A multi-image complementary information tracking method with dynamic activation mechanism

Abstract

The application provides a multi-image complementary information tracking method with a dynamic activation mechanism, comprising the following steps: tracking a target by using a built network to obtain a tracking result, wherein the built network comprises a Transfomer feature extraction network with a multi-head attention module to calculate a mixed template feature and a scene perception activation module, a multi-view mutual information module, and a head prediction network with an added bounding box prediction branch based on a classification branch and a regression branch.The multi-image complementary information tracking method of the application has better performance under the premise of meeting the real-time running speed in the face of changes in viewing angle, similar object interference, partial occlusion and rapid motion, and can effectively improve the tracking success rate and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target tracking of unmanned aerial vehicles, in particular, to a multi-image complementary information tracking method with a dynamic activation mechanism. BACKGROUND

[0002] In the field of computer vision, target tracking algorithms have always been of great concern, and many researchers at home and abroad have devoted themselves to the research of this technology. The Transformer tracker has a large computational overhead, especially when the background is simple, which will produce a lot of redundant calculations; in the process of unmanned aerial vehicle target tracking, the rapid movement of the unmanned aerial vehicle leads to frequent scale changes and extreme changes in the viewing angle, and the traditional template features cannot guarantee the stability of the target features in the viewing angle changes, thereby limiting the performance of the tracker.

[0003] In the prior art, OStrack uses a ViT backbone to combine the feature learning and feature fusion processes together, effectively utilizing the information flow between the target template features and the search region features; SeqTrack regards the target tracking task as a sequence generation task, uses a simple encoder-decoder architecture, the encoder is responsible for feature extraction, and the decoder generates the target bounding box coordinate sequence frame by frame, making the tracking framework more efficient; MATTrack independently encodes and decodes the visual features of the template and the search region when learning the target tracking representation, and enhances the discriminability of learning through masking random processing. Litetrack proposes to use layer pruning technology to greatly simplify the Transformer model, so as to greatly improve the tracking speed; SparseTT proposes a sparse attention mechanism, which focuses on the most relevant information in the search region, effectively reduces the interference of the background, and improves the tracking efficiency and accuracy; HiFT uses a multi-layer convolutional network to generate a hierarchical similarity map, and inputs these maps into the Transformer, so as to realize the interactive fusion of spatial (shallow) and semantic (deep) features, the structure is more efficient, and a higher real-time tracking speed can be maintained.

[0004] However, the above-mentioned prior art methods have the following common problems: the main problem of the above-mentioned first three methods is that the tracking accuracy is high, but the tracking efficiency is very low, the tracking speed is difficult to achieve real-time tracking effect, and the actual deployment on embedded devices is impossible, the last three algorithms greatly simplify the backbone network of the model through layer pruning, attention sparsification and other ways, but the accuracy decreases to a certain extent, and the performance in the challenging scenarios such as viewing angle changes and scale changes in the process of unmanned aerial vehicle target tracking is poor.

[0005] Therefore, the present application is proposed. SUMMARY

[0006] Therefore, the present application provides a multi-image complementary information tracking method with a dynamic activation mechanism, which abandons the traditional single template image modeling method and the fixed calculation path in the feature extraction process, and aims to solve the problems that the algorithm is difficult to cope with the change of the visual angle and the excessive redundant calculation in the unmanned aerial vehicle target tracking.

[0007] Specifically, the present application is realized by the following technical solutions:

[0008] The present application provides a multi-image complementary information tracking method with a dynamic activation mechanism, comprising the following steps:

[0009] Implementing abnormality detection based on a multi-dimensional abnormality evaluation unit and fault diagnosis based on a double-path fine-grained feature positioning structure on the photovoltaic system;

[0010] Tracking the target by using the built network to obtain tracking results, wherein the built network comprises: a Transfomer feature extraction network with a multi-head attention module to calculate mixed template features and a scene awareness activation module, a multi-view mutual information module, and a head prediction network with an added bounding box prediction branch based on a classification branch and a regression branch.

[0011] Optionally, the construction method of the scene awareness activation module comprises:

[0012] Inputting part of the slices in the token sequence generated by the search branch Transformer encoder, and performing normalization and entropy calculation on the token slices to generate a background entropy value H, then the module accepts the target classification confidence score s generated by the classification branch in the head prediction network, and obtains a scene complexity weight by combining H and s;

[0013] By setting the preliminary activation probability p of the current Transformer block i , the final activation probability P is further calculated by combining the scene complexity weight i , so as to realize dynamic activation or skipping of the Transformer layer according to the input features and different scenes.

[0014] Optionally, the calculation method of the scene complexity weight by combining H and s comprises:

[0015]

[0016] C=γ·(1-s)+δ·H (2)

[0017] Wherein, x iis the normalized value for the i-th feature, N is the total number of tokens, and γ and δ are the weights controlling s and H. s represents the recognition confidence of the model to the target, and H represents the complexity of the background information.

[0018] Optionally, the preliminary activation probability p i is calculated as follows:

[0019] p i = σ(L i (r i-1 )) (3)

[0020] where r i-1 is the selected token segment, L i is a linear change, and σ is a Sigmoid activation function.

[0021] Optionally, the final activation probability P i is further calculated by combining the scene complexity weight as follows:

[0022] P i = p i ·(1+C) (4)

[0023] If P i > α (wherein α ∈ (0.7, 1)), the i-th Transformer encoder is activated to further extract feature information; if P i < α, the i-th layer is skipped and the output result of the previous transformer block is directly transmitted to the i+1-th layer.

[0024] Optionally, the method for building the multi-view mutual information module comprises:

[0025] Three kinds of feature information F st , F sr , and F i are calculated, where F st , F sr represent token sequences from the output of the last layer of the Transformer encoder of the static template branch and the search branch, and F i represents a maximum difference frame feature sequence obtained based on a frame difference method.

[0026] The three kinds of feature information are input into an MIE mutual information estimator to output mutual information features, which are used to optimize the alignment between the template features and the search features.

[0027] Optionally, the calculation processes of F st , F sr , and F i are as follows:

[0028] Fst = Attn st (5)

[0029] F sr = Attn sr (6)

[0030]

[0031] Where, θ(F) st ,f i F is the cosine similarity between the features of the candidate frame and the search frame. i This represents the frame with the largest difference selected from all candidate frames.

[0032] Optionally, after inputting the three types of feature information into the MIE mutual information estimator to output mutual information features, the Jensen-Shannon divergence estimation method is used to learn the stable viewpoint features of the multi-view. The loss function used to learn the stable viewpoint feature representation of the multi-view is defined as follows:

[0033]

[0034] The features output by the static template through the Transformer block are denoted as follows: The features of the search region output by the Transformer block are denoted as follows: This represents the feature of the frame with the greatest difference.

[0035] In tracking engineering, the rapid movement of UAVs and the movement of the tracked target lead to frequent scale changes and extreme viewpoint changes, causing tracking failures. To overcome this problem, this invention proposes a multi-image complementary tracking algorithm, AMV-Tracker, with a dynamic activation mechanism. By establishing multi-view mutual information, robust features are built under different viewpoints, thus better adapting to viewpoint changes. A schematic diagram of the AMV-Tracker target tracking algorithm proposed in this invention is shown below. Figure 1 As shown: It mainly includes three network components: (1) Transformer feature extraction network, including multi-head attention module MHA to calculate hybrid template features and scene-aware activation module SAAM to realize dynamic activation of Transformer blocks under different scenes to reduce redundant calculations; (2) multi-angle mutual information module MVMI, which obtains more robust feature representation by modeling template features and searching for mutual information between features and keyframe features; (3) head prediction network, which adds a bounding box prediction branch on the basis of classification and regression branches, accurately predicts the boundary of the target according to the scale change of the target, and obtains a bounding box that is more adaptable to the change of the target, thus realizing accurate target localization.

[0036] The present application aims at the problems of lack of adaptability to view angle change, large network computing overhead, tracking performance decline in large-scale motion scene and lack of long-term stability and short-term adaptability of single template in traditional target tracking network, and proposes a multi-image complementary information double-template tracking algorithm with a dynamic activation mechanism, which gives up the traditional single template image modeling method and the fixed calculation path in the feature extraction process, aiming to solve the problems of difficulty in coping with view angle change and excessive redundant calculation of the proposed algorithm in unmanned aerial vehicle target tracking. Under the premise of meeting the real-time running speed, the method has better performance in the scenes of view angle change, similar object interference, partial occlusion and rapid motion, and can effectively improve the tracking success rate and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0037] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments with reference made to the accompanying drawings, which serve as best modes of carrying out the application and are not intended to limit the same, and in which like reference numerals designate similar parts throughout the several views:

[0038] Figure 1 The AMV-Tracker network structure of the embodiment of the present application is shown in the figure;

[0039] Figure 2 The scene awareness activation module SAAM of the embodiment of the present application is shown in the figure;

[0040] Figure 3 The multi-view mutual information module MVMI of the embodiment of the present application is shown in the figure;

[0041] Figure 4 The multi-sequence comparison experiment result of the embodiment of the present application is shown in the figure;

[0042] Figure 5 The UAV123 dataset visualization test result of the embodiment of the present application is shown in the figure;

[0043] Figure 6 The structure schematic diagram of a computer device is shown in the figure. DETAILED DESCRIPTION

[0044] The exemplary embodiments will be described in detail hereinbelow with reference to the drawings, wherein the same or similar components are designated by the same reference numerals if not described otherwise, and the embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present disclosure, but rather only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims,

[0045] The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting thereof, the use of the singular herein will include the plural unless the context clearly dictates otherwise, it is further to be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more associated listed items,

[0046] It should be understood that although the terms first, second, third, etc. can be employed in this disclosure to describe various information, such information should not be limited to these terms solely, these terms are only used to distinguish one piece of information from another piece of information, for example, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information, depending on the context, the word "if" as used herein can be interpreted as "when" or "in response to determining" that,

[0047] Embodiments

[0048] The present application provides a multi-image complementary information tracking method with a dynamic activation mechanism, comprising the following steps:

[0049] The target is tracked by using the built network to obtain the tracking result, the built network comprises: a Transfomer feature extraction network with a multi-head attention module to calculate a mixed template feature and a scene-aware activation module, a multi-view mutual information module, and a head prediction network with an additional bounding box prediction branch based on a classification branch and a regression branch.

[0050] Specifically, the steps of the following embodiments are performed:

[0051] Embodiment 1

[0052] 1) Scene-aware activation module SSAM

[0053] In order to dynamically adjust the calculation path according to the scene complexity, a scene-aware activation module is designed as shown in Figure 2 According to the input feature and the different scenes, the Transformer layer is dynamically activated or skipped, and the network calculation efficiency is improved.

[0054] First, input part of the slice in the token sequence generated by the search branch Transformer encoder, and normalize and entropy calculate the token slice to generate the background entropy value H, then, the module accepts the target classification confidence score s generated by the classification branch in the head prediction network, and calculates the scene complexity weight C jointly with H and s, the above calculation process is as follows:

[0055]

[0056] C = γ · (1 - s) + δ · H (2)

[0057] where x i is the normalized value of the i-th feature value, N is the total number of tokens, and γ and δ are the weights controlling s and H. s is used to represent the recognition confidence of the model to the target, and the lower the confidence score, the lower the probability of the model recognizing the target of interest, and the model needs to extract more in-depth features; H is used to represent the complexity of the background information, and the higher the background entropy value means the more complex the scene, and more transformer encoders need to be activated to model the target with higher dimension information.

[0058] Secondly, the tokens generated after feature extraction of the input template image and the search image are selected, and then a part of the token fragments are selected to pass through a linear layer for feature dimension reduction and aggregation, and are mapped to a scalar. Then, the scalar is mapped to the interval corresponding to the interval by using the Sigmoid function to obtain the preliminary activation probability p i of the current transformer block.

[0059] p i = σ (L i (r i-1 )) (3)

[0060] where r i-1 is the selected token fragment, L i is a linear change, and σ is a Sigmoid activation function.

[0061] Finally, the scene complexity weight and the preliminary activation probability p i are further calculated to obtain the final activation probability P i . If P i > α (wherein α ∈ (0.7, 1)), the i-th transformer encoder is activated to further extract feature information; if P i < α, the i-th layer is directly passed to the i+1-th layer to realize the reduction of the calculation amount in a simple scene and the increase of the calculation capacity in a complex scene, thereby improving the efficiency and robustness of the model. P i The calculation formula is as follows:

[0062] P i = p i · (1 + C) (4)

[0063] When the target moves in a single-color background, due to the simplicity of the scene, the model has higher recognition confidence to the target, and can realize effective tracking based on the color contrast between the target and the background. At this time, the calculation result of C is lower, and the final activation probability Pi The closer to p i , the greater the probability of skipping the current Transformer encoder to reduce unnecessary calculations. When the target is navigating in a cluttered environment, the tracking task becomes more complex, and the model's confidence in identifying the target decreases. In this case, the calculation of C will be higher, activating more Transformer encoders to participate in further feature extraction, capturing and analyzing sufficient semantic features and relationships in the chaotic scene to ensure the robustness of the tracking effect.

[0064] 2) Multi-view mutual information module MVMI

[0065] The multi-view mutual information module establishes mutual information between the search branch output feature and the template branch output feature, as shown in Figure 3 , ensuring that the model maintains feature consistency and effectively tracks the target even in cases of large changes in viewing angle. In addition, to prevent the two features from being unable to calculate effective mutual information due to too high feature similarity or too large viewing angle difference, the key frame feature is introduced to establish mutual information with the above two features, providing motion information of the target over a longer time span, achieving a smoother mutual information establishment process, and enhancing the viewing angle invariance of the features.

[0066] F st , F sr represent the token sequences from the last layer Transformer encoder outputs of the static template branch and the search branch, f i to f i+k represent k historical frame features within a certain time, F i represents the maximum difference frame feature sequence obtained based on the frame difference method. F st , F sr , F i The calculation process is as follows:

[0067] F st = Attn st (5)

[0068] F sr = Attn sr (6)

[0069]

[0070] where θ(F st ,f i ) is the cosine similarity calculation between the candidate frame and the search frame feature, F i represents the maximum difference frame selected from all candidate frames.

[0071] After calculating the three characteristic information, input them to the MIE mutual information estimator to output the mutual information characteristics, which are used to optimize the alignment between the template characteristics and the search characteristics. The MIE estimator adopts the Jensen-Shannon divergence (JSD) estimation method to learn the stable view angle characteristics of multi-view. The characteristics of the static template output by the Transform block are denoted as The characteristics of the search area output by the Transform block are denoted as The loss function for learning the stable view angle feature representation of multi-view is defined as follows:

[0072]

[0073] After introducing the maximum difference frame characteristics, the mutual information module further enhances the consistency across the view angles, ensures that the optimization process is smoother, and alleviates the problem that the template and the search image are difficult to calculate effective mutual information due to too small or too large view angle difference. By maximizing the mutual information between multi-views, it can be ensured that the feature representation of the template and the search image remains consistent under different view angles, which helps the model to reduce the influence of factors such as view angle change, scaling, deformation, etc. Finally, these characteristics are used to optimize the positioning and regression tasks of the target, improving the accuracy and robustness of tracking.

[0074] The above is the specific application of the multi-view mutual information and the scene perception activation module in the method of the present application. In order to verify the effectiveness of the algorithm of the present application, first, an ablation experiment is performed on a specific data set to verify the effect, and the experimental results are shown in Table 1 below.

[0075] Table 1 Ablation experiment results on LaSOT dataset

[0076]

[0077] The baseline algorithm running speed is 163.2FPS, the parameter quantity is 38.97M, the success rate (SR), precision (Pn) and normalized precision (P) are 0.621, 0.715 and 0.651 respectively. In experiment 2, after adding the SAAM module, the parameter quantity reaches 40.86M, but due to the selective skipping of part of the calculation of the SAAM module, the subsequent calculation amount of the model has a certain degree of decline, and the running speed can reach 172.1 frames, while the SR, Pn and P have an increase of 0.008, 0.014 and 0.02. Experiment 3 increases the MVMI module on the basis of experiment 2, and the parameter quantity increases by 3.35M. Since the main function of the MVMI module is to calculate the mutual information between the template frame and the search frame features, the calculation result is a loss value, and no extra feature information is output, but is used to optimize the feature relationship between the template features and the search features in the tracking network, so the running speed is almost the same as that of experiment 2, which is 172.3FPS / s, and the SR, Pn and P accuracy are increased by 0.009, 0.004 and 0.013 respectively. Through the above ablation experiments, it can be well shown that the model proposed strategy of the present application is effective.

[0078] To further show the superiority of the network built by the present application, some classic tracking algorithms are compared on the UAV dataset, including TransT, Stark, BANDT, LiteTrack, SparseTT, HiFT, OStrack, TCTrack and AVTrack. As shown in Figure 4 , it is the success rate curve of different algorithms in various challenging scenarios. The method proposed in the present application achieves the best performance in view angle change, camera motion, scale change and partial occlusion sequences, with a success rate of 0.724, 0.712, 0.687 and 0.649 respectively. In addition, it also achieves excellent performance in similar objects, fast motion and light change

[0079] As shown in Table 2, the comparison results of each algorithm. As can be seen from the table, the algorithm proposed in the present application is slightly higher than StarK in tracking performance, but the running speed is obviously better than StarK and other algorithms, and the processing speed of 157.4 can be transplanted, with higher running efficiency.

[0080] Table 2 LaSOT comparison test results

[0081]

[0082] To avoid too many algorithms causing tracking results in video sequences to be chaotic and difficult to distinguish, the proposed algorithm is compared with existing algorithms to more clearly show the tracking performance of the algorithm. The visual results of part of the UAV123 dataset video sequences are shown, and five algorithms, SparseTT, TCTrack, HiFT, Stark and LiteTrack, are selected to verify the actual tracking effect in the visualization sequence, and the results are shown in Figure 5 .

[0083] In Figure 5 (a), the tracking target constantly moves and the shooting platform changes the view angle, resulting in frequent scale changes. Stark and LiteTrack cannot accurately track the complete target due to the lack of adaptability to target scale changes.

[0084] In Figure 5 (b), the shooting platform needs to track the fast-moving target from multiple angles. LiteTrack cannot fully track the target due to the difficulty in adapting to the view angle changes caused by fast motion.

[0085] In Figure 5 (c), the shooting platform moves a large range during tracking, resulting in extreme view angle changes. The tracker needs to track the target at more angles, and there are different degrees of similar interference problems. SparseTT, TCTrack, HiFT and LiteTrack frequently drift during tracking, making it difficult to accurately track the target throughout the process. However, the algorithm in this paper still has great adaptability to targets under angle changes and can accurately track the target of interest.

[0086] Through the visual analysis of various tracking scenes, the algorithm in this paper successfully solves the challenges brought by target scale changes, fast motion and extreme view angle changes, and demonstrates the overall performance of the algorithm in various scenes.

[0087] Figure 6 A structural schematic diagram of a computer device is disclosed. Referring to Figure 6 , the computer device 400 at least includes a memory 402 and a processor 401. The memory 402 is connected to the processor through a communication bus 403 and is used to store computer instructions executable by the processor 401. The processor 401 is used to read computer instructions from the memory 402 to implement the steps of the method described in any of the above embodiments.

[0088] For the above apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be seen from the parts of the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present disclosure according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0089] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0090] It should also be understood that, although the terms "first" and "second" can be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another. Thus, a first element discussed below could be termed a second element without departing from the scope of the present application. Similarly, it should be understood that, although the terms "including", "includes" and / or "having" can be used herein, these terms are used in the sense of "comprising", "includes" and / or "having" to specify the presence of stated features, integers, steps and / or components but not to the exclusion of one or more other features, integers, steps, components or equivalents thereof.

[0091] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such order, nor that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0092] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0093] The above-described embodiments are merely possible implementations of the present disclosure, and do not limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the present disclosure.

Claims

1. A multi-image complementary information tracking method with a dynamic activation mechanism, characterized in that, The method comprises the following steps: Tracking the target by using the built network to obtain tracking results, wherein the built network comprises a Transformer feature extraction network with a multi-head attention module to calculate mixed template features and a scene-aware activation module, a multi-view mutual information module, and a head prediction network with an added bounding box prediction branch based on a classification branch and a regression branch; The construction method of the scene-aware activation module comprises: Inputting part of the slices in the token sequence generated by the search branch Transformer encoder, normalizing and entropy calculating the token slices to generate a background entropy value H, then the module accepting a target classification confidence score s generated by the classification branch in the head prediction network, and obtaining a scene complexity weight by combining H and s; By setting the current Transformer block preliminary activation probability , the scene complexity weight is combined to further calculate the final activation probability , and the Transformer layer is activated or skipped according to the input characteristics and different scenes. The construction method of the multi-view mutual information module comprises: Three kinds of feature information are calculated wherein represents the token sequence from the static template branch and the last layer of the Transformer encoder output of the search branch, represents the maximum difference frame feature sequence obtained based on the frame difference method; Inputting the three kinds of feature information into an MIE mutual information estimator to output mutual information features for optimizing the alignment between the template features and the search features.

2. The multi-image complementary information tracking method of claim 1, wherein, The calculation method of the scene complexity weight obtained by combining H and s comprises: (1) (2) wherein, is the normalized value for the ith feature value, N is the total number of tokens, and is the weight controlling s and s represents the model's recognition confidence for the target, represents the complexity of the background information.

3. The multi-image complementary information tracking method of claim 1, wherein, said preliminary activation probability The calculation method of the preliminary activation probability is as follows: (3) where, is the selected token piece, is a linear change, is a Sigmoid activation function.

4. The multi-image complementary information tracking method of claim 1, wherein, The scene complexity weight is combined to further calculate a final activation probability The calculation method is: (4) If (wherein ∈(0.7, 1)), the i-th transformer encoder is activated to further extract feature information; if , the i-th layer directly passes the output result of the previous transformer block to the i+1-th layer.

5. The multi-image complementary information tracking method of claim 1, wherein, , , The calculation proceeds as follows: (5) (6) (7) wherein, is the cosine similarity calculation between the candidate frame and the search frame features, represents the maximum difference frame selected among all candidate frames.

6. The multi-image complementary information tracking method of claim 1, wherein, After inputting the three kinds of feature information into the MIE mutual information estimator to output mutual information features, a Jensen-Shannon divergence estimation method is used to learn the stable view angle features of the multi-view, and a loss function for learning the stable view angle feature representation of the multi-view is defined as follows: (8) The features output by the Transformer block for the static template are denoted as , and the features output by the Transformer block for the search region are denoted as , is the maximum difference frame feature.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by the processor to execute the steps of the multi-image complementary information tracking method according to any one of claims 1-6.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the multi-image complementary information tracking method according to any one of claims 1-6.

Images