Any complex scene scale adaptive single target memory tracking method and device, medium and program product

Abstract

The invention provides a single-target memory tracking method and device adaptive to any complex scene scale, a medium and a program product. According to the method, an FDSST tracking algorithm is improved, fhog features are extracted under non-complete shielding, correlation filtering and scale filtering are carried out, and target scale self-adaptive tracking is achieved; a target historical filtering template and a target historical track are cached, correlation operation is carried out on the historical filtering template and a current filtering template, target shielding loss judgment is carried out, and if a target is lost, a memory tracking state is entered; and secondly, searching and repositioning are carried out in subsequent frames by using cached target features, FDSST tracking is recovered when the target appears again and a shielding response value is higher than a shielding threshold value, and track pre-pushing after target tracking shielding is lost and target repositioning and tracking recovery after shielding disappears can be realized through the method. And finally, target scale adaptive memory tracking is realized, and the short-time shielding resistance of target tracking is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of target tracking technology, and in particular to a single target memory tracking method, device, medium and program product that is adaptive to scale of any complex scene. Background Art

[0002] Image target tracking is widely used in various fields to achieve stable and continuous tracking of any type of target in any complex scene. In actual use, for any complex application scene, due to the camera zooming or the displacement of the target optical axis under the fixed-focus camera, the scale of the target in the image changes greatly, causing the target tracking to drift or even lose tracking; at the same time, complex occlusions can also cause drastic changes in target features and cause tracking loss. Therefore, solving the tracking instability caused by target deformation and improving the target's anti-occlusion performance are the keys to stable and continuous tracking of any target in a complex background.

[0003] Traditional single target tracking algorithms are mainly based on detection tracking methods such as CSK, KCF, MOSSE and other related filtering methods. Although such methods have high real-time performance, they can only predict the position of the target and cannot predict and adapt to the scale change of the target. In actual use, for any complex application scenario, due to the factors of camera zooming or fixed-focus camera displacement of the target optical axis, the scale of the target in the image changes greatly, causing the target tracking to drift or even lose tracking; at the same time, complex occlusions can also cause drastic changes in target features, which can also lead to tracking loss.

[0004] FDSST can predict both the change in the position of the target and the change in the scale of the target. It has better adaptability than the above-mentioned related filtering algorithms, but it still cannot solve the problem of memory tracking when the target is occluded. When the target is completely occluded for a long time and then reappears, it cannot automatically re-track the target. In recent years, deep learning has made great progress and achieved good results in the field of image tracking. Single target tracking algorithms based on deep learning, such as SiameseFC, DCFNet, and MDNet, have high tracking performance, but due to the large amount of neural network calculations, they cannot be realized in real time on embedded hardware platforms with limited computing power, and there is also the problem of being unable to remember and track the target after it is occluded. Summary of the invention

[0005] The present invention aims to provide a single target memory tracking method, device, medium and program product with arbitrarily complex scene scale self-adaptation, so as to solve the problem of target tracking loss caused by large target scale changes and target occlusion in arbitrarily complex scenes.

[0006] The present invention provides a single target memory tracking method for arbitrarily complex scene scale adaptation, comprising the following steps:

[0007] Step 1: Initialize the FDSST single target image tracking algorithm to obtain the target filter template, including the position filter template and the scale filter template;

[0008] Step 2: Obtain real-time video frames to predict the position and scale of the target, and perform online training to update the filter template of the target;

[0009] Step 3: Cache the target filter template, perform correlation filtering on the cached target filter template and the historical filter template to obtain the first occlusion response value of the target;

[0010] Step 4: Determine whether the first occlusion response value is lower than the set occlusion threshold:

[0011] If the first occlusion response value is lower than the set occlusion threshold, the target is determined to be completely occluded, and the memory tracking state is entered. The trajectory of the target is pre-pushed and the scale is maintained according to the historical trajectory, and step 5 is performed;

[0012] If the first occlusion response value is not lower than the set occlusion threshold, it is determined that the target is not occluded, and the process returns to step 2;

[0013] Step 5: Continue to perform correlation filtering on the target filter template and the historical filter template to obtain the second occlusion response value of the target:

[0014] If the second occlusion response value is lower than the set occlusion threshold, the memory tracking state is maintained and step 6 is performed;

[0015] If the second occlusion response value is not lower than the set occlusion threshold, restore to FDSST tracking and return to step 2;

[0016] Step 6: Determine whether the memory tracking state time exceeds the set time threshold:

[0017] If the memory tracking state time exceeds the set time threshold, the target is considered lost, and the single target tracking is exited, waiting for receiving a new target and returning to step 1;

[0018] If the memory tracking state time does not exceed the set time threshold, the memory tracking state is maintained.

[0019] In some embodiments, step 1 specifically includes the following sub-steps:

[0020] Step 1.1, extract the fhog features of the target;

[0021] Step 1.2, using the extracted fhog features to train the FDSST single target image tracking algorithm to obtain a filter template of the target, including a position filter template and a scale filter template.

[0022] In some embodiments, the filtering template of the target is represented as:

[0023]

[0024] Among them, H l represents the filter template, d represents the fhog feature dimension, and l represents a certain dimension feature; the desired filter response output result is subjected to discrete Fourier transform and conjugated to obtain The result of discrete Fourier transform after the feature of the lth dimension of the sample is mapped to a high dimension by Gaussian kernel function is F l ; and so on F k It represents the result of discrete Fourier transform after the k-th dimension feature of the sample is mapped to a high dimension by Gaussian kernel function. F k , λ is a small quantity that prevents the denominator from dividing by zero.

[0025] In some embodiments, the extracted fhog features are used to train an FDSST single target image tracking algorithm, including:

[0026] The filter template H l is equivalent to:

[0027]

[0028] The target prediction is achieved by performing correlation filtering operation on the fhog feature map of the current search area and the filter template of the previous frame, thereby updating A and B to complete the filter template update, which is expressed as:

[0029]

[0030] in, and Represents the filter template H of the previous frame l The numerator and denominator of , η is the learning rate.

[0031] In some embodiments, in step 3, correlation filtering is performed on the cached target filter template and the historical filter template to obtain a predicted response map, and the maximum response value in the predicted response map is the first occlusion response value of the target;

[0032] The predicted response graph y is expressed as:

[0033]

[0034] Among them, Z l Represents the fhog feature map of the search area in the current frame, F -1 Represents the inverse Fourier transform operation after frequency domain multiplication.

[0035] In some embodiments, in step 4, performing trajectory prediction and scale maintenance on the target according to the historical trajectory includes:

[0036] Step 4.1, smoothing the target historical trajectory;

[0037] Step 4.2: Predict particle trajectory and maintain scale based on the smoothed target historical trajectory.

[0038] In some embodiments, in step 4, when the first occlusion response value is lower than the set occlusion threshold value for multiple consecutive times, it is determined that the target is completely occluded and the memory tracking state is entered.

[0039] In a second aspect, the present invention further provides an electronic device, comprising:

[0040] at least one processor; and a memory communicatively coupled to the at least one processor;

[0041] The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the above method by executing the instructions stored in the memory.

[0042] In a third aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store instructions, and when the instructions are executed, the above method is implemented.

[0043] In a fourth aspect, the present invention further provides a computer program product, which, when called by a computer, enables the computer to execute the above method.

[0044] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0045] The present invention improves the FDSST tracking algorithm, extracts the fhog features of the target in a normal tracking state of non-complete occlusion, performs correlation filtering and scale filtering respectively, realizes simultaneous prediction of the target position and scale to achieve adaptive tracking of the target scale; by caching the target history filter template and the target history trajectory, adopts the history filter template and the current filter template to perform correlation operations, performs target occlusion loss judgment, if the target is lost, enters the memory tracking state, and maintains the target motion state through the history trajectory feature; secondly, in the subsequent frames, the cached target features are used to search and relocate the center point neighborhood template, and when the target appears again, the occlusion response value is higher than the occlusion threshold, and the FDSST tracking is restored. By this method, the trajectory pre-pushing after the target tracking occlusion is lost and the target relocation recovery tracking after the occlusion disappears can be realized, and the target scale adaptive memory tracking is finally realized, and the anti-short-term occlusion performance of the target tracking is enhanced. Therefore, the method of the present invention effectively solves the problem of target tracking loss caused by large target scale changes and target occlusion in any complex scene, and realizes target scale adaptive tracking and anti-occlusion memory tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a flow chart of a single target memory tracking method capable of adaptively tracking the scale of any complex scene in an embodiment of the present invention.

[0047] Figure 2 This is a rendering of the single target memory tracking method that is scale-adaptive for arbitrarily complex scenes in an embodiment of the present invention.

[0048] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In order to solve the problem of target tracking loss caused by large changes in target scale and target occlusion in any complex scene, we finally realize target scale adaptive tracking and anti-occlusion memory tracking. Figure 1 As shown, the above-mentioned single target memory tracking method for arbitrary complex scene scale adaptation includes the following steps:

[0052] Step 1: Initialize the FDSST single target image tracking algorithm and obtain the target filter template, including the position filter template and the scale filter template.

[0053] In some embodiments, step 1 specifically includes the following sub-steps:

[0054] Step 1.1, extracting the fhog feature of the target; generally, the user draws an initial frame (usually a rectangular frame) through the upper computer display and control interactive interface to select the target to be tracked, and uses the initial frame to extract the fhog feature;

[0055] Step 1.2, using the extracted fhog features to train the FDSST single target image tracking algorithm to obtain a position filter template and a scale filter template.

[0056] The fhog feature is an improved optimization of the traditional hog feature, and has fewer feature dimensions than the traditional oriented gradient histogram feature. Taking the traditional oriented gradient histogram that is insensitive to direction as an example, the feature dimension of each cell is 36 dimensions. If fhog is used to sum the rows and columns of the feature, the dimension can be reduced to a 13-dimensional vector. In order to further enhance the robustness of the feature, fhog finally integrates both the feature extraction methods that are sensitive to direction and the feature extraction methods that are insensitive to direction. For a cell of 8×8 pixels, the feature is finally compressed from 4*(18+9)=108 dimensions to 31 dimensions, which greatly reduces the amount of calculation.

[0057] The calculation formula of the filter template obtained after initialization is shown in (1):

[0058]

[0059] Among them, H l represents the filter template, d represents the fhog feature dimension, and l represents a certain dimension feature; the desired filter response output result is subjected to discrete Fourier transform and conjugated to obtain The result of discrete Fourier transform after the feature of the lth dimension of the sample is mapped to a high dimension by Gaussian kernel function is F l ; and so on F k It represents the result of discrete Fourier transform after the k-th dimension feature of the sample is mapped to a high dimension by Gaussian kernel function. F kThe conjugate of λ is a small quantity to prevent the denominator from dividing by zero. In the initialization process, the position filter template and scale filter template are initialized by formula (1) to complete the initialization of the FDSST single target image tracking algorithm. The position filter template and scale filter template are used as the initial template for the first frame of target tracking.

[0060] Step 2: Obtain real-time video frames to predict the position and scale of the target, and perform online training to update the target's filter template.

[0061] Filter template H l It can be equivalent to formula (2):

[0062]

[0063] In this way, in the update phase, A and B are actually updated. The filter template update at the current moment is equivalent to updating A and B. The updated learning rate is η, as shown in equations (3) and (4):

[0064]

[0065] in, and Represents the filter template H of the previous frame l The numerator and denominator of , the learning rate η is set to 0.001 here.

[0066] The prediction is done by searching the fhog feature map of the area and the filter template H l Correlation filtering is performed in the frequency domain and then Fourier inverse transform is performed to obtain the response map. The maximum response value in the response map is the new predicted position. The specific calculation formula is shown in (5):

[0067]

[0068] Among them, Z l Represents the fhog feature map of the search area in the current frame, F -1 It represents the inverse Fourier transform operation after frequency domain multiplication, and finally obtains the predicted response map y. The result is generally the correlation filtering operation between the fhog feature map of the current search area and the filter template of the previous frame to achieve target prediction. Similarly, the target scale prediction is based on the result of position prediction to perform scale filtering and update the scale filter according to the current scale feature.

[0069] Step 3: Cache the target filter template, perform correlation filtering on the cached target filter template and the historical filter template to obtain the first occlusion response value of the target.

[0070] Based on the prediction and update in step 2, a structure instance sequence of the target position filter template and the target centroid track is maintained. The sequence length can be set as needed. For example, the maximum sequence length can be set to 100. The fhog feature map Z calculated in the current frame l Perform correlation filtering with the first target position filter template in the sequence, as shown in formula (5), calculate the maximum response value and use this value as the target feature change measurement value. This value represents the correlation between the current feature of the target and the historical feature, which can reflect the degree of occlusion of the target. Therefore, this value is used as the target occlusion response value to determine whether the target is completely occluded.

[0071] Step 4: Determine whether the first occlusion response value is lower than the set occlusion threshold:

[0072] If the first occlusion response value is lower than the set occlusion threshold (which can be set based on experience, for example, 0.2), the target is determined to be completely occluded, and the memory tracking state is entered. The trajectory of the target is pre-pushed and the scale is maintained according to the historical trajectory, and step 5 is performed;

[0073] If the first occlusion response value is not lower than the set occlusion threshold, it is determined that the target is not occluded, and the process returns to step 2.

[0074] In some embodiments, in order to prevent misjudgment, the judgment method of step 4 can be changed to multiple judgments, that is, when the first occlusion response value is lower than the set occlusion threshold for multiple consecutive times (for example, 3 times), it is determined that the target is completely occluded and enters the memory tracking state.

[0075] In some embodiments, performing trajectory prediction and scale maintenance on a target based on a historical trajectory includes:

[0076] Step 4.1, use Savitzky-Golay filter to smooth the target historical trajectory;

[0077] The Savitzky-Golay filter is a convolution smoothing filter. Assume that the filter window width n = 2m-1, the measurement point set P = {(x -m ,y -m ),(x -m+1 ,y -m+1 )…(x 0 ,y 0 )…(x m ,y m )}, now construct a k-order polynomial y=a 0 +a 1 x+a 2 x 2 +…+a k x k, at this time, since the filtering window is n, there are n polynomials, and n>k, the coefficients of the polynomials can be obtained by using the least square method. In this embodiment, m=2, k=3, that is, 5-point 3-order Savitzky-Golay filtering is used. The filtering process can be obtained by the least square method as shown in formula (6):

[0078]

[0079] Step 4.2: Predict particle trajectory and maintain scale based on the smoothed target historical trajectory.

[0080] In some embodiments, a three-point linear prediction method is used to predict the position (x) of the target in the next frame of image. c ,y c ), in order to realize the prediction of particle trajectory, the general three-point linear prediction equation is shown in formula (7):

[0081]

[0082] Among them, f(k+1 / k) represents the center of mass position of the target at the next moment (next frame), f(k) represents the center of mass position at the current moment, and f(k-1) and f(k-2) represent the center of mass positions at two historical moments relative to the current moment. In the memory tracking state, only the center of mass trajectory is predicted and the scale of the target before it is lost remains unchanged. At this time, FDSST tracking prediction and update are no longer performed.

[0083] Step 5: After completing the trajectory pre-pushing, continue to perform correlation filtering on the target filter template and the historical filter template to obtain the second occlusion response value of the target:

[0084] If the second occlusion response value is lower than the set occlusion threshold, the memory tracking state is maintained and step 6 is performed;

[0085] If the second occlusion response value is not lower than the set occlusion threshold, the FDSST tracking is restored and the process returns to step 2.

[0086] In the memory tracking state, the cached target filter template is no longer updated. During the target trajectory pre-pushing process, the fhog features of the target search area are still calculated in real time, and the occlusion response value is obtained by correlation filtering with the first feature filter of the cache feature queue (that is, the historical filter template) according to formula (5). If the occlusion response value is higher than the occlusion threshold of 0.2 when the target appears again, the system exits the memory tracking state, trajectory smoothing and pre-pushing stop, and FDSST single target tracking is performed again.

[0087] Step 6: Determine whether the memory tracking state time exceeds the set time threshold:

[0088] If the memory tracking state time exceeds the set time threshold (which can be set as needed, for example, 10 seconds), the target is determined to be lost, and the single target tracking is exited, the tracking state is exited, the tracking cache is cleared, and the new target is received and the process returns to step 1;

[0089] If the memory tracking state time does not exceed the set time threshold, the memory tracking state is maintained.

[0090] Figure 2 The figure is a rendering of a scale-adaptive single target memory tracking method for arbitrary complex scenes in an embodiment of the present invention. It can be seen that the embodiment of the present invention proposes a scale-adaptive single target memory tracking method for arbitrary complex scenes. By improving the FDSST tracking algorithm, the fhog features of the target are extracted in a normal tracking state without complete occlusion, and correlation filtering and scale filtering are performed respectively to achieve simultaneous prediction of the target position and scale to achieve scale-adaptive tracking of the target. By caching the target historical filter template and the target historical trajectory, the historical filter template is used to perform correlation operations with the current filter template to judge the target occlusion loss. If the target is lost, the memory tracking state is entered, and the target motion state is maintained by the historical trajectory features. Secondly, the cached target features are used in subsequent frames to search and relocate the center point neighborhood template. When the target appears again, the occlusion response value is higher than the occlusion threshold, and the FDSST tracking is restored. This method can realize the trajectory pre-pushing after the target tracking occlusion is lost and the target relocation and recovery tracking after the occlusion disappears, and finally realize the scale-adaptive memory tracking of the target, and enhance the anti-short-term occlusion performance of the target tracking. Therefore, the method of the present invention effectively solves the problem of target tracking loss caused by large target scale changes and target occlusion in any complex scene, and realizes target scale adaptive tracking and anti-occlusion memory tracking.

[0091] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, which can implement the single target memory tracking method process of any complex scene scale adaptive provided in the above embodiment of the present application. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device. Figure 3 As shown, the electronic device may include:

[0092] At least one processor, and a memory connected to the at least one processor. The specific connection medium between the processor and the memory is not limited in the embodiment of the present invention. Figure 3 The example in this article is that the processor and memory are connected through a bus. Figure 3 The connections between other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into address bus, data bus, control bus, etc. Figure 3In the diagram, only one thick line is used, but this does not mean that there is only one bus or only one type of bus. Alternatively, a processor can also be called a controller, and there is no limitation on the name.

[0093] In an embodiment of the present invention, the memory stores instructions that can be executed by at least one processor. By executing the instructions stored in the memory, the at least one processor can execute the above-mentioned single target memory tracking method for arbitrarily complex scene scale adaptation. The processor can implement Figure 3 The functions of each module in the device shown.

[0094] Among them, the processor is the control center of the device, which can use various interfaces and lines to connect the various parts of the entire control device, and monitor the device as a whole by running or executing instructions stored in the memory and calling data stored in the memory, various functions of the device and processing data.

[0095] In an optional design, the processor may include one or more processing units, and the processor may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor. In some embodiments, the processor and the memory may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0096] The processor may be a general-purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of a single-target memory tracking method for adaptive scale of an arbitrary complex scene disclosed in the embodiments of the present invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0097] As a non-volatile computer-readable storage medium, the memory can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiment of the present invention can also be a circuit or any other device that can realize a storage function, used to store program instructions and / or data.

[0098] By designing and programming the processor, the code corresponding to the method for single target memory tracking in an arbitrarily complex scene scale adaptive mode introduced in the above embodiment can be fixed into the chip, so that the chip can perform the steps of the method for single target memory tracking in an arbitrarily complex scene scale adaptive mode in the above embodiment when running. How to design and program the processor is a technology known to those skilled in the art and will not be described in detail here.

[0099] Based on the same inventive concept, an embodiment of the present invention further provides a storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer executes the single target memory tracking method for arbitrarily complex scene scale adaptation discussed above.

[0100] In some optional embodiments, the present application also provides various aspects of a single target memory tracking method that is scale-adaptive for arbitrarily complex scenes, which can also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of a single target memory tracking method that is scale-adaptive for arbitrarily complex scenes according to various exemplary embodiments of the present application described above in this specification.

[0101] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of a unit described above can be further divided into multiple units to be embodied. In addition, although the operation of the method of the present application is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a server, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] Program code for performing the operations of the present application may be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0105] Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A single target memory tracking method with adaptive scale in any complex scene, characterized in that: The steps include: Step 1: Initialize the FDSST single target image tracking algorithm to obtain the target filter template, including the position filter template and the scale filter template; Step 2: Obtain real-time video frames to predict the position and scale of the target, and perform online training to update the filter template of the target; Step 3: Cache the target filter template, perform correlation filtering on the cached target filter template and the historical filter template to obtain the first occlusion response value of the target; Step 4: Determine whether the first occlusion response value is lower than the set occlusion threshold: If the first occlusion response value is lower than the set occlusion threshold, the target is determined to be completely occluded, and the memory tracking state is entered. The trajectory of the target is pre-pushed and the scale is maintained according to the historical trajectory, and step 5 is performed; If the first occlusion response value is not lower than the set occlusion threshold, it is determined that the target is not occluded, and the process returns to step 2; Step 5: Continue to perform correlation filtering on the target filter template and the historical filter template to obtain the second occlusion response value of the target: If the second occlusion response value is lower than the set occlusion threshold, the memory tracking state is maintained and step 6 is performed; If the second occlusion response value is not lower than the set occlusion threshold, restore to FDSST tracking and return to step 2; Step 6: Determine whether the memory tracking state time exceeds the set time threshold: If the memory tracking state time exceeds the set time threshold, the target is considered lost, and the single target tracking is exited, waiting for receiving a new target and returning to step 1; If the memory tracking state time does not exceed the set time threshold, the memory tracking state is maintained.

2. The scale-adaptive single target memory tracking method for arbitrary complex scenes according to claim 1 is characterized in that: Step 1 specifically includes the following sub-steps: Step 1.1, extract the fhog features of the target; Step 1.2, using the extracted fhog features to train the FDSST single target image tracking algorithm to obtain a filter template of the target, including a position filter template and a scale filter template.

3. The scale-adaptive single target memory tracking method for arbitrary complex scenes according to claim 2 is characterized in that: The filtering template of the target is expressed as: Among them, H l represents the filter template, d represents the fhog feature dimension, and l represents a certain dimension feature; the desired filter response output result is subjected to discrete Fourier transform and conjugated to obtain The result of discrete Fourier transform after the feature of the lth dimension of the sample is mapped to a high dimension by Gaussian kernel function is F l ; and so on F k It represents the result of discrete Fourier transform after the k-th dimension feature of the sample is mapped to a high dimension by Gaussian kernel function. F k , λ is a small quantity that prevents the denominator from dividing by zero.

4. The scale-adaptive single target memory tracking method for arbitrary complex scenes according to claim 3 is characterized in that: The extracted fhog features are used to train the FDSST single target image tracking algorithm, including: The filter template H l is equivalent to: The target prediction is achieved by performing correlation filtering operation on the fhog feature map of the current search area and the filter template of the previous frame, thereby updating A and B to complete the filter template update, which is expressed as: in, and Represents the filter template H of the previous frame l The numerator and denominator of , η is the learning rate.

5. The scale-adaptive single target memory tracking method for arbitrary complex scenes according to claim 4 is characterized in that: In step 3, the cached target filter template and the historical filter template are correlated and filtered to obtain a predicted response map, and the maximum response value in the predicted response map is the first occlusion response value of the target; The predicted response graph y is expressed as: Among them, Z l Represents the fhog feature map of the search area in the current frame, F -1 Represents the inverse Fourier transform operation after frequency domain multiplication.

6. The scale-adaptive single target memory tracking method for any complex scene according to claim 1, characterized in that: In step 4, the trajectory prediction and scale maintenance of the target based on the historical trajectory include: Step 4.1, smoothing the target historical trajectory; Step 4.2: Predict particle trajectory and maintain scale based on the smoothed target historical trajectory.

7. The scale-adaptive single target memory tracking method for arbitrary complex scenes according to claim 1 is characterized in that: In step 4, when the first occlusion response value is lower than the set occlusion threshold value for multiple consecutive times, the target is determined to be completely occluded and enters the memory tracking state.

8. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the method as described in any one of claims 1 to 7 by executing the instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that When the computer program product is called by a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images