Target tracking method, device, electronic device and storage medium

Abstract

The present application proposes a target tracking method, device, electronic device and storage medium, including: S1, obtaining an initial video frame, training a position-related filter; S2, adding a short-term memory block to the position-related filter; S3, obtaining the next video frame, obtaining the target position through the position-related filter, calculating the first motion state information of the current video frame and updating it to the short-term memory block; S4, expanding the target tracking frame according to the first motion state information of the previous video frame to obtain the target search frame of the current video frame; S5, proportionally enlarging or reducing the target tracking frame of the previous video frame to obtain a multi-scale sliding detection frame, and traversing the sliding detection in the target search frame of the current video frame in turn, and updating the sliding detection frame corresponding to the maximum response value to the target tracking frame of the current video frame; S6, repeating steps S3 to S5, traversing all video frames. The target tracking method of the present application is not easy to lose the target, and improves reliability.

Description

Technical Field

[0001] The present application relates to the technical field of target tracking, and in particular to a target tracking method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of UAV technology, UAVs have been widely used in agriculture, surveying and mapping, film production, security monitoring and other fields. Among them, the target tracking technology of UAV gimbal camera has been a hot topic in recent years, which can realize automatic recognition and continuous tracking of ground or air targets.

[0003] Target tracking is to select the target of interest in each frame of the video and then track it continuously. Traditional target tracking usually uses a correlation filter algorithm. First, an image block is trained through the first frame of the image. The size of the trained image block and the image block to be subsequently detected must be consistent. Then, a search box with a fixed size range is determined in the next frame of the image. The detected image block is traversed and slidingly detected in the search box. When the global maximum response value is detected, it means that the target is found again in the next frame of the image.

[0004] However, during the target tracking process, due to the different flying altitudes of the drones, the near and far scales of the target in the drone's perspective are quite different, and the scale of the same target at the same position will also have a certain deviation under different shooting angles. That is to say, the image block size trained in the first frame may not match the target in the subsequent frame image, resulting in target information loss or the addition of some unnecessary background information, leading to detection failure. In addition, since the size range of the search box is fixed, once the target has partially or completely left the search box area when entering the next frame image, it will also cause target information loss, resulting in tracking failure. Summary of the invention

[0005] In order to solve the technical problem that the UAV target tracking method in the prior art is prone to target loss and cannot continue tracking, the present application proposes a target tracking method, device, electronic device and storage medium.

[0006] According to the first aspect of the present application, a target tracking method is proposed, comprising the steps of:

[0007] S1, obtaining an initial video frame containing a tracked target, and training a position-related filter according to the initial video frame, wherein the initial video frame includes a target tracking frame containing the tracked target, and the position-related filter is used to predict a target position of a subsequent video frame according to a target position of a previous video frame;

[0008] S2, adding a short-term memory block to the position-dependent filter;

[0009] S3, obtaining the next video frame, recording it as the current video frame, obtaining the target position of the current video frame through the position-related filter, obtaining the first motion state information of the tracked target in the current video frame according to the target positions of the previous and next two video frames, and updating and saving the first motion state information in the short-term memory block, wherein the first motion state information includes direction information and speed information;

[0010] S4, enlarging the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expanding the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame to obtain a target search frame of the current video frame;

[0011] S5, enlarging and / or reducing the target tracking frame of the previous video frame of the current video frame in equal proportion to obtain a multi-scale sliding detection frame, and sequentially traversing the sliding detection of the multi-scale sliding detection frame in the target search frame of the current video frame, calculating the response value by the position-related filter, updating the sliding detection frame corresponding to the maximum response value as the target tracking frame of the current video frame, and training and updating the position-related filter;

[0012] S6. Repeat steps S3 to S5 until all video frames are traversed.

[0013] Preferably, the step S2 further comprises:

[0014] Adding a long-term memory block to the position-dependent filter;

[0015] The step S3 further comprises:

[0016] Recording the first motion state information of the historical video frame, obtaining a motion state information set and saving it in the long-term memory block;

[0017] Before step S4, the method further includes:

[0018] Determine whether the direction information of N video frames before the current video frame is consistent, where N is a preset value;

[0019] If yes, average acceleration information is calculated by using the speed information of N video frames before the current video frame, and second motion state information of the current video frame is predicted based on the average acceleration information and the first motion state information of the video frame before the current video frame;

[0020] A target tracking frame of a video frame preceding the current video frame is enlarged according to a preset ratio, and the enlarged target tracking frame is expanded according to the second motion state information to obtain a target search frame of the current video frame;

[0021] If not, execute step S4.

[0022] Preferably, the step S5 specifically includes:

[0023] Based on the direction information of a video frame before the current video frame, the multi-scale sliding detection frame is sequentially traversed for sliding detection in a target search frame of the current video frame toward a specified direction;

[0024] During the sliding detection process of the sliding detection frame at each scale, when the position-dependent filter calculates a peak response value, the sliding detection is stopped and the peak response value is recorded;

[0025] The peak response values ​​of the multi-scale sliding detection frames are compared, and the sliding detection frame corresponding to the maximum response value is updated as the target tracking frame of the current video frame.

[0026] Preferably, in step S4, the enlarging the amplified target tracking frame according to the first motion state information of the previous video frame of the current video frame specifically includes:

[0027] Obtaining a moving direction of the tracking target according to the direction information of a video frame before the current video frame, and determining an expansion direction of the target tracking frame based on the moving direction of the tracking target;

[0028] The moving distance L1 of the tracking target is obtained according to the speed information of the previous video frame of the current video frame, and the enlarged distance L2 of the target tracking frame is determined based on the moving distance L1 of the tracking target, wherein L2≥L1.

[0029] Preferably, the step S2 further comprises:

[0030] adding a scaling factor to the position-dependent filter;

[0031] The expression of the scaling factor is as follows:

[0032] β ±i , i=0,1,2,3,...,k

[0033] Among them, β is the scaling factor, and k is used to represent the number of scales;

[0034] In step S5, the target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced in equal proportion to obtain a multi-scale sliding detection frame, including:

[0035] The target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced according to the scaling factor to obtain a multi-scale sliding detection frame.

[0036] Preferably, in step S1, the training process of the position-dependent filter specifically includes:

[0037] Selecting a target tracking frame including the tracking target in the initial video frame;

[0038] Extracting HOG features from the target tracking frame of the tracking target and using it as a training sample;

[0039] The position-dependent filter is obtained by training according to the training samples.

[0040] According to a second aspect of the present application, a target tracking device is proposed, comprising:

[0041] A training module is configured to obtain an initial video frame including a tracked target, and train a position-related filter according to the initial video frame, wherein the initial video frame includes a target tracking frame including the tracked target, and the position-related filter is used to predict a target position of a subsequent video frame according to a target position of a previous video frame;

[0042] An improvement module configured to add a short-term memory block to the position-dependent filter;

[0043] a calculation module configured to obtain a next video frame, record it as a current video frame, obtain a target position of the current video frame through the position-related filter, obtain first motion state information of the tracked target in the current video frame according to the target positions of the previous and next two video frames, and update and save the first motion state information in the short-term memory block, wherein the first motion state information includes direction information and speed information;

[0044] A first execution module is configured to enlarge the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expand the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame to obtain the target search frame of the current video frame;

[0045] A tracking module is configured to enlarge and / or reduce the target tracking frame of the previous video frame of the current video frame in equal proportion to obtain a multi-scale sliding detection frame, and sequentially perform sliding detection on the multi-scale sliding detection frame in the target search frame of the current video frame, calculate a response value using the position-related filter, update the sliding detection frame corresponding to the maximum response value to the target tracking frame of the current video frame, and train and update the position-related filter;

[0046] A loop module is configured to repeat the calculation module, the first execution module and the tracking module in sequence until all video frames are traversed.

[0047] Preferably, the improvement module is further configured to add a long-term memory block to the position-dependent filter;

[0048] The calculation module is further configured to record the first motion state information of the historical video frame, obtain a motion state information set and save it in the long-term memory block;

[0049] The device further comprises a judgment module before entering the first execution module;

[0050] The judging module is configured to judge whether the direction information of the N video frames before the current video frame is consistent, where N is a preset value; if so, enter the second execution module; if not, enter the first execution module;

[0051] The second execution module is configured to calculate average acceleration information through the speed information of the previous N video frames of the current video frame, predict the second motion state information of the current video frame according to the average acceleration information and the first motion state information of the previous video frame of the current video frame; enlarge the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expand the enlarged target tracking frame according to the second motion state information to obtain the target search frame of the current video frame;

[0052] The loop module is configured to repeat the calculation module, the judgment module, the second execution module, the first execution module and the tracking module in sequence until all video frames are traversed.

[0053] According to the third aspect of the present application, an electronic device is proposed, comprising one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements a target tracking method as provided in any one of the embodiments of the first aspect above.

[0054] According to a fourth aspect of the present application, a computer-readable storage medium is proposed, on which a computer program is stored. When the program is executed by a processor, the target tracking method provided in any embodiment of the first aspect is implemented.

[0055] The present application proposes a target tracking method, device, electronic device and storage medium, which at least have the following beneficial effects:

[0056] (1) Based on the traditional correlation filter algorithm, a short-term memory block is added to the position correlation filter and a short-term memory strategy is introduced. First, the target tracking frame of the previous video frame is enlarged according to a preset ratio, and then the enlarged target tracking frame is enlarged according to the first motion state information of the previous video frame to obtain the target search frame of the next video frame. Then, a multi-scale sliding detection frame is obtained by introducing a scaling factor. The multi-scale sliding detection frame is used to perform sliding detection in the target search frame to obtain the target tracking frame of the next video frame, thereby achieving target tracking. Since the target search frame introduces a short-term memory strategy, it can largely avoid the target leaving the target search frame in the next video frame, resulting in target information loss. In addition, the target tracking frame can adaptively change its size according to the scale change of the tracked target during the motion process, thereby avoiding the loss of target information or the addition of some unnecessary background information, thereby avoiding the situation where the target is lost and cannot be tracked.

[0057] (2) Since target tracking is a continuous process, a long-term memory block is added to the position-related filter and a long-term memory strategy is introduced. When it is determined that the direction information of the tracked target in the previous N video frames is consistent, it can be determined that the tracked target is moving in the same direction during this period. Therefore, by calculating the average acceleration of the tracked target in the previous N video frames and superimposing the movement speed of the tracked target in the previous video frame with the average acceleration, the movement speed of the tracked target in the next video frame can be predicted, thereby more accurately expanding the target search box in the next video frame and improving the target detection accuracy.

[0058] (3) The moving direction of the tracking target can be determined based on the direction information of the tracking target in the previous video frame. Therefore, a detection strategy is introduced for the position-related filter. The sliding detection frame at each scale performs sliding detection in the target search frame according to the moving direction of the tracking target. When the peak response value is detected, it is considered that the sliding detection frame at this scale is already in the best detection position, and the detection is stopped. Then, by comparing the peak response values ​​of the sliding detection frames at multiple scales, the sliding detection frame corresponding to the maximum response value is used as the target tracking frame of the re-detected tracking target, thereby improving the efficiency of re-detecting the target and further improving the efficiency of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.

[0060] Figure 1 is a flow chart of a target tracking method according to a specific embodiment of the present application;

[0061] Figure 2 is a schematic diagram of generating a target search box according to a specific embodiment of the present application;

[0062] Figure 3 is a flow chart of a detection strategy according to a specific embodiment of the present application;

[0063] Figure 4 is a flow chart of a long-term memory strategy according to a specific embodiment of the present application;

[0064] Figure 5 It is an architectural diagram of a target tracking device according to a specific embodiment of the present application. DETAILED DESCRIPTION

[0065] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0066] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0067] This application proposes a target tracking method. Figure 1 A flow chart of a target tracking method according to a specific embodiment of the present application is shown. Figure 1 As shown, the method comprises the steps of:

[0068] S1. Obtain an initial video frame containing a tracked target, and train a position-related filter based on the initial video frame. The initial video frame includes a target tracking frame containing the tracked target. The position-related filter is used to predict a target position of a subsequent video frame based on a target position of a previous video frame.

[0069] Specifically, the video stream of the tracking target is collected by an image acquisition device, and the nth frame is taken as the initial video frame, where n≥1. In this embodiment, the first frame of the video frame is taken as the initial video frame, and the initial video frame is used as a training sample to first train a position-related filter, and the position-related filter is used to predict the target position of the next frame of the video frame according to the target position of the previous frame of the video frame. The training process of the position-related filter is as follows:

[0070] A target tracking frame containing the tracked target is manually selected in the initial video frame, and the target position of the initial video frame is known, that is, the center coordinates of the target tracking frame; HOG features are extracted from the target tracking frame of the tracked target as a training sample; and a position-related filter is obtained by training based on the training sample.

[0071] In this embodiment, the image acquisition device is specifically a gimbal pod carried by a drone. In other embodiments, the image acquisition device can be any other device with an image acquisition function, which is not limited here.

[0072] S2. Add a short-term memory block to the position-dependent filter.

[0073] Target tracking is a continuous process. If contextual connections are added between two or more consecutive video frames, the reliability of target tracking can be improved. In this embodiment, a short-term memory block is added to the position-related filter and a short-term memory strategy is introduced. The short-term memory block is updated as the position-related filter enters the next video frame iterative training update. The short-term memory block is used to update and store the information of the previous video frame.

[0074] S3. Get the next video frame and record it as the current video frame. Obtain the target position of the current video frame through a position-related filter. Obtain the first motion state information of the tracked target in the current video frame according to the target positions of the two previous and next video frames. Update and save the first motion state information in the short-term memory block. The first motion state information includes direction information and speed information.

[0075] In a specific embodiment, when the video frame rate is known, the time between each frame can be obtained, and the movement speed of the target position in the current video frame can be obtained by calculating the offset of the target position of the current video frame relative to the target position of the previous video frame, thereby obtaining the first movement state information. The first movement state information is a motion vector, including the direction information of the tracking target in the x and y directions, and the speed information of the tracking target in the x and y directions.

[0076] It should be noted that, when the current video frame is the second video frame, the previous video frame of the current video frame (i.e., the first video frame) has no motion state information. At this time, the short-term memory block does not update the first motion state information. Therefore, the default tracking target does not move in the x and y directions, and the motion speed is 0.

[0077] S4, enlarging the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expanding the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame to obtain a target search frame of the current video frame.

[0078] Figure 2 A schematic diagram of generating a target search box according to a specific embodiment of the present application is shown. Figure 2 Specifically, the moving direction of the target is obtained according to the direction information of the previous video frame of the current video frame, and the enlargement direction of the target tracking frame is determined based on the moving direction of the target. In this embodiment, the moving direction of the target is directly used as the enlargement direction of the target tracking frame.

[0079] It should be noted that, considering that when the tracking target changes from a static state to a moving state, the tracking target has no speed in the previous video frame, and after entering the next video frame, the first motion state information of the previous video frame is 0. In this case, if the target search frame of the next video frame is completely determined by the first motion state information of the previous video frame, the target search frame is actually the target tracking frame of the previous video frame. But in fact, the tracking target has a moving speed after the next video frame changes to a moving state. At this time, the tracking target has partially or completely run out of the target tracking frame of the previous video frame. At this time, the complete tracking target can no longer be detected in the target search frame.

[0080] Therefore, in this embodiment, the target tracking frame of the previous video frame of the current video frame is first enlarged according to a preset ratio, and then the enlarged target tracking frame is enlarged according to the first motion state information of the previous video frame of the current video frame to obtain the target search frame of the current video frame. Since the target tracking frame of the previous video frame is enlarged once in all directions, when the tracking target changes from a static state to a moving state, no matter in which direction the tracking target moves, it is still in the target tracking frame of the previous video frame. In this way, it is ensured that when the tracking target enters the next video frame, the tracking target is still in the target search frame of the next video frame.

[0081] After the target tracking frame of the previous video frame of the current video frame is enlarged, the enlarged distance of the target tracking frame in all directions is X1, and then the moving distance L1 of the tracking target is obtained according to the speed information of the previous video frame of the current video frame, and the enlarged distance L2 of the target tracking frame is determined based on the moving distance L1 of the tracking target, where L2 ≥ L1, and the moving distance L1 of the tracking target in the x and y directions can be directly calculated by the moving speed of the tracking target in the x and y directions. Finally, the superimposed enlarged distance of the target tracking frame in the moving direction of the tracking target is X1+L2.

[0082] In this embodiment, L1=L2 is taken, and the moving distance of the tracking target is directly used as the expansion distance of the target tracking frame to obtain the target search frame of the current video frame. In other embodiments, considering that the tracking target may just appear near the boundary of the target search frame, the boundary information of the tracking target may be filtered out, resulting in the inability to re-detect the complete tracking target, so L2>L1 can be set, and the value of L2 can be slightly larger than L1, that is, an empirical value is added on the basis of L1, and the empirical value can be set according to actual needs.

[0083] It should also be noted that when the current video frame is the second video frame, the previous video frame of the current video frame (i.e., the first video frame) has no motion state information, and the default tracking target does not move in the x and y directions, and the motion speed is 0. Therefore, at this time, L1=0, and when L1=L2, the target tracking frame of the first video frame is enlarged according to the preset ratio to become the target search frame of the second video frame.

[0084] S5. The target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced in equal proportion to obtain a multi-scale sliding detection frame, and the multi-scale sliding detection frame is sequentially traversed for sliding detection in the target search frame of the current video frame. The position-related filter calculates the response value, and the sliding detection frame corresponding to the maximum response value is updated to the target tracking frame of the current video frame, and the position-related filter is trained and updated.

[0085] In this embodiment, step S2 further includes: adding a scaling factor to the position-dependent filter. The scaling factor is expressed as follows:

[0086] β ±i , i=0,1,2,3,...,k

[0087] Among them, β is the scaling factor, and k is used to represent the number of scales, that is, the number of scales is 2k+1.

[0088] In this embodiment, in step S5, the target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced in equal proportion to obtain a multi-scale sliding detection frame, specifically including: enlarging and / or reducing the target tracking frame of the previous video frame of the current video frame according to the scaling factor to obtain a multi-scale sliding detection frame.

[0089] Specifically, by scaling the target tracking frame of the previous video frame of the current video frame according to a series of scaling ratios released by the scaling factor, a multi-scale sliding detection frame can be obtained. Then, the multi-scale sliding detection frame is sequentially traversed in the target search frame of the current video frame for sliding detection, and then combined with the response value output by the position-related filter, the sliding detection frame corresponding to the maximum response value is updated to the target tracking frame of the current video frame. Thus, the target tracking frame can adaptively change its size according to the scale change of the tracked target during the motion process, avoiding the loss of target information or the addition of some unnecessary background information, thereby avoiding the situation where the target is lost and cannot be tracked.

[0090] In this embodiment, the series of scaling ratios released by the scaling factor include scaling ratios less than 1, 1, and greater than 1. In other implementations, the series of scaling ratios released by the scaling factor may include only less than 1 or only greater than 1, which is not limited here.

[0091] It should be noted that after entering each new video frame and updating the target tracking frame, the position-dependent filter will be retrained to ensure the target tracking accuracy when entering the next video frame.

[0092] S6. Repeat steps S3 to S5 until all video frames are traversed.

[0093] In this embodiment, in order to improve the sliding detection efficiency, a detection strategy is introduced into the position correlation filter. Figure 3 A flow chart of a detection strategy according to a specific embodiment of the present application is shown. Figure 3 As shown, step S5 specifically includes:

[0094] S51 , based on the direction information of a video frame before the current video frame, slide detection is performed in sequence on a multi-scale sliding detection frame in a target search frame of the current video frame in a specified direction.

[0095] Specifically, the sliding detection frame starts from the initial position in the target search frame and performs sliding detection according to the moving direction of the tracked target. Assuming that the moving direction of the tracked target is the lower right corner direction of the target search frame, the initial position is the upper left corner of the target search frame.

[0096] S52 . During the sliding detection process of the sliding detection frame at each scale, when the position-dependent filter calculates a peak response value, the sliding detection is stopped and the peak response value is recorded.

[0097] S53, comparing the peak response values ​​of the sliding detection frames of multiple scales, and updating the sliding detection frame corresponding to the maximum response value as the target tracking frame of the current video frame.

[0098] Specifically, the sliding detection frame at each scale performs sliding detection in the target search frame according to the movement direction of the tracking target, and the position-related filter calculates the response value. When the peak response value appears, it can be considered that the sliding detection frame at this scale is already in the best detection position, and the detection is stopped at this time. The sliding detection frames at multiple scales are traversed and detected, and the peak response values ​​of the sliding detection frames at multiple scales are compared, and the sliding detection frame corresponding to the maximum response value is used as the target tracking frame of the re-detected tracking target.

[0099] The detection strategy provided in this embodiment can save sliding detection of part of the target search box area, thereby improving the efficiency of re-detecting the target and further improving the efficiency of target tracking.

[0100] Based on the target tracking method provided in the above embodiment, the present application also provides another preferred embodiment of the target tracking method, which introduces a long-term memory strategy.

[0101] In this embodiment, step S2 further includes: adding a long-term memory block to the position-dependent filter.

[0102] In this embodiment, step S3 further includes: recording the first motion state information of the historical video frame, obtaining a motion state information set and saving it in a long-term memory block.

[0103] Figure 4 A flowchart of a long-term memory strategy according to a specific embodiment of the present application is shown. Figure 4 As shown, after step S3 and before step S4, it also includes:

[0104] S3a, determine whether the direction information of the N video frames before the current video frame is consistent, where N is a preset value; if so, execute steps S3b and S3c in sequence, and directly enter step S5; if not, execute step S4.

[0105] When it is determined that the direction information of the N video frames before the current video frame is consistent, it can be determined that the tracked target is continuously moving in one direction during the period.

[0106] S3b, calculating average acceleration information through speed information of N video frames before the current video frame, and predicting second motion state information of the current video frame according to the average acceleration information and first motion state information of the video frame before the current video frame.

[0107] After determining that the tracked target is moving continuously in one direction during the period, the movement speed of the tracked target in the current video frame can be more accurately predicted by introducing the average acceleration and superimposing it with the movement speed of the tracked target in the previous video frame.

[0108] S3c, enlarging the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expanding the enlarged target tracking frame according to the second motion state information to obtain a target search frame of the current video frame.

[0109] Since target tracking is a continuous process, a long-term memory block is added to the position-related filter and a long-term memory strategy is introduced to increase contextual connections. When it is determined that the direction information of the tracked target in the first N video frames is consistent, it can be determined that the tracked target is moving in the same direction during this period. Therefore, by calculating the average acceleration of the tracked target in the first N video frames, the movement speed of the tracked target in the previous video frame is superimposed with the average acceleration, and the movement speed of the tracked target in the next video frame can be predicted. Then, the predicted movement speed of the tracked target in the next video frame is converted into the movement distance of the tracked target in the next video frame, so as to more accurately expand the target search box of the next video frame. At this time, the target search box is closer to the area where the tracked target may appear, thereby improving the accuracy of target detection.

[0110] In one embodiment provided in the present application, the long-term memory block may only update and retain the first motion state information of the N video frames before the current video frame, and after entering each next video frame, update the data of the long-term memory block to keep the data length of the long-term memory block at N.

[0111] The long-term memory strategy of this embodiment has a good tracking effect when the target is continuously accelerating or decelerating, and the long-term memory strategy does not take up too much memory and computing power. It is adapted to the fact that drones can only be equipped with low-computing power chips and is suitable for application scenarios of drone target tracking.

[0112] Based on the above target tracking method, the present application also proposes a target tracking device. Figure 5 FIG. 4 shows an architecture diagram of a target tracking device according to a specific embodiment of the present application. Figure 5 As shown, the device includes:

[0113] The training module 100 is configured to obtain an initial video frame including a tracked target, and to obtain a position-related filter based on training of the initial video frame. The initial video frame includes a target tracking frame including the tracked target. The position-related filter is used to predict a target position of a subsequent video frame based on a target position of a previous video frame.

[0114] The improvement module 200 is configured to add a short-term memory block and a long-term memory block to the position-related filter.

[0115] The calculation module 300 is configured to obtain the next frame of video frame, record it as the current video frame, obtain the target position of the current video frame through a position-related filter, obtain the first motion state information of the tracked target in the current video frame according to the target positions of the previous and next two frames of video frames, and update and save the first motion state information in a short-term memory block, wherein the first motion state information includes direction information and speed information; record the first motion state information of the historical video frame, obtain a motion state information set and save it in a long-term memory block.

[0116] The judgment module 400 is configured to judge whether the direction information of the N video frames before the current video frame is consistent, where N is a preset value; if so, enter the second execution module; if not, enter the first execution module.

[0117] The second execution module 500 is configured to calculate average acceleration information through speed information of N frames before the current video frame, predict second motion state information of the current video frame based on the average acceleration information and first motion state information of the frame before the current video frame; enlarge the target tracking frame of the frame before the current video frame according to a preset ratio, and expand the enlarged target tracking frame according to the second motion state information to obtain a target search frame of the current video frame.

[0118] The first execution module 600 is configured to expand the target tracking frame based on the target tracking frame of the previous video frame of the current video frame according to the first motion state information of the previous video frame of the current video frame to obtain the target search frame of the current video frame.

[0119] The tracking module 700 is configured to enlarge or reduce the target tracking frame of the previous video frame of the current video frame in equal proportion to obtain a multi-scale sliding detection frame, and sequentially traverse the multi-scale sliding detection frame in the target search frame of the current video frame for sliding detection, calculate the response value by the position-related filter, update the sliding detection frame corresponding to the maximum response value to the target tracking frame of the current video frame, and train and update the position-related filter.

[0120] The loop module 800 is configured to repeat the calculation module, the judgment module, the second execution module, the first execution module and the tracking module in sequence until all video frames are traversed.

[0121] Based on the above target tracking method, the present application also proposes an electronic device, including: one or more processors, and a memory.

[0122] Among them, the memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the electronic device implements the target tracking method provided by any of the above-mentioned embodiments.

[0123] Based on the above target tracking method, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the target tracking method provided by any of the above embodiments is implemented.

[0124] In the embodiments of the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device / system / method embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0126] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0128] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and changes are within the scope of the claims of the present invention and their equivalents, the present invention is also intended to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recorded in mutually different dependent claims does not indicate that the combination of these measures cannot be used to profit. Any reference numerals in the claims should not be considered to limit the scope.

Claims

1. A target tracking method, characterized in that: Includes steps: S1, obtaining an initial video frame containing a tracked target, and training a position-related filter according to the initial video frame, wherein the initial video frame includes a target tracking frame containing the tracked target, and the position-related filter is used to predict a target position of a subsequent video frame according to a target position of a previous video frame; S2, adding a short-term memory block to the position-dependent filter; S3, obtaining the next video frame, recording it as the current video frame, obtaining the target position of the current video frame through the position-related filter, obtaining the first motion state information of the tracked target in the current video frame according to the target positions of the previous and next two video frames, and updating and saving the first motion state information in the short-term memory block, wherein the first motion state information includes direction information and speed information; S4, enlarging the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expanding the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame to obtain a target search frame of the current video frame; S5, enlarging and / or reducing the target tracking frame of the previous video frame of the current video frame in equal proportion to obtain a multi-scale sliding detection frame, and sequentially traversing the sliding detection of the multi-scale sliding detection frame in the target search frame of the current video frame, calculating the response value by the position-related filter, updating the sliding detection frame corresponding to the maximum response value as the target tracking frame of the current video frame, and training and updating the position-related filter; S6, repeating steps S3 to S5 until all video frames are traversed; Wherein, in the step S4, the enlarging the amplified target tracking frame according to the first motion state information of the previous video frame of the current video frame specifically includes: Obtaining a moving direction of the tracking target according to the direction information of a video frame before the current video frame, and determining an expansion direction of the target tracking frame based on the moving direction of the tracking target; Obtaining a movement distance L1 of the tracking target according to the speed information of a video frame before the current video frame, and determining an expansion distance L2 of the target tracking frame based on the movement distance L1 of the tracking target, wherein L2≥L1; The step S5 specifically includes: Based on the direction information of a video frame before the current video frame, the multi-scale sliding detection frame is sequentially traversed for sliding detection in a target search frame of the current video frame toward a specified direction; During the sliding detection process of the sliding detection frame at each scale, when the position-dependent filter calculates a peak response value, the sliding detection is stopped and the peak response value is recorded; The peak response values ​​of the multi-scale sliding detection frames are compared, and the sliding detection frame corresponding to the maximum response value is updated as the target tracking frame of the current video frame.

2. The method according to claim 1, characterized in that The step S2 further comprises: Adding a long-term memory block to the position-dependent filter; The step S3 further comprises: Recording the first motion state information of the historical video frame, obtaining a motion state information set and saving it in the long-term memory block; Before step S4, the method further includes: Determine whether the direction information of N video frames before the current video frame is consistent, where N is a preset value; If yes, average acceleration information is calculated by using the speed information of N video frames before the current video frame, and second motion state information of the current video frame is predicted based on the average acceleration information and the first motion state information of the video frame before the current video frame; A target tracking frame of a video frame preceding the current video frame is enlarged according to a preset ratio, and the enlarged target tracking frame is expanded according to the second motion state information to obtain a target search frame of the current video frame; If not, execute step S4.

3. The method according to claim 1, characterized in that The step S2 further comprises: adding a scaling factor to the position-dependent filter; The expression of the scaling factor is as follows: β ±i ,i=0,1,2,3,...,k Among them, β is the scaling factor, and k is used to represent the number of scales; In step S5, the target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced in equal proportion to obtain a multi-scale sliding detection frame, including: The target tracking frame of the previous video frame of the current video frame is enlarged and / or reduced according to the scaling factor to obtain a multi-scale sliding detection frame.

4. The method according to claim 1, characterized in that In step S1, the training process of the position-dependent filter specifically includes: Selecting a target tracking frame including the tracking target in the initial video frame; Extracting HOG features from the target tracking frame of the tracking target and using it as a training sample; The position-dependent filter is obtained by training according to the training samples.

5. A target tracking device, characterized in that: include: A training module is configured to obtain an initial video frame including a tracked target, and train a position-related filter according to the initial video frame, wherein the initial video frame includes a target tracking frame including the tracked target, and the position-related filter is used to predict a target position of a subsequent video frame according to a target position of a previous video frame; An improvement module configured to add a short-term memory block to the position-dependent filter; a calculation module configured to obtain a next video frame, record it as a current video frame, obtain a target position of the current video frame through the position-related filter, obtain first motion state information of the tracked target in the current video frame according to the target positions of the previous and next two video frames, and update and save the first motion state information in the short-term memory block, wherein the first motion state information includes direction information and speed information; A first execution module is configured to enlarge the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expand the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame to obtain the target search frame of the current video frame; wherein, the enlarging the enlarged target tracking frame according to the first motion state information of the previous video frame of the current video frame specifically includes: obtaining the motion direction of the tracked target according to the direction information of the previous video frame of the current video frame, and determining the enlargement direction of the target tracking frame based on the motion direction of the tracked target; obtaining the motion distance L1 of the tracked target according to the speed information of the previous video frame of the current video frame, and determining the enlargement distance L2 of the target tracking frame based on the motion distance L1 of the tracked target, wherein L2≥L1; A tracking module is configured to enlarge and / or reduce the target tracking frame of the previous video frame of the current video frame in equal proportion to obtain a multi-scale sliding detection frame, and the multi-scale sliding detection frame is sequentially subjected to sliding detection in the target search frame of the current video frame, the position-related filter calculates a response value, and the sliding detection frame corresponding to the maximum response value is updated as the target tracking frame of the current video frame, and the position-related filter is trained and updated; specifically comprising: based on the direction information of the previous video frame of the current video frame, the multi-scale sliding detection frame is sequentially subjected to sliding detection in the target search frame of the current video frame toward a specified direction; in the sliding detection process of the sliding detection frame of each scale, when the position-related filter calculates a peak response value, the sliding detection is stopped and the peak response value is recorded; the peak response values ​​of the multi-scale sliding detection frames are compared, and the sliding detection frame corresponding to the maximum response value is updated as the target tracking frame of the current video frame; A loop module is configured to repeat the calculation module, the first execution module and the tracking module in sequence until all video frames are traversed.

6. The device according to claim 5, characterized in that: The improvement module is also configured to add a long-term memory block to the position-dependent filter; The calculation module is further configured to record the first motion state information of the historical video frame, obtain a motion state information set and save it in the long-term memory block; The device further comprises a judgment module before entering the first execution module; The judging module is configured to judge whether the direction information of the N video frames before the current video frame is consistent, where N is a preset value; if so, enter the second execution module; if not, enter the first execution module; The second execution module is configured to calculate average acceleration information through the speed information of the previous N video frames of the current video frame, predict the second motion state information of the current video frame according to the average acceleration information and the first motion state information of the previous video frame of the current video frame; enlarge the target tracking frame of the previous video frame of the current video frame according to a preset ratio, and expand the enlarged target tracking frame according to the second motion state information to obtain the target search frame of the current video frame; The loop module is configured to repeat the calculation module, the judgment module, the second execution module, the first execution module and the tracking module in sequence until all video frames are traversed.

7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method as claimed in any one of claims 1 to 4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images