Target tracking method, device, electronic device and storage medium under occlusion state

Abstract

The present application proposes a method, device, electronic device and storage medium for tracking a target in an occlusion state, including: obtaining a current video frame containing a target object, the target object having an occlusion state and motion state information; in response to the target object being occluded, predicting the second motion state information of the target object in the current video frame based on the first motion state information of the target object in the historical video frame; based on the first detection frame of the target object in the previous video frame of the current video frame, determining the second detection frame of the target object in the current video frame according to the second motion state information of the target object in the current video frame. The present application can continuously track the target object when the target object is occluded, and the detection frame of the target object can adaptively change with the scale of the target object, thereby improving the tracking reliability of the target object.

Description

Technical Field

[0001] The present application relates to the technical field of target tracking, and in particular to a method, device, electronic device and storage medium for tracking a target under an occluded state. Background Art

[0002] With the rapid development of UAV technology, UAVs have been widely used in agriculture, surveying and mapping, film production, security monitoring and other fields. Among them, the target tracking technology of UAV gimbal camera has been a hot topic in recent years, which can realize automatic recognition and continuous tracking of ground or air targets.

[0003] The target tracking algorithm in the prior art includes the kernel correlation filter algorithm (KCF), which significantly speeds up the tracking algorithm by introducing the idea of ​​correlation filtering in signal processing into target tracking. However, in practical applications, target tracking is easily disturbed by occlusion, background, environment, etc., and the reliability of the tracking effect will be affected. At the same time, in the process of drone tracking, once the target is occluded and the flight altitude and shooting angle of the drone change, the scale of the target will change significantly, and the KCF algorithm can only continuously track targets of the same scale, and its target detection frame cannot change adaptively with the change of the scale of the target, resulting in a decrease in the reliability of target tracking. Summary of the invention

[0004] In order to solve the technical problem in the prior art that the tracking reliability of UAV target tracking methods decreases when the target is obscured, the present application proposes a target tracking method, device, electronic device and storage medium under an obscured state.

[0005] According to a first aspect of the present application, a method for tracking a target in an occluded state is proposed, comprising:

[0006] Acquire a current video frame containing a target object, wherein the target object has occlusion state and motion state information, wherein the occlusion state includes being occluded and being unoccluded, and the motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information;

[0007] In response to the target object being blocked, predicting second motion state information of the target object in the current video frame based on first motion state information of the target object in a historical video frame;

[0008] Based on the first detection frame of the target object in the previous video frame of the current video frame, and according to the second motion state information of the target object in the current video frame, a second detection frame of the target object in the current video frame is determined.

[0009] Preferably, predicting the second motion state information of the target object in the current video frame based on the first motion state information of the target object in the historical video frame includes:

[0010] Based on the motion speed and scale change information of the target object in the motion direction in the historical video frames, the motion speed and scale change information of the target object in the motion direction of the current video frame are obtained by using an exponentially weighted sliding average calculation;

[0011] The exponentially weighted moving average calculation expression is as follows:

[0012] , , ;

[0013] in, , are the moving speeds of the target object in the x and y directions of the current video frame, respectively; N is the previous N video frames; e is a natural constant; V xi , V yi are the movement speeds of the target object in the x and y directions of the i-th video frame in the previous N video frames, is the scale change information of the target object in the current video frame, K i It is the scale change information of the target object in the i-th video frame of the previous N video frames in history.

[0014] Preferably, the determining, based on the first detection frame of the target object in a previous video frame of the current video frame and according to the second motion state information of the target object in the current video frame, a second detection frame of the target object in the current video frame comprises:

[0015] Determine the moving distance in the x and y directions according to the moving speed of the target object in the x and y directions of the current video frame;

[0016] Based on the first coordinate information of the target object in a previous video frame of the current video frame, determining the second coordinate information of the target object in the current video frame according to the movement distance of the target object in the x and y directions of the current video frame;

[0017] Moving the first detection frame based on the second coordinate information;

[0018] According to the scale change information of the target object in the current video frame, the moved first detection frame is scaled to obtain the second detection frame.

[0019] Preferably, the moving the first detection frame based on the second coordinate information includes:

[0020] Based on the coordinate information of the target object in the historical video frame, a polynomial trajectory fitting is performed, and the objective function is optimized using the least squares method to obtain the trajectory prediction function of the target object. The objective function expression is as follows:

[0021] ;

[0022] in, F (x) is the trajectory prediction function, N is the previous N video frames, T is the order of the high-order polynomial, a is the coefficient of the objective function, and x i ,y i The coordinate information of the target object in the i-th video frame in the previous N video frames in history;

[0023] Substituting the abscissa x1 of the second coordinate information into the trajectory prediction function to calculate the predicted ordinate y2, averaging the ordinate y1 of the second coordinate information and the predicted ordinate y2 to obtain a final ordinate y3, and obtaining the third coordinate information of the target object in the current video frame based on the abscissa x1 of the second coordinate information and the final ordinate y3;

[0024] The first detection frame is moved according to the third coordinate information.

[0025] Preferably, the step of acquiring the coordinate information, the moving direction, and the moving speed in the moving direction of the target object comprises:

[0026] A kernel correlation filter is obtained based on the kernel correlation filter algorithm training;

[0027] According to the coordinate information of the target object in the previous video frame of the current video frame, the coordinate information of the target object in the current video frame is predicted by the kernel correlation filter;

[0028] According to the change of the coordinate information of the target object in the two previous and next video frames, the moving direction and the moving speed of the target object in the current video frame are calculated.

[0029] Preferably, the step of acquiring the scale change information of the target object includes:

[0030] Enlarging the first detection frame according to a first preset ratio and / or a specified direction to obtain a search frame of the target object in the current video frame;

[0031] Enlarging and / or reducing the first detection frame according to a second preset ratio to obtain a multi-scale sliding frame;

[0032] The multi-scale sliding boxes are sequentially traversed in the search box for sliding detection, the kernel correlation filter calculates a response value, and the scale corresponding to the sliding box with the maximum response value is used as the scale change information of the current video frame.

[0033] Preferably, the step of determining the occlusion state of the target object comprises:

[0034] Determine whether the maximum response value calculated by the kernel correlation filter is greater than a preset threshold; if so, determine that the occlusion state of the target object is not occluded; if not, determine that the occlusion state of the target object is occluded.

[0035] According to a second aspect of the present application, a target tracking device in an occluded state is proposed, comprising:

[0036] an acquisition module configured to acquire a current video frame containing a target object, wherein the target object has an occlusion state and motion state information, wherein the occlusion state includes being occluded and being unoccluded, and the motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information;

[0037] A prediction module, configured to predict, in response to the target object being blocked, second motion state information of the target object in the current video frame based on first motion state information of the target object in a historical video frame;

[0038] The tracking module is configured to establish a second detection frame of the target object in the current video frame based on the first detection frame of the target object in the previous video frame of the current video frame and according to the second motion state information of the target object in the current video frame.

[0039] According to the third aspect of the present application, an electronic device is proposed, comprising: one or more processors; a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements a target tracking method under an occluded state as provided in any embodiment of the first aspect above.

[0040] According to a fourth aspect of the present application, a computer-readable storage medium is proposed, on which a computer program is stored. When the program is executed by a processor, a target tracking method under an occluded state as provided in any embodiment of the first aspect is implemented.

[0041] The present application proposes a method, device, electronic device and storage medium for tracking a target under an occlusion state. When the target object is occluded, the motion speed and scale change information of the target object in the current video frame are predicted by an exponentially weighted sliding average calculation method based on the motion speed and scale change information of the target object in the historical video frame, thereby re-determining the detection frame position of the target object. At the same time, a polynomial fitting is performed on the coordinate information of the target object in the historical video frame, and the motion trajectory of the target object is predicted using the least squares method. The prediction result of the exponentially weighted sliding average is then fused with the prediction result of the least squares method, thereby further improving the prediction accuracy. The present application can continuously track the target object when the target object is occluded, and the detection frame of the target object can adaptively change with the scale change of the target object, thereby improving the tracking reliability of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.

[0043] Figure 1 is a flow chart of a method for tracking a target under an occlusion state according to a specific embodiment of the present application;

[0044] Figure 2 is a flow chart of generating a second detection frame according to a specific embodiment of the present application;

[0045] Figure 3 is a flow chart for generating third coordinate information according to a specific embodiment of the present application;

[0046] Figure 4 It is a framework diagram of a target tracking device under an occluded state according to a specific embodiment of the present application. DETAILED DESCRIPTION

[0047] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0048] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0049] The present application proposes a method for tracking a target in an occluded state. Figure 1 A flow chart of a method for tracking a target under an occlusion state according to a specific embodiment of the present application is shown. Figure 1 As shown, the target tracking method includes:

[0050] S1. Acquire a current video frame containing a target object. The target object has occlusion state and motion state information. The occlusion state includes occluded and unoccluded. The motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information.

[0051] Specifically, the current video frame of the target object is captured by an image capture device. In this embodiment, the image capture device is specifically a gimbal pod carried by a drone. In other embodiments, the image capture device can be any other device with an image capture function, which is not limited here.

[0052] In a specific embodiment, each video frame generates a detection frame for tracking and detecting the target object, and the coordinate information is the center coordinate of the target object, that is, the center coordinate of the detection frame. The target object has a moving direction when in motion, and the moving speed in the moving direction can be decomposed into the moving speed in the x direction and the moving speed in the y direction. The scale change information is the scale change ratio of the detection frame of the target object of the current video frame relative to the detection frame of the target object of the previous video frame.

[0053] It should be noted that, in other implementations, the coordinate information may also be the coordinates of the corner points of the target object or the coordinates of any other point, that is, the coordinates of the corner points of the detection frame or the coordinates of any point in the detection frame, which is not limited here.

[0054] In a specific embodiment provided in the present application, the steps of acquiring the coordinate information, the moving direction, and the moving speed in the moving direction of the target object include:

[0055] S11a, a kernel correlation filter is obtained by training based on a kernel correlation filtering algorithm.

[0056] S11b, according to the coordinate information of the target object in the previous video frame of the current video frame, obtain the coordinate information of the target object in the current video frame through kernel correlation filter prediction.

[0057] S11c, calculating the moving direction and moving speed of the target object in the current video frame according to the change of the coordinate information of the target object in the two previous and next video frames.

[0058] The target tracking algorithm of this embodiment is based on the kernel correlation filter algorithm (KCF). The kernel correlation filter is obtained by training the KCF algorithm. The kernel correlation filter can predict the coordinate information of the target object in the next video frame based on the coordinate information (i.e., the center coordinate) of the target object in the previous video frame. Therefore, the moving direction and speed of the target object in the moving direction can be calculated based on the coordinate information difference between the previous and next two video frames.

[0059] In a specific embodiment provided in the present application, the step of acquiring scale change information of the target object in the current video frame includes:

[0060] S12a, enlarging a first detection frame of the target object in a video frame previous to the current video frame according to a first preset ratio and / or a specified direction to obtain a search frame of the target object in the current video frame.

[0061] In a specific embodiment, the center of the first detection frame can be used as a reference point and expanded at a preset ratio to obtain a rectangular search frame with a larger area. The area of ​​the search frame is considered to be the area where the target object may appear in the current video frame. Similarly, according to the movement direction of the target object in the previous video frame of the current video frame, one or more sides of the first detection frame can be expanded in the direction of the side to obtain a rectangular search frame with a larger area; or the first detection frame can be expanded at a preset ratio and in the direction of one or more sides at the same time to obtain a rectangular search frame with a larger area, and no specific restrictions are made here.

[0062] S12b: Enlarge and / or reduce the first detection frame according to a second preset ratio to obtain a multi-scale sliding frame.

[0063] In a specific embodiment, the first detection frame is enlarged and reduced at the same time according to a preset ratio to obtain a smaller and a larger rectangular sliding frame. In other implementations, only the first detection frame may be enlarged or reduced at a preset ratio according to actual needs to obtain multiple smaller or larger rectangular sliding frames, which is not limited here.

[0064] S12c, sliding detection is performed on the sliding boxes of multiple scales in the search box in turn, a kernel correlation filter is calculated to obtain a response value, and the scale corresponding to the sliding box with the maximum response value is used as the scale change information of the current video frame.

[0065] In a specific embodiment, the sliding box of each scale is sequentially traversed and detected in the search box. The detection order of the sliding box can be detected in a specified direction or in a "Z"-shaped detection order, which is not limited here. After the sliding box of each scale is traversed and detected, the scale of the sliding box corresponding to the maximum response value calculated by the kernel correlation filter is used as the scale change information of the current video frame.

[0066] It should be noted that when the target object is not blocked, the multi-scale sliding box traverses the sliding detection in the search box in turn, and the sliding box corresponding to the maximum response value calculated by the kernel correlation filter is the second detection box of the target object in the current video frame, and the kernel correlation filter will be iteratively trained and updated synchronously. When the target object is blocked, the scale of the sliding box corresponding to the maximum response value calculated by the kernel correlation filter is only used as a prediction reference for the current video frame, and the kernel correlation filter is not iteratively trained and updated to ensure the prediction accuracy of the kernel correlation filter.

[0067] In a specific embodiment provided in the present application, the step of determining the occlusion state of the target object includes:

[0068] S13a, determining whether the maximum response value calculated by the kernel correlation filter is greater than a preset threshold; if so, determining that the occlusion state of the target object is not occluded; if not, determining that the occlusion state of the target object is occluded.

[0069] In a specific embodiment, by giving a preset threshold of a response value, when the maximum response value calculated by the kernel correlation filter is greater than the preset threshold, it means that the detected target object is basically close to the complete target object and the target object is not obscured; when the maximum response value calculated by the kernel correlation filter is less than the preset threshold, it means that there is a certain deviation between the detected target object and the complete target object and the target object is obscured.

[0070] Continue to refer to Figure 1 , the target tracking algorithm also includes:

[0071] S2. In response to the target object being blocked, predicting second motion state information of the target object in the current video frame based on first motion state information of the target object in historical video frames.

[0072] In a specific embodiment, after the target object is determined to be blocked in step S13a, the second motion state information of the target object in the current video frame is predicted by the first motion state information of the target object in the historical video frame. The following is a detailed introduction to this step. Step S2 includes:

[0073] S21. Based on the motion speed and scale change information of the target object in the motion direction in the historical video frames, the motion speed and scale change information of the target object in the motion direction of the current video frame are obtained by using an exponentially weighted sliding average calculation.

[0074] Among them, the exponentially weighted moving average calculation expression is as follows:

[0075] , ,

[0076] in, , are the moving speeds of the target object in the x and y directions of the current video frame, N is the previous N video frames, e is a natural constant, V xi , V yi are the moving speeds of the target object in the x and y directions of the i-th video frame in the previous N video frames, is the scale change information of the target object in the current video frame, K i It is the scale change information of the target object in the i-th video frame of the previous N video frames in history.

[0077] In this embodiment, an exponentially weighted sliding average calculation method is used to predict the movement speed and scale change information of the target object in the movement direction of the current video frame based on the movement speed and scale change information of the target object in the movement direction in the historical video frames, so that the target object can be continuously tracked and predicted when the target object is occluded.

[0078] Continue to refer to Figure 1 , the target tracking algorithm also includes:

[0079] S3. Determine a second detection frame of the target object in the current video frame based on the first detection frame of the target object in the previous video frame of the current video frame and according to the second motion state information of the target object in the current video frame.

[0080] Specifically, after predicting the motion speed and scale change information of the target object in the motion direction of the current video frame, the second detection frame of the target object in the current video frame is obtained by moving and scaling the first detection frame of the previous video frame.

[0081] Figure 2 FIG. 4 shows a flow chart of generating a second detection frame according to a specific embodiment of the present application, as shown in FIG. Figure 2 As shown, step S3 is specifically as follows:

[0082] S31. Determine the moving distance in the x and y directions according to the moving speed of the target object in the x and y directions of the current video frame.

[0083] After predicting the moving speed of the target object in the x and y directions of the current video frame, the moving distances L1 and L2 of the target object in the x and y directions of the current video frame can be calculated according to the time interval between the two previous and next video frames.

[0084] S32. Determine second coordinate information of the target object in the current video frame based on the first coordinate information of the target object in the previous video frame and according to the movement distance of the target object in the x and y directions in the current video frame.

[0085] After determining the movement distances L1 and L2 of the target object in the x and y directions of the current video frame, the first coordinate information (x, y) of the target object in the previous video frame of the current video frame is moved to obtain the second coordinate information (x1, y1) of the target object in the current video frame.

[0086] S33. Move the first detection frame based on the second coordinate information.

[0087] After determining the second coordinate information (x1, y1) of the target object in the current video frame, the first detection frame of the target object in the previous video frame of the current video frame is moved to the center point with the coordinate point as the center point.

[0088] S34. According to the scale change information of the target object in the current video frame, the moved first detection frame is scaled to obtain a second detection frame.

[0089] According to the predicted scale change information of the target object in the current video frame, the moved first detection frame is scaled to obtain a second detection frame of the target object in the current video frame.

[0090] In this embodiment, the motion speed and scale change of the target object are predicted based on the data of the historical video frames using an exponentially weighted sliding average calculation method. Since the prediction is of the motion speed change and scale change of the target object in an obscured state, it has a high reliability in a short period of time. However, as the target object is obscured for a longer time, especially for the prediction of the motion speed change, the error will become larger and larger, and the accuracy of the prediction will decrease faster and faster. Therefore, the present application also proposes a polynomial trajectory fitting prediction method based on the least squares method for the coordinate information of the target object. By integrating the exponentially weighted sliding average calculation method with the polynomial trajectory fitting prediction method based on the least squares method, the prediction error is reduced and the reliability of target tracking is further improved.

[0091] Figure 3 FIG. 4 shows a flow chart of generating the third coordinate information according to a specific embodiment of the present application. Figure 3 As shown, step S33 specifically includes:

[0092] S331 , performing polynomial trajectory fitting based on the coordinate information of the target object in the historical video frame, optimizing the objective function using the least squares method, and obtaining a trajectory prediction function of the target object.

[0093] Specifically, the objective function expression is as follows:

[0094]

[0095] in, F (x) is the trajectory prediction function, N is the previous N video frames, T is the order of the high-order polynomial, a is the coefficient of the objective function, and x i ,y i It is the coordinate information of the target object in the i-th video frame of the previous N video frames in history.

[0096] S332, substitute the horizontal coordinate x1 of the second coordinate information into the trajectory prediction function, calculate the predicted vertical coordinate y2, average the vertical coordinate y1 of the second coordinate information and the predicted vertical coordinate y2 to obtain the final vertical coordinate y3, and obtain the third coordinate information of the target object in the current video frame based on the horizontal coordinate x1 of the second coordinate information and the final vertical coordinate y3.

[0097] In this embodiment, the ordinate y1 of the second coordinate information and the predicted ordinate y2 are arithmetic averaged. In other implementations, different coefficients may be used to perform weighted average calculation on the ordinate y1 of the second coordinate information and the predicted ordinate y2, which is not limited here.

[0098] S333. Move the first detection frame according to the third coordinate information.

[0099] Specifically, the least squares-based polynomial trajectory fitting prediction method predicts the center coordinate motion trajectory of the target object. Therefore, during a period of time when the target object is occluded, the prediction of the motion trajectory is more reliable, and the prediction accuracy decreases slowly, with a relatively small error. Therefore, by integrating the exponentially weighted moving average calculation method with the least squares-based polynomial trajectory fitting prediction method, the defect of large errors in predicting motion speed over a long period of time using the exponentially weighted moving average calculation method is made up, thereby improving the tracking reliability of the target object under long-term occlusion.

[0100] The target tracking method under occlusion state provided by the present application predicts the target object's motion speed and scale change information in the motion direction of the current video frame through an exponentially weighted sliding average calculation method based on the motion speed and scale change information of the target object in the motion direction in the historical video frame, thereby re-determining the detection frame position of the target object. At the same time, a polynomial fitting is performed on the coordinate information of the target object in the historical video frame, and the motion trajectory of the target object is predicted using the least squares method. The prediction result of the exponentially weighted sliding average is then fused with the prediction result of the least squares method to further determine the detection frame position of the target object, thereby reducing the prediction error and improving the prediction accuracy. The present application can continuously track the target object when the target object is occluded, and the detection frame of the target object can adaptively change with the scale change of the target object, thereby improving the tracking reliability of the target object.

[0101] Based on the above target tracking method under occlusion state, the present application also proposes a target tracking device under occlusion state. Figure 4 A framework diagram of a target tracking device under an occlusion state according to a specific embodiment of the present application is shown. Figure 4 As shown, the target tracking device comprises:

[0102] The acquisition module 100 is configured to acquire a current video frame containing a target object, wherein the target object has occlusion state and motion state information, wherein the occlusion state includes occluded and unoccluded, and the motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information.

[0103] The prediction module 200 is configured to predict the second motion state information of the target object in the current video frame based on the first motion state information of the target object in the historical video frame in response to the target object being blocked.

[0104] The tracking module 300 is configured to establish a second detection frame of the target object in the current video frame based on the first detection frame of the target object in the previous video frame of the current video frame and according to the second motion state information of the target object in the current video frame.

[0105] Based on the above-mentioned target tracking method in an occluded state, the present application also proposes an electronic device, including: one or more processors, and a memory.

[0106] Among them, the memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the electronic device implements the target tracking method under the occlusion state provided by any of the above-mentioned embodiments.

[0107] Based on the above target tracking method, the present application also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the target tracking method under an occluded state provided by any of the above embodiments is implemented.

[0108] In the embodiments of the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device / system / method embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0109] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0110] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0112] Obviously, those skilled in the art can make various modifications and changes to the embodiments of the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and changes are within the scope of the claims of the present invention and their equivalents, the present invention is also intended to cover these modifications and changes. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are recorded in mutually different dependent claims does not indicate that the combination of these measures cannot be used to profit. Any reference numerals in the claims should not be considered to limit the scope.

Claims

1. A method for tracking a target under occlusion, characterized in that: include: Acquire a current video frame containing a target object, wherein the target object has occlusion state and motion state information, wherein the occlusion state includes being occluded and being unoccluded, and the motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information; In response to the target object not being blocked, obtaining a kernel correlation filter based on kernel correlation filtering algorithm training, and predicting motion state information of the target object in the current video frame according to the kernel correlation filter; In response to the target object being blocked, predicting the second motion state information of the target object in the current video frame based on the first motion state information of the target object in the historical video frame, including: obtaining the motion speed and scale change information of the target object in the motion direction of the current video frame by using an exponentially weighted sliding average calculation based on the motion speed and scale change information of the target object in the motion direction in the historical video frame; Based on the first detection frame of the target object in the previous video frame of the current video frame, according to the second motion state information of the target object in the current video frame, or according to the second motion state information of the target object in the current video frame, determine the second detection frame of the target object in the current video frame.

2. The method according to claim 1, characterized in that The exponentially weighted moving average calculation expression is as follows: ， ， in, , are the moving speeds of the target object in the x and y directions of the current video frame, respectively; N is the previous N video frames; e is a natural constant; , are the movement speeds of the target object in the x and y directions of the i-th video frame in the previous N video frames, is the scale change information of the target object in the current video frame, It is the scale change information of the target object in the i-th video frame of the previous N video frames in history.

3. The method according to claim 2, characterized in that The determining, based on the first detection frame of the target object in a previous video frame of the current video frame, and according to the second motion state information of the target object in the current video frame, a second detection frame of the target object in the current video frame comprises: Determine the moving distance in the x and y directions according to the moving speed of the target object in the x and y directions of the current video frame; Based on the first coordinate information of the target object in a previous video frame of the current video frame, determining the second coordinate information of the target object in the current video frame according to the movement distance of the target object in the x and y directions of the current video frame; Moving the first detection frame based on the second coordinate information; According to the scale change information of the target object in the current video frame, the moved first detection frame is scaled to obtain the second detection frame.

4. The method according to claim 3, characterized in that The moving the first detection frame based on the second coordinate information includes: Based on the coordinate information of the target object in the historical video frame, a polynomial trajectory fitting is performed, and the objective function is optimized using the least squares method to obtain the trajectory prediction function of the target object. The objective function expression is as follows: in, is the trajectory prediction function, N is the previous N video frames, T is the order of the high-order polynomial, a is the coefficient of the objective function, , The coordinate information of the target object in the i-th video frame in the previous N video frames in history; Substituting the abscissa x1 of the second coordinate information into the trajectory prediction function to calculate the predicted ordinate y2, averaging the ordinate y1 of the second coordinate information and the predicted ordinate y2 to obtain a final ordinate y3, and obtaining the third coordinate information of the target object in the current video frame based on the abscissa x1 of the second coordinate information and the final ordinate y3; The first detection frame is moved according to the third coordinate information.

5. The method according to claim 1, characterized in that The steps of acquiring the coordinate information, the moving direction, and the moving speed in the moving direction of the target object include: A kernel correlation filter is obtained based on the kernel correlation filter algorithm training; According to the coordinate information of the target object in the previous video frame of the current video frame, the coordinate information of the target object in the current video frame is predicted by the kernel correlation filter; According to the change of the coordinate information of the target object in the two previous and next video frames, the moving direction and the moving speed of the target object in the current video frame are calculated.

6. The method according to claim 5, characterized in that The step of acquiring the scale change information of the target object comprises: Enlarging the first detection frame according to a first preset ratio and / or a specified direction to obtain a search frame of the target object in the current video frame; Enlarging and / or reducing the first detection frame according to a second preset ratio to obtain a multi-scale sliding frame; The multi-scale sliding boxes are sequentially traversed in the search box for sliding detection, the kernel correlation filter calculates a response value, and the scale corresponding to the sliding box with the maximum response value is used as the scale change information of the current video frame.

7. The method according to claim 6, characterized in that The step of determining the occlusion state of the target object comprises: Determine whether the maximum response value calculated by the kernel correlation filter is greater than a preset threshold; if so, determine that the occlusion state of the target object is not occluded; if not, determine that the occlusion state of the target object is occluded.

8. A target tracking device in an occluded state, characterized in that: include: an acquisition module configured to acquire a current video frame containing a target object, wherein the target object has an occlusion state and motion state information, wherein the occlusion state includes being occluded and being unoccluded, and the motion state information includes coordinate information, motion direction, motion speed in the motion direction, and scale change information; A prediction module is configured to, in response to the target object not being blocked, obtain a kernel correlation filter based on a kernel correlation filter algorithm training, and predict the motion state information of the target object in the current video frame according to the kernel correlation filter; in response to the target object being blocked, predict the second motion state information of the target object in the current video frame based on the first motion state information of the target object in the historical video frame, including: based on the motion speed and scale change information of the target object in the motion direction in the historical video frame, using an exponentially weighted sliding average calculation to obtain the motion speed and scale change information of the target object in the motion direction of the current video frame; A tracking module is configured to establish a second detection frame of the target object in the current video frame based on a first detection frame of the target object in a previous video frame of the current video frame, according to the second motion state information of the target object in the current video frame, or according to the second motion state information of the target object in the current video frame.

9. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images