A target tracking method and device
Through the image tracking method, corner point detection and sparse optical flow method are used to calculate the average displacement of the target object, reducing the tracking calculation volume, solving the problem of high CPU consumption when the number of faces is high, and improving the processing of live beauty efficiency.
Patent Information
- Application Number
- CN202110002099.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-04
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-01-04
AI Technical Summary
The existing target tracking scheme based on deep learning is large when there are many faces in the image, and the calculation volume is large and time-consuming, resulting in excessive CPU consumption during live beauty, making it difficult to meet the needs of low energy consumption.
The image tracking method is adopted to calculate the average displacement of the target object by obtaining the feature corner points set of images in the previous frame, and use corner point detection technology and sparse optical flow method to reduce the tracking calculation amount, reduce the number of target detection times, and improve tracking efficiency.
It effectively reduces tracking and computing overhead, improves image processing efficiency, and is suitable for mobile phones, especially when there are many faces, meeting the processing rate requirements of more than 25fps.
Smart Images

Figure CN113808159B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to computer technology, and in particular to a target tracking method and device. Background Art
[0002] With the rapid development of live video streaming, live streaming has gradually become an important development direction for major e-commerce companies. The beauty technology of live streaming software can make the anchor more beautiful, which is of great significance to attracting consumers to consume. Therefore, live streaming beauty technology has become an important demand of live streaming platforms. However, most anchors usually use smartphones as live streaming devices, and the live streaming time is long. This poses new challenges to the beauty algorithm, which needs to meet the needs of live streaming beauty with lower energy consumption. In the existing beauty solutions, face detection in images takes up a lot of time and CPU consumption. Therefore, tracking faces in live streaming to reduce the number of face detections has become an important demand.
[0003] Tracking based on deep learning is a common target tracking solution. This solution uses convolutional neural networks to train and learn the relevant features of the target to achieve target tracking.
[0004] In the process of implementing the present invention, the inventors found that the above target tracking solution has the problems of large amount of calculation and long time consumption. The specific reasons are analyzed as follows:
[0005] The existing target tracking scheme based on deep learning needs to input two frames of images in succession, and use the deep learning network to analyze the correlation of the input images. However, the analysis computation of the deep learning network is large, and accordingly, the overall tracking takes a long time. In addition, as the number of faces in the image increases, the number of targets to be tracked also increases. In this way, the computation and time consumption also increase linearly with the number of faces. Therefore, when there are more faces in the image, the above problem is more serious. Summary of the invention
[0006] In view of this, the main purpose of the present invention is to provide an image tracking method and device, which can reduce computational overhead and improve tracking efficiency. In order to achieve the above purpose, the technical solution proposed in the embodiment of the present invention is:
[0007] A target tracking method, comprising:
[0008] Get the image frame k that needs to be detected currently;
[0009] Determine a detection method for the image frame k according to a detection method or a detection result of a previous image frame k-1, and detect each target object in the image frame k according to the determined detection method;
[0010] Wherein, when the determined detection mode is target detection, after detecting the detection frame of the target object, detecting a set of characteristic corner points of the target object in the corresponding detection frame;
[0011] When the determined detection mode is tracking, the detection mode of the next frame obtained when detecting the image frame k-1 is used as the detection mode of the image frame k.
[0012] Preferably, when the determined detection mode is tracking, detecting each target object in the image frame k includes:
[0013] Determine a set of feature corner points of each target object in image frame k according to a set of feature corner points of the target object in image frame k-1;
[0014] For each target object n, determine the average displacement of the target object n in the image frame k-1 and the image frame k according to the feature corner point sets of the target object n respectively; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target;
[0015] If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection method of the next frame is determined to be target detection, otherwise, the detection method of the next frame is determined to be tracking.
[0016] Preferably, the detection method for determining the image frame k includes:
[0017] If the detection mode of the image frame k-1 is target detection, determining that the detection mode of the image frame k is tracking;
[0018] If the detection mode for the image frame k-1 is tracking, the detection mode for the image frame k is obtained according to the detection mode for the next frame determined when the image frame k-1 is detected.
[0019] Preferably, detecting a set of feature corner points of the target object in a corresponding detection frame includes:
[0020] A local binary pattern (LBP) operator is used as a corner point detection operator to detect a set of feature corner points of the target object in a corresponding detection frame.
[0021] Preferably, determining a set of characteristic corner points of each target object in the image frame k comprises:
[0022] Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1;
[0023] For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
[0024] Preferably, determining the average displacement of the target object n in the image frame k comprises:
[0025] According to d n =(F k n,m -F k-1 n,m ) / M, calculate the average displacement d of the target object n in the image frame k n ;
[0026] Among them, F k n,m is the coordinate of the mth feature corner point of the target object n in the image frame k, F k-1 n,m is the coordinate of the mth feature corner point of the target object n in the image frame k-1; M is the number of feature corner points of the target object n in the image frame k.
[0027] An embodiment of the present invention provides a target tracking device, comprising:
[0028] An image acquisition module is used to acquire the image frame k that needs to be detected currently;
[0029] A detection module is used to determine the detection method of the image frame k according to the detection method or detection result of the previous image frame k-1, and detect each target object in the image frame k according to the determined detection method; wherein, when the determined detection method is target detection, after detecting the detection frame of the target object, the feature corner point set of the target object in the corresponding detection frame is detected; when the determined detection method is tracking, the target object is detected according to the feature corner point set and the detection frame of the target object in the image frame k-1.
[0030] Preferably, the detection module is specifically used to detect each target object in the image frame k when the determined detection mode is tracking, including:
[0031] Determine a set of feature corner points of each target object in image frame k according to a set of feature corner points of the target object in image frame k-1;
[0032] For each target object n, determine the average displacement of the target object n in the image frame k-1 and the image frame k according to the feature corner point sets of the target object n respectively; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target;
[0033] If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection method of the next frame is determined to be target detection, otherwise, the detection method of the next frame is determined to be tracking.
[0034] Preferably, the detection module is specifically used to determine the detection method of the image frame k, including:
[0035] If the detection mode of the image frame k-1 is target detection, determining that the detection mode of the image frame k is tracking;
[0036] If the detection mode for the image frame k-1 is tracking, the detection mode of the next frame obtained when the image frame k-1 is detected is used as the detection mode of the image frame k.
[0037] Preferably, the detection module is specifically used to detect a set of characteristic corner points of the target object in a corresponding detection frame, including:
[0038] A local binary pattern (LBP) operator is used as a corner point detection operator to detect a set of characteristic corner points of the target object in a corresponding detection frame.
[0039] Preferably, the detection module is specifically used to determine a set of characteristic corner points of each target object in the image frame k, including:
[0040] Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1;
[0041] For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
[0042] Preferably, the detection module is specifically used to determine the average displacement of the target object n in the image frame k, including:
[0043] According to d n =(F k n,m -F k-1 n,m ) / M, calculate the average displacement d of the target object n in the image frame k n ;
[0044] Among them, F k n,m is the coordinate of the mth feature corner point of the target object n in the image frame k, F k-1 n,m is the coordinate of the mth feature corner point of the target object n in the image frame k-1; M is the number of feature corner points of the target object n in the image frame k.
[0045] The embodiment of the present application further discloses a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the processor executes the steps of the image tracking method as described above.
[0046] An embodiment of the present application also discloses an electronic device, including the non-volatile computer-readable storage medium as described above, and the processor capable of accessing the non-volatile computer-readable storage medium.
[0047] In summary, the image tracking method and device proposed in the present invention, by using corner detection technology, determines the feature corner point set of each target object in the image frame. When tracking is required, the feature corner point set and detection frame of the target object in the previous frame image can be used to determine the detection frame of the target object in the next frame. In this way, tracking can be used to effectively reduce the number of times target detection is used, thereby reducing the overall image processing calculation amount. In addition, since there is no need to use a deep learning network to obtain the correlation between frames, compared with the existing solutions that require the use of a deep learning network for tracking, the amount of calculation is greatly reduced, effectively reducing the tracking calculation overhead and improving the tracking efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention;
[0049] Figure 2Schematic diagram of the device structure of an embodiment of the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0051] Figure 1 Schematic diagram of the method flow of an embodiment of the present invention, as shown in Figure 1 As shown, the target tracking method implemented in this embodiment mainly includes:
[0052] Step 101: Obtain the image frame k that currently needs to be detected.
[0053] In this step, the image frame currently requiring the detection of the target object is obtained from the input image frame sequence, so as to detect the target object therein.
[0054] In practical applications, those skilled in the art may set the targets to be detected in the image according to actual needs, such as faces, animals, plants, etc.
[0055] Step 102: determine the detection method of the image frame k according to the detection method or detection result of the previous image frame k-1, and detect each target object in the image frame k according to the determined detection method; wherein, when the determined detection method is target detection, after detecting the detection frame of the target object, detect the feature corner point set of the target object in the corresponding detection frame; when the determined detection method is tracking, detect the target object according to the feature corner point set and the detection frame of the target object in the image frame k-1.
[0056] In this step, the corner detection technology is used to determine the feature corner point set of each target object in the image frame. When tracking is required, the feature corner point set of the target object in the previous frame is used to translate the detection frame of the target object in the previous frame to obtain the detection frame of the target object in the current frame. In this way, compared with the existing solution that requires the use of a deep learning network for tracking, the complexity of the tracking operation is greatly reduced and simplified, thereby effectively reducing the amount of tracking operations and improving the tracking efficiency. In this way, even if there are a large number of target objects in the image, it will not lead to high tracking operation overhead and low operating efficiency, thereby affecting the applicability of the tracking solution. Therefore, it is suitable for the mobile phone side. Especially when performing face detection, when the number of faces is large, the processing rate requirement of more than 25fps can be met.
[0057] It should be noted here that corner detection is a method used in computer vision systems to obtain image features. Corner points are usually defined as the intersection of two edges. There are many operators used for corner detection, such as Haar (a digital image feature used for object recognition), local binary pattern (LBP), Hog and other feature extractors.
[0058] In order to improve tracking efficiency and reduce tracking calculation amount so that the present invention can be better applied to mobile terminals, in one embodiment, the LBP operator can be used as a corner point detection operator to detect the feature corner point set of the target object in the corresponding detection frame.
[0059] In one implementation, in the above step 102, when the determined detection method is tracking, the following method may be specifically used to detect each target object in the image frame k:
[0060] Step a1: determine the feature corner point set of each target object in the image frame k according to the feature corner point set of the target object in the image frame k-1.
[0061] In this step, the feature corner point set of the target object in the current frame is determined based on the feature corner point set of the target object in the previous frame image.
[0062] Preferably, in order to more accurately and efficiently determine the feature corner point set of each target object in the current frame k, in one embodiment, a sparse optical flow (Lucas–Kanade, LK) method can be used to determine the feature corner point set of each target object in the image frame k:
[0063] Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1;
[0064] For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
[0065] In the above method, optical flow tracking is performed based on the collection of feature corner point sets of all target objects in image frame k-1 in image frame k-1 (i.e., the feature corner point set of image frame k-1). In this way, only one optical flow tracking is required to obtain the feature corner point set of all target objects in image frame k. Therefore, the feature corner point set of each target object in image frame k can be obtained with minimal optical flow processing overhead.
[0066] It should be noted here that the optical flow method is a method that uses the changes in pixels in the image sequence in the time domain and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames.
[0067] Commonly used optical flow methods are divided into sparse optical flow and dense optical flow. The LK optical flow method has three assumptions:
[0068] (1) Constant brightness: The brightness value (pixel grayscale value) of a pixel remains constant as time goes by. This is the basic setting of the optical flow method. All optical flow methods must meet this requirement.
[0069] (2) Small motion: Changes in time will not cause drastic changes in position. In this way, the change in grayscale value caused by the change in position between adjacent frames can be used to obtain the partial derivative of grayscale with respect to position. All optical flow methods must meet this requirement.
[0070] (3) Spatial consistency: that is, adjacent pixels in the previous frame are also adjacent in the next frame. This is a unique assumption of the LK optical flow method. In order to obtain the speed in the x and y directions, multiple equations need to be established and solved simultaneously. The spatial consistency assumption can use the n pixels in the neighborhood to establish n equations.
[0071] The image difference between frames in a video frame sequence is very small. Therefore, the assumptions of the optical flow algorithm are met. When the kth frame is used to track the feature corner points of the k-1 frame, it is easy to accurately obtain the feature point set of the kth frame.
[0072] Step a2: for each target object n, determine the average displacement of the target object n in the image frame k according to the feature corner point sets of the target object n in the image frame k-1 and the image frame k; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target.
[0073] In this step, the detection frame is tracked by calculating the average displacement of the corner features in the target object detection frame between frames and performing a corresponding translation on the detection frame.
[0074] In one embodiment, the following method may be used to determine the average displacement of the target object n in the image frame k:
[0075] According to dn =(F k n,m -F k-1 n,m ) / M, calculate the average displacement d of the target object n in the image frame k n ;
[0076] Among them, F k n,m is the coordinate of the mth feature corner point of the target object n in the image frame k, F k-1 n,m is the coordinate of the mth feature corner point of the target object n in the image frame k-1; M is the number of feature corner points of the target object n in the image frame k.
[0077] The average displacement d of the target object n in the image frame k is obtained. n After that, the detection box of the target object n in the image frame k-1 needs to be translated to obtain the detection box of the target object n in the image frame k k n ,in,
[0078] Step a3: If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection mode of the next frame is determined to be target detection; otherwise, the detection mode of the next frame is determined to be tracking.
[0079] It should be noted here that the detection box after tracking can be obtained by using the above steps a1 and a2 k n However, in actual applications, corner points or optical flow detection may drift. At this time, the direction and distance of the detection frame movement may be problematic. Therefore, through step a3, the image in the translated detection frame is classified and judged to determine whether the image in the frame is the target object (for example, whether it is a face). If so, it means that the current tracking result is accurate, and the tracking method can be continued in the next frame to determine the detection frame of the target object. Otherwise, if it is not the target object, it means that the tracking result of the current frame is inaccurate, and target detection needs to be performed again in the next frame. In order to improve the tracking efficiency and accuracy in the above step a3, as long as the detection result of a current target object is inaccurate, it is determined that all target objects in the next frame image are uniformly detected again.
[0080] In one embodiment, in order to use the tracking method as much as possible to detect the target object in the image, so as to reduce the number of target detections and thus reduce the image processing overhead, the following method may be specifically used in step 102 to determine the detection method of the image frame k:
[0081] If the detection method of image frame k-1 is target detection, then the detection method of the image frame k is determined to be tracking; if the detection method of image frame k-1 is tracking, then the detection method of the next frame obtained when detecting image frame k-1 is used as the detection method of the image frame k.
[0082] Here, when the target object is a human face, the above target detection is specifically human face detection. When it is specifically implemented, the existing human face detection solution can be adopted, which will not be described in detail here.
[0083] It can be known from the above embodiments that in the above method embodiments, by using the corner point detection technology, the feature corner point set of each target object in each frame of the image frame sequence is determined. When tracking is required, the feature corner point set of the target object in the two frames before and after is used to calculate the average displacement of the target object in the two frames before and after, and the detection frame of the target object in the previous frame is translated based on the average displacement to obtain the detection frame of the target object in the next frame, thereby completing the detection of the target object. In this way, since it is only necessary to calculate the displacement of the target object in the two frames before and after based on the feature corner point set of the two frames before and after, and to translate the detection frame of the previous frame using the displacement to achieve tracking of the target object, in this way, compared with the computational overhead of the deep learning network, the computational overhead is very small, and therefore, the tracking computational overhead can be greatly reduced, thereby greatly improving the tracking efficiency.
[0084] Corresponding to the above method embodiment, the embodiment of the present invention also provides a target tracking device, such as Figure 2 Shown include:
[0085] An image acquisition module is used to acquire the image frame k that needs to be detected currently;
[0086] A detection module is used to determine the detection method of the image frame k according to the detection method or detection result of the previous image frame k-1, and detect each target object in the image frame k according to the determined detection method; wherein, when the determined detection method is target detection, after detecting the detection frame of the target object, the feature corner point set of the target object in the corresponding detection frame is detected; when the determined detection method is tracking, the target object is detected according to the feature corner point set and the detection frame of the target object in the image frame k-1.
[0087] Preferably, the detection module is specifically used to detect each target object in the image frame k when the determined detection mode is tracking, including:
[0088] Determine a set of feature corner points of each target object in image frame k according to a set of feature corner points of the target object in image frame k-1;
[0089] For each target object n, determine the average displacement of the target object n in the image frame k-1 and the image frame k according to the feature corner point sets of the target object n respectively; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target;
[0090] If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection method of the next frame is determined to be target detection, otherwise, the detection method of the next frame is determined to be tracking.
[0091] Preferably, the detection module is specifically used to determine the detection method of the image frame k, including:
[0092] If the detection mode of the image frame k-1 is target detection, determining that the detection mode of the image frame k is tracking;
[0093] If the detection mode for the image frame k-1 is tracking, the detection mode of the next frame obtained when the image frame k-1 is detected is used as the detection mode of the image frame k.
[0094] Preferably, the detection module is specifically used to detect a set of characteristic corner points of the target object in a corresponding detection frame, including:
[0095] The LBP operator is used as a corner point detection operator to detect a set of feature corner points of the target object in a corresponding detection frame.
[0096] Preferably, the detection module is specifically used to determine a set of characteristic corner points of each target object in the image frame k, including:
[0097] Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1;
[0098] For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
[0099] Preferably, the detection module is specifically used to determine the average displacement of the target object n in the image frame k, including:
[0100] According to d n =(F k n,m -F k-1 n,m ) / M, calculate the average displacement d of the target object n in the image frame k n ;
[0101] Among them, F k n,m is the coordinate of the mth feature corner point of the target object n in the image frame k, F k-1 n,m is the coordinate of the mth feature corner point of the target object n in the image frame k-1; M is the number of feature corner points of the target object n in the image frame k.
[0102] An embodiment of the present invention further discloses a target tracking device, which includes: a processor and a memory; wherein the memory stores an application program executable by the processor, which is used to enable the processor to execute any of the target tracking methods described above.
[0103] The memory may be implemented as various storage media such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), and a programmable program read-only memory (PROM). The processor may be implemented as including one or more central processing units or one or more field programmable gate arrays, wherein the field programmable gate array integrates one or more central processing unit cores. Specifically, the central processing unit or the central processing unit core may be implemented as a CPU or an MCU.
[0104] It should be noted that not all steps and modules in the above processes and structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The division of each module is only for the convenience of describing the functional division adopted. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can also be implemented by the same module. These modules can be located in the same device or in different devices.
[0105] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (such as a dedicated processor, such as an FPGA or ASIC) for performing a specific operation. The hardware module may also include a programmable logic device or circuit (such as a general-purpose processor or other programmable processor) temporarily configured by software to perform a specific operation. As for whether to implement the hardware module mechanically, or using a dedicated permanent circuit, or using a temporarily configured circuit (such as configured by software), it can be decided based on cost and time considerations.
[0106] The embodiment of the present invention also provides a machine-readable storage medium, storing instructions for making a machine perform a method as described in the present application. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code for realizing the function of any of the embodiments described in the above embodiments is stored, and the computer (or CPU or MPU) of the system or device reads out and executes the program code stored in the storage medium. In addition, the operating system etc. operated on the computer can also be completed part or all of the actual operation by instructions based on the program code. The program code read out from the storage medium can also be written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then the CPU etc. installed on the expansion board or expansion unit are made to perform part and all of the actual operation based on the instructions of the program code, thereby realizing the function of any of the embodiments described in the above embodiments.
[0107] Storage media implementations for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, program code can be downloaded from a server computer or cloud via a communication network.
[0108] In this article, "schematic" means "serving as an example, instance or explanation", and any diagram or implementation method described as "schematic" in this article should not be interpreted as a more preferred or more advantageous technical solution. In order to make the drawings concise, only the parts related to the present invention are schematically shown in each figure, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and easy to understand, in some figures, only one of the parts with the same structure or function is schematically drawn, or only one of them is marked. In this article, "one" does not mean that the number of the relevant parts of the present invention is limited to "only one", and "one" does not mean that the number of the relevant parts of the present invention is "more than one". In this article, "upper", "lower", "front", "back", "left", "right", "inside", "outside", etc. are only used to indicate the relative position relationship between the relevant parts, rather than to limit the absolute position of these relevant parts.
[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A target tracking method, characterized in that: include: Get the image frame k that needs to be detected currently; Determine a detection method for the image frame k according to a detection method or a detection result of a previous image frame k-1, and detect each target object in the image frame k according to the determined detection method; Wherein, when the determined detection mode is target detection, after detecting the detection frame of the target object, detecting a set of characteristic corner points of the target object in the corresponding detection frame; When the determined detection mode is tracking, the target object is detected according to the feature corner point set and the detection frame of the target object in the image frame k-1; When the determined detection mode is tracking, detecting each target object in the image frame k includes: Determine a set of feature corner points of each target object in image frame k according to a set of feature corner points of the target object in image frame k-1; For each target object n, determine the average displacement of the target object n in the image frame k-1 and the image frame k according to the feature corner point sets of the target object n respectively; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target; If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection method of the next frame is determined to be target detection, otherwise, the detection method of the next frame is determined to be tracking.
2. The method according to claim 1, characterized in that The detection method for determining the image frame k includes: If the detection mode of the image frame k-1 is target detection, determining that the detection mode of the image frame k is tracking; If the detection mode for the image frame k-1 is tracking, the detection mode of the next frame obtained when the image frame k-1 is detected is used as the detection mode of the image frame k.
3. The method according to claim 1, characterized in that The detecting of the set of characteristic corner points of the target object in the corresponding detection frame comprises: A local binary pattern (LBP) operator is used as a corner point detection operator to detect a set of characteristic corner points of the target object in a corresponding detection frame.
4. The method according to claim 1, characterized in that: Determining a set of feature corner points of each target object in the image frame k comprises: Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1; For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
5. The method according to claim 1, characterized in that: Determining the average displacement of the target object n in the image frame k comprises: According to d n =(F k n,m -F k-1 n,m ) / M, calculate the average displacement d of the target object n in the image frame k n ; Among them, F k n,m is the coordinate of the mth feature corner point of the target object n in the image frame k, F k-1 n,m is the coordinate of the mth feature corner point of the target object n in the image frame k-1; M is the number of feature corner points of the target object n in the image frame k.
6. A target tracking device, characterized in that: include: An image acquisition module is used to acquire the image frame k that needs to be detected currently; A detection module, configured to determine a detection method for the image frame k according to a detection method or a detection result of a previous image frame k-1, and detect each target object in the image frame k according to the determined detection method; wherein, when the determined detection method is target detection, after detecting a detection frame of the target object, a feature corner point set of the target object in the corresponding detection frame is detected; when the determined detection method is tracking, the target object is detected according to the feature corner point set and the detection frame of the target object in the image frame k-1; The detection module is specifically used to detect each target object in the image frame k when the determined detection mode is tracking, including: Determine a set of feature corner points of each target object in image frame k according to a set of feature corner points of the target object in image frame k-1; For each target object n, determine the average displacement of the target object n in the image frame k-1 and the image frame k according to the feature corner point sets of the target object n respectively; and detect the detection box box of the target object n in the image frame k-1 according to the average displacement. k-1 n Translate to obtain the detection box of the target object n in the image frame k k n , and determine the detection box k n Whether the image in belongs to the detection target; If the image of at least one target object in the detection frame of the image frame k does not belong to the detection target, the detection method of the next frame is determined to be target detection, otherwise, the detection method of the next frame is determined to be tracking.
7. The device according to claim 6, characterized in that The detection module is specifically used to determine the detection method of the image frame k, including: If the detection mode of the image frame k-1 is target detection, determining that the detection mode of the image frame k is tracking; If the detection mode for the image frame k-1 is tracking, the detection mode of the next frame obtained when the image frame k-1 is detected is used as the detection mode of the image frame k.
8. The device according to claim 6, characterized in that The detection module is specifically used to determine a set of characteristic corner points of each target object in the image frame k, including: Using a sparse optical flow method, optical flow tracking is performed on the feature corner points in the feature corner point set of the image frame k-1 to obtain the feature corner point set of the image frame k; the feature corner point set of the image frame k-1 is the collection of the feature corner point sets of all the target objects in the image frame k-1; For each of the target objects, the feature corner points corresponding to the target object are acquired from the feature corner point set of the image frame k, so as to obtain the feature corner point set of the target object in the image frame k.
9. A non-volatile computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by a processor, the processor is caused to perform the steps of the target tracking method according to any one of claims 1 to 5.
10. An electronic device, characterized in that: The invention comprises the non-volatile computer-readable storage medium of claim 9, and the processor capable of accessing the non-volatile computer-readable storage medium.
