Target tracking method, system and terminal based on image processing

By using HOG feature templates and grayscale image templates for feature representation, combined with template library matching weighted and memory mode position recursive prediction, the problem of target loss caused by occlusion and scale changes in target tracking is solved, and high-precision target tracking is achieved.

CN118918141APending Publication Date: 2024-11-08XIAN TIANYUAN ELECTRO-OPTICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410948979.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing target tracking algorithms are prone to target loss when dealing with occlusion and changes in target scale, resulting in poor overall tracking performance.

Method used

An image processing-based target tracking method is adopted, which uses HOG feature templates and grayscale image templates for feature representation. It combines template library matching weighting mechanism and position recursion prediction under memory mode to solve the target loss problem caused by target deformation, occlusion and scale change.

Benefits of technology

It achieves high-precision tracking of targets, effectively avoids target loss during radial movement, solves the target loss problem caused by occlusion and rapid deformation, and improves the overall tracking effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918141A_ABST
    Figure CN118918141A_ABST
Patent Text Reader

Abstract

The invention discloses a target tracking method, system and terminal based on image processing, and relates to the technical field of target tracking. The target tracking method is based on a known target initial position and a target size, and comprises the following steps: carrying out feature representation on an initial frame target area image; acquiring a next frame of image grey-scale map, configuring a search box consistent with a target in size, and traversing a search area along a search path; judging whether the search point position coordinate corresponding to the maximum value of the HOG feature matching coefficient is equal to the search point position coordinate corresponding to the maximum value of the gray feature matching coefficient, if so, calculating a final matching coefficient, otherwise, assigning the maximum value of the two to the final matching coefficient; and configuring a matching coefficient threshold, and determining whether to output a predicted position coordinate based on a comparison result of the final matching coefficient and the matching coefficient threshold. According to the method, the problem of target loss caused by target deformation, shielding and target scale change can be solved, and the overall tracking effect is relatively good.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of target tracking, and in particular relates to a target tracking method, system and terminal based on image processing. Background Art

[0002] With the development of science and technology, the field of computer vision has developed rapidly. As a very challenging research direction in the field of computer vision, target tracking algorithms have become a research hotspot. Target tracking algorithms are widely used in life, mainly in security, real-time analysis of competition videos, drone tracking, human-computer interaction and many other fields.

[0003] The target tracking algorithms provided by the existing technology mainly include optical flow method, Kalman, Meanshift, correlation filtering and deep learning algorithm, etc. In practical applications, the factors affecting the tracking effect mainly include changes in ambient lighting, target deformation, target occlusion and target scale changes.

[0004] The optical flow algorithm can accurately detect and identify the position of moving targets to achieve target tracking without relying on any scene information. However, its tracking effect is poor when the light changes or the target moves too fast.

[0005] Although the Meanshift algorithm can achieve better target tracking in multiple scenarios, it cannot perform adaptive tracking when the size of the tracked target changes.

[0006] The correlation filter algorithm uses discrete Fourier transform to accelerate the calculation of the filter and achieve faster operation speed. This makes the correlation filter algorithm widely used in real-time target tracking and high-definition video target tracking scenarios. However, it is sensitive to occlusion. When the tracked target is severely occluded, the correlation filter algorithm has a probability of losing the target.

[0007] The current mainstream complex target tracking algorithms have high tracking accuracy, but the algorithm complexity is also high, the real-time performance is poor, and the processing speed is insufficient in high-definition real-time video stream tracking scenarios. Although simple target tracking algorithms meet the application scenarios in terms of real-time performance, they can easily cause the target to be lost when dealing with occlusion and changes in target scale, resulting in poor overall tracking results. Summary of the invention

[0008] The object of the present invention is to provide a target tracking method, system and terminal based on image processing, which can solve the problem of target loss caused by target deformation, occlusion and target scale change.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] In a first aspect, the present invention provides a target tracking method based on image processing, having multiple search cycles, wherein the target initial position and the target size are known for each search cycle, and the target tracking method comprises the following steps in each search cycle:

[0011] S10. Applying HOG feature template and grayscale image template to feature represent the target area image of the initial frame;

[0012] S11. Obtain the grayscale image of the next frame, determine the search area with the target initial position as the center and in combination with the empirical value; configure a search box with the same size as the target; determine the starting point of the search box in the search area and the search path composed of multiple search points, and traverse the search area along the search path from the starting point;

[0013] S12. Calculate the search box HOG value and the search box grayscale feature at the starting point and each search point;

[0014] S13. For the starting point and each search point, determine the HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, form a HOG feature matching coefficient matrix by multiple HOG feature matching coefficients, and determine the HOG feature matching coefficient maximum value and its corresponding search point position coordinates based on the HOG feature matching coefficient matrix;

[0015] S14. For the starting point and each search point, determine the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template, form a grayscale feature matching coefficient matrix from multiple grayscale feature matching coefficients, and determine the maximum grayscale feature matching coefficient and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix;

[0016] S15. Determine whether the position coordinates of the two search points are equal. If they are equal, calculate the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient; otherwise, assign the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient to the final matching coefficient, and record the corresponding position coordinates as the predicted position coordinates;

[0017] S16. Configure a matching coefficient threshold, and determine whether to output the predicted position coordinates based on the comparison result between the final matching coefficient and the matching coefficient threshold.

[0018] As a possible implementation method, S10 specifically includes the following steps:

[0019] S100. Obtaining an initial frame target area image;

[0020] S101. Grayscale processing of the initial frame target area image to obtain the initial frame target area grayscale image;

[0021] S102. spatially downsample the grayscale image of the target region of the initial frame to obtain a multi-scale grayscale image of the target region of the initial frame;

[0022] S103. Perform HOG feature extraction on the multi-scale grayscale image of the target area of ​​the initial frame to obtain a HOG feature template;

[0023] S104. The multi-scale grayscale image of the target area of ​​the initial frame is used as a grayscale image template, and the HOG feature template is stored in the template library to complete the feature representation of the target area image of the initial frame.

[0024] As a possible implementation method, the final matching coefficient of the HOG feature matching coefficient maximum and the grayscale feature matching coefficient maximum is calculated by the following method:

[0025] α max =AC max +BP max

[0026] Among them, α max is the final matching coefficient, C max is the maximum value of HOG feature matching coefficient, P max is the maximum value of the grayscale feature matching coefficient; A and B are weighted coefficients, A+B=1.

[0027] As a possible implementation method, S16 specifically includes: defining the matching coefficient threshold as α t , when α max >α t When , the predicted position coordinates are output and stored in the predicted point library.

[0028] As a possible implementation method, S16 specifically includes: defining the matching coefficient threshold as α t , when 0.8*α t ≤α max ≤α t , output the predicted position coordinates and store them in the template library.

[0029] As a possible implementation method, define the matching coefficient threshold as α t , when α max <0.8*α t , it enters the memory mode and uses the position recursion method to determine the predicted position coordinates.

[0030] As a possible implementation method, after entering the memory mode, perform the following steps:

[0031] Determine the number of prediction points in the prediction point library. If the number of points is greater than or equal to the point threshold, it is judged to meet the prediction conditions. At this time, perform position recursive prediction and calculate the optimal matching position in the search area. If the number of points is less than the point threshold, it is judged to not meet the prediction conditions. At this time, output the optimal matching position at the previous moment.

[0032] As a possible implementation method, the specific method of position recursive prediction is:

[0033] x n+1 =Cx n +D

[0034] y n+1 =Cy n +D

[0035] Among them, (x n ,y n ) is the target position at the previous moment, (x n+1 ,y n+1 ) is the predicted target position at the next moment; C and D are the coefficients of the linear prediction equation.

[0036] In a second aspect, the present invention provides a target tracking system based on image processing, comprising:

[0037] A feature representation unit, used for applying a HOG feature template and a grayscale image template to perform feature representation on the target area image of the initial frame;

[0038] A search unit is used to obtain a grayscale image of the next frame, determine a search area with the target initial position as the center and in combination with empirical values; configure a search box that is consistent with the size of the target; determine a starting point of the search box in the search area and a search path composed of multiple search points, and traverse the search area along the search path starting from the starting point;

[0039] A calculation unit, used to calculate the HOG value and grayscale feature of the search box at the starting point and each search point;

[0040] A HOG feature matching coefficient determination unit determines, for the starting point and each search point, a HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, a HOG feature matching coefficient matrix is ​​formed by multiple HOG feature matching coefficients, and a HOG feature matching coefficient maximum value and its corresponding search point position coordinates are determined based on the HOG feature matching coefficient matrix;

[0041] A grayscale feature matching coefficient determination unit determines the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template for the starting point and each search point, forms a grayscale feature matching coefficient matrix from a plurality of grayscale feature matching coefficients, and determines the grayscale feature matching coefficient maximum value and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix;

[0042] The predicted position coordinate determination unit determines whether the position coordinates of the two search points are equal. If they are equal, the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is calculated; otherwise, the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is assigned to the final matching coefficient, and the corresponding position coordinate is recorded as the predicted position coordinate;

[0043] The predicted position coordinate output unit is configured with a matching coefficient threshold, and determines whether to output the predicted position coordinate based on a comparison result between the final matching coefficient and the matching coefficient threshold.

[0044] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the image processing-based target tracking method provided in the first aspect.

[0045] Compared with the prior art, the present invention has the following effects:

[0046] 1. The target tracking method based on image processing proposed in the present invention adopts a template library matching weighted mechanism, which can track the target more accurately and effectively avoid the problem of target disappearance when the target moves radially.

[0047] 2. The target tracking method based on image processing proposed in the present invention adopts a template library matching weighting mechanism and a template updating mechanism, which can well solve the problems of target deformation and scale change, and can achieve high-precision tracking of the target.

[0048] 3. The target tracking method based on image processing proposed in the present invention adopts position recursive prediction under memory mode, which can effectively solve the problem of target loss caused by occlusion or rapid deformation of the target, and the overall tracking effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0050] Figure 1 A flow chart of a target tracking method based on image processing provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a square search area and a search box starting point area provided in an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of a square search area and a search path provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0054] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0056] Existing target tracking algorithms can easily cause the problem of target loss when dealing with occlusion and changes in target scale, resulting in poor overall tracking effect. The present invention provides a target tracking method, system and terminal based on image processing, which can solve the problem of target loss caused by target deformation, occlusion and changes in target scale.

[0057] In a first aspect, the present invention provides a target tracking method based on image processing, having multiple search cycles, wherein the target initial position and target size are known for each search cycle. For example, the target initial position (x, y) and target size ws are known, see Figure 1 ,In each search cycle, the target tracking method includes the following steps:

[0058] S10. Applying HOG feature template and grayscale image template to feature represent the target area image of the initial frame;

[0059] As a possible implementation method, S10 specifically includes the following steps:

[0060] S100. Obtaining an initial frame target area image;

[0061] S101. Grayscale processing of the initial frame target area image to obtain the initial frame target area grayscale image;

[0062] As an example, the target area image of the initial frame is a square or rectangular area image that just includes the target when the target is at the initial position. The grayscale processing of the target area image of the initial frame is as follows:

[0063] Gray=0.3*R+0.59*G+0.11*B (1)

[0064] Among them, Gray is a grayscale image, R, G, and B are the red, green, and blue three-channel images of the target area image of the original initial frame respectively;

[0065] Grayscale processing is used to filter out irrelevant colors and obtain the grayscale image of the target area in the initial frame.

[0066] S102. spatially downsample the grayscale image of the target region of the initial frame to obtain a multi-scale grayscale image of the target region of the initial frame;

[0067] As an example, the spatial downsampling algorithm can be any one of the existing technologies, such as Gaussian pyramid algorithm, BalanceCascade algorithm, Probabilistic Roadmap (PRM) algorithm, etc., which are not specifically limited here. Since the target is constantly changing, the image size becomes smaller when it is far away, and the image size becomes larger when it is close. The initial frame target area grayscale image is spatially downsampled to obtain a multi-scale grayscale image of the initial frame target area. Exemplarily, assuming that the size of the initial frame target area grayscale image is 50*50, when the target is far away, a multi-scale grayscale image of the initial frame target area with a size of 51*51, 52*52, 53*53... can be obtained; when the target is close, a multi-scale grayscale image of the initial frame target area with a size of 49*49, 48*48, 47*47... can be obtained, which can effectively avoid the problem of target disappearance when the target moves radially.

[0068] S103. Perform HOG feature extraction on the multi-scale grayscale image of the target area of ​​the initial frame to obtain a HOG feature template;

[0069] As an example, the following algorithm is used to extract HOG features from the multi-scale grayscale image of the target area in the initial frame:

[0070]

[0071] Where I(x+1, y) is the grayscale value at the position (x+1, y) in the grayscale image; I(x, y-1) is the grayscale value at the position (x, y-1) in the grayscale image; I(x, y+1) is the grayscale value at the position (x, y+1) in the grayscale image; I(x, y-1) is the grayscale value at the position (x, y-1) in the grayscale image; G x (x, y) is the horizontal gradient of the position (x, y); G y (x, y) is the vertical gradient at the position (x, y); G(x, y) is the directional gradient amplitude at the position (x, y); θ (x,y) is the gradient direction at the (x, y) position.

[0072] S104. The multi-scale grayscale image of the target area of ​​the initial frame is used as a grayscale image template, and the HOG feature template is stored in the template library to complete the feature representation of the target area image of the initial frame.

[0073] S11. Obtain the grayscale image of the next frame, determine the search area with the target initial position as the center and in combination with the empirical value; configure a search box with the same size as the target; determine the starting point of the search box in the search area and the search path composed of multiple search points, and traverse the search area along the search path from the starting point;

[0074] As an example, with the target initial position (x, y) as the center and W as the search radius, a circular search area is determined for target search, or with the target initial position (x, y) as the center, a square search area is determined for target search with a length of W from the center in the four directions of top, bottom, left and right. Configure a search box of size ws, determine the starting point of the search box in the search area, the starting point is the initial position to start the search, and can be located in the upper left corner, upper right corner, lower left corner, lower right corner or other positions of the search area; after determining the starting point, move the search box to obtain multiple search points, and the starting point and multiple search points constitute a search path. Exemplarily, the search path can be various, such as horizontal first and then vertical, vertical first and then horizontal, or other paths, until the entire search area is traversed.

[0075] S12. Calculate the HOG value of the search box at the starting point and each search point using formula (2); calculate the grayscale feature of the search box at the starting point and each search point using formula (1);

[0076] S13. For the starting point and each search point, determine the HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, form a HOG feature matching coefficient matrix by multiple HOG feature matching coefficients, and determine the HOG feature matching coefficient maximum value and its corresponding search point position coordinates based on the HOG feature matching coefficient matrix;

[0077] As an example of determining the HOG feature matching coefficient between the search box HOG value and the HOG feature template, first, the distance between the search box HOG value and the HOG feature template is calculated according to the following formula:

[0078]

[0079] Among them, x i is the HOG feature vector, y i is the HOG feature vector in the template library, n is the number of feature vectors, and θ is the angle of the HOG feature vector;

[0080] By moving the search box to form a new search box, the distance between the HOG value of the new search box and the HOG feature template is calculated using formula (3), and the HOG feature matching coefficients between the HOG values ​​of all search boxes and the HOG feature templates in the search area are obtained. Multiple HOG feature matching coefficients constitute the HOG feature matching coefficient matrix cos(θ): (x,y) , based on this matrix, calculate the maximum value C of the HOG feature matching coefficient max and its corresponding search point position coordinates (cx max ,cy max ).

[0081] S14. For the starting point and each search point, determine the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template, form a grayscale feature matching coefficient matrix from multiple grayscale feature matching coefficients, and determine the maximum grayscale feature matching coefficient and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix;

[0082] As an example of determining the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template, the grayscale feature matching degree between the grayscale feature of the search box and the grayscale image template is calculated according to the following formula:

[0083]

[0084] In the formula,

[0085]

[0086] Where: R(x, y) is the distance between the grayscale image template and the grayscale feature of the search box at the position (x, y), T(x ′ ,y ′ ) is the grayscale image template in the template library, I(x+x ′ ,y+y ′ ) is the grayscale feature of the search box at the position (x, y). W is the width of the grayscale template in the template library, and h is the height of the grayscale template in the template library. For T(x ′ ,y ′ ) to sum up all the gray values, For I(x+x ′ ,y+y ′ ) to sum up all gray values. ′ (x ′ ,y ′ ) is the grayscale image template T(x ′ ,y ′ ) The normalized value, I ′ (x+x ′ ,y+y ′ ) is the grayscale feature of the search box I(x+x ′ ,y+y ′ )The normalized value;

[0087] By moving the search box to form a new search box, the grayscale feature matching degree between the grayscale feature of the new search box and the grayscale image template is calculated using formula (4), and the grayscale feature matching coefficients between the grayscale features of all search boxes and the grayscale image template in the search area are obtained. Multiple grayscale feature matching coefficients constitute a grayscale feature matching coefficient matrix R(x, y), and the grayscale feature matching coefficient maximum value P is calculated based on the matrix max and its corresponding search point position coordinates (pxmax ,py max ).

[0088] S15. Determine whether the position coordinates of the two search points are equal. If they are equal, calculate the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient; otherwise, assign the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient to the final matching coefficient, and record the corresponding position coordinates as the predicted position coordinates;

[0089] As a possible implementation, the search point position coordinates (cx max ,cy max ) and (px max ,py max ) are equal, the final matching coefficient of the maximum HOG feature matching coefficient and the maximum grayscale feature matching coefficient is calculated by the following method:

[0090] α max =AC max +BP max (5)

[0091] Among them, α max is the final matching coefficient, C max is the maximum value of HOG feature matching coefficient, P max is the maximum value of the grayscale feature matching coefficient; A and B are weighted coefficients, A+B=1.

[0092] For example, the search point position coordinates (cx max ,cy max ) and (px max ,py max ) are not equal, the maximum value of the HOG feature matching coefficient maximum value and the grayscale feature matching coefficient maximum value is assigned to the final matching coefficient; that is, if C max >P max , then the final matching coefficient α max =C max If P max >C max , then the final matching coefficient α max =P max .

[0093] The present invention adopts a template library matching weighted mechanism, which can track the target more accurately and effectively avoid the problem of target disappearance when the target moves radially.

[0094] S16. Configure a matching coefficient threshold, and determine whether to output the predicted position coordinates based on the comparison result between the final matching coefficient and the matching coefficient threshold.

[0095] As a possible implementation method, S16 specifically includes: defining the matching coefficient threshold as α t , when α max >α t When , the predicted position coordinates are output and stored in the predicted point library.

[0096] As a possible implementation method, S16 specifically includes: defining the matching coefficient threshold as α t , when 0.8*α t ≤α max ≤α t , output the predicted position coordinates and store them in the template library.

[0097] As an example of step S16, the matching coefficient threshold is determined based on the empirical value as α t , compare the final matching coefficient α max And the matching coefficient threshold α t The size of α max >α t When , the final matching coefficient α is output max The corresponding position coordinates are the predicted position coordinates, and the predicted position coordinates are stored in the prediction point library.

[0098] When 0.8*α t ≤α max ≤α t When , the final matching coefficient α is output max The corresponding position coordinates are the predicted position coordinates, and the predicted position coordinates are stored in the template library; it needs to be further explained that 0.8 in the above judgment formula is a better value determined based on experience, and it does not mean that only 0.8 can be selected.

[0099] The present invention adopts a template library matching weighting mechanism and a template updating mechanism, which can well solve the problems of target deformation and scale change, and can achieve high-precision tracking of the target.

[0100] As a possible implementation method, define the matching coefficient threshold as α t , when α max <0.8*α t , it enters the memory mode and uses the position recursion method to determine the predicted position coordinates.

[0101] As an example, the matching coefficient threshold is determined to be α based on the empirical value. t , compare the final matching coefficient α max and 0.8*α t The size of α needs further explanation. The 0.8 in the above judgment formula is a better value determined based on experience, which does not mean that only 0.8 can be selected. max <0.8*αt , enters the memory mode and applies the position recursion method to determine the predicted position coordinates.

[0102] As a possible implementation method, after entering the memory mode, perform the following steps:

[0103] Determine the number of prediction points in the prediction point library. If the number of points is greater than or equal to the point threshold, it is judged to meet the prediction conditions. At this time, perform position recursive prediction and calculate the optimal matching position in the search area. If the number of points is less than the point threshold, it is judged to not meet the prediction conditions. At this time, output the optimal matching position at the previous moment.

[0104] As an example after entering the memory mode, the point threshold is determined to be N based on experience. When the number of predicted points in the prediction point library is greater than or equal to N, the prediction condition is met. At this time, position recursive prediction is performed to calculate the optimal matching position in the search area.

[0105] As a possible implementation method, the specific method of position recursive prediction is:

[0106] x n+1 =Cx n +D

[0107] y n+1 =Cy n +D

[0108] Among them, (x n ,y n ) is the target position at the previous moment, (x n+1 ,y n+1 ) is the predicted target position at the next moment; C and D are the coefficients of the linear prediction equation.

[0109] For example, when the number of prediction points in the prediction point library is less than N, it is judged that the prediction condition is not met. At this time, the best matching position at the previous moment is output, that is, (x n ,y n ).

[0110] The present invention adopts position recursive prediction in memory mode, which can well solve the problem of target loss caused by occlusion or rapid deformation of the target, and the overall tracking effect is better.

[0111] To facilitate understanding of the technical solution of the present application, further description is given below in conjunction with specific embodiments.

[0112] A target tracking method using the present invention, knowing that the initial position of the target is (x0, y0), the target size is 50*50, and performing the following steps to track the target:

[0113] S20. The camera captures the initial frame image I1, with an image resolution of 1920*1080, and the target area image is recorded as T1. The target area image T1 is grayed to obtain the target area grayscale image T11. The target area grayscale image T11 is spatially downsampled twice to obtain the initial frame target area multi-scale grayscale images T12 and T13. HOG feature extraction is performed on the grayscale images T11, T12 and T13 respectively to obtain the HOG feature template, and store it in the template library; grayscale images T11, T12 and T13 are grayed to obtain the grayscale image template, and store it in the template library.

[0114] S21. The camera continues to capture the next frame of image I2, and grayscales the image I2 to obtain a grayscale image I21 of the image I2. Figure 2 , with the target initial position (x0, y0) as the center, the search area is a square formed by the top, bottom, left and right with a length of 200 from the center.

[0115] S22. See Figure 2 , configure a search box of size 50*50, select the first area with a width and height of 50*50 starting from the upper left corner of the search area as the starting point area, extract the HOG value and grayscale features of the starting point area, calculate the distance between the HOG value of the starting point area and the HOG feature template, and the grayscale feature matching degree between the grayscale features of the starting point area and the grayscale image template.

[0116] S23. Move the search box to the first search point area, extract the HOG value and grayscale features of the first search point area, calculate the distance between the HOG value of the first search point area and the HOG feature template, and the grayscale feature matching degree between the grayscale features of the first search point area and the grayscale image template; see Figure 3 , continue to move the search box along the search path, repeat the above steps until the distance between the HOG value and the HOG feature template of the starting point and all search point areas in the search area and the grayscale feature matching degree between the grayscale feature and the grayscale image template are calculated, and the HOG feature matching coefficients between the HOG values ​​of all search boxes and the HOG feature templates in the search area and the grayscale feature matching coefficients between the grayscale features of all search boxes and the grayscale image template are obtained. Multiple HOG feature matching coefficients constitute a HOG feature matching coefficient matrix, and multiple grayscale feature matching coefficients constitute a grayscale feature matching coefficient matrix.

[0117] S24. Count the maximum values ​​in each HOG feature matching coefficient matrix and grayscale feature matching coefficient matrix and their corresponding search point position coordinates respectively.

[0118] In this embodiment, the search point position coordinates in the two matrices are not equal, and the maximum value of the HOG feature matching coefficient maximum value and the grayscale feature matching coefficient maximum value is assigned to the final matching coefficient to obtain the final matching coefficient P and its corresponding search point position coordinates (x1, y1).

[0119] S25. Configure the matching coefficient threshold to 0.95, and determine whether the final matching coefficient P is greater than 0.95. If so, output the predicted position coordinates (x1, y1) and add the predicted position coordinates to the prediction point library; if less than, continue to determine whether P is greater than 0.95*0.8=0.76. If so, output the predicted position coordinates (x1, y1) and add the predicted position coordinates to the template library; if P is less than 0.76, enter the memory mode.

[0120] In memory mode, the configuration point threshold is 50, and it is determined whether the number of predicted points in the prediction point library is greater than or equal to 50. If so, position recursion is used to predict the position of the next point; if the number of predicted points is less than 50, the target position (x0, y0) at the previous moment is output.

[0121] S26. Repeat steps S20 to S25 until the entire search area is traversed, output the final target position, and complete target tracking.

[0122] In a second aspect, the present invention provides a target tracking system based on image processing, comprising:

[0123] A feature representation unit, used for applying a HOG feature template and a grayscale image template to perform feature representation on the target area image of the initial frame;

[0124] A search unit is used to obtain a grayscale image of the next frame, determine a search area with the target initial position as the center and in combination with empirical values; configure a search box that is consistent with the size of the target; determine a starting point of the search box in the search area and a search path composed of multiple search points, and traverse the search area along the search path starting from the starting point;

[0125] A calculation unit, used to calculate the HOG value and grayscale feature of the search box at the starting point and each search point;

[0126] A HOG feature matching coefficient determination unit determines, for the starting point and each search point, a HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, a HOG feature matching coefficient matrix is ​​formed by multiple HOG feature matching coefficients, and a HOG feature matching coefficient maximum value and its corresponding search point position coordinates are determined based on the HOG feature matching coefficient matrix;

[0127] A grayscale feature matching coefficient determination unit determines the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template for the starting point and each search point, forms a grayscale feature matching coefficient matrix from a plurality of grayscale feature matching coefficients, and determines the grayscale feature matching coefficient maximum value and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix;

[0128] The predicted position coordinate determination unit determines whether the position coordinates of the two search points are equal. If they are equal, the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is calculated; otherwise, the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is assigned to the final matching coefficient, and the corresponding position coordinate is recorded as the predicted position coordinate;

[0129] The predicted position coordinate output unit is configured with a matching coefficient threshold, and determines whether to output the predicted position coordinate based on a comparison result between the final matching coefficient and the matching coefficient threshold.

[0130] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the image processing-based target tracking method provided in the first aspect.

[0131] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0132] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.

Claims

1. A target tracking method based on image processing, having multiple search cycles, wherein the target initial position and target size are known for each search cycle, characterized in that: In each search cycle, the target tracking method includes the following steps: S10. Applying HOG feature template and grayscale image template to feature represent the target area image of the initial frame; S11. Obtain the grayscale image of the next frame, determine the search area with the target initial position as the center and in combination with the empirical value; configure a search box that is consistent with the size of the target; determine the starting point of the search box in the search area and a search path composed of multiple search points, and traverse the search area along the search path starting from the starting point; S12. Calculate the search box HOG value and the search box grayscale feature at the starting point and each search point; S13. For the starting point and each search point, determine the HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, form a HOG feature matching coefficient matrix by multiple HOG feature matching coefficients, and determine the HOG feature matching coefficient maximum value and its corresponding search point position coordinates based on the HOG feature matching coefficient matrix; S14. For the starting point and each search point, determine the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template, form a grayscale feature matching coefficient matrix from multiple grayscale feature matching coefficients, and determine the maximum grayscale feature matching coefficient and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix; S15. Determine whether the position coordinates of the two search points are equal. If they are equal, calculate the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient; otherwise, assign the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient to the final matching coefficient, and record the corresponding position coordinates as the predicted position coordinates; S16. Configure a matching coefficient threshold, and determine whether to output the predicted position coordinates based on the comparison result between the final matching coefficient and the matching coefficient threshold.

2. According to the target tracking method based on image processing in claim 1, the step S10 specifically comprises the following steps: S100. Obtaining an initial frame target area image; S101. Grayscale processing of the initial frame target area image to obtain the initial frame target area grayscale image; S102. spatially downsample the grayscale image of the target region of the initial frame to obtain a multi-scale grayscale image of the target region of the initial frame; S103. Perform HOG feature extraction on the multi-scale grayscale image of the target area of ​​the initial frame to obtain a HOG feature template; S104. The multi-scale grayscale image of the target area of ​​the initial frame is used as a grayscale image template, and the HOG feature template is stored in the template library to complete the feature representation of the target area image of the initial frame.

3. The target tracking method based on image processing according to claim 1, characterized in that: The final matching coefficients of the HOG feature matching coefficient maximum and the grayscale feature matching coefficient maximum are calculated by the following method: α max =AC max +BP max Among them, α max is the final matching coefficient, C max is the maximum value of HOG feature matching coefficient, P max is the maximum value of the grayscale feature matching coefficient; A and B are weighted coefficients, A+B=1.

4. The target tracking method based on image processing according to claim 3, characterized in that: The S16 specifically includes: defining the matching coefficient threshold as α t , when α max >α t When the predicted position coordinates are output, the predicted position coordinates are stored in the predicted point library.

5. The target tracking method based on image processing according to claim 3, characterized in that: The S16 specifically includes: defining the matching coefficient threshold as α t , when 0.8*α t ≤α max ≤α t When the predicted position coordinates are output, the predicted position coordinates are stored in the template library.

6. The target tracking method based on image processing according to claim 3, characterized in that: Define the matching coefficient threshold as α t , when α max <0.8*α t , it enters the memory mode and uses the position recursion method to determine the predicted position coordinates.

7. The target tracking method based on image processing according to claim 6, characterized in that: After entering the memory mode, follow the steps below: Determine the number of prediction points in the prediction point library. If the number of prediction points is greater than or equal to the threshold, the prediction condition is determined to be met. At this time, position recursive prediction is performed to calculate the optimal matching position in the search area. When the number of points is less than the point threshold, it is judged that the prediction condition is not met. At this time, the best matching position at the previous moment is output.

8. The target tracking method based on image processing according to claim 7, characterized in that: The specific method of position recursive prediction is: x n+1 =Cx n +D <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> n+1 <h2 style=";text-align:left;direction:ltr"> =Cy<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> +D Among them, (x n ,y n ) is the target position at the previous moment, (x n+1 ,y n+1 ) is the predicted target position at the next moment; C and D are the coefficients of the linear prediction equation.

9. A target tracking system based on image processing, characterized in that: The target tracking system comprises: A feature representation unit, used for applying a HOG feature template and a grayscale image template to perform feature representation on the target area image of the initial frame; A search unit is used to obtain a grayscale image of the next frame, determine a search area with the target initial position as the center and in combination with empirical values; configure a search box that is consistent with the size of the target; determine a starting point of the search box in the search area and a search path composed of multiple search points, and traverse the search area along the search path starting from the starting point; A calculation unit, used to calculate the HOG value and grayscale feature of the search box at the starting point and each search point; A HOG feature matching coefficient determination unit, for the starting point and each search point, determines the HOG feature matching coefficient between the corresponding search box HOG value and the HOG feature template, a HOG feature matching coefficient matrix is ​​formed by multiple HOG feature matching coefficients, and a HOG feature matching coefficient maximum value and its corresponding search point position coordinates are determined based on the HOG feature matching coefficient matrix; A grayscale feature matching coefficient determination unit determines the grayscale feature matching coefficient between the grayscale feature of the search box and the grayscale image template for the starting point and each search point, forms a grayscale feature matching coefficient matrix from a plurality of grayscale feature matching coefficients, and determines the grayscale feature matching coefficient maximum value and its corresponding search point position coordinates based on the grayscale feature matching coefficient matrix; The predicted position coordinate determination unit determines whether the position coordinates of the two search points are equal. If they are equal, the final matching coefficient of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is calculated; otherwise, the maximum value of the maximum value of the HOG feature matching coefficient and the maximum value of the grayscale feature matching coefficient is assigned to the final matching coefficient, and the corresponding position coordinate is recorded as the predicted position coordinate; The predicted position coordinate output unit is configured with a matching coefficient threshold, and determines whether to output the predicted position coordinate based on a comparison result between the final matching coefficient and the matching coefficient threshold.

10. A terminal, characterized in that: It comprises a processor and a communication interface coupled to the processor, and the processor is used to run a computer program or instruction to implement the target tracking method based on image processing as described in claims 1 to 8.