A video target tracking method based on optical flow points

Abstract

The invention discloses a video target tracking method based on optical flow points. The method adopts a linear approximation method to roughly select a marginal marker point set of a target graphic, uses an ORB operator to detect feature key points and calculate corresponding feature description vectors to obtain an initial key point set, and simultaneously identifies the grayscale gradient of the target graphic. The marginal marker point set is overlapped and compared with the initial key point set, and key optical flow points are obtained by weighting. Then, the steps are repeated from the second frame to the six hundredth frame to construct an initial feature library. When the target is tracked, the initial feature library captures the key optical flow points in the current tracking frame, and uses a pyramid Lucas-Kanade optical flow method to evaluate forward and backward optical flows. Then, an ORB algorithm is used to detect and describe the current frame video image to obtain a key point set, and then three groups of three-point positioning configurations are used to perform a re-check with the initial feature library to complete tracking and confirm positioning. The method has the advantage that optical flow tracking and RGB grayscale tracking and positioning are effectively integrated, and the amount of calculation is effectively reduced.

Description

Technical Field

[0001] The invention relates to the field of video target tracking, in particular to a video target tracking method based on optical flow points. Background Art

[0002] Moving target tracking is one of the core technologies in the field of computer vision, and is also a key application technology in many fields such as security monitoring, human-computer interaction, intelligent transportation, aerospace, and medical diagnosis. So far, visual tracking has formed a set of basic theories and accumulated a large number of research results. Common moving target tracking technologies include detection-based tracking, matching-based tracking, filtering-based tracking, and fusion-based tracking. These methods analyze the characteristics of the visual tracking process from different perspectives and establish corresponding models to handle them, but a single tracking method has certain limitations or inherent defects. For example, the conventional matching-based algorithm has the characteristics of stable tracking and high accuracy, but because the target appearance model is generally based on global features or regional block features, it leads to a large search area, large amount of calculation, and poor real-time performance. When the size, shape, and posture of the target change or the target is occluded, the target model is not updated in time or updated excessively, which will affect the stability and effectiveness of the algorithm.

[0003] Optical flow is the instantaneous speed of pixel movement of a moving object in space on the observation imaging plane. The optical flow method uses the changes in pixels in the time domain in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of the object between adjacent frames. The instantaneous rate of change of grayscale at a specific coordinate point on a two-dimensional image plane is usually defined as an optical flow vector. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a video target tracking method based on optical flow points in view of the shortcomings of the prior art.

[0005] Technical solution: The video target tracking method based on optical flow points described in the present invention comprises the following steps:

[0006] S1, decomposing the video to be processed, decomposing the video to be processed frame by frame;

[0007] S2, taking the first frame of the video to be processed, manually selecting the target graphic, and using the linear approximation method to roughly select the marginal marker point set of the target graphic;

[0008] S3, using the ORB operator to detect feature key points and calculate the corresponding feature description vector, obtain the initial key point set, and identify the grayscale gradient of the target graphic;

[0009] S4, compare the overlap of the marginal mark point set with the initial key point set, and weight each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with a weighted value exceeding 0.5 are key optical flow points.

[0010] S5, continue to take the second frame to the 600th frame, repeat the steps of S2 to S4, and at the same time, arrange the marginal marker point set and the initial key point set obtained from the first frame to the 600th frame in three dimensions according to the grayscale gradient obtained from the first frame to the 600th frame, and obtain the change trend of the key optical flow points in the optical flow of the first frame to the 600th frame according to the grayscale gradient variable reflected in the three-dimensional arrangement, and obtain the initial feature library;

[0011] S6. When the target graphics need to be tracked in the subsequent video, the key optical flow points in the current tracking frame are captured according to the initial feature library, and the pyramid Lucas-Kanade optical flow method is used to evaluate the forward and backward optical flows. Then, the ORB algorithm is used to detect and describe the current frame video image to obtain the key point set.

[0012] S7, taking three groups of three-point positioning configurations from the key points obtained in S6 to double-check with the initial feature library to confirm that they meet the definition of the key optical flow point weights of the target graphic, thereby completing the tracking and positioning of the video target.

[0013] Preferably, after obtaining the key optical flow points in S4, each key optical flow point is compounded in the three-dimensional arrangement obtained in S5, the geometric center point of the target graphic in the three-dimensional arrangement state is marked, and the obtained geometric center point is included in the initial feature library.

[0014] Preferably, in S7, three groups of three-point positioning configurations are selected for re-check with the initial feature library, and at the same time, the geometric center point is checked for coincidence matching with the selected three groups of three-point positioning configurations.

[0015] Preferably, in S4, points with weighted values ​​exceeding 0.75 are key optical flow points.

[0016] Compared with the prior art, the present invention has the following beneficial effects: it effectively integrates optical flow tracking with RGB grayscale tracking and positioning, and can construct an initial feature library with larger data volume and more in line with the actual three-dimensional configuration of the target in the preparation stage, and compares the overlap of the marginal marker point set with the initial key point set, and weights each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0-1, and the points with a weighting value exceeding 0.5 are key optical flow points. The accuracy of tracking can be confirmed by double-checking the key point set with the initial feature library, and the amount of calculation is greatly reduced, thereby making the tracking calculation more efficient and effectively reducing the delay in the real-time video tracking process. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0019] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integrated connection; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Embodiment 1: A video target tracking method based on optical flow points comprises the following steps:

[0022] S1, decomposing the video to be processed, decomposing the video to be processed frame by frame;

[0023] S2, taking the first frame of the video to be processed, manually selecting the target graphic, and using the linear approximation method to roughly select the marginal marker point set of the target graphic;

[0024] S3, using the ORB operator to detect feature key points and calculate the corresponding feature description vector, obtain the initial key point set, and identify the grayscale gradient of the target graphic;

[0025] S4, compare the overlap of the marginal mark point set with the initial key point set, and weight each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with a weighted value exceeding 0.5 are key optical flow points.

[0026] S5, continue to take the second frame to the 600th frame, repeat the steps of S2 to S4, and at the same time, arrange the marginal marker point set and the initial key point set obtained from the first frame to the 600th frame in three dimensions according to the grayscale gradient obtained from the first frame to the 600th frame, and obtain the change trend of the key optical flow points in the optical flow of the first frame to the 600th frame according to the grayscale gradient variable reflected in the three-dimensional arrangement, and obtain the initial feature library;

[0027] S6. When the target graphics need to be tracked in the subsequent video, the key optical flow points in the current tracking frame are captured according to the initial feature library, and the pyramid Lucas-Kanade optical flow method is used to evaluate the forward and backward optical flows. Then, the ORB algorithm is used to detect and describe the current frame video image to obtain the key point set.

[0028] S7, taking three groups of three-point positioning configurations from the key points obtained in S6 to double-check with the initial feature library to confirm that they meet the definition of the key optical flow point weights of the target graphic, thereby completing the tracking and positioning of the video target.

[0029] Embodiment 2: A video target tracking method based on optical flow points comprises the following steps:

[0030] S1, decomposing the video to be processed, decomposing the video to be processed frame by frame;

[0031] S2, taking the first frame of the video to be processed, manually selecting the target graphic, and using the linear approximation method to roughly select the marginal marker point set of the target graphic;

[0032] S3, using the ORB operator to detect feature key points and calculate the corresponding feature description vector, obtain the initial key point set, and identify the grayscale gradient of the target graphic;

[0033] S4, compare the overlap of the marginal mark point set with the initial key point set, and weight each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with a weighted value exceeding 0.5 are key optical flow points.

[0034] S5, continue to take the second frame to the 600th frame, repeat the steps of S2 to S4, and at the same time, perform stereoscopic arrangement of the marginal marking point set and the initial key point set obtained from the first frame to the 600th frame according to the grayscale gradient obtained from the first frame to the 600th frame, and multiplex each key optical flow point in the stereoscopic arrangement, mark the geometric center point of the target figure in the stereoscopic arrangement state, and obtain the change trend of the key optical flow point in the optical flow of the first frame to the 600th frame according to the grayscale gradient variable reflected in the stereoscopic arrangement, obtain the initial feature library, and at the same time, include the obtained geometric center point into the initial feature library;

[0035] S6. When the target graphics need to be tracked in the subsequent video, the key optical flow points in the current tracking frame are captured according to the initial feature library, and the pyramid Lucas-Kanade optical flow method is used to evaluate the forward and backward optical flows. Then, the ORB algorithm is used to detect and describe the current frame video image to obtain the key point set.

[0036] S7. Collect three groups of three-point positioning configurations from the key points obtained in S6 to check against the initial feature library to confirm that they meet the definition of the key optical flow point weights of the target graphic. At the same time, check the geometric center point for coincidence and matching with the selected three groups of three-point positioning configurations, thereby completing the tracking and positioning of the video target.

[0037] Embodiment 3: A video target tracking method based on optical flow points, comprising the following steps:

[0038] S1, decomposing the video to be processed, decomposing the video to be processed frame by frame;

[0039] S2, taking the first frame of the video to be processed, manually selecting the target graphic, and using the linear approximation method to roughly select the marginal marker point set of the target graphic;

[0040] S3, using the ORB operator to detect feature key points and calculate the corresponding feature description vector, obtain the initial key point set, and identify the grayscale gradient of the target graphic;

[0041] S4, compare the overlap of the marginal mark point set with the initial key point set, and weight each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with a weighted value exceeding 0.75 are key optical flow points.

[0042] S5, continue to take the second frame to the 600th frame, repeat the steps of S2 to S4, and at the same time, perform stereoscopic arrangement of the marginal marking point set and the initial key point set obtained from the first frame to the 600th frame according to the grayscale gradient obtained from the first frame to the 600th frame, and multiplex each key optical flow point in the stereoscopic arrangement, mark the geometric center point of the target figure in the stereoscopic arrangement state, and obtain the change trend of the key optical flow point in the optical flow of the first frame to the 600th frame according to the grayscale gradient variable reflected in the stereoscopic arrangement, obtain the initial feature library, and at the same time, include the obtained geometric center point into the initial feature library;

[0043] S6. When the target graphics need to be tracked in the subsequent video, the key optical flow points in the current tracking frame are captured according to the initial feature library, and the pyramid Lucas-Kanade optical flow method is used to evaluate the forward and backward optical flows. Then, the ORB algorithm is used to detect and describe the current frame video image to obtain the key point set.

[0044] S7. Collect three groups of three-point positioning configurations from the key points obtained in S6 to check against the initial feature library to confirm that they meet the definition of the key optical flow point weights of the target graphic. At the same time, check the geometric center point for coincidence and matching with the selected three groups of three-point positioning configurations, thereby completing the tracking and positioning of the video target.

[0045] The advantage of this technical solution is that it effectively integrates optical flow tracking with RGB grayscale tracking and positioning. In the preparation stage, an initial feature library with larger data volume and more in line with the actual three-dimensional configuration of the target can be constructed, and the marginal marker point set and the initial key point set are overlapped and compared. Each point is weighted according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with weights exceeding 0.5 are key optical flow points. The accuracy of tracking can be confirmed by double-checking the key point set with the initial feature library. The amount of calculation is greatly reduced, making the tracking calculation more efficient and effectively reducing the delay in the real-time video tracking process.

[0046] In the present invention, unless otherwise clearly specified and limited, the first feature being "above" or "below" the second feature may mean that the first feature is in direct contact with the second feature, or the first feature and the second feature are in indirect contact through an intermediate medium. Moreover, the first feature being "above", "above" and "above" the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature being "below", "below" and "below" the second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature. In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example.

[0047] Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradicting each other.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video target tracking method based on optical flow points, characterized in that: The following steps are involved: S1, decomposing the video to be processed, decomposing the video to be processed frame by frame; S2, taking the first frame of the video to be processed, manually selecting the target graphic, and using the linear approximation method to roughly select the marginal marker point set of the target graphic; S3, using the ORB operator to detect feature key points and calculate the corresponding feature description vector, obtain the initial key point set, and identify the grayscale gradient of the target graphic; S4, compare the overlap of the marginal mark point set with the initial key point set, and weight each point according to the degree of freedom between the corresponding points of the two. The lower the degree of freedom, the higher the weight. The weighting value range is 0 to 1. Points with a weighted value exceeding 0.5 are key optical flow points. S5, continue to take the second frame to the 600th frame, repeat the steps of S2 to S4, and at the same time, arrange the marginal marker point set and the initial key point set obtained from the first frame to the 600th frame in three dimensions according to the grayscale gradient obtained from the first frame to the 600th frame, and obtain the change trend of the key optical flow points in the optical flow of the first frame to the 600th frame according to the grayscale gradient variable reflected in the three-dimensional arrangement, and obtain the initial feature library; S6. When the target graphics need to be tracked in the subsequent video, the key optical flow points in the current tracking frame are captured according to the initial feature library, and the pyramid Lucas-Kanade optical flow method is used to evaluate the forward and backward optical flows. Then, the ORB algorithm is used to detect and describe the current frame video image to obtain the key point set. S7, taking three groups of three-point positioning configurations from the key points obtained in S6 to double-check with the initial feature library to confirm that they meet the definition of the key optical flow point weights of the target graphic, thereby completing the tracking and positioning of the video target.

2. The video target tracking method based on optical flow points according to claim 1, characterized in that: After the key optical flow points are obtained in S4, the three-dimensional arrangement obtained in S5 is compounded after each key optical flow point, the geometric center point of the target figure in the three-dimensional arrangement state is marked, and the obtained geometric center point is included in the initial feature library.

3. The video target tracking method based on optical flow points according to claim 2 is characterized in that: In S7, three groups of three-point positioning configurations are selected for re-checking with the initial feature library, and at the same time, the geometric center point is checked for coincidence matching with the selected three groups of three-point positioning configurations.

4. The video target tracking method based on optical flow points according to claim 1, characterized in that: In S4, points with weighted values ​​exceeding 0.75 are key optical flow points.