Tracking method combining affine transformation correction and improved generalized Hough transform
By combining affine transformation and improving generalized Hough transformation, the relative relationship between feature points and tracking points is corrected, and the tracking problem when partial areas of the target image are dynamically missing is solved, achieving high-precision and stable target tracking, which is suitable for target tracking applications in complex environments.
Patent Information
- Application Number
- CN202510575069.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-06
- Publication Date
- 2025-08-01
- Estimated Expiration
- 2045-05-06
AI Technical Summary
The prior art cannot achieve continuous tracking when partial areas of the target image are dynamically missing, especially under the influence of factors such as atmospheric turbulence and rapid motion, the tracking accuracy is insufficient.
Combining affine transformation correction and improving generalized Hough transform, the relative relationship between feature points and tracking points is corrected by calculating the affine transformation matrix between the front and back frames, a Hough table is established and the target position is reverse solved.
It improves tracking accuracy and stability in the absence of dynamics in some areas, achieves continuous and accurate target tracking, reduces the requirements for high-precision tracking system hardware, and enhances environmental adaptability.
Smart Images
Figure CN120088517B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of optical systems and image processing, and particularly relates to a tracking method combining affine transformation correction and improved generalized Hough transform. Background Art
[0002] Visual target recognition technology has been widely applied in both civilian and defense fields. In recent years, target tracking technology has always been the focus of research at home and abroad. The tracking accuracy of traditional ATP (acquisition, tracking, and pointing system) tracking mode is relatively low. Modern tracking systems generally add a fast steering mirror and a fine tracking sensor at the back, and use the target miss distance provided by the fine tracking sensor to control the fast steering mirror to achieve more accurate tracking. This structure is called a high-precision tracking system. The adaptive optical system includes a tip-tilt mirror, a deformable mirror, a detector, and a wavefront processor, etc., which can detect and correct wavefront errors in real time, enabling the target to be imaged with a quality close to the optical diffraction limit. However, the correction ability of the adaptive optical system is limited, and it can only correct targets with a scale of several to more than a dozen times the diffraction limit. For some high-frequency errors, effective correction still cannot be achieved. The image after AO correction still has the characteristics of blurred image contours, and there may also be a phenomenon of dynamic loss in some areas of the target image, resulting in problems such as the inability to continuously track the target.
[0003] Regarding the target tracking with dynamic loss in some areas caused by occlusion, researchers have conducted a large number of studies, and the main achievements are as follows: In 2016, Mu Xiaokai et al. used the frame difference method to detect the target and smoothed the target trajectory through a Kalman filter. When the target was occluded, a suitable fitting segment was selected from the smoothed trajectory, and the least squares method was used to adaptively fit the motion trajectory of the target to predict the target position. In 2017, Shen Chenggen et al. utilized the fused data of a camera and a lidar sensor to create an occlusion map based on the relative distance, azimuth angle, and size of the existing target, and compared this map with the trajectory prediction state to predict the occlusion situation between targets. When the target was occluded, a background tracker was used to analyze the motion trajectory of background points to predict the target position. In २०१८, Gu Chenggang et al. proposed a tracking method based on feature fusion and occlusion detection. This method fused two complementary features, HOG (histogram of oriented gradients) and color features, through a weighting strategy, and then proposed an occlusion detection method according to the response function of the fused features. In 2018, Ruan Weijian et al. constructed multiple CFs using the global and local appearances of the target, jointly measured the overall and local correlation responses within the search window to estimate the target position, proposed a triangular structure model to describe the spatial relationship between local parts, and used structural similarity to constrain the target position.
[0004] Analyze the current reported tracking research when there is dynamic loss in some regions of the image. During the tracking process of any position on the target image, when the designated tracking position cannot be continuously tracked due to the dynamic loss of some occluded regions, there is no relevant report at home and abroad. Summary of the Invention
[0005] The technical problem to be solved by the present invention: During the extended target tracking process, factors such as atmospheric turbulence and rapid movement cause dynamic loss of some regions of the image, making it impossible to achieve continuous tracking. In response to this problem, a tracking method combining affine transformation correction and improved generalized Hough transform is proposed to improve the tracking accuracy in the case of dynamic loss of some regions and achieve continuous, accurate and stable tracking. This method is based on the improved generalized Hough transform and calculates the affine transformation matrix between the front and back frames to correct the relative relationship between the feature points and the designated tracking points, which can improve the tracking accuracy. This method is stable and easy to implement, providing technical support for the better application of extended target tracking in fields such as astronomical observation, high-quality laser beam transmission, and target reconnaissance.
[0006] The technical solution adopted by the present invention to solve its technical problem is:
[0007] A tracking method combining affine transformation correction and improved generalized Hough transform, and the implementation steps are as follows:
[0008] Step S110, obtain the extended target image sequence I;
[0009] Step S120, read the extended target image sequence I, select the tracking point C in the k-th frame image, where k is a positive integer and less than the total number of images in the extended target image sequence. Based on the SURF algorithm, extract N feature points from each frame of the image. The position of the i-th feature point in the k-th frame image is represented by , the i-th feature point in the (k + 1)-th frame image is represented by , i = 1... N, and define the degree of loss as , and match the feature points extracted from the k-th frame image with the feature points extracted from the (k + 1)-th frame image;
[0010] Step S130, calculate the affine transformation matrix T from the k-th frame to the (k + 1)-th frame. According to the affine transformation matrix T, correct the position of the feature points extracted from the k-th frame image from to , the position of the tracking point from to , and correct the feature point direction to , and establish the position of the corrected feature points in the k-th frame and the position of the corrected tracking point Hough table of the relative relationship
[0011] Step S140: Index the relative relationship between the corresponding feature points and the tracking points in the Hough table established in the previous frame by using the feature points matched in the (k + 1)-th frame, and inversely solve the position of the tracking points in the (k + 1)-th frame.
[0012] Step S150: Read the next frame, and repeat the above steps until all frames are processed.
[0013] A computing device, comprising: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device is caused to execute the tracking method based on the combination of affine transformation correction and improved generalized Hough transform.
[0014] A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device is caused to execute the tracking method based on the combination of affine transformation correction and improved generalized Hough transform.
[0015] The present invention has the following beneficial effects compared with the prior art:
[0016] 1. The present invention realizes effective improvement of tracking accuracy and stability and improvement of tracking precision during the tracking of an extended target with dynamic missing in a partial area, and realizes continuous and accurate tracking.
[0017] 2. The present invention combines image processing with algorithm optimization, effectively solves the problem of continuous and accurate tracking when there is dynamic missing in a partial area of an extended target, and provides a technical option for the application of target tracking in a high-precision tracking system, such as developing a tracking prediction model of deep learning by using the relationship between the information of the front and rear frames.
[0018] 3. The present invention uses the improved generalized Hough transform to realize continuous tracking when there is dynamic missing in a partial area of an extended target image, reduces the requirements for the hardware of a high-precision tracking system to a certain extent, and improves the environmental adaptability of the system to extended targets. Description of the Drawings
[0019] Figure 1 is the flow of the tracking method based on the combination of affine transformation correction and improved generalized Hough transform of the present invention;
[0020] Figure 2(a) is the original image of the k-th frame;
[0021] Figure 2(b) is the original image of the (k + 1)-th frame;
[0022] Figure 3(a) is the extraction of SURF feature points from the k-th frame image;
[0023] Figure 3(b) shows the extraction of SURF feature points from the (k + 1)-th frame image;
[0024] Figure 4 To match the feature points extracted from the k-th and (k + 1)-th frame images;
[0025] Figure 5 Is the Hough voting result for the (k + 1)-th frame image;
[0026] Figure 6 Is the position of the tracking point in the (k + 1)-th frame image. Detailed implementation manner
[0027] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art.
[0028] Figure 1 Is a flowchart of the tracking method of the present invention based on the combination of affine transformation correction and improved generalized Hough transform. As Figure 1 shown, the method includes the following steps:
[0029] Step S110, obtaining the extended target image sequence I, as shown in Figures 2(a) and 2(b). In one embodiment, the pixel size of the extended target image I is 442×448.
[0030] Step S120, reading the extended target image sequence I, and selecting the tracking point C in the k-th (k is a positive integer and less than the total number of images in the image sequence) frame image shown in Figure 2(a) , based on the SURF (Speeded Up Robust Features) algorithm, extracting N feature points (N is generally not less than 20) from each frame of the image. As shown in Figure 3(a), the position of the i-th feature point in the k-th frame image is represented by (i = 1…N). As shown in Figure 3(b), the i-th feature point in the (k + 1)-th frame image is represented by , and the degree of missing is defined as , and the feature points extracted from the k-th frame image are matched with the feature points extracted from the (k + 1)-th frame image (adjacent frame matching), as Figure 4 shown.
[0031] In one embodiment, step S120 may include: reading the extended target image sequence I, and specifying the tracking point C in the first frame image , based on the improved generalized Hough transform framework, the SURF algorithm is used to extract feature points from each frame of the image. The coordinates of the feature points in the first frame of the image are defined as , the coordinates of the feature points in the second frame of the image are , and the degree of missing is defined as . The a total of 286 feature points extracted from the first frame of the image are matched with the a total of 181 feature points extracted from the second frame of the image, and 24 matching point pairs are obtained. The corresponding first-frame and second-frame feature point index numbers of these 24 matching point pairs are: . Only some of the feature points and matching point pairs are listed above. Those skilled in the art can know the result data of extracting feature points and performing feature point matching through the SURF algorithm. It should be understood that different feature point data can also be obtained by changing the parameters of the SURF algorithm according to actual needs.
[0032] Degree of missing The calculation process includes: after each frame of the image is extracted by the Canny edge extraction operator, the number of pixels within the closed image is calculated with the number of pixels extracted from the reference image to calculate the degree of missing. The expression is:
[0033] ,
[0034] The greater the degree of missing, the greater it is, and its value range is [0, 1]. The closed image mentioned above refers to the closed image formed after the edges of the image are extracted by the Canny edge operator, and the reference image refers to the original image without missing.
[0035] Tracking point C can be located in the non-missing part of the target or in the missing part of the target.
[0036] Step S120 also includes: after matching the feature points extracted from the k-th frame of the image with the feature points extracted from the k + 1-th frame of the image, the random sample consensus algorithm (RANSAC) is used to screen out the wrong matching points.
[0037] Step S130, calculate the affine transformation matrix T from the k-th frame to the k + 1-th frame. According to the affine transformation matrix T, the position of the feature points extracted from the k-th frame of the image is changed from to , and the position of the tracking point is changed from to . The feature point direction is changed to . Establish the position of the corrected feature points in the k-th frame The Hough table of the relative relationship with the corrected tracking point position between them.
[0038] In a specific implementation, the process of calculating the affine transformation matrix includes:
[0039] The affine transformation between the feature points in the (k + 1)-th frame image and the feature points in the k-th frame image can be written in matrix form:
[0040] (1)
[0041] where are the rotation, scaling, and shear parameters, are the translation parameters. Denote the affine transformation matrix as T. The process of finding the affine transformation matrix T is actually to make the transformed feature points of the previous frame as close as possible to the feature points of the current frame, that is, to solve:
[0042] (2)
[0043] Transform the problem into a system of linear equations. For each pair of matching points and , write the following equation:
[0044] (3)
[0045] In the specific example of the 1st frame and the 2nd frame, the above equation (3) can be expressed as: Combine the equations of the points matched in adjacent frames to obtain a system of linear equations:
[0046] (4)
[0047] Represented in matrix form as:
[0048] (5)
[0049] where A is the coordinate vector of the feature points in the k-th frame image, B is the coordinate vector of the feature points in the (k + 1)-th frame image, A can be regarded as the coefficient matrix, with size , is the number of matched feature points.
[0050] where are the parameters to be solved:
[0051] (6)
[0052] B is the coordinate vector of the feature points in the (k + 1)-th frame image:
[0053] (7)
[0054] The number of matching point pairs is usually greater than the number of unknowns ( ), and the unknowns refer to , and the least squares method is used to solve the optimal solution:
[0055] (8)
[0056] The digital matrix at the rightmost end of the equal sign in the above formula (8) is the solution for the specific example of the first and second frames. To keep the space concise, the solutions obtained from formula (8) in the above example are directly listed in formula (8). The same applies to the following formulas such as formula (9) and (10).
[0057] According to the affine transformation matrix T, the position of the feature points extracted from the k-th frame image is changed from to , and the position of the tracking point is changed from to , and the process of changing the direction of the feature points to includes:
[0058] (9)
[0059] Calculate the relative vector between the corrected feature points and the corrected tracking points in the k-th frame:
[0060] (10)
[0061] Calculate the rotation angle :
[0062] (11)
[0063] Change the direction of the feature points to , that is:
[0064] (12)
[0065] The process of establishing the Hough table for the relative relationship between the position of the corrected feature points and the position of the corrected tracking points in the k-th frame includes:
[0066] Map the direction of the corrected feature points into the direction interval. Define the number of direction intervals M as 30, then the angular range of each interval is:
[0067] (13)
[0068] That is to say, the direction of the feature points is corrected to , and the interval division of the feature point directions is performed again.
[0069] Calculate the direction interval index j:
[0070] (14)
[0071] Establish the relative relationship between the position of the corrected feature points in the k-th frame and the position of the corrected tracking points in the Hough table as shown in Table 1:
[0072] Table 1 Hough table
[0073]
[0074] As shown above, the structure of the Hough table includes: the direction interval index calculated by formula (14), each angular range calculated by formula (13) and the relative vector calculated by formula (10), and the set of relative vectors assigned to the corresponding intervals through formulas (12) and (14).
[0075] Step S140, use the feature points matched in the (k + 1)-th frame to index the relative relationship between the corresponding feature points and tracking points in the Hough table established in the previous frame, and then solve the position of the tracking points in the (k + 1)-th frame inversely :
[0076] (15)
[0077] is the candidate position of the tracking point.
[0078] And vote through the accumulator matrix. As Figure 5 shown, in Figure 5 , the accumulator matrix response and the accumulative response refer to the matrix representation of the accumulator array and contain all possible cumulative voting values:
[0079] (16)
[0080] is the accumulator matrix, which is used to record the accumulative result in the Hough voting.
[0081] The position with the most votes in the accumulator matrix is the position of the tracking point inversely solved in the (k + 1)-th frame, that is, the simulation position, as Figure 6 shown:
[0082] (17)
[0083] Calculate the average pixel error in the X direction and the average pixel error in the Y direction according to the true position and the simulated position of the tracking points in the (k + 1)-th frame, as shown in Table 2. The above true position is the manually marked ground truth. When performing Hough voting, each matched feature point will be voted at different positions. Therefore, the position pointed to by the vote of each matched feature point is called the candidate position, and the position with the most votes in the candidate positions is the simulated position.
[0084] Table 2 True / Simulated Position - Average Pixel Error Table
[0085]
[0086] Step S150, read the next frame, and repeat the above steps until all frames are processed.
[0087] The method according to the present invention is based on the affine transformation matrix of adjacent frames and realizes target tracking by correcting the improved generalized Hough transform. The idea of the improved generalized Hough transform is to establish a Hough table of the feature points in the previous frame and the reference points, and use the feature points in the next frame and the Hough table established in the previous frame to inversely solve the position of the reference points in the next frame. The present invention combines the affine transformation matrix with the improved generalized Hough transform, improving the tracking accuracy when there is dynamic loss in some areas of the extended target. This tracking method is stable and easy to implement, providing technical support for realizing high-precision and stable tracking of extended targets in complex environments, so as to be better applied to technical fields such as astronomical observation, high-quality laser beam transmission, and target reconnaissance.
[0088] According to an embodiment of the present invention, there is also provided a computing device, including: at least one processor and a memory storing program instructions; when the program instructions are read and executed by the processor, the computing device is enabled to execute a tracking method combining affine transformation correction and improved generalized Hough transform.
[0089] According to an embodiment of the present invention, there is also provided a readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device is enabled to execute a tracking method combining affine transformation correction and improved generalized Hough transform.
[0090] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and technologies have not been shown in detail so as not to obscure the understanding of this specification.
[0091] Although the invention has been described in terms of a limited number of embodiments, those skilled in the art, having the benefit of the foregoing description, will appreciate that other embodiments can be devised within the scope of the invention as described herein. Additionally, it should be noted that the language used in this specification has been principally selected for readability and instructional purposes and not to limit or circumscribe the inventive subject matter.
Claims
1. A tracking method combining affine transformation correction and improved generalized Hough transform, characterized in that: The implementation steps are as follows: Step S110: Obtain the extended target image sequence I; Step S120: read the extended target image sequence I and select the tracking point C in the kth frame image. , k is a positive integer, which is less than the total number of images in the extended target image sequence. Based on the SURF algorithm, N feature points are extracted from each frame of the image. The position of the i-th feature point in the k-th frame image is expressed as Indicates that the i-th feature point of the k+1-th frame image is represented by Indicates that i=1...N, and defines the missing degree as , match the feature points extracted from the k-th frame image with the feature points extracted from the k+1-th frame image; Step S130, calculate the affine transformation matrix T from the k-th frame to the (k + 1)-th frame. According to the affine transformation matrix T, the position of the feature points extracted from the k-th frame image is changed from to , the tracking point position is changed from to , the feature point direction is changed to , establish a Hough table for the relative relationship between the position of the corrected feature points in the k-th frame and the position of the corrected tracking points ; Step S140: Use the feature points matched in the (k + 1)-th frame to index the relative relationship between the corresponding feature points and the tracking points in the Hough table established in the previous frame, and inversely solve the position of the tracking points in the (k + 1)-th frame; Step S150: Read the next frame, and repeat the above steps until all frames are processed; In step S130, the process of calculating the affine transformation matrix T from the k-th frame to the (k + 1)-th frame includes: The affine transformation between the feature points in the (k + 1)-th frame image and the feature points in the k-th frame image is written in matrix form: （1） Among them, are the rotation, scaling, and shear parameters, is the translation parameter. Denote the affine transformation matrix as T, and solve: （2） Convert the problem into a system of linear equations. For each set of matching points and , write the following equations: （3） It is represented in matrix form as: （5） Among them, A is the coordinate vector of the feature points of the k-th frame image, and B is the coordinate vector of the feature points of the (k + 1)-th frame image, where is the parameter to be solved: （6） The least squares method is used to solve the optimal solution: （8）。 2. The tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: In step S120, after matching the feature points extracted from the k-th frame image with the feature points extracted from the (k + 1)-th frame image, the random sample consensus algorithm is used to filter out the wrong matching points.
3. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: The degree of missing defined in step S120 is the number of pixels within the closed image after extracting each frame of the image by the Canny edge extraction operator and the number of pixels extracted from the reference image calculated, and the expression is: ， The greater the degree of loss, the greater, and its value range is [0, 1].
4. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: The tracking point C selected in step S120 is located in the non-missing part of the target.
5. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: The tracking point C selected in step S120 is located in the missing part of the target.
6. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: In step S130, according to the affine transformation matrix T, the position of the feature points extracted from the k-th frame image is changed from to , and the process of changing the position of the tracking point from to includes: （9）。 7. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: In step S130, establish the position of the feature points after correction for the k-th frame and the position of the tracking points after correction The process of the Hough table for the relative relationship between them includes: Calculate the relative vector between the corrected feature points and the corrected reference points of the k-th frame : （10） Calculate the rotation angle Then correct the direction of the feature point to , that is: （12） Map the corrected direction into the direction interval. Define the number of direction intervals as M, then the angular range of each interval is as follows: （13） Calculate the direction interval index j: （14） Establish the position of the corrected feature points in the k-th frame and the position of the corrected tracking points The Hough table of the relative relationship between them. The structure of the Hough table includes: the direction interval index calculated by formula (14), each angle range calculated by formula (13) and the relative vector of the reference point calculated by formula (10) , and the relative vectors assigned to the corresponding intervals through formulas (12) and (14) set.
8. A tracking method based on the combination of affine transformation correction and improved generalized Hough transform according to claim 1, characterized in that: In step S140, the relative relationship between the corrected feature point and the tracking point is indexed in the Hough table established in the previous frame using the feature points matched in the (k + 1)-th frame, so as to inversely solve the position of the tracking point in the (k + 1)-th frame, including: The position of: Solve the candidate position of the tracking points: （15） is a candidate position of the tracking point; And vote through the accumulator matrix: （16） The position with the most votes in the accumulator matrix is the position of the tracking points inversely solved in the (k + 1)-th frame: （17）。