Image edge tracking method based on sub-pixel positioning

Abstract

The invention relates to an image edge tracking method based on sub-pixel positioning, which relates to the field of computer vision technology application, and comprises the following steps: selecting an initial local anchor point; determining the search range of the next edge pixel point, and obtaining a plurality of candidate edge pixel points; selecting the candidate point with the maximum confidence coefficient as the next edge point, and updating the edge direction; continuing tracking until the inner product energy values of all the pixel points in the search coding range are 0, and turning to the other edge direction of the initial pixel point for tracking; combining the edge pixel points obtained by tracking in the two directions; and carrying out sub-pixel relocation on each pixel obtained by edge tracking. According to the image edge tracking method based on sub-pixel positioning, the edge detection effect is good for the image with the vague edge and the low contrast ratio. The edge of the extracted image is not too wide, redundant information is reduced, smoothness and robustness are high, and follow-up processing of the image is simplified.

Description

Technical Field

[0001] The present invention relates to the application field of computer vision technology, and in particular to an image edge tracking method based on sub-pixel positioning. Background Art

[0002] Images are an important source of information for humans, and image edges, as the main low-level visual features of images, are important carriers of image information. With the rapid development of computer technology, computer vision technology is also changing with each passing day. As a key computer vision technology, image edge detection is widely used in industrial and agricultural production, visual navigation, medical imaging, engineering robots and other fields. The effect of image edge detection directly affects the subsequent image processing work, such as target recognition and target posture measurement. Sobel operator method, Prewitt operator method, Roberts operator method and Canny algorithm are the most widely used concentrated image edge detection methods. For images with blurred edges and low contrast, the commonly used image edge detection effect is poor, the obtained image edges are discontinuous, and have a certain width, which increases the difficulty of subsequent image processing. When using the classic Canny algorithm to detect image edges, it is necessary to set a global threshold according to the brightness of the external environment, so the algorithm is cumbersome and inefficient.

[0003] Usually, the coordinates of each edge pixel point obtained by the image edge detection method are expressed in an integer form, which introduces truncation errors and causes a certain deviation between the detected image edge and the actual edge. The commonly used sub-pixel positioning method is the interpolation method, which is limited by its weak anti-interference ability and cannot be widely used. When using deep learning to perform sub-pixel relocation, although the effect is good, it is not interpretable enough and has high requirements for computing resources. For fields such as aerospace, due to the small number of reference samples, the problem of insufficient training is prone to occur during deep learning. Summary of the invention

[0004] The present invention aims to solve the technical problems in the prior art and provide an image edge tracking method based on sub-pixel positioning involving image recognition, target detection and three-dimensional visual measurement. The single-pixel edge tracking strategy of the method of the present invention is based on the assumption that the image edge direction is continuous, and refers to the local extreme value and gradient direction of the edge pixel point to extract the curve set composed of the image edge pixels. For images with blurred edges and low contrast, the edge detection effect is good.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] An image edge tracking method based on sub-pixel positioning comprises the following steps:

[0007] Step 1: Select the pixel point with the largest inner product energy value among the adjacent pixels in the gradient direction and the same edge direction as the initial local anchor point;

[0008] Step 2: Based on the assumption that the edge directions of the edge pixels of the image are continuous, determine the search range of the next edge pixel and obtain multiple candidate edge pixels;

[0009] Step 3: Referring to the rules for selecting the next edge pixel point, the confidence of multiple candidate edge pixel points is obtained after weighted calculation, the candidate point with the largest confidence is selected as the next edge point, and the edge direction is updated;

[0010] Step 4: Continue tracking by referring to the method of step 2 and step 3 until the inner product energy values ​​of all pixels within the search coding range are 0, and then switch to another edge direction of the initial pixel for tracking;

[0011] Step 5: Combine the edge pixels obtained by tracking in two directions to obtain a complete image edge;

[0012] Step 6: Using the edge pixel gradient direction as a constraint, perform sub-pixel relocation on each pixel obtained by edge tracking.

[0013] In the above technical solution, the rules for selecting the next edge pixel in step 3 include:

[0014] 1) The pixel is a maximum point in its neighborhood;

[0015] 2) The inner product energy amplitude is the largest;

[0016] 3) The edge direction is the same as the previous edge pixel.

[0017] In the above technical solution, the number of candidate edge pixel points in step 2 is 3.

[0018] In the above technical solution, step 6 specifically includes:

[0019] Take an edge pixel p as the starting point of sub-pixel relocation;

[0020] All pixels in the neighborhood S(p) of edge pixel p satisfy the equation:

[0021] G(q i )·(pq i )=0

[0022] Reposition the image edge pixels p, Under the premise of taking the minimum value, obtain the coordinates of the edge pixel p after relocation value.

[0023] In the above technical solution, for equation:

[0024] G(q i )·(pq i )=0

[0025] The solution process is:

[0026] Suppose the gradient value of edge pixel p(x0,y0) is (g x ,g y ), the straight line passing through the edge pixel is:

[0027] y=kx+b,k=-g y / g x ,b=y0-kx0

[0028] In the formula, k and b are the slope and constant intercept of the straight line respectively. If it is limited to the straight line l, then:

[0029]

[0030] The Lagrange multiplier method is used to solve the least squares estimate that satisfies the linear constraints of the model:

[0031]

[0032] Among them, λ represents the Lagrange multiplier; the auxiliary function equation is constructed as follows:

[0033] F(β,λ)=‖Y-Xβ‖ 2 +2λ(Aβ-B)=(Y-Xβ)′(Y-Xβ)+2λ(Aβ-B) (8)

[0034] Where A, X and B are parameter matrices.

[0035] Set the partial derivatives of each component of parameter β in the auxiliary function equation to 0, solve the partial derivatives of each component of β in the auxiliary function F(β,λ), and make the partial derivatives of each component of β equal to 0, and we get:

[0036]

[0037] Among them, X ′ is the transposed matrix of matrix X, A ′ is the transposed matrix of matrix A;

[0038] Constrained Least Squares Solution It is expressed as:

[0039]

[0040] in, is the estimated value of the Lagrangian operator;

[0041] Unconstrained least squares solution

[0042]

[0043] have to:

[0044]

[0045] Since the rank of matrix A is 1, the parameter matrix A(X ′ X) -1 A ′ is a 1st order reversible matrix, then:

[0046]

[0047] Get the coordinates of edge pixel p after sub-pixel relocation

[0048]

[0049] In the above technical solution, in order to ensure the convergence of the sub-pixel positioning algorithm, the distance between the original pixel and the relocated pixel is solved Compared with the distance threshold τ, when , the edge pixel relocation is deemed invalid and the original edge pixel coordinate value is retained.

[0050] The present invention has the following beneficial effects:

[0051] The image edge tracking method based on sub-pixel positioning of the present invention, wherein the single-pixel edge tracking strategy involved is based on the assumption that the image edge direction is continuous, and refers to the local extreme value and gradient direction of the edge pixel point to extract a curve set composed of the image edge pixels. The method of the present invention has a good effect of edge detection for images with blurred edges and low contrast. The extracted image edge does not have an excessive width, reduces the amount of redundant information, is smooth, has strong robustness, and simplifies the subsequent processing of the image.

[0052] The image edge tracking method based on sub-pixel positioning of the present invention uses the gradient direction as a constraint and refers to the grayscale values ​​of pixels in adjacent areas to perform sub-pixel repositioning of edge pixels in order to reduce the deviation between the extracted image edge and the actual edge. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0054] Figure 1Schematic diagram of the anchor point detection method in the image edge tracking method based on sub-pixel positioning of the present invention, wherein (a) is a non-anchor point (does not satisfy property 1 that the selected local anchor point must meet), (b) is a non-anchor point (does not satisfy property 2 that the selected local anchor point must meet), and (c) is an anchor point.

[0055] Figure 2 The complete edge in the image edge tracking method based on sub-pixel positioning of the present invention is a schematic diagram of the union of the edges tracked in two directions initialized by the anchor point.

[0056] Figure 3 It is a schematic diagram of the search range of the next edge point in the image edge tracking method based on sub-pixel positioning of the present invention.

[0057] Figure 4 Schematic diagram of the gradient direction of pixels in the local image.

[0058] Figure 5 Schematic diagram of the inner product energy value of the pixel and the search range of the edge pixel.

[0059] Figure 6 It is a schematic diagram of the sub-pixel repositioning edge pixel range in the image edge tracking method based on sub-pixel positioning of the present invention. DETAILED DESCRIPTION

[0060] The image edge tracking method based on sub-pixel positioning of the present invention comprises the following steps:

[0061] Step 1: Select the pixel point with the largest inner product energy value among the adjacent pixels in the gradient direction and the same edge direction as the initial local anchor point;

[0062] Step 2: Based on the assumption that the edge directions of the edge pixels of the image are continuous, the search range of the next edge pixel is determined, and three candidate edge pixels are obtained;

[0063] Step 3: Refer to the three rules for selecting the next edge pixel point, and obtain the confidence conf of the three candidate edge pixel points after weighted calculation. Select the candidate point with the largest confidence as the next edge point, and update the edge direction;

[0064] Step 4: Continue tracking by referring to the method of step 2 and step 3 until the inner product energy values ​​of all pixels within the search coding range are 0, and then switch to another edge direction of the initial pixel for tracking;

[0065] Step 5: Combine the edge pixels obtained by tracking in two directions to obtain a complete image edge;

[0066] Step 6: Using the edge pixel gradient direction as a constraint, perform sub-pixel relocation on each pixel obtained by edge tracking.

[0067] Combine the following Figures 1 to 6 , the tracking method of the present invention is described in more detail.

[0068] The image edge tracking method based on sub-pixel positioning of the present invention is divided into two parts: a single-pixel edge tracking strategy and a sub-pixel positioning strategy.

[0069] 1. Single-pixel edge tracking strategy

[0070] The edge direction of objects in nature (i.e., the normal direction of the gradient direction of the edge pixels of the image) generally has obvious continuity. Based on this, the single-pixel edge tracking strategy selects the local anchor point as the starting point, tracks the edge pixels one by one along the edge extension direction, and strives to extract a set of curves with a width of a single pixel.

[0071] In order to conveniently and accurately describe the process of image edge tracking, it is necessary to introduce the concepts of inner product energy value and local aiming point. For an edge pixel p, the surrounding local image with the point as the geometric center is defined as the neighborhood S(p) of the point, and the gradient G(p) of the edge pixel p is:

[0072] G(p)=(G x (p),G y (p)) (1)

[0073] Among them, G x (p) and G y (p) represent the horizontal gradient and vertical gradient of edge pixel p respectively; take any pixel point q in the neighborhood S(p) of edge pixel p i , the pixel gradient value G(q i )for:

[0074] G(q i )=(G x (q i ),G x (q i )) (2)

[0075] The inner product energy value IP(p) of point p is obtained by performing inner product accumulation on the gradient values ​​of point p and all the pixels in its neighborhood:

[0076]

[0077] The selected local anchor point must meet the following two conditions and properties at the same time:

[0078] Property 1: The edge direction of the pixel is the same as that of the adjacent pixel in the edge direction;

[0079] Property 2: Compared with its neighboring pixels in the gradient direction, the inner product energy value of this pixel is the largest.

[0080] The method for selecting local anchor points is as follows: Figure 1 As shown, Figure 1 The inner product energy value of point A in (a) is not the maximum point in the gradient direction, so point A cannot be selected as the anchor point; Figure 1 In (b), the pixels in the neighborhood of point A and located on the edge direction (the two pixels adjacent to point A on the left and right) have different edge directions and cannot be selected as anchor points. Figure 1 Point A in (c) satisfies both properties 1 and 2 and can be selected as an anchor point. The dotted arrow and the solid arrow represent the gradient direction and edge direction of the pixel, respectively, and the value in the bracket represents the inner product energy value of the pixel. Reasonable selection of local anchor points can effectively filter out interfering edges and improve the efficiency of subsequent image processing.

[0081] The local anchor point as the starting point of edge tracking has two edge tracking directions, which are opposite and perpendicular to the gradient direction. Figure 2 As shown, the initial anchor point X has two edge tracking, and the image edge E(X) is obtained by tracking to the left. - ={X,A,B,C}, track to the right to get the image edge E(X) + = {X, D, E, F}, the image edge obtained is E(X) = E(X) + ∪E(X) - The gradient direction of the initial anchor point X determines the tracking direction of the starting point, and the edge direction determines the tracking direction of subsequent edge pixels.

[0082] The search range for the next edge pixel is as follows: Figure 3 As shown, based on the assumption that the image edge direction is continuous, assuming that the edge direction of the current image edge pixel is encoded as a, the direction of the search range of the next image edge pixel is encoded as {a, (a+1) & 0x7, (a-1) & 0x7}, with a total of 3 candidate edge pixel points.

[0083] The rules for selecting the next edge pixel from the candidate edge pixels need to consider the following factors:

[0084] 1) The pixel is a maximum point in its neighborhood;

[0085] 2) The inner product energy amplitude is the largest;

[0086] 3) The edge direction is the same as the previous edge pixel.

[0087] The above three factors are given the same weight, and the confidence of the candidate edge pixels is obtained through weighted calculation. The pixel with the largest confidence is the next edge pixel point, the edge direction is updated, and the next edge pixel is found.

[0088] If the next edge pixel cannot be obtained by the above method, the edge tracking in this direction is terminated and the tracking is performed along the other edge direction of the starting anchor point X.

[0089] Figure 4 Given the gradient direction of the pixel in the local image, Figure 5 The inner product energy value of the pixel and the search range of the edge pixel are given. Figure 4 and 5 , the process of single-pixel edge tracking can be described as:

[0090] (1) The edge direction of the initial pixel P0 depends on the gradient direction of the pixel. The gradient direction of P0 is coded as 7, so the search range for the next edge pixel is {6,7,0}. The pixels corresponding to the direction codes 6 and 0 do not meet the three elements of edge pixel screening proposed above, and the confidence is 0; the pixel corresponding to the direction code 7 is the maximum point in the neighborhood, and the inner product energy amplitude of this point is the largest, and the confidence is 2, so point P1 is the next edge pixel.

[0091] (2) The code of the P0-P1 edge direction is 7 (upper right direction), so the search range of the next edge pixel is {6,7,0}. The pixels corresponding to the direction codes 6 and 7 only meet factor 3 and the confidence is 1. The pixel corresponding to the direction code 0 meets all three factors and has a confidence of 3, which is the next edge pixel.

[0092] (3) Continue edge tracking by referring to the method described in steps (1) and (2). When the inner product energy amplitudes of the three "candidate" edge pixels (i.e., candidate edge pixel points) are all 0, jump to the initial pixel P0. Perform edge tracking along the other direction (direction code is 3). Combine the edge pixels selected in the two directions to finally obtain the complete image edge.

[0093] 2. Sub-pixel positioning strategy

[0094] The coordinates of each edge pixel point obtained by the image edge detection method are expressed in an integer form, which introduces truncation errors and causes a certain deviation between the detected image edge and the actual edge. By performing sub-pixel repositioning on all edge pixels, we strive to reduce the image edge tracking error and improve the edge detection accuracy.

[0095] Take an edge pixel p as the starting point of sub-pixel relocation, and the gradient relationship of its neighborhood is as follows: Figure 5 As shown in Figure 2. For edge pixels, the pixels in its neighborhood S(p) can be divided into pixels on the same edge (pixel q0) and pixels in the low texture area (pixels q1 and q2). Based on the assumption that the image edge has significant continuity mentioned above, pixels p and q0 have the same edge direction. The gradient direction of pixel q0 is the same as the vector Vertical, that is, G(q0)·(p-q0)=0. For pixels in low texture areas, the gradient value is almost 0, so it can be concluded that the pixels in the neighborhood S(p) of edge pixel p all satisfy the following equation:

[0096] G(q i )·(pq i )=0 (4)

[0097] Therefore, the process of relocating the edge pixels of the image can be described as: Under the premise of taking the minimum value, obtain the coordinates of the edge pixels after relocation The process of value.

[0098] The least squares method is used to solve the above equation. To prevent the solution process from not converging, it is necessary to constrain The value range of .

[0099] like Figure 6 As shown in , since the image edge is consistent with the gradient direction of point p, we can The search range is limited to the normal line of point p. Suppose the gradient value of edge pixel p(x0,y0) is (g x ,g y ), the straight line passing through the edge pixel is:

[0100] y=kx+b,k=-g y / g x ,b=y0-kx0 (5)

[0101] In the formula, k and b are the slope and constant intercept of the straight line respectively. If it is limited to the straight line l, then:

[0102]

[0103] Rewriting equation (5) into the form of Aβ=B, we can obtain A=(k,-1), B=-b, and β=(x p ,y p ) t is the parameter vector to be determined. The process of solving the parameter vector β is transformed into the process of solving the extreme value of g(x) under the constraint of the conditional function f(x) = 0. The least squares estimate that satisfies the linear constraints of the model is solved by the Lagrange multiplier method:

[0104]

[0105] Among them, λ represents the Lagrange multiplier. The auxiliary function equation is constructed as:

[0106] F(β,λ)=‖Y-Xβ‖ 2+2λ(Aβ-B)=(Y-Xβ)′(Y-Xβ)+2λ(Aβ-B) (8)

[0107] In the formula, A, X and B are parameter matrices, and the parameters parameter Set the partial derivatives of each component of parameter β in the auxiliary function equation to 0, solve the partial derivatives of each component of β in the auxiliary function F(β,λ), make the partial derivatives of each component of β equal to 0, simplify equation (8) to the system of parallel equations, and we can get:

[0108]

[0109] Among them, X ′ is the transposed matrix of matrix X, A ′ is the transposed matrix of matrix A;

[0110] (X ′ X) -1 Multiply the first equation in (9) on the left and solve equation (9) to obtain the constrained least squares solution It can be expressed as:

[0111]

[0112] in, is the estimated value of the Lagrangian operator;

[0113] Find unconstrained least squares solution

[0114]

[0115] Converting formula (11) to the form of Aβ=B, we can obtain:

[0116]

[0117] Since the rank of matrix A is 1, the parameter matrix A(X′X) in equation (12) is -1 A′ is a 1st order reversible matrix, then:

[0118]

[0119] Substituting equation (13) into equation (10), we finally get the coordinates of edge pixel p after sub-pixel relocation:

[0120]

[0121] To ensure the convergence of the sub-pixel positioning algorithm, the distance between the original pixel and the relocated pixel is solved, that is, Compare with the distance threshold τ. , it can be determined that the edge pixel relocation is invalid and the coordinate value of the original edge pixel is retained.

[0122] The image edge tracking method based on sub-pixel positioning of the present invention, wherein the single-pixel edge tracking strategy involved is based on the assumption that the image edge direction is continuous, and refers to the local extreme value and gradient direction of the edge pixel point to extract a curve set composed of the image edge pixels. The method of the present invention has a good effect of edge detection for images with blurred edges and low contrast. The extracted image edge does not have an excessive width, reduces the amount of redundant information, is smooth, has strong robustness, and simplifies the subsequent processing of the image.

[0123] The image edge tracking method based on sub-pixel positioning of the present invention uses the gradient direction as a constraint and refers to the grayscale values ​​of pixels in adjacent areas to perform sub-pixel repositioning of edge pixels in order to reduce the deviation between the extracted image edge and the actual edge.

[0124] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. An image edge tracking method based on sub-pixel positioning, characterized in that: The following steps are involved: Step 1: Select the pixel point with the largest inner product energy value among the adjacent pixels in the gradient direction and the same edge direction as the initial local anchor point; Step 2: Based on the assumption that the edge directions of the edge pixels of the image are continuous, determine the search range of the next edge pixel and obtain multiple candidate edge pixels; Step 3: Referring to the rules for selecting the next edge pixel point, the confidence of multiple candidate edge pixel points is obtained after weighted calculation, the candidate point with the largest confidence is selected as the next edge point, and the edge direction is updated; Step 4: Continue tracking by referring to the method of step 2 and step 3 until the inner product energy values ​​of all pixels within the search coding range are 0, and then switch to another edge direction of the initial pixel for tracking; Step 5: Combine the edge pixels obtained by tracking in two directions to obtain a complete image edge; Step 6: Using the edge pixel gradient direction as a constraint, perform sub-pixel relocation on each pixel obtained by edge tracking.

2. The image edge tracking method based on sub-pixel positioning according to claim 1, characterized in that: The rules for selecting the next edge pixel in step 3 include: 1) The pixel is a maximum point in its neighborhood; 2) The inner product energy amplitude is the largest; 3) The edge direction is the same as the previous edge pixel.

3. The image edge tracking method based on sub-pixel positioning according to claim 1, characterized in that: The number of candidate edge pixels in step 2 is 3.

4. The image edge tracking method based on sub-pixel positioning according to claim 1, characterized in that: Step 6 specifically includes: Take an edge pixel p as the starting point of sub-pixel relocation; All pixels in the neighborhood S(p) of edge pixel p satisfy the equation: G(q i )·(p-q i )=0 Reposition the image edge pixels p, Under the premise of taking the minimum value, obtain the coordinates of the edge pixel p after relocation value.

5. The image edge tracking method based on sub-pixel positioning according to claim 4, characterized in that: For the equation: G(q i )·(p-q i )=0 The solution process is: Suppose the gradient value of edge pixel p(x0,y0) is (g x ,g y ), the straight line passing through the edge pixel is: y=kx+b,k=-g y / g x ,b=y0-kx0 In the formula, k and b are the slope and constant intercept of the straight line respectively. If it is limited to the straight line l, then: The Lagrange multiplier method is used to solve the least squares estimate that satisfies the linear constraints of the model: Among them, λ represents the Lagrange multiplier; the auxiliary function equation is constructed as follows: F(β,λ)=‖Y-Xβ‖ 2 +2λ(Aβ-B)=(Y-Xβ)′(Y-Xβ)+2λ(Aβ-B) (8) Where A, X and B are parameter matrices. Set the partial derivatives of each component of parameter β in the auxiliary function equation to 0, solve the partial derivatives of each component of β in the auxiliary function F(β,λ), and make the partial derivatives of each component of β equal to 0, and we get: Where X′ is the transposed matrix of matrix X, and A′ is the transposed matrix of matrix A; the constrained least squares solution is It is expressed as: in, is the estimated value of the Lagrangian operator; Unconstrained least squares solution have to: Since the rank of matrix A is 1, the parameter matrix A(X′X) -1 A′ is a 1st order reversible matrix, then: Get the coordinates of edge pixel p after sub-pixel relocation 6. The image edge tracking method based on sub-pixel positioning according to claim 5, characterized in that: To ensure the convergence of the sub-pixel localization algorithm, the distance between the original pixel and the relocated pixel is calculated. Compared with the distance threshold τ, when , the edge pixel relocation is deemed invalid and the original edge pixel coordinate value is retained.

Images