Unidirectional blurred image registration method based on linear features

Abstract

The invention relates to the technical field of image recognition, in particular to a one-way blurred image registration method based on linear features, which comprises the following steps of: confirming a clear direction of a one-way blurred image, and extracting and matching linear features which form an included angle of less than 45 degrees with the clear direction; confirming a linear equation coefficient of the matched line feature and the length of the line feature; constructing a matrix equation, and introducing a line segment length weight coefficient; solving the coefficient X based on the least square method in the step S3 to obtain a homography matrix; and solving the coefficient X according to a least square method to obtain a homography matrix H, and performing image registration by using the homography matrix. Due to the fact that the definition of the one-way blurred image is high in the non-blurring direction, by extracting the line features in the other clear direction, the method with high precision is extracted, and compared with the method that after the line segments are extracted, the intersection points of the straight lines where every two line segments are located are calculated, and then the homography matrix is calculated through the intersection points, the method has higher stability and robustness in numerical value.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a one-way fuzzy image registration method based on straight line features. Background Art

[0002] Image registration is a key step in digital image processing and is widely used in remote sensing technology, computer vision, and medical image analysis. However, in practical applications, due to camera defocus, high-speed motion, and other reasons, images may have unidirectional blur, resulting in insufficient accuracy of traditional feature point detection methods, affecting the registration effect.

[0003] Existing registration methods for solving unidirectional blurred images include image enhancement, using feature detection operators with scale invariance, etc.

[0004] The image enhancement method is to enhance the original image, restore or improve the image quality of the blurred image to a certain extent, improve the accuracy of feature point detection, and thus improve the subsequent registration accuracy. However, this method is very dependent on the specific conditions of the image, and excessive sharpening of the image will introduce artifacts, affecting the final registration accuracy.

[0005] Feature point algorithms such as SIFT and SURF are scale invariant and rotation invariant, and have good robustness to partial blur. They can improve the accuracy of registration to a certain extent. However, they are sensitive to severe blur, which may lead to inaccurate coordinates of feature points and have extremely high computational costs.

[0006] In view of this, the present invention provides a unidirectional fuzzy image registration method based on straight line features. Summary of the invention

[0007] The purpose of the present invention is to provide a one-way fuzzy image registration method based on straight line features, so as to solve the problem of unsatisfactory registration effect caused by poor extraction accuracy of feature points of one-way fuzzy images.

[0008] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a unidirectional fuzzy image registration method based on straight line features, comprising the following steps: Step S1: confirm the clear direction of the unidirectional blurred image, extract and match the line features whose angle with the clear direction is less than 45°; Step S2: confirming the linear equation coefficient of the matched line feature and the length of the line feature; Step S3: construct a matrix equation and introduce a line segment length weight coefficient; Step S4: Solve the coefficient X based on the least square method in step S3 to obtain a homography matrix; Step S5: Solve the coefficient X according to the least square method to obtain the homography matrix H, and use the homography matrix to perform image registration.

[0009] As a preferred implementation of the first aspect of the present invention, the confirmation logic of the clear direction of the unidirectional blurred image is: Use the line feature detection operator to perform line detection on the image to extract line features; The direction distribution of the extracted line features is statistically analyzed, and the directions that meet the conditions and are concentrated are selected as clear directions according to the preset frequency.

[0010] As a preferred implementation of the first aspect of the present invention, the logic for extracting and matching line features with an angle less than 45° with the clear direction is: By comparing line features in different directions, the line features with an angle less than 45° with the clear direction are selected; The line features in the blurred image being screened and the reference image are matched, and the number of matched line features is not less than 4 pairs.

[0011] As a preferred implementation of the first aspect of the present invention, the inclination angle of the line segment has a degree of distinction.

[0012] As a preferred implementation of the first aspect of the present invention, corresponding straight line equation coefficients A and B are extracted based on the point coordinates in the blurred image; Extract the corresponding straight line equation coefficients C and D based on the point coordinates in the reference image; The correspondence between the coordinates of the homography matrix matching points converted between the blurred image and the reference image is obtained; the linear equation coefficients of the corresponding matching line features between the blurred image and the reference image can be obtained.

[0013] As a preferred implementation of the first aspect of the present invention, the straight line equation coefficient is converted into a line segment length with a line segment length weight coefficient based on the matching line feature, thereby extracting the number of corresponding matching line segments between the blurred image and the reference image.

[0014] As a preferred implementation of the first aspect of the present invention, a matrix equation is constructed based on the number of matching line segments, the matrix equation includes a matrix M and a matrix N, and the matrix equation coefficients corresponding to the matrix equation are solved using the least squares method.

[0015] As a preferred implementation of the first aspect of the present invention, the registration of blurred images is achieved based on specific values ​​of matrix equation coefficients X and homography matrix H.

[0016] In a second aspect, the present invention provides an electronic device, comprising: at least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0017] In a third aspect, the present invention provides a computer-readable storage medium storing instructions, which, when executed on a computer, causes the computer to execute the method described in the first aspect.

[0018] Compared with the prior art, the present invention has the following beneficial effects: For one-way blurred images, the present invention has higher stability and robustness in numerical terms because the one-way blurred images have higher clarity in the non-blurred direction and the line features along another clear direction are extracted with higher accuracy. After the line segments are extracted, the intersection points of the line segments are calculated and the homography matrix is ​​calculated using the intersection points.

[0019] The angle between the extracted line features and the clear direction of the image is small, that is, the angle between the line features is small. When calculating the intersection point, due to the small angle between the straight lines, the intersection point of the two straight lines may be outside the image range or even at infinity, affecting the numerical stability of the point coordinates, thereby achieving high-precision registration of one-way blurred images.

[0020] By introducing the length of the line segment as the weight coefficient of each line feature, the influence of the extraction accuracy of line features of different lengths on the final solution is fully considered. Generally speaking, the longer the line segment, the higher its reliability and the greater its contribution to the solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the one-way fuzzy image registration method of straight line features of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "vertical", "up", "down", "horizontal", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0024] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] Example 1

[0026] See also Figure 1 The present invention provides a technical solution: a one-way fuzzy image registration method based on straight line features, comprising the following steps: Step S1: confirm the clear direction of the blurred image, extract and match line features whose angle with the clear direction is less than 45°.

[0027] Specifically, based on the combination of line feature extraction and gradient analysis, the clear direction is determined by statistically analyzing the direction distribution of line features.

[0028] The confirmation logic of the clear direction of the blurred image is: Use the line feature detection operator to perform line detection on the image to extract line features; The direction distribution of the extracted line features is statistically analyzed, and the directions that meet the conditions and are concentrated are selected as clear directions according to the preset frequency.

[0029] The logic for extracting and matching line features with an angle less than 45° to the clear direction is: By comparing line features in different directions, the line features with an angle less than 45° with the clear direction are selected; The line features in the blurred image being screened and the reference image are matched, and the number of matched line features is not less than 4 pairs.

[0030] It should be noted that in practical applications, due to camera motion and other reasons, the captured image is unidirectionally blurred, that is, a blurred image. When registering blurred images, one is the image to be registered and the other is the reference image. Image registration is achieved through image transformation, so that the shooting angle and content of the two images are exactly the same. The blurred image is the image to be registered, and the reference image is the image that the image to be registered becomes after transformation.

[0031] In image registration, the inclination angles of line features should have a certain angle difference, be able to cover different directions of the image, and avoid completely parallel line features. Find 4 lines from the blurred image and the reference image respectively, so that the 4 lines can be matched; for example, if both images have the same building, then the edge lines of the roofs in the two images can be regarded as a pair of matching line segments, that is, the edge lines of the roofs of the blurred image and the roofs of the reference image are matched. By reasonably selecting and distributing line features, the accuracy and robustness of image registration can be effectively improved; therefore, the inclination angles of the line segments have discrimination.

[0032] After extracting the line features, the line features are screened to remove those line features that are too close in direction. For example, an angle threshold (such as 30° or 45°) can be set to only retain those line features whose inclination angle difference is greater than the threshold.

[0033] When extracting line features, try to select line features that are evenly distributed. For example, you can divide the image into multiple regions and extract line features in different directions from each region to ensure the direction diversity of the line features.

[0034] According to the geometric structure of the image, select line features that can reflect the main features of the image. For example, in building images, you can select horizontal and vertical line features; in natural images, you can select edge line features in different directions.

[0035] Step S2: confirming the linear equation coefficient of the matched line feature and the length of the line feature; Specifically, the confirmation logic of the linear equation coefficients of the line feature is: The coordinates of the midpoint of the blurred image are (x, y), the equation of the feature line segment is Ax + By + 1 = 0, the coordinates of the midpoint of the reference image are (u, v), and the corresponding line segment equation is Cu + Dv + 1 = 0, that is: the linear equation coefficients A and B corresponding to the blurred image; the linear equation coefficients C and D corresponding to the reference image; The homography matrix H for the transformation between two images is recorded as:

[0036] Then the corresponding relationship between the coordinates of the points in the two images can be expressed as:

[0037] In the above formula, λ is the scaling factor. According to the above formula, we can get:

[0038] Substituting u and v into Cu + Dv + 1 = 0, we get:

[0039] x, y itself satisfies Ax + By + 1 = 0. Comparing with the above formula, we can get:

[0040] Arranging the above formula can get the following two equations:

[0041] In the above formula, A, B, C, and D are a set of straight line equation coefficients for matching line features.

[0042] Step S3: introducing the line segment length weight coefficient and constructing the matrix equation; Considering that the lengths of the matched line features are different, the line segment length weight coefficient is introduced into the equation group. Let the line segment length in the reference image be L, then the above two equations can be written as:

[0043] Furthermore, after obtaining N pairs of matching line segments, a matrix equation is constructed, including matrix M and matrix N, and the formula is:

[0044] .

[0045] Step S4: Solve the coefficient X based on the least square method in step S3 to obtain the homography matrix.

[0046] Specifically, using the least squares method to solve the equation MX=N, X can be expressed as .

[0047] Step S5: Solve the coefficient X according to the least square method to obtain the homography matrix H, and use the homography matrix to perform image registration.

[0048] Specifically, according to the least square solution of X, that is, the specific value of the homography matrix H, the registration of the blurred image can be realized, and the blurred image can be aligned with the reference image after correction. The key to registration is to obtain the homography matrix H, that is, the value of a1~a8.

[0049] Example 2

[0050] The parts not described in this embodiment are as in Embodiment 1. This embodiment shows an electronic device, including: at least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the one-way fuzzy image registration method based on straight line features described in the first aspect.

[0051] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memories store at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the one-way fuzzy image registration method based on straight line features provided in the above-mentioned various method embodiments.

[0052] The electronic device may also include other components for realizing the functions of the device, for example, the electronic device may also have components such as a wired or wireless network interface and an input / output interface for input and output. The embodiments of the present application will not be described in detail here.

[0053] Example 3

[0054] The parts not described in this embodiment are as in Embodiment 1. This embodiment further provides a computer-readable storage medium storing instructions. When the instructions are executed on a computer, the computer executes the one-way fuzzy image registration method based on straight line features.

[0055] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0056] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0057] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A one-way fuzzy image registration method based on straight line features, characterized by: The following steps are involved: Step S1: confirm the clear direction of the unidirectional blurred image, extract and match the line features whose angle with the clear direction is less than 45°; Step S2: confirming the linear equation coefficient of the matched line feature and the length of the line feature; Step S3: construct a matrix equation and introduce a line segment length weight coefficient; Step S4: Solve the coefficient X based on the least square method in step S3 to obtain a homography matrix; Step S5: Solve the coefficient X according to the least square method to obtain the homography matrix H, and use the homography matrix to perform image registration.

2. The one-way fuzzy image registration method based on straight line features according to claim 1 is characterized in that: The confirmation logic of the clear direction of the one-way blurred image is: Use the line feature detection operator to perform line detection on the image to extract line features; The direction distribution of the extracted line features is statistically analyzed, and the directions that meet the conditions and are concentrated are selected as clear directions according to the preset frequency.

3. The one-way fuzzy image registration method based on straight line features according to claim 2 is characterized in that: The logic for extracting and matching line features with an angle less than 45° to the clear direction is: By comparing line features in different directions, the line features with an angle less than 45° with the clear direction are selected; The line features in the blurred image being screened and the reference image are matched, and the number of matched line features is not less than 4 pairs.

4. The one-way fuzzy image registration method based on straight line features according to claim 3 is characterized in that: The inclination angle of the line segment is discriminative.

5. The one-way fuzzy image registration method based on straight line features according to claim 4 is characterized in that: Extract the corresponding straight line equation coefficients A and B based on the coordinates of the points on the straight line in the blurred image; Extract the corresponding straight line equation coefficients C and D based on the coordinates of the points on the straight line in the reference image; By matching the line features extracted from the blurred image and the reference image, the linear equation coefficients of the corresponding matching line features between the blurred image and the reference image can be obtained.

6. The one-way fuzzy image registration method based on straight line features according to claim 5 is characterized in that: Based on the line equation coefficients of the matching line features, the line equation coefficients are converted into line segment lengths with line segment length weight coefficients, thereby extracting the number of corresponding matching line segments between the blurred image and the reference image.

7. The one-way fuzzy image registration method based on straight line features according to claim 6 is characterized in that: A matrix equation is constructed based on the number of matching line segments, wherein the matrix equation includes a matrix M and a matrix N, and matrix equation coefficients corresponding to the matrix equation are solved using a least squares method.

8. The one-way fuzzy image registration method based on straight line features according to claim 7 is characterized in that: The registration of blurred images is achieved based on the specific values ​​of the matrix equation coefficients X and the homography matrix H.

9. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the one-way fuzzy image registration method based on straight line features as described in any one of claims 1-8.

10. A computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the one-way fuzzy image registration method based on straight line features as described in any one of claims 1 to 8.

Images