One-way fuzzy image registration method based on linear features

Abstract

The present invention relates to the technical field of image recognition, specifically a one-way blurred image registration method based on line features, comprising the following steps: confirming the clear direction of the one-way blurred image, extracting line features with an included angle less than 45° with the clear direction and matching them; confirming the linear equation coefficients of the matched line features and the lengths of the line features; constructing a matrix equation and introducing a line segment length weight coefficient; solving the coefficient X based on the least squares method in step S3 to obtain a homography matrix; obtaining the homography matrix H by solving the coefficient X according to the least squares method, and using the homography matrix for image registration. Since the one-way blurred image has higher clarity in the non-blurred direction, by extracting line features along the other clear direction, the extraction has higher accuracy. Compared with the method of calculating the intersection points of the straight lines where the two line segments are located after extracting the line segments and then using the intersection points to calculate the homography matrix, it has higher numerical stability and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and specifically to a one-way blurred image registration method based on line features. Background Technique

[0002] Image registration is a key step in digital image processing and is widely used in fields such as remote sensing technology, computer vision, and medical image analysis. However, in practical applications, due to reasons such as camera defocus and high-speed movement, the image may have a one-way blur problem, resulting in insufficient accuracy of traditional feature point detection methods and affecting the registration effect.

[0003] Existing registration methods for one-way blurred images include image enhancement and using feature detection operators with scale invariance.

[0004] The method of image enhancement is to perform enhancement processing on the original image, to a certain extent, restore or improve the image quality of the blurred image, improve the accuracy of feature point detection, and thus improve the subsequent registration accuracy. However, this method is highly dependent on the specific situation of the image, and excessive sharpening of the image will introduce artifacts and affect the final registration accuracy.

[0005] Feature point algorithms such as SIFT and SURF have scale invariance and rotation invariance, have good robustness to partial blur, and can improve the registration accuracy to a certain extent. However, they are sensitive to severe blur, may cause inaccurate coordinates of feature points, and have extremely high computational costs.

[0006] In view of this, the present invention provides a one-way blurred image registration method based on line features. Summary of the Invention

[0007] The purpose of the present invention is to provide a one-way blurred image registration method based on line features to solve the problem of unsatisfactory registration effect caused by poor accuracy of feature point extraction of one-way blurred images.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] In the first aspect, the present invention provides a one-way blurred image registration method based on line features, including the following steps:

[0010] Step S1: Confirm the clear direction of the one-way blurred image, extract line features with an angle less than 45° with the clear direction and match them;

[0011] Step S2: Confirm the coefficients of the straight line equation of the matched line features and the lengths of the line features;

[0012] Step S3: Construct a matrix equation and introduce a line segment length weight coefficient;

[0013] Step S4: Solve the coefficient X based on the least squares method in Step S3 to obtain the homography matrix;

[0014] Step S5: Solve the coefficient X according to the least squares method to obtain the homography matrix H, and use the homography matrix for image registration.

[0015] As a preferred embodiment of the first aspect of the present invention, the confirmation logic of the clear direction of the unidirectional blurred image is as follows:

[0016] Use a line feature detection operator to perform line detection on the image to extract line features;

[0017] Statistically analyze the direction distribution of the extracted line features, and screen the directions that meet the conditions and are concentrated as the clear direction according to a preset frequency.

[0018] As a preferred embodiment of the first aspect of the present invention, the logic for extracting and matching the line features with an angle less than 45° from the clear direction is as follows:

[0019] By comparing the line features in different directions, screen out the line features with an angle less than 45° from the clear direction;

[0020] Match the line features in the blurred image and the reference image during the screening, and the number of matched line features is not less than 4 pairs.

[0021] As a preferred embodiment of the first aspect of the present invention, the inclination angle of the line segment has distinctiveness.

[0022] As a preferred embodiment of the first aspect of the present invention, extract the corresponding straight line equation coefficients A and B based on the point coordinates in the blurred image;

[0023] Extract the corresponding straight line equation coefficients C and D based on the point coordinates in the reference image;

[0024] Match the corresponding relationship of the point coordinates of the homography matrix between the blurred image and the reference image; the straight line equation coefficients of the corresponding matching line features between the blurred image and the reference image can be obtained.

[0025] As a preferred embodiment of the first aspect of the present invention, based on the straight line equation coefficients of the matching line features, convert the straight line equation coefficients into the line segment lengths with line segment length weight coefficients, so as to extract the number of corresponding matching line segments between the blurred image and the reference image.

[0026] As a preferred embodiment of the first aspect of the present invention, construct a matrix equation based on the number of matching line segments. The matrix equation includes matrix M and matrix N, and use the least squares method to solve the matrix equation coefficients corresponding to the matrix equation.

[0027] As a preferred embodiment of the first aspect of the present invention, the registration of the blurred image is realized based on the specific numerical values of the matrix equation coefficient X and the homography matrix H.

[0028] In a second aspect, the present invention provides an electronic device, comprising:

[0029] at least one processor; and,

[0030] a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect.

[0031] In a third aspect, the present invention provides a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the method described in the first aspect.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] For the unidirectional blurred image of the present invention, since the unidirectional blurred image has higher clarity in the non-blurred direction, by extracting the line features along the other clear direction, the extraction has higher accuracy. Compared with the method of calculating the intersection points of the line segments and then using the intersection points to calculate the homography matrix, it has higher numerical stability and robustness.

[0034] The included angle between the extracted line features and the clear direction of the image is small, that is, the included angle between the line features is small. When calculating the intersection points, due to the small included angle between the straight lines, the situation where the intersection points of the two straight lines are outside the image range or even at infinity will occur, affecting the numerical stability of the point coordinates, and realizing the high-precision registration problem of the unidirectional blurred image.

[0035] By introducing the length of the line segment as the weight coefficient of each line feature, the influence of the extraction accuracy of line features with different lengths on the final solution is fully considered. Generally speaking, the longer the line segment, the higher its reliability and the greater its contribution to the solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of the method for registering a unidirectional blurred image of the straight line feature of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "vertical", "upper", "lower", "horizontal", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention.

[0039] In the description of the present invention, it should also be noted that unless otherwise clearly specified and limited, the terms "set", "installed", "connected", "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0040] Embodiment 1

[0041] Please refer to Figure 1 , the present invention provides a technical solution: a one-way fuzzy image registration method based on line features, including the following steps:

[0042] Step S1: Confirm the clear direction of the fuzzy image, extract the line features with an angle less than 45° with the clear direction and match them.

[0043] Specifically, based on the combination of line feature extraction and gradient analysis, the clear direction is determined by statistically analyzing the direction distribution of line features.

[0044] The confirmation logic of the clear direction of the fuzzy image is as follows:

[0045] Use a line feature detection operator to perform line detection on the image to extract line features;

[0046] Statistically analyze the direction distribution of the extracted line features, and select the direction that meets the conditions and is concentrated as the clear direction according to the preset frequency.

[0047] The logic of extracting and matching the line features with an angle less than 45° with the clear direction is as follows:

[0048] By comparing the line features in different directions, select the line features with an angle less than 45° with the clear direction;

[0049] Match the line features in the fuzzy image and the reference image during the screening, and the number of matched line features is not less than 4 pairs.

[0050] It should be noted that in practical applications, due to reasons such as camera movement, the captured images have unidirectional blur, that is, blurred images. When registering the blurred images, one is the image to be registered and the other is the reference image. Image registration is achieved through image transformation, making the shooting perspectives and contents of the two images exactly the same. The blurred image is the image to be registered, and the reference image is the image that the image to be registered is desired to become after transformation.

[0051] In image registration, the inclination angles of line features should have a certain angular difference, which can cover different directions of the image and avoid completely parallel line features. Four lines are respectively found from the blurred image and the reference image so that the four lines can be matched; for example, if there is the same building in both images, then the edge lines of the building top in the two images can be regarded as a pair of matching line segments, that is, the edge line of the building top in the blurred image and the edge line of the building top in the reference image are matched. By reasonably screening and distributing line features, the accuracy and robustness of image registration can be effectively improved; therefore, the inclination angles of the line segments have distinctiveness.

[0052] After extracting the line features, screen the line features and remove those line features with overly close directions. For example, an angle threshold (such as 30° or 45°) can be set, and only those line features with an inclination angle difference greater than the threshold are retained.

[0053] When extracting line features, try to select line features with uniform distribution. For example, the image can be divided into multiple regions, and line features in different directions are extracted from each region to ensure the direction diversity of line features.

[0054] According to the geometric structure of the image, select line features that can reflect the main features of the image. For example, in a building image, horizontal and vertical line features can be selected; in a natural image, edge line features in different directions can be selected.

[0055] Step S2: Confirm the coefficients of the straight-line equation of the matched line features and the lengths of the line features;

[0056] Specifically, the confirmation logic of the coefficients of the straight-line equation of the line features is as follows:

[0057] Denote the point coordinates in the blurred image as (x, y), the equation of the feature line segment as Ax + By + 1 = 0, the point coordinates in the reference image as (u, v), and the corresponding line segment equation as Cu + Dv + 1 = 0, that is: the coefficients A and B of the straight-line equation corresponding to the blurred image; the coefficients C and D of the straight-line equation corresponding to the reference image;

[0058] The homography matrix H for the transformation between the two images is denoted as:

[0059]

[0060] The corresponding relationship of the point coordinates in the two figures can be expressed as:

[0061]

[0062] In the above formula, λ is the scaling factor. According to the above formula, we can get:

[0063]

[0064] Substitute u and v into Cu + Dv + 1 = 0, and we can get:

[0065]

[0066] x and y themselves satisfy Ax + By + 1 = 0. Comparing with the above formula, we can get:

[0067]

[0068] Rearranging the above formula, we can get the following two equations:

[0069]

[0070] In the above formula, A, B, C, and D are the coefficients of the straight-line equation of a set of matching line features.

[0071] Step S3: Introduce the line segment length weight coefficient and construct a matrix equation;

[0072] Considering that the lengths of the matching line features are different, a line segment length weight coefficient is introduced into the system of equations. Denote the line segment length of the line segment in the reference image as L. Then the above two equations can be written as:

[0073]

[0074] Furthermore, after obtaining N pairs of matching line segments, construct a matrix equation, including matrix M and matrix N. The formula is;

[0075]

[0076] .

[0077] Step S4: Solve the coefficient X based on the least squares method in Step S3 to obtain the homography matrix.

[0078] Specifically, use the least squares method to solve the equation MX = N, and X can be expressed as .

[0079] Step S5: According to the least squares method, solve the coefficient X to obtain the homography matrix H, and use the homography matrix for image registration.

[0080] Specifically, according to the obtained least-squares solution of X, the specific values of the homography matrix H are obtained, enabling the registration of the blurred image. After correction, the blurred image can be aligned with the reference image. The key to registration is to obtain the homography matrix H, that is, the values of a1 to a8.

[0081] Embodiment 2

[0082] The unstated parts of this embodiment are the same as those in Embodiment 1. An electronic device shown in this embodiment includes:

[0083] At least one processor; and,

[0084] A memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the one-way blurred image registration method based on line features described in the first aspect.

[0085] This electronic device may vary significantly due to configuration or performance differences, and can include one or more processors (Central Processing Units, CPUs) and one or more memories. Among them, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the one-way blurred image registration method based on line features provided in the above various method embodiments.

[0086] This electronic device may also include other components for implementing device functions. For example, this electronic device may also have components such as wired or wireless network interfaces and input / output interfaces for input / output. Details are not described in this application embodiment.

[0087] Embodiment 3

[0088] The unstated parts of this embodiment are the same as those in Embodiment 1. This embodiment also provides a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to execute the one-way blurred image registration method based on line features.

[0089] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present invention can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0090] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0091] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A one-way fuzzy image registration method based on straight line features, characterized by: The following steps are involved: Step S1: confirm the clear direction of the unidirectional blurred image, extract the line features with an angle less than 45° with the clear direction, match the line features in the filtered blurred image and the reference image, ensure that the matching line features are line features of the same scene in the two images, and the number of selected matching line features is not less than 4 pairs; Step S2: confirming the linear equation coefficient of the matched line feature and the length of the line feature; Step S3: construct a matrix equation and introduce a line segment length weight coefficient; Step S4: Solve the coefficient X based on the least square method in step S3 to obtain a homography matrix; Step S5: Solve the coefficient X according to the least square method to obtain the homography matrix H, and use the homography matrix to perform image registration.

2. The one-way fuzzy image registration method based on straight line features according to claim 1 is characterized in that: The confirmation logic of the clear direction of the one-way blurred image is: Use the line feature detection operator to perform line detection on the image to extract line features; The direction distribution of the extracted line features is statistically analyzed, and the directions that meet the conditions and are concentrated are selected as clear directions according to the preset frequency.

3. The one-way fuzzy image registration method based on straight line features according to claim 2 is characterized in that: The inclination angle of the line segment is discriminative.

4. The one-way fuzzy image registration method based on straight line features according to claim 3 is characterized in that: Extract the corresponding straight line equation coefficients A and B based on the coordinates of the points on the straight line in the blurred image; Extract the corresponding straight line equation coefficients C and D based on the coordinates of the points on the straight line in the reference image; By matching the line features extracted from the blurred image and the reference image, the linear equation coefficients of the corresponding matching line features between the blurred image and the reference image can be obtained.

5. The one-way fuzzy image registration method based on straight line features according to claim 4 is characterized in that: Based on the line equation coefficients of the matching line features, the line equation coefficients are converted into line segment lengths with line segment length weight coefficients, thereby extracting the number of corresponding matching line segments between the blurred image and the reference image.

6. The one-way fuzzy image registration method based on straight line features according to claim 5 is characterized in that: A matrix equation is constructed based on the number of matching line segments, wherein the matrix equation includes a matrix M and a matrix N, and matrix equation coefficients corresponding to the matrix equation are solved using a least squares method.

7. The one-way fuzzy image registration method based on straight line features according to claim 6 is characterized in that: The registration of blurred images is achieved based on the specific values ​​of the matrix equation coefficients X and the homography matrix H.

8. An electronic device, characterized in that: include: at least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the one-way fuzzy image registration method based on straight line features as described in any one of claims 1-7.

9. A computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the one-way fuzzy image registration method based on straight line features as described in any one of claims 1 to 7.

Images