Image registration method, electronic device and computer readable storage medium

By combining sliding matching and feature point matching methods in image registration, the problem of error-prone image matching in the prior art is solved, and the accuracy and accuracy of image registration are improved, especially when the features are not significant or repeated, it is effectively improved. The accuracy of feature point pairing is achieved.

CN112435283BActive Publication Date: 2024-12-06ZHEJIANG DAHUA TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202011217920.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-04
Publication Date
2024-12-06
Estimated Expiration
2040-11-04

AI Technical Summary

Technical Problem

Existing image registration methods are prone to matching errors when the image features are close, the image is blurred, or the feature points are concentrated, resulting in incorrect registration results.

Method used

By acquiring the first image and the second image to be registered, first slide on the second image to find the matching area with the highest correlation and size related to the first image, perform feature point extraction, and determine whether it is in the preset area of ​​the feature point in the feature point There are matching feature points, and the transformation relationship is calculated and registered.

Benefits of technology

The accuracy and accuracy of image matching are improved, especially when the image features are not significant or repeated, effectively improving the accuracy of feature point pairing, further improving the registration accuracy and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112435283B_ABST
    Figure CN112435283B_ABST
Patent Text Reader

Abstract

The present invention discloses an image registration method, an electronic device, and a computer-readable storage medium. The image registration method includes: obtaining a first image and a second image to be registered; wherein the size of the first image is smaller than the size of the second image; sliding the first image on the second image, and finding a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image; in the matching area, extracting feature points of the first image and the second image respectively, and obtaining a first feature point corresponding to the first image and a second feature point corresponding to the second image; in response to the presence of a matching second feature point within a preset area of ​​the first feature point, determining that the match is successful; using the position information of the successfully matched feature point to calculate the transformation relationship between the first image and the second image; and using the transformation relationship to register the pixel points of the first image to the second image. In the above manner, the present invention can achieve accurate image registration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image registration, and in particular to an image registration method, an electronic device and a computer-readable storage medium. Background Art

[0002] Image registration is a typical problem in the field of image processing research. Its purpose is to compare or fuse images of the same object acquired under different conditions (for example, different acquisition equipment, time, shooting angle). Image registration in the usual sense means that for two images, one image is mapped to the plane where the other image is located by finding a transformation, so that the points corresponding to the same position in the space in the two images correspond one to one, thereby achieving the purpose of information fusion. Image registration has been widely used in image processing as the most important technology in image stitching, image fusion, stereo vision, 3D reconstruction, depth estimation and image measurement.

[0003] Existing image registration methods are prone to mismatching when image features are close, the image is blurred, or the feature points are very concentrated, resulting in errors in the final calculated registration results. Summary of the invention

[0004] The present invention provides an image registration method, an electronic device and a computer-readable storage medium to solve the problem of easy error in image matching in the prior art.

[0005] To solve the above technical problems, the present invention provides an image registration method, comprising: acquiring a first image and a second image to be registered; wherein the size of the first image is smaller than the size of the second image; sliding the first image on the second image, and finding a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image; extracting feature points of the first image and the second image in the matching area respectively, to obtain first feature points corresponding to the first image and second feature points corresponding to the second image; determining that the match is successful in response to the presence of a matching second feature point within a preset area of ​​the first feature point; calculating a transformation relationship between the first image and the second image using the position information of the successfully matched feature points; and registering the first image with the second image using the transformation relationship.

[0006] Among them, the step of sliding the first image on the second image and finding a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image includes: sliding the first image on the second image to obtain multiple reference areas on the second image that have the same size as the first image; based on the number of pixels, pixel coordinates, pixel grayscale value, pixel average and pixel standard deviation of the first image and each reference area, using normalized cross-correlation to calculate the correlation between the first image and each reference area; and taking the reference area with the highest correlation as the matching area.

[0007] Before the step of sliding the first image on the second image to obtain a plurality of reference areas of the second image related to the size of the first image, the method includes: reducing the first image and the second image by a preset multiple and graying the reduced first image and the second image.

[0008] Among them, the step of sliding the first image on the second image to obtain multiple reference areas on the second image related to the size of the first image specifically includes: performing jump sliding comparison on the first image on the second image to obtain multiple reference areas on the second image related to the size of the first image.

[0009] Among them, within the matching area, the step of extracting feature points of the first image and the second image respectively to obtain first feature points corresponding to the first image and second feature points corresponding to the second image includes: extracting feature points of the first image and the second image respectively by an accelerated robust feature algorithm to obtain first feature points corresponding to the first image and second feature points corresponding to the matching area of ​​the second image.

[0010] Among them, in response to the existence of a matching second feature point within the preset area range of the first feature point, the step of determining that the match is successful includes: judging whether there is a corresponding second feature point within the preset area range of each first feature point; if so, calculating whether the square difference between the feature vector of the first feature point and the feature vector of the second feature point satisfies a first preset condition; if so, the match is successful.

[0011] Among them, the step of determining whether there is a second feature point within the preset area range of the first feature point also includes: obtaining the number of second feature points corresponding to the preset area range of the first feature point; if the number is 1, executing the step of calculating whether the square difference between the first feature point and the second feature point meets the first preset condition; if the number is greater than and equal to 2, determining whether the quotient between a group of first feature points and the second feature point with the smallest square difference of the feature vector and a group of first feature points and the second feature point with the second smallest square difference of the feature vector meet the second preset condition, and if so, the match is successful.

[0012] Among them, the step of calculating the transformation relationship between the first image and the second image using the position information of the successfully matched feature points includes: dividing the first image and the matching area into multiple equal areas; using the position information of a group of point pairs with the smallest square difference in the feature vector of the successfully matched feature points in each equal area to calculate the transformation relationship.

[0013] The number of the multiple equal areas is greater than or equal to 4.

[0014] The step of registering the pixels of the first image into the second image using the transformation relationship includes: registering the target object of the first image into the second image using the transformation relationship to achieve registration of the first image and the second image.

[0015] To solve the above technical problems, the present invention further provides an electronic device, which includes: a memory and a processor coupled to each other, and the processor is used to execute program instructions stored in the memory to implement any of the above image registration methods.

[0016] In order to solve the above technical problems, the present invention further provides a computer-readable storage medium, which stores program data, and the program data can be executed to implement any of the above-mentioned image registration methods.

[0017] The beneficial effect of the present invention is as follows: Different from the prior art, the image registration method of the present invention performs sliding matching on the first image on the second image based on the information of the first image to obtain the matching area on the second image with the highest correlation with the first image, thereby firstly performing rough matching on the first image and the second image, limiting the matching area, and improving the matching accuracy. Feature points are then extracted from the first image and the matching area respectively to obtain the first feature point and the second feature point, and with the first feature point as the center, it is determined whether there is a matching second feature point within the preset area of ​​the first feature point. If so, it is determined that the feature point pair is successfully matched, and finally the transformation relationship between the first image and the second image is calculated based on the successfully matched feature point pair, and the first image and the second image are registered using the transformation relationship. This embodiment combines sliding matching and feature point matching methods to improve the precision and accuracy of image matching, and then uses the feature point matching method with area restrictions, which can effectively improve the accuracy of feature point pairing when the image features are not significant or repeated, and further improve the registration precision and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of an embodiment of an image registration method provided by the present invention;

[0019] Figure 2 is a flow chart of another embodiment of the image registration method provided by the present invention;

[0020] Figure 3 is a schematic diagram of the sliding comparison in step S23;

[0021] Figure 4 Schematic diagram of feature point extraction of the accelerated robust feature algorithm in this embodiment;

[0022] Figure 5 is a structural schematic diagram of an embodiment of an electronic device provided by the present invention;

[0023] Figure 6 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1 , Figure 1 It is a flow chart of an embodiment of an image registration method provided by the present invention.

[0026] Step S11: acquiring a first image and a second image to be registered; wherein the size of the first image is smaller than the size of the second image.

[0027] The first image and the second image to be registered are obtained. In a specific application scenario, the first image and the second image have the same target object, but the angles or reference systems of the target objects of the first image and the second image are different. In a specific application scenario, the first image may be a front view of a water cup, and the second image may be an oblique view of the water cup, and image registration is performed on the front view and the oblique view of the water cup.

[0028] In a specific application scenario, the size of the target object in the first image is the same as that in the second image. If the size of the target object in the first image is different from that in the second image, the size of the first image or the second image is adjusted so that the size of the target object in the first image is the same as that in the second image. In this embodiment, when the size of the target object in the first image is the same as that in the second image, the size of the first image is smaller than that of the second image.

[0029] Step S12: Slide the first image on the second image, and find a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image.

[0030] Slide the smaller first image on the second image to find the matching area in the second image that has the highest correlation with the first image and is related to the size of the first image. In a specific application scenario, the first image can be slid horizontally on the second image for multiple rows to find the matching area. In a specific application scenario, the first image can be slid vertically on the second image for multiple columns to find the matching area. In a specific application scenario, the first image can also be slid in a circle on the second image for matching. The specific sliding matching method is not limited here.

[0031] During the sliding process, the first image is used as a reference to find a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image. The matching area has the highest correlation with the first image and has the same size to ensure that the target object in the matching area and the target object in the first image can accurately correspond.

[0032] Step S13: extracting feature points from the first image and the second image in the matching area respectively, to obtain first feature points corresponding to the first image and second feature points corresponding to the second image.

[0033] Feature points are extracted from the first image to obtain first feature points corresponding to the first image. Feature points are also extracted from the matching area of ​​the second image to obtain second feature points corresponding to the matching area in the second image.

[0034] In a specific application scenario, feature points of the first image and the second image in the matching area can be extracted by Sift (Scale Invariant Feature Transform) feature point extraction algorithm to obtain first feature points and second feature points. In a specific application scenario, feature points of the first image and the second image in the matching area can be extracted by Harris feature point extraction algorithm to obtain first feature points and second feature points. In a specific application scenario, feature points of the first image and the second image in the matching area can also be extracted by SURF (Speeded Up Robust Features) feature point extraction algorithm to obtain first feature points and second feature points. The feature point extraction algorithm is not limited here.

[0035] Step S14: In response to the presence of a matching second feature point within the preset area of ​​the first feature point, determining that the match is successful.

[0036] The first feature points of the first image are matched with the second feature points of the matching area of ​​the second image. When the matching is completed, it is determined whether there is a successfully matched second feature point within the preset area of ​​the first feature point. If there is a matched second feature point within the preset area of ​​the first feature point, the first feature point is successfully matched with the second feature point.

[0037] In this step, the second feature point is limited by setting a preset area range, thereby reducing the problem of mismatching when feature point matching is prone to occur when there are repeated image features in the image, thereby improving the accuracy of image registration.

[0038] Step S15: Calculate the transformation relationship between the first image and the second image using the position information of the successfully matched feature points.

[0039] When the first feature point on the first image is successfully matched with the second feature point in the matching area of ​​the second image, the transformation relationship between the first image and the second image is calculated using the position information of the feature point pair of the successfully matched first feature point and the second feature point. In a specific application scenario, the position information of the feature point pair may include the coordinate information of each feature point in the feature point pair to calculate the transformation relationship between the first image and the second image.

[0040] Step S16: registering the first image and the second image using the transformation relationship.

[0041] In a specific application scenario, any pixel point on the first image can be transformed and registered to the second image based on the transformation relationship, so as to achieve image registration of each pixel point of the first image with the corresponding pixel point of the second image. In a specific application scenario, any pixel point in the matching area of ​​the second image can also be transformed and registered to the first image based on the transformation relationship, so as to achieve image registration of each pixel point of the first image with the corresponding pixel point of the second image. This is not limited here.

[0042] Through the above scheme, the image registration method of this embodiment slides the first image on the second image based on the information of the first image to obtain the matching area on the second image with the highest correlation with the first image, so as to first roughly match the first image and the second image, limit the matching area, and improve the matching accuracy. Feature points are then extracted from the first image and the matching area respectively to obtain the first feature point and the second feature point, and with the first feature point as the center, it is determined whether there is a matching second feature point within the preset area of ​​the first feature point. If so, it is determined that the feature point pair is successfully matched, and finally the transformation relationship between the first image and the second image is calculated based on the successfully matched feature point pair, and the first image and the second image are registered using the transformation relationship. This embodiment uses a feature point matching method with area restrictions, which can effectively improve the accuracy of feature point pairing when the image features are not significant or repeated, and further improve the registration accuracy.

[0043] See also Figure 2 , Figure 2 It is a flow chart of another embodiment of the image registration method provided by the present invention.

[0044] Step S21: acquiring a first image and a second image to be registered; wherein the size of the first image is smaller than the size of the second image.

[0045] A first image and a second image to be registered are obtained, wherein the first image and the second image contain the same target object. In a specific application scenario, the same target object in the two images can be registered and overlapped. The target object may include an object, a background, a scene, and the like.

[0046] In a specific application scenario, when the size of the target object in the first image is the same as that in the second image, the size of the first image is compared with that of the second image, wherein the size of the first image is smaller than that of the second image. In a specific application scenario, when the size of the target object in the first image is different from that in the second image, the size of the first image or the second image is adjusted so that the size of the target object in the first image is the same as that in the second image. Among them, when the size of the target object in the first image is the same as that in the second image, the size of the first image is smaller than that of the second image. In the above manner, this embodiment can achieve accurate registration of images with different resolutions.

[0047] Step S22: reducing the first image and the second image by a preset multiple, and performing grayscale processing on the reduced first image and the second image.

[0048] In a specific application scenario, in order to improve the accuracy of subsequent image registration, the first image and the second image may be preprocessed before sliding registration is performed on the first image and the second image. Specifically, the preprocessing may include adjusting the image size of the first image or the second image to a preset size (e.g., 256*256). Alternatively, the preprocessing may also include normalizing the image intensity of the first image and the second image to a preset range (e.g., a range of 0 to 1). This facilitates feature comparison between the first image and the second image.

[0049] In a specific application scenario, the first image and the second image can be reduced by a preset multiple to simplify the image features of the first image and the second image, thereby improving the comparison efficiency of subsequent sliding comparison. The preset multiple can be 5 times, 10 times, 15 times, etc. The specific multiple can be set based on the image complexity of the first image and the second image in the actual registration, and is not limited here.

[0050] In a specific application scenario, the first image and the second image after the reduction processing can be grayed. Among them, the color image is composed of three different components, which is a three-channel image. When processing a color image, it is often necessary to process the three channels in sequence, which will consume a lot of time. Therefore, in order to increase the processing and comparison speed of the first image and the second image, it is necessary to reduce the amount of data required to be processed. Therefore, in this step, the color image of RGB three-channel data is converted into a grayscale image of single-channel data through the grayscale processing of the image, so as to reduce the amount of data for sliding comparison and improve the subsequent comparison efficiency of the first image and the second image.

[0051] Step S23: performing a jump sliding comparison on the first image on the second image to obtain a plurality of reference areas on the second image having the same size as the first image.

[0052] In a specific application scenario, a first image with a smaller size is used as a template, and a sliding comparison is performed on a second image with a larger size to find a matching area on the second image that is closest to the template.

[0053] See also Figure 3 , Figure 3 It is a schematic diagram of the sliding comparison in step S23.

[0054] The first image 01 with a smaller size is compared by translation and sliding on the second image 02 with a larger size, so as to obtain a plurality of reference areas 03 related to the first image 01 on the second image 02. The size of the reference area 03 is the same as that of the first image 01.

[0055] In this embodiment, the first image 01 is translated horizontally in multiple rows to perform sliding comparison on the second image 02 to obtain multiple reference areas 03 arranged in parallel. In other embodiments, the first image 01 can be translated and compared with other sliding rules, which is not limited here.

[0056] In a specific application scenario, the first image 01 can be compared with the second image 02 in a jump-sliding manner to obtain a plurality of reference areas 03 arranged at intervals on the second image 02, so as to reduce the amount of data for the sliding comparison and improve the efficiency of template matching.

[0057] Step S24: Based on the number of pixels, pixel coordinates, pixel grayscale value, pixel average value and pixel standard deviation of the first image and each reference area, the correlation between the first image and each reference area is calculated using normalized cross correlation, and the reference area with the highest correlation is used as the matching area.

[0058] When the first image is slidingly matched on the second image and the reference region is obtained, the correlation between the reference region and the first image is calculated, wherein the correlation calculation may use the NCC (Normalized Cross Correlation) algorithm as an evaluation criterion.

[0059] Specifically, the correlation between the first image and each reference area can be calculated based on the number of pixels, pixel coordinates, pixel grayscale value, pixel average and pixel standard deviation of the first image and each reference area using normalized cross correlation, and the reference area with the highest correlation is used as the matching area.

[0060] The NCC algorithm is a mathematical method to determine whether there is a relationship between two sets of data. It is a common correlation evaluation standard. Its calculation formula is as follows:

[0061]

[0062] Where n is the number of pixels, f is the pixel grayscale value of the first image, t is the pixel grayscale value of the second image, μ represents the average pixel value of the first image and the second image, σ f represents the standard deviation of the first image pixels, σ t represents the standard deviation of the second image pixel. f(x,y) is the coordinate of the first image pixel, and t(x,y) is the coordinate of the second image pixel.

[0063] The correlation between the first image and each reference area is calculated by the above formula, wherein the reference area with the largest correlation is used as the matching area closest to the first image. In the subsequent registration process, the matching area is used as the area where the target object in the second image is located to be registered with the target object in the first image.

[0064] Step S25: extracting feature points from the first image and the second image respectively by using an accelerated robust feature algorithm to obtain first feature points corresponding to the first image and second feature points corresponding to the matching area of ​​the second image.

[0065] The first image and the second image are respectively subjected to feature point extraction by using a Speeded Up Robust Features (SURF) algorithm to obtain first feature points corresponding to the first image and second feature points corresponding to a matching area of ​​the second image.

[0066] See also Figure 4 , Figure 4 This is a schematic diagram of feature point extraction for accelerating the robust feature algorithm in this embodiment. The details are as follows:

[0067] Step S251: construct a Hessian matrix.

[0068] The Hessian Matrix is ​​a square matrix composed of the second-order partial derivatives of a multivariate function, which describes the local curvature of the multivariate function. The purpose of constructing the Hessian matrix is ​​to generate stable edge points (mutation points) of the image, which lays a foundation for subsequent feature extraction. A Hessian matrix can be calculated for each pixel.

[0069] In this step, a Hessian matrix is ​​constructed for the matching areas of the first image and the second image respectively.

[0070] Step S252: construct a scale space.

[0071] The scale space of the accelerated robust feature algorithm is composed of O groups and L layers. In the accelerated robust feature algorithm, the sizes of images in different groups are consistent, but the template sizes of the box filters used in different groups gradually increase. The same size of filters are used in different layers of the same group, but the fuzzy coefficient of the filter gradually increases, thereby constructing the scale space of the matching area of ​​the first image and the second image respectively, where the size of the first image is the same as that of the matching area.

[0072] Step S253: Feature point positioning.

[0073] The first image processed by the Hessian matrix and each pixel point in the matching area are compared with 26 points in the two-dimensional image space and the scale space neighborhood, and the key points are preliminarily located. Then, the key points with relatively weak energy and the wrongly located key points are filtered out, and the final stable feature points are screened out, so as to locate the first feature point of the first image and the second feature point in the matching area of ​​the second image.

[0074] Step S254: assigning main directions of feature points.

[0075] In the accelerated robust feature algorithm, the main direction of the feature point is assigned by counting the Haar wavelet features in the circular neighborhood of the feature point. Specifically, in the circular neighborhood of each feature point, the sum of the horizontal and vertical Haar wavelet features of all points in a 60-degree sector is counted, and then the sector is rotated at a certain interval and the Haar wavelet feature values ​​in the area are counted again. After the rotation is completed 360 degrees, the direction of the sector with the largest value is taken as the main direction of the feature point.

[0076] Step S255: Generate feature point descriptors.

[0077] A 4*4 rectangular area block is taken around each first feature point and second feature point, but the direction of the obtained rectangular area is the same as the main direction of the corresponding feature point. Each sub-area counts the Haar wavelet features of 25 pixels in the horizontal and vertical directions, where the horizontal and vertical directions are relative to the main direction. The Haar wavelet features are the sum of the horizontal direction values, the sum of the vertical direction values, the sum of the horizontal direction absolute values, and the sum of the vertical direction absolute values ​​in four directions, thereby generating the descriptors of each first feature point and each second feature point respectively.

[0078] Step S256: Feature point matching.

[0079] The matching degree between each pair of feature points is determined by calculating the Euclidean distance between each first feature point and each second feature point. The shorter the Euclidean distance, the better the matching degree between the two feature points.

[0080] By accelerating the robust feature algorithm, the number of feature point descriptors can be reduced, thereby increasing the speed and efficiency of feature point extraction and feature point description.

[0081] Step S26: Determine whether there is a corresponding second feature point within the preset area of ​​each first feature point. If so, calculate whether the square difference between the feature vector of the first feature point and the feature vector of the second feature point meets the first preset condition. If so, the match is successful.

[0082] After obtaining the first feature points of the first image and the second feature points of the matching area, all the first feature points are matched with all the second feature points. Specifically, by judging whether there is a second feature point in the matching area corresponding to the preset area range of each first feature point, if so, the square difference between the feature vector of the first feature point and the feature vector of the second feature point is calculated to see whether it meets the first preset condition. If so, the matching is successful, and if not, the matching fails. The first preset condition may be exceeding a first threshold value, and the specific value of the first threshold value may be set based on actual applications and is not limited here.

[0083] In a specific application scenario, the preset area range can be set based on the actual application. The smaller the preset area range, the more accurate the feature point matching is, and there is no limitation here.

[0084] In a specific application scenario, when the square difference between a feature vector of a second feature point and a feature vector of a first feature point satisfies a first preset condition, but the second feature point is not within the matching area corresponding to the preset area range of the first feature point, the second feature point fails to match the first feature point.

[0085] In a specific application scenario, when performing feature matching on the first feature point, the number of corresponding second feature points within the preset area of ​​the first feature point can be obtained first. If the number of second feature points is 1, the step of calculating whether the square difference between the feature vector of the first feature point and the feature vector of the second feature point meets the first preset condition is executed; if so, the first feature point and the second feature point are matched successfully; if not, the matching fails.

[0086] If the number of second feature points is greater than and equal to 2, then the quotient between a group of first feature points with the smallest square difference of feature vectors and the second feature points and a group of first feature points with the second smallest square difference of feature vectors and the second feature points is calculated to see whether it meets the second preset condition. If so, the first feature point is successfully matched with the plurality of second feature points; if not, the first feature point fails to match. The second preset condition may be exceeding a second threshold value, and the specific value of the second threshold value may be set based on actual applications and is not limited here.

[0087] In a specific application scenario, the steps of feature point matching can be: first obtain a first feature point, determine whether there is a second feature point within the preset area of ​​the first feature point, and if not, re-select the first feature point to re-match. If so, determine whether the number of second feature points within the preset area of ​​the first feature point is 1, and if it is 1, calculate whether the square difference between the feature vector of the first feature point and the feature vector of the second feature point meets the first preset condition to match the feature points, and if it is not 1, calculate whether the quotient between a group of first feature points with the smallest square difference of feature vectors and the second feature point and a group of first feature points with the second smallest square difference of feature vectors and the second feature point meets the second preset condition to match the feature points. After the feature point matching of the first feature point is completed, continue to obtain other first feature points for feature point matching.

[0088] This step uses a feature point matching method with area restrictions to effectively improve the accuracy of feature point pairing and registration accuracy when image features are not significant or repeated.

[0089] Step S27: Divide the first image and the matching area into a plurality of equal areas, and calculate the transformation relationship using the position information of a group of point pairs with the smallest square difference in the feature vectors of the successfully matched feature point pairs in each equal area.

[0090] After the feature point matching is completed, the first image and the matching area are divided into a plurality of equal areas, and the transformation relationship is calculated using the position information of a group of point pairs with the smallest square difference in the feature vector of the successfully matched feature point pairs in each equal area. This step is used to screen the successfully matched feature point pairs to make them evenly distributed, thereby improving the accuracy of the transformation relationship.

[0091] In a specific application scenario, the first image and the matching area can be divided into M*N equal parts, and the transformation relationship can be calculated by using the position information of a group of point pairs with the smallest square difference in the feature vectors of the successfully matched feature point pairs in each equal part.

[0092] The calculation formula of the transformation relationship is as follows:

[0093] U=Hinv*X (2)

[0094] By expanding formula (2), we can obtain:

[0095]

[0096] Where u and v are the pixel coordinates of the first image, and x and y are the pixel coordinates of the second image. A, B, C, D, E, F, G, H, and I are transformation parameters, which are unknown quantities. The transformation parameters are subsequently calculated and solved to obtain a specific transformation relationship, thereby registering the first image and the matching area.

[0097] After converting formula (3), we get formula (4):

[0098]

[0099] Assuming that I is 1, after substituting the coordinates of at least 4 pairs of successfully matched feature point pairs into formula (4), at least 8 equation groups can be obtained:

[0100]

[0101] Obviously, after setting the value of I, 8 sets of equations can be obtained based on the coordinates of 4 feature point pairs, that is, 8 feature points. Therefore, based on the above 8 sets of equations, the 8 transformation parameters A, B, C, D, E, F, G, and H can be solved to obtain the calculation formula of the specific transformation relationship.

[0102] Therefore, Hinv=[AB CDEFGH]' is the inverse matrix of the registration matrix of the transformation relationship, so Hinv=X\U, and the calculation formula of the transformation relationship can be obtained through the above calculation.

[0103] The transformation relationship may be converted based on each pixel point of the first image to be aligned with the corresponding pixel point in the matching area.

[0104] Step S28: registering the target object of the first image to the second image using the transformation relationship, so as to achieve registration of the first image and the second image.

[0105] Each pixel point of the target object in the first image is registered to the second image using the transformation relationship obtained in the previous step, so as to achieve registration of the first image and the second image.

[0106] In a specific application scenario, any pixel point on the first image can be transformed and registered to the second image based on the transformation relationship, so as to achieve image registration of each pixel point of the first image with the corresponding pixel point of the second image. In a specific application scenario, any pixel point in the matching area of ​​the second image can also be transformed and registered to the first image based on the transformation relationship, so as to achieve image registration of each pixel point of the first image with the corresponding pixel point of the second image. This is not limited here.

[0107] Through the above scheme, the image registration method of this embodiment first reduces the first image and the second image by a preset multiple and grayscales the image features to simplify the image features, thereby reducing the amount of data for template matching, so as to improve the problem of excessive time consumption of the template matching algorithm. Then, based on the template matching algorithm, the matching area on the second image that is most relevant to the first image is found, and after the first feature point and the second feature point are extracted by the accelerated robust feature algorithm, after the second feature point can be successfully matched within the preset area of ​​each first feature point, the first image and the matching area are correspondingly divided into a plurality of equal areas, and the transformation relationship is calculated using the feature point pairs that are successfully matched in each equal area, so that the feature point pairs are evenly distributed by dividing the image area into blocks and selecting a set of matching point pairs for each block, and the problem of huge registration errors in some areas will not occur, thereby improving the final registration effect. In addition, this embodiment adopts a method of combining template matching with feature point matching, so that images of different resolutions can be automatically registered without manual participation, thereby improving the efficiency and effect of image registration.

[0108] Based on the same inventive concept, the present invention also proposes an electronic device, which can be executed to implement the image registration method of any of the above embodiments, see Figure 5 , Figure 5It is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. The electronic device includes a processor 51 and a memory 52.

[0109] The processor 51 is used to execute the program instructions stored in the memory 52 to implement the steps of any of the above-mentioned image registration method embodiments. In a specific implementation scenario, the electronic device may include but is not limited to: a microcomputer, a server, and in addition, the electronic device may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0110] Specifically, the processor 51 is used to control itself and the memory 52 to implement the steps of the above-mentioned registration method embodiment of any one of the images. The processor 51 can also be called a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 51 can be implemented by an integrated circuit chip.

[0111] The above solution can achieve accurate image registration.

[0112] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, see Figure 6 , Figure 6 6 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer-readable storage medium 60 stores at least one program data 61, and the program data 61 is used to implement any of the above methods. In one embodiment, the computer-readable storage medium 60 includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0113] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0116] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium.

[0117] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image registration method, characterized in that: The image registration method comprises: Acquire a first image and a second image to be registered; wherein the size of the first image is smaller than the size of the second image; Slide the first image on the second image, and find a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image; Extracting feature points from the first image and the second image in the matching area respectively to obtain first feature points corresponding to the first image and second feature points corresponding to the second image; In response to the presence of a matching second feature point within a preset area of ​​the first feature point, determining that the match is successful; The transformation relationship between the first image and the second image is calculated using the position information of the successfully matched feature points; registering the first image with the second image using the transformation relationship; The step of calculating the transformation relationship between the first image and the second image by using the position information of the successfully matched feature points comprises: The first image and the matching area are divided into a plurality of equal areas respectively; the transformation relationship is calculated by using the position information of a group of point pairs with the smallest square difference in the feature vectors of the successfully matched feature point pairs in each of the equal areas.

2. The image registration method according to claim 1, characterized in that: The step of sliding the first image on the second image and finding a matching area in the second image that has the highest correlation with the first image and is related to the size of the first image comprises: Slide the first image on the second image to obtain a plurality of reference areas on the second image that have the same size as the first image; Calculating the correlation between the first image and each of the reference areas by using normalized cross correlation based on the number of pixels, pixel coordinates, pixel grayscale values, pixel average values, and pixel standard deviations of the first image and each of the reference areas; The reference area with the highest correlation is used as the matching area.

3. The image registration method according to claim 2, characterized in that: Before the step of sliding the first image on the second image to obtain a plurality of reference areas of the second image that are related to the size of the first image, the step includes: The first image and the second image are reduced by a preset multiple, and grayscale processing is performed on the reduced first image and the second image.

4. The image registration method according to claim 2, characterized in that: The step of sliding the first image on the second image to obtain a plurality of reference areas on the second image that are related to the size of the first image specifically includes: The first image is compared with the second image by sliding in a jump manner to obtain a plurality of reference areas on the second image that are related to the size of the first image.

5. The image registration method according to claim 1, characterized in that: The step of extracting feature points from the first image and the second image in the matching area to obtain first feature points corresponding to the first image and second feature points corresponding to the second image includes: Feature points of the first image and the second image are extracted respectively by using an accelerated robust feature algorithm to obtain first feature points corresponding to the first image and second feature points corresponding to a matching area of ​​the second image.

6. The image registration method according to claim 1, characterized in that: In response to the presence of a matching second feature point within a preset area of ​​the first feature point, the step of determining that the match is successful includes: Determine whether there is a corresponding second feature point within the preset area of ​​each of the first feature points; If so, calculating whether the square difference between the first feature point feature vector and the second feature point feature vector satisfies a first preset condition; If satisfied, the match is successful.

7. The image registration method according to claim 6, characterized in that: The step of determining whether a second feature point exists within a preset area of ​​the first feature point further includes: Obtaining the number of second feature points corresponding to the preset area of ​​the first feature point; If the number is 1, the step of calculating whether the square difference between the first feature point and the second feature point satisfies a first preset condition is performed; If the number is greater than and equal to 2, it is determined whether the quotient between a group of first feature points and a second feature point with the smallest square difference of feature vectors and a group of first feature points and a second feature point with the second smallest square difference of feature vectors meets the second preset condition. If so, the match is successful.

8. The image registration method according to claim 1, characterized in that: The number range of the multiple equal areas is greater than or equal to 4.

9. The image registration method according to claim 1, characterized in that: The step of registering the pixels of the first image to the second image using the transformation relationship comprises: The target object of the first image is registered into the second image by using the transformation relationship, so as to achieve registration of the first image and the second image.

10. An electronic device, characterized in that: The electronic device comprises: a memory and a processor coupled to each other, wherein the processor is used to execute program instructions stored in the memory to implement the image registration method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and the program data can be executed to implement the image registration method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Image matching method and mobile phone application test platform

    CN105513038A