Image registration methods, apparatus, electronic devices and storage media

Abstract

This application discloses an image registration method, apparatus, electronic device, and storage medium. The method includes: performing feature point detection and contour line detection on a first image and a second image respectively; obtaining a first feature point set and a second feature point set corresponding to the first image and the second image respectively; a first contour point set corresponding to a first contour line in the first image; and a second contour point set corresponding to a second contour line in the second image respectively; calculating spatial transformation parameters based on the first feature point set, the second feature point set, each first contour point set, and each second contour point set; and performing registration processing on the first image and the second image based on the spatial transformation parameters. Implementing this application embodiment can improve the accuracy of image registration.

Description

Technical Field

[0001] This application relates to the field of imaging technology, specifically to image registration methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Currently, when electronic devices encounter multiple frames of images with a certain spatial similarity (such as multiple photos taken consecutively by an electronic device in the same scene), they often need to first register these multiple frames to determine the spatial transformation relationship between each frame, and then perform further image processing on each frame based on the spatial transformation relationship between the frames. However, in practice, it has been found that existing image registration methods have the problem of low registration accuracy. Summary of the Invention

[0003] This application discloses an image registration method, apparatus, electronic device, and storage medium, which can improve the accuracy of image registration.

[0004] This application discloses an image registration method, comprising: performing feature point detection on a first image and a second image respectively to obtain a first feature point set corresponding to the first image and a second feature point set corresponding to the second image; performing contour line detection on the first image and the second image respectively to obtain a first contour point set corresponding to a first contour line in the first image and a second contour point set corresponding to a second contour line in the second image; calculating spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line; and performing registration processing on the first image and the second image according to the spatial transformation parameters.

[0005] This application discloses an image registration device, comprising: a feature point detection module, configured to perform feature point detection on a first image and a second image respectively, to obtain a first feature point set corresponding to the first image and a second feature point set corresponding to the second image; a contour detection module, configured to perform contour line detection on the first image and the second image respectively, to obtain a first contour point set corresponding to a first contour line in the first image and a second contour point set corresponding to a second contour line in the second image; a calculation module, configured to calculate spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line; and a registration module, configured to perform registration processing on the first image and the second image based on the spatial transformation parameters.

[0006] This application discloses an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor implements any of the image registration methods disclosed in this application.

[0007] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the image registration methods disclosed in this application.

[0008] Compared with related technologies, the embodiments of this application have the following beneficial effects:

[0009] Feature point detection and contour line detection are performed on the first image and the second image respectively to obtain a first feature point set and a first contour line set corresponding to the first image, and a second feature point set and a second contour line set corresponding to the second image. Control point sampling is performed on the first contour lines included in the first contour line set and the second contour lines included in the second contour line set to obtain a first contour point set corresponding to each first contour line and a second contour point set corresponding to each second contour line. Spatial transformation parameters are calculated based on the first feature point set, the second feature point set, the first contour point set, and the second contour point set, and then the first image and the second image are registered according to the calculated spatial transformation parameters. It can be seen that in this embodiment, feature point information and contour point information in the image are fused for image registration, thereby further fusing contour features in the image on the basis of feature points to improve the accuracy of image registration. In application scenarios with a large number of contour features, such as indoor shooting, high registration accuracy can be achieved, and the algorithm has strong robustness and can adapt to various complex indoor shooting scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the structure of an image processing circuit disclosed in one embodiment;

[0012] Figure 2 This is a flowchart illustrating an image registration method disclosed in one embodiment;

[0013] Figure 3 This is a schematic flowchart of another image registration method disclosed in one embodiment;

[0014] Figure 4This is a flowchart illustrating another image registration method disclosed in one embodiment;

[0015] Figure 5 This is a schematic diagram of the structure of an image registration device disclosed in one embodiment;

[0016] Figure 6 This is a schematic diagram of the structure of an electronic device disclosed in one embodiment. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0019] In related technologies, image registration methods can include keypoint-based registration methods and descriptor-based registration methods. For example, there are registration methods based on ORB (Oriented Fast and Rotated BRIEF) keypoints and registration methods based on Scale-Invariant Feature Transform (SIFT) feature points. However, these image registration methods have the following drawbacks:

[0020] First, the aforementioned key points and descriptors are often isolated in space, unlike image transformations or lines which are spatially continuous image features. Second, the aforementioned methods generally describe the grayscale features of points through descriptors, without fully utilizing the spatial location information of points. Third, the aforementioned methods generally use one-to-one hard decision to directly determine the correspondence between feature points, making them more sensitive to mismatches.

[0021] In addition, contour-based image registration methods also exist. However, contour-based image registration methods tend to ignore the image information provided by feature points. In mobile phone photography, there are often cases where the contours are blurred or lack sufficient contour features, in which case contour-based image registration methods often fail to guarantee the accuracy of image registration.

[0022] This application discloses an image registration method, apparatus, electronic device, and storage medium, which can improve the accuracy of image registration. These will be described in detail below.

[0023] Please see Figure 1 , Figure 1 This is a schematic diagram of an image processing circuit disclosed in one embodiment. This image processing circuit can be used to implement the image registration method disclosed in this application embodiment. Exemplarily, this image processing circuit can be applied to electronic devices such as mobile phones, smart wearable devices, and tablet computers, but is not limited thereto. Figure 1 As shown, the image processing circuit may include an imaging device (camera) 110, an attitude sensor 120, an image memory 130, an image signal processing (ISP) processor 140, a control logic unit 150, and a display 160.

[0024] The image data captured by the imaging device 110 can first be processed by the ISP processor 140. For example, the ISP processor 140 can analyze the image data to obtain image statistical information that can be used to determine one or more control parameters of the imaging device 110. The imaging device 110 may include one or more lenses 112 and an image sensor 114. The image sensor 114 may include a color filter array (e.g., a Bayer filter). The image sensor 114 can acquire the light intensity and wavelength information captured by each imaging pixel and provide a set of raw image data that can be processed by the ISP processor 140. The attitude sensor 120 (e.g., a three-axis gyroscope, Hall sensor, accelerometer, etc.) can provide the ISP processor 140 with parameters for image processing (e.g., image stabilization parameters) based on the attitude sensor 120 interface type. The attitude sensor 120 interface can be an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

[0025] In addition, image sensor 114 can also send raw image data to attitude sensor 120. Attitude sensor 120 can provide the raw image data to ISP processor 140 based on the interface type of attitude sensor 120, or store the raw image data in image memory 130.

[0026] The ISP processor 140 can process raw image data pixel by pixel in various formats. For example, each image pixel can have a bit depth of 8, 10, 12, or 14 bits. The ISP processor 140 can perform one or more image processing operations on the raw image data and collect statistical information about the image data. The image processing operations can be performed with the same or different bit depth precision.

[0027] The ISP processor 140 can also receive image data from the image memory 130. For example, the attitude sensor 120 interface sends raw image data to the image memory 130, and the raw image data in the image memory 130 is then provided to the ISP processor 140 for processing. The image memory 130 may be part of a memory device, a storage device, or a separate dedicated memory within an electronic device, and may include DMA (Direct Memory Access) features.

[0028] Upon receiving raw image data from the image sensor 114 interface, the attitude sensor 120 interface, or the image memory 130, the ISP processor 140 can perform one or more image processing operations, such as temporal filtering, frequency filtering, etc. The processed image data can be sent to the image memory 130 for further processing before display. The ISP processor 140 receives processed data from the image memory 130 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The processed image data can be output to the display 160 for user viewing and / or further processed by the graphics engine or GPU (Graphics Processing Unit). Furthermore, the output of the ISP processor 140 can also be sent to the image memory 130, and the display 160 can read image data from the image memory 130. In one embodiment, the image memory 130 can be configured to implement one or more frame buffers.

[0029] The statistical data determined by the ISP processor 140 can be sent to the control logic unit 150. For example, the statistical data may include image sensor 114 statistics such as gyroscope vibration frequency, auto exposure, auto white balance, auto focus, flicker detection, black level compensation, and lens 112 shading correction. The control logic unit 150 may include a processor and / or microcontroller executing one or more routines (such as firmware) that determine control parameters for the imaging device 110 and the ISP processor 140 based on the received statistical data. For example, the control parameters for the imaging device 110 may include attitude sensor 120 control parameters (e.g., gain, integration time for exposure control, image stabilization parameters, etc.), camera flash control parameters, camera image stabilization shift parameters, lens 112 control parameters (e.g., focal length for focusing or zooming), or combinations of these parameters. The ISP control parameters may include gain levels and color correction matrices for auto white balance and color adjustment (e.g., during RGB processing), and lens 112 shading correction parameters.

[0030] In some embodiments, the lens 112 and image sensor 114 in the imaging device (camera) 110 can acquire multiple frames of images captured in succession, and send these multiple frames of images to the ISP processor 140. Since the multiple frames of images captured in succession often have a certain spatial similarity, the ISP processor 140 can register the above-mentioned multiple frames of images to determine the spatial transformation relationship between each frame of images, and then use the registered images to realize further image processing (such as image blurring, image fusion, image super-resolution, etc.).

[0031] For example, taking image registration of two frames, namely a first image and a second image, as an example, the ISP processor 140 can perform feature point detection on the first image and the second image respectively to obtain a first feature point set corresponding to the first image and a second feature point set corresponding to the second image. Further, the ISP processor 140 performs contour line detection on the first image and the second image respectively to obtain a first contour line set corresponding to the first image and a second contour line set corresponding to the second image. Then, it further performs control point sampling on the first contour lines included in the first contour line set and the second contour lines included in the second contour line set to obtain a first contour point set corresponding to each first contour line and a second contour point set corresponding to each second contour line. The ISP processor 140 can calculate spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line, and then perform registration processing on the first image and the second image based on the spatial transformation parameters.

[0032] By implementing the above image registration method, the electronic device can fuse feature point information and contour feature information in the first image and the second image to jointly calculate the spatial transformation parameters used for registering the first image and the second image. These spatial transformation parameters can accurately characterize the spatial transformation relationship between the first image and the second image. Using these spatial transformation parameters to register the first image and the second image helps to improve the accuracy of image registration.

[0033] Optionally, the ISP processor 140 can also perform further image processing using the registered first and second images. Based on this, the ISP processor 140 can output the processed image data to the display 160 for user viewing.

[0034] In other embodiments, when the ISP processor 140 retrieves the first and second images to be registered from the image memory 130, the first and second images may also be image data originally stored locally on the electronic device, or image data obtained from an external source via a data transmission module. For image data from other sources (i.e., not captured by the imaging device 110), the ISP processor 140 may also use the above-described image registration method to register them without affecting the realization of the corresponding image registration effect.

[0035] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image registration method disclosed in one embodiment. This method can be applied to electronic devices including the image processing circuit described above. Figure 2 As shown, the image registration method may include the following steps:

[0036] 210. Perform feature point detection on the first image and the second image respectively to obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image.

[0037] After acquiring the first image and the second image to be registered, the electronic device can perform feature point detection on the first image and the second image respectively based on a certain feature point detection algorithm to obtain the first feature point detected in the first image and the second feature point detected in the second image.

[0038] For example, feature point detection algorithms may include those based on Harris corners, ORB keypoints, SIFT feature points, or Speeded Up Robust Features (SURF) feature points, etc., without any specific limitation.

[0039] The electronic device can form a first feature point set by combining all detected first feature points, and form a second feature point set by combining all detected second feature points. Alternatively, the electronic device can further filter all feature points detected from the first image and the second image to remove some feature points, thereby obtaining a first feature point set composed of the first feature points retained in the first image and a second feature point set composed of the second feature points retained in the second image.

[0040] Optionally, the electronic device may calculate the similarity between each first feature point and each second feature point, and filter each first feature point and each second feature point based on the similarity.

[0041] The electronic device can first determine matching point pairs based on the similarity between the first feature point and the second feature point. Each matching point pair includes a first feature point and a second feature point that matches the first feature point; the second feature point that matches the first feature point can be the second feature point with the highest similarity among all the second feature points included in the second image. Further, the electronic device can filter the first and second feature points included in each matching point pair based on the similarity corresponding to each pair.

[0042] The filtering methods may include any one or more of the following, but are not limited to:

[0043] Method 1: For each pair of matching points, if the similarity of the pair is lower than the similarity threshold, the first and second feature points included in the pair are removed; if the similarity of the pair is greater than or equal to the similarity threshold, the first and second feature points included in the pair are retained.

[0044] The similarity threshold can be set with reference to actual business needs. When the similarity of a matching point pair is lower than the similarity threshold, it can be considered that the first feature point and the second feature point included in the matching point pair are feature points that are mismatched.

[0045] Method 2: Sort each pair of matching points in descending order of similarity, and retain the first and second feature points of the first N pairs or the first N% of matching point pairs.

[0046] For example, the electronic device can first eliminate matching point pairs with similarity lower than the similarity threshold in each matching point pair based on the aforementioned method one, then sort the remaining matching point pairs based on method two, and then retain the top 30% of the sorted matching point pairs.

[0047] 220. Perform contour line detection on the first image and the second image respectively, and obtain the first contour point set corresponding to the first contour line in the first image and the second contour point set corresponding to the second contour line in the second image.

[0048] Electronic devices can perform contour line detection on a first image and a second image based on a certain edge detection algorithm to obtain a first contour line detected in the first image and a second contour line detected in the second image.

[0049] For example, edge detection algorithms may include edge detection algorithms based on Canny operators, Sobel operators, Prewitt operators, or Roberts operators, etc., without any specific limitation.

[0050] The electronic device can form a first contour set by combining all detected first contour lines, and a second contour set by combining all detected second contour lines. Alternatively, the electronic device can further filter all contour lines detected from the first and second images to remove some contour lines, thereby obtaining a first contour set composed of the first contour lines retained in the first image and a second contour set composed of the second contour lines retained in the second image.

[0051] Optionally, the electronic device may filter based on the number of pixels included in the contour line and / or the image position of the contour line in the image. For ease of description, the following uses a target image as an example to illustrate a specific implementation method for contour line filtering. The target image can be any frame from either the first image or the second image. Wherein:

[0052] Filtering based on the number of pixels contained in a contour line may include: for each contour line detected from the target image, counting the number of pixels contained in each contour line; removing contour lines from the multiple contour lines detected in the target image that contain fewer than a certain number of pixels, resulting in a set of contour lines corresponding to the target image, which consists of the retained contour lines. The aforementioned threshold can be set according to actual business needs and is not specifically limited. That is, electronic devices can remove shorter contour lines from the target image and retain longer contour lines.

[0053] Filtering based on the image position of contour lines may include: determining the image position of each contour line detected from the target image; removing contour lines whose image positions are located in the boundary region of the target image from the multiple contour lines detected in the target image, resulting in a set of contour lines corresponding to the target image, which consists of the retained contour lines. That is, the electronic device can remove contour lines near the image boundary.

[0054] The aforementioned first set of contour lines may include one or more first contour lines. Similarly, the second set of contour lines may also include one or more second contour lines.

[0055] After obtaining the first contour line detected from the first image and the second contour line detected from the second image, the electronic device can sample control points for each first contour line and each second contour line. The control point can be a pixel selected by the electronic device from the contour line, such as a corner point, an endpoint, any pixel between two adjacent corner points, any pixel between two adjacent endpoints, or any pixel between adjacent corner points and endpoints, without any specific limitation.

[0056] The electronic device samples control points for each first contour line to obtain a first contour point set corresponding to that first contour line. The first contour point set includes multiple first contour points selected from that first contour line. Similarly, the electronic device samples control points for each second contour line to obtain a second contour point set corresponding to that second contour line. The second contour point set includes multiple second contour points selected from that second contour line.

[0057] The electronic device can sample multiple first contour point sets and multiple second contour point sets. The number of first contour point sets can correspond to the number of first contour lines in the first contour line set, and the number of second contour point sets can correspond to the number of second contour lines in the second contour line set.

[0058] To facilitate the differentiation of different contour point sets, the electronic device can number each first contour line included in the first contour line set and each second contour line included in the second contour line set, and use the contour line numbers to label each contour point set.

[0059] 230. Calculate the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line.

[0060] After extracting the first feature point set, the second feature point set, the first contour point set, and the second contour point set from the first image and the second image, the electronic device can calculate spatial transformation parameters based on the feature points and contour points included in each of the above point sets. These spatial transformation parameters can be used to register the first image and the second image.

[0061] For example, the spatial transformation parameters mentioned above may include mapping parameters, such as mapping vectors, mapping matrices, affine transformation parameters, etc., without any specific limitation.

[0062] 250. Perform registration processing on the first image and the second image according to the spatial transformation parameters.

[0063] In some embodiments, after calculating the corresponding spatial transformation parameters such as mapping vectors or mapping matrices based on the aforementioned point sets, the electronic device can perform matrix operations on the first image (or the second image) based on the mapping vectors or mapping matrices to obtain the registered first image (or the second image).

[0064] In other embodiments, after calculating the affine transformation parameters, the electronic device can rotate the first image (or second image) to be registered according to the transformation matrix in the affine transformation parameters, and then translate the rotated first image (or second image) according to the transformation vector in the affine transformation parameters to obtain the first image (or second image) registered after affine transformation.

[0065] It should be noted that, when the image to be registered includes multiple frames, the electronic device can use one frame as a reference image as the first image, and sequentially use the other frames as second images, and then use the image registration method in this embodiment to register the first image and each frame of the second image sequentially. In some embodiments, the electronic device can also first select an initial first image and a second image, register the first image and the second image, and then use the registered first image or the second image as a reference to continue registering other images, and repeat the above process until all images are registered.

[0066] As can be seen, in the aforementioned embodiments, the electronic device can calculate the spatial transformation parameters for image registration by combining the image feature information provided by feature points and contour edges. Feature points and contour edges can complement each other, making full use of their respective advantages, which can improve the accuracy of spatial transformation parameters, thereby improving the accuracy of image registration. In application scenarios with a large number of contour features, such as indoor shooting, the electronic device can effectively fuse feature point and contour edge information to perform accurate image registration operations, and can adapt to various complex indoor shooting scenarios.

[0067] Please see Figure 3 , Figure 3 This is a schematic flowchart of another image registration method disclosed in one embodiment, which can be applied to electronic devices including the image processing circuit described above. Figure 3 As shown, the image registration method may include the following steps:

[0068] 310. Perform feature point detection on the first image and the second image respectively to obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image.

[0069] 320. Perform contour line detection on the first image and the second image respectively to obtain the first contour line set corresponding to the first image and the second contour line set corresponding to the second image.

[0070] Steps 310 and 320 are similar to steps 210 and 220 above, and will not be described again here.

[0071] 330. Control point sampling is performed on the first contour lines included in the first contour line set and the second contour lines included in the second contour line set to obtain a first contour point set corresponding to each first contour line and a second contour point set corresponding to each second contour line.

[0072] In this embodiment, for each first or second contour line, the electronic device can employ different control point sampling strategies based on the contour line type to perform control point sampling on the contour line. The following description uses a target contour line as an example, where the target contour line can be any first or second contour line.

[0073] In one embodiment, if the target contour line includes corner points, the electronic device can perform control point sampling on the target contour line starting from the corner points. Specifically, the electronic device can perform control point sampling from the corner points in the direction of any endpoint of the target contour. The sampling can be uniform sampling with uniform pixel spacing or non-uniform sampling with uneven pixel spacing; the specific method is not limited.

[0074] Optionally, if the target contour line includes a corner point, the electronic device can use that corner point as the starting point to perform control point sampling on the target contour line at a first pixel interval in any direction to one or two endpoints of the target contour line.

[0075] Optionally, if the target contour line includes at least two corner points, the electronic device can perform control point sampling between two adjacent corner points at a second pixel interval, and between adjacent endpoints and corner points of the target contour line at a third pixel interval, where the second pixel interval is greater than the third pixel interval. That is, the electronic device can perform relatively high-density, evenly spaced, uniform sampling on line segments between corner points, and relatively low-density, evenly spaced, uniform sampling on line segments between angles and endpoints. For the target contour line as a whole, this constitutes non-uniform sampling with unequal intervals.

[0076] It should be noted that the aforementioned first pixel spacing can be the same as or different from the second or third pixel spacing; no specific limitation is imposed.

[0077] In another embodiment, if the target contour does not include corner points and is a non-closed contour, the electronic device can use any pixel of the target contour as a starting point to perform control point sampling with uniform pixel spacing on the target contour.

[0078] In another embodiment, if the target contour does not include corner points and is a closed contour, the electronic device can sample the centroid of the target contour as the control point of the target contour.

[0079] The electronic device samples control points for each first contour line and each second contour line based on the aforementioned control point sampling strategies, thereby obtaining multiple sets of first contour points and multiple sets of second contour points.

[0080] 340. Based on the first feature point set and the second feature point set, determine the correspondence between the feature points corresponding to the multiple feature point pairs respectively.

[0081] In the embodiments of this application, each feature point pair includes a first feature point from a first feature point set and a second feature point from a second feature point set.

[0082] It should be noted that each first feature point in the first feature point set can form a feature point pair with each second feature point in the second feature point set. For example, if both the first and second feature point sets contain 10 feature points, then a total of 10 × 10 = 100 feature point pairs can be formed.

[0083] In the embodiments of this application, the feature point correspondence of each feature point pair can be used to indicate whether the first feature point and the second feature point included in the feature point pair are the same feature point belonging to different images, which can be characterized by the similarity probability of the first feature point and the second feature point. If the similarity probability of the first feature point and the second feature point included in the feature point pair is higher, the feature point correspondence of the feature point pair is stronger.

[0084] Optionally, for each feature point pair, the electronic device can calculate the first Euclidean distance between the first feature point and the second feature point included in the feature point pair, and construct a feature point correspondence relationship with the feature point pair based on the first Euclidean distance, thereby obtaining the feature point correspondence relationships corresponding to multiple feature point pairs respectively.

[0085] For example, the correspondence between feature points of a pair can be represented by the following formula:

[0086]

[0087] in, It can be used to represent the p-th first feature point. It can be used to represent the q-th second feature point, where p and q are both positive integers; It can be used to represent feature point pairs The probability of similarity; It can be used to represent the Euclidean distance between the p-th first feature point and the q-th second feature point; T1 can be used to represent temperature parameters, which can be set according to actual business needs.

[0088] 350. Based on the first set of contour points corresponding to each first contour line and the second set of contour points corresponding to each second contour line, determine the contour point correspondence relationship between multiple contour point pairs respectively.

[0089] In the embodiments of this application, each contour point pair includes a first contour point in a first contour point set and a second contour point in a second contour point set.

[0090] It should be noted that when the number of both the first contour point set and the second contour point set is greater than or equal to 2, each first contour point in each first contour point set can form a contour point pair with each second contour point in each second feature point set. For example, if both the first contour point set and the second contour point set contain 3 feature points, then a total of 3 × 3 = 9 contour point pairs can be formed.

[0091] In this embodiment, the feature point correspondence of each contour point pair can be used to indicate whether the first contour point and the second contour point included in the contour point pair are the same contour point belonging to different images, which can be characterized by the similarity probability of the first contour point and the second contour point. The similarity probability of the first contour point and the second contour point belonging to the same contour line is higher than the similarity probability of the first contour point and the second contour point not belonging to the same contour line.

[0092] Optionally, for each pair of contour points, the electronic device can determine from multiple first contour point sets the first target contour point set to which the first contour points included in the pair belong, and from multiple second contour point sets the second target contour point set to which the second contour points included in the pair belong. The electronic device first calculates the chamfer distance between the first target contour point set and the second target contour point set, and calculates the second Euclidean distance between the first contour points included in the pair and the second contour points included in the pair. Then, based on the aforementioned chamfer distance and second Euclidean distance, it constructs the contour point correspondence of the pair, thereby obtaining the feature point correspondence of multiple contour point pairs respectively.

[0093] For example, the correspondence between contour points in a pair can be represented by the following formula:

[0094]

[0095] in, It can be used to represent the p-th first contour point in the set of first contour points corresponding to the i-th first contour line. It can be used to represent the qth second contour point in the set of second contour points corresponding to the jth second contour line, where i, j, p, and q are all positive integers; Can be used to represent contour point pairs The probability of similarity; It can be used to represent the chamfer distance between the first contour point set corresponding to the i-th first contour line and the second contour point set corresponding to the j-th second contour line, that is, the chamfer distance between the first target contour point set and the second target contour point set; T1 and T2 can be used to represent the Euclidean distance between the p-th first contour point and the q-th second contour point mentioned above; T2 and T3 can be used to represent temperature parameters, which can be set according to actual business needs.

[0096] As can be seen from formula (2), in this embodiment of the application, the contour point correspondence of contour point pairs can be constructed based on the assumption that points on the same contour line are still on the same contour after transformation. This can effectively improve the accuracy of determining the contour point correspondence, thereby improving the accuracy of spatial transformation parameter calculation.

[0097] That is, the correspondence between contour points in a contour point pair is related not only to the Euclidean distance between the contour points, but also to the contour line to which the contour points belong. If the contour points included in a contour point pair belong to the same contour line, then the chamfer distance between the first target contour point set and the second target contour point set corresponding to the contour point pair is smaller, and the similarity probability of the contour point pair is greater.

[0098] The chamfer distance between any two sets of first and second contour points can be calculated in the following way:

[0099] First, for each first contour point included in the first contour point set, a second contour point corresponding to each first contour point is determined in the second contour point set according to the nearest neighbor criterion; and a first average distance corresponding to the first contour point set is calculated based on the Euclidean distance between each first contour point in the first contour point set and the corresponding second contour point.

[0100] For example, the first average distance can be expressed by the following formula:

[0101]

[0102] in, It can be used to represent the Euclidean distance between the corresponding first and second contour points of the k-th pair, where k is a positive integer; n(i) can be used to represent the distance between the first contour point set and the second contour point. Based on the nearest neighbor criterion, from the second contour point set The number of corresponding contour point pairs determined in the process.

[0103] Then, for each second contour point included in the second contour point set, a first contour point corresponding to each second contour point is determined in the first contour point set according to the nearest neighbor criterion; and a second average distance corresponding to the second contour point set is calculated based on the Euclidean distance between each second contour point in the second target contour point set and the corresponding first contour point.

[0104] For example, the second average distance can be expressed by the following formula:

[0105]

[0106] in, It can be used to represent the Euclidean distance between the k-th pair of corresponding second contour points and the first contour point, where k is a positive integer; n(j) can be used to represent the number of mutually corresponding contour point pairs determined from the first contour point set based on the nearest neighbor criterion, with the second contour point set as the reference.

[0107] Finally, the chamfer distance is calculated based on the first average distance and the second average distance.

[0108] For example, the chamfer distance can be expressed by the following formula:

[0109]

[0110] in, It can be used to represent the chamfer distance between the first set of contour points and the second set of contour points.

[0111] 360. Construct a correspondence matrix based on the correspondence between multiple feature point pairs and the correspondence between multiple contour point pairs.

[0112] In this embodiment, the first feature points and first contour points detected from the first image can form a template point set, and the second feature points and second contour points detected from the second image can form a target point set. That is, the template points included in the template point set can be first feature points or first contour points, and the target points included in the target point set can be second feature points or second contour points.

[0113] The correspondence matrix can include the correspondence between each template point and each target point. That is, the correspondence matrix can include the feature point correspondence between each feature point pair, the contour point correspondence between each contour point pair, and the correspondence between feature points and contour points, such as the correspondence between the first feature point and the first contour point, the correspondence between the first feature point and the second contour point, and the correspondence between the second feature point and the first and second contour points respectively.

[0114] For example, the correspondence between feature points in a pair can be calculated according to the aforementioned formula (1), and the correspondence between contour points in a pair can be calculated according to the aforementioned formula (2). The correspondence between feature points and contour points can be set with reference to the following rules: if a feature point and a contour point belong to the same pixel, the correspondence between the feature point and the contour point can be set to 1; if a feature point and a contour point do not belong to the same pixel, the correspondence between the feature point and the contour point can be set to 0. Alternatively, the correspondence between feature points and contour points can be uniformly set to 0 without considering that feature points and contour points may belong to the same pixel.

[0115] In some embodiments, to facilitate computation and suppress outliers, the electronic device may also normalize the correspondence matrix. For example, the normalized correspondence matrix can be represented by the following formula:

[0116]

[0117] Where, p mn It can be used to represent the normalized correspondence matrix; It can be used to represent the correspondence between the m-th template point and the n-th target point, where the m-th template point belongs to the template point set corresponding to the first image, and the n-th target point belongs to the target point set corresponding to the second image; It can be used to represent the correspondence between the l-th template point and the n-th target point, where the l-th template point belongs to the template point set corresponding to the first image; M can be used to represent the total number of template points included in the template point set; τ can be used to represent the parameters for suppressing outliers.

[0118] 370. Calculate the spatial transformation parameters based on the correspondence matrix.

[0119] 380. Perform registration processing on the first image and the second image according to the spatial transformation parameters.

[0120] In this embodiment of the application, after determining the correspondence matrix, the electronic device can further calculate the corresponding spatial transformation parameters based on the correspondence matrix. The spatial transformation parameters can be used to register the first image and the second image.

[0121] For example, the spatial transformation parameters mentioned above may include mapping parameters, such as mapping vectors, mapping matrices, affine transformation parameters, etc. In some embodiments, taking affine transformation parameters as an example, the electronic device may first determine the registration cost function based on the above correspondence matrix, and then calculate the affine transformation parameters between the first image and the second image based on the registration cost function, as the spatial transformation parameters for registering the first image and the second image.

[0122] For example, the cost function for registering the first image with the second image can be expressed by the following formula:

[0123]

[0124] Where, p mn A can be used to represent the normalized correspondence matrix, A can be used to represent the transformation matrix of rotation transformation, and t can be used to represent the transformation vector of translation transformation. The transformation matrix A and the transformation vector t together constitute the affine transformation parameters.

[0125] By taking the partial derivatives of the transformation matrix A and the transformation vector t in the above formula (7) and setting them equal to 0, we can obtain the following:

[0126]

[0127] t = u x -Au Y ;Formula (9);

[0128] In formula (8) above, "1" represents a unit vector, P can be used to represent the correspondence matrix, X can be used to represent the template point set, and Y can be used to represent the target point set.

[0129] Combining formulas (8) and (9) above, the affine transformation performed on the second image can be expressed by the following formula:

[0130]

[0131] Using the transformation matrix A and transformation vector t described above, the target point set can be updated to obtain... Updated target point set It can be used to characterize the new image positions of each second feature point and second contour point in the second image after performing an affine transformation on the second image, thus achieving image registration between the first image and the second image.

[0132] As can be seen, in the aforementioned embodiments, the electronic device integrates feature information from feature points and contour edges to calculate spatial transformation parameters, which helps to improve the accuracy of spatial transformation parameters. In addition, the electronic device can also construct the contour point correspondence relationship of contour point pairs based on the assumption that points on the same contour line remain on the same contour after transformation, which can effectively improve the accuracy of determining the contour point correspondence relationship, thereby further improving the calculation accuracy of spatial transformation parameters.

[0133] In some embodiments, the transformation matrix A and transformation vector t calculated based on the aforementioned formulas may be accurate spatial transformation parameters that can be directly applied to register the second image.

[0134] In other embodiments, the accuracy of the transformation matrix A and transformation vector t calculated based on the foregoing formulas can be further improved to enhance the accuracy of image registration. Wherein:

[0135] The electronic device can obtain accurate spatial transformation parameters by iteratively solving the simulated annealing algorithm. That is, the electronic device can adjust the target point set (the second feature point set and each second contour point set) based on the spatial transformation parameters it has calculated, and re-execute the steps of calculating the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line, until the iteration termination condition is met.

[0136] For a clearer explanation of the specific implementation of iteratively solving for spatial transformation parameters, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic flowchart illustrating another image registration method disclosed in one embodiment. This method can be applied to electronic devices including the image processing circuit described above. Figure 4 As shown, the image registration method may include the following steps:

[0137] 401. Perform feature point detection on the first image and the second image respectively to obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image.

[0138] 403 Perform contour line detection on the first image and the second image respectively to obtain the first contour line set corresponding to the first image and the second contour line set corresponding to the second image.

[0139] 405. Perform control point sampling on the first contour lines included in the first contour line set and the second contour lines included in the second contour line set respectively to obtain the first contour point set corresponding to each first contour line and the second contour point set corresponding to each second contour line.

[0140] Steps 401 to 405 are similar to steps 310 and 330 above, and will not be repeated here.

[0141] 407. Calculate the initial spatial transformation parameters based on the matching relationship between each first feature point included in the first feature point set and each second feature point included in the second feature point set, and use the initial spatial transformation parameters to perform the initial update of the second feature point set and each second contour point set.

[0142] The electronic device can find the matching relationship between the first feature point and the second feature point based on the nearest neighbor criterion; or, it can determine the matching relationship between each first feature point and the second feature point based on the similarity between the first feature point and the second feature point. Therefore, the electronic device can calculate the initial spatial transformation parameters based on the initial detection of the matching relationship between the first feature point and the second feature point from the first image and the second image. The initial spatial transformation parameters can be used as the initial values ​​for the iteration of the spatial transformation parameters in the simulated annealing algorithm.

[0143] 409. Based on the first feature point set and the second feature point set, determine the correspondence between the feature points corresponding to the multiple feature point pairs respectively.

[0144] In step 409, the correspondence between feature points of a feature point pair can be calculated according to the aforementioned formula (1).

[0145] 411. Based on the first set of contour points corresponding to each first contour line and the second set of contour points corresponding to each second contour line, determine the correspondence between contour points of multiple contour point pairs respectively.

[0146] In step 411, the correspondence between contour points of contour point pairs can be calculated according to the aforementioned formula (2).

[0147] It should be noted that during the iteration process, whenever the second feature point set and each second contour point set are updated, the feature point correspondence calculated based on the aforementioned formula (1) for the same feature point pair may be different from the feature point correspondence calculated before the update; the contour point correspondence calculated based on the aforementioned formula (2) for the same contour point pair may be different from the contour point correspondence calculated before the update.

[0148] 412. Construct a correspondence matrix based on the correspondence between feature point pairs and contour point pairs.

[0149] In step 412, the correspondence matrix can be calculated according to the aforementioned formula (6).

[0150] 415. Calculate the spatial transformation parameters based on the correspondence matrix.

[0151] In step 415, the spatial transformation parameters can be calculated based on the aforementioned formulas (7), (8) and (9).

[0152] 417. Determine whether the iteration termination condition is met; if yes, proceed to step 419; if no, proceed to step 421.

[0153] In step 47, the iteration termination condition may include an iteration count condition and / or an iteration parameter condition.

[0154] The aforementioned iteration count condition can include a specified number of iterations, such as 100, 150, or 200. By performing a finite number of iterations, a balance can be achieved between the computing power of the electronic device and the accuracy of its calculated spatial transformation parameters. This ensures that the calculated spatial transformation parameters achieve a certain level of accuracy while minimizing the computation time of the electronic device, thereby reducing power consumption.

[0155] The aforementioned iterative parameter conditions may include a specified iterative parameter precision. This iterative parameter can be the temperature parameter of the simulated annealing algorithm, which can be represented by T1, T2, and T3 in formulas (1) and (2). By performing the above iterative calculation process based on the deterministic annealing algorithm, the temperature parameter obtained in each iteration can gradually decrease. When the temperature parameter reaches the specified temperature parameter precision (e.g., specifying T...), the temperature parameter is considered to be at a certain precision. stop =10 -5 When the temperature parameter T1≤T stop When the iteration parameter conditions are met, the iteration can be stopped, thus ensuring that the spatial transformation parameters calculated by the electronic device achieve the required accuracy, thereby improving the accuracy and reliability of image registration by the electronic device.

[0156] 419. Perform registration processing on the first image and the second image based on the spatial transformation parameters updated when the iteration ends.

[0157] When the iteration termination condition is met, the updated spatial transformation parameters can be relatively accurate spatial transformation parameters, which can be used to perform registration processing on the first and second images.

[0158] 421. Reduce the temperature parameter, update the second feature point set and each second contour point set based on the spatial transformation parameter, and return to execute the aforementioned step 409.

[0159] In step 421, the electronic device updates the target point set (the second feature point set and each of the second contour point sets) instead of updating the entire second image, which reduces the computational load. The update of the second feature point set and each of the second contour point sets can be calculated according to the aforementioned formula (10).

[0160] As can be seen, the electronic device iteratively updates the spatial transformation parameters based on the simulated annealing algorithm. Compared with the method of directly determining the correspondence of feature points for image registration through one-to-one hard decision, the method of iteratively solving the spatial transformation parameters in the aforementioned embodiment can reduce the sensitivity to matching errors, thereby improving the accuracy of the solved spatial transformation parameters and thus improving the accuracy of image registration.

[0161] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an image registration device disclosed in one embodiment. This image registration device can be applied to electronic devices including the above-described image processing circuit. Figure 5 As shown, the image registration device 500 may include a feature point detection module 510, a contour detection module 520, a calculation module 530, and a registration module 540, wherein:

[0162] The feature point detection module 510 is used to perform feature point detection on the first image and the second image respectively, and obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image.

[0163] The contour detection module 520 is used to perform contour detection on the first image and the second image respectively, and obtain the first contour point set corresponding to the first contour line in the first image and the second contour point set corresponding to the second contour line in the second image.

[0164] The calculation module 530 is used to calculate spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line.

[0165] The registration module 550 is used to perform registration processing on the first image and the second image according to the spatial transformation parameters.

[0166] In one embodiment, the feature point detection module 510 may include: a first detection unit, a similarity calculation unit, and a first filtering unit.

[0167] The first detection unit can be used to perform feature point detection on the first image and the second image respectively, to obtain the first feature point in the first image and the second feature point in the second image;

[0168] The similarity calculation unit can be used to calculate the similarity between each first feature point in the first image and each second feature point in the second image;

[0169] The first filtering unit can be used to remove some feature points from the first feature points included in the first image and the second feature points included in the second image based on similarity, so as to obtain a first feature point set composed of the retained first feature points and a second feature point set composed of the retained second feature points.

[0170] In one embodiment, the contour detection module 520 may include a contour line detection unit and a contour point sampling unit.

[0171] The contour detection unit can be used to perform contour detection on the first image and the second image respectively, and obtain a first contour set corresponding to the first image and a second contour set corresponding to the second image.

[0172] The contour point sampling unit can be used to sample control points for the first contour lines included in the first contour line set and the second contour lines included in the second contour line set, respectively, to obtain a first contour point set corresponding to each first contour line and a second contour point set corresponding to each second contour line.

[0173] In one embodiment, the contour detection unit may include: a second detection subunit and a second filtering subunit.

[0174] The second detection subunit can be used to detect contour lines in the target image to obtain multiple contour lines in the target image; the target image is either the first image or the second image.

[0175] The second filtering subunit can be used to remove contour lines from multiple contour lines that contain fewer than a certain number of pixels, thereby obtaining a set of contour lines corresponding to the target image; the corresponding set of contour lines consists of the contour lines retained from the multiple contour lines; and / or,

[0176] The second filtering subunit can be used to remove contour lines whose image positions are located in the boundary region of the target image from multiple contour lines, so as to obtain a set of contour lines corresponding to the target image; the corresponding set of contour lines consists of the contour lines retained from the multiple contour lines.

[0177] In one embodiment, the contour point sampling unit can also be used to perform control point sampling on the target contour line, starting from the corner points, where the target contour line includes corner points; or...

[0178] The contour point sampling unit can also be used to sample control points evenly spaced along the target contour line, starting from any pixel, when the target contour line does not include corner points and is a non-closed contour line; or...

[0179] The contour point sampling unit can also be used to sample the centroid of the target contour as the control point of the target contour when the target contour does not include corner points and is a closed contour.

[0180] The target contour line can be any first contour line or any second contour line.

[0181] In one embodiment, the contour point sampling unit can also be used to perform uniformly spaced control point sampling of the target contour line, starting from the corner point and moving towards the endpoint of the target contour line when the target contour line includes a corner point; or...

[0182] The contour point sampling unit can also be used to sample control points between two adjacent corner points according to a first pixel interval when the target contour line includes at least two corner points, and to sample control points between adjacent endpoints and corner points of the target contour line according to a second pixel interval; the first pixel interval is greater than the second pixel interval.

[0183] In one embodiment, the calculation module 530 may include: a first determining unit, a second determining unit, a construction unit, and a transformation parameter calculation unit.

[0184] The first determining unit can be used to determine the feature point correspondence relationship between multiple feature point pairs according to the first feature point set and the second feature point set; each feature point pair includes a first feature point in the first feature point set and a second feature point in the second feature point set.

[0185] The second determining unit can be used to determine the first contour point set corresponding to each first contour line and the second contour point set corresponding to each second contour line; each contour point pair includes a first contour point in the first contour point set and a second contour point in the second contour point set.

[0186] The building unit can be used to construct a correspondence matrix based on the correspondence between multiple feature point pairs and the correspondence between multiple contour point pairs.

[0187] The transformation parameter calculation unit can be used to construct the feature point correspondence of feature point pairs based on the first Euclidean distance.

[0188] In one embodiment, the first determining unit can also be used to calculate, for each feature point pair, a first Euclidean distance between the first feature point included in the feature point pair and the second feature point included in the feature point pair; and to construct a feature point correspondence relationship of the feature point pair based on the first Euclidean distance.

[0189] In one embodiment, the second determining unit can also be configured to, for each pair of contour points, determine a first target contour point set to which the first contour points included in the contour point pair belong from a plurality of first contour point sets, and determine a second target contour point set to which the second contour points included in the contour point pair belong from a plurality of second contour point sets; and calculate the chamfer distance between the first target contour point set and the second target contour point set; and calculate the second Euclidean distance between the first contour points included in the contour point pair and the second contour points included in the contour point pair; and construct the contour point correspondence relationship of the contour point pair based on the chamfer distance and the second Euclidean distance.

[0190] In one embodiment, the second determining unit can be further configured to: for each first contour point included in the first target contour point set, determine a second contour point corresponding to each first contour point in the second target contour point set according to the nearest neighbor criterion; calculate a first average distance corresponding to the first target contour point set according to the Euclidean distance between each first contour point in the first target contour point set and the corresponding second contour point; for each second contour point included in the second target contour point set, determine a first contour point corresponding to each second contour point in the first target contour point set according to the nearest neighbor criterion; calculate a second average distance corresponding to the second target contour point set according to the Euclidean distance between each second contour point in the second target contour point set and the corresponding first contour point; and calculate a chamfer distance based on the first average distance and the second average distance.

[0191] In one embodiment, the image registration device may further include an update module.

[0192] The update module can be used to update the second feature point set and each second contour point set based on the spatial transformation parameters, and trigger the aforementioned calculation module 530 to repeatedly perform the operation of calculating the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line, until the iteration termination condition is met.

[0193] The registration module 540 can also be used to register the first image and the second image according to the spatial transformation parameters updated when the iteration end condition is met.

[0194] The image registration apparatus disclosed in the foregoing embodiments can calculate spatial transformation parameters for image registration by combining image feature information provided by feature points and contour edges. Feature points and contour edges can complement each other, making full use of their respective advantages, which can improve the accuracy of spatial transformation parameters, thereby improving the accuracy of image registration. In application scenarios with a large number of contour features, such as indoor shooting, the image registration apparatus can effectively fuse feature point and contour edge information to perform accurate image registration operations, and can adapt to various complex indoor shooting scenarios.

[0195] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device disclosed in one embodiment. For example... Figure 6 As shown, the electronic device 600 may include:

[0196] Memory 610 storing executable program code;

[0197] Processor 620 coupled to memory 610;

[0198] The processor 620 can call the executable program code stored in the memory 610 to execute all or part of the steps in any of the image registration methods described in the above embodiments.

[0199] Furthermore, embodiments of this application disclose a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program enables a computer to perform all or part of the steps in any of the image registration methods described in the above embodiments.

[0200] Furthermore, this application further discloses a computer program product that, when run on a computer, enables the computer to execute all or part of the steps in any of the image registration methods described in the above embodiments.

[0201] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0202] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0203] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0205] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0206] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0207] The image registration method, apparatus, electronic device, and storage medium disclosed in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image registration method, characterized in that, include: Feature point detection is performed on the first image and the second image respectively to obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image; Contour line detection is performed on the first image and the second image respectively to obtain the first contour point set corresponding to the first contour line in the first image and the second contour point set corresponding to the second contour line in the second image. Calculate the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line; The first image and the second image are registered according to the spatial transformation parameters. The first contour point set is obtained by sampling control points on the first contour line based on the type of the first contour line, and the second contour point set is obtained by sampling control points on the second contour line based on the type of the second contour line. The control points corresponding to the first contour point set are pixels selected from the first contour line, and the control points corresponding to the second contour point set are pixels selected from the second contour line.

2. The method according to claim 1, characterized in that, The step of performing feature point detection on the first image and the second image respectively to obtain a first feature point set corresponding to the first image and a second feature point set corresponding to the second image includes: Feature point detection is performed on the first image and the second image respectively to obtain the first feature point in the first image and the second feature point in the second image; Calculate the similarity between each first feature point in the first image and each second feature point in the second image; Based on the similarity, some feature points are removed from the first feature points included in the first image and the second feature points included in the second image to obtain a first feature point set consisting of the retained first feature points and a second feature point set consisting of the retained second feature points.

3. The method according to claim 1, characterized in that, The step of performing contour line detection on the first image and the second image respectively, and obtaining the first contour point set corresponding to the first contour line in the first image, and the second contour point set corresponding to the second contour line in the second image, includes: Contour line detection is performed on the first image and the second image respectively to obtain a first contour line set corresponding to the first image and a second contour line set corresponding to the second image; Control point sampling is performed on the first contour lines included in the first contour line set and the second contour lines included in the second contour line set to obtain a first contour point set corresponding to each first contour line and a second contour point set corresponding to each second contour line.

4. The method according to claim 3, characterized in that The step of performing contour line detection on the first image and the second image respectively to obtain a first contour line set corresponding to the first image and a second contour line set corresponding to the second image includes: Contour line detection is performed on the target image to obtain multiple contour lines in the target image; the target image includes the first image and / or the second image; From the plurality of contour lines, contour lines containing fewer than a certain number of pixels are removed to obtain a set of contour lines corresponding to the target image; and / or, contour lines whose image positions are located in the boundary region of the target image are removed from the plurality of contour lines to obtain a set of contour lines corresponding to the target image. The corresponding set of contour lines consists of the contour lines retained from the plurality of contour lines.

5. The method according to claim 3, characterized in that, The step of sampling control points for the first contour lines included in the first contour line set and the second contour lines included in the second contour line set includes: If the target contour line includes corner points, then control point sampling is performed on the target contour line starting from the corner points; or, If the target contour does not include corner points and is a non-closed contour, then starting from any pixel of the target contour, control point sampling with uniform pixel spacing is performed on the target contour; or, If the target contour does not include corner points and is a closed contour, then the centroid of the target contour is sampled as the control point of the target contour. The target contour line includes any one of the first contour lines and / or any one of the second contour lines.

6. The method according to claim 5, characterized in that, If the target contour line includes corner points, then control point sampling is performed on the target contour line starting from the corner points, including: If the target contour line includes a corner point, then starting from the corner point, control point sampling is performed on the target contour line at a first pixel interval towards the endpoint; or... If the target contour line includes at least two corner points, control points are sampled between two adjacent corner points at a second pixel interval, and control points are sampled between adjacent endpoints and corner points of the target contour line at a third pixel interval; the second pixel interval is greater than the third pixel interval.

7. The method according to any one of claims 1 to 6, characterized in that, The step of calculating spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line includes: Based on the first feature point set and the second feature point set, a feature point correspondence relationship is determined for multiple feature point pairs respectively; wherein, each feature point pair includes a first feature point in the first feature point set and a second feature point in the second feature point set; Based on the first set of contour points corresponding to each first contour line and the second set of contour points corresponding to each second contour line, a contour point correspondence relationship is determined for multiple contour point pairs respectively; each contour point pair includes a first contour point in the first set of contour points and a second contour point in the second set of contour points. A correspondence matrix is ​​constructed based on the correspondence between the feature points corresponding to the multiple feature point pairs and the correspondence between the contour points corresponding to the multiple contour point pairs. Calculate the spatial transformation parameters based on the correspondence matrix.

8. The method according to claim 7, characterized in that, The step of determining the correspondence between multiple feature point pairs based on the first feature point set and the second feature point set includes: For each pair of feature points, calculate the first Euclidean distance between the first feature point included in the pair and the second feature point included in the pair; The feature point correspondence of the feature point pair is constructed based on the first Euclidean distance.

9. The method according to claim 7, characterized in that, The step of determining the contour point correspondence relationship between multiple contour point pairs based on the first contour point set corresponding to each first contour line and the second contour point set corresponding to each second contour line includes: For each pair of contour points, a first target contour point set to which the first contour points included in the pair belong is determined from a plurality of first contour point sets, and a second target contour point set to which the second contour points included in the pair belong is determined from a plurality of second contour point sets; wherein, each first contour point set corresponds to a first contour line, and each second contour point set corresponds to a second contour line. Calculate the chamfer distance between the first target contour point set and the second target contour point set; Calculate the second Euclidean distance between the first contour point included in the contour point pair and the second contour point included in the contour point pair; The contour point correspondence of the contour point pair is constructed based on the chamfer distance and the second Euclidean distance.

10. The method according to claim 9, characterized in that, The calculation of the chamfer distance between the first target contour point set and the second target contour point set includes: For each first contour point included in the first target contour point set, a second contour point corresponding to each first contour point is determined in the second target contour point set according to the nearest neighbor criterion. Calculate the first average distance corresponding to the first target contour point set based on the Euclidean distance between each first contour point in the first target contour point set and the corresponding second contour point; For each second contour point included in the second target contour point set, the first contour point corresponding to each second contour point is determined in the first target contour point set according to the nearest neighbor criterion. Calculate the second average distance corresponding to the second target contour point set based on the Euclidean distance between each second contour point in the second target contour point set and the corresponding first contour point; The chamfer distance is calculated based on the first average distance and the second average distance.

11. The method according to claim 1, characterized in that, After calculating the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line, the method further includes: The second feature point set and each of the second contour point sets are updated based on the spatial transformation parameters, and the step of calculating the spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to each first contour line, and the second contour point set corresponding to each second contour line is re-executed until the iteration termination condition is met. And, the registration process of the first image and the second image according to the spatial transformation parameters includes: The first image and the second image are registered based on the updated spatial transformation parameters when the iteration termination condition is met.

12. An image registration device, characterized in that, include: The feature point detection module is used to perform feature point detection on the first image and the second image respectively, and obtain the first feature point set corresponding to the first image and the second feature point set corresponding to the second image. The contour detection module is used to perform contour line detection on the first image and the second image respectively, and obtain the first contour point set corresponding to the first contour line in the first image and the second contour point set corresponding to the second contour line in the second image. The calculation module is used to calculate spatial transformation parameters based on the first feature point set, the second feature point set, the first contour point set corresponding to the first contour line, and the second contour point set corresponding to the second contour line. A registration module is used to perform registration processing on the first image and the second image according to the spatial transformation parameters; The first contour point set is obtained by sampling control points on the first contour line based on the type of the first contour line, and the second contour point set is obtained by sampling control points on the second contour line based on the type of the second contour line. The control points corresponding to the first contour point set are pixels selected from the first contour line, and the control points corresponding to the second contour point set are pixels selected from the second contour line.

13. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

Images