Image Registration Method, Apparatus and Storage Medium

Abstract

The present application provides an image registration method, apparatus, and storage medium. The image registration method includes: obtaining repeated structures in a first image and a second image; the first image and the second image are two images of the same object; performing image registration based on the contour point position information corresponding to the repeated structures to obtain geometric transformation parameters between the first image and the second image. The image registration method, apparatus, and storage medium provided by the present application use the contour points corresponding to the repeated structures in the first image and the second image as the object features in the images, perform image registration based on the contour point position information, solve the problem of missing key points, and can be applied to various new scenarios, improving the accuracy of image registration.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to an image registration method, apparatus, and storage medium. Background Art

[0002] Image registration is an important underlying technology in the field of computer vision. Its task is to find the correspondence between two images and solve their geometric transformation. Image registration technology is widely used in scenarios such as medicine and remote sensing.

[0003] In recent years, image registration technology often uses a method that combines local feature descriptors and the Random Sample Consensus (RANSAC) algorithm. Such methods have standard implementations in OpenCV. With the explosion of deep learning, image registration methods have also started to use deep models to solve problems.

[0004] However, the local feature descriptors in existing methods cannot solve problems such as the lack of key points caused by smooth and reflective object surfaces, resulting in low accuracy of image registration; while the accuracy of methods based on deep models depends on the richness of training data. In new scenarios, due to the lack of data, the accuracy of image registration will also be low. Summary of the Invention

[0005] Embodiments of this application provide an image registration method, apparatus, and storage medium to solve the technical problem of low accuracy of image registration in the prior art.

[0006] In a first aspect, embodiments of this application provide an image registration method, including:

[0007] Obtain the repeated structures in the first image and the second image; the first image and the second image are two images of the same object;

[0008] Perform image registration based on the position information of the contour points corresponding to the repeated structures to obtain the geometric transformation parameters between the first image and the second image.

[0009] In some embodiments, the performing image registration based on the position information of the contour points corresponding to the repeated structures to obtain the geometric transformation parameters between the first image and the second image includes:

[0010] Based on the position information of the contour points corresponding to the repeated structures, use the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to the first contour point; the first contour point is the center point of the repeated structure located at the contour in the first image;

[0011] Based on the k-nearest neighbors, use the Random Sample Consensus RANSAC algorithm to determine the geometric transformation parameters for mapping the second image onto the first image.

[0012] In some embodiments, determining, based on the contour point position information corresponding to the repeating structure, the k-nearest neighbors corresponding to the first contour points by using the iterative closest point k-ICP algorithm includes:

[0013] Mapping a second contour point onto the first image based on the contour point position information corresponding to the repeating structure; the second contour point is the center point of the repeating structure located at the contour in the second image;

[0014] Determining, by using the iterative closest point k-ICP algorithm, the k-nearest neighbors corresponding to each first contour point from the mapped second contour points.

[0015] In some embodiments, the method further includes:

[0016] Obtaining initial position information of the repeating structure;

[0017] Initializing geometric transformation parameters between the first image and the second image according to the initial position information.

[0018] In some embodiments, the contour point position information includes coordinate information of the repeating structure located at the contour in the first image and coordinate information of the repeating structure located at the contour in the second image.

[0019] In some embodiments, determining, based on the k-nearest neighbors, geometric transformation parameters for mapping the second image onto the first image by using the random sample consensus RANSAC algorithm includes:

[0020] Detecting inliers among the k-nearest neighbors by using the random sample consensus RANSAC algorithm;

[0021] Determining geometric transformation parameters for mapping the second image onto the first image based on the position information of the inliers.

[0022] In some embodiments, the method further includes:

[0023] Obtaining the center points of the repeating structures located at the contour in the first image and the center points of the repeating structures located at the contour in the second image by using the connected component analysis method or the Alpha-Shape algorithm;

[0024] Taking the center points of the repeating structures located at the contour in the first image and the center points of the repeating structures located at the contour in the second image as the contour points corresponding to the repeating structure.

[0025] In a second aspect, an embodiment of the present application provides an image registration device, including:

[0026] A first acquisition module, configured to acquire a repeated structure in a first image and a second image; the first image and the second image are two images of the same object.

[0027] A registration module, configured to perform image registration based on the contour point position information corresponding to the repeated structure, and obtain geometric transformation parameters between the first image and the second image.

[0028] In some embodiments, the registration module includes:

[0029] A first determination unit, configured to determine k-nearest neighbors corresponding to a first contour point based on the contour point position information corresponding to the repeated structure, using the iterative closest point k-ICP algorithm; the first contour point is the center point of the repeated structure located at the contour in the first image.

[0030] A second determination unit, configured to determine geometric transformation parameters for mapping the second image onto the first image based on the k-nearest neighbors, using the random sample consensus RANSAC algorithm.

[0031] In some embodiments, the first determination unit includes:

[0032] A mapping subunit, configured to map a second contour point onto the first image based on the contour point position information corresponding to the repeated structure; the second contour point is the center point of the repeated structure located at the contour in the second image.

[0033] A first determination subunit, configured to determine k-nearest neighbors corresponding to each first contour point from the mapped second contour points, using the iterative closest point k-ICP algorithm.

[0034] In some embodiments, the first determination unit further includes:

[0035] An acquisition subunit, configured to acquire initial position information of the repeated structure.

[0036] An initialization subunit, configured to initialize geometric transformation parameters between the first image and the second image according to the initial position information.

[0037] In some embodiments, the contour point position information includes coordinate information of the repeated structure located at the contour in the first image, and coordinate information of the repeated structure located at the contour in the second image.

[0038] In some embodiments, the second determination unit includes:

[0039] A detection subunit, configured to detect inliers among the k-nearest neighbors, using the random sample consensus RANSAC algorithm.

[0040] A second determination subunit, configured to determine geometric transformation parameters for mapping a second image onto a first image based on the position information of the inliers.

[0041] In some embodiments, it further includes:

[0042] A second acquisition module, configured to acquire the center points of the repeated structures located at the contours in the first image and the center points of the repeated structures located at the contours in the second image by using a connected component analysis method or an Alpha-Shape algorithm;

[0043] A third acquisition module, configured to use the center points of the repeated structures located at the contours in the first image and the center points of the repeated structures located at the contours in the second image as the contour points corresponding to the repeated structures.

[0044] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the image registration method as described in the first aspect above.

[0045] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the image registration method as described in the first aspect above.

[0046] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the image registration method as described in the first aspect above.

[0047] The image registration method, device, and storage medium provided by the embodiments of the present application obtain repeated structures by performing repeated detection on a first image and a second image, and perform image registration based on the position information of the contour points corresponding to the repeated structures. Using the contour points corresponding to the repeated structures as the object features in the images improves the effectiveness of the obtained features, can be flexibly applied to various new scenarios, and ensures a high accuracy rate of image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is a flowchart of the image registration method provided by the embodiment of the present application;

[0050] Figure 2 It is a schematic structural diagram of an image registration device provided by an embodiment of the present application;

[0051] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Specific embodiments

[0052] In the prior art, the most popular image registration method is the method that combines local feature descriptors and the RANSAC algorithm. Among them, the most commonly used in local feature descriptors is the Scale-invariant feature transform (SIFT) algorithm. The following takes SIFT and Homography Matrix estimation as examples to illustrate the basic process of image registration.

[0053] Given two images containing the same scene or object, use the SIFT algorithm to find key points in the image pyramid, and estimate their scale and orientation information. Then, based on this information and pixel values, extract geometrically invariant feature vectors (e.g., 128-dimensional). Compare the feature vectors of M and N key points in the two images to obtain K pairs of successfully matched key points. When the number of key points is large, generally use methods such as Approximate Nearest Neighbor (ANN) to accelerate; due to the presence of noise in the K pairs of key points, generally use the RANSAC algorithm to iteratively detect inliers (correct matches) and outliers (incorrect matches), and at the same time estimate the geometric transformation parameters of the two images.

[0054] The image registration method based on a deep model directly regresses the geometric transformation parameters from the pixel values of the two images without relying on key point detection. Therefore, it can better handle smooth and reflective objects and has lower requirements for image quality. However, the generalization of the deep model depends on the richness of the training data, and its accuracy drops significantly for unseen scenarios.

[0055] However, local feature descriptors cannot solve the problem of missing key points caused by smooth and reflective object surfaces, nor can they handle the problem of a large number of "repeating structures" in the image. Although the deep model can skip the key point detection stage and avoid the above problems, its accuracy depends on the richness of the training data and it has low adaptability for long-tail applications.

[0056] Based on the above technical problems, an embodiment of the present application proposes an image registration method. By repeatedly detecting the first image and the second image to obtain repeated structures, and performing image registration based on the position information of the contour points corresponding to the repeated structures, using the contour points corresponding to the repeated structures as the object features in the image, the effectiveness of the obtained features is improved, which can be flexibly applied to various new scenarios, and a high accuracy of image registration is ensured; the problem of missing key points caused by the smooth and reflective surface of the object and the problem of registration errors caused by a large number of repeated structures are solved, and the accuracy of image registration is improved.

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0058] Figure 1 is a schematic flowchart of the image registration method provided by an embodiment of the present application. As Figure 1 shown, an embodiment of the present application provides an image registration method. The method includes:

[0059] Step 101, obtain the repeated structures in the first image and the second image; the first image and the second image are two images of the same object.

[0060] Specifically, the first image and the second image are two images of the same object in a certain application scenario, and these two images can be obtained under different camera positions, angles, illuminances, or time conditions respectively. When the first image is the moving image in image registration, the second image is the fixed image; when the first image is the fixed image, the second image is the moving image.

[0061] For the object in the image, there may be many structures with similar appearance features in the object, that is, repeated structures. First, it is necessary to obtain the repeated structures in the first image and the second image.

[0062] For example, a semantic segmentation algorithm based on a deep feature aggregation network (DFANet) or a pyramid scene parsing network (PSPNet) is used to perform semantic segmentation on the first image and the second image respectively to detect the repeated structures in the images.

[0063] For another example, an instance segmentation algorithm based on the deep model Mask-RCNN is used to detect duplicate structures in the first image and the second image. Mask-RCNN can outline the edges of each object segmented from the image, further improving the accuracy of duplicate structure segmentation.

[0064] Step 102: Perform image registration based on the contour point position information corresponding to the duplicate structure to obtain the geometric transformation parameters between the first image and the second image.

[0065] Specifically, after obtaining the duplicate structures in the first image and the second image, determine the duplicate structures at the contours in the first image and the second image, and then determine the contour point position information corresponding to the duplicate structures. Perform image registration based on the contour point position information to make the contour points corresponding to the same position in space in the first image and the second image correspond one by one, and obtain the geometric transformation parameters between the first image and the second image.

[0066] For example, use the Iterative Closed Point (ICP) algorithm combined with the Stochastic Gradient Descent (SGD) algorithm for iterative solution. First, use the ICP algorithm to map the contour points corresponding to the duplicate structures in the second image to the corresponding positions in the first image, establish the mapping relationship between the contour points corresponding to the duplicate structures in the first image and the contour points corresponding to the duplicate structures in the second image, and then use the SGD algorithm to estimate the geometric transformation parameters between the first image and the second image, and improve the stability by combining the pixel error loss function.

[0067] For another example, use the Iterative k-Closed Points (k-ICP) algorithm combined with the Random Sample Consensus (RANSAC) algorithm for iterative solution. First, use the k-ICP algorithm to find the mapping relationship between the contour points corresponding to the duplicate structures in the first image and the contour points corresponding to the duplicate structures in the second image. Among them, each contour point corresponding to the duplicate structure in the first image corresponds to multiple (k) contour points corresponding to the duplicate structures in the second image. Some of these multiple contour points corresponding to the duplicate structures in the second image are correctly matched points, that is, inliers, and some are incorrectly matched points, that is, outliers. Then, use the RANSAC algorithm to detect the inliers and estimate the geometric transformation parameters between the first image and the second image. Iterate the above steps until the model converges to obtain the final geometric transformation parameters between the first image and the second image.

[0068] The image registration method provided by the embodiments of the present application obtains the position information of the contour points by detecting the repeated structures in the images and performing contour analysis on them, and obtains the position information that best represents the object features in the images. Based on the position information of the contour points, the geometric transformation parameters between the images are obtained, improving the accuracy of image registration.

[0069] In some embodiments, the position information of the contour points includes the coordinate information of the repeated structures located at the contour in the first image and the coordinate information of the repeated structures located at the contour in the second image.

[0070] Specifically, in the embodiments of the present application, the contour points (or boundary points) include the contour points in the first image and the contour points in the second image. The contour points in the first image refer to the position points of the repeated structures at the contour in the first image, and the contour points in the second image refer to the position points of the repeated structures at the contour in the second image. The position information of the contour points includes the coordinate information of the contour points in the first image and the coordinate information of the contour points in the second image.

[0071] The image registration method provided by the embodiments of the present application performs image registration based on obtaining the position information of the contour points corresponding to the repeated structures, avoiding the situation of incorrect point pair matching caused by a large number of repeated structures in the images, and being able to better handle smooth and reflective objects, improving the accuracy of image registration in new scenarios.

[0072] In some embodiments, the method further includes:

[0073] Using the connected component analysis method or the Alpha-Shape algorithm to obtain the center points of the repeated structures located at the contour in the first image and the center points of the repeated structures located at the contour in the second image;

[0074] Taking the center points of the repeated structures located at the contour in the first image and the center points of the repeated structures located at the contour in the second image as the contour points corresponding to the repeated structures.

[0075] Specifically, in the embodiments of the present application, since the appearance features of each repeated structure are almost the same, only the position information of the repeated structures is matched. And in the position matching process, the information of the contour points is the richest and the ambiguity is the smallest. Therefore, methods such as the connected component analysis or the contour point extraction algorithm Alpha-Shape are used to locate the repeated structures located at the contour.

[0076] First, use the connected component analysis method or the Alpha-Shape algorithm to obtain the center points of the repeated structures located at the contour in the first image and the center points of the repeated structures located at the contour in the second image, and then take these center points as the contour points corresponding to the repeated structures.

[0077] For example, after obtaining the repeating structures in the first image and the second image, perform connected component analysis on the repeating structures in the first image and the second image, retain the connected components that meet the preset contour conditions, record the repeating structures at the boundaries in the connected components, and use the center points of these repeating structures as contour points.

[0078] For another example, after obtaining the repeating structures in the first image and the second image, use the Alpha-Shape algorithm to extract the repeating structures at the boundaries, and use the center points of these repeating structures as contour points.

[0079] The image registration method provided by the embodiments of the present application locates the repeating structures at the contour through various methods such as connected component analysis or Alpha-Shape, thereby determining the contour points corresponding to the repeating structures, improving the flexibility of the application, and being applicable to different scenario requirements.

[0080] In some embodiments, the method further includes:

[0081] Obtain the initial position information of the repeating structures;

[0082] Initialize the geometric transformation parameters between the first image and the second image according to the initial position information.

[0083] Specifically, before performing image registration, it is necessary to initialize the geometric transformation parameters between the first image and the second image based on the initial position information of the repeating structures. The initial position information is the rough positioning information of the repeating structures in the image, such as the bounding box of the whole formed by all the repeating structures in the image, or the minimum bounding box containing all the repeating structures in the image and other rough positioning information.

[0084] For example, obtain the minimum bounding box containing all the repeating structures in each image, and obtain the minimum bounding box of the repeating structures in the first image and the minimum bounding box of the repeating structures in the second image. Initialize the geometric transformation parameters between the first image and the second image according to the position of the minimum bounding box of the repeating structures in the first image and the position of the minimum bounding box of the repeating structures in the second image.

[0085] The image registration method provided by the embodiments of the present application initializes the geometric transformation parameters between the first image and the second image through the rough positioning information, ensuring the correct mapping between the first image and the second image.

[0086] In some embodiments, the performing image registration based on the position information of the contour points corresponding to the repeating structures to obtain the geometric transformation parameters between the first image and the second image includes:

[0087] Based on the contour point position information corresponding to the repeated structure, use the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to the first contour point; the first contour point is the center point of the repeated structure located at the contour in the first image.

[0088] Based on the k-nearest neighbors, use the random sample consensus RANSAC algorithm to determine the geometric transformation parameters for mapping the second image onto the first image.

[0089] Specifically, after initializing the geometric transformation parameters between the first image and the second image, based on the determined contour point position information corresponding to the repeated structure, use the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to the first contour point, and establish a one-to-many mapping relationship, where some mappings are inliers of correct mappings and some are outliers of incorrect mappings. The first contour point is the center point of the repeated structure located at the contour in the first image. Then, based on the k-nearest neighbors, use the RANSAC algorithm to detect the correct mappings and estimate the geometric transformation parameters for mapping the second image onto the first image.

[0090] Iterate the above steps until the model converges to obtain the final geometric transformation parameters between the first image and the second image.

[0091] The image registration method provided by the embodiments of the present application determines the k-nearest neighbors corresponding to the first contour point through the iterative closest point k-ICP algorithm, establishes a one-to-many mapping relationship, improves the registration accuracy, reduces the number of iterations, and reduces the complexity. And use the RANSAC algorithm to calculate the correct geometric transformation parameters from the noisy data, improving the accuracy of image registration.

[0092] In some embodiments, the step of determining the k-nearest neighbors corresponding to the first contour point based on the contour point position information corresponding to the repeated structure and using the iterative closest point k-ICP algorithm includes:

[0093] Map the second contour point onto the first image based on the contour point position information corresponding to the repeated structure; the second contour point is the center point of the repeated structure located at the contour in the second image.

[0094] Use the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to each first contour point from the mapped second contour points.

[0095] Specifically, for the determination of the k-nearest neighbors corresponding to the first contour point, first map the second contour point onto the first image based on the contour point position information corresponding to the repeated structure. The second contour point is the center point of the repeated structure located at the contour in the second image. Then use the k-ICP algorithm to determine the k-nearest neighbors corresponding to each first contour point from the mapped second contour points.

[0096] The image registration method provided by the embodiment of the present application determines the k-nearest neighbors corresponding to each first contour point through the k-ICP algorithm, can obtain a reasonable proportion of inliers and outliers, establish a more reasonable one-to-many mapping relationship, improve the accuracy of image registration, reduce the number of iterations at the same time, and reduce the complexity.

[0097] In some embodiments, using the random sample consensus (RANSAC) algorithm based on the k-nearest neighbors to determine the geometric transformation parameters for mapping the second image onto the first image includes:

[0098] Detecting inliers among the k-nearest neighbors using the RANSAC algorithm;

[0099] Determining the geometric transformation parameters for mapping the second image onto the first image based on the position information of the inliers.

[0100] Specifically, after obtaining the k-nearest neighbors corresponding to each first contour point, use the RANSAC algorithm to detect the inliers among the k-nearest neighbors, and determine the geometric transformation parameters for mapping the second image onto the first image based on the position information of the inliers.

[0101] The image registration method provided by the embodiment of the present application calculates the correct geometric transformation parameters from the noise data using the RANSAC algorithm, improving the accuracy of image registration. By repeatedly detecting the first image and the second image to obtain the repeated structures, and performing image registration based on the position information of the contour points corresponding to the repeated structures, using the contour points corresponding to the repeated structures as the object features in the image, the effectiveness of the obtained features is improved, it can be flexibly applied to various new scenarios, and a high accuracy of image registration is ensured; it solves the problem of missing key points caused by the smooth and reflective surface of the object and the problem of registration errors caused by a large number of repeated structures, improving the accuracy of image registration.

[0102] Figure 2 It is a schematic structural diagram of an image registration device provided by the embodiment of the present application, as Figure 2 shown, the embodiment of the present application provides an image registration device, including a first acquisition module 201 and a registration module 202.

[0103] The first acquisition module 201 is used to acquire the repeated structures in the first image and the second image; the first image and the second image are two images of the same object.

[0104] The registration module 202 is used to perform image registration based on the position information of the contour points corresponding to the repeated structures to obtain the geometric transformation parameters between the first image and the second image.

[0105] In some embodiments, the registration module includes:

[0106] A first determination unit, configured to determine k nearest neighbors corresponding to a first contour point by using an iterative closest point k-ICP algorithm based on the contour point position information corresponding to the repeating structure; the first contour point is the center point of the repeating structure located at the contour in the first image.

[0107] A second determination unit, configured to determine geometric transformation parameters for mapping the second image onto the first image based on the k nearest neighbors by using a random sample consensus RANSAC algorithm.

[0108] In some embodiments, the first determination unit includes:

[0109] A mapping subunit, configured to map a second contour point onto the first image based on the contour point position information corresponding to the repeating structure; the second contour point is the center point of the repeating structure located at the contour in the second image.

[0110] A first determination subunit, configured to determine k nearest neighbors corresponding to each first contour point from the mapped second contour points by using an iterative closest point k-ICP algorithm.

[0111] In some embodiments, the first determination unit further includes:

[0112] An acquisition subunit, configured to acquire initial position information of the repeating structure.

[0113] An initialization subunit, configured to initialize geometric transformation parameters between the first image and the second image according to the initial position information.

[0114] In some embodiments, the contour point position information includes coordinate information of the repeating structure located at the contour in the first image and coordinate information of the repeating structure located at the contour in the second image.

[0115] In some embodiments, the second determination unit includes:

[0116] A detection subunit, configured to detect inliers among the k nearest neighbors by using a random sample consensus RANSAC algorithm.

[0117] A second determination subunit, configured to determine geometric transformation parameters for mapping the second image onto the first image based on the position information of the inliers.

[0118] In some embodiments, it further includes:

[0119] A second acquisition module, configured to acquire the center points of the repeating structures located at the contours in the first image and the center points of the repeating structures located at the contours in the second image by using a connected component analysis method or an Alpha-Shape algorithm.

[0120] A third acquisition module, configured to use the center points of the repeating structures located at the contours in the first image and the center points of the repeating structures located at the contours in the second image as the contour points corresponding to the repeating structures.

[0121] Specifically, the image registration device provided in the embodiments of the present application can implement all the method steps implemented in the embodiments of the above image registration method, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0122] It should be noted that the division of units / modules in the above embodiments of the present application is illustrative, merely a logical function division, and there may be other division methods in actual implementation. In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated units may be implemented in the form of hardware or in the form of software functional units.

[0123] Figure 3 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. As Figure 3 shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304. The processor 301 can call the logical instructions in the memory 303 to execute an image registration method, and the method includes:

[0124] Obtain repeating structures in a first image and a second image; the first image and the second image are two images of the same object;

[0125] Perform image registration based on the position information of the contour points corresponding to the repeating structures to obtain geometric transformation parameters between the first image and the second image.

[0126] Specifically, the processor 301 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or a Complex Programmable Logic Device (CPLD). The processor may also adopt a multi-core architecture.

[0127] When the logical instructions in the memory 303 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disks, or optical discs that can store program codes.

[0128] In some embodiments, a computer program product is also provided. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image registration method provided in each of the above method embodiments. The method includes:

[0129] Obtain the repeated structures in the first image and the second image; the first image and the second image are two images of the same object;

[0130] Perform image registration based on the contour point position information corresponding to the repeated structures to obtain the geometric transformation parameters between the first image and the second image.

[0131] Specifically, the above computer program product provided by the embodiments of this application can implement all the method steps implemented by each of the above method embodiments and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described in this embodiment.

[0132] In some embodiments, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and the computer program is used to cause a computer to execute the image registration method provided in each of the above method embodiments. The method includes:

[0133] Obtain the repeating structures in the first image and the second image; the first image and the second image are two images of the same object;

[0134] Perform image registration based on the contour point position information corresponding to the repeating structures to obtain the geometric transformation parameters between the first image and the second image.

[0135] Specifically, the above computer-readable storage medium provided in the embodiments of the present application can implement all the method steps implemented in each of the above method embodiments, and can achieve the same technical effects. The same parts and beneficial effects as those in the method embodiments will not be specifically described herein again.

[0136] It should be noted that: The computer-readable storage medium can be any available medium or data storage device that can be accessed by a processor, including but not limited to magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical discs (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid state drives (SSD)).

[0137] In addition, it should be noted that: The terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or multiple.

[0138] The term "and / or" in the embodiments of the present application describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0139] The term "plurality" in the embodiments of the present application refers to two or more, and other quantifiers are similar thereto.

[0140] "Determining B based on A" in this application means that the factor A should be considered when determining B. It is not limited to "determining B only based on A", but also includes: "determining B based on A and C", "determining B based on A, C, and E", "determining C based on A and further determining B based on C", etc. Additionally, it can also include using A as a condition for determining B. For example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B"; for yet another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be using A as a condition for the factor of determining B. For example, "when A meets the first condition, use the first method to determine C and further determine B based on C", etc.

[0141] Those skilled in the art should understand that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) that contain computer-usable program code.

[0142] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0143] These processor-executable instructions can also be stored in a processor-readable memory that can guide the computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the processor-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0144] These processor-executable instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the processing in the process Figure 1 a process or processes and / or blocks Figure 1 steps for implementing the functions specified in a block or blocks.

[0145] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. An image registration method, characterized in that, it includes: Obtaining repeated structures in a first image and a second image; the first image and the second image are two images of the same object under different camera positions, angles, illuminations or time conditions; the repeated structures are multiple structures with approximate appearance features that make up the object; Based on the position information of the contour points corresponding to the repeated structures, performing association mapping on the contour points corresponding to the repeated structures in the first image and the contour points corresponding to the repeated structures in the second image to obtain geometric transformation parameters between the first image and the second image.

2. The image registration method according to claim 1, characterized in that, The performing association mapping on the contour points corresponding to the repeated structures in the first image and the contour points corresponding to the repeated structures in the second image based on the position information of the contour points corresponding to the repeated structures to obtain geometric transformation parameters between the first image and the second image includes: Based on the position information of the contour points corresponding to the repeated structures, using the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to the first contour point; the first contour point is the center point of the repeated structure located at the contour in the first image; Based on the k-nearest neighbors, using the random sample consensus RANSAC algorithm to determine the geometric transformation parameters for mapping the second image onto the first image.

3. The image registration method according to claim 2, characterized in that, The based on the position information of the contour points corresponding to the repeated structures, using the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to the first contour point includes: Based on the position information of the contour points corresponding to the repeated structures, mapping the second contour point onto the first image; the second contour point is the center point of the repeated structure located at the contour in the second image; Using the iterative closest point k-ICP algorithm to determine the k-nearest neighbors corresponding to each first contour point from the mapped second contour points.

4. The image registration method according to claim 3, characterized in that, The method further includes: Obtaining the initial position information of the repeated structure; Initializing the geometric transformation parameters between the first image and the second image according to the initial position information.

5. The image registration method according to any one of claims 1-3, characterized in that, The contour point position information includes the coordinate information of the repeated structures located at the contour in the first image and the coordinate information of the repeated structures located at the contour in the second image.

6. The image registration method according to claim 2, characterized in that, The based on the k-nearest neighbors, using the random sample consensus RANSAC algorithm to determine the geometric transformation parameters for mapping the second image onto the first image includes: Using the random sample consensus RANSAC algorithm to detect the inliers among the k-nearest neighbors; Based on the position information of the inliers, determining the geometric transformation parameters for mapping the second image onto the first image.

7. The image registration method according to claim 1, characterized in that, The method further includes: Obtain the center points of the repetitive structures located at the contours in the first image and the center points of the repetitive structures located at the contours in the second image by using the connected component analysis method or the Alpha-Shape algorithm; Use the center points of the repetitive structures located at the contours in the first image and the center points of the repetitive structures located at the contours in the second image as the contour points corresponding to the repetitive structures.

8. An image registration device Characterized in that It includes: A first acquisition module, configured to acquire repetitive structures in a first image and a second image; the first image and the second image are two images of the same object under different camera positions, angles, illuminations or time conditions; the repetitive structures are multiple structures with approximate appearance features that make up the object; A registration module, configured to perform association mapping on the contour points corresponding to the repetitive structures in the first image and the contour points corresponding to the repetitive structures in the second image based on the position information of the contour points corresponding to the repetitive structures, to obtain the geometric transformation parameters between the first image and the second image.

9. An electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor, Characterized in that When the processor executes the program, it implements the image registration method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, on which a computer program is stored, Characterized in that When the computer program is executed by a processor, it implements the image registration method according to any one of claims 1 to 7.

Images