Image registration method, device, computer equipment and storage medium

Abstract

The present application relates to an image registration method, device, computer equipment and storage medium. The method comprises: performing different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched; obtaining a reference image and determining at least two local feature areas in the reference image; determining candidate areas that match each local feature area respectively from each of the images to be matched, and filtering out a target matching image from the images to be matched according to the candidate areas; determining the target transformation parameters between the reference image and the image to be registered according to the candidate areas that match the at least two local feature areas in the target matching image; registering the images to be registered based on the target transformation parameters to obtain the registered images. The use of this method can improve the accuracy of image registration.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image registration method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of computer technology, image registration technology has emerged. Image registration is the process of matching and superimposing two or more images acquired at different times, with different imaging devices or under different conditions (such as climate, illumination, camera position and angle, etc.). It has been widely used in remote sensing data analysis, computer vision, image processing and other fields.

[0003] However, traditional image registration methods are prone to matching errors when image features are close, the image is blurred, or the feature points are very concentrated, resulting in inaccurate image registration. Summary of the invention

[0004] Based on this, it is necessary to provide an image registration method, device, computer equipment and storage medium that can improve accuracy in response to the above technical problems.

[0005] An image registration method, the method comprising:

[0006] Perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched;

[0007] Acquire a reference image, and determine at least two local feature areas in the reference image;

[0008] Determining candidate regions that match each local feature region from each of the images to be matched, and screening target matching images from the images to be matched based on the candidate regions;

[0009] Determining target transformation parameters between the reference image and the image to be registered according to the candidate region in the target matching image that matches the at least two local feature regions;

[0010] The image to be registered is registered based on the target transformation parameter to obtain a registered image.

[0011] An image registration device, comprising:

[0012] A transformation module is used to perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched;

[0013] An acquisition module, used for acquiring a reference image and determining at least two local feature areas in the reference image;

[0014] A screening module, used to determine candidate regions that match each local feature region from each of the images to be matched, and screen target matching images from the images to be matched based on the candidate regions;

[0015] A determination module, configured to determine target transformation parameters between the reference image and the image to be registered according to a candidate region in the target matching image that matches the at least two local feature regions;

[0016] The registration module is used to register the image to be registered based on the target transformation parameters to obtain a registered image.

[0017] In one embodiment, the transformation module is also used to obtain image transformation parameters corresponding to multiple transformation methods, and the image transformation parameters include at least one of rotation parameters and scaling parameters; based on each of the image transformation parameters, the image to be registered is subjected to image transformation processing to obtain multiple images to be matched.

[0018] In one embodiment, the acquisition module is further used to acquire a reference image, divide the reference image into at least two reference frames, and obtain a local feature region contained in each reference frame;

[0019] The screening module is further used to determine, for each image to be matched, a candidate frame corresponding to each reference frame in the image to be matched, and perform an outward expansion process on the candidate frame; for each outward expanded candidate frame, search the outward expanded candidate frame for an area that matches the local feature area contained in the corresponding reference frame as the candidate area that matches the local feature area.

[0020] In one embodiment, the screening module is also used to traverse the area in the corresponding expanded candidate frame through the local feature area in the reference frame, and determine the confidence corresponding to the traversed area in each traversal; and use the area corresponding to the confidence that meets the confidence condition as the candidate area matching the local feature area.

[0021] In one embodiment, the screening module is also used to determine, for each image to be matched, candidate regions that match each local feature region respectively from the corresponding image to be matched, and determine the candidate similarities corresponding to each candidate region; based on the candidate similarities of each candidate region included in each image to be matched, screen out the target matching image from the image to be matched.

[0022] In one embodiment, the screening module is also used to, for each local feature region, use the highest similarity among the candidate similarities between the same local feature region and each matching candidate region as the target similarity corresponding to the corresponding local feature region; when the candidate regions corresponding to the target similarities are in the same image to be matched, use the same image to be matched as the target matching image; when the candidate regions corresponding to the target similarities are in different images to be matched, determine the number of target similarities corresponding to each image to be matched in the different images to be matched, and use the image to be matched with the largest number of corresponding target similarities as the target image.

[0023] In one embodiment, the determination module is also used to determine the reference center point of each of the at least two local feature areas, and the candidate center point of each candidate area in the target matching image; the reference center point of each of the local feature areas and the candidate center point in the corresponding candidate area are used as a matching point pair, and the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pairs.

[0024] In one embodiment, the determination module is also used to obtain target image transformation parameters corresponding to the transformation method corresponding to the target matching image; perform image inverse transformation processing on the target matching image based on the target image transformation parameters to obtain an inverse transformation image; and determine the target transformation parameters between the reference image and the image to be registered based on the position of the candidate center point in the matching point pair in the inverse transformation image and the position of the reference center point in the corresponding local feature area in the reference image.

[0025] In one embodiment, the determination module is further used to combine the matching point pairs, each combination including at least two matching point pairs; for each combination, based on the matching point pairs included in the combination, the combined transformation parameters between the reference image and the image to be registered are calculated to obtain the combined transformation parameters corresponding to each combination; for each combined transformation parameter, it is determined whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination to obtain the first applicable quantity corresponding to each combined transformation parameter; the combined transformation parameters corresponding to the first applicable quantity that meets the first preset quantity condition are used as the target transformation parameters between the reference image and the image to be registered.

[0026] In one embodiment, the determination module is also used to obtain, for each combination, the positions of candidate center points in the remaining matching point pairs except the matching point pairs included in the combination, and transform the obtained candidate center points based on the combination transformation parameters corresponding to the corresponding combination to obtain the combined transformation position; for each remaining matching point pair corresponding to each combination, respectively calculate the first error between the combined transformation position in each remaining matching point pair and the reference position of the corresponding reference center point; based on the first error, determine the first applicable number of matching point pairs to which the combination transformation parameters of the corresponding combination are applicable.

[0027] In one embodiment, the determination module is further used to select at least two matching point pairs from each matching point pair, calculate candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time; determine a second error generated by the matching point pairs that are not selected this time under the candidate transformation parameters, and determine a second applicable number of matching point pairs applicable to the candidate transformation parameters based on the second error; continue to select at least two matching point pairs, return to the step of calculating the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time and continue to execute until a preset stop condition is reached, and obtain the second applicable number corresponding to each candidate transformation parameter; and use the candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition as the target transformation parameters between the reference image and the image to be registered.

[0028] In one embodiment, the image to be registered and the reference image are both images of the same industrial device; the reference image is an image taken after debugging of the industrial device is completed, and each of the at least two local feature areas in the reference image is unique.

[0029] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched;

[0031] Acquire a reference image, and determine at least two local feature areas in the reference image;

[0032] Determining candidate regions that match each local feature region from each of the images to be matched, and screening target matching images from the images to be matched based on the candidate regions;

[0033] Determining target transformation parameters between the reference image and the image to be registered according to the candidate region in the target matching image that matches the at least two local feature regions;

[0034] The image to be registered is registered based on the target transformation parameter to obtain a registered image.

[0035] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0036] Perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched;

[0037] Acquire a reference image, and determine at least two local feature areas in the reference image;

[0038] Determining candidate regions that match each local feature region from each of the images to be matched, and screening target matching images from the images to be matched based on the candidate regions;

[0039] Determining target transformation parameters between the reference image and the image to be registered according to the candidate region in the target matching image that matches the at least two local feature regions;

[0040] The image to be registered is registered based on the target transformation parameter to obtain a registered image.

[0041] A computer program product or computer program includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the following steps: performing different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched; obtaining a reference image and determining at least two local feature areas in the reference image; determining candidate areas that match each local feature area respectively from each of the images to be matched, and screening out a target matching image from the images to be matched based on the candidate areas; determining target transformation parameters between the reference image and the image to be registered based on the candidate areas that match the at least two local feature areas in the target matching image; and registering the images to be registered based on the target transformation parameters to obtain a registered image.

[0042] The above-mentioned image registration method, device, computer equipment, storage medium and computer program, by performing different image transformation processing on the image to be registered according to multiple transformation methods, obtain multiple images to be matched, determine at least two local feature areas in the reference image, and search for corresponding matching candidate areas in the image to be matched through the local feature areas of the reference image, and can accurately screen out the target matching image that is most similar to the reference image from multiple images to be matched based on the matching areas between the images. And screening based on the image area can effectively avoid the problem of inaccurate matching due to the similarity of a single feature point, thereby improving the accuracy of screening the target matching image. According to the candidate area in the target matching image that matches at least two local feature areas, the target transformation parameters between the reference image and the image to be registered are accurately calculated, so that the image to be registered can be more accurately registered based on the target transformation parameters, thereby obtaining the image after being registered to the reference image. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A diagram of an application environment of an image registration method in an embodiment;

[0044] Figure 2 is a schematic diagram of a flow chart of an image registration method in one embodiment;

[0045] Figure 3 A schematic diagram of a process of selecting a target matching image from images to be matched based on the candidate similarities of each candidate region included in each image to be matched in one embodiment;

[0046] Figure 4 is a schematic diagram of a flow chart of determining target transformation parameters between a reference image and an image to be registered based on matching point pairs in another embodiment;

[0047] Figure 5 It is a schematic diagram of a process of determining target transformation parameters between a reference image and an image to be registered based on matching point pairs in one embodiment;

[0048] Figure 6 is a schematic diagram of a flow chart of determining target transformation parameters between a reference image and an image to be registered based on matching point pairs in another embodiment;

[0049] Figure 7 A schematic diagram of a process of an image registration method based on template matching in one embodiment;

[0050] Figure 8 is a schematic diagram of comparing a reference image and an image to be matched in one embodiment;

[0051] Fig. 9 A schematic diagram of a process applied to an industrial quality inspection scenario in an embodiment;

[0052] Fig.10 is a structural block diagram of an image registration device in one embodiment;

[0053] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The present application relates to the field of artificial intelligence (AI) technology, wherein artificial intelligence is the theory, method, technology and application system of simulating, extending and expanding human intelligence, perceiving the environment, acquiring knowledge and using knowledge to obtain the best results using digital computers or machines controlled by digital computers. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The scheme provided in the embodiments of the present application relates to an image registration method of artificial intelligence, which is specifically described by the following embodiments.

[0056] The image registration method provided in this application can be applied to Figure 1 The image registration system shown in Figure 1 is shown in Figure 2. Figure 1As shown, the image registration system includes a terminal 110 and a server 120. In one embodiment, the terminal 110 and the server 120 can each independently perform the image registration method provided in the embodiment of the present application. The terminal 110 and the server 120 can also be used in conjunction to perform the image registration method provided in the embodiment of the present application. When the terminal 110 and the server 120 are used in conjunction to perform the image registration method provided in the embodiment of the present application, the terminal 110 obtains the image to be registered and the reference image, and sends the image to be registered and the reference image to the server 120. The server 120 performs different image transformation processing on the image to be registered according to a variety of transformation methods to obtain a plurality of images to be matched. The server 120 determines at least two local feature areas in the reference image, determines candidate areas that match each local feature area from each image to be matched, and selects a target matching image from the image to be matched according to the candidate areas. The server 120 determines the target transformation parameters between the reference image and the image to be registered according to the candidate areas that match at least two local feature areas in the target matching image. The server 120 registers the image to be registered based on the target transformation parameters to obtain a registered image, and returns the registered image to the terminal 120 .

[0057] The server 120 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services or a cloud server cluster composed of multiple cloud servers. The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited thereto. The terminal 110 and the server 120 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0058] In one embodiment, multiple servers may form a blockchain, and the servers are nodes on the blockchain.

[0059] In one embodiment, data related to the image registration method can be stored on the blockchain, such as the image to be registered, multiple transformation methods, the image to be matched, the reference image, the target transformation parameters, and the registered image. Similarly, data related to the recognition model training method can also be stored on the blockchain.

[0060] It should be noted that the quantities such as "multiple" mentioned in the embodiments of the present application all refer to the quantity of "at least two", for example, "multiple" refers to at least two, "multiple" refers to at least two, and "multiple" refers to at least two.

[0061] In one embodiment, Figure 2As shown, an image registration method is provided, and the method is applied to a computer device (the computer device can be specifically Figure 1 The terminal or server in the example is used to illustrate, including the following steps:

[0062] Step S202 , performing different image transformation processes on the image to be registered according to a plurality of transformation methods to obtain a plurality of images to be matched.

[0063] The image to be registered is an image that needs to be registered. The image to be registered can be any one of an RGB (Red, Green, Blue) image, a grayscale image, a depth image, an image corresponding to the Y component in a YUV image, etc., but is not limited thereto. The "Y" in the YUV image represents brightness (Luminance or Luma), that is, the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. The image to be registered can be an image taken of any scene, such as a person image, a landscape image, or an industrial device image, but is not limited thereto.

[0064] The image to be matched is an image obtained by transforming the image to be registered. Different transformation methods can obtain different images to be matched. The image transformation method includes at least one of rotation transformation and scale transformation. At least one of the direction and size of the image to be registered can be changed by image transformation. The rotation transformation includes transformation of different rotation parameters, and the scale transformation includes transformation of different scaling parameters.

[0065] Specifically, the computer device can obtain the image to be registered and obtain at least two image transformation modes, and perform image transformation processing on the image to be registered according to each transformation mode to obtain the image to be matched corresponding to each transformation mode.

[0066] Step S204: acquiring a reference image, and determining at least two local feature regions in the reference image.

[0067] The reference image is a comparison image used to register the image to be registered, and both the reference image and the image to be registered are images obtained by shooting the same scene. The reference image can be any one of an RGB image, a grayscale image, a depth image, an image corresponding to the Y component in a YUV image, etc., but is not limited thereto.

[0068] Specifically, the computer device obtains a reference image and selects at least two local feature regions from the reference image. Any two of the at least two local feature regions may have the same part or may be different from each other.

[0069] In one embodiment, a scene may be photographed by a terminal to obtain frames of images, the first frame of image is used as a reference image, and the remaining frames of image except the first frame of image are used as images to be registered.

[0070] Step S206 , determining candidate regions that match each local feature region from each to-be-matched image, and selecting a target matching image from the to-be-matched images based on the candidate regions.

[0071] Specifically, after the computer device determines the candidate regions that match each local feature region in each image to be matched, it screens out a target matching image from multiple images to be matched according to the candidate regions included in each image to be matched.

[0072] In one embodiment, the computer device calculates the similarity between each candidate region and the matching local feature region, and selects the target matching image from the images to be matched based on the similarity.

[0073] Step S208, determining target transformation parameters between the reference image and the image to be registered according to the candidate region in the target matching image that matches the at least two local feature regions.

[0074] Specifically, the computer device selects feature points from each candidate area in the target matching image and at least two local feature areas in the reference image, and selects feature points from the corresponding local feature areas to form matching point pairs. Based on the obtained matching point pairs, the target transformation parameters between the reference image and the image to be registered are calculated.

[0075] Step S210 , registering the image to be registered based on the target transformation parameters to obtain a registered image.

[0076] Among them, the alignment and registration processing refers to the process of aligning two or more images in spatial position to match and superimpose two or more images acquired at different times, different imaging devices or different conditions.

[0077] Specifically, after obtaining the target transformation parameters, the computer device can obtain each pixel in the image to be registered, and map each pixel to the same image space as the reference image based on the target transformation parameters to obtain the registered image.

[0078] In one embodiment, the computer device may extract feature points from the image to be registered, and map each feature point to the same image space as the reference image based on the target transformation parameters to obtain a registered image.

[0079] In one embodiment, the reference image and each to-be-matched image are respectively subjected to histogram equalization processing to enhance the contrast of the images and effectively improve the robustness against illumination changes. Step S206 and subsequent steps are performed after the histogram equalization processing is performed.

[0080] In the above-mentioned image registration method, by performing different image transformation processing on the image to be registered according to multiple transformation methods, multiple images to be matched are obtained, and at least two local feature areas in the reference image are determined, so that the corresponding matching candidate areas are searched in the image to be matched through the local feature areas of the reference image, and the target matching image that is most similar to the reference image can be accurately screened out from the multiple images to be matched based on the matching areas between the images. And the screening based on the image area can effectively avoid the problem of inaccurate matching due to the similarity of a single feature point, thereby improving the accuracy of the target matching image screening. According to the candidate areas in the target matching image that match at least two local feature areas, the target transformation parameters between the reference image and the image to be registered are accurately calculated, so that the image to be registered can be more accurately registered based on the target transformation parameters, thereby obtaining the image after being registered to the reference image.

[0081] In one embodiment, the image to be registered is subjected to different image transformation processes according to a plurality of transformation modes to obtain a plurality of images to be matched, including:

[0082] The image transformation parameters corresponding to the plurality of transformation modes are obtained, wherein the image transformation parameters include at least one of a rotation parameter and a scaling parameter; and image transformation processing is performed on the images to be registered based on the image transformation parameters to obtain a plurality of images to be matched.

[0083] The image transformation parameters include at least one of a rotation parameter and a scaling parameter, and different transformation modes correspond to different image transformation parameters. For example, the rotation parameter is The scaling parameter is Image transformation parameters can include rotation parameters Any parameter in , or can include scaling parameters Any one of the parameters may also include a combination of the two, such as the image transformation parameters include the rotation parameter R and the scaling parameter S.

[0084] Specifically, the computer device obtains multiple transformation modes and obtains image transformation parameters corresponding to each transformation mode. For each transformation mode, the image to be registered is subjected to image transformation processing according to the image transformation parameters corresponding to each transformation mode to obtain the image to be matched corresponding to each image transformation mode.

[0085] Furthermore, for each transformation mode, the computer device can obtain the rotation parameter and the scaling parameter corresponding to the transformation mode, perform a rotation transformation on the image to be registered according to the rotation parameter, and then perform a scale transformation on the image after the rotation transformation according to the scaling parameter to obtain the image to be registered corresponding to the transformation mode. According to the same process, the image to be registered corresponding to each transformation mode can be obtained.

[0086] In one embodiment, for each transformation mode, the computer device may obtain the rotation parameter and the scaling parameter corresponding to the transformation mode, scale the image to be registered according to the scaling parameter, and then rotate the scale-transformed image according to the rotation parameter to obtain the image to be registered corresponding to the transformation mode. According to the same process, the image to be registered corresponding to each transformation mode can be obtained.

[0087] In this embodiment, image transformation parameters corresponding to a plurality of transformation modes are obtained, and the image transformation parameters include at least one of a rotation parameter and a scaling parameter. Image transformation processing is performed on the image to be registered based on each image transformation parameter to obtain an image to be matched generated after the image to be registered undergoes different rotations and different scalings. The rotation error and scaling error generated by the image to be registered relative to the reference image can be understood through the image to be matched under different rotations and different scalings, so as to accurately register the image to be registered.

[0088] In one embodiment, obtaining a reference image and determining at least two local feature regions in the reference image include:

[0089] Acquire a reference image, divide the reference image into at least two reference frames, and obtain a local feature region contained in each reference frame;

[0090] Determine candidate regions that match each local feature region from each image to be matched, including:

[0091] For each image to be matched, a candidate frame corresponding to each reference frame is determined in the image to be matched, and the candidate frame is expanded; for each expanded candidate frame, an area matching the local feature area contained in the corresponding reference frame is searched in the expanded candidate frame as the candidate area matching the local feature area.

[0092] Specifically, the computer device obtains a reference image, divides the reference image into at least two reference frames, and obtains a local feature area contained in each reference frame.

[0093] In one embodiment, the computer device can perform subject recognition on the image, and divide the recognized subject by at least two reference frames, each of which contains a local feature area of ​​the subject. For example, the subject in the reference image is divided by five reference frames, each of which contains a local feature area of ​​the subject.

[0094] In one embodiment, the local feature regions included in each reference frame may have the same part. For example, the five reference frames respectively include corresponding local feature regions, the local feature region in the first reference frame and the local feature region in the second reference frame may have the same part, and the local feature region in the third reference frame and the local feature region in the second reference frame may have the same part.

[0095] The computer device can determine the position of each reference frame in the reference image, and for an image to be matched, determine a candidate frame in the image to be matched that has the same position as the reference frame according to the position of each reference frame. The candidate frame in the image to be matched is expanded to obtain an expanded candidate frame and an image area contained in the expanded candidate frame. For each expanded candidate frame in the image to be matched, the local feature area contained in the reference frame is traversed on the image area contained in the corresponding expanded candidate frame to search for an area in the image area that matches the local feature area, and the area found to match the local feature area is used as a candidate area that matches the local feature area. According to the corresponding processing method, a candidate area that matches the local feature area contained in the corresponding reference frame can be found in each expanded candidate frame in the image to be matched.

[0096] In one embodiment, the computer device may determine the coordinates of each reference frame in the reference image, and for an image to be matched, determine a candidate frame having the same coordinates as the reference frame in the image to be matched according to the coordinates of each reference frame.

[0097] Each image to be matched is processed in the above manner to obtain a candidate region that matches each local feature region in each image to be matched.

[0098] In this embodiment, a reference image is obtained, and the reference image is divided by at least two reference frames, so as to extract each local feature area of ​​the reference image through the candidate frame. For each image to be matched, a candidate frame corresponding to each reference frame is determined in the image to be matched, and the candidate frame is expanded, so that for each expanded candidate frame, a candidate area matching the local feature area contained in the corresponding reference frame can be accurately found in the expanded candidate frame, so that the corresponding matching candidate area can be found in the image to be matched through the local feature area of ​​the reference image, avoiding the problem of inaccurate matching caused by direct feature point matching.

[0099] In one embodiment, searching for an area matching the local feature area included in the corresponding reference frame in the expanded candidate frame as the candidate area matching the local feature area includes:

[0100] Through the local feature area in the reference frame, the area in the corresponding expanded candidate frame is traversed, and the confidence corresponding to the traversed area is determined in each traversal; the area corresponding to the confidence that meets the confidence condition is used as the candidate area matching the local feature area.

[0101] The confidence level refers to the degree of confidence that each region matches the local feature region among the multiple regions traversed by the local feature region, that is, the degree of confidence that each region is a candidate region. The confidence level condition may refer to the highest confidence level or the confidence level meeting the confidence level threshold, etc.

[0102] Specifically, for each local feature region in a reference frame, the computer device determines the candidate frame after expansion corresponding to the reference frame, and uses the local feature region in the reference frame to traverse the candidate frame after expansion. The confidence corresponding to the currently traversed region is calculated each time the traversal is completed, and the confidence corresponding to each traversed region is obtained when the traversal is completed. The computer device obtains the confidence condition, selects the target confidence that meets the confidence condition from each confidence, and uses the area corresponding to the target confidence as the candidate area matching the local feature region. According to the same processing method, the candidate area matching each local feature region can be determined.

[0103] In one embodiment, the computer device selects the highest confidence from various confidences as the target confidence, and uses the area corresponding to the target confidence as the candidate area that matches the corresponding local feature area.

[0104] In this embodiment, the area in the corresponding expanded candidate frame is traversed by the local feature area in the reference frame, and the confidence corresponding to the traversed area is determined in each traversal. The area corresponding to the confidence that meets the confidence condition can be used as the candidate area matching the local feature area, thereby accurately screening the candidate area most similar to each local feature area based on the confidence.

[0105] In one embodiment, determining candidate regions that match each local feature region from each to-be-matched image, and screening target matching images from the to-be-matched images according to the candidate regions, includes:

[0106] For each image to be matched, candidate regions that match each local feature region are determined from the corresponding image to be matched, and the candidate similarities corresponding to each candidate region are determined; based on the candidate similarities of each candidate region included in each image to be matched, the target matching image is screened out from the image to be matched.

[0107] Specifically, for each image to be matched, the computer device determines the candidate regions that match each local feature region from the image to be matched, and calculates the similarity between each candidate region and the matched local feature region to obtain the candidate similarity corresponding to each candidate region. Based on the candidate similarities corresponding to each candidate region included in each image to be matched, the target matching image is screened out from the image to be matched.

[0108] In one embodiment, based on the candidate similarities of each candidate area included in each image to be matched, a target matching image is screened out from the images to be matched, including: determining a target similarity that meets a similarity condition among the candidate similarities, and taking the image to be matched where the candidate area corresponding to the target similarity is located as the target matching image.

[0109] The computer device selects the candidate similarities that meet the similarity conditions from the candidate similarities corresponding to each candidate area included in each image to be matched as the target similarity, and obtains the target similarity corresponding to each local feature area. The computer device can determine the image to be matched where the candidate area corresponding to each target similarity is located, and use the image to be matched where the candidate area corresponding to the target similarity is located as the target matching image.

[0110] In one embodiment, a target similarity that meets a similarity condition among candidate similarities is determined, and an image to be matched in which a candidate region corresponding to the target similarity is located is used as a target matching image, including: for each local feature region, taking the highest similarity among the candidate similarities between the same local feature region and each matching candidate region as the target similarity corresponding to the corresponding local feature region; when the candidate regions corresponding to the target similarities are in the same image to be matched, taking the same image to be matched as the target matching image; when the candidate regions corresponding to the target similarities are in different images to be matched, determining the number of target similarities corresponding to each image to be matched in different images to be matched, and taking the image to be matched with the largest number of corresponding target similarities as the target image.

[0111] In this embodiment, for each image to be matched, candidate areas that match each local feature area are determined from the corresponding image to be matched, and the candidate similarities corresponding to each candidate area are determined. Based on the candidate similarities of each candidate area included in each image to be matched, the target matching image can be accurately screened out from the images to be matched. Using the similarity between image areas as a screening condition can further improve the accuracy of screening.

[0112] In one embodiment, Figure 3 As shown, based on the candidate similarities of each candidate region included in each image to be matched, a target matching image is screened out from the image to be matched, including:

[0113] Step S302 : for each local feature region, the highest similarity among the candidate similarities between the same local feature region and each matching candidate region is used as the target similarity corresponding to the corresponding local feature region.

[0114] Specifically, for each local feature region, the computer device determines the highest similarity from the candidate similarities corresponding to the same local feature region and each matching candidate region, and uses the highest similarity as the target similarity corresponding to the same local feature region. According to the same processing method, the target similarity corresponding to each local feature region can be obtained. For example, the reference image has 3 local feature regions, and each of the 25 images to be matched has 3 corresponding candidate regions. Then each local feature region corresponds to 25 candidate regions, and the similarity between each local feature region and the corresponding 25 candidate regions is calculated. The highest similarity is selected from the 25 similarities corresponding to a local feature region as the target similarity of the local feature region.

[0115] Step S304: when the candidate regions corresponding to the target similarities are in the same image to be matched, the same image to be matched is used as the target matching image.

[0116] Specifically, the computer device may determine the image to be matched in which the candidate region corresponding to each target similarity is located. When the candidate regions corresponding to the target similarities are all in the same image to be matched, the same image to be matched is used as the target matching image.

[0117] Step S306, when the candidate regions corresponding to the target similarities are in different to-be-matched images, the number of target similarities corresponding to each to-be-matched image in the different to-be-matched images is determined, and the to-be-matched image with the largest number of corresponding target similarities is used as the target image.

[0118] Specifically, when the candidate regions corresponding to the target similarities are in different images to be matched, the computer device can determine the number of target similarities corresponding to each of the different images to be matched. The computer device determines the image to be matched with the highest number of target similarities and uses the image to be matched as the target image.

[0119] In one implementation, when the candidate regions corresponding to the target similarities are in different images to be matched, the mean similarity of the candidate similarities in the same image is determined according to the candidate similarities corresponding to each image to be matched, and the image to be matched with the highest mean similarity is used as the target image.

[0120] In one implementation, when the candidate regions corresponding to the target similarities are in different images to be matched, the sum of the similarities of the candidate similarities in the same image is determined according to the candidate similarities corresponding to each image to be matched, and the image to be matched with the highest sum of similarities is used as the target matching image.

[0121] In this embodiment, for each local feature region, the highest similarity among the candidate similarities between the same local feature region and each matching candidate region is used as the target similarity corresponding to the corresponding local feature region, so that the candidate region most similar to each local feature can be screened out. When the candidate regions corresponding to the target similarities are in the same image to be matched, the same image to be matched is used as the target matching image. When the candidate regions corresponding to the target similarities are in different images to be matched, the number of target similarities corresponding to each image to be matched in the different images to be matched is determined, and the image to be matched with the largest number of corresponding target similarities is used as the target image. Therefore, on the basis of screening out the candidate regions most similar to the local feature regions, the image to be matched that is most similar to the reference image can be further screened out, and the accuracy of the target matching image screening can be further improved through two screenings.

[0122] In one embodiment, determining target transformation parameters between a reference image and an image to be registered according to a candidate region in a target matching image that matches at least two local feature regions includes:

[0123] Determine a reference center point of each of at least two local feature areas and a candidate center point of each candidate area in the target matching image; use the reference center point of each local feature area and the candidate center point in the corresponding candidate area as a matching point pair, and determine the target transformation parameters between the reference image and the image to be registered based on the matching point pair.

[0124] Specifically, the computer device determines the reference center point of each of the at least two local feature regions, and determines the candidate center point of each candidate region in the target matching image. For each local feature region, the candidate center point corresponding to the candidate region matching the local feature region and the reference center point of the local feature region are used as a matching point pair to obtain a candidate center point matched with each reference center point, thereby obtaining at least two matching point pairs. The target transformation parameters between the reference image and the image to be registered are calculated based on the at least two matching point pairs.

[0125] In one embodiment, a computer device uses the reference center point of each local feature area and the candidate center point in the corresponding candidate area as a matching point pair, determines the position of each reference center point in the reference image, determines the position of each candidate center point in the target matching image, and calculates the target transformation parameters between the reference image and the image to be registered based on the position of the reference center point and the position of the candidate center point in the matching point pair.

[0126] In one embodiment, the position of the reference center point in the reference image may be the coordinates of the reference center point in the reference image, and the position of the candidate center point in the target matching image may be the coordinates of the candidate center point in the target matching image.

[0127] In this embodiment, the reference center point of each local feature area in at least two local feature areas and the candidate center point of each candidate area in the target matching image are determined, the reference center point of each local feature area and the candidate center point in the corresponding candidate area are used as matching point pairs, and the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pairs, which can avoid the mismatching problem caused by directly automatically generating feature points when the feature points in the area are relatively similar. In addition, selecting the center point of the area as the matching point pair can not only accurately obtain the matching point pair, but also reduce the number of matching point pairs, thereby improving the calculation efficiency.

[0128] In one embodiment, Figure 4 As shown, after the reference center point of each local feature region and the candidate center point in the corresponding candidate region are used as a matching point pair, steps S402 to S404 are also included:

[0129] Step S402, obtaining target image transformation parameters corresponding to the transformation method corresponding to the target matching image.

[0130] The target image transformation parameters refer to the rotation parameters and scaling parameters corresponding to the target matching image.

[0131] Specifically, the computer device may obtain the transformation method used when the image to be registered is subjected to image transformation processing, thereby obtaining the rotation parameters and scaling parameters corresponding to the transformation method.

[0132] Step S404: performing image inverse transformation processing on the target matching image based on the target image transformation parameters to obtain an inverse transformation image.

[0133] The inverse image transformation process refers to transforming the target matching image into the image to be registered through a process opposite to that of generating the target matching image.

[0134] Specifically, the computer device determines the processing process corresponding to the transformation method used when the image to be registered generates the target matching image, and performs image inverse transformation processing on the target matching image according to the opposite processing process to obtain an inverse transformation image. The inverse transformation image is the image to be registered.

[0135] In one embodiment, the computer device may enlarge the target matching image based on the scaling parameter, the enlargement value of the enlargement process is the same as the value of the scaling parameter, and the enlarged image is rotated in the opposite direction of the rotation parameter, the rotation angle is the same as the angle of the rotation parameter. It can be understood that the order of the rotation process and the scaling process in the image inverse transformation process is opposite to the order of the rotation process and the scaling process in the transformation method.

[0136] For example, the image to be registered is first rotated 3° to the right and then scaled 0.8 times to obtain the target matching image. The target matching image is then enlarged 0.8 times and then rotated 3° to the left to obtain the inverse transformation image. The inverse transformation image is actually the image to be registered.

[0137] Determining target transformation parameters between the reference image and the image to be registered based on the matching point pairs includes step S406:

[0138] Step S406, determining target transformation parameters between the reference image and the image to be registered according to the position of the candidate center point in the matching point pair in the inverse transformation image and the position of the reference center point in the corresponding local feature region in the reference image.

[0139] Specifically, in one embodiment, a computer device uses the reference center point of each local feature region and the candidate center point in the corresponding candidate region as a matching point pair, determines the position of each reference center point in the reference image, determines the position of each candidate center point in the inverse transformation image, and calculates the target transformation parameters between the reference image and the image to be registered based on the position of the reference center point and the position of the candidate center point in the matching point pair.

[0140] In one embodiment, the position of the reference center point in the reference image may be the coordinates of the reference center point in the reference image, and the position of the candidate center point in the inverse transformation image may be the coordinates of the candidate center point in the inverse transformation image. According to the coordinates corresponding to the matching point pair, the target transformation parameters between the reference image and the inverse transformation image may be calculated, and the target transformation parameters are the target transformation parameters between the reference image and the image to be registered.

[0141] In this embodiment, the target image transformation parameters corresponding to the transformation method corresponding to the target matching image are obtained, and the target matching image is subjected to image inverse transformation processing based on the target image transformation parameters, so that the target matching image can be subjected to image inverse transformation processing to obtain the original image to be registered, and at this time, the matching points with the reference image have been determined in the image to be registered, avoiding the phenomenon of inaccurate matching caused by directly matching the feature points of the image to be registered and the reference image, thereby effectively improving the accuracy of image registration. The position of the candidate center point in the matching point pair in the inverse transformation image and the position of the reference center point in the corresponding local feature area in the reference image are used, so that the reference image is still the target transformation parameter calculated with the image to be registered, and the calculated target transformation parameters are more accurate.

[0142] In one embodiment, Figure 5 As shown, the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pairs, including:

[0143] Step S502: combine the matching point pairs, where each combination includes at least two matching point pairs.

[0144] Specifically, the computer device calculates at least two matching point pairs between the reference image and the target matching image. When the at least two matching point pairs contain only two matching point pairs, the two matching point pairs are taken as a combination. When the at least two matching point pairs contain more than two matching point pairs, each matching point pair is combined with other matching point pairs to obtain respective combinations. Each combination includes at least two matching point pairs.

[0145] For example, if the matching point pairs A, B, C, D and E are calculated, then 10 combinations of AB, AC, AD, AE, BC, BD, BE, CD, CE and DE can be obtained.

[0146] Step S504: for each combination, a combined transformation parameter between the reference image and the image to be registered is calculated according to the matching point pairs included in the combination, so as to obtain a combined transformation parameter corresponding to each combination.

[0147] Specifically, for each combination, the computer device can calculate the combined transformation parameters between the reference image and the image to be registered through the matching point pairs included in the combination, and each combination can calculate the corresponding combined transformation parameters. For example, for the above 10 combinations, 10 combined transformation parameters can be calculated.

[0148] Step S506: for each combined transformation parameter, determine whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination, so as to obtain first applicable quantities corresponding to each combined transformation parameter.

[0149] Specifically, for each combined transformation parameter, the computer device may determine whether the combined transformation parameter is applicable to other matching point pairs outside the combination corresponding to the combined transformation parameter, thereby determining the first applicable quantity corresponding to each combined transformation parameter.

[0150] Further, after calculating a combined transformation parameter, the computer device can determine that the combined transformation parameter is applicable to other matching point pairs to obtain a first applicable quantity corresponding to the combined transformation parameter. According to the same processing method, the first applicable quantity corresponding to each combined transformation parameter can be obtained.

[0151] Step S508: using the combined transformation parameters corresponding to the first applicable quantity that meets the first preset quantity condition as target transformation parameters between the reference image and the image to be registered.

[0152] Specifically, the computer device obtains a first preset quantity condition, selects a first applicable quantity that meets the first preset quantity condition from the first applicable quantities corresponding to each combined transformation parameter, and uses the combined transformation parameters corresponding to the selected first applicable quantity as the target transformation parameters between the reference image and the image to be registered.

[0153] In one embodiment, a first applicable number of matching point pairs applicable to the combined transformation parameters corresponding to each combination is determined, and a first applicable number greater than a first number threshold is screened out, and the combined transformation parameters corresponding to the screened out first applicable number are used as target transformation parameters between the reference image and the image to be registered.

[0154] In one embodiment, a first applicable number of matching point pairs applicable to the combined transformation parameters corresponding to each combination is determined, and the combined transformation parameters with the largest first applicable number are used as target transformation parameters between the reference image and the image to be registered.

[0155] In this embodiment, each matching point pair is combined, each combination includes at least two matching point pairs, and the combined transformation parameters between the reference image and the image to be registered are calculated based on the matching point pairs included in the combination to obtain the combined transformation parameters corresponding to each combination. For each combined transformation parameter, it is determined whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination, and it is possible to determine which matching point pairs each combined transformation parameter is applicable to, so as to obtain the first applicable quantity corresponding to each combined transformation parameter, so that the most applicable combined transformation parameter can be screened out based on the first preset quantity condition as the target transformation parameter between the reference image and the image to be registered. On the basis of calculating multiple combined transformation parameters, the most applicable combined transformation parameter is further screened, so that the subsequent registration processing of the image to be registered is more accurate and reliable.

[0156] In one embodiment, for each combined transformation parameter, determining whether the combined transformation parameter is applicable to the remaining matching point pairs outside the corresponding combination to obtain the first applicable quantities corresponding to each combined transformation parameter includes:

[0157] For each combination, the positions of candidate center points in the remaining matching point pairs except the matching point pairs included in the combination are obtained, and based on the combination transformation parameters corresponding to the corresponding combination, the obtained candidate center points are transformed to obtain the combined transformed positions; for each of the remaining matching point pairs corresponding to each combination, the first errors between the combined transformed positions in the remaining matching point pairs and the reference positions of the corresponding reference center points are calculated respectively; based on the first error, the first applicable number of matching point pairs to which the combination transformation parameters of the corresponding combination are applicable is determined.

[0158] Specifically, after the computer device calculates the combined transformation parameters corresponding to a combination through the matching point pairs in a combination, it obtains the remaining matching point pairs except the matching point pairs included in the combination, and obtains the positions of the candidate center points in the remaining matching point pairs. The computer device calculates the combined transformation position according to the positions of the candidate center points and the combined transformation parameters. The computer device obtains the positions of the reference center points in the remaining matching point pairs, and determines the matching point pairs applicable to the combined transformation parameters according to the difference between the combined transformation position and the position of the corresponding reference center point. If the matching point pair is applicable to the combined transformation parameters, the combined transformation position calculated by the combined transformation parameters and the candidate center points should be basically consistent with the position of the reference center point. It can be considered that when the difference between the combined transformation position and the position of the corresponding reference center point is very small, it means that the combined transformation position is the position of the reference center point, and it can be determined that the matching point pair is applicable to the combined transformation parameters.

[0159] Likewise, the combined transformed position calculated by combining the transform parameters and the reference center point should be substantially consistent with the position of the candidate center point.

[0160] Further, the computer device calculates a first error between the combined transformation position and the position of the corresponding reference center point, determines a matching point pair applicable to the combined transformation parameter according to the first error, and thereby determines a first applicable number of matching point pairs applicable to the combined transformation parameter. In the same processing manner, the first applicable number corresponding to each combined transformation parameter can be determined.

[0161] In one embodiment, the computer device calculates a first error between the combined transformation position and the position of the corresponding reference center point, and when the first error is less than or equal to an error threshold, it indicates that the combined transformation parameter is applicable to the matching point pair. When the first error is greater than the error threshold, it indicates that the combined transformation parameter is not applicable to the matching point pair.

[0162] In this embodiment, for each combination, the positions of the candidate center points in the remaining matching point pairs except the matching point pairs included in the combination are obtained, and based on the combination transformation parameters corresponding to the corresponding combination, the obtained candidate center points are transformed to obtain the combination transformation position, and for each remaining matching point pair corresponding to each combination, the first error between the combination transformation position in each remaining matching point pair and the reference position of the corresponding reference center point is calculated respectively, so that it is possible to verify whether the combination transformation parameter is applicable to all matching point pairs through the first error between the combination transformation position and the reference position of the reference center point. According to the first error, it is determined which matching point pairs and how many matching point pairs the combination transformation parameters of the corresponding combination are applicable to, so that the combination transformation parameters applicable to all matching point pairs can be accurately screened, or the combination transformation parameters applicable to the most matching point pairs can be screened. On the basis of calculating each combination transformation parameter, the most applicable combination transformation parameter is further screened, so that the registration processing of the image to be registered is more accurate and reliable.

[0163] In one embodiment, for each combination, the positions of the candidate center points in the remaining matching point pairs except the matching point pairs included in the combination are obtained, and based on the combination transformation parameters corresponding to the corresponding combination, the obtained reference center point is transformed to obtain the combined transformation position; for each remaining matching point pair corresponding to each combination, the third error between the combined transformation position in each remaining matching point pair and the position of the corresponding candidate center point is calculated respectively; based on the third error, the first applicable number of matching point pairs to which the combination transformation parameters of the corresponding combination are applicable is determined. The specific processing process of this embodiment can refer to the processing process related to the first error, which will not be repeated here.

[0164] In one embodiment, Figure 6 As shown, the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pairs, including:

[0165] Step S602: Select at least two matching point pairs from each matching point pair, and calculate candidate transformation parameters between the reference image and the image to be registered according to the matching point pairs selected this time.

[0166] Specifically, after the computer device determines each matching point pair, it can select at least two matching point pairs from each matching point pair, and calculate the candidate transformation parameters between the reference image and the image to be registered based on the position of the reference center point and the position of the candidate center point in the matching point pairs selected this time.

[0167] In one embodiment, candidate transformation parameters between the reference image and the image to be registered are calculated based on the position of the reference center point in the currently selected matching point pair in the reference image and the position of the candidate center point in the target matching image.

[0168] In one embodiment, candidate transformation parameters between the reference image and the image to be registered are calculated based on the position of the reference center point in the currently selected matching point pair in the reference image and the position of the candidate center point in the inverse transformation image.

[0169] Step S604, determining a second error generated by the matching point pairs that are not selected this time under the candidate transformation parameters, and determining a second applicable number of matching point pairs applicable to the candidate transformation parameters according to the second error.

[0170] Specifically, the computer device determines the matching point pairs that are not selected at this time, obtains the positions of the candidate center points in the matching point pairs that are not selected, and transforms the positions of the obtained candidate center points based on the combined transformation parameters to obtain corresponding candidate transformation positions. The computer device calculates the second error between the candidate transformation position and the reference position of the corresponding reference center point, determines the matching point pairs to which the candidate transformation parameters of this time are applicable based on the second errors corresponding to each matching point pair, and obtains the second applicable number corresponding to the candidate transformation parameters of this time.

[0171] In one embodiment, the computer device determines the matching point pairs that are not selected at this time, obtains the positions of the candidate center points in the matching point pairs that are not selected, and transforms the positions of the obtained reference center points based on the candidate transformation parameters to obtain corresponding candidate transformation positions. The computer device calculates the second error between the candidate transformation position and the position of the corresponding candidate center point, and determines the matching point pairs to which the candidate transformation parameters at this time are applicable based on the second error corresponding to each matching point pair, thereby obtaining the second applicable number corresponding to the candidate transformation parameters at this time.

[0172] In one embodiment, the computer device calculates a second error between the candidate transformation position and the position of the corresponding reference center point, and when the second error is less than or equal to an error threshold, it indicates that the candidate transformation parameter is applicable to the matching point pair. When the second error is greater than the error threshold, it indicates that the candidate transformation parameter is not applicable to the matching point pair.

[0173] Step S606, continue to select at least two matching point pairs, return to the step of calculating the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time and continue to execute until the preset stop condition is reached, and obtain the second applicable number corresponding to each candidate transformation parameter.

[0174] The preset stop condition is that the number of iterations reaches the preset number of iterations, or there are no unselected matching point pairs, or each matching point pair has been used to calculate candidate transformation parameters with other matching point pairs.

[0175] Specifically, after determining the second applicable number of matching point pairs applicable to the previous candidate transformation parameter, continue to select at least two matching point pairs. The matching point pairs selected each time may not be completely the same, or may be completely different. For example, matching point pairs 1 and 2 are selected for the first time, matching point pairs 1 and 3 are selected for the second time, and matching point pairs 4 and 5 may be selected for the third time.

[0176] After continuing to select at least two matching point pairs, return to step S602 and continue to calculate the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time, so as to calculate the second applicable number corresponding to the candidate transformation parameters of this time, and then perform the calculation of the next candidate transformation parameter until the preset stop condition is met, and obtain the second applicable number corresponding to each candidate transformation parameter.

[0177] In one embodiment, the preset stop condition is that the number of iterations reaches the preset number of iterations. The process from selecting at least two matching point pairs to calculating the second applicable number corresponding to the candidate transformation parameters of the current time is regarded as one iteration. After each iteration is completed, the computer device can determine that the current iteration has reached the preset number of iterations, and when the preset number of iterations is not reached, the next iteration is performed, and when the preset number of iterations is reached, the iteration stops, and the second applicable number corresponding to the candidate transformation parameters in each iteration is obtained.

[0178] In one embodiment, when the number of matching point pairs is large, the preset stop condition can be set to the number of iterations reaching the preset number of iterations. When the number of matching point pairs is large, a limited number of iterations can be set, which is conducive to improving calculation efficiency. When the number of matching point pairs is small, the preset stop condition can be set to the number of unselected matching point pairs, or each matching point pair has been used to calculate candidate transformation parameters with other matching point pairs. When the number of matching point pairs is small, all matching point pairs can be used as much as possible to calculate candidate transformation parameters, so that the target transformation parameters finally determined can be applicable to more matching point pairs, making the candidate image registration more comprehensive and accurate.

[0179] Step S608: taking the candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition as target transformation parameters between the reference image and the image to be registered.

[0180] Specifically, the computer device obtains a second preset quantity condition, filters out a second applicable quantity that meets the second preset quantity condition from the second applicable quantities corresponding to each candidate transformation parameter, and uses the candidate transformation parameters corresponding to the filtered second applicable quantity as the target transformation parameters between the reference image and the image to be registered.

[0181] In one embodiment, a second applicable number of matching point pairs applicable to each candidate transformation parameter is determined, and a second applicable number greater than a second number threshold is screened out, and the candidate transformation parameters corresponding to the screened second applicable number are used as target transformation parameters between the reference image and the image to be registered.

[0182] In one embodiment, a second applicable number of matching point pairs applicable to each candidate transformation parameter is determined, and the candidate transformation parameter with the largest second applicable number is used as the target transformation parameter between the reference image and the image to be registered.

[0183] In this embodiment, at least two matching point pairs are selected from each matching point pair, candidate transformation parameters between the reference image and the image to be registered are calculated based on the matching point pairs selected this time, the second error generated by the matching point pairs not selected this time under the candidate transformation parameters is determined, the second applicable number of matching point pairs applicable to the candidate transformation parameters is determined based on the second error, at least two matching point pairs are continuously selected, the step of calculating the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time is returned and the execution is continued until the preset stop condition is reached, and the second applicable number corresponding to each candidate transformation parameter can be calculated through multiple loop iterations. The candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition can be used to screen out the most applicable candidate transformation parameters as the transformation relationship between the reference image and the image to be registered, which can effectively improve the accuracy of registering the image to be registered to the reference image.

[0184] like Figure 7 FIG. 1 is a flow chart of an image registration method based on template matching in one embodiment.

[0185] Step S702, obtain a reference image, and execute step S704, select N local feature areas in the reference image through N reference frames, and execute step S706.

[0186] Step S706: performing histogram equalization processing on the reference image to increase the contrast of the reference image.

[0187] It can be understood that the processing from step S702 to step S706 and the processing from step S708 to step S712 can be performed independently, with no limitation on the order of execution, and can also be performed simultaneously.

[0188] Step S708 , obtaining the image to be registered, so as to execute step S710 , that is, performing scale transformation and rotation transformation on the image to be registered, to obtain a plurality of images to be matched.

[0189] Next, step S712 is executed to perform histogram equalization processing on the multiple images to be matched.

[0190] Next, step S714 is executed to determine a candidate frame having the same position as the N reference frames in each image to be matched, and perform an expansion process on the candidate frame.

[0191] Through template matching, in the area of ​​the expanded candidate frame, the candidate area that matches the local feature area in the corresponding reference frame is searched to obtain N candidate areas in each image to be matched. Based on the N candidate areas in each image to be matched, the target matching image with the highest similarity to the reference image is selected.

[0192] The reference center points of the N local feature regions and the candidate center points of the corresponding N candidate regions in the target matching image are used as matching point pairs to execute step S716.

[0193] Step S716, use the Ransac (random sampling consensus) algorithm to filter out external points and obtain the target transformation parameters. The Ransac algorithm steps are as follows: (1) Randomly select 2 groups of matching point pairs from each matching pair and calculate the corresponding transformation parameter H; (2) Calculate the error generated by other matching point pairs under the transformation parameter H. If the error is less than the threshold, the matching point pair is considered to be an inlier and added to the inlier set I; (3) If the number of matching point pairs in the current inlier set I is greater than the maximum number I_best, update I_best=I and record the current transformation parameter H; (4) If the current number of iterations reaches the preset number of iterations, stop the iteration, otherwise increase the number of iterations by 1, and repeat the above steps until the current number of iterations reaches the preset number of iterations, and stop, and use the transformation parameter H corresponding to the maximum number I_best as the target transformation parameter. By filtering out external points, the influence of incorrect matching point pair registration can be eliminated, and the robustness to imaging defects can be enhanced.

[0194] Step S716: After the target transformation parameters are obtained, the image to be registered can be corrected by similarity transformation.

[0195] In one embodiment, the image to be registered and the reference image are both images of the same industrial device; the reference image is an image taken after the industrial device is debugged, and each of the at least two local feature areas in the reference image is unique.

[0196] Specifically, in industrial automation systems, if it is necessary to conduct quality inspection on the industrial devices produced, a machine for photographing the industrial devices will be placed at a fixed position. Once the machine is debugged, the imaging angle will be fixed, and the first image taken by the machine after debugging will be selected as the reference image. In the subsequent production process, all images taken at this angle will only have small position changes and lighting changes compared to the reference image. The images taken in the subsequent production process are used as the images to be registered, and the images to be registered are registered with the reference image to correct the images to be registered.

[0197] After obtaining the reference image, four local feature regions can be selected in the reference image through four candidate frames, and each selected local feature region is unique.

[0198] The imaging error range of the machine in the industrial automation system is counted, including the translation parameter range t∈(-t,t), the rotation parameter range r∈(-r,r), and the scaling parameter range s∈(-s,s). The rotation parameter can be set as The scaling parameter is By combining the rotation parameters and the scaling parameters respectively, 25 transformation modes can be obtained. Then, the images to be registered in the production project are rotated and scaled according to each transformation mode, and 25 images to be matched can be obtained for each image to be registered.

[0199] For each image to be matched, the four candidate frames in the reference image are mapped to the image to be matched according to the coordinates, so as to obtain four candidate frames in each image to be matched.

[0200] like Figure 8 As shown, Figure 8 (a) in the figure is the image to be matched. Figure 8 (b) in the figure is the reference image. The four reference frames in the reference image are mapped to the image to be matched to obtain the corresponding four candidate frames.

[0201] Each candidate box is expanded so that all situations within the error range can be included, making the processing method more robust to position changes.

[0202] In each expanded candidate frame, a candidate region matching the corresponding local feature region is searched by template matching. For each local feature region, there are 25 corresponding candidate regions. For each of the four local feature regions, the candidate similarity with the 25 candidate regions can be calculated respectively, and the highest similarity is selected from the 25 candidate similarities as the corresponding target similarity. Then, the four local feature regions can obtain four target similarities.

[0203] When determining whether the candidate areas corresponding to the four target similarities are in the same image to be matched, if so, the image to be matched is used as the target matching image; otherwise, the images to be matched in which the candidate areas corresponding to the four target similarities are located are determined, the number of target similarities corresponding to each image to be matched is determined, and the image to be matched with the largest number of corresponding target similarities is used as the target matching image.

[0204] Determine the candidate center points of the four candidate regions in the target matching image and the reference center points of the four local feature regions to form four matching point pairs A, B, C and D. Figure 8 As shown by the dotted lines in , three pairs of matching points between the reference image and the image to be matched are given.

[0205] After determining the target matching image, the target matching image is subjected to image inverse transformation processing, so that the size of the image to be registered and the reference image are consistent. Determine the coordinates of the four candidate center points in the image to be registered, and determine the coordinates of the four reference center points in the reference image. Use the matching point pairs AB to calculate the transformation parameter 1, and use the matching point pairs C and D to verify the transformation parameter 1 to determine whether the transformation parameter 1 is applicable to the matching point pairs C and D. The specific calculation method is as follows: transform the coordinates of the candidate center point in the matching point pair C by using the transformation parameter 1 to obtain the transformed coordinates, calculate the error between the transformed coordinates and the coordinates of the reference center point in the matching point pair C, if the error is less than or equal to the error threshold, it means that the transformed coordinates are consistent with the coordinates of the reference center point, and the transformation parameter 1 is applicable to the matching point pair C. If the error is greater than the error threshold, it means that the transformed coordinates are inconsistent with the coordinates of the reference center point, and the transformation parameter 1 is not applicable to the matching point pair C. Use the same method to process the matching point pair D, so as to calculate the number of matching point pairs to which the transformation parameter 1 is applicable.

[0206] Similarly, transformation parameter 2 is calculated using matching point pair AC, transformation parameter 3 is calculated using matching point pair AD, transformation parameter 4 is calculated using matching point pair BC, transformation parameter 5 is calculated using matching point pair BD, and transformation parameter 6 is calculated using matching point pair CD, and the number of matching point pairs applicable to each transformation parameter is calculated respectively.

[0207] The transformation parameter that is applicable to the largest number of matching point pairs is used as the target transformation parameter. For example, if the transformation parameter 5 is applicable to the largest number of matching point pairs, then the transformation parameter 5 is used as the target transformation parameter, and the registration processing is performed on the image to be registered by using the transformation parameter 5 to achieve correction of the image to be registered.

[0208] It is understandable that there are multiple images to be registered in industrial production, and each image to be registered is processed according to the method of this embodiment, so as to achieve correction of each image to be registered.

[0209] In this embodiment, the image registration method can be applied to the registration scene of industrial device images. The image to be registered and the reference image are both images of the same industrial device. The reference image is an image taken after the industrial device is debugged. Each of the at least two local feature areas in the reference image is unique, so that the selected local feature area has key features, which can effectively reduce the situation of mismatching after automatically generating feature points. Industrial devices are often similar in color, shape, etc., so the feature points of each part are also similar. Directly using feature point matching is prone to mismatching, resulting in inaccurate image registration. In this embodiment, matching through the local feature area of ​​the image is more accurate than matching a single feature point, and after the local feature area is matched, the center point of the local feature area is selected to form a matching point pair, which can reduce the number of matching point pairs, save the time of feature point matching, and further improve the accuracy of the matching point pair. After obtaining the matching point pair, the most suitable target transformation parameters are further screened from the transformation parameters calculated from each matching point pair, effectively improving the precision and accuracy of industrial device image calibration.

[0210] In one embodiment, the image registration method can be applied to industrial quality inspection scenarios. Fig. 9 As shown, it is a schematic diagram of the process of industrial quality inspection in one embodiment. In the current industrial intelligent quality inspection platform, industrially manufactured components, especially 3C components, are usually small and precisely constructed, so the cameras are designed to shoot at multiple angles for locations where defects frequently occur. When it comes to a specific camera, there will be a fixed "region of interest" with clear imaging, and the rest of the area will be relatively blurred, leaving it to other cameras to take pictures. The image registration method of this embodiment is to align the photographed "region of interest" with the reference image through image registration processing to obtain a registered image, so that in the subsequent defect comparison learning, the defects in the "region of interest" can be effectively located and identified.

[0211] In one embodiment, an image registration method is provided, which is applied to a computer device and includes:

[0212] Acquire multiple frames of images, take the first frame of image as the reference frame of image, and take the remaining frames of image as images to be registered.

[0213] Image transformation parameters corresponding to the plurality of transformation modes are obtained, where the image transformation parameters include at least one of a rotation parameter and a scaling parameter.

[0214] The images to be registered are transformed based on the image transformation parameters to obtain multiple images to be matched.

[0215] The reference image is divided by at least two reference frames to obtain a local feature area contained in each reference frame.

[0216] For each image to be matched, a candidate frame corresponding to each reference frame is determined in the image to be matched, and the candidate frame is expanded.

[0217] Through the local feature area in the reference frame, the area in the corresponding expanded candidate frame is traversed, and the confidence corresponding to the traversed area is determined in each traversal.

[0218] The area corresponding to the confidence satisfying the confidence condition is taken as the candidate area matching the local feature area.

[0219] The candidate similarity between each candidate region included in each image to be matched and the corresponding local feature region is calculated.

[0220] For each local feature region, the highest similarity among the candidate similarities between the same local feature region and each matching candidate region is used as the target similarity corresponding to the corresponding local feature region.

[0221] Optionally, when the candidate regions corresponding to the target similarities are in the same image to be matched, the same image to be matched is used as the target matching image.

[0222] Optionally, when the candidate regions corresponding to the target similarities are in different images to be matched, the number of target similarities corresponding to each image to be matched in the different images to be matched is determined, and the image to be matched with the largest number of corresponding target similarities is used as the target matching image.

[0223] A reference center point of each local feature area in at least two local feature areas and a position of each reference center point in a reference image are determined, and a candidate center point of each candidate area in a target matching image is determined.

[0224] The reference center point of each local feature region and the candidate center point in the corresponding candidate region are used as a matching point pair to obtain target image transformation parameters corresponding to the transformation method corresponding to the target matching image.

[0225] The target matching image is subjected to image inverse transformation processing based on the target image transformation parameters to obtain an inverse transformation image.

[0226] At least two matching point pairs are selected from each matching point pair, and the candidate transformation parameters between the reference image and the image to be registered are calculated based on the position of the candidate center point of the selected matching point in the inverse transformation image and the position of the corresponding reference center point in the reference image.

[0227] The position of the candidate center point among the matching points that are not selected at the time is determined, and based on the corresponding candidate transformation parameters, the obtained candidate center point is transformed to obtain the corresponding candidate transformation position.

[0228] The second errors between the candidate transformation positions of the matching points that were not selected this time and the reference positions of the corresponding reference center points are calculated respectively, and the second applicable number of the matching point pairs to which the candidate transformation parameters are applicable is determined according to the second errors. Further, when the second error is less than or equal to the error threshold, it indicates that the matching point pair is applicable to the candidate transformation parameters. When the second error is greater than the error threshold, it indicates that the matching point pair is not applicable to the candidate transformation parameters.

[0229] Continue to select at least two matching point pairs, return to the step of calculating candidate transformation parameters between the reference image and the image to be registered according to the matching point pairs selected this time, and continue to execute until a preset stop condition is reached, and obtain a second applicable number corresponding to each candidate transformation parameter;

[0230] The candidate transformation parameters corresponding to the largest second applicable number are used as target transformation parameters between the reference image and the image to be registered.

[0231] The image to be registered is registered based on the target transformation parameters to obtain the registered image.

[0232] For each image to be registered, the target transformation parameters between the image and the reference image are calculated according to the above processing method, so as to perform registration based on the target transformation parameters to obtain each image registered to the reference image.

[0233] In this embodiment, by performing different image transformation processing on the image to be registered according to multiple transformation methods, multiple images to be matched are obtained, and at least two local feature areas in the reference image are determined, so as to search for corresponding matching candidate areas in the image to be matched through the local feature areas of the reference image. Matching through the local feature areas of the image is more accurate than matching a single feature point, and after the local feature areas are matched, the center points of the local feature areas are selected to form matching point pairs, which can reduce the number of matching point pairs, save the time of feature point matching, and further improve the accuracy of matching point pairs. After obtaining the matching point pairs, the most suitable target transformation parameters are further screened out from the transformation parameters calculated from each matching point pair, effectively improving the precision and accuracy of image calibration.

[0234] It should be understood that although Figure 2-Figure 9 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 9 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0235] In one embodiment, Fig.10 As shown, an image registration device 1000 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a transformation module 1002, an acquisition module 1004, a screening module 1006, a determination module 1008 and a registration module 1010, wherein:

[0236] The transformation module 1002 is used to perform different image transformation processes on the image to be registered according to multiple transformation methods to obtain multiple images to be matched.

[0237] The acquisition module 1004 is used to acquire a reference image and determine at least two local feature regions in the reference image.

[0238] The screening module 1006 is used to determine candidate regions that match each local feature region from each to-be-matched image, and screen out target matching images from the to-be-matched images based on the candidate regions.

[0239] The determination module 1008 is used to determine the target transformation parameters between the reference image and the image to be registered according to the candidate area in the target matching image that matches the at least two local feature areas.

[0240] The registration module 1010 is used to register the image to be registered based on the target transformation parameters to obtain a registered image.

[0241] In this embodiment, by performing different image transformation processing on the image to be registered according to multiple transformation methods, multiple images to be matched are obtained, and at least two local feature areas in the reference image are determined, so that the corresponding matching candidate areas can be found in the image to be matched through the local feature areas of the reference image, and the target matching image that is most similar to the reference image can be accurately screened out from the multiple images to be matched based on the matching areas between the images. And screening based on the image area can effectively avoid the problem of inaccurate matching due to the similarity of a single feature point, thereby improving the accuracy of target matching image screening. According to the candidate areas in the target matching image that match at least two local feature areas, the target transformation parameters between the reference image and the image to be registered are accurately calculated, so that the image to be registered can be more accurately registered based on the target transformation parameters, thereby obtaining an image after being registered to the reference image.

[0242] In one embodiment, the transformation module 1002 is also used to obtain image transformation parameters corresponding to multiple transformation methods, where the image transformation parameters include at least one of a rotation parameter and a scaling parameter; based on each image transformation parameter, image transformation processing is performed on the image to be registered to obtain multiple images to be matched.

[0243] In this embodiment, image transformation parameters corresponding to a plurality of transformation modes are obtained, and the image transformation parameters include at least one of a rotation parameter and a scaling parameter. Image transformation processing is performed on the image to be registered based on each image transformation parameter to obtain an image to be matched generated after the image to be registered undergoes different rotations and different scalings. The rotation error and scaling error generated by the image to be registered corresponding to the reference image can be understood through the image to be matched under different rotations and different scalings, so as to accurately align the image to be registered.

[0244] In one embodiment, the acquisition module 1004 is further used to acquire a reference image, divide the reference image into at least two reference frames, and obtain a local feature region contained in each reference frame;

[0245] The screening module 1006 is also used to determine, for each image to be matched, a candidate frame corresponding to each reference frame in the image to be matched, and perform an outward expansion process on the candidate frame; for each outward expanded candidate frame, search the outward expanded candidate frame for an area that matches the local feature area contained in the corresponding reference frame as a candidate area that matches the local feature area.

[0246] In this embodiment, a reference image is obtained, and the reference image is divided by at least two reference frames, so as to extract each local feature area of ​​the reference image through the candidate frame. For each image to be matched, a candidate frame corresponding to each reference frame is determined in the image to be matched, and the candidate frame is expanded, so that for each expanded candidate frame, a candidate area matching the local feature area contained in the corresponding reference frame can be accurately found in the expanded candidate frame, so that the corresponding matching candidate area can be found in the image to be matched through the local feature area of ​​the reference image, avoiding the problem of inaccurate matching caused by direct feature point matching.

[0247] In one embodiment, the screening module 1006 is also used to traverse the area in the corresponding expanded candidate frame through the local feature area in the reference frame, and determine the confidence corresponding to the traversed area in each traversal; the area corresponding to the confidence that meets the confidence condition is used as the candidate area matching the local feature area.

[0248] In this embodiment, the area in the corresponding expanded candidate frame is traversed by the local feature area in the reference frame, and the confidence corresponding to the traversed area is determined in each traversal. The area corresponding to the confidence that meets the confidence condition can be used as the candidate area matching the local feature area, thereby accurately screening the candidate area most similar to each local feature area based on the confidence.

[0249] In one embodiment, the screening module 1006 is also used to determine, for each image to be matched, candidate regions that match each local feature region from the corresponding image to be matched, and determine the candidate similarities corresponding to each candidate region; based on the candidate similarities of each candidate region included in each image to be matched, screen out the target matching image from the image to be matched.

[0250] In this embodiment, for each image to be matched, candidate areas that match each local feature area are determined from the corresponding image to be matched, and the candidate similarities corresponding to each candidate area are determined. Based on the candidate similarities of each candidate area included in each image to be matched, the target matching image can be accurately screened out from the images to be matched. Using the similarity between image areas as a screening condition can further improve the accuracy of screening.

[0251] In one embodiment, the screening module 1006 is also used to, for each local feature region, use the highest similarity among the candidate similarities between the same local feature region and each matching candidate region as the target similarity corresponding to the corresponding local feature region; when the candidate regions corresponding to the target similarities are in the same image to be matched, use the same image to be matched as the target matching image; when the candidate regions corresponding to the target similarities are in different images to be matched, determine the number of target similarities corresponding to each image to be matched in different images to be matched, and use the image to be matched with the largest number of corresponding target similarities as the target image.

[0252] In this embodiment, for each local feature region, the highest similarity among the candidate similarities between the same local feature region and each matching candidate region is used as the target similarity corresponding to the corresponding local feature region, so that the candidate region most similar to each local feature can be screened out. When the candidate regions corresponding to the target similarities are in the same image to be matched, the same image to be matched is used as the target matching image. When the candidate regions corresponding to the target similarities are in different images to be matched, the number of target similarities corresponding to each image to be matched in the different images to be matched is determined, and the image to be matched with the largest number of corresponding target similarities is used as the target image. Therefore, on the basis of screening out the candidate regions most similar to the local feature regions, the image to be matched that is most similar to the reference image can be further screened out, and the accuracy of the target matching image screening can be further improved through two screenings.

[0253] In one embodiment, the determination module 1008 is also used to determine the reference center point of each local feature area in at least two local feature areas, and the candidate center point of each candidate area in the target matching image; the reference center point of each local feature area and the candidate center point in the corresponding candidate area are used as a matching point pair, and the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pair.

[0254] In this embodiment, the reference center point of each local feature area in at least two local feature areas and the candidate center point of each candidate area in the target matching image are determined, the reference center point of each local feature area and the candidate center point in the corresponding candidate area are used as matching point pairs, and the target transformation parameters between the reference image and the image to be registered are determined based on the matching point pairs, which can avoid the mismatching problem caused by directly automatically generating feature points when the feature points in the area are relatively similar. In addition, selecting the center point of the area as the matching point pair can not only accurately obtain the matching point pair, but also reduce the number of matching point pairs, thereby improving the calculation efficiency.

[0255] In one embodiment, the determination module 1008 is further used to obtain target image transformation parameters corresponding to the transformation method corresponding to the target matching image; perform image inverse transformation processing on the target matching image based on the target image transformation parameters to obtain an inverse transformation image; and determine the target transformation parameters between the reference image and the image to be registered according to the position of the candidate center point in the matching point pair in the inverse transformation image and the position of the reference center point in the corresponding local feature area in the reference image.

[0256] In this embodiment, the target image transformation parameters corresponding to the transformation method corresponding to the target matching image are obtained, and the target matching image is subjected to image inverse transformation processing based on the target image transformation parameters, so that the target matching image can be subjected to image inverse transformation processing to obtain the original image to be registered, and at this time, the matching points with the reference image have been determined in the image to be registered, avoiding the phenomenon of inaccurate matching caused by directly matching the feature points of the image to be registered and the reference image, thereby effectively improving the accuracy of image registration. The position of the candidate center point in the matching point pair in the inverse transformation image and the position of the reference center point in the corresponding local feature area in the reference image are used, so that the reference image is still the target transformation parameter calculated with the image to be registered, and the calculated target transformation parameters are more accurate.

[0257] In one embodiment, the determination module 1008 is further used to combine the matching point pairs, each combination including at least two matching point pairs; for each combination, based on the matching point pairs included in the combination, the combined transformation parameters between the reference image and the image to be registered are calculated to obtain the combined transformation parameters corresponding to each combination; for each combined transformation parameter, it is determined whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination to obtain the first applicable quantities corresponding to each combined transformation parameter; the combined transformation parameters corresponding to the first applicable quantity that meets the first preset quantity condition are used as the target transformation parameters between the reference image and the image to be registered.

[0258] In this embodiment, each matching point pair is combined, each combination includes at least two matching point pairs, and the combined transformation parameters between the reference image and the image to be registered are calculated based on the matching point pairs included in the combination to obtain the combined transformation parameters corresponding to each combination. For each combined transformation parameter, it is determined whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination, and it is possible to determine which matching point pairs each combined transformation parameter is applicable to, so as to obtain the first applicable quantity corresponding to each combined transformation parameter, so that the most applicable combined transformation parameter can be screened out based on the first preset quantity condition as the target transformation parameter between the reference image and the image to be registered. On the basis of calculating multiple combined transformation parameters, the most applicable combined transformation parameter is further screened, so that the subsequent registration processing of the image to be registered is more accurate and reliable.

[0259] In one embodiment, the determination module 1008 is further used to obtain, for each combination, the positions of candidate center points in the remaining matching point pairs except the matching point pairs included in the combination, and transform the obtained candidate center points based on the combination transformation parameters corresponding to the corresponding combination to obtain the combined transformation position; for each remaining matching point pair corresponding to each combination, respectively calculate the first error between the combined transformation position in each remaining matching point pair and the reference position of the corresponding reference center point; and determine, based on the first error, the first applicable number of matching point pairs to which the combination transformation parameters of the corresponding combination are applicable.

[0260] In this embodiment, for each combination, the positions of the candidate center points in the remaining matching point pairs except the matching point pairs included in the combination are obtained, and based on the combination transformation parameters corresponding to the corresponding combination, the obtained candidate center points are transformed to obtain the combination transformation position, and for each remaining matching point pair corresponding to each combination, the first error between the combination transformation position in each remaining matching point pair and the reference position of the corresponding reference center point is calculated respectively, so that it is possible to verify whether the combination transformation parameter is applicable to all matching point pairs through the first error between the combination transformation position and the reference position of the reference center point. According to the first error, it is determined which matching point pairs and how many matching point pairs the combination transformation parameters of the corresponding combination are applicable to, so that the combination transformation parameters applicable to all matching point pairs can be accurately screened, or the combination transformation parameters applicable to the most matching point pairs can be screened. On the basis of calculating each combination transformation parameter, the most applicable combination transformation parameter is further screened, so that the registration processing of the image to be registered is more accurate and reliable.

[0261] In one embodiment, the determination module 1008 is further used to select at least two matching point pairs from each matching point pair, calculate candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time; determine a second error generated by the matching point pairs that are not selected this time under the candidate transformation parameters, and determine a second applicable number of matching point pairs applicable to the candidate transformation parameters based on the second error; continue to select at least two matching point pairs, return to the step of calculating the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time and continue to execute until a preset stop condition is reached, and obtain the second applicable number corresponding to each candidate transformation parameter; and use the candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition as the target transformation parameters between the reference image and the image to be registered.

[0262] In this embodiment, at least two matching point pairs are selected from each matching point pair, candidate transformation parameters between the reference image and the image to be registered are calculated based on the matching point pairs selected this time, the second error generated by the matching point pairs not selected this time under the candidate transformation parameters is determined, the second applicable number of matching point pairs applicable to the candidate transformation parameters is determined based on the second error, at least two matching point pairs are continuously selected, the step of calculating the candidate transformation parameters between the reference image and the image to be registered based on the matching point pairs selected this time is returned and the execution is continued until the preset stop condition is reached, and the second applicable number corresponding to each candidate transformation parameter can be calculated through multiple loop iterations. The candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition can be used to screen out the most applicable candidate transformation parameters as the transformation relationship between the reference image and the image to be registered, which can effectively improve the accuracy of registering the image to be registered to the reference image.

[0263] In one embodiment, the image to be registered and the reference image are both images of the same industrial device; the reference image is an image taken after the industrial device is debugged, and each of the at least two local feature areas in the reference image is unique.

[0264] In this embodiment, the image registration method can be applied to the registration scene of industrial device images. The image to be registered and the reference image are both images of the same industrial device. The reference image is an image taken after the industrial device is debugged. Each of the at least two local feature areas in the reference image is unique, so that the selected local feature area has key features, which can effectively reduce the situation of mismatching after automatically generating feature points. Industrial devices are often similar in color, shape, etc., so the feature points of each part are also similar. Directly using feature point matching is prone to mismatching, resulting in inaccurate image registration. In this embodiment, matching through the local feature area of ​​the image is more accurate than matching a single feature point, and after the local feature area is matched, the center point of the local feature area is selected to form a matching point pair, which can reduce the number of matching point pairs, save the time of feature point matching, and further improve the accuracy of the matching point pair. After obtaining the matching point pair, the most suitable target transformation parameters are further screened from the transformation parameters calculated from each matching point pair, effectively improving the precision and accuracy of industrial device image calibration.

[0265] For the specific definition of the image registration device, please refer to the definition of the image registration method above, which will not be repeated here. Each module in the above-mentioned image registration device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0266] In one embodiment, a computer device is provided, which may be a terminal or a server. This embodiment takes a terminal as an example, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image registration method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0267] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0268] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0269] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0270] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0271] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0272] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0273] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image registration method, characterized in that: The method comprises: Perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched; Acquire a reference image, and determine at least two local feature areas in the reference image, wherein the image to be registered and the reference image are images obtained by shooting the same scene; Determining candidate regions that match each local feature region from each of the images to be matched, and screening target matching images from the images to be matched based on the candidate regions; Selecting feature points from the at least two local feature regions and the candidate region of the target matching image respectively, and taking the feature points of each local feature region and the feature points of the corresponding candidate region as matching point pairs; Combining the matching point pairs, each combination includes at least two matching point pairs; For each combination, determining a combined transformation parameter between the reference image and the image to be registered according to the matching point pairs in the combination, and obtaining a combined transformation parameter corresponding to the combination; For each combination transformation parameter corresponding to the combination, determine the number of matching point pairs applicable to the combination transformation parameter, and obtain a first applicable number corresponding to the combination transformation parameter; Using the combined transformation parameters corresponding to the first applicable quantity that meets the first preset quantity condition as the target transformation parameters between the reference image and the image to be registered; The image to be registered is registered based on the target transformation parameter to obtain a registered image.

2. The method according to claim 1, characterized in that: The image to be registered is subjected to different image transformation processing according to a plurality of transformation modes to obtain a plurality of images to be matched, including: Acquire image transformation parameters corresponding to the plurality of transformation modes, wherein the image transformation parameters include at least one of a rotation parameter and a scaling parameter; Based on each of the image transformation parameters, image transformation processing is performed on the images to be registered to obtain multiple images to be matched.

3. The method according to claim 1, characterized in that The acquiring a reference image and determining at least two local feature areas in the reference image includes: Acquire a reference image, divide the reference image into at least two reference frames, and obtain a local feature region contained in each reference frame; The step of determining, from each of the images to be matched, candidate regions that match each local feature region, respectively, includes: For each image to be matched, determining a candidate frame corresponding to each reference frame in the image to be matched, and performing an outward expansion process on the candidate frame; For each expanded candidate frame, a region matching the local feature region included in the corresponding reference frame is searched in the expanded candidate frame as the candidate region matching the local feature region.

4. The method according to claim 3, characterized in that The step of searching, in the expanded candidate frame, for an area matching the local feature area contained in the corresponding reference frame as the candidate region matching the local feature area comprises: Traversing the area in the corresponding expanded candidate frame through the local feature area in the reference frame, and determining the confidence level corresponding to the traversed area in each traversal; The region corresponding to the confidence satisfying the confidence condition is taken as a candidate region matching the local feature region.

5. The method according to claim 1, characterized in that The step of determining candidate regions that match each local feature region from each of the images to be matched, and selecting target matching images from the images to be matched according to the candidate regions, comprises: For each image to be matched, candidate regions that match each local feature region are determined from the corresponding image to be matched, and candidate similarities corresponding to each candidate region are determined; Based on the candidate similarities of the candidate regions included in each image to be matched, a target matching image is screened out from the images to be matched.

6. The method according to claim 5, characterized in that The step of selecting a target matching image from the images to be matched based on the candidate similarities of the candidate regions included in each image to be matched comprises: For each local feature region, the highest similarity among the candidate similarities between the same local feature region and each matching candidate region is taken as the target similarity corresponding to the corresponding local feature region; When the candidate regions corresponding to the target similarities are in the same image to be matched, the same image to be matched is used as the target matching image; When the candidate areas corresponding to the target similarities are in different images to be matched, the number of target similarities corresponding to each image to be matched in the different images to be matched is determined, and the image to be matched with the largest number of corresponding target similarities is used as the target image.

7. The method according to claim 1, characterized in that The selecting feature points from the at least two local feature regions and the candidate region of the target matching image respectively, and taking the feature points of each local feature region and the feature points in the corresponding candidate region as matching point pairs, comprises: Determine a reference center point of each of the at least two local feature areas and a candidate center point of each candidate area in the target matching image; The reference center point of each local feature region and the candidate center point in the corresponding candidate region are used as a matching point pair.

8. The method according to claim 7, characterized in that The method further comprises: Obtaining target image transformation parameters corresponding to the transformation method corresponding to the target matching image; Performing image inverse transformation processing on the target matching image based on the target image transformation parameters to obtain an inverse transformation image; The step of determining, for each combination, a combined transformation parameter between the reference image and the image to be registered according to a pair of matching points in the combination, and obtaining the combined transformation parameter corresponding to the combination includes: For each combination, the combined transformation parameters between the reference image and the image to be registered are determined based on the position of the candidate center point in the matching point pair in the combination and the position of the reference center point in the corresponding local feature area in the reference image, so as to obtain the combined transformation parameters corresponding to the combination.

9. The method according to claim 7, characterized in that: The step of determining the number of matching point pairs applicable to the combined transformation parameter for each combination, and obtaining the first applicable number corresponding to the combined transformation parameter, comprises: For each combination, the positions of candidate center points in the remaining matching point pairs except the matching point pairs included in the combination are obtained, and based on the combination transformation parameters corresponding to the corresponding combination, the obtained candidate center points are transformed to obtain the combined transformation position; For each remaining matching point pair corresponding to each combination, respectively calculating a first error between a combined transformed position in each remaining matching point pair and a reference position of a corresponding reference center point; A first applicable number of matching point pairs to which the combined transformation parameters of the corresponding combination are applicable is determined according to the first error.

10. The method according to claim 7, characterized in that The method further comprises: Selecting at least two matching point pairs from each matching point pair, and calculating candidate transformation parameters between the reference image and the image to be registered according to the matching point pairs selected at that time; Determine a second error generated by the matching point pairs that are not selected this time under the candidate transformation parameters, and determine a second applicable number of matching point pairs applicable to the candidate transformation parameters according to the second error; Continue to select at least two matching point pairs, return to the step of calculating the candidate transformation parameters between the reference image and the image to be registered according to the matching point pairs selected this time, and continue to execute until a preset stop condition is reached, and obtain a second applicable number corresponding to each candidate transformation parameter; The candidate transformation parameters corresponding to the second applicable number that meet the second preset number condition are used as target transformation parameters between the reference image and the image to be registered.

11. The method according to any one of claims 1 to 10, characterized in that: The image to be registered and the reference image are both images of the same industrial device; the reference image is an image taken after the industrial device is debugged, and each of the at least two local feature areas in the reference image is unique.

12. An image registration device, characterized in that: The device comprises: A transformation module is used to perform different image transformation processing on the image to be registered according to multiple transformation methods to obtain multiple images to be matched; An acquisition module, used for acquiring a reference image and determining at least two local feature areas in the reference image, wherein the image to be registered and the reference image are images obtained by shooting the same scene; A screening module, used to determine candidate regions that match each local feature region from each of the images to be matched, and screen target matching images from the images to be matched based on the candidate regions; A determination module, used to select feature points from the at least two local feature areas and the candidate area of ​​the target matching image respectively, and use the feature points of each local feature area and the feature points in the corresponding candidate area as matching point pairs; combine the matching point pairs, each combination including at least two matching point pairs; for each combination, calculate the combined transformation parameters between the reference image and the image to be registered according to the matching point pairs in the combination, and obtain the combined transformation parameters corresponding to the combination; for the combined transformation parameters corresponding to each combination, determine the number of matching point pairs applicable to the combined transformation parameters, and obtain a first applicable number corresponding to the combined transformation parameters; use the combined transformation parameters corresponding to the first applicable number that meets the first preset number condition as the target transformation parameters between the reference image and the image to be registered; The registration module is used to register the image to be registered based on the target transformation parameters to obtain a registered image.

13. The device according to claim 12, characterized in that The transformation module is also used to obtain image transformation parameters corresponding to multiple transformation methods, wherein the image transformation parameters include at least one of a rotation parameter and a scaling parameter; based on each of the image transformation parameters, image transformation processing is performed on the image to be registered to obtain multiple images to be matched.

14. The device according to claim 12, characterized in that The acquisition module is further used to acquire a reference image, divide the reference image into at least two reference frames, and obtain a local feature area contained in each reference frame; The screening module is further used to determine, for each image to be matched, a candidate frame corresponding to each reference frame in the image to be matched, and perform an outward expansion process on the candidate frame; for each outward expanded candidate frame, search the outward expanded candidate frame for an area that matches the local feature area contained in the corresponding reference frame as the candidate area that matches the local feature area.

15. The device according to claim 14, characterized in that The screening module is also used to traverse the area in the corresponding expanded candidate frame through the local feature area in the reference frame, and determine the confidence corresponding to the traversed area in each traversal; and use the area corresponding to the confidence that meets the confidence condition as the candidate area matching the local feature area.

16. The device according to claim 12, characterized in that The screening module is also used to determine, for each image to be matched, candidate regions that match each local feature region from the corresponding image to be matched, and determine the candidate similarities corresponding to each candidate region; based on the candidate similarities of each candidate region included in each image to be matched, screen out the target matching image from the image to be matched.

17. The device according to claim 16, characterized in that The screening module is further used to, for each local feature region, use the highest similarity among the candidate similarities between the same local feature region and each matching candidate region as the target similarity corresponding to the corresponding local feature region; when the candidate regions corresponding to the target similarities are in the same image to be matched, use the same image to be matched as the target matching image; when the candidate regions corresponding to the target similarities are in different images to be matched, determine the number of target similarities corresponding to each image to be matched in the different images to be matched, and use the image to be matched with the largest number of corresponding target similarities as the target image.

18. The device according to claim 12, characterized in that The determination module is also used to determine the reference center point of each local feature area in the at least two local feature areas, and the candidate center point of each candidate area in the target matching image; and use the reference center point of each local feature area and the candidate center point in the corresponding candidate area as a matching point pair.

19. The device according to claim 18, characterized in that The determination module is further used to obtain target image transformation parameters corresponding to the transformation method corresponding to the target matching image; perform image inverse transformation processing on the target matching image based on the target image transformation parameters to obtain an inverse transformation image; for each combination, determine the combined transformation parameters between the reference image and the image to be registered according to the position of the candidate center point in the matching point pair in the combination in the inverse transformation image and the position of the reference center point in the corresponding local feature area in the reference image, so as to obtain the combined transformation parameters corresponding to the combination.

20. The device according to claim 18, characterized in that The determination module is further used to combine the matching point pairs, each combination including at least two matching point pairs; for each combination, based on the matching point pairs included in the combination, calculate the combined transformation parameters between the reference image and the image to be registered, and obtain the combined transformation parameters corresponding to each combination; for each combined transformation parameter, determine whether the combined transformation parameter is applicable to other matching point pairs outside the corresponding combination, so as to obtain the first applicable quantities corresponding to each combined transformation parameter; and use the combined transformation parameters corresponding to the first applicable quantity that meets the first preset quantity condition as the target transformation parameters between the reference image and the image to be registered.

21. The device according to claim 18, characterized in that The determination module is further used to obtain, for each combination, the positions of candidate center points in the remaining matching point pairs except the matching point pairs included in the combination, and transform the obtained candidate center points based on the combination transformation parameters corresponding to the corresponding combination to obtain the combined transformation position; for each remaining matching point pair corresponding to each combination, respectively calculate the first error between the combined transformation position in each remaining matching point pair and the reference position of the corresponding reference center point; and determine, based on the first error, a first applicable number of matching point pairs to which the combination transformation parameters of the corresponding combination are applicable.

22. The device according to any one of claims 12 to 21, characterized in that The image to be registered and the reference image are both images of the same industrial device; the reference image is an image taken after the industrial device is debugged, and each of the at least two local feature areas in the reference image is unique.

23. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

24. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

25. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images