A large baseline image registration method and system based on feature separation

Abstract

The present invention discloses a large baseline image registration method and system based on feature separation, which collects images of a circuit board at a preset overlap rate to obtain a plurality of local images of the circuit board, performs distortion correction on the plurality of local images of the circuit board, composes image pairs of the local images of the circuit board after distortion correction and constructs a training set, builds an image registration network, uses the training set to train the image registration network and calculates the reconstruction loss, obtains the trained image registration network, obtains the large baseline image to be registered in a real scene, composes image pairs after distortion correction on the large baseline image to be registered and inputs them into the trained image registration network for processing, outputs the image transformation matrix of the large baseline image pair to be registered in the real scene, and realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix. The method can significantly improve the accuracy, efficiency and stability of image registration in actual industrial applications.

Description

Technical Field

[0001] The present invention relates to the field of industrial vision technology, and in particular to a large baseline image registration method and system based on feature separation. Background Art

[0002] Large baseline images specifically refer to images with low overlap rates. In the field of intelligent manufacturing, with the continuous advancement of production technology, large baseline image registration of circuit boards has become an important link to ensure product quality and production efficiency. Large baseline image registration refers to the splicing and alignment of multiple locally captured circuit board images with low overlap rates so that they accurately correspond to each other in the same coordinate system. Traditional image registration methods, such as those based on feature point matching, often have difficulty in achieving ideal results when dealing with complex backgrounds, large parallax, and image distortion. Especially under large baseline conditions, due to changes in shooting angles and increased parallax, the similarity between different images is significantly reduced, which greatly increases the difficulty of image registration. In addition, circuit boards have complex texture structures and diverse shapes, which further aggravates the complexity and uncertainty of the registration process. Through precise image registration, accurate detection, measurement, and analysis of circuit boards can be achieved, which is of great significance for ensuring the production quality of circuit boards, discovering potential manufacturing defects, and improving the reliability of automated detection. Summary of the invention

[0003] In order to significantly improve the efficiency and stability of image registration in practical industrial applications, the present invention proposes a large baseline image registration method based on feature separation, which comprises the following steps:

[0004] S1. Capturing images of the circuit board at a preset overlap rate to obtain a number of local images of the circuit board;

[0005] S2, performing distortion correction on a plurality of circuit board partial images, and forming image pairs from the distortion-corrected circuit board partial images to obtain a training set consisting of a plurality of circuit board image pairs;

[0006] S3, build an image registration network, use several circuit board image pairs in the training set to train the image registration network, the image registration network performs feature mapping, feature separation and feature exchange processing on the input image pair, obtains the reconstructed image of the input image pair according to the exchanged features, calculates the reconstruction loss according to the input image pair and its reconstructed image, and selects the network with the minimum reconstruction loss as the trained image registration network;

[0007] S4. Acquire a large baseline image to be registered in a real scene, perform distortion correction on the large baseline image to be registered, and input the image pair into a trained image registration network for processing. The trained image registration network performs feature mapping and feature separation on the input image pair, and outputs an image transformation matrix of the large baseline image pair to be registered in the real scene according to the overlapping area features obtained after feature separation. The registration of the large baseline image pair to be registered in the real scene is achieved according to the image transformation matrix.

[0008] Preferably, in S2, distortion correction is performed on a plurality of local images of the circuit board, which is specifically expressed by the formula:

[0009] ;

[0010] in, ;

[0011] In the formula, is the pixel coordinate on the local image of the circuit board after distortion correction, and are the physical sizes of the pixels on the local image plane of the circuit board in the horizontal and vertical directions, and are the pixel coordinates of the horizontal and vertical positions of the pixels in the local image plane of the circuit board, is the pixel coordinate on the local image of the circuit board before distortion correction, represents the local image coordinates of the circuit board after distortion correction, and are the distortion coefficients, Indicates the radial distance of pixels on the local image of the circuit board before distortion correction.

[0012] Preferably, the image registration network in S3 includes a twin pyramid module and an image reconstruction module and a parameter regression module connected in parallel to the twin pyramid module. The image pair is input into the twin pyramid module for processing to obtain a feature map, and feature separation and feature exchange processing are performed on the feature map, and exchange features and overlapping area features are output. The image reconstruction module receives and processes the exchange features and outputs the reconstructed image of the image pair. The parameter regression module receives and processes the overlapping area features and outputs the image transformation matrix of the image pair.

[0013] Preferably, each circuit board image pair in the training set includes a target image and a reference image. In S3, several circuit board image pairs in the training set are used to train the image registration network. The image registration network performs feature mapping, feature separation and feature exchange processing on the input image pairs, obtains a reconstructed image of the input image pair according to the exchanged features, calculates a reconstruction loss according to the input image pair and its reconstructed image, and selects a network with the minimum reconstruction loss as the trained image registration network. The specific process is as follows:

[0014] S31, preset the number of training times, perform the nth training on the image registration network, select an image pair from a number of circuit board image pairs in the training set at random and input it into the image registration network, the twin pyramid module receives and processes the selected image pair, and outputs a feature map corresponding to the selected image pair, including a feature map of the target image and a feature map of the reference image;

[0015] S32, performing feature separation on the feature map of the target image and the feature map of the reference image, respectively, to obtain overlapping region features and independent features corresponding to the target image and the reference image, respectively, and performing feature exchange on the overlapping region features of the target image and the overlapping region features of the reference image, to obtain exchange features corresponding to the target image and the reference image respectively;

[0016] S33, the image reconstruction module receives the exchange features corresponding to the target image and the reference image respectively and performs image reconstruction respectively to obtain the reconstructed images corresponding to the respective ones, and calculates the reconstruction loss according to the target image, the reference image and the reconstructed images corresponding to the target image and the reference image respectively;

[0017] S34, randomly selecting another image pair from the plurality of circuit board image pairs in the training set until all the circuit board image pairs are selected, and repeating S31 to S33 to obtain the reconstruction loss corresponding to each image pair after the nth training;

[0018] S35, training the image registration network for the n+1th time, repeating S31 to S34 until the preset number of training times is reached, obtaining the reconstruction loss corresponding to each image pair after each training, and selecting the network with the minimum reconstruction loss as the trained image registration network.

[0019] Preferably, the twin pyramid module in S31 receives and processes the selected image pair, and outputs a feature map corresponding to the selected image pair, which is specifically expressed by the formula:

[0020] ;

[0021] ;

[0022] In the formula, represents the reference image in the selected image pair, represents the target image in the selected image pair, Indicates the reference image for the selected image alignment The feature map of Indicates that the selected image is centered on the target image The feature map of and are the selected image centering reference images Overlapping regional features and independent features, and Align the selected images to the target image Overlapping regional features and independent features, represents the convolutional layer in the twin pyramid module, represents the normalization operation, Indicates activation operation. Indicates repeating 3 times.

[0023] Preferably, the image reconstruction module in S33 receives the exchange features corresponding to the target image and the reference image respectively and performs image reconstruction respectively to obtain the corresponding reconstructed images, which are specifically expressed by the formula: ;

[0024] In the formula, and denote the reconstructed reference image and the reconstructed target image respectively, represents the upsampling module in the image reconstruction module, represents the normalization operation, Indicates activation operation. Indicates repeating 4 times.

[0025] Preferably, in S33, the reconstruction loss is calculated according to the target image, the reference image, and the reconstructed images corresponding to the target image and the reference image, and the calculation formula of the reconstruction loss is specifically:

[0026] ;

[0027] In the formula, represents the 1 norm, represents the reconstruction loss.

[0028] Preferably, in S4, the large baseline image to be registered is subjected to distortion correction to form an image pair, which is input into the trained image registration network for processing. The trained image registration network performs feature mapping and feature separation processing on the input image pair, and outputs the image transformation matrix of the large baseline image pair to be registered in the real scene according to the overlapping area features obtained after feature separation. The specific process is as follows:

[0029] S41, performing distortion correction on the large baseline image to be registered, and inputting the distortion-corrected images into a large baseline image pair to be registered into the trained image registration network, wherein the large baseline image pair to be registered includes a target image to be registered and a reference image to be registered;

[0030] S42, the twin pyramid module in the trained image registration network receives and processes the target image to be registered and the reference image to be registered, and outputs a feature map of the target image to be registered and a feature map of the reference image to be registered;

[0031] S43, performing feature separation on the feature map of the target image to be registered and the feature map of the reference image to be registered, respectively, and obtaining overlapping region features corresponding to the target image to be registered and the reference image to be registered;

[0032] S44, the parameter regression module in the trained image registration network receives and processes the overlapping area features corresponding to the target image to be registered and the reference image to be registered, and outputs the image transformation matrix of the large baseline image pair to be registered.

[0033] Preferably, in S4, the registration of the large baseline image to be registered in the real scene is realized according to the image transformation matrix of the large baseline image pair to be registered, which is specifically expressed by the formula:

[0034] ;

[0035] In the formula, and Respectively represent the reference image and target image in the large baseline image pair to be registered, represents the first pixels, Represents the image transformation matrix corresponding to the large baseline image pair to be registered, represents the first The corresponding position of pixels on the reference image, and are the reference images in the large baseline image pair to be registered and the target image The corresponding overlapping area features, Represents a parameter regression module.

[0036] On the other hand, the present invention provides a large baseline image registration system, which uses the large baseline image registration method based on feature separation to perform image registration. The registration system includes an image acquisition module, an image processing module, an image registration network and a computer system. The image acquisition module is connected to the computer system, and the image processing module and the image registration network are connected in sequence and arranged in the computer system, wherein:

[0037] The image acquisition module is used to acquire the image to be registered and transmit the image to be registered to the computer system;

[0038] The image processing module in the computer system receives the images to be registered and performs distortion correction processing, and the processed images are combined into an image pair to be registered;

[0039] The image registration network in the computer system receives the image pair to be registered and performs registration processing, outputs the image transformation matrix of the image pair to be registered, realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix, and obtains the registered image.

[0040] The above-mentioned large baseline image registration method and system based on feature separation first collects several local images of circuit boards with low overlap rates and performs distortion correction, composes image pairs of the local images of circuit boards after distortion correction and constructs a training set, designs an image registration network, uses the training set to train the image registration network and calculates the reconstruction loss, obtains the trained image registration network, obtains the large baseline image to be registered in the real scene, composes image pairs after distortion correction and inputs them into the trained image registration network for processing, outputs the image transformation matrix of the large baseline image pair to be registered in the real scene, and realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix. The method performs feature mapping, feature separation and feature exchange processing on the input image pair through the image registration network, wherein feature separation can reduce the influence of redundant features on the image transformation matrix, and can significantly improve the accuracy, efficiency and stability of image registration in actual industrial applications, thereby providing strong technical support for automated detection, quality control and production optimization in intelligent manufacturing. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a large baseline image registration method based on feature separation in one embodiment of the present invention;

[0042] Figure 2 Schematic diagram of the structure of the image registration network in one embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0044] See also Figure 1 , a large baseline image registration method based on feature separation, comprising the following steps:

[0045] S1. Capturing images of a circuit board at a preset overlap rate to obtain a plurality of local images of the circuit board.

[0046] Specifically, in order to ensure the accuracy and efficiency of subsequent image registration, the overlap rate is set to about 20%, for example, 15%, 20% or 25%, and a lens (such as XF-PTL03723-C-VI) and an industrial camera (such as MV-CH250-90TC-C-NF) are used to capture images of the circuit board to obtain several local images of the circuit board.

[0047] S2. Perform distortion correction on a plurality of circuit board local images, and form image pairs with the distortion-corrected circuit board local images to obtain a training set consisting of a plurality of circuit board image pairs.

[0048] Specifically, due to the camera's reasons, the captured images are distorted. In order to ensure the geometric accuracy of the local images of the circuit board and alleviate the subsequent impact on image registration, several local images of the circuit board are subjected to distortion correction. Adjacent local images of the circuit board after distortion correction (that is, images with overlapping areas) are combined into image pairs to obtain several circuit board image pairs. Several circuit board image pairs constitute a training set.

[0049] Furthermore, in S2, distortion correction is performed on several local images of the circuit board, which is specifically expressed by the formula:

[0050] (1)

[0051] in, (1) -1

[0052] In the formula, is the pixel coordinate on the local image of the circuit board after distortion correction, and are the physical sizes of the pixels on the local image plane of the circuit board in the horizontal and vertical directions, and are the pixel coordinates of the horizontal and vertical positions of the pixels in the local image plane of the circuit board, is the pixel coordinate on the local image of the circuit board before distortion correction, represents the local image coordinates of the circuit board after distortion correction, and are the distortion coefficients, Represents the radial distance of pixels on the local image of the circuit board before distortion correction, .

[0053] S3. Build an image registration network and use several circuit board image pairs in the training set to train the image registration network. The image registration network performs feature mapping, feature separation and feature exchange on the input image pairs, and obtains the reconstructed image of the input image pair based on the exchanged features. The reconstruction loss is calculated based on the input image pair and its reconstructed image, and the network with the smallest reconstruction loss is selected as the trained image registration network.

[0054] Further, see Figure 2 The image registration network in S3 includes a twin pyramid module, an image reconstruction module and a parameter regression module connected in parallel with the twin pyramid module. The image pair is input into the twin pyramid module for processing to obtain a feature map, and feature separation and feature exchange processing are performed on the feature map, and exchange features and overlapping area features are output. The image reconstruction module receives and processes the exchange features and outputs the reconstructed image of the image pair. The parameter regression module receives and processes the overlapping area features and outputs the image transformation matrix of the image pair.

[0055] Furthermore, each circuit board image pair in the training set includes a target image and a reference image. In S3, several circuit board image pairs in the training set are used to train the image registration network. The image registration network performs feature mapping, feature separation and feature exchange processing on the input image pair, obtains a reconstructed image of the input image pair according to the exchanged features, calculates the reconstruction loss according to the input image pair and its reconstructed image, and selects the network with the minimum reconstruction loss as the trained image registration network. The specific process is as follows:

[0056] S31, preset the number of training times, perform the nth training on the image registration network, select any one image pair from several circuit board image pairs in the training set and input it into the image registration network, the twin pyramid module receives and processes the selected image pair, and outputs the feature map corresponding to the selected image pair, including the feature map of the target image and the feature map of the reference image.

[0057] Furthermore, the twin pyramid module in S31 receives and processes the selected image pair, and outputs a feature map corresponding to the selected image pair, which can be specifically expressed as:

[0058] (2)

[0059] (3)

[0060] In the formula, represents the reference image in the selected image pair, represents the target image in the selected image pair, Indicates the reference image for the selected image alignment The feature map of Indicates that the selected image is centered on the target image The feature map of and are the selected image centering reference images Overlapping regional features and independent features, and Align the selected images to the target image Overlapping regional features and independent features, represents the twin pyramid module, represents the convolutional layer in the twin pyramid module, represents the normalization operation, Indicates activation operation. Indicates repeating 3 times.

[0061] Specifically, the twin pyramid module includes four convolutional layers with a stride of 2. Normalization and ReLu activation operations are performed in sequence after the first three convolutional layers. The twin pyramid module receives and processes the selected image pairs and outputs the feature maps corresponding to the selected image pairs.

[0062] S32, perform feature separation on the feature map of the target image and the feature map of the reference image respectively, and obtain the overlapping area features and independent features corresponding to the target image and the reference image respectively, and perform feature exchange on the overlapping area features of the target image and the overlapping area features of the reference image to obtain the exchange features corresponding to the target image and the reference image respectively.

[0063] Specifically, the feature map of the target image and the feature map of the reference image can be divided into two parts, one part is the overlapping area features of the image pair, and the other part is the independent features. When training the image registration network, for each input image pair, the overlapping area features of the target image and the overlapping area features of the reference image are exchanged, and the exchanged overlapping area features and the original independent features constitute the exchange features corresponding to the target image and the reference image respectively. The exchange features are used to reconstruct the image and calculate the reconstruction loss, and the overlapping area features are used to calculate the image transformation matrix of the input image pair.

[0064] S33, the image reconstruction module receives the exchange features corresponding to the target image and the reference image respectively and performs image reconstruction respectively to obtain the reconstructed images corresponding to each other, and calculates the reconstruction loss according to the target image, the reference image and the reconstructed images corresponding to each other.

[0065] Furthermore, the image reconstruction module in S33 receives the exchange features corresponding to the target image and the reference image respectively and performs image reconstruction respectively to obtain the corresponding reconstructed images, which are specifically expressed as:

[0066] (4)

[0067] (5)

[0068] In the formula, and denote the reconstructed reference image and the reconstructed target image respectively, represents the image reconstruction module, represents the upsampling module in the image reconstruction module, represents the normalization operation, Indicates activation operation. Indicates repeating 4 times.

[0069] Specifically, the image reconstruction module includes four upsampling modules connected in sequence, and normalization and ReLu activation operations are performed in sequence after each upsampling module. The four upsampling modules connected in sequence receive the interaction features and perform upsampling processing to obtain a reconstructed image pair.

[0070] Furthermore, in S33, the reconstruction loss is calculated according to the target image, the reference image, and the reconstructed images corresponding to the target image and the reference image. The calculation formula of the reconstruction loss is specifically:

[0071] (6)

[0072] In the formula, represents the 1 norm, represents the reconstruction loss.

[0073] S34, randomly selecting another image pair from the plurality of circuit board image pairs in the training set until all the circuit board image pairs are selected, and repeating S31 to S33 to obtain the reconstruction loss corresponding to each image pair after the nth training;

[0074] S35, training the image registration network for the n+1th time, repeating S31 to S34 until the preset number of training times is reached, obtaining the reconstruction loss corresponding to each image pair after each training, and selecting the network with the minimum reconstruction loss as the trained image registration network.

[0075] It should be noted that during the training process, the parameter regression module will also receive and process the overlapping area features corresponding to the target image and the reference image to obtain the image transformation matrix corresponding to the selected image pair. In other words, each input image pair corresponds to an image transformation matrix, but if the calculated reconstruction loss is large, then the registration of the input image pair using the obtained image transformation matrix will not be so accurate.

[0076] Furthermore, the parameter regression module is composed of a modified ResNet-34. In order to regress the image transformation matrix parameters using the features of the overlapping area, the last fully connected layer of the network is modified to 8 (the image transformation matrix parameters are 8 values) based on the existing ResNet-34 network. The image transformation matrix corresponding to the selected image pair can be specifically expressed as:

[0077] (7)

[0078] In the formula, Represents the image transformation matrix corresponding to the selected image pair, represents the parameter regression module, and Respectively represent the reference image and target image in the selected image pair, and are the reference images in the selected image pair. and the target image The corresponding overlapping area features.

[0079] The image reconstruction module is mainly used to reconstruct the interactive features corresponding to the target image and the reference image when training the image registration network, and obtain the reconstructed images corresponding to the target image and the reference image. By calculating the reconstruction loss between the reconstructed image output after feature exchange and the original image through feature interaction, the training process of the image registration network can be better supervised, so that the trained image registration network can better achieve feature separation.

[0080] S4. Acquire a large baseline image to be registered in a real scene, perform distortion correction on the large baseline image to be registered, and input the image pair into a trained image registration network for processing. The trained image registration network performs feature mapping and feature separation on the input image pair, and outputs an image transformation matrix of the large baseline image pair to be registered in the real scene according to the overlapping area features obtained after feature separation. The registration of the large baseline image pair to be registered in the real scene is achieved according to the image transformation matrix.

[0081] Furthermore, in S4, the large baseline image to be registered is subjected to distortion correction to form an image pair, which is input into the trained image registration network for processing, and the image transformation matrix of the large baseline image pair to be registered in the real scene is output. The specific process is as follows:

[0082] S41, performing distortion correction on the large baseline image to be registered, and inputting the distortion-corrected images into a large baseline image pair to be registered into the trained image registration network, wherein the large baseline image pair to be registered includes a target image to be registered and a reference image to be registered;

[0083] S42, the twin pyramid module in the trained image registration network receives and processes the target image to be registered and the reference image to be registered, and outputs a feature map of the target image to be registered and a feature map of the reference image to be registered;

[0084] S43, performing feature separation on the feature map of the target image to be registered and the feature map of the reference image to be registered, respectively, and obtaining overlapping region features corresponding to the target image to be registered and the reference image to be registered;

[0085] S44, the parameter regression module in the trained image registration network receives and processes the overlapping area features corresponding to the target image to be registered and the reference image to be registered, and outputs the image transformation matrix of the large baseline image pair to be registered.

[0086] Furthermore, the parameter regression module in S44 receives and processes the overlapping region features, and outputs an image transformation matrix of the image transformation matrix of the large baseline image pair to be registered, which is specifically expressed by the formula:

[0087] (8)

[0088] In the formula, Represents the image transformation matrix corresponding to the large baseline image pair to be registered, represents the parameter regression module, and Respectively represent the reference image and target image in the large baseline image pair to be registered, and are the reference images in the large baseline image pair to be registered and the target image The corresponding overlapping area features.

[0089] Furthermore, in S4, the registration of the large baseline image to be registered in the real scene is realized according to the image transformation matrix of the large baseline image pair to be registered, which is specifically expressed by the formula:

[0090] (9)

[0091] In the formula, represents the first pixels, represents the first The corresponding position of pixels in the reference image.

[0092] Specifically, in real scenarios, the trained image registration network can effectively extract the overlapping area features of the input large baseline image pair to be registered. At this time, the image reconstruction module no longer participates in image registration. The twin pyramid module in the trained image registration network receives and processes the target image to be registered and the reference image to be registered, outputs the feature maps corresponding to the target image to be registered and the reference image, separates the overlapping area features from the corresponding feature maps, and the parameter regression module receives and processes the overlapping area features, outputs the image transformation matrix of the large baseline image pair to be registered, and registers the large baseline image pair to be registered according to the image transformation matrix, specifically mapping the pixel values ​​of the target image to the reference image, thereby achieving matching alignment of the large baseline image pair to be registered.

[0093] In one embodiment, a large baseline image registration system is provided, which uses the large baseline image registration method based on feature separation to perform image registration. The system includes an image acquisition module, an image processing module, an image registration network and a computer system. The image acquisition module is connected to the computer system, and the image processing module and the image registration network are connected in sequence and arranged in the computer system, wherein:

[0094] The image acquisition module is used to acquire the image to be registered and transmit the image to be registered to the computer system;

[0095] The image processing module in the computer system receives the images to be registered and performs distortion correction processing, and the processed images are combined into an image pair to be registered;

[0096] The image registration network in the computer system receives the image pair to be registered and performs registration processing, outputs the image transformation matrix of the image pair to be registered, realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix, and obtains the registered image.

[0097] Specifically, the image registration system includes an image acquisition module and a computer system. The image acquisition module is composed of a lens XF-PTL03723-C-VI and is equipped with an industrial camera MV-CH250-90TC-C-NF. The computer system is provided with an image processing module and an image registration network connected in sequence. The local image of a high-definition circuit board with a low overlap rate acquired by the image acquisition module is subjected to image distortion correction by the image processing module. Then, the images after distortion correction are combined into an image pair to be registered and sent to the image registration network to obtain an image transformation matrix. The image transformation matrix is ​​used to achieve matching alignment of the image pair to be registered.

[0098] The above-mentioned large baseline image registration method and system based on feature separation first collects several local images of circuit boards with low overlap rates and performs distortion correction, composes image pairs of the local images of circuit boards after distortion correction and constructs a training set, designs an image registration network, uses the training set to train the image registration network and calculates the reconstruction loss, obtains the trained image registration network, obtains the large baseline image to be registered in the real scene, composes image pairs after distortion correction and inputs them into the trained image registration network for processing, outputs the image transformation matrix of the large baseline image pair to be registered in the real scene, and realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix. The method performs feature mapping, feature separation and feature exchange processing on the input image pair through the image registration network, wherein feature separation can reduce the influence of redundant features on the image transformation matrix, and can significantly improve the accuracy, efficiency and stability of image registration in actual industrial applications, thereby providing strong technical support for automated detection, quality control and production optimization in intelligent manufacturing.

[0099] The above is a detailed introduction to a large baseline image registration method and system based on feature separation provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A large baseline image registration method based on feature separation, characterized in that: The method comprises the following steps: S1. Capturing images of the circuit board at a preset overlap rate to obtain a number of local images of the circuit board; S2, performing distortion correction on a plurality of circuit board partial images, and forming image pairs from the distortion-corrected circuit board partial images to obtain a training set consisting of a plurality of circuit board image pairs; S3, build an image registration network, use several circuit board image pairs in the training set to train the image registration network, the image registration network performs feature mapping, feature separation and feature exchange processing on the input image pair, obtains the reconstructed image of the input image pair according to the exchanged features, calculates the reconstruction loss according to the input image pair and its reconstructed image, and selects the network with the minimum reconstruction loss as the trained image registration network; the image registration network includes a twin pyramid module and an image reconstruction module and a parameter regression module connected in parallel with the twin pyramid module, the image pair is input into the twin pyramid module for processing, feature mapping is obtained, feature separation and feature exchange processing are performed on the feature mapping, exchange features and overlapping area features are output, the image reconstruction module receives and processes the exchange features, outputs the reconstructed image of the image pair, the parameter regression module receives and processes the overlapping area features, and outputs the image transformation matrix of the image pair; S4. Acquire a large baseline image to be registered in a real scene, perform distortion correction on the large baseline image to be registered, and input the image pair into a trained image registration network for processing. The trained image registration network performs feature mapping and feature separation on the input image pair, and outputs an image transformation matrix of the large baseline image pair to be registered in the real scene according to the overlapping area features obtained after feature separation. The registration of the large baseline image pair to be registered in the real scene is achieved according to the image transformation matrix.

2. The large baseline image registration method based on feature separation as claimed in claim 1, characterized in that: In S2, several local images of circuit boards are subjected to distortion correction, which is specifically expressed by the formula: ； in, ; In the formula, is the pixel coordinate on the local image of the circuit board after distortion correction, and are the physical sizes of the pixels on the local image plane of the circuit board in the horizontal and vertical directions, and are the pixel coordinates of the horizontal and vertical positions of the pixels in the local image plane of the circuit board, is the pixel coordinate on the local image of the circuit board before distortion correction, represents the local image coordinates of the circuit board after distortion correction, and are the distortion coefficients, Indicates the radial distance of pixels on the local image of the circuit board before distortion correction.

3. The large baseline image registration method based on feature separation as claimed in claim 2, characterized in that: Each circuit board image pair in the training set includes a target image and a reference image. S3 uses several circuit board image pairs in the training set to train the image registration network. The image registration network performs feature mapping, feature separation and feature exchange on the input image pairs, and obtains the reconstructed image of the input image pair based on the exchanged features. The reconstruction loss is calculated based on the input image pair and its reconstructed image, and the network with the smallest reconstruction loss is selected as the trained image registration network. The specific process is as follows: S31, preset the number of training times, perform the nth training on the image registration network, select an image pair from a number of circuit board image pairs in the training set at random and input it into the image registration network, the twin pyramid module receives and processes the selected image pair, and outputs a feature map corresponding to the selected image pair, including a feature map of the target image and a feature map of the reference image; S32, performing feature separation on the feature map of the target image and the feature map of the reference image, respectively, to obtain overlapping region features and independent features corresponding to the target image and the reference image, respectively, and performing feature exchange on the overlapping region features of the target image and the overlapping region features of the reference image, to obtain exchange features corresponding to the target image and the reference image respectively; S33, the image reconstruction module receives the exchange features corresponding to the target image and the reference image respectively and performs image reconstruction respectively to obtain the reconstructed images corresponding to the respective ones, and calculates the reconstruction loss according to the target image, the reference image and the reconstructed images corresponding to the target image and the reference image respectively; S34, randomly selecting another image pair from the plurality of circuit board image pairs in the training set until all the circuit board image pairs are selected, and repeating S31 to S33 to obtain the reconstruction loss corresponding to each image pair after the nth training; S35, training the image registration network for the n+1th time, repeating S31 to S34 until the preset number of training times is reached, obtaining the reconstruction loss corresponding to each image pair after each training, and selecting the network with the minimum reconstruction loss as the trained image registration network.

4. The large baseline image registration method based on feature separation as claimed in claim 3, characterized in that: The twin pyramid module in S31 receives and processes the selected image pair and outputs the feature map corresponding to the selected image pair, which is specifically expressed by the formula: ； ； In the formula, represents the reference image in the selected image pair, represents the target image in the selected image pair, Indicates the reference image for the selected image alignment The feature map of Indicates that the selected image is centered on the target image The feature map of and are the selected image centering reference images Overlapping regional features and independent features, and Align the selected images to the target image Overlapping regional features and independent features, represents the convolutional layer in the twin pyramid module, represents the normalization operation, Indicates activation operation. Indicates repeating 3 times.

5. The large baseline image registration method based on feature separation as claimed in claim 4, characterized in that: The image reconstruction module in S33 receives the exchange features corresponding to the target image and the reference image and reconstructs the images respectively to obtain the corresponding reconstructed images, which are specifically expressed by the formula: ； In the formula, and denote the reconstructed reference image and the reconstructed target image respectively, represents the upsampling module in the image reconstruction module, represents the normalization operation, Indicates activation operation. Indicates repeating 4 times.

6. The large baseline image registration method based on feature separation as claimed in claim 5, characterized in that: In S33, the reconstruction loss is calculated according to the target image, the reference image, and the reconstructed images corresponding to the target image and the reference image. The calculation formula of the reconstruction loss is specifically: ； In the formula, represents the 1 norm, represents the reconstruction loss.

7. The large baseline image registration method based on feature separation as claimed in claim 6, characterized in that: In S4, the large baseline image to be registered is subjected to distortion correction to form an image pair, which is input into the trained image registration network for processing. The trained image registration network performs feature mapping and feature separation on the input image pair, and outputs the image transformation matrix of the large baseline image pair to be registered in the real scene according to the overlapping area features obtained after feature separation. The specific process is as follows: S41, performing distortion correction on the large baseline image to be registered, and inputting the distortion-corrected images into a large baseline image pair to be registered into the trained image registration network, wherein the large baseline image pair to be registered includes a target image to be registered and a reference image to be registered; S42, the twin pyramid module in the trained image registration network receives and processes the target image to be registered and the reference image to be registered, and outputs a feature map of the target image to be registered and a feature map of the reference image to be registered; S43, performing feature separation on the feature map of the target image to be registered and the feature map of the reference image to be registered, respectively, and obtaining overlapping region features corresponding to the target image to be registered and the reference image to be registered; S44, the parameter regression module in the trained image registration network receives and processes the overlapping area features corresponding to the target image to be registered and the reference image to be registered, and outputs the image transformation matrix of the large baseline image pair to be registered.

8. The large baseline image registration method based on feature separation as claimed in claim 1, characterized in that: In S4, the registration of the large baseline image to be registered in the real scene is realized according to the image transformation matrix of the large baseline image pair to be registered, which is specifically expressed by the formula: ； In the formula, and Respectively represent the reference image and target image in the large baseline image pair to be registered, represents the first pixels, Represents the image transformation matrix corresponding to the large baseline image pair to be registered, represents the first The corresponding position of pixels on the reference image, and are the reference images in the large baseline image pair to be registered and the target image The corresponding overlapping area features, Represents a parameter regression module.

9. A large baseline image registration system, using the large baseline image registration method based on feature separation as described in any one of claims 1 to 8 for image registration, characterized in that: The registration system comprises an image acquisition module, an image processing module, an image registration network and a computer system, wherein the image acquisition module is connected to the computer system, and the image processing module and the image registration network are sequentially connected and arranged in the computer system, wherein: The image acquisition module is used to acquire the image to be registered and transmit the image to be registered to the computer system; The image processing module in the computer system receives the images to be registered and performs distortion correction processing, and the processed images are combined into an image pair to be registered; The image registration network in the computer system receives the image pair to be registered and performs registration processing, outputs the image transformation matrix of the image pair to be registered, realizes the registration of the large baseline image pair to be registered in the real scene according to the image transformation matrix, and obtains the registered image.

Images