Image registration method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image registration method, device, electronic equipment and storage medium, which can be applied to the field of financial technology. The method preprocesses a first image to be registered to obtain a second image to be registered consistent with the size of a pre-specified standard image, inputs the second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image, and inputs the fine registration image into a spatial transformation network model to obtain a target image after registration. The VGG network model is trained based on at least one training image group, and each training image group includes a first work order image uploaded by a user and a standard work order image. In this technical solution, the image to be registered is first cropped to be consistent with the size of the standard image, and then based on the VGG network model, the registration accuracy and efficiency of the image are improved in identifying the registration of the user-uploaded image and the standard image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of financial technology, and particularly relates to an image registration method and device, electronic equipment and storage medium. BACKGROUND

[0002] Image registration technology is a key technology of image recognition and is also an indispensable process in image fusion processing. Registration is to find an optimal spatial transformation mapping between input image pairs so that the points in space between the input image pairs are one-to-one corresponding. Image registration is mainly divided into registration based on traditional algorithms and registration based on deep learning, which is a recent research hotspot.

[0003] The image registration method based on deep learning is mainly divided into two categories. One is to learn the similarity metric between the image pairs to be registered by the deep learning through the network, to calculate the difference between the image pairs by using a suitable loss function, and to feed back to the deep network for optimization, so as to minimize the difference between the images. The other is to use the network to train and predict the conversion parameters between the input images.

[0004] However, in the prior art, the similarity metric is estimated by using deep learning, and the traditional registration method is still used for iterative optimization. The time consumption of registration is relatively large, and the accuracy is low. SUMMARY

[0005] The present disclosure provides an image registration method, device, electronic equipment and storage medium to solve the technical problem of low efficiency and accuracy in image registration operation in the prior art.

[0006] In a first aspect, an embodiment of the present disclosure provides an image registration method, comprising:

[0007] preprocessing a first image to be registered to obtain a second image to be registered consistent in size with a pre-specified standard image;

[0008] inputting the second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image;

[0009] inputting the fine registration image into a spatial transformation network model to obtain a target image after registration;

[0010] The VGG network model is trained based on at least one training image group, and each training image group includes a first work order image uploaded by a user and a standard work order image.

[0011] In a possible implementation of the first aspect, the inputting the second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image comprises:

[0012] obtaining the standard image;

[0013] inputting the second image to be registered and the standard image into a first layer of the VGG network model to obtain a first deformation field between the second image to be registered and the standard image;

[0014] warping the standard image based on the first deformation field to obtain a coarse registration image;

[0015] determining the fine registration image according to the coarse registration image and the second image to be registered.

[0016] Optionally, the determining the fine registration image according to the coarse registration image and the second image to be registered comprises:

[0017] inputting the coarse registration image and the second image to be registered into a second layer of the VGG network model to obtain a second deformation field between the coarse registration image and the second image to be registered;

[0018] warping the coarse registration image based on the second deformation field to obtain the fine registration image.

[0019] In another possible implementation of the first aspect, before the inputting the second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image, the method further comprises:

[0020] for each training image group, inputting a first work order image and a standard work order image in the training image group into the VGG network model to obtain a corresponding registration image;

[0021] evaluating a similarity between the registration image and the standard image to optimize the VGG network model by using a stochastic gradient descent.

[0022] In still another possible implementation of the first aspect, the spatial transformation network model is used to guide the VGG network model to perform linear transformation of translation, warping, and rotation.

[0023] In yet another possible implementation of the first aspect, before the inputting the second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image, the method further comprises:

[0024] perform image enhancement processing on the second to-be-registered image to obtain a second to-be-registered image after image enhancement.

[0025] In a second aspect, the embodiments of the present disclosure provide an image registration device, comprising:

[0026] a processing module configured to perform preprocessing on a first to-be-registered image to obtain a second to-be-registered image consistent in size with a pre-specified standard image;

[0027] a first input module configured to input the second to-be-registered image and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine-registered image;

[0028] a second input module configured to input the fine-registered image into a spatial transformation network model to obtain a target image after registration;

[0029] The VGG network model is trained based on at least one training image group, and each training image group includes a first work order image uploaded by a user and a standard work order image.

[0030] In a possible implementation of the second aspect, the first input module is specifically configured to:

[0031] obtain the standard image;

[0032] input the second to-be-registered image and the standard image into a first layer of the VGG network model to obtain a first deformation field between the second to-be-registered image and the standard image;

[0033] perform warping deformation on the standard image based on the first deformation field to obtain a coarse-registered image;

[0034] determine the fine-registered image according to the coarse-registered image and the second to-be-registered image.

[0035] Optionally, the first input module determines the fine-registered image according to the coarse-registered image and the second to-be-registered image, and is specifically configured to:

[0036] input the coarse-registered image and the second to-be-registered image into a second layer of the VGG network model to obtain a second deformation field between the coarse-registered image and the second to-be-registered image;

[0037] perform warping deformation on the coarse-registered image based on the second deformation field to obtain the fine-registered image.

[0038] In a second possible implementation of the second aspect, before the inputting the second image to be registered and the standard image into the pre-trained visual geometry group (VGG) network model to obtain a fine registration image, the processing module is further configured to:

[0039] For each training image group, input a first work order image and a standard work order image in the training image group into the VGG network model to obtain a corresponding registration image.

[0040] Evaluate the similarity between the registration image and the standard image to optimize the VGG network model using a stochastic gradient descent.

[0041] In a third possible implementation of the second aspect, the spatial transformation network model is configured to guide the VGG network model to perform linear transformation of translation, distortion, and rotation.

[0042] In a fourth possible implementation of the second aspect, before the inputting the second image to be registered and the standard image into the pre-trained visual geometry group (VGG) network model to obtain a fine registration image, the processing module is further configured to:

[0043] Perform image enhancement processing on the second image to be registered to obtain an image-enhanced second image to be registered.

[0044] In a third aspect, the embodiments of the present disclosure provide an electronic device, including a processor, and a memory and a transceiver connected to the processor in communication;

[0045] The memory stores computer-executable instructions; and the transceiver is configured to transmit and receive data.

[0046] The processor executes the computer-executable instructions stored in the memory to implement the image registration method of the first aspect or any of the manners described above.

[0047] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the image registration method of the first aspect or any of the manners described above.

[0048] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the image registration method of the first aspect or any of the manners described above.

[0049] The image registration method, apparatus, electronic device, and storage medium provided in this disclosure can be applied to the financial technology field. The method preprocesses a first image to be registered to obtain a second image to be registered with the same size as a pre-defined standard image. The second image to be registered and the standard image are then input into a pre-trained Visual Geometric Group (VGG) network model to obtain a finely registered image. This finely registered image is then input into a spatial transformation network model to obtain the registered target image. The VGG network model is trained based on at least one set of training images, each set including a user-uploaded first work order image and a standard work order image. This technical solution first crops the image to be registered to match the size of the standard image. Then, based on the VGG network model, it improves the accuracy and efficiency of image registration in identifying the user-uploaded image and the standard image for registration. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0051] Figure 1 A schematic diagram of a neural network model provided in an embodiment of this disclosure;

[0052] Figure 2 Flowchart of the image registration method provided in the embodiments of this disclosure Figure 1 ;

[0053] Figure 3 This is a schematic diagram of a spatial transformation network model provided in an embodiment of the present disclosure;

[0054] Figure 4 A flowchart illustrating the image registration method provided in this embodiment. Figure 2 ;

[0055] Figure 5 Flowchart of the image registration method provided in the embodiments of this disclosure Figure 3 ;

[0056] Figure 6 A schematic diagram of the registration framework provided for embodiments of this disclosure;

[0057] Figure 7 This is a schematic diagram of the structure of the image registration device provided in the embodiments of this disclosure;

[0058] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0059] The specific embodiments of the present disclosure have been shown by the above drawings, and will be described in more detail hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present disclosure in any way, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0060] In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present disclosure.

[0061] Before introducing the embodiments of the present disclosure, the professional terms and technical backgrounds involved in the embodiments of the present disclosure are first explained:

[0062] Visual Geometry Group (VGG)-16 original neural network model: as shown in Figure 1 , the neural network model schematic diagram provided by the embodiments of the present disclosure is shown. Figure 1

[0063] As shown in Figure 1 , the neural network model schematic diagram provided by the embodiments of the present disclosure is shown.

[0064] First part: an original picture is changed to (224, 224, 3).

[0065] Second part: five convolutional blocks for feature extraction, specifically:

[0066] 1. The first convolutional block: conv1 twice [3, 3] convolutional network, the output feature layer is 64, the output is (224, 224, 64), and then 2X2 maximum pooling, the output net is (112, 112, 64);

[0067] 2. The second convolutional block: conv2 twice [3, 3] convolutional network, the output feature layer is 128, the output net is (112, 112, 128), and then 2X2 maximum pooling, the output net is (56, 56, 128).

[0068] 3. The third convolutional block: conv3 three times [3, 3] convolutional network, the output feature layer is 256, the output net is (56, 56, 256), and then 2X2 maximum pooling, the output net is (28, 28, 256).

[0069] ​4. The fourth convolutional block: conv3 three times [3, 3] convolution network, the output feature layer is 256, the output net is (28, 28, 512), and then 2X2 maximum pooling, the output net is (14, 14, 512).

[0070] 5. The fifth convolutional block: conv3 three times [3, 3] convolution network, the output feature layer is 256, the output net is (14, 14, 512), and then 2X2 maximum pooling, the output net is (7, 7, 512) = 25088 parameters.

[0071] The third part: operation of the network for classification task, specifically:

[0072] 1. Using the convolutional method to simulate the full connection layer, the effect is equivalent, the output net is (1, 1, 4096) - 25088*4096+4096 = 102764544. A total of twice.

[0073] 2. Using the convolutional method to simulate the full connection layer, the effect is equivalent, the output net is (1, 1, 1000). The final output is the prediction of each class.

[0074] Image registration technology is a key technology of image recognition, and is also an essential process in image fusion processing. Registration is to find an optimal spatial transformation mapping between input image pairs, so that the points between the input image pairs are one-to-one corresponding in space. Image registration mainly includes traditional algorithm-based registration and the recent research hotspot of deep learning-based registration.

[0075] The deep learning-based image registration method mainly includes two categories. One is to use deep learning to learn the similarity metric between the to-be-registered image pairs through the network, calculate the difference between the image pairs by using a suitable loss function, and feed back to the deep network for optimization, so as to minimize the difference between the images. The other is to use the network to train and predict the conversion parameters between the input images.

[0076] The former method only estimates the similarity metric by using deep learning, and still uses the traditional registration method for iterative optimization, which cannot shorten the time consumption of registration well.

[0077] The existing problems in the prior art need to be solved by the embodiments of the present disclosure: how to improve the efficiency and accuracy of registration.

[0078] In view of the technical problems existing in the prior art, the inventor of the present disclosure conceives as follows: the work order images uploaded by users are different in angle and size, which affects the classification effect of the work order images. Therefore, a standard work order image template is provided, and more and more reliable image feature points are obtained after registration with the image uploaded by the user, so as to improve the image quality and prepare for subsequent image classification. Then, an image registration method based on unsupervised deep learning of iteration VGG-16 (Visual Geometry Group) can be used, and the network model of iteration VGG-16 is used to improve the accuracy and efficiency of image registration in identifying the registration of the image uploaded by the user and the standard image.

[0079] In the following, the technical solutions of the present disclosure will be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in detail in some embodiments.

[0080] It should be noted that the application field of the image registration method, device, electronic equipment and storage medium of the present disclosure is not limited, and can be related to the field of financial technology or other related fields.

[0081] Among them, the execution subject of the present disclosure is an electronic device, which can be a personal computer, a mobile terminal, a server, etc.

[0082] Figure 2 Flowchart of the image registration method provided by the embodiment of the present disclosure Figure 1 As shown in the image registration method, the image registration method can include the following steps: Figure 2

[0083] Step 21, pre-processing the first to-be-registered image to obtain a second to-be-registered image consistent in size with a pre-specified standard image.

[0084] In this step, before the user performs registration processing on the first to-be-registered image and the pre-specified standard image, the first to-be-registered image is pre-processed to obtain a second to-be-registered image consistent in size with the standard image.

[0085] Optionally, the pre-processing process can be: scaling or enlarging the first to-be-registered image to crop a second to-be-registered image consistent in size with the standard image.

[0086] Further, the second to-be-registered image is subjected to image enhancement processing to obtain an image-enhanced second to-be-registered image.

[0087] In this implementation, a high-pass filter can be used to enhance image quality, which can better extract work order number features in the second to-be-registered image. ​

[0088] Step 22, inputting the second image to be registered and the standard image into the pre-trained VGG network model to obtain a fine registration image.

[0089] The VGG network model is trained based on at least one training image group, and each training image group includes a first work order image uploaded by a user and a standard work order image.

[0090] In this step, the second image to be registered after image enhancement and the pre-specified standard image are input into the pre-trained VGG network model, and a fine registration image is output.

[0091] Before this step, the VGG network model also needs to be trained to obtain a model capable of processing registration images, which can be:

[0092] The input is a pair of M images and F images (where M is a first work order image uploaded by a user, and F is a provided standard work order image). After the M image and the F image pair to be registered first enter the first layer network of the VGG, a deformation field between the M image and the F image is obtained for the first time, and the F image is deformed to obtain a coarsely registered image.

[0093] Then, the VGG neural network is trained for the second time to achieve fine registration. At this time, the input of the VGG network is the coarsely registered image and the M image, and the deformation field between the M image and the coarsely registered image is obtained again. The coarsely registered image is deformed for the second time to obtain a fine registration image.

[0094] In order to optimize the registration result of the M image and the F image, a mean square error and gradient descent loss function is used to estimate the difference value of the M and F images, and the difference value is fed back to the VGG neural network to update the parameters of the VGG neural network.

[0095] Step 23, inputting the fine registration image into the spatial transformation network model to obtain a target image after registration.

[0096] The spatial transformation network model is used to guide the VGG network model to linearly transform translation, distortion, and rotation.

[0097] In this step, since the VGG neural network cannot well realize the invariance of the network model to translation, rotation, and other distortion, the learning of linear transformations such as translation, distortion, and rotation of image morphology is not accurate enough. The spatial transformation network is added after the VGG neural network to guide the deformation of the image, that is, the fine registration image is input into the spatial transformation network model to obtain a target image after registration.

[0098] The spatial transformation network model provided by the embodiment of the disclosure is shown in the following schematic diagram. Figure 3 The spatial transformation network model provided by the embodiment of the disclosure is shown in the following schematic diagram.Figure 3 As shown in the following figure:

[0099] Spatial Transformer Networks (STN) is a new learning model in convolutional neural networks, which can be used as a separate module or added anywhere in the convolutional neural network. Since the convolutional neural network cannot truly achieve invariance for large-scale spatial transformation, the spatial transformation network can actively transform the features of the neural network without additional network training, and achieve the invariance of the network model to translation, rotation and other distortions. Spatial transformation network consists of three parts: localization network, parameterized grid generator and differentiable image sampling.

[0100] That is, input the fine registration image U, and output the target image V.

[0101] The image registration method provided by the embodiments of the present disclosure can be applied to the field of financial technology. The method pre-processes the first to-be-registered image to obtain a second to-be-registered image consistent in size with a pre-specified standard image, inputs the second to-be-registered image and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image, and inputs the fine registration image into a spatial transformation network model to obtain a target image after registration. The VGG network model is trained based on at least one training image group, and each training image group includes a first work order image uploaded by a user and a standard work order image. In this technical solution, the to-be-registered image is first cropped to be consistent in size with the standard image, and then based on the VGG network model, the registration accuracy and efficiency of the image are improved in identifying the registration of the user-uploaded image and the standard image.

[0102] On the basis of the above-mentioned embodiments, Figure 4 The flowchart of the image registration method provided by the embodiments of the present disclosure is shown in the following figure: Figure 2 As shown in the following figure, step 22 in the image registration method can be implemented in the following way: Figure 4

[0103] Step 41, obtaining a standard image.

[0104] In this step, the standard image can be a reference image in image registration, which is an image preset for the user.

[0105] ​​For example, the standard image can be a diffusion-weighted imaging (DWI) image.

[0106] Step 42, input the second to-be-registered image and the standard image into the first layer of the VGG network model to obtain a first deformation field between the second to-be-registered image and the standard image.

[0107] In this step, the input is a pair of the second to-be-registered image and the standard image, and the second to-be-registered image and the standard image pair first enter the first layer of the VGG network model to obtain a deformation field between the second to-be-registered image and the standard image for the first time, i.e., a first deformation field.

[0108] For example, the second to-be-registered image can be a dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) image.

[0109] Step 43, warp the standard image based on the first deformation field to obtain a coarse registration image.

[0110] In this step, the standard image is warped based on the first deformation field to obtain a coarse registration image.

[0111] Step 44, determine a fine registration image based on the coarse registration image and the second to-be-registered image.

[0112] In this step, after obtaining the coarse registration image, the fine registration image is determined based on the coarse registration image and the second to-be-registered image.

[0113] Optionally, the implementation of this step can include:

[0114] Step 1, input the coarse registration image and the second to-be-registered image into the second layer of the VGG network model to obtain a second deformation field between the coarse registration image and the second to-be-registered image;

[0115] The VGG network model is used to implement fine registration. At this time, the input of the VGG network model is the coarse registration image and the second to-be-registered image, and a deformation field between the coarse registration image and the second to-be-registered image is obtained again, i.e., a second deformation field.

[0116] Step 2, warp the coarse registration image based on the second deformation field to obtain a fine registration image.

[0117] The coarse registration image is warped based on the second deformation field to obtain a fine registration image.

[0118] The image registration method provided by the embodiments of the present disclosure comprises the following steps: obtaining a standard image; inputting a second to-be-registered image and the standard image into a first layer of a VGG network model to obtain a first deformation field between the second to-be-registered image and the standard image; performing a warping deformation on the standard image based on the first deformation field to obtain a coarse registration image; and determining a fine registration image according to the coarse registration image and the second to-be-registered image. In the technical solution, the to-be-registered image is registered twice to obtain the fine registration image, starting from two layers of the VGG network model.

[0119] On the basis of the above embodiments, Figure 5 The flowchart of the image registration method provided by the embodiments of the present disclosure is shown in FIG. 1. Figure 3 As shown in FIG. 2, the image registration method can further comprise the following steps before step 22: Figure 5

[0120] Step 51: For each training image group, input a first work order image and a standard work order image in the training image group into the VGG network model to obtain a corresponding registration image.

[0121] For each training image group, the input of the VGG network model is a pair of first work order image M image and standard work order image F image. After the to-be-registered M image and F image pair first enter the first layer network of the VGG, a deformation field between the M image and the F image is obtained initially, and a coarse registration image is obtained by warping and deforming the F image according to the deformation field.

[0122] Then, the VGG neural network is trained for the second time to achieve fine registration. At this time, the input of the VGG network is the coarse registration image and the M image, and a deformation field between the M image and the coarse registration image is obtained again, and a fine registration image is obtained by deforming the coarse registration image for the second time by using the deformation field.

[0123] Step 52: Evaluate the similarity between the registration image and the standard image to optimize the VGG network model by using the stochastic gradient descent.

[0124] Optionally, Figure 6 The registration framework provided by the embodiments of the present disclosure is shown in FIG. 3. The embodiments of the present disclosure are described in combination with FIG. 3. Figure 6 The embodiments of the present disclosure are described as follows:

[0125] In order to optimize the registration result of the M and F images, the loss function of the mean square error and the gradient descent is used to estimate the difference value of the M and F images, and the difference value is fed back to the convolutional neural network. The loss function is as follows:

[0126] For the image data set, for the data set in this paper, where N∈R represents the number of all samples, the input of the model is a pair of F image and D image, and the pair of F image and D image is represented as ​The registration is represented as The optimization problem is defined as:

[0127] Equation 1:

[0128] Equation 2:

[0129] where φ represents the displacement vector field, L sim represents the similarity measure, L smooth represents the regularization imposed on φ, and λ is the coefficient of the regularization term. The registration is achieved by minimizing the value of Equation 1.

[0130] After the input image passes through the fully convolutional neural network, the fully convolutional neural network generates a deformation field Φ by calculation. According to the generated deformation field, the floating image is deformed by a spatial transformation function to obtain a deformed image M(φ(t)). The formula is as follows:

[0131] Equation 3:

[0132] where t represents a pixel on the image, and φ(t) represents the pixel position corresponding to each t on the deformed image, represents the adjacent pixels of φ(t), and the entire formula is differentiable, so that the error can be backpropagated during the optimization of the parameters. During training, the similarity between the deformed image M and the fixed image F is evaluated, and stochastic gradient descent is used to optimize the objective function. During backpropagation, the value of L is fed back to the neural network, and the loss function value is continuously optimized to be minimized.

[0133] The loss function uses Mean Square Error (MSE), as shown in Equation 4, L smooth is a smooth constraint on the deformation field, usually a linear operation on the spatial gradient of φ, as shown in Equation 5, so the overall loss function is defined as Equation 6:

[0134] Equation 4:

[0135] Equation 5:

[0136] Equation 6:

[0137]

[0138] where λ1 and λ2 are the weight coefficients of coarse registration and fine registration, respectively, and are set to 1.0 and 0.2, respectively.

[0139] is the deformation field obtained in fine registration.

[0140] The image registration method provided by the embodiments of the present disclosure comprises: inputting a first work order image and a standard work order image in a VGG network model to obtain a corresponding registration image, and then evaluating the similarity between the registration image and the standard image to optimize the VGG network model by using a stochastic gradient descent. In the technical solution, the training of the VGG network model and the optimization of the parameters are realized, which provides a guarantee for the accuracy and efficiency of subsequent image registration.

[0141] The following is an embodiment of the image registration device of the present disclosure, which can be used to execute the image registration method embodiment of the present disclosure. For details not disclosed in the image registration device embodiment of the present disclosure, please refer to the image registration method embodiment of the present disclosure.

[0142] Figure 7 The structure diagram of the image registration device provided by the embodiments of the present disclosure is shown in the figure. Figure 7 As shown in the figure, the image registration device comprises:

[0143] The processing module 71 is configured to pre-process the first to-be-registered image to obtain a second to-be-registered image consistent in size with a pre-specified standard image.

[0144] The first input module 72 is configured to input the second to-be-registered image and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image.

[0145] The second input module 73 is configured to input the fine registration image into a spatial transformation network model to obtain a target image after registration.

[0146] The VGG network model is trained based on at least one training image group, and each training image group comprises a first work order image uploaded by a user and a standard work order image.

[0147] In a possible implementation of the embodiments of the present disclosure, the first input module 72 is specifically configured to:

[0148] obtain the standard image;

[0149] input the second to-be-registered image and the standard image into the first layer of the VGG network model to obtain a first deformation field between the second to-be-registered image and the standard image;

[0150] deform the standard image based on the first deformation field to obtain a coarse registration image;

[0151] determine a fine registration image according to the coarse registration image and the second to-be-registered image.

[0152] Optionally, the first input module 72 determines the fine registration image according to the coarse registration image and the second to-be-registered image, and specifically is configured to:

[0153] input the coarse registration image and the second to-be-registered image into the second layer of the VGG network model to obtain a second deformation field between the coarse registration image and the second to-be-registered image;

[0154] deform the coarse registration image based on the second deformation field to obtain the fine registration image.

[0155] In another possible implementation of the embodiments of the present disclosure, before the second to-be-registered image and the standard image are input into the pre-trained VGG network model to obtain the fine registration image, the processing module 71 is further configured to:

[0156] for each training image group, input the first work order image and the standard work order image in the training image group into the VGG network model to obtain a corresponding registration image;

[0157] evaluate the similarity between the registration image and the standard image to optimize the VGG network model by using a stochastic gradient descent.

[0158] In another possible implementation of the embodiments of the present disclosure, the spatial transformation network model is used to guide the VGG network model to perform linear transformation of translation, distortion and rotation.

[0159] In another possible implementation of the embodiments of the present disclosure, before the second to-be-registered image and the standard image are input into the pre-trained VGG network model to obtain the fine registration image, the processing module 71 is further configured to:

[0160] perform image enhancement processing on the second to-be-registered image to obtain an image-enhanced second to-be-registered image.

[0161] The image registration apparatus provided by the embodiments of the present disclosure can be used to execute the image registration method in any of the above embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0162] It should be noted that the division of each module of the above apparatus is only a logical function division, and all or part of the modules can be integrated into one physical entity or physically separated in actual implementation. The modules can all be implemented in the form of software invoked by a processing element, or all be implemented in the form of hardware, or part of the modules are implemented in the form of software invoked by a processing element and part of the modules are implemented in the form of hardware. In addition, all or part of the modules can be integrated together or independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by an integrated logic circuit of hardware in the processing element or an instruction in the form of software.

[0163] Figure 8 A structural schematic diagram of an electronic device provided by the embodiments of the present disclosure is shown in FIG. 8. The electronic device can include a processor 81, a memory 82, and computer program instructions stored in the memory 82 and executable on the processor 81, and the processor 81 implements the image registration method provided by any of the above embodiments when executing the computer program instructions. Figure 8

[0164] Optionally, each device of the electronic device can be connected through a system bus.

[0165] The memory 82 can be a separate storage unit or a storage unit integrated in the processor 81. The number of processors 81 is one or more.

[0166] It should be understood that the processor 81 can be a central processing unit (CPU), and can also be other general-purpose processors 81, digital signal processors 81 (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor 81 can be a microprocessor or can also be any conventional processor 81, etc. The steps of the method disclosed in combination with the present disclosure can be directly embodied as hardware processor 81 execution or executed by a combination of hardware and software modules in the processor 81.

[0167] ​The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The system bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The memory 82 can include a random access memory 82 (RAM) and can also include a non-volatile memory 82 (NVM), such as at least one disk memory 82.

[0168] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a readable memory 82. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory 82 (storage medium) includes a read-only memory 82 (ROM), a RAM, a flash memory 82, a hard disk, a solid state disk, a magnetic tape (English: magnetic tape), a floppy disk (English: floppy disk), an optical disc (English: optical disc), and any combination thereof.

[0169] The electronic device provided by the embodiments of the present disclosure can be used to execute the image registration method provided by any of the above-mentioned method embodiments, and the implementation principles and technical effects are similar, which will not be described here.

[0170] The embodiments of the present disclosure provide a computer readable storage medium, which stores computer instructions, when the computer instructions run on a computer, the computer executes the above-mentioned image registration method.

[0171] The above-mentioned computer readable storage medium, the above-mentioned readable storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0172] Optionally, a readable storage medium is coupled to the processor, such that the processor is enabled to read information from, and write information to, the readable storage medium. Of course, the readable storage medium can be a part of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium can also be located in a device as discrete components.

[0173] The embodiments of the present disclosure further provide a computer program product, which comprises a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the above method.

[0174] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image registration method characterized by, The method comprises: preprocessing a first image to be registered to obtain a second image to be registered consistent in size with a pre-specified standard image, the preprocessing comprising image cropping and size adjustment; performing image enhancement processing on the second image to be registered to obtain an image-enhanced second image to be registered; inputting the image-enhanced second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine registration image, wherein the VGG network model is trained based on at least one training image group, each training image group comprising a first work order image and a standard work order image uploaded by a user, and a random gradient descent is used to optimize the VGG network model during the training process; inputting the fine registration image into a spatial transformation network model to obtain a target image after registration, wherein the spatial transformation network model is used to guide the VGG network model to perform linear transformation of translation, distortion, and rotation.

2. The method of claim 1, wherein, The method comprises: obtaining the standard image; inputting the second image to be registered and the standard image into a first layer of the VGG network model to obtain a first deformation field between the second image to be registered and the standard image; distorting the standard image based on the first deformation field to obtain a coarse registration image; determining the fine registration image according to the coarse registration image and the second image to be registered.

3. The method of claim 2, wherein, The method comprises: inputting the coarse registration image and the second image to be registered into a second layer of the VGG network model to obtain a second deformation field between the coarse registration image and the second image to be registered; distorting the coarse registration image based on the second deformation field to obtain the fine registration image.

4. The method according to any one of claims 1 to 3, characterized in that, Before the step of inputting the second image to be registered and the standard image into the pre-trained VGG network model to obtain the fine registration image, the method further comprises: for each training image group, inputting the first work order image and the standard work order image in the training image group into the VGG network model to obtain a corresponding registration image; evaluating the similarity between the registration image and the standard image to optimize the VGG network model using a random gradient descent.

5. An image registration apparatus characterized by comprising: The method comprises: a processing module configured to preprocess a first image to be registered to obtain a second image to be registered consistent in size with a pre-specified standard image, the preprocessing comprising image cropping and size adjustment; performing image enhancement processing on the second image to be registered to obtain an image-enhanced second image to be registered; The first input module is configured to input the image-enhanced second image to be registered and the standard image into a pre-trained visual geometry group (VGG) network model to obtain a fine-registered image, wherein the VGG network model is trained based on at least one training image group, each training image group including a first work order image uploaded by a user and a standard work order image, and a random gradient descent is used to optimize the VGG network model during the training process. The second input module is configured to input the fine-registered image into a spatial transformation network model to obtain a target image after registration, wherein the spatial transformation network model is used to guide the VGG network model to perform linear transformation of translation, distortion, and rotation.

6. An electronic device, comprising: The image registration method comprises: a processor, and a memory connected to the processor in communication; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the image registration method according to any one of claims 1 to 4.

7. A computer readable storage medium characterized by The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the image registration method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the image registration method according to any one of claims 1 to 4.

Images