An image registration method based on packet motion estimation and neighborhood refinement sampling

Abstract

The application discloses an image registration method based on grouping motion estimation and neighborhood refinement sampling. The method comprises the following steps: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module; obtaining a source image and a target image, and training the image registration network, extracting shared features, grouping motion estimation and neighborhood refinement sampling are sequentially performed until the loss function converges to complete the training; and finally obtaining a registration motion flow by processing the source image and the target image to be registered through the network, and registering the target image to the source image. The method introduces grouping motion estimation to perform sparse modeling on input features, combines an adaptive fusion mechanism to enhance local deformation expression capability, designs a neighborhood refinement sampling method, introduces a learnable local weighting correction mechanism in the motion flow sampling process, effectively improves the boundary definition and detail accuracy of the dense motion field, and significantly improves the accuracy and robustness of image registration.

Description

TECHNICAL FIELD

[0001] The present application relates to an image registration method, relates to the field of digital image processing, and particularly relates to an image registration method based on grouping motion estimation and neighborhood refinement sampling. BACKGROUND

[0002] As a basic task in computer vision and image processing, the core goal of image registration is to align images from different viewpoints, different time points or different sensors to a unified coordinate system, so as to realize information fusion, comparison or further analysis. This technology is widely used in medical image analysis, remote sensing data processing, augmented reality, video stabilization, high dynamic range imaging and other fields. With the wide application of three-dimensional scenes, traditional image registration methods perform poorly in the face of complex deformation, multi-plane structure and non-rigid deformation.

[0003] The image registration method based on deep learning mainly relies on geometric transformation models, such as homography transformation, affine transformation and thin plate spline interpolation models. These methods can provide stable and fast registration results under ideal conditions, but in actual application, there are the following significant problems: 1) plane assumption does not hold: homography transformation is essentially a model for planar scenes, when there are multiple depth layers or large parallax in real scenes, local regions need to be deformed independently, at this time the homography model cannot accurately describe the mapping relationship between images. 2) contradiction between global consistency and local freedom: local transformation methods based on grids (such as multi-grid homography or thin plate spline interpolation) introduce local control points to improve flexibility, but their deformation fields are often subject to global smoothness constraints, adjusting local control points may cause disturbance to the entire image space, affecting the overall registration accuracy. 3) parameter redundancy and low computational efficiency: in order to improve the registration resolution, a high-density control grid is usually used, which leads to a sharp increase in the number of parameters, and then brings problems such as training difficulty, slow reasoning speed, etc., which is difficult to meet the needs of industrial real-time applications. 4) boundary blur and detail loss: in the dense motion flow estimation method, the commonly used bilinear upsampling strategy is computationally efficient, but it will cause the motion field boundary to be blurred, and cannot preserve the fine structures in the image, especially in the edge, corner and other key areas, which is prone to mismatch. SUMMARY

[0004] To solve the problems in the background art, the application provides an image registration method based on grouping motion estimation and neighborhood refinement sampling. The application is particularly suitable for high-precision image alignment tasks in multi-view stereo scenes with complex non-rigid deformation. The method of the application divides the input features into multiple subgroups for sparse modeling through a grouping motion estimation unit, and integrates local motion information through an adaptive fusion mechanism, significantly reducing the computational overhead while enhancing the local modeling capability. In addition, a neighborhood refinement sampling mechanism is introduced, which introduces a learnable local neighborhood weighting mechanism during the upsampling process of the motion flow, combines bilinear interpolation and residual correction, and effectively improves the detail accuracy of the motion field, avoiding the blurring effect caused by upsampling. The image registration method of the application enhances the localized modeling capability, realizes differential modeling of different regions in the image, and improves the adaptability to complex deformation. In addition, a motion estimation module with less parameters and efficient calculation is designed to support deployment on resource-constrained devices, meeting the real-time and power requirements of the industry.

[0005] The technical scheme adopted by the application is:

[0006] The image registration method based on grouping motion estimation and neighborhood refinement sampling of the application comprises:

[0007] Step S1: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module connected in sequence.

[0008] Step S2: obtaining a plurality of pairs of source images and target images of different views thereof and inputting them into the image registration network for training, extracting shared features through the shared feature extraction module, performing grouping motion estimation through the sparse motion module, and finally performing neighborhood refinement sampling based on the dense motion module until the loss function of the image registration network converges, obtaining the trained image registration network; the source images and the target images of different views thereof are all visible light images captured by a camera, and there is an overlap of part of the content in each pair of source images and target images.

[0009] Step S3: inputting the source image to be registered and the target image thereof into the trained image registration network to obtain the dense motion flow of the source image to be registered and the target image thereof, and then obtaining the registration motion flow, registering the target image to the source image based on the registration motion flow, outputting the registered target image after processing, and realizing image registration.

[0010] In step S1, the shared feature extraction module specifically adopts a convolutional neural network MobileNet network with shared weights, and inputs the source image I s and the target image I t with the same resolution into the convolutional neural network MobileNet network for forward propagation, and extracts the source image I sHigh-resolution feature map F s (m) and low-resolution feature map F s (n) and target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) , m < n Source Image I s High-resolution feature map F s (m) and low-resolution feature map F s (n) The image sizes are respectively the source image I s 1 / of the image size m and 1 / n Target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) The image sizes are respectively the target image I t 1 / of the image size m and 1 / n .

[0011] In step S1, the source image I is processed by the sparse motion module. s low-resolution feature map F s (n) and target image I t low-resolution feature map F t (n) The source image I is processed together as two input feature maps. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) After undergoing deformation transformation, the two input feature maps are processed together by the sparse motion module. The source image I after deformation transformation s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) as follows:

[0012] F T (m) =warp(F T (m)D1'', T = s or t

[0013] wherein F s (m) and F t (m) are high resolution feature maps of the source image I s and the target image I t after deformation transformation respectively; warp() is a deformation transformation operation; D1'' is a final dense motion flow obtained by high resolution feature maps F s of the source image I s (m) and high resolution feature maps F t of the target image I t (m) .

[0014] In the sparse motion module, firstly, a similarity matrix S g between two feature maps is constructed, then motion latent features F c are extracted by a motion latent feature extractor, F c ∈R Cc , and then input into a grouped motion estimation unit for processing to obtain a sparse motion flow.

[0015] The grouped motion estimation unit specifically comprises the following steps: firstly, the motion latent features F c are evenly divided into N g groups of sub-feature groups, each group having C g sub-features, C g =C c / N g , C c being the total number of sub-features of the motion latent features F c ; then, for each group of sub-feature groups, linear transformation processing is performed on each sub-feature in the sub-feature group, and then the processed groups of sub-feature groups are spliced and sequentially subjected to activation function ReLU and linear layer processing to obtain a sparse motion flow.

[0016] In the step S1, the dense motion module firstly processes the similarity matrix S g between two feature maps through a neighborhood sampling mask generator to obtain a neighborhood sampling mask M, and then based on the sparse motion flow and the neighborhood sampling mask M, processes through a neighborhood refinement sampling unit to obtain a final dense motion flow.

[0017] The motion hidden feature extractor comprises N sequentially connected basic convolution blocks, the neighborhood sampling mask generator comprises P sequentially connected basic convolution blocks, and the basic convolution block comprises sequentially connected convolution layers and an activation function RELU.

[0018] The neighborhood refinement sampling unit first performs bilinear upsampling processing on the sparse motion flow to obtain an initial dense motion flow D', and performs softmax normalization on the neighborhood sampling mask M to obtain a normalized neighborhood sampling mask M'; then, the initial dense motion flow D' and the normalized neighborhood sampling mask M' are subjected to neighborhood weighted correction processing to obtain a final dense motion flow D'', and the specific process is as follows:

[0019] D''= D'+∑ k=1 9  (M' k ·D')

[0020] Wherein, M' k is the i-th neighborhood position mask value in the normalized neighborhood sampling mask M'. k

[0021] In step S2, the loss function adopts an image content loss L align , and the specific process is as follows:

[0022] L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+

[0023] λ0|| I t ·warp(O, D1'') - warp(I s , (D1'') T ) ||1+

[0024] λ1|| I s ·warp(O, (D F ) T ) - warp(I t , D F ) ||1+

[0025] λ1|| I t ·warp(O, D F ) - warp(I s , (D F ) T ) ||1

[0026] ​Wherein, λ0 and λ1 are initial and final registration error weights respectively; || ||1 is 1 norm; I s And I t Respectively, the source image and the target image; warp() is a deformation transformation operation; O is a matrix with all elements being 1; D1'' is the final dense motion flow obtained by high-resolution feature map F s Of the source image I s (m) And high-resolution feature map F t Of the target image I t (m) Obtained; T is transposition; D F Is the registration motion flow, D F =D1''+D2'', D2'' is the final dense motion flow obtained by low-resolution feature map F s Of the source image I s (n) Low-resolution feature map F t Of the target image I t (n) The registration motion flow overcomes the blurring effect problem caused by traditional bilinear interpolation, and effectively captures more local motion details.

[0027] Based on image content loss L align , when the image registration network is trained, the image registration network is optimized using gradient back propagation, and the trained image registration network is obtained.

[0028] The electronic device of the present application comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method as described above.

[0029] The computer readable storage medium of the present application has program data stored thereon, and the program data is executed by the processor to implement the method as described above.

[0030] The present application has the following advantages:

[0031] 1. Efficient parameter modeling, reducing calculation redundancy: In view of the problems of parameter redundancy and low calculation efficiency caused by high-density control grid, the present application adopts grouping motion estimation strategy, which significantly reduces the model parameter amount while maintaining local degrees of freedom; The method improves the inference speed without sacrificing accuracy, and is particularly suitable for deployment on resource-limited devices, meeting the needs of industrial real-time applications.

[0032] 2. Neighborhood refinement sampling mechanism, preserving the details of the motion field: To solve the problem of boundary blur and detail loss caused by traditional bilinear up-sampling, the application introduces a learnable neighborhood refinement sampling mechanism, combining bilinear interpolation and residual correction, dynamically adjusting the neighborhood weight distribution during the generation of dense motion flow, effectively preserving the key edge and corner information in the image, significantly improving the visual quality and geometric accuracy of the registration result.

[0033] The method of the application sparsely models the input features by introducing grouped motion estimation, and enhances the local deformation expression ability by combining an adaptive fusion mechanism, while designing a neighborhood refinement sampling method to introduce a learnable local weighted correction mechanism in the motion flow sampling process, effectively improving the boundary clarity and detail accuracy of the dense motion field. The application can effectively cope with challenges such as complex non-rigid deformation, large disparity and low overlap rate in multi-view stereo scenes, significantly improving the accuracy and robustness of image registration, and has the advantages of high computational efficiency, strong detail preservation ability, etc., and is suitable for various practical applications such as image stitching and video stabilization. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a flowchart of the method of the application;

[0035] Figure 2 is a flowchart of the grouped motion estimation of the application;

[0036] Figure 3 is an example one of the registration results of the method of the application;

[0037] Figure 4 is an example two of the registration results of the method of the application;

[0038] Figure 5 is an example three of the registration results of the method of the application. DETAILED DESCRIPTION

[0039] The application will be further described in detail below in combination with the drawings and specific embodiments.

[0040] As shown in the drawings, Figure 1 The image registration method of the application based on grouped motion estimation and neighborhood refinement sampling is as follows:

[0041] The application first performs data preparation and preprocessing, collects multi-view image data sets for image registration tasks, covering real stereo scene images with large disparity or non-rigid deformation. All images are uniformly adjusted to a fixed resolution and normalized, and the input images are uniformly set to 512 size, i.e. H=W=512.

[0042] Then, an image registration network is constructed, comprising a shared feature extraction module, a sparse motion module, and a dense motion module connected sequentially. The shared feature extraction module specifically employs a MobileNet convolutional neural network with shared weights, registering source images of the same resolution. s and target image I t The input is processed through a convolutional neural MobileNet network for forward propagation, and the source image I is extracted respectively. s High-resolution feature map F s (m) and low-resolution feature map F s (n) and target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) , m < n Source Image I s High-resolution feature map F s (m) and low-resolution feature map F s (n) The image sizes are respectively the source image I s 1 / of the image size m and 1 / n Target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) The image sizes are respectively the target image I t 1 / of the image size m and 1 / n In specific implementation, m =4, n =8, meaning the downsampling factors are 4 and 8 respectively; I s I t ∈R H×W×3 H and W are the length and width, respectively; F s (4) ∈R (H / 4)×(W / 4)×C4 F s (8) ∈R (H / 8)×(W / 8)×C8 F t (4) ∈R (H / 4)×(W / 4)×C4 F8 (8) ∈R (H / 8)×(W / 8)×C8 C4 and C8 represent the number of channels at the corresponding level, respectively.

[0043] The network model constructed by the application uses MobileNet as the backbone network, and outputs two scale feature maps: a high resolution of 128x128 and a channel number C4=64; and a low resolution of 64x64 and a channel number C8=96. All convolutional layers use a ReLU activation function and batch normalization BN (BatchNorm) to accelerate convergence. The local matching window size K=5, that is, a 5x5 neighborhood around each reference point is taken for feature matching. The motion hidden feature extractor is composed of N=4 basic convolutional blocks, and the output channel number is 128. The feature is divided into N g =4 groups. The neighborhood sampling mask generator contains P=3 basic convolutional blocks.

[0044] In the network model, the multi-scale feature maps of the source image and the target image are extracted through the shared feature extraction module, thereby providing a basis for subsequent deformation solving. The low resolution feature maps F s of the source image I s (n) and the low resolution feature maps F t of the target image I t (n) are processed together as two input feature maps. The high resolution feature maps F s of the source image I s (m) and the high resolution feature maps F t of the target image I t (m) are subjected to deformation transformation and then processed together as two input feature maps through the sparse motion module. The high resolution feature maps F s of the source image I s (m) and the high resolution feature maps F t of the target image I t (m) after deformation transformation are as follows:

[0045] F T (m) '=warp(F T (m) ,D1''), T = s or t

[0046] wherein F s (m) ' and F t (m) ' are the high resolution feature maps of the source image I s and the target image I t after deformation transformation; warp() is a deformation transformation operation; D1'' is a deformation transformation operation on the low resolution feature maps F s of the source image I s (n) and the low resolution feature maps F t of the target image I t (n) .s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The final dense motion flow obtained.

[0047] In the sparse motion module, the similarity matrix S between the two feature maps is first constructed. g Then, the motion latent features F are extracted using a motion latent feature extractor. c F c ∈R Cc Then, the data is input into the grouped motion estimation unit for processing to obtain the sparse motion flow, D. a ∈R (H / 8)×(W / 8)×2 a=1 or 2, D2 is the result of passing through source image I s low-resolution feature map F s (n) and the low-resolution feature map F of the target image t (n) The obtained sparse motion flow.

[0048] In practical implementation, source image I s low-resolution feature map F s (8) and target image I t low-resolution feature map F t (8) Similarity matrix S g Specifically as follows:

[0049] S g (x s ,y s ,x t ,y t )=

[0050] ∑ (K / 2) i,j=-(K / 2) ( <F s,(xs+i,ys+j) (8) ,F t,(xt+i,yt+j) (8) > / || F s,(xs+i,ys+j) (8) ||·||F t,(xt+i,yt+j)) (8) ||)

[0051] S g ∈R H / 8×W / 8×H / 8×W / 8

[0052] Among them, (x s ,y s ) and (x t ,y t) are the source images I s Feature map and target image I t Feature map F t (8) The coordinates of the points on the graph; K represents the preset local window size, <> is the dot product, and || is the modulus.

[0053] Source image I after deformation transformation s High-resolution feature map F s (4) 'and target image I t High-resolution feature map F t (4) Similarity matrix S g The details are as follows:

[0054] S g '(x s ,y s ,x t ,y t )=

[0055] ∑ (K / 2) i,j=-(K / 2) ( <F s,(xs+i,ys+j) (4) ',F t,(xt+i,yt+j) (4) > / || F s,(xs+i,ys+j) (4) ' ||·||F t,(xt+i,yt+j)) (4) ||)

[0056] S g ∈R H / 8×W / 8×H / 8×W / 8

[0057] The similarity matrix reflects the similarity relationship between each local region in the source image and all possible matching regions in the target image, providing a basis for subsequent motion field prediction.

[0058] like Figure 2 As shown, the grouped motion estimation unit specifically involves first processing the motion latent features F... c Divide evenly into N g Group sub-feature groups, each group contains C g Individual characteristics, C g =C c / N g C c For motion latent features F cThe total number of sub-features is calculated; then, for each sub-feature group, a linear transformation is performed on each sub-feature within the sub-feature group. The processed sub-feature groups are then concatenated and sequentially processed through the ReLU activation function and a linear layer to obtain the sparse motion flow. Each sub-feature group is processed independently, with independent linear transformations performed, as follows:

[0059] F g,k '=W k F g,k +B k

[0060] Among them, F g,k ' is the th after linear transformation k Sub-features in a group of sub-features, F g,k For the first k Sub-features within a group of sub-features; W k and B k The first k The first and second learnable parameters of the sub-feature group, W k ∈R Cg'×Cg B k ∈R Cg' C g 'W represents the sub-feature dimension after linear transformation; k and B k Iterative optimization is performed during network training.

[0061] In a specific implementation, the inference time of grouped motion estimation was compared, as shown in Table 1 below:

[0062] Table 1

[0063]

[0064] As can be seen, the group motion estimation method proposed in this invention effectively reduces the inference time of the model while still maintaining considerable registration accuracy, proving the effectiveness of the proposed method in reducing computational overhead.

[0065] The dense motion module first compares the similarity matrix S between two feature maps. g The neighborhood sampling mask M, M∈R, is obtained after processing by the neighborhood sampling mask generator. 9×8×8×H / 8×W / 8 Then, based on the sparse motion flow and the neighborhood sampling mask M, the final dense motion flow is obtained by processing through neighborhood refinement sampling units.

[0066] The motion latent feature extractor consists of N sequentially connected basic convolutional blocks, and the neighborhood sampling mask generator consists of P sequentially connected basic convolutional blocks. Each basic convolutional block consists of a sequentially connected 3×3 convolutional layer and the activation function ReLU.

[0067] The neighborhood refinement sampling unit first performs bilinear upsampling processing on the sparse motion stream to obtain an initial dense motion stream D', and performs softmax normalization on the neighborhood sampling mask M to obtain a normalized neighborhood sampling mask M'; then, the initial dense motion stream D' and the normalized neighborhood sampling mask M' are subjected to neighborhood weighted correction processing to obtain a final dense motion stream D'', which is as follows:

[0068] D''= D'+∑ k=1 9  (M' k ·D')

[0069] Wherein, M' k is the i-th neighborhood position mask value in the normalized neighborhood sampling mask M'. k

[0070] The neighborhood weighted correction combines the local neighborhood pixels and the mask weight to calculate the residual high-resolution motion stream, realizes the refinement of the motion, and thus reduces the blurring effect caused by the bilinear interpolation.

[0071] The present application carries out neighborhood refinement sampling registration accuracy comparison in specific implementation, as shown in the following table 2:

[0072] Table 2

[0073]

[0074] It can be seen that the neighborhood refinement sampling proposed in the present application improves the upper limit of the local deformation transformation of the model, thereby improving the registration accuracy of the model, and the registration accuracy is significantly improved.

[0075] After the model construction is completed, the processed multi-view image data set is input into the image registration network for training, which contains a plurality of pairs of source images and target images of different perspectives thereof, shared features are extracted through a shared feature extraction module, grouping motion estimation is performed through a sparse motion module, and finally neighborhood refinement sampling is performed based on a dense motion module until the loss function of the image registration network converges, and a trained image registration network is obtained; the source images and the target images of different perspectives thereof are all visible light images photographed by a camera, and there is a partial content overlap in each pair of source images and target images.

[0076] The loss function adopted by the present application is an image content loss L align , for network parameter optimization, which is as follows:

[0077] L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+​

[0078] λ0|| I t ·warp(O, D1'') - warp(I s , (D1'') T ) ||1+

[0079] λ1|| I s ·warp(O, (D F ) T ) - warp(I t , D F ) ||1+

[0080] λ1|| I t ·warp(O, D F ) - warp(I s , (D F ) T ) ||1

[0081] Wherein, λ0 and λ1 are initial and final registration error weights, in specific implementation, λ0=1, λ1=2; || ||1 is 1 norm; I s and I t are source image and target image; warp( ) is deformation transformation operation; O is a matrix whose elements are all 1; D1'' is the final dense motion flow obtained by high resolution feature map F s of source image I s (m) and high resolution feature map F t of target image I t (m) ; T is transpose; D F is registration motion flow, D F =D1''+D2'', D2'' is the final dense motion flow obtained by low resolution feature map F s of source image I s (n) and low resolution feature map F t of target image I t (n) ; Registration motion flow overcomes the problem of blur effect caused by traditional bilinear interpolation, and effectively captures more local motion details.

[0082] Based on image content loss L align , when the image registration network is trained, the image registration network is optimized using gradient back propagation, and the optimizer uses Adam, the initial learning rate lr=1e -4The learning rate is exponentially attenuated every 10 batch epochs, with an attenuation rate of 0.95. The batch size is 32, and 50 batch epochs are trained. The network is initialized using the Kaiming initialization. Gradient clipping (gradient clipping norm clipnorm=1.0) is used to prevent gradient explosion. The model is trained using a single A100 card. Finally, the trained image registration network is obtained.

[0083] Finally, the source image to be registered and its target image are input into the trained image registration network to obtain the dense motion flow of the source image to be registered and its target image, and then the registration motion flow is obtained, the target image is registered to the source image based on the registration motion flow, and the registered target image is output after processing, thereby realizing image registration. In specific implementation, the optimal model weight is loaded for testing, a pair of unseen source image and target image are input, the network is forward inferred, and the final registration motion flow D F The image is morphed to generate the registered target image. The method can be widely applied to image stitching and video stabilization scenes. Figure 3 、 Figure 4 and Figure 5 As shown in the figures, the source image, the target image and the target image registered based on the method are respectively a source image, a target image and a target image registered based on the method, which are captured by a camera at a certain position of a region. It can be seen that the registration effect of the method is good.

[0084] The application provides a feasible and advanced image registration method, which has good engineering implementation value and wide practical prospect. For those skilled in the image processing field, the above embodiments are only exemplary and should not be construed as limiting the scope of protection of the application. Any equivalent replacement or improvement based on the idea of the application falls within the scope of protection of the application.

Claims

1. An image registration method based on grouped motion estimation and neighborhood refinement sampling, characterized in that, include: Step S1: Construct an image registration network comprising a shared feature extraction module, a sparse motion module, and a dense motion module connected in sequence; Step S2: Obtain several pairs of source images and target images from different perspectives and input them into the image registration network for training. Extract shared features through the shared feature extraction module, perform group motion estimation through the sparse motion module, and finally perform neighborhood refinement sampling based on the dense motion module until the loss function of the image registration network converges, thus obtaining the trained image registration network. Step S3: Input the source image to be registered and its target image into the trained image registration network to obtain the dense motion flow of the source image to be registered and its target image, and then obtain the registration motion flow. Based on the registration motion flow, register the target image to the source image, process and output the registered target image to achieve image registration. In step S1, the source image I is processed by the sparse motion module. s low-resolution feature map F s (n) and target image I t low-resolution feature map F t (n) The source image I is processed together as two input feature maps. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) After undergoing deformation transformation, the two input feature maps are processed together by the sparse motion module. The source image I after deformation transformation s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) as follows: F T (m) =warp(F T (m) ,D1''), T = s or t Among them, F s (m) 'and F t (m) 'These are the source images I after deformation transformation.' s High-resolution feature maps and target images I t High-resolution feature map; warp() is the deformation transformation operation; D1'' is the high-resolution feature map obtained from the source image I. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The final dense flow of motion obtained; In the sparse motion module, the similarity matrix S between the two feature maps is first constructed. g Then, the motion latent features F are extracted using a motion latent feature extractor. c Then, the data is input into the grouped motion estimation unit for processing to obtain the sparse motion flow. The grouped motion estimation unit specifically involves first processing the motion latent features F... c Divide evenly into N g Group sub-feature groups, each group contains C g Individual characteristics, C g =C c / N g C c For motion latent features F c The total number of sub-features; then for each sub-feature group, perform linear transformation on each sub-feature in the sub-feature group, then concatenate the processed sub-feature groups and then process them sequentially through the activation function ReLU and a linear layer to obtain the sparse motion flow; In step S1, the dense motion module first sets the similarity matrix S between the two feature maps. g The neighborhood sampling mask M is obtained after processing by the neighborhood sampling mask generator. Then, based on the sparse motion flow and the neighborhood sampling mask M, the final dense motion flow is obtained by processing through the neighborhood refinement sampling unit.

2. The image registration method based on group motion estimation and neighborhood refinement sampling according to claim 1, characterized in that: In step S1, the shared feature extraction module specifically employs a MobileNet convolutional neural network with shared weights to extract source images I of the same resolution. s and target image I t The input is processed through a convolutional neural MobileNet network for forward propagation, and the source image I is extracted respectively. s High-resolution feature map F s (m) and low-resolution feature map F s (n) and target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) , m < n Source Image I s High-resolution feature map F s (m) and low-resolution feature map F s (n) The image sizes are respectively the source image I s 1 / of the image size m and 1 / n Target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) The image sizes are respectively the target image I t 1 / of the image size m and 1 / n .

3. The image registration method based on group motion estimation and neighborhood refinement sampling according to claim 1, characterized in that: The motion latent feature extractor includes N sequentially connected basic convolutional blocks, and the neighborhood sampling mask generator includes P sequentially connected basic convolutional blocks. Each basic convolutional block includes a sequentially connected convolutional layer and an activation function ReLU.

4. The image registration method based on group motion estimation and neighborhood refinement sampling according to claim 1, characterized in that: The neighborhood refinement sampling unit first performs bilinear upsampling on the sparse motion flow to obtain an initial dense motion flow D', and then performs softmax normalization on the neighborhood sampling mask M to obtain a normalized neighborhood sampling mask M'. Then, neighborhood weighted correction is performed on the initial dense motion flow D' and the normalized neighborhood sampling mask M' to obtain the final dense motion flow D'', as detailed below: D''=D'+∑ k=1 9  (M' k ·D') Among them, M' k The first normalized neighborhood sampling mask M' k Each neighborhood location mask value.

5. The image registration method based on group motion estimation and neighborhood refinement sampling according to claim 1, characterized in that: In step S2, the loss function is image content loss L. align The details are as follows: L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+ λ0|| I t ·warp(O, D1'') - warp(I s , (D1'') T ) ||1+ λ1|| I s ·warp(O, (D F ) T ) - warp(I t , D F ) ||1+ λ1|| I t ·warp(O, D F ) - warp(I s , (D F ) T ) ||1 Where λ0 and λ1 are the initial and final registration error weights, respectively; |||1 is the 1-norm; I s and I t These are the source image and the target image, respectively; `warp()` is the warp transformation operation; `O` is a matrix with all elements equal to 1; `D1''` is the image obtained by passing through the source image I... s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The resulting dense motion flow; T is the transpose; D F To register the moving flow, D F =D1''+D2'', where D2'' is the value obtained from the source image I. s low-resolution feature map F s (n) Target image I t low-resolution feature map F t (n) The final dense flow of motion obtained; Based on image content loss L align During the training of the image registration network, gradient backpropagation is used to optimize the image registration network, resulting in a trained image registration network.

6. An electronic device, characterized in that, include: A memory and a processor are coupled to each other, wherein the memory stores program data, and the processor invokes the program data to perform the method as described in any one of claims 1-5.

7. A computer-readable storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, the method as described in any one of claims 1-5 is implemented.

Images