Image registration method based on grouped motion estimation and neighborhood refinement sampling
By using grouped motion estimation and neighborhood refinement sampling, the problem of low accuracy and low efficiency of traditional image registration methods in complex deformation and non-rigid deformation is solved, achieving efficient and accurate image registration, which is suitable for multi-view stereo scenes and resource-constrained devices.
Patent Information
- Application Number
- CN202511631595.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-10
- Publication Date
- 2025-12-09
- Estimated Expiration
- 2045-11-10
AI Technical Summary
Traditional image registration methods are not accurate and computationally efficient when dealing with complex deformations, multi-planar structures and non-rigid deformations. Furthermore, bilinear upsampling leads to blurred boundaries and loss of details.
An image registration method using grouped motion estimation and neighborhood refinement sampling is adopted. Sparse modeling is performed through grouped motion estimation units and combined with an adaptive fusion mechanism. A learnable local neighborhood weighting mechanism is introduced to improve local modeling capability and computational efficiency. A motion estimation module with a small number of parameters is designed.
It significantly improves the accuracy and robustness of image registration, adapts to complex deformations, meets the needs of industrial-grade real-time applications, preserves key edge and corner information, and enhances visual quality and geometric accuracy.
Smart Images

Figure CN121095301A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to an image registration method, relates to the field of digital image processing, and particularly relates to an image registration method based on grouping motion estimation and neighborhood refinement sampling. BACKGROUND
[0002] As a basic task in computer vision and image processing, the core goal of image registration is to align images from different viewpoints, different time points or different sensors to a unified coordinate system, so as to realize information fusion, comparison or further analysis. This technology is widely used in medical image analysis, remote sensing data processing, augmented reality, video stabilization, high dynamic range imaging and other fields. With the wide application of three-dimensional scenes, traditional image registration methods perform poorly in the face of complex deformation, multi-plane structure and non-rigid deformation.
[0003] The image registration method based on deep learning mainly relies on geometric transformation models, such as homography transformation, affine transformation and thin plate spline interpolation models. These methods can provide stable and fast registration results under ideal conditions, but in actual application, there are the following significant problems: 1) plane assumption does not hold: homography transformation is essentially a model for planar scenes, when there are multiple depth layers or large parallax in real scenes, local regions need to be deformed independently, at this time the homography model cannot accurately describe the mapping relationship between images. 2) contradiction between global consistency and local freedom: local transformation methods based on grids (such as multi-grid homography or thin plate spline interpolation) introduce local control points to improve flexibility, but their deformation fields are often subject to global smoothness constraints, adjusting local control points may cause disturbance to the entire image space, affecting the overall registration accuracy. 3) parameter redundancy and low computational efficiency: in order to improve the registration resolution, a high-density control grid is usually used, which leads to a sharp increase in the number of parameters, and then brings problems such as training difficulty, slow reasoning speed, etc., which is difficult to meet the needs of industrial real-time applications. 4) boundary blur and detail loss: in the dense motion flow estimation method, the commonly used bilinear upsampling strategy is computationally efficient, but it will cause the motion field boundary to be blurred, and cannot preserve the fine structures in the image, especially in the edge, corner and other key areas, which is prone to mismatch. SUMMARY
[0004] To solve the problems in the background art, the application provides an image registration method based on grouping motion estimation and neighborhood refinement sampling. The application is particularly suitable for high-precision image alignment tasks in multi-view stereo scenes with complex non-rigid deformation. The application method divides the input features into multiple subgroups for sparse modeling through a grouping motion estimation unit, and integrates local motion information through an adaptive fusion mechanism, significantly reducing the computational overhead while enhancing the local modeling capability. In addition, a neighborhood refinement sampling mechanism is introduced, which introduces a learnable local neighborhood weighting mechanism during the motion flow upsampling process, combining bilinear interpolation and residual correction to effectively improve the detail accuracy of the motion field and avoid the blurring effect caused by upsampling. The image registration method of the application enhances the local modeling capability, realizes differential modeling of different regions in the image, and improves the adaptability to complex deformation. In addition, a motion estimation module with less parameters and efficient calculation is designed to support deployment on resource-constrained devices, meeting the real-time and power requirements of the industry.
[0005] The technical scheme adopted by the application is: The image registration method based on grouping motion estimation and neighborhood refinement sampling of the application comprises: Step S1: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module connected in turn.
[0006] Step S2: obtaining a plurality of pairs of source images and target images of different perspectives thereof and inputting them into the image registration network for training, extracting shared features through the shared feature extraction module, performing grouping motion estimation through the sparse motion module, and finally performing neighborhood refinement sampling based on the dense motion module until the loss function of the image registration network converges, obtaining the trained image registration network; the source images and the target images of different perspectives thereof are all visible light images captured by a camera, and there is a partial overlap of content in each pair of source images and target images.
[0007] Step S3: inputting the source image to be registered and the target image thereof into the trained image registration network to obtain the dense motion flow of the source image to be registered and the target image thereof, and then obtaining the registration motion flow, registering the target image to the source image based on the registration motion flow, outputting the registered target image after processing, and realizing image registration.
[0008] In step S1, the shared feature extraction module specifically adopts a convolutional neural MobileNet network with shared weights, and inputs the source image I s and the target image I t with the same resolution into the convolutional neural MobileNet network for forward propagation, respectively extracting the high-resolution feature map F s of the source image I s (m)and low-resolution feature map F s (n) and target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) , m < n Source Image I s High-resolution feature map F s (m) and low-resolution feature map F s (n) The image sizes are respectively the source image I s 1 / of the image size m and 1 / n Target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) The image sizes are respectively the target image I t 1 / of the image size m and 1 / n .
[0009] In step S1, the source image I is processed by the sparse motion module. s low-resolution feature map F s (n) and target image I t low-resolution feature map F t (n) The source image I is processed together as two input feature maps. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) After undergoing deformation transformation, the two input feature maps are processed together by the sparse motion module. The source image I after deformation transformation s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) as follows: F T (m) =warp(F T (m) ,D1'' ), T = s or t wherein F s (m) and F t (m) are high resolution feature maps of the source image I s and the target image I t respectively after deformation transformation; warp() is a deformation transformation operation; D1'' is a final dense motion flow obtained by high resolution feature maps F s of the source image I s (m) and high resolution feature maps F t of the target image I t (m) . In the sparse motion module, a similarity matrix S g between the two feature maps is first constructed, then motion latent features F c are extracted by a motion latent feature extractor, F c ∈R Cc , and then input into a grouped motion estimation unit for processing to obtain a sparse motion flow.
[0010] The grouped motion estimation unit specifically uniformly divides the motion latent features F c into N g groups of sub-feature groups, each group having C g sub-features, C g =C c / N g , C c being the total number of sub-features of the motion latent features F c ; then for each group of sub-feature groups, linear transformation processing is performed on each sub-feature in the sub-feature group, and then the processed groups of sub-feature groups are spliced and sequentially processed through an activation function ReLU and a linear layer to obtain a sparse motion flow.
[0011] In step S1, the dense motion module first processes the similarity matrix S g between the two feature maps through a neighborhood sampling mask generator to obtain a neighborhood sampling mask M, and then based on the sparse motion flow and the neighborhood sampling mask M, processes through a neighborhood refinement sampling unit to obtain a final dense motion flow.
[0012] The motion latent feature extractor includes N basic convolutional blocks connected in sequence, and the neighborhood sampling mask generator includes P basic convolutional blocks connected in sequence, and the basic convolutional block includes a convolutional layer and an activation function ReLU connected in sequence.
[0013] The neighborhood refinement sampling unit first performs bilinear up-sampling on the sparse motion flow to obtain an initial dense motion flow D', and performs softmax normalization on the neighborhood sampling mask M to obtain a normalized neighborhood sampling mask M'; then performs neighborhood weighted correction processing on the initial dense motion flow D' and the normalized neighborhood sampling mask M' to obtain a final dense motion flow D'', specifically as follows: D''= D'+∑ k=1 9 (M' k ·D') Wherein, M' k is the i-th neighborhood position mask value in the normalized neighborhood sampling mask M'. k
[0014] In step S2, the loss function adopts an image content loss L align , specifically as follows: L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+ λ0|| I t ·warp(O, D1'') - warp(I s , (D1'') T ) ||1+ λ1|| I s ·warp(O, (D F ) T ) - warp(I t , D F ) ||1+ λ1|| I t ·warp(O, D F ) - warp(I s , (D F ) T ) ||1 Wherein, λ0 and λ1 are initial and final registration error weights respectively; || ||1 is 1 norm; I s and I t are source and target images respectively; warp() is a warping operation; O is a matrix with all elements being 1; D1'' is a dense motion flow obtained by warping the high-resolution feature map F s of the source image I s (m) and the high-resolution feature map F t of the target image I t (m) The final dense motion field obtained; T is transposition; D F For registration motion field, D F =D1''+D2'', D2'' is the low-resolution feature map F s of the source image I s (n) The low-resolution feature map F t of the target image I t (n) The final dense motion field obtained; the registration motion field overcomes the problem of blurring effect caused by traditional bilinear interpolation, and effectively captures more local motion details.
[0015] Based on the image content loss L align When the image registration network is trained, the image registration network is optimized using gradient back propagation, and a trained image registration network is obtained.
[0016] The electronic device of the present application comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method as described above.
[0017] The computer readable storage medium of the present application has program data stored thereon, and the program data is executed by the processor to implement the method as described above.
[0018] The present application has the following advantages: 1. Efficient parameter modeling, reducing computational redundancy: In order to solve the problem of parameter redundancy and low computational efficiency caused by high-density control grid, the present application adopts a grouping motion estimation strategy, which significantly reduces the number of model parameters while maintaining local degrees of freedom; The method improves the inference speed without sacrificing accuracy, and is particularly suitable for deployment on resource-constrained devices, meeting the needs of industrial real-time applications.
[0019] 2. Neighborhood refinement sampling mechanism, preserving motion field details: In order to solve the problem of boundary blurring and detail loss caused by traditional bilinear upsampling, the present application introduces a learnable neighborhood refinement sampling mechanism, which combines bilinear interpolation and residual correction to dynamically adjust the neighborhood weight distribution during the generation of dense motion field, effectively preserving the key edge and corner information in the image, and significantly improving the visual quality and geometric accuracy of the registration result.
[0020] The method of the present application can effectively deal with the challenges of complex non-rigid deformation, large disparity and low overlap rate in multi-view stereo scenes, significantly improve the accuracy and robustness of image registration, and has the advantages of high computational efficiency, strong detail retention capability, etc., and is suitable for image stitching, video stabilization and other practical applications. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 is a flowchart of the method of the present application; Figure 2 is a flowchart of the grouping motion estimation of the present application; Figure 3 is an example one of the registration results of the method of the present application; Figure 4 is an example two of the registration results of the method of the present application; Figure 5 is an example three of the registration results of the method of the present application. DETAILED DESCRIPTION
[0022] The present application will be further described in detail below in combination with the drawings and specific embodiments.
[0023] As shown in the drawings, Figure 1 the image registration method of the present application based on grouping motion estimation and neighborhood refinement sampling is as follows: The present application first performs data preparation and preprocessing, collects multi-view image data sets for image registration tasks, covering real stereo scene images with large disparity or non-rigid deformation. All images are uniformly adjusted to a fixed resolution and normalized, and the input images are uniformly to 512 size, i.e. H=W=512.
[0024] Then the image registration network including the sequentially connected shared feature extraction module, sparse motion module and dense motion module is constructed. The shared feature extraction module specifically adopts the convolutional neural network MobileNet with shared weights, and the source image I s and the target image I t with the same resolution are input into the convolutional neural network MobileNet for forward propagation, and the high-resolution feature map F s and the low-resolution feature map F s (m) of the source image I s (n) and the high-resolution feature map F t of the target image I t(m) and low resolution feature maps F t (n) , m < n , source image I s high resolution feature maps F s (m) and low resolution feature maps F s (n) The image sizes of the high resolution feature maps F s m and low resolution feature maps F n of the target image I t t (m) and low resolution feature maps F t (n) The image sizes of the high resolution feature maps F t m and low resolution feature maps F n ; in particular implementation, m = 4, n = 8, that is, the downsampling multiples are 4 and 8 respectively; I s , I t ∈ R H×W×3 , H and W are the length and width respectively; F s (4) ∈ R (H / 4)×(W / 4)×C4 , F s (8) ∈ R (H / 8)×(W / 8)×C8 , F t (4) ∈ R (H / 4)×(W / 4)×C4 , F8 (8) ∈ R (H / 8)×(W / 8)×C8 , C4 and C8 represent the number of channels of the corresponding levels respectively.
[0025] The network model constructed by the application uses MobileNet as the backbone network, and outputs two scale feature maps: the high resolution is 128x128, and the number of channels C4 = 64; the low resolution is 64x64, and the number of channels C8 = 96. All convolutional layers use ReLU activation function and batch normalization BN (BatchNorm) to accelerate convergence. The size of the local matching window K = 5, that is, a 5x5 neighborhood around each reference point is taken for feature matching. The motion hidden feature extractor is composed of N = 4 basic convolutional blocks, and the output channel number is 128. The features are divided into N g = 4 groups. The neighborhood sampling mask generator contains P = 3 basic convolutional blocks.
[0026] In the network model, the multi-scale feature maps of the source image and the target image are extracted through the shared feature extraction module, which provides a basis for subsequent deformation solving. Through the sparse motion module, the multi-scale feature maps of the source image Is low-resolution feature map F s (n) and target image I t low-resolution feature map F t (n) The source image I is processed together as two input feature maps. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) After undergoing deformation transformation, the two input feature maps are processed together by the sparse motion module. The source image I after deformation transformation s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) as follows: F T (m) =warp(F T (m) ,D1''), T = s or t Among them, F s (m) 'and F t (m) 'These are the source images I after deformation transformation.' s High-resolution feature maps and target images I t High-resolution feature map; warp() is the deformation transformation operation; D1'' is the high-resolution feature map obtained from the source image I. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The final dense motion flow obtained.
[0027] In the sparse motion module, the similarity matrix S between the two feature maps is first constructed. g Then, the motion latent features F are extracted using a motion latent feature extractor. c F c ∈R Cc Then, the data is input into the grouped motion estimation unit for processing to obtain the sparse motion flow, D. a ∈R (H / 8)×(W / 8)×2 a=1 or 2, D2 is the result of passing through source image I s low-resolution feature map Fs (n) and the low-resolution feature map F of the target image t (n) The obtained sparse motion flow.
[0028] In practical implementation, source image I s low-resolution feature map F s (8) and target image I t low-resolution feature map F t (8) Similarity matrix S g Specifically as follows: S g (x s ,y s ,x t ,y t )= ∑ (K / 2) i,j=-(K / 2) ( <F s,(xs+i,ys+j) (8) ,F t,(xt+i,yt+j) (8) > / || F s,(xs+i,ys+j) (8) ||·||F t,(xt+i,yt+j)) (8) ||) S g ∈R H / 8×W / 8×H / 8×W / 8 Among them, (x s ,y s ) and (x t ,y t ) are the source images I s Feature map and target image I t Feature map F t (8) The coordinates of the points on the graph; K represents the preset local window size, <> is the dot product, and || is the modulus.
[0029] Source image I after deformation transformation s High-resolution feature map F s (4) 'and target image I t High-resolution feature map F t (4) Similarity matrix S g The details are as follows: S g '(x s ,y s ,x t ,y t )= ∑ (K / 2)i,j=-(K / 2) (<F s,(xs+i,ys+j) (4) ',F t,(xt+i,yt+j) (4) > / || F s,(xs+i,ys+j) (4) ' ||·||F t,(xt+i,yt+j)) (4) ||) S g ∈R H / 8×W / 8×H / 8×W / 8 The similarity matrix reflects the similarity relationship between each local region on the source image and all possible matching regions on the target image, and provides a basis for subsequent motion field prediction.
[0030] As Figure 2 shown, the grouped motion estimation unit specifically comprises: first, uniformly dividing the motion hidden feature F c into N g groups of sub-feature groups, each group having C g sub-features, C g =C c / N g , C c being the total number of sub-features of the motion hidden feature F c ; then, for each group of sub-feature groups, performing linear transformation processing on each sub-feature in the sub-feature group, and then splicing the processed sub-feature groups, and then sequentially performing activation function ReLU and linear layer processing to obtain sparse motion flow. Each group of sub-feature groups is independently processed and independently linearly transformed, specifically as follows: F g,k '=W k F g,k +B k Wherein, F g,k ' is a sub-feature in the C k th group of sub-feature groups after linear transformation, F g,k is a sub-feature in the C k th group of sub-feature groups; W k and B k are the first and second learnable parameters of the C k th group of sub-feature groups, W k ∈R Cg'×Cg , B k ∈R Cg' , C g ' is the dimension of the sub-feature after linear transformation; W k and B k are iteratively optimized during network training.
[0031] The present application compares the reasoning time of grouped motion estimation in specific implementation, as shown in the following Table 1: Table 1
[0032] It can be seen that the group motion estimation method proposed in the application effectively reduces the inference time of the model while still having relatively high registration accuracy, proving the effectiveness of the method in reducing computational overhead.
[0033] The dense motion module first processes the similarity matrix S between the two feature maps g The neighborhood sampling mask M is obtained after processing by the neighborhood sampling mask generator, and M is in R 9×8×8×H / 8×W / 8 Then, based on the sparse motion flow and the neighborhood sampling mask M, the neighborhood refinement sampling unit is processed to obtain the final dense motion flow.
[0034] The motion hidden feature extractor includes N basic convolutional blocks connected in sequence, the neighborhood sampling mask generator includes P basic convolutional blocks connected in sequence, and the basic convolutional block includes a 3x3 convolutional layer and an activation function RELU connected in sequence.
[0035] The neighborhood refinement sampling unit first performs bilinear upsampling processing on the sparse motion flow to obtain the initial dense motion flow D', and performs softmax normalization on the neighborhood sampling mask M to obtain the normalized neighborhood sampling mask M'; then the initial dense motion flow D' and the normalized neighborhood sampling mask M' are subjected to neighborhood weighted correction processing to obtain the final dense motion flow D'', which is as follows: D''= D'+∑ k=1 9 (M' k ·D') Where M' k is the i-th neighborhood position mask value in the normalized neighborhood sampling mask M'. k
[0036] The neighborhood weighted correction combines the local neighborhood pixels and the mask weight to calculate the residual high-resolution motion flow, realizes the refinement of the motion, and thus reduces the blurring effect caused by the bilinear interpolation.
[0037] The neighborhood refinement sampling registration accuracy is compared in the specific implementation of the application, as shown in the following table 2: Table 2
[0038] It can be seen that the neighborhood refinement sampling proposed in the application improves the upper limit of the local deformation transformation of the model, thereby improving the accuracy of the model registration, and the registration accuracy is significantly improved.
[0039] After the model is built, the processed multi-view image dataset is input into the image registration network for training. It contains several pairs of source images and target images from different perspectives. Shared features are extracted through the shared feature extraction module, group motion estimation is performed through the sparse motion module, and finally neighborhood refinement sampling is performed based on the dense motion module until the loss function of the image registration network converges, thus obtaining the trained image registration network. The source images and target images from different perspectives are all visible light images taken by a camera, and there is some overlap in the content of each pair of source images and target images.
[0040] The loss function used in this invention is image content loss L. align In order to optimize network parameters, the following is a detailed description: L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+ λ0|| I t ·warp(O, D1'') - warp(I s , (D1'') T ) ||1+ λ1|| I s ·warp(O, (D F ) T ) - warp(I t D F ) ||1+ λ1|| I t ·warp(O, D F ) - warp(I s , (D F ) T ) ||1 Where λ0 and λ1 are the initial and final registration error weights, respectively; in practice, λ0=1, λ1=2; |||1 is the 1-norm; I s and I t These are the source image and the target image, respectively; `warp()` is the warp transformation operation; `O` is a matrix with all elements equal to 1; `D1''` is the image obtained by passing through the source image I... s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The resulting dense motion flow; T is the transpose; D F To register the moving flow, D FD1''+D2'', D2'' is a low-resolution feature map F s of the target image I s (n) obtained by the source image I t of the target image I t (n) The final dense motion flow obtained; the registration motion flow overcomes the blurring effect problem caused by the traditional bilinear interpolation, and effectively captures more local motion details.
[0041] Based on the image content loss L align When training the image registration network, the image registration network is optimized using gradient back propagation, the optimizer adopts Adam, the initial learning rate lr=1e -4 . The learning rate is reduced in an exponential decay manner every 10 batches epoch, and the decay rate is 0.95. The batch size is 32, and the training is 50 batches epoch. The network initialization method adopts the Kaiming initialization. Gradient clipping (gradient clipping norm clipnorm=1.0) is used to prevent gradient explosion. A single card A100 is used for model training. Finally, the trained image registration network is obtained.
[0042] Finally, the source image to be registered and the target image thereof are input into the trained image registration network, and the dense motion flow of the source image to be registered and the target image thereof is obtained, and then the registration motion flow is obtained, the target image is registered based on the registration motion flow, and the registered target image is output after processing, thereby realizing image registration. In specific implementation, the optimal model weight is loaded for testing, a pair of unseen source image and target image are input, the final registration motion flow D F is obtained through network forward inference, and the image is deformed to generate the registered target image. The method can be widely applied to image stitching and video stabilization scenes. Figure 3 、 Figure 4 and Figure 5 As shown, the registration results in the specific implementation of the method are a source image, a target image and a target image registered based on the method, which are captured by a camera at a certain position in a region, and it can be seen that the registration effect of the method is good.
[0043] The application provides a feasible and advanced image registration method, which has good engineering implementation value and wide practical prospect. For those skilled in the image processing field, the above embodiments are only exemplary and should not be construed as limiting the scope of protection of the application. Any equivalent replacement or improvement based on the idea of the application falls within the scope of protection of the application.
Claims
1. An image registration method based on packet motion estimation and neighborhood refinement sampling, characterized in that, The method comprises the steps of: Step S1: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module connected in sequence; Step S2: obtaining a plurality of pairs of source images and target images of different perspectives thereof and inputting the image registration network for training, extracting shared features through the shared feature extraction module, performing grouped motion estimation through the sparse motion module, and finally performing neighborhood refinement sampling based on the dense motion module until the loss function of the image registration network converges, thereby obtaining a trained image registration network; Step S3: inputting the source image to be registered and the target image thereof into the trained image registration network, obtaining the dense motion flow of the source image to be registered and the target image thereof, and then obtaining the registration motion flow, registering the target image to the source image based on the registration motion flow, outputting the registered target image after processing, and realizing image registration.
2. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 1, wherein: In step S1, the shared feature extraction module specifically employs a MobileNet convolutional neural network with shared weights to extract source images I of the same resolution. s and target image I t The input is processed through a convolutional neural MobileNet network for forward propagation, and the source image I is extracted respectively. s High-resolution feature map F s (m) and low-resolution feature map F s (n) and target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) , m < n Source Image I s High-resolution feature map F s (m) and low-resolution feature map F s (n) The image sizes are respectively the source image I s 1 / of the image size m and 1 / n Target image I t High-resolution feature map F t (m) and low-resolution feature map F t (n) The image sizes are respectively the target image I t 1 / of the image size m and 1 / n .
3. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 2, wherein: The step S1, by sparse motion module to source image I s The low resolution feature map F s (n) And the low resolution feature map F t Target image I t (n) As two input feature maps, the high resolution feature map F s Of source image I s (m) And the high resolution feature map F t Target image I t (m) After morphing transformation, as two input feature maps through sparse motion module, the high resolution feature map F s Of source image I s (m) And the high resolution feature map F t Target image I t (m) As follows: F T (m) '=warp(F T (m) ,D1''), T = s or t Among them, F s (m) 'and F t (m) 'These are the source images I after deformation transformation.' s High-resolution feature maps and target images I t High-resolution feature map; warp() is the deformation transformation operation; D1'' is the high-resolution feature map obtained from the source image I. s High-resolution feature map F s (m) and target image I t High-resolution feature map F t (m) The final dense flow of motion obtained; In the sparse motion module, first, a similarity matrix S between two feature maps is constructed g Then, the motion latent feature F is extracted by the motion latent feature extractor c Then, it is input into the grouped motion estimation unit for processing to obtain the sparse motion flow.
4. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 3, wherein: The grouping motion estimation unit specifically comprises: first, motion hidden features F c are evenly divided into N g groups of sub-feature groups, each group having C g sub-features, C g =C c / N g , C c is the total number of sub-features of the motion hidden features F c ; then, for each group of sub-feature groups, each sub-feature in the sub-feature group is subjected to linear transformation processing, and then the processed groups of sub-feature groups are spliced and sequentially subjected to activation function ReLU and linear layer processing to obtain a sparse motion flow.
5. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 3, wherein: In the step S1, the dense motion module first calculates the similarity matrix S between the two feature maps g After the neighborhood sampling mask generator processing, the neighborhood sampling mask M is obtained, and then based on the sparse motion flow and the neighborhood sampling mask M, the neighborhood refinement sampling unit is processed to obtain the final dense motion flow.
6. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 5, wherein: The motion hidden feature extractor comprises N basic convolutional blocks connected in sequence, and the neighborhood sampling mask generator comprises P basic convolutional blocks connected in sequence.
7. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 5, wherein: The neighborhood refinement sampling unit first performs bilinear upsampling processing on the sparse motion flow to obtain an initial dense motion flow D', and simultaneously performs softmax normalization on the neighborhood sampling mask M to obtain a normalized neighborhood sampling mask M'; then, the initial dense motion flow D' and the normalized neighborhood sampling mask M' are subjected to neighborhood weighted correction processing to obtain a final dense motion flow D'', and the specific process is as follows: D" = D' + ∑ k=1 9 (M' k ·D') where M' = M / N k is the normalized neighborhood sample mask M' in the i-th k neighborhood position mask value.
8. The image registration method based on packet motion estimation and neighborhood refinement sampling of claim 5, wherein: In the step S2, the loss function adopts an image content loss L align , and specifically as follows: L align =λ0|| I s ·warp(O, (D1'' ) T ) - warp(I t , D1'') ||1+ || I t • warp(0, D1'') - warp(I s , (D1'') T ) ||1+ λ1|| I s • warp(O, (D F ) T ) - warp(I t , D F )||1+ λ1|| I t • warp(O, D F ) s , (D F ) T ) ||1 where λ0and λ1are the initial and final registration error weights, respectively; || ||1is the 1-norm; I s and I t are the source and target images, respectively; warp() is the warping operation; O is a matrix of all ones; D1'' is the final dense motion field obtained from the high resolution feature maps F s of the source image I s and the high resolution feature maps F (m) of the target image I t ; T is the transpose; D t is the registered motion field, D (m) = D1'' + D2'', D2'' is the final dense motion field obtained from the low resolution feature maps F F of the source image I F and the low resolution feature maps F s of the target image I s ; and (n) t t (n) Based on the image content loss L align When the image registration network is trained, the image registration network is optimized using gradient back propagation, and a trained image registration network is obtained.
9. An electronic device, comprising: The method comprises the steps of: The memory and the processor are coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1-8.
10. A computer readable storage medium having stored thereon program data, wherein, The program data is executed by the processor to realize the method according to any one of claims 1-8.
Citation Information
Patent Citations
Joint sparse representation hyperspectral image classification method based on dual neighborhood constraints
CN120564030A
Guided sparse feature matching via coarsely defined dense matches
US20190318196A1
Generating digital images utilizing high-resolution sparse attention and semantic layout manipulation neural networks
US20220327657A1
