Image splicing method based on multi-stage feature matching
By combining multi-stage feature matching and network optimization, the problem of poor image stitching effect in existing technologies is solved, and high-quality image stitching with correct semantic structure and natural texture transition at the seams is achieved.
Patent Information
- Application Number
- CN202411143510.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-26
AI Technical Summary
Existing image stitching methods for autonomous driving are insufficient in terms of homography estimation accuracy between complex images and visual effects at seams, especially in images with large parallax and low feature points.
An image stitching method based on multi-stage feature matching is adopted. By combining stitching and optimization networks, a transformation module, a convolution module, a feature matching module, and a symmetric dynamic fusion module are used to perform matching and fusion between feature maps at different scales. Deformable convolutional layers and sigmoid functions are combined to optimize the weight relationship at the seam. A GAN discriminator is introduced for training to improve stitching accuracy and quality.
This method achieves image stitching results that include both correct semantic contours and fine detail textures at the seams, improving the accuracy of homography estimation and stitching quality between complex images, reducing artifacts, and generating more realistic stitched images.
Smart Images

Figure CN119027311A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an image stitching method based on multi-stage feature matching. Background Art
[0002] With the continuous development of artificial intelligence technology, autonomous driving and path planning have become indispensable intelligent technologies for unmanned vehicles. The premise for unmanned vehicles to realize these technologies is to use multiple visual sensors to capture information about the surrounding environment. However, ordinary single visual sensors can only capture environmental information within a certain angle range, and the environmental information captured by panoramic visual sensors has problems such as distortion and distortion, which cannot guarantee accurate autonomous navigation and driving. Therefore, using image stitching technology to stitch the image information captured by multiple sensors into a whole panoramic information is crucial for unmanned vehicles to perform autonomous driving.
[0003] Image stitching methods are divided into four steps: image matching, reprojection, stitching, and fusion. In traditional stitching methods, the research focus is mainly on the two steps of image matching and fusion. The methods focusing on the image matching step include the image stitching method based on underlying features and the image stitching method based on contours. Among them, the algorithm proposed by Joshi et al., which combines the Harris corner detector and the SURF detector, is a representative image stitching method based on underlying features. The image stitching method based on contours is represented by the method of Deshmukh et al., which uses the centroid of the image contour as the starting point to perform registration according to the centroid of the extracted contour. Although these methods are based on high-level features, they ignore the complexity between high-level contours, making it easy for the stitched images to have ghosting problems at the seams. The methods focusing on the fusion step include the image stitching method based on smooth transition and the image stitching method based on optimal seams. Among them, the gradient domain image stitching (GIST) method proposed by Levin et al. is a representative image stitching method based on smooth transition. This method reduces seam ghosting by optimizing the gradient at the seams. Representative image stitching methods based on optimal seams include Wen et al., who used dynamic programming to continuously update the selected optimal seams and analyzed the grayscale relationship between the two regions between the selected seams to further determine the optimal seams. The above traditional image stitching methods have good stitching effects on images with multiple features and at the same parallax level, but have poor stitching effects on images with many feature points, large parallax, and distortion.
[0004] The stitching methods based on deep learning are still in the development stage. Among them, DeTone et al. first applied convolutional neural network (CNN) to the image stitching algorithm and proposed a deep homography method. This method uses CNN network to learn to match the feature points of the reference image and the target image, and estimates the homography relationship of the same feature points on the two images. According to this homography, the target image is distorted to maintain the same baseline level as the reference image, which provides a guarantee for the subsequent stitching operation. Chang et al. proposed a cascaded image alignment network, which uses CNN combined with Lucas Kanade method to predict the motion parameters of pixels in multi-scale feature images to achieve accurate image alignment. Zhang et al. proposed an unsupervised content-aware method, which uses its own network to learn a mask as an attention map to cover the dynamic area, and selects reliable fixed areas for homography estimation. Finally, this method achieves good homography estimation performance in small baseline scenes. The above methods only use deep learning knowledge in the stages of feature point detection or feature matching, and do not incorporate deep learning knowledge in all stages of image stitching. Nie et al. first proposed a complete framework model based on deep learning. The model uses CNN combined with DLT to calculate the homography relationship between images, then uses the homography to warp the transformed image for stitching, and finally uses the VGG network for optimization. Although the network can handle most views with different baseline levels, it can only handle input images with fixed resolution. To solve this problem, Nie et al. combined the ideas of cascade and pyramid frameworks to propose an edge-preserving deep image stitching method, which connects feature maps of different scale levels to achieve stitching of images of different resolutions, improving its generalization ability. However, this method only uses ordinary convolution to estimate the homography relationship between two images, and the stitching effect of images with large parallax and low feature points is poor.
[0005] Existing image stitching methods for autonomous driving are mainly based on deep learning and use convolutional neural networks to perform homography estimation. However, they have shortcomings in the accuracy of homography estimation between complex images and the visual effects at the seams. Summary of the invention
[0006] In view of the problems and shortcomings of the prior art, the object of the present invention is to provide an image stitching method based on multi-stage feature matching.
[0007] To achieve the purpose of the invention, the technical solution adopted by the present invention is as follows:
[0008] A first aspect of the present invention provides an image stitching method based on multi-stage feature matching, comprising: inputting a reference image R and a target image T to be stitched into a pre-trained image stitching network model, using the image stitching network model to perform stitching processing, obtaining a final stitched image of the reference image R and the target image T and outputting it; wherein the image stitching network model consists of a stitching network and an optimization network; the stitching network is used to match the nonlinear contour features of the same object between the reference image R and the target image T to be stitched, and extract the homography relationship between the reference image R and the target image T to be stitched, and use the homography relationship to distort and deform the reference image R and the target image T to be stitched and perform dynamic fusion, the input of the stitching network is the reference image R and the target image T, and the output of the stitching network is a primary stitched image of the reference image R and the target image T to be stitched; the optimization network is used to optimize the detail texture, semantic structure and overall quality of the primary stitched image to obtain a final stitched image, the input of the optimization network is the initial stitched image, and the output of the optimization network is the final stitched image.
[0009] According to the above-mentioned image stitching method, preferably, the stitching network includes a conversion module, a first convolution module, a second convolution module, a first feature matching module, a second feature matching module, a third feature matching module, a first symmetric dynamic fusion module, a second symmetric dynamic fusion module, and a third symmetric dynamic fusion module; the stitching network stitching process of the reference image R and the target image T includes four stages, the first stage is: using the conversion module to convert the input reference image R and the target image T, and converting the reference image R into three reference images R1, reference image R2, and reference image T with different resolutions. R3 and output, convert the target image T into three target images T1, target image T2, and target image T3 with different resolutions and output them; the second stage is: use the first feature matching module to perform feature matching on the target image T3 and the reference image R3 to obtain the offset feature map T'3 and the offset feature map R'3 and output them, use the first symmetric dynamic fusion module to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'3 and the offset feature map R'3 in turn, obtain the first fused feature map and output it, then perform channel addition on the first fused feature map and the target image T2 to obtain T2 H The third stage is: using the first convolution module to perform convolution processing on the reference image R2 (wherein the first convolution module is composed of four groups of convolution layers, and the purpose of using the first convolution module to perform convolution processing on the reference image R2 is to capture features to balance the network carrying capacity), and then using the second feature matching module to perform convolution processing on T2 HThe feature map is matched with the reference image R2 processed by the first convolution module to obtain the offset feature map T'2 and the offset feature map R'2 and output them. The second symmetric dynamic fusion module is used to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'2 and the offset feature map R'2 in turn to obtain the second fused feature map and output it. Then, the second fused feature map is channel-added with the target image T1 to obtain The fourth stage is: using the second convolution module to perform convolution processing on the reference image R1 (wherein the second convolution module is composed of eight groups of convolution layers), and then using the third feature matching module to The feature map is matched with the reference image R1 processed by the second convolution module to obtain the offset feature map T'1 and the offset feature map R'1 and output them. The third symmetric dynamic fusion module is used to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'1 and the offset feature map R'1 in turn to obtain the primary splicing image S F And output. The second, third and fourth stages of the stitching process of the splicing network of the present invention for the reference image R and the target image T correspond to the matching and fusion of three different scale feature maps of the reference image R and the target image T, respectively, wherein the feature matching of the low-scale feature map can include the global contour of the same object between the two images R and T, and the feature matching of the high-scale feature map can include the internal details of the same object in the two images. The low-scale feature map is then fused into the high-scale feature map by the symmetrical dynamic fusion module, so that the stitching result can finally include both the correct semantic contour and the fine detail texture at the seam.
[0010] According to the above-mentioned image stitching method, preferably, the network structures of the first feature matching module, the second feature matching module, and the third feature matching module are the same, and the first feature matching module, the second feature matching module, and the third feature matching module all include a channel fusion layer, a first convolution layer, a second convolution layer, a first deformable convolution layer DCN, and a second deformable convolution layer DCN; the first feature matching module, the second feature matching module, and the third feature matching module perform the same process of image matching. Among them, taking the first feature matching module performing feature matching processing on the target image T3 and the reference image R3 as an example, the specific process of the first feature matching module performing feature matching processing on the target image T3 and the reference image R3 is: the target image T3 and the reference image R3 are subjected to channel fusion through the channel fusion layer and then subjected to convolution processing using the first convolution layer to obtain an offset The offset The reference image R3 is input into the first variable convolution layer DCN for processing, and the offset feature map R'3 with the offset of the reference image R3 relative to the target image T3 is obtained and output; the offset feature map R'3 is convolved through the second convolution layer to obtain the offset The offset With offset An element-wise multiplication operation is performed to obtain a total offset, and the total offset and the target image T3 are input into a second deformable convolutional layer DCN for processing to obtain an offset feature map T'3 with an offset of the target image T3 relative to the reference image R3.
[0011] According to the above-mentioned image stitching method, preferably, the network structures of the first symmetrical dynamic fusion module, the second symmetrical dynamic fusion module, and the third symmetrical dynamic fusion module are the same, and are all symmetrical interlaced fusion structures; the first symmetrical dynamic fusion module, the second symmetrical dynamic fusion module, and the third symmetrical dynamic fusion module have the same process of homography prediction, distortion deformation, and stitching and fusion processing on the image; taking the first symmetrical dynamic fusion module as an example, it includes an offset feature map R'3 processing branch and an offset feature map T'3 processing branch, and the offset feature map R'3 processing branch and the offset feature map T'3 processing branch both include a direct linear transformation module, a spatial transformation network, a channel fusion module, a convolution module, and a Sigmoid activation function module; the offset feature map T'3 and the offset feature map R'3 input into the first symmetrical dynamic fusion module are respectively processed by the offset feature map T'3 processing branch and the offset feature map R'3 processing branch, and then channel fusion and convolution processing are performed to obtain the first fused feature map.
[0012] According to the above-mentioned image stitching method, preferably, the process of the first symmetric dynamic fusion module performing homography prediction, distortion deformation and stitching fusion processing on the input offset feature map T'3 and the offset feature map R'3 is divided into four stages;
[0013] The first stage is: for the offset feature map R'3 processing branch, the direct linear transformation module (solvingDLT) is first used to directly linearly transform the input offset feature map R'3 to obtain a homography matrix with eight offset coordinates Then the homography matrix Together with the offset feature map R'3, it is input into the spatial transformer network (STN) for distortion and deformation processing to obtain a feature map W at the same baseline level. R ; For the offset feature map T'3 processing branch, the direct linear transformation module (solving DLT) is first used to directly linearly transform the input offset feature map T'3 to obtain a homography matrix with eight offset coordinates Then the homography matrix Together with the offset feature map T'3, it is input into the spatial transformer network (STN) for distortion and deformation processing to obtain a feature map W at the same baseline level. T ;
[0014] The second stage is: for the offset feature map R'3 processing branch, the channel fusion module is used to transform the feature map W R and feature map W T After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map. The convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a (0, 1) distribution to obtain a feature map W. R and feature map W T Relative weight relationship diagram S R ; For the offset feature map T'3 processing branch, the channel fusion module is used to transform the feature map W T and feature map W R After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map. The convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a (0, 1) distribution to obtain a feature map W. T and feature map W R Relative weight relationship diagram S T ; Relative weight relationship diagram S R And the relative weight relationship diagram S T Used to dynamically adjust the pixel information of the image in the overlapping area;
[0015] The third stage is: for the offset feature map R'3 processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S R Multiply (this multiplication operation can ensure that the network dynamically optimizes the overlapping area, reduces the artifacts in the area, and corrects the incorrect semantic transition area) to obtain the feature map R"3, and then add the feature map R"3 to the feature map feature map W R Splicing is performed along the channel direction to obtain the feature map R"'3, which is the final output of the offset feature map R'3 processing branch; for the offset feature map T'3 processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S T Multiply (this multiplication operation can ensure that the network dynamically optimizes the overlapping area, reduces the artifacts in the area, and corrects the incorrect semantic transition area) to obtain the feature map T"3, and then add the feature map T"3 to the feature map feature map W TSplicing is performed along the channel direction to obtain the feature map T'3, which is the final output of the offset feature map T'3 processing branch;
[0016] The fourth stage is: using the channel fusion module to splice the feature map R'3 output by the offset feature map processing branch and the feature map T'3 output by the offset feature map processing branch along the channel direction, and then use a 1×1 convolution layer to perform convolution processing to obtain the first fused feature map.
[0017] According to the above-mentioned image stitching method, preferably, the optimization network includes a Conv convolution module, a convolution network and a deconvolution network, the convolution network is used to extract features from the primary stitching image, and extract key features from the primary stitching image; the deconvolution network is used to perform structural reconstruction, and a jump connection is provided between the convolution network and the deconvolution network.
[0018] According to the above-mentioned image stitching method, preferably, the optimization network performs the optimization on the input primary stitching image S F The processing process is as follows: first, the primary spliced image S is processed by the Conv convolution module F Convolution processing is performed to obtain a convolution feature map; the convolution feature map is input into the convolution network for key feature extraction, and the extracted key features are processed by the Conv convolution module and then input into the deconvolution network for upsampling to gradually restore the spatial resolution of the image and increase its dimension. In addition, in order to retain the image details, a skip connection is set between the convolution network and the deconvolution network to ensure the synchronization of the parameters of the convolution network and the deconvolution network; then the feature map output by the deconvolution network is combined with the primary spliced image S F The convolution feature map output after processing by the Conv convolution module performs element-wise addition to obtain the final spliced image and output it.
[0019] According to the above-mentioned image stitching method, preferably, in order to reduce memory usage and computing resources, when the conversion module is used to convert the input reference image R and the target image T, the conversion module first converts the reference image R and the target image T into grayscale images, and then performs three two-layer convolution processes on the grayscale image of the reference image R and the grayscale image of the target image T respectively, to obtain three resolution reference images R1, reference image R2, reference image R3 and three target images T1, target image T2, target image T3 with different resolutions.
[0020] According to the above-mentioned image stitching method, preferably, the resolutions of the reference image R1, reference image R2, and reference image R3 are respectively 1 times, 0.5 times, and 0.25 times the resolution of the reference image R; the resolutions of the target image T1, target image T2, and target image T3 are respectively 1 times, 0.5 times, and 0.25 times the resolution of the target image T.
[0021] A second aspect of the present invention provides a method for training an image splicing network model, characterized in that it comprises the following steps:
[0022] S1: Collect two images of different viewpoints and a full viewpoint image of each of n scenes, record the two images of different viewpoints as reference image and target image respectively, and there is overlap between the reference image and the target image; the reference image and the target image of each scene form a pair of samples, and the full viewpoint image of each scene is used as the real image of the pair of samples, and a total of n pairs of samples are obtained, and the n pairs of samples are divided into a training sample set and a verification sample set according to proportion;
[0023] S2: using the training sample set to train the pre-built image stitching network model, optimizing the parameters of the image stitching network model, and obtaining the trained image stitching network model;
[0024] S3: Use the verification sample set to verify all trained image stitching network models, evaluate the image processing performance of the trained image stitching network models, select the optimal trained image stitching network, and complete the training of the image stitching network model; wherein the image stitching network model is the image stitching network model described in the first aspect above.
[0025] According to the above training method, preferably, the specific operation of step S2 is: inputting the training samples in the training sample set into the image stitching network model for processing, obtaining and outputting the final stitching image of the reference image R and the target image T in the training sample; calculating the difference loss between the final stitching image and the real image corresponding to the training sample, and then inputting the final stitching image and the real image corresponding to the training sample into the GAN (Generative Adversarial Network) discriminator for comparison to obtain the GAN discriminator loss, constructing a loss function using the difference loss and the GAN discriminator loss, and adjusting the parameters of the image stitching network model using the back propagation algorithm according to the constructed loss function, so that the loss function converges, and obtaining the trained image stitching network model; wherein the calculation formula of the loss function is as follows:
[0026] L=λ1×L rec +λ2×L D
[0027]
[0028]
[0029] Where λ1 and λ2 are weight parameters to balance the two losses, L rec is the difference loss function, L D is the GAN discriminator loss, I i is the pixel value of the real image, Ji is the pixel value of the final stitched image generated, N is the total number of pixels, D is the discriminator, I is the real image, and J is the generated image.
[0030] A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image stitching method as described in the first aspect or the training method as described in the second aspect is implemented.
[0031] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the image stitching method as described in the first aspect above or the training method as described in the second aspect above is implemented.
[0032] Compared with the prior art, the technical effects achieved by the present invention are as follows:
[0033] (1) The stitching network in the image stitching network model of the present invention includes a conversion module, a first convolution module, a second convolution module, a first feature matching module, a second feature matching module, a third feature matching module, a first symmetric dynamic fusion module, a second symmetric dynamic fusion module, and a third symmetric dynamic fusion module; the second, third, and fourth stages of the stitching process of the reference image R and the target image T using the stitching network correspond to the matching and fusion of three different scale feature maps of the reference image R and the target image T, respectively, wherein the feature matching of the low-scale feature map can include the global contour of the same object between the two images R and T, and the feature matching of the high-scale feature map can include the internal details of the same object in the two images, and then the low-scale feature map is fused into the high-scale feature map by the symmetric dynamic fusion module, so that the stitching result can finally include both the correct semantic contour and the fine detail texture at the seam.
[0034] (2) The existing stitching method only uses ordinary convolution to extract the homography relationship between images and only uses channel addition to stitch images, resulting in poor stitching effect for complex images. The feature matching module in the stitching network of the present invention introduces a deformable convolution layer DCN to learn the homography relationship between the stitched images, which can better learn the nonlinear contours of the same object between the stitched images and more accurately estimate the homography matrix between the stitched images, thereby achieving accurate registration of the same features between complex images; and also fuses the homography matrices between multiple high-scale and low-scale feature maps to further improve the homography mapping accuracy between the stitched images, laying a foundation for the subsequent accurate distortion and deformation of the stitched images; in addition, the feature matching module can learn the relative position relationship between different contour points of the same target object in the target image relative to the reference image in the image feature space, and can learn the deformation process of the object contour on the target image T from the reference image R and digitize it; therefore, the feature matching module has a stronger ability to capture the features of complex contour images, and can dynamically learn the mutual position transformation relationship between the target image and the reference image, providing a good feature matching and alignment basis for the subsequent stitching of the two images.
[0035] (3) The symmetrical dynamic fusion module in the splicing network of the present invention adopts a symmetrical staggered fusion structure and also introduces a sigmoid function, which can continuously adjust the weight relationship of the same object in the seam area between the distorted and deformed images during the overall network training process, and dynamically adjust the weight parameters to further reduce the ghosting at the seams, improve the splicing quality, and ensure that the semantic structure of the spliced image at the seams is correct and the texture transition is natural.
[0036] (4) The optimization network of the present invention includes a convolutional network and a deconvolutional network. The important semantic features of the primary splicing feature map are processed and learned through the aggregation network, and then the structure is reconstructed through the Fanju Aji network. In addition, a jump connection is provided between the convolutional network and the deconvolutional network. The spliced image after optimization processing contains finer details and more accurate semantic structure, thereby improving the detail texture and overall quality of the spliced image.
[0037] (5) The GAN discriminator is introduced into the training process of the image splicing network model of the present invention to construct the GAN loss, so that the overall splicing network can still splice and generate more realistic samples that are closer to the real data distribution with a low training cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 Schematic diagram of the network architecture of the splicing network in the image splicing network model of the present invention;
[0039] Figure 2 A schematic diagram of the network architecture of a conversion module in a splicing network of the present invention;
[0040] Figure 3A schematic diagram of the network architecture of a feature matching module in a splicing network of the present invention;
[0041] Figure 4 A schematic diagram of the network architecture of a symmetrical dynamic fusion module in a splicing network of the present invention;
[0042] Figure 5 A schematic diagram of the network architecture of the optimized network in the image stitching network model of the present invention;
[0043] Figure 6 This is a schematic diagram of the image stitching network model training process of the present invention;
[0044] Figure 7 The following are the splicing effect diagrams of images using the method of the present invention and the comparison method under different light levels;
[0045] Figure 8 The diagrams are the image stitching effects of the method of the present invention and the comparative method under different overlap rates and parallaxes. DETAILED DESCRIPTION
[0046] The present invention is further described in detail below through specific examples, but the scope of the present invention is not limited thereto.
[0047] Embodiment 1:
[0048] A method for image stitching based on multi-stage feature matching, wherein a reference image R and a target image T to be stitched are input into a pre-trained image stitching network model, the image stitching network model is used to perform stitching processing, and a final stitched image of the reference image R and the target image T is obtained and output.
[0049] The image stitching network model is composed of a stitching network and an optimization network; the stitching network is used to match the nonlinear contour features of the same object between the reference image R and the target image T to be stitched, and extract the homography relationship between the reference image R and the target image T to be stitched, and use the homography relationship to distort the reference image R and the target image T to be stitched and dynamically fuse them, the input of the stitching network is the reference image R and the target image T, and the output of the stitching network is the primary stitching image of the reference image R and the target image T to be stitched; the optimization network is used to optimize the detail texture, semantic structure and overall quality of the primary stitching image to obtain the final stitching image, the input of the optimization network is the initial stitching image, and the output of the optimization network is the final stitching image.
[0050] Splicing network (such as Figure 1The splicing network includes a conversion module, a first convolution module, a second convolution module, a first feature matching module, a second feature matching module, a third feature matching module, a first symmetric dynamic fusion module, a second symmetric dynamic fusion module, and a third symmetric dynamic fusion module. The splicing network includes four stages for the splicing process of the reference image R and the target image T, which are as follows:
[0051] The first stage is: using the conversion module to convert the input reference image R and the target image T, converting the reference image R into three reference images R1, R2, and R3 with different resolutions and outputting them, and converting the target image T into three target images T1, T2, and T3 with different resolutions and outputting them. Furthermore, in order to reduce memory usage and computing resources, when using the conversion module to convert the input reference image R and the target image T, the conversion module first converts the reference image R and the target image T into grayscale images, and then performs three convolutions on the grayscale image of the reference image R and the grayscale image of the target image T (the convolution module used in the convolution process consists of two convolution layers), to obtain three resolution reference images R1, R2, and R3 and three target images T1, T2, and T3 with different resolutions, and in the process of performing three convolutions on the grayscale image of the reference image R and the grayscale image of the target image T, a shared weight (such as Figure 2 As shown), the convolution process can further learn the correlation between the corresponding features of the reference image R and the target image T. Furthermore, the resolutions of the reference images R1, R2, and R3 are 1, 0.5, and 0.25 times the resolution of the reference image R, respectively; the resolutions of the target images T1, T2, and T3 are 1, 0.5, and 0.25 times the resolution of the target image T, respectively.
[0052] The second stage is: using the first feature matching module to perform feature matching on the target image T3 and the reference image R3, obtaining the offset feature map T'3 and the offset feature map R'3 and outputting them, using the first symmetric dynamic fusion module to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'3 and the offset feature map R'3 in turn, obtaining the first fused feature map and outputting it, and then performing channel addition on the first fused feature map and the target image T2 to obtain Feature map.
[0053] The third stage is: using the first convolution module to perform convolution processing on the reference image R2, and then using the second feature matching module to The feature map is matched with the reference image R2 processed by the first convolution module to obtain the offset feature map T'2 and the offset feature map R'2 and output them. The second symmetric dynamic fusion module is used to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'2 and the offset feature map R'2 in turn to obtain the second fused feature map and output it. Then, the second fused feature map is channel-added with the target image T1 to obtain Feature map.
[0054] The fourth stage is: using the second convolution module to perform convolution processing on the reference image R1, and then using the third feature matching module to The feature map is matched with the reference image R1 processed by the second convolution module to obtain the offset feature map T'1 and the offset feature map R'1 and output them. The third symmetric dynamic fusion module is used to perform homography prediction, distortion deformation and splicing fusion processing on the offset feature map T'1 and the offset feature map R'1 in turn to obtain the primary splicing image S F And output.
[0055] The network structure of the first feature matching module, the second feature matching module, and the third feature matching module in the splicing network (such as Figure 3 The first feature matching module, the second feature matching module, and the third feature matching module all include a channel fusion layer, a first convolution layer, a second convolution layer, a first deformable convolution layer DCN, and a second deformable convolution layer DCN. Taking the first feature matching module as an example, the specific process of the first feature matching module performing feature matching processing on the target image T3 and the reference image R3 is as follows: the target image T3 and the reference image R3 are subjected to channel fusion by the channel fusion layer and then subjected to convolution processing by the first convolution layer to obtain an offset The offset The reference image R3 is input into the first variable convolution layer DCN for processing, and the offset feature map R'3 with the offset of the reference image R3 relative to the target image T3 is obtained and output; the offset feature map R'3 is convolved through the second convolution layer to obtain the offset The offset With offset Perform element-wise multiplication to obtain a total offset, and input the total offset and the target image T3 into the second deformable convolution layer DCN for processing to obtain an offset feature map T'3 with the offset of the target image T3 relative to the reference image R3. The process of the second feature matching module and the third feature matching module matching the image is basically the same as that of the first feature matching module, except that the images input into the second feature matching module and the third feature matching module are different. Therefore, the process of the second feature matching module and the third feature matching module matching the image can refer to the first feature matching module, which will not be described in detail here.
[0056] The network structures of the first symmetric dynamic fusion module, the second symmetric dynamic fusion module, and the third symmetric dynamic fusion module in the splicing network are the same, and are all symmetrical interlaced fusion structures (such as Figure 4 As shown); taking the first symmetric dynamic fusion module as an example, it includes an offset feature map R'3 processing branch and an offset feature map T'3 processing branch, and the offset feature map R'3 processing branch and the offset feature map T'3 processing branch both include a direct linear transformation module, a spatial transformation network, a channel fusion module, a convolution module, and a Sigmoid activation function module; the offset feature map T'3 and the offset feature map R'3 input into the first symmetric dynamic fusion module are respectively processed by the offset feature map T'3 processing branch and the offset feature map R'3 processing branch, and then channel fusion and convolution are performed to obtain the first fused feature map.
[0057] The process of the first symmetric dynamic fusion module performing homography prediction, distortion deformation, and splicing fusion processing on the input offset feature map T'3 and the offset feature map R'3 is divided into four stages, as follows:
[0058] The first stage is: for the offset feature map R'3 processing branch, the direct linear transformation module (solvingDLT) is first used to directly linearly transform the input offset feature map R'3 to obtain a homography matrix with eight offset coordinates Then the homography matrix Together with the offset feature map R'3, it is input into the spatial transformer network (STN) for distortion and deformation processing to obtain a feature map W at the same baseline level. R ; For the offset feature map T'3 processing branch, the direct linear transformation module (solving DLT) is first used to directly linearly transform the input offset feature map T'3 to obtain a homography matrix with eight offset coordinates Then the homography matrix Together with the offset feature map T'3, it is input into the spatial transformer network (STN) for distortion and deformation processing to obtain a feature map W at the same baseline level. T ;
[0059] The second stage is: for the offset feature map R'3 processing branch, the channel fusion module is used to transform the feature map W R and feature map W T After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map. The convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a (0, 1) distribution to obtain a feature map W. R and feature map W T Relative weight relationship diagram S R ; For the offset feature map T'3 processing branch, the channel fusion module is used to transform the feature map W T and feature map W R After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map. The convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a (0, 1) distribution to obtain a feature map W. T and feature map W R Relative weight relationship diagram S T ; Relative weight relationship diagram S R And the relative weight relationship diagram S T Used to dynamically adjust the pixel information of the image in the overlapping area;
[0060] The third stage is: for the offset feature map R'3 processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S R Multiply (this multiplication operation can ensure that the network dynamically optimizes the overlapping area, reduces the artifacts in the area, and corrects the incorrect semantic transition area) to obtain the feature map R"3, and then add the feature map R"3 to the feature map feature map W R Splicing is performed along the channel direction to obtain the feature map R"'3, which is the final output of the offset feature map R'3 processing branch; for the offset feature map T'3 processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S T Multiply (this multiplication operation can ensure that the network dynamically optimizes the overlapping area, reduces the artifacts in the area, and corrects the incorrect semantic transition area) to obtain the feature map T"3, and then add the feature map T"3 to the feature map feature map W T Splicing is performed along the channel direction to obtain the feature map T'3, which is the final output of the offset feature map T'3 processing branch;
[0061] The fourth stage is: using the channel fusion module to splice the feature map R'3" output by the offset feature map R'3 processing branch and the feature map T'3" output by the offset feature map T'3 processing branch along the channel direction, and then use a 1×1 convolution layer to perform convolution processing to obtain the first fused feature map.
[0062] The process of performing homography prediction, distortion, deformation and splicing and fusion processing on the image by the first symmetrical dynamic fusion module, the second symmetrical dynamic fusion module and the third symmetrical dynamic fusion module is basically the same as that of the first symmetrical dynamic fusion module. The difference lies in that the images input into the second symmetrical dynamic fusion module and the third symmetrical dynamic fusion module are different. Therefore, the process of performing homography prediction, distortion, deformation and splicing and fusion processing on the image by the second symmetrical dynamic fusion module and the third symmetrical dynamic fusion module can be referred to the first feature matching module and will not be described in detail here.
[0063] The optimization network in the image stitching network model (such as Figure 5 The optimization network includes a Conv convolution module, a convolution network and a deconvolution network. The convolution network is used to extract features from the primary spliced image and extract key features from the primary spliced image. The deconvolution network is used to perform structure reconstruction. A jump connection is provided between the convolution network and the deconvolution network. The optimization network performs a multi-step optimization on the input primary spliced image S. F The processing process is as follows: first, the primary spliced image S is processed by the Conv convolution module F Convolution processing is performed to obtain a convolution feature map; the convolution feature map is input into the convolution network for key feature extraction, and the extracted key features are processed by the Conv convolution module and then input into the deconvolution network for upsampling to gradually restore the spatial resolution of the image and increase its dimension. In addition, in order to retain the image details, a skip connection is set between the convolution network and the deconvolution network to ensure the synchronization of the parameters of the convolution network and the deconvolution network; then the feature map output by the deconvolution network is combined with the primary spliced image S F The convolution feature map output after processing by the Conv convolution module performs element-wise addition to obtain the final spliced image and output it.
[0064] Embodiment 2:
[0065] A training method for an image splicing network model comprises the following steps:
[0066] S1: Collect two images with different viewpoints and a full viewpoint image of each of n scenes, and record the two images with different viewpoints as reference image and target image respectively, with overlap between the reference image and the target image; the reference image and the target image of each scene form a pair of samples, and the full viewpoint image of each scene is used as the real image of the pair of samples, and a total of n pairs of samples are obtained, which are divided into a training sample set and a verification sample set according to proportion.
[0067] S2: Input the training samples in the training sample set into a pre-constructed image stitching network model for processing, obtain the final stitching image of the reference image R and the target image T in the training sample and output it, wherein the image stitching network model is the image stitching network model described in Example 1; calculate the difference loss (also called reconstruction loss) between the final stitching image and the real image corresponding to the training sample, then input the final stitching image and the real image corresponding to the training sample into the GAN discriminator for comparison, obtain the GAN discriminator loss (also called GAN loss), construct a loss function using the difference loss and the GAN discriminator loss, and adjust the parameters of the image stitching network model using the back propagation algorithm according to the constructed loss function, so that the loss function converges, and obtain the trained image stitching network model; wherein the calculation formula of the loss function is as follows:
[0068] L=λ1×L rec +λ2×L D
[0069]
[0070]
[0071] Where λ1 and λ2 are weight parameters to balance the two losses, L rec is the difference loss function, L D is the GAN discriminator loss, I i is the pixel value of the real image, J i is the pixel value of the final stitched image generated, N is the total number of pixels, D is the discriminator, I is the real image, and J is the generated image.
[0072] S3: Use the verification sample set to verify all trained image stitching network models, evaluate the image processing performance of the trained image stitching network models, select the optimal trained image stitching network model, and complete the training of the image stitching network model.
[0073] In order to ensure the smooth experiment of the image stitching algorithm, the inventor used the dataset preparation method of Nie et al. to prepare 11,600 pairs of training images, 1,200 pairs of verification images and 1,200 pairs of test images with a resolution of 256×256 and real stitching labels based on the UDIS-D dataset, and used 11,600 pairs of training images as training sample sets, 1,200 pairs of verification images as verification sample sets, and 1,200 pairs of test images as test sets to test the trained image stitching network model. The hardware platform used for the image stitching network model training experiment is shown in Table 1. During the training of the stitching network, four NVIDIA GeForceRTX1080s were used simultaneously for training in parallel. In order to achieve the optimal training effect of the image stitching network model, this paper continuously adjusted the training parameters and finally used the parameters in Table 2 for network training. Among them, the initial learning rate is 0.0001, the batch size is 14, and the total iteration is 200 times. The network parameters are initialized to kaiming distribution. The network activation functions include LReLU and PReLU.
[0074] Table 1 Hardware platform used in the image stitching network model training experiment
[0075]
[0076] Table 2 Training parameters
[0077]
[0078]
[0079] Embodiment 3:
[0080] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image stitching method as described in the above embodiment 1 or the image stitching network model training method as described in the above embodiment 2 is implemented.
[0081] Embodiment 4:
[0082] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the image stitching method as described in the above-mentioned embodiment 1 or the training method of the image stitching network model as described in the above-mentioned embodiment 2 is implemented.
[0083] In order to verify the image stitching performance of the image stitching method based on multi-stage feature matching described in Example 1 of the present invention, the image stitching method based on multi-stage feature matching of the present invention and three representative image stitching algorithms are used to stitch images under different light levels, different overlap rates and parallaxes on the UDIS-D dataset. The stitching effects are shown in the figure below. Figure 7 and Figure 8 As shown in the figure (In the figure, Ours represents the stitching image obtained by stitching using the image stitching method based on multi-stage feature matching of the present invention). Among them, three representative image stitching algorithms are: the traditional image stitching algorithm SIFT+RANSAC, the viewless image stitching network VFISNet with a complete deep learning framework, and the image stitching algorithm LEPIS that integrates multi-scale features.
[0084] Figure 7 and Figure 8 The red box in the fourth column shows the problems of images stitched by other methods. The traditional image stitching method SIFT+RANSAC matches and stitches based on the detected interest points. It is only suitable for images with low feature points and small baseline levels, and the stitching effect is poor for complex and large parallax images. VFISNet benefits from the powerful feature extraction and structure reconstruction capabilities of convolutional neural networks. This method has a robust stitching effect between images in different scenes ( Figure 7 and Figure 8 However, the VFISNet method does not process the image stitching process, which results in fuzzy content and semantic confusion at the seams of the stitched images ( Figure 7 and Figure 8 The LEPIS method uses a pyramid framework structure to fuse the high-scale feature map detail texture and the low-scale feature map global semantic contour, thereby reducing the ghosting at the seams; however, this method still does not pay attention to the image stitching process, so Figure 7 , 8 There are still problems such as pixel blur and semantic confusion in the red box. Figure 7 , Figure 8 It can be seen from the last column that the stitched image obtained by the image stitching method based on multi-stage feature matching of the present invention has correct semantic structure at the seam and natural texture transition, which is almost the same as the real image. This is because the present invention uses the powerful nonlinear contour extraction capability of DCN in the feature matching stage to capture the contour features of the same object between the stitched images, and then fuses the semantic information of the multi-scale feature map. In addition, the symmetric dynamic fusion module is used in the stitching stage to make the network pay more attention to the stitching process, and the GAN discriminator is used in the training process to ensure the authenticity of the stitched image.
[0085] In addition, the present invention also uses three evaluation indicators to evaluate the difference between the stitching image obtained by the stitching method of the present invention, the traditional image stitching algorithm SIFT+RANSAC, the viewless image stitching network VFISNet and the image stitching algorithm LEPIS that integrates multi-scale features and the real image. The three evaluation indicators are L1 distance, structural similarity index SSIM and peak signal-to-noise ratio PSNR, where the smaller the L1 distance and the larger the SSIM and PSNR values, the better the stitching result. The results are shown in Table 3.
[0086] Table 3 Comparison results of L1 distance, SSIM, and PSNR between the image stitching method of the present invention and the existing image stitching method
[0087]
[0088] As shown in Table 3, the L1 distance of the image stitching method of the present invention is as low as 2.579%, the PSNR value is 0.762 higher than that of LEPIS, and the SSIM is 0.009 higher than that of LEPIS. Therefore, the stitching performance of the image stitching method of the present invention is better than the above-mentioned comparison algorithms.
[0089] Finally, it should be noted that the above embodiments are only preferred embodiments of the present invention, and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the above technical content as inspiration to make changes or modifications. This is an equivalent embodiment of equivalent changes. However, any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical concept of the present invention still fall within the scope of protection of the claims of the present invention.
Claims
1. An image stitching method based on multi-stage feature matching, It is characterized in that The reference image R and the target image T to be stitched are input into a pre-trained image stitching network model, and the image stitching network model is used for stitching processing to obtain and output the final stitching image of the reference image R and the target image T; wherein the image stitching network model is composed of a stitching network and an optimization network; the stitching network is used to match the nonlinear contour features of the same object between the reference image R and the target image T to be stitched, and extract the homography relationship between the reference image R and the target image T to be stitched, and use the homography relationship to distort and deform the reference image R and the target image T to be stitched and perform dynamic fusion, the input of the stitching network is the reference image R and the target image T, and the output of the stitching network is the primary stitching image of the reference image R and the target image T to be stitched; the optimization network is used to optimize the detail texture, semantic structure and overall quality of the primary stitching image to obtain the final stitching image, the input of the optimization network is the initial stitching image, and the output of the optimization network is the final stitching image.
2. The image stitching method according to claim 1, It is characterized in that The splicing network includes a conversion module, a first convolution module, a second convolution module, a first feature matching module, a second feature matching module, a third feature matching module, a first symmetric dynamic fusion module, a second symmetric dynamic fusion module, and a third symmetric dynamic fusion module; the splicing network includes four stages for the splicing process of the reference image R and the target image T. The first stage is: using the conversion module to convert the input reference image R and the target image T, and converting the reference image R into three reference images R with different resolutions. 1 , reference image R 2 and the reference image R 3 And output, convert the target image T into three target images T with different resolutions 1 , target image T 2 , target image T 3 The second stage is: using the first feature matching module to match the target image T 3 With the reference image R 3 Perform feature matching to obtain the offset feature map T' 3 , offset feature map R' 3 And output, using the first symmetric dynamic fusion module to offset the feature map T' 3 , offset feature map E' 3 Perform homography prediction, distortion, and splicing and fusion processing in sequence to obtain a first fused feature map and output it, and then combine the first fused feature map with the target image T 2 After channel addition, we get feature map; the third stage is: using the first convolution module to the reference image R 2 Convolution processing is performed, and then the second feature matching module is used to Feature map and the reference image R processed by the first convolution module 2 Perform feature matching to obtain the offset feature map T' 2 , offset feature map R' 2 And output, use the second symmetric dynamic fusion module to offset the feature map T' 2 , offset feature map E' 2 Perform homography prediction, distortion, and splicing and fusion processing in sequence to obtain a second fused feature map and output it, and then combine the second fused feature map with the target image T 1 After channel addition, we get feature map; the fourth stage is: using the second convolution module to the reference image R 1 Convolution processing is performed, and then the third feature matching module is used to Feature map and the reference image R processed by the second convolution module 1 Perform feature matching to obtain the offset feature map T' 1 , offset feature map R' 1 And output, using the third symmetric dynamic fusion module to offset the feature map T' 1 , offset feature map R' 1 Perform homography prediction, distortion, and splicing and fusion processing in sequence to obtain the primary splicing image S F And output.
3. The image stitching method according to claim 2, It is characterized in that The network structures of the first feature matching module, the second feature matching module and the third feature matching module are the same. The first feature matching module, the second feature matching module and the third feature matching module all include a channel fusion layer, a first convolution layer, a second convolution layer, a first deformable convolution layer DCN and a second deformable convolution layer DCN. The first feature matching module, the second feature matching module and the third feature matching module perform the same image matching process. The first feature matching module processes the target image T 3 With the reference image R 3 The specific process of feature matching is as follows: 3 With the reference image R 3 After the channel fusion layer performs channel fusion, the first convolution layer is used to perform convolution processing to obtain the offset The offset and the reference image R 3 Input the first variable convolution layer DCN for processing to obtain the reference image R 3 Relative target image T 3 The offset feature map R' 3 And output; offset feature map R' 3 After convolution processing in the second convolution layer, the offset is obtained The offset With offset Perform element-wise multiplication to obtain the total offset, and multiply the total offset by the target image T 3 Input into the second deformable convolutional layer DCN for processing to obtain the target image T 3 Relative reference image R 3 The offset feature map T' 3 .
4. The image stitching method according to claim 2 or 3, It is characterized in that The network structures of the first symmetric dynamic fusion module, the second symmetric dynamic fusion module, and the third symmetric dynamic fusion module are the same, which are all symmetrical interlaced fusion structures; the first symmetric dynamic fusion module, the second symmetric dynamic fusion module, and the third symmetric dynamic fusion module perform the same process of homography prediction, distortion deformation, and splicing fusion processing on the image; taking the first symmetric dynamic fusion module as an example, it includes the offset feature map R' 3 Processing branch and offset feature map T' 3 Processing branch, offset feature map R' 3 Processing branch and offset feature map T' 3 The processing branches all include direct linear transformation module, spatial transformation network, channel fusion module, convolution module, and Sigmoid activation function module; the offset feature map T' of the first symmetric dynamic fusion module is input 3 , offset feature map R' 3 The offset feature map T' 3 Processing branch, offset feature map R' 3 After the branch processing, channel fusion and convolution processing are performed to obtain the first fused feature map.
5. The image stitching method according to claim 4, It is characterized in that The first symmetric dynamic fusion module inputs the offset feature map T' 3 , offset feature map R' 3 The process of homography prediction, distortion, and splicing and fusion processing is divided into four stages; The first stage is: for the offset feature map R' 3 The processing branch first uses a direct linear transformation module to transform the input offset feature map R' 3 Perform a direct linear transformation to obtain the homography matrix with offset coordinates Then the homography matrix And the offset feature map R' 3 The spatial transformation network is input together for distortion and deformation processing to obtain the feature map W R ; For the offset feature map T' 3 The processing branch first uses a direct linear transformation module to transform the input offset feature map T' 3 Perform a direct linear transformation to obtain the homography matrix with offset coordinates Then the homography matrix and offset feature map T' 3 The spatial transformation network is input together for distortion and deformation processing to obtain the feature map W T ; The second stage is: for the offset feature map R' 3 Processing branch, using channel fusion module to transform feature map W R and feature map W T After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map, and the convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a distribution between (0, 1) to obtain a feature map W. R and feature map W T Relative weight relationship diagram S R ; For the offset feature map T' 3 Processing branch, using channel fusion module to transform feature map W T and feature map W R After splicing along the channel direction, the convolution module is input for convolution processing to obtain a convolution feature map, and the convolution feature map is input into the Sigmoid activation function module. The Sigmoid activation function module maps the data of the convolution feature map to a distribution between (0, 1) to obtain a feature map W. T and feature map W R Relative weight relationship diagram S T ; The third stage is: for the offset feature map R' 3 Processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S R Multiply them to get the feature map R″ 3 , the feature map R″ 3 and feature map feature map W R Splice along the channel direction to get the feature map R″′ 3 , feature map R″′ 3 is the offset feature map R' 3 The final output of the processing branch; for the offset feature map T' 3 Processing branch, the feature map W R and feature map W T After adding the elements, the relative weight relationship diagram S T Multiply them together to get the feature map T″ 3 , the feature map T″ 3 and feature map feature map W T Splice along the channel direction to get the feature map T' 3 ”, feature map T' 3 ” is the offset feature map T' 3 Process the final output of the branch; The fourth stage is: using the channel fusion module to offset the feature map R' 3 Process the feature map R' output by the branch 3 " and offset feature map T' 3 Process the feature map T' output by the branch 3 After splicing along the channel direction, a 1×1 convolution layer is used for convolution processing to obtain the first fusion feature map.
6. A training method for an image stitching network model, It is characterized in that The following steps are involved: S1: Collect two images of different viewpoints and a full viewpoint image of each scene in n scenes, record the two images of different viewpoints as reference image and target image respectively, and there is overlap between the reference image and the target image; the reference image and the target image of each scene form a pair of samples, and the full viewpoint image of each scene is used as the real image of the pair of samples, and a total of n pairs of samples are obtained, and the n pairs of samples are divided into a training sample set and a verification sample set according to proportion; S2: using the training sample set to train the pre-built image stitching network model, optimizing the parameters of the image stitching network model, and obtaining the trained image stitching network model; S3: Use the validation sample set to validate all trained image stitching network models, evaluate the image processing performance of the trained image stitching network models, select the optimal trained image stitching network model, and complete the training of the image stitching network model; Wherein, the image stitching network model is the image stitching network model described in any one of claims 1-5.
7. The training method according to claim 6, It is characterized in that The specific operation of step S2 is: inputting the training samples in the training sample set into the image stitching network model for processing, obtaining and outputting the final stitching image of the reference image R and the target image T in the training samples; The difference loss between the final spliced image and the real image corresponding to the training sample is calculated, and then the final spliced image and the real image corresponding to the training sample are input into the GAN discriminator for comparison to obtain the GAN discriminator loss, and the loss function is constructed using the difference loss and the GAN discriminator loss. According to the constructed loss function, the parameters of the image splicing network model are adjusted using the back propagation algorithm to make the loss function converge, and the trained image splicing network model is obtained; wherein the calculation formula of the loss function is as follows: L=λ 1 ×L rec +λ 2 ×L D In the formula, λ 1 and λ 2 is the weight parameter to balance the two losses, L rec is the difference loss function, L D is the GAN discriminator loss, I i is the pixel value of the real image, J i is the pixel value of the final stitched image generated, N is the total number of pixels, D is the discriminator, I is the real image, and J is the generated image.
8. An electronic device comprising a memory and a processor, It is characterized in that The memory stores a computer program, and when the processor executes the computer program, the image stitching method according to any one of claims 1 to 5, or the training method according to claim 6 or 7 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the image stitching method according to any one of claims 1 to 5 or the training method according to claim 6 or 7 is implemented.
Citation Information
Cited By
Image stitching method and system of camera and image stitching camera
CN120997044A
