An image stitching method, device and storage medium

Abstract

The application discloses an image splicing method and device and a storage medium; the method comprises the following steps: fusing a pair or a group of to-be-spliced images into a fused image through a splicing model; the splicing model comprises a splicing network, a correlation analysis network and a fusion network which are connected in sequence; the splicing network is used for extracting deep semantic features and shallow detail features of the to-be-spliced images; the correlation analysis network can more accurately judge the overlapping relationship and the spatial position between the images by mining the context correlation information between the images; in combination with the dynamic determination of the size of the minimum circumscribed rectangle, the registration deviation is greatly reduced. In the coding stage of the fusion network, the feature expression ability is enhanced by reducing the size of the feature map; in the decoding stage, the size is restored to realize the fine fusion of the deep and shallow layer features. The three networks are cooperated to effectively solve the problems of high calculation complexity, low registration accuracy and poor splicing quality of the traditional method, and meet the application requirements of mobile devices and complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image stitching, and in particular to an image stitching method, device and storage medium. BACKGROUND

[0002] With the rapid development of computer vision, remote sensing monitoring, virtual reality and other fields, the demand for large-scale, high-resolution images is increasingly urgent. As a key means of obtaining such images, image stitching technology can break through the limitations of single image view by fusing multiple images with overlapping regions into a complete panoramic image, providing important support for scene understanding, target detection, geographic information collection and other applications.

[0003] However, existing image stitching methods often have problems such as insufficient registration accuracy, blurred stitching edges, and loss of detail information when facing complex scenes such as dynamic object interference, severe changes in lighting, and many repeated texture regions. Traditional image stitching methods cannot effectively fuse deep semantic information and shallow detail features, inevitably causing loss of high-frequency details, resulting in problems such as blurred edges and reduced texture contrast in the stitched image, affecting image clarity and subsequent analysis and application effects. SUMMARY

[0004] The technical problem solved by the present application is to provide an image stitching method that can effectively fuse deep semantic information and shallow detail features, avoiding the loss of high-frequency details and the resulting problems such as blurred edges and reduced texture contrast in the stitched image, affecting image clarity and subsequent analysis and application effects.

[0005] To solve the above technical problems, one technical solution adopted by the present application is to provide an image stitching method, comprising the steps of:

[0006] fusing pairs or groups of images to be stitched into a fused image through a stitching model; the stitching model includes a stitching network, a correlation analysis network and a fusion network connected in sequence;

[0007] The stitching network receives the images to be stitched, and the stitching network is used to extract deep semantic features and shallow detail features in the images to be stitched to obtain a stitching feature map;

[0008] The correlation analysis network receives the images to be stitched and the stitching feature map, and the correlation analysis network mines contextual association information in the images to be stitched, determines the minimum bounding rectangle size of the images to be stitched, and fuses the features of the images to be stitched and the stitching feature map to obtain a registration feature map;

[0009] The fusion network receives the registration feature map, encodes and correspondingly decodes the registration feature map, reduces the size of the registration feature map in the encoding to increase the feature expression of the registration feature map, increases the size of the registration feature map in the corresponding decoding, fuses deep semantic features and shallow detail features in the registration feature map, and obtains a fused image.

[0010] The application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image stitching method when executing the computer program.

[0011] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the image stitching method.

[0012] The application has the following beneficial effects: in the application, the deep semantic features (such as target contour and scene structure) and the shallow detail features (such as texture and color) of the image to be stitched are extracted by the stitching network, which provides a rich feature basis for subsequent fusion and effectively avoids the missing of stitching information caused by single features. The correlation analysis network can more accurately determine the overlapping relationship and spatial position between images by mining the context correlation information between images, and in combination with the dynamic determination of the size of the minimum bounding rectangle, the registration deviation is greatly reduced. In the encoding stage of the fusion network, the feature map size is reduced to enhance the feature expression ability, and in the decoding stage, the size is restored to realize the fine fusion of deep and shallow features, which not only ensures the global coherence of the stitched image, but also retains the clarity of local details, and significantly improves the transition naturalness of the stitching edge. The three networks cooperatively realize the end-to-end image stitching process of “efficient feature extraction-accurate geometric registration-seamless fusion optimization”, effectively solve the problems of high computational complexity, low registration accuracy and poor stitching quality of traditional methods, and adapt to the application requirements of mobile devices and complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart according to an embodiment of the application;

[0014] Figure 2 is a structural schematic diagram according to an embodiment of the application;

[0015] Figure 3 is a structural schematic diagram of a stitching network according to an embodiment of the application;

[0016] Figure 4 is a structural schematic diagram of a first structural residual block according to an embodiment of the application;

[0017] Figure 5is a structural schematic diagram of a residual block according to an embodiment of the present application;

[0018] Figure 6 is a structural schematic diagram of a correlation analysis network according to an embodiment of the present application;

[0019] Figure 7 is a structural schematic diagram of a fusion network according to an embodiment of the present application;

[0020] Figure 8 is a fusion result schematic diagram according to an embodiment of the present application;

[0021] Figure 9 is another fusion result schematic diagram according to an embodiment of the present application;

[0022] Figure 10 is another fusion result schematic diagram according to an embodiment of the present application;

[0023] Figure 11 is another fusion result schematic diagram according to an embodiment of the present application;

[0024] Figure 12 is another fusion result schematic diagram according to an embodiment of the present application;

[0025] Figure 13 is another fusion result schematic diagram according to an embodiment of the present application;

[0026] Fig. 1 is a structural schematic diagram of a stitching network according to an embodiment of the present application; Fig. 2 is a structural schematic diagram of a correlation analysis network according to an embodiment of the present application; Fig. 3 is a structural schematic diagram of a fusion network according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to facilitate understanding of the present application, the present application will be described in more detail below in combination with the drawings and specific embodiments. The preferred embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms, and is not limited to the embodiments described in the specification. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0028] It should be noted that, unless otherwise defined, all technical and scientific terms used in the present specification shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the present application solely for the purpose of describing a specific embodiment of the present application, and is not intended to limit the present application. The term "and / or" used in the present specification includes any and all combinations of one or more of the associated listed items.

[0029] Figure 1 and Figure 2 Embodiments of the image stitching method of the present application are shown, including:

[0030] Step S1: fusing a pair or a group of to-be-stitched images into a fused image through a stitching model; the stitching model includes a stitching network 1, a correlation analysis network 2 and a fusion network 3 connected in sequence;

[0031] Step S2: the stitching network 1 receives the to-be-stitched images, and the stitching network 1 is used to extract deep semantic features and shallow detail features in the to-be-stitched images, to obtain a stitching feature map;

[0032] Step S3: the correlation analysis network 2 receives the to-be-stitched images and the stitching feature map, the correlation analysis network 2 mines context-related information in the to-be-stitched images, determines the minimum bounding rectangle size of the to-be-stitched images, and fuses the features of the to-be-stitched images and the stitching feature map, to obtain a registration feature map;

[0033] Step S4: the fusion network 3 receives the registration feature map, the fusion network 3 encodes and correspondingly decodes the registration feature map, reduces the size of the registration feature map in the encoding to increase the feature expression of the registration feature map, increases the size of the registration feature map in the corresponding decoding, and fuses the deep semantic features and the shallow detail features in the registration feature map, to obtain a fused image.

[0034] In the present application, the deep semantic features (such as target contour, scene structure) and shallow detail features (such as texture, color) of the to-be-stitched images are extracted through the stitching network 1, providing a rich feature basis for subsequent fusion and effectively avoiding the missing of stitching information caused by single features. The correlation analysis network 2 can more accurately determine the overlapping relationship and spatial position between images by mining the contextual correlation information between images, combined with the dynamic determination of the size of the minimum bounding rectangle, which greatly reduces the registration deviation. The encoding stage of the fusion network 3 enhances the feature expression ability by reducing the size of the feature map, and the decoding stage realizes the fine fusion of deep and shallow features by restoring the size, which not only ensures the global coherence of the stitched image, but also retains the clarity of local details, significantly improving the transition naturalness of the stitching edge. The three networks cooperatively realize the end-to-end image stitching process of "efficient feature extraction-accurate geometric registration-seamless fusion optimization", effectively solving the problems of high computational complexity, low registration accuracy and poor stitching quality of traditional methods, and adapting to the application requirements of mobile devices and complex scenes.

[0035] The stitching network 1 can be arranged in pairs or groups, and each stitching network 1 can correspond to processing one to-be-stitched image. The stitching network 1 avoids information loss caused by channel dimension reduction in traditional methods through multi-scale parallel branches and feature grouping strategies. Compared with the single-scale feature extraction scheme, this architecture reduces the computational complexity by about 35% while maintaining the feature expression ability, and is more suitable for lightweight training requirements in unsupervised learning where data labeling is missing.

[0036] In some embodiments, as shown in Figure 3 The stitching network 1 includes a stitching preprocessing module 11, a plurality of stitching feature extraction modules 12, an attention processing module 13 and a plurality of sampling modules 14 connected in sequence, the stitching preprocessing module 11 is used to reduce the size of the to-be-stitched image and expand the channel number of the to-be-stitched image, to generate a first feature map, the stitching feature extraction module 12 is used to extract features in the first feature map to generate a plurality of intermediate feature maps with different sizes and channel numbers; the attention processing module 13 is used to capture the global dependency relationship between channels in the intermediate feature map to generate an attention feature map, and the sampling module 14 is used to perform up-sampling processing on the attention feature map to generate a plurality of sampling feature maps, and fuse the up-sampling feature map with the intermediate feature map corresponding to its channel number to generate a stitching feature map.

[0037] In some embodiments, after the to-be-stitched image is input into the stitching network 1, the stitching preprocessing module 11 first performs preliminary processing on the to-be-stitched image, the stitching preprocessing module 11 includes a convolution layer and a max-pooling layer, and the size of the to-be-stitched image is reduced and the channel number of the to-be-stitched image is expanded through the convolution layer and the max-pooling layer to generate a first feature map.

[0038] Specifically, a to-be-stitched image with a size of 512x512 pixels is input into the stitching network 1. The to-be-stitched image is first subjected to preliminary processing by a 3x3 ordinary convolution layer and a maximum pooling layer, which compresses the size of the to-be-stitched image to 1 / 2 of the original size and expands the number of channels to 16, thereby obtaining a first feature map with a size and a channel represented as (1 / 2, 16). The stitching preprocessing module 11 can effectively reduce the spatial dimension of the to-be-stitched image, greatly reduce the subsequent calculation amount, and extract basic image features to lay the foundation for deep feature extraction. The stitching preprocessing module 11 automatically down-samples the original high-resolution image (such as 4K or 8K) to a standard size of 512x512 pixels in proportion and records the scaling coefficient of the image length and width (such as 1 / 8 or 1 / 16). This operation significantly reduces the calculation complexity of feature extraction and network inference, enabling the lightweight model to reduce the memory occupancy by 90% while increasing the inference speed by 5-8 times.

[0039] In some embodiments, as shown in Figure 3 After the stitching preprocessing module 11 generates the first feature map, the first feature map is input into the stitching feature extraction module 12, and the stitching feature extraction module 12 extracts features in the first feature map to generate a plurality of intermediate feature maps with different sizes and channel numbers.

[0040] In some embodiments, the stitching feature extraction module 12 includes four stitching feature extraction modules 12, each of which outputs an intermediate feature map with different sizes and channel numbers. The four intermediate feature maps output by the four stitching feature extraction modules 12 can be a first intermediate sub-feature map, a second intermediate sub-feature map, a third intermediate sub-feature map, and a fourth intermediate sub-feature map. The first feature map is input into the first stitching feature extraction module 12, the output of the first stitching feature extraction module 12 is the first intermediate sub-feature map, the first intermediate sub-feature map is input into the second stitching feature extraction module 12, the output of the second stitching feature extraction module 12 is the second intermediate sub-feature map, the second intermediate sub-feature map is input into the third stitching feature extraction module 12, the output of the third stitching feature extraction module 12 is the third intermediate sub-feature map, the third intermediate sub-feature map is input into the fourth stitching feature extraction module 12, and the output of the fourth stitching feature extraction module 12 is the fourth intermediate sub-feature map. The corresponding sizes and channel numbers of the first intermediate sub-feature map, the second intermediate sub-feature map, the third intermediate sub-feature map, and the fourth intermediate sub-feature map are (1 / 4, 32), (1 / 8, 64), (1 / 16, 128), and (1 / 32, 256), respectively.

[0041] In some embodiments, as shown in Figure 4As shown, the splicing feature extraction module 12 includes at least two residual blocks and a maximum pooling layer connected in sequence, and the residual block includes an expansion convolution unit 121, a depth separable convolution unit 122, an attention unit 123, an adjustment convolution unit 124, an addition unit 125 and an activation unit 126 connected in sequence. The expansion convolution unit 121 is used to expand the number of channels to widen the feature expression space; the depth separable convolution unit 122 is used to extract spatial and channel features, the attention unit 123 is used to select the local optimal coverage range of cross-channel interaction, capture the key dependency relationship between channels, and focus on discriminative features; the adjustment convolution unit 124 is used to adjust the number of channels; the addition unit 125 is used to fuse the features, and the activation unit 126 is a rectified linear unit (Relu) function.

[0042] In some embodiments, as Figure 5 As shown, the expansion convolution unit 121 includes a convolution layer (Conv) 1x1, a normalization (BN) layer and a rectified linear unit (Relu) function arranged in sequence, and the expansion convolution unit 121 is used to expand the number of channels of the input image to twice to widen the feature expression space.

[0043] The depth separable convolution unit 122 includes two depth shift convolution (DSconv) layers, and the depth separable convolution unit 122 is used to extract spatial and channel features. The depth separable convolution unit 122 is introduced in the residual block, and this lightweight convolution structure significantly reduces the parameter quantity and calculation cost through the decoupling operation of the spatial and channel dimensions while maintaining the feature extraction capability, thereby releasing more calculation resources for subsequent network layers. The calculation method of the parameter quantity of the ordinary convolution layer is as follows: assuming that the input channel number is Cin, the output channel number is Cout, and the convolution kernel size is KxK, the parameter quantity is CinxCoutxKxK; and the depth separable convolution unit 122 decomposes the standard convolution operation into spatial dimension convolution (channel-by-channel convolution) and channel dimension convolution (point-by-point convolution), and the parameter quantity is CinxKxK+CinxCout. Taking Cin=32, Cout=64 and K=3 as an example, the parameter quantity of the ordinary convolution is 32x64x3x3=18432, the parameter quantity of the depth separable convolution unit 122 is 32x3x3+32x64=2304, the parameter quantity is reduced by about 87.5%, the calculation cost is significantly reduced, the calculation efficiency is improved, and the demand for lightweight models in the unsupervised learning scenario is met.

[0044] In some embodiments, the attention unit 123 is based on efficient channel attention (ECA), which adaptively selects the locally optimal coverage range of cross-channel interaction, accurately captures the key dependency between channels, and focuses on discriminative features. The attention unit 123 avoids the dimension reduction loss of traditional attention, dynamically weights the channel features at a minimal computational cost (only 0.1% increase in parameter quantity), focuses on discriminative features (such as edges, corner points, and other key features), and improves the robustness of feature extraction in an unsupervised scene. Thus, the residual block can focus on the extraction and strengthening of discriminative features by adaptively selecting the locally optimal coverage range of cross-channel interaction and accurately capturing the key dependency between channels.

[0045] In some embodiments, the adjustment convolution unit 124 includes a convolution layer (Conv) 1x1 and a normalization (BN) layer, and is used to adjust the channel number of the image to a suitable dimension.

[0046] In some embodiments, the addition unit 125 is used to realize feature fusion and enhance the learning ability of the network.

[0047] Taking the residual block in the first concatenation feature extraction module 12 as an example, the first feature map is input into the first concatenation feature extraction module 12, and the addition unit 125 adds and fuses the input first feature map and the first feature map processed by the expansion convolution unit 121, the depth separable convolution unit 122, the attention unit 123, and the adjustment convolution unit 124. The addition result is processed by the rectified linear unit (Relu) function of the activation unit 126, and an activated feature map is output. This structure effectively avoids the gradient disappearance problem while adjusting the channel dimension, and improves the network training stability.

[0048] In some embodiments, the residual block further includes an additional adjustment convolution unit 127, which is used to adjust the channel number of the image output by the previous residual block. The additional adjustment convolution unit 127 includes a convolution layer (Conv) 1x1 and a normalization (BN) layer.

[0049] Taking the residual block in the first concatenation feature extraction module 12 as an example, the addition unit 125 adds and fuses the first feature map processed by the additional adjustment convolution unit 127 and the first feature map processed by the expansion convolution unit 121, the depth separable convolution unit 122, the attention unit 123, and the adjustment convolution unit 124, and the addition result is processed by the rectified linear unit (Relu) function, and an activated feature map is output.

[0050] In some embodiments, the residual blocks without the additional adjustment convolution unit 127 are taken as first structure residual blocks, the residual blocks with the additional adjustment convolution unit 127 are taken as second structure residual blocks, and the number of residual blocks in the four splicing feature extraction modules 12 is respectively 2, 3, 2 and 2. Among these residual blocks, the residual blocks on the front side adopt the first structure residual block, and the last residual block adopts the second structure residual block, thereby effectively avoiding the gradient disappearance problem and improving the network training stability.

[0051] The traditional residual block adopts standard convolution, which has large parameter quantity and redundant calculation. The deep separable convolution unit 122 is introduced in the present application, which decouples the spatial and channel dimensions, reduces the parameter quantity of a single residual block by about 80% compared with the standard convolution, and significantly improves the calculation efficiency. Combined with the efficient channel attention mechanism (only increasing 0.1% parameter quantity), the key dependency relationship between channels is dynamically enhanced without increasing the calculation cost, avoiding the feature representation limitation caused by the fixed channel interaction range of the traditional attention (such as SE). This combination enables the feature extraction module to capture discriminative features (such as edges and corners) at a lower calculation cost in an unsupervised scene, supports deeper network structure design, and improves the model robustness.

[0052] In the above, through the plurality of splicing feature extraction modules 12 in the splicing network 1, a plurality of intermediate feature maps with different sizes and channel numbers can be generated. After generating the intermediate feature maps, the attention processing module 13 in the splicing network 1 processes the intermediate feature maps to obtain the attention feature map.

[0053] In some embodiments, the attention processing module 13 adopts the efficient multi-scale attention (EMA) mechanism, and the size and channel number of the intermediate feature map remain unchanged during the processing of the attention processing module 13.

[0054] In some embodiments, the attention processing module 13 adopts a multi-scale parallel subnetwork design, in which the 1x1 convolution branch focuses on capturing the global dependency relationship between channels, and the 3x3 convolution branch is responsible for capturing local texture information, effectively avoiding the information loss problem caused by channel dimension reduction. At the same time, by cross-dimension interaction, different scale features are dynamically aggregated to form richer semantic representations, automatically mine the corresponding relationship between images under unsupervised conditions, and enhance the feature response in the overlapping area.

[0055] The attention processing module 13 efficiently captures multi-scale spatial semantic features while preserving channel information, and enhances pixel-level context modeling through cross-spatial interaction, effectively improving the stability and accuracy of the model's image feature capture. The attention processing module 13 fuses the output features of parallel branches through a cross-spatial learning method, effectively capturing pixel-level long-range dependencies. Specifically, the attention processing module 13 encodes global spatial information using two-dimensional global average pooling and generates a spatial attention map through cross-dimensional interaction, thereby highlighting the semantic information of key regions (such as overlapping regions). This design enables the model to automatically discover correspondences between images under unsupervised conditions, such as optimizing stitching alignment by enhancing the feature responses of overlapping regions. Furthermore, the attention processing module 13 further improves the uniform distribution of spatial semantic features and reduces noise interference through a feature grouping strategy (dividing the channel dimension into multiple sub-feature groups).

[0056] The stitching network 1 upsamples the attention feature map through multiple sampling modules 14 to generate multiple sampled feature maps. These upsampled feature maps are then fused with the intermediate feature maps corresponding to their respective channel numbers to generate a stitched feature map. This stitched feature map organically combines deep semantic features with shallow detail features, enabling the stitching network 1 to accurately capture image detail and texture information while also accurately grasping the overall structural contour, providing comprehensive and precise feature information support for subsequent image registration.

[0057] In some embodiments, such as Figure 3 As shown, the sampling module 14 includes an upsampling unit, a fusion unit, and a residual block arranged sequentially. The upsampling unit is used to upsample the attention feature map, and the fusion unit is used to fuse the attention feature map with the intermediate feature map to generate a stitched feature map. For example, after the attention feature map is upsampled by the upsampling unit, the size and number of channels of the attention feature map are (1 / 16, 128). The image output by the upsampling unit is input into the fusion unit, and the third intermediate sub-feature map is also input into the fusion unit, with the size and number of channels of the third intermediate sub-feature map also being (1 / 16, 128). The image obtained after the fusion unit fuses the attention feature map and the third intermediate sub-feature map is then input into the first residual unit or the second residual unit, which processes the input image. After processing, the processed image is input into the next sampling module 14. After upsampling by the upsampling unit of the next sampling module 14, an image with a size and number of channels of (1 / 8, 64) is obtained. This image is then fused with the second intermediate sub-feature map, which also has a size and number of channels of (1 / 8, 64). Thus, a stitched feature map with a size and number of channels of (1 / 8, 64) is obtained.

[0058] Single scale features cannot balance between detail information (high resolution features) and semantic information (low resolution features), directly using deep features may lose key registration details such as edges, while shallow features lack global structure information. In this application, the deep semantic features (structure outline) and shallow detail features (texture edge) are spliced and fused by the sampling module 14 to organically integrate the features of the image at different resolutions, generating a spliced feature map containing multi-scale information. Solving the problem of "detail loss" or "semantic ambiguity" of traditional single scale features. After fusing multi-scale features, the feature matching accuracy is improved by 22% compared with single scale scheme.

[0059] The spliced feature map contains both the texture details (such as pixel-level color differences) and the structural semantics (such as regional shape features) of the image, which enables the model to capture both the detailed texture details of the image and the overall structural semantics. Provide more comprehensive input for subsequent context association analysis; The sampling module 14 adapts to scenes with variable object sizes in unsupervised image stitching, and improves the generalization ability of matching different size objects through multi-scale feature matching.

[0060] In some embodiments, the spliced feature map and the images to be spliced (including the image to be spliced a and the image to be spliced b) are input into the correlation analysis network 2, and the correlation analysis network 2 processes the spliced feature map and the image to be spliced to obtain the registration feature map (including the registration feature map a and the registration feature map b) and the mask (including the mask a and the mask b).

[0061] In some embodiments, as shown in Figure 6 The correlation analysis network 2 includes a context correlation module 21, a related feature extraction module 22, a full connection module 23, a control point offset module 24, a direct linear transformation module 25 and a stitching domain transformation module 26 connected in turn. The context correlation module 21 accurately captures the spatial correspondence clues in the spliced feature map based on the directional feature interaction mechanism by separating the feature dependency relationship in the horizontal and vertical directions. The related feature extraction module 22 is used to extract the features in the spliced feature map, the full connection module 23 is used to adjust the feature vector of the spliced feature map, the control point offset module 24 is used to control the offset of the pixel points in the spliced feature map, the direct linear transformation module 25 is used to transform the spliced feature map and generate a warped feature map, and the warped feature map is fused with one of the images to be spliced. Through the stitching domain transformation module 26, the other image to be spliced is spliced to generate the registration feature map and the mask.

[0062] In some embodiments, as shown in Figure 6 The context correlation module 21 includes a cluster center loss (CCL) layer and a convolution layer, and the cluster center loss layer performs correlation calculation in the horizontal and vertical directions on the input spliced feature map:

[0063] Horizontal feature flow generation: compute dot product similarity of feature vectors along row dimension, generate feature flow matrix describing horizontal displacement trend;

[0064] Vertical feature flow generation: compute cosine similarity of feature vectors along column dimension, generate feature flow matrix describing vertical displacement trend.

[0065] The concatenated feature map is input into the context correlation module 21, and the size and channel number of the feature flow matrix output by the cluster center loss layer in the context correlation module 21 are (1 / 8, 2). The feature flow matrix output by the cluster center loss layer is expanded to a first correlation feature map with a size and channel number of (1 / 8, 64) through a 1x1 convolution layer.

[0066] The core of image registration is the spatial correspondence at the pixel level, and the displacement in the horizontal / vertical direction is a basic component of geometric transformation. The feature interaction method in the prior art (such as point-by-point calculation) does not show the separation direction dependence, and it is difficult to capture long-distance context association (such as row and column feature matching across image boundaries). In this application, the context correlation module 21 is used to deeply mine the context association information between image pairs. The context correlation module 21 can analyze the feature interaction relationship of the image pair in the horizontal and vertical directions in detail, generate horizontal and vertical feature flows by calculating the feature correlation in the row / column dimension, and thus accurately describe the displacement trend of the image pair in the horizontal / vertical direction, providing structured context clues for subsequent offset regression; reduce the feature mismatch problem caused by complex geometric transformation in unsupervised learning, especially for image pairs with rotation and scaling, directional feature flow can enhance the modeling of the corresponding relationship of local regions. The context correlation module 21 shows and analyzes the horizontal / vertical direction feature interaction, and generates structured directional feature flow. Compared with the traditional point-by-point feature interaction method, the registration error of this mechanism for rotation and scaling scenes is reduced by 18%, and it is particularly advantageous in modeling long-distance context association across image boundaries (such as accurately capturing the horizontal displacement distribution of a row of pixels in the left image in the right image).

[0067] In some embodiments, as Figure 6As shown, the relevant feature extraction module 22 includes three residual blocks and a maximum pooling layer connected in sequence. The residual blocks have the same structure as the residual blocks in the above-mentioned splicing feature extraction module 12. The first relevant feature map is input into the relevant feature extraction module 22, and the relevant feature extraction module 22 gradually compresses the size of the input first relevant feature map to 1 / 16, 1 / 32 and 1 / 64, and expands the channel number to 128, 256 and 512. For example, the relevant feature extraction module 22 expands the channel number from 64 to 128 through 1x1 convolution, strengthens the discriminative features through the depth separable convolution unit and the attention unit, and then keeps the channel number unchanged through 1x1 convolution, and cooperates with the residual connection to avoid gradient disappearance; the maximum pooling layer performs 2x2 kernel downsampling, so that the size of the first relevant feature map is compressed from 1 / 8 to 1 / 16.

[0068] In some embodiments, global average pooling is performed on the image output by the relevant feature extraction module 22, the spatial dimension is compressed to 1x1, and a 512-dimensional feature vector is obtained. The vector is sequentially passed through a fully connected (FC) module 23 and a control point offset module 24, and the control point offset of a 12x12 grid is regressed to generate a second relevant feature map. In the fully connected module 23, the change of the 512-dimensional feature vector is: 512→256→288 (12x12x2), wherein the hidden layer adopts a rectified linear unit (Relu) function activation. In the control point offset module 24, the predicted offset (Δx, Δy) of each grid vertex indicates the displacement (unit: pixel) of the point in the original image coordinate system relative to the reference image.

[0069] The present application divides the image into a plurality of grid units. In this way, the image is decomposed into local regions, each region independently regresses the offset, supporting more flexible local geometric transformation modeling. The local control mechanism enables the model to adapt to complex scenes (such as multi-plane splicing and curved surface images), each grid can independently learn the transformation parameters such as scaling, rotation and translation, and the registration accuracy is improved; the non-linear mapping capability of the fully connected layer captures the complex relationship between the control point offset and the features (such as the offset in the texture dense area needs to be adjusted more finely), avoiding the dependence of traditional methods (such as manual feature matching) on specific scenes. Thus, the offset of each control point is accurately predicted. In this way, accurate registration of the image is realized.

[0070] In some embodiments, in the direct linear transformation module 25, for each 12x12 grid (a total of 13x13 control points) in the second relevant feature map, a direct linear transformation (DLT) is used to calculate the local homography matrix H from the offset. The mathematical expression of the homography transformation is:

[0071] ;

[0072] in, The coordinates of the source image. The coordinates of the target image; The elements are in the local homography matrix H; the direct linear transformation module solves the following overdetermined system of equations by least squares:

[0073] ;

[0074] Transform into matrix form . The optimal solution is obtained by finding the eigenvector corresponding to the minimum singular value through singular value decomposition, where T represents the transpose. The distorted feature map can then be obtained through the direct linear transformation module 25.

[0075] In existing technologies, the homography matrix is ​​a standard mathematical tool for describing planar transformations. Direct linear transformation methods ensure geometric consistency of local transformations through least-squares fitting. However, direct stitching may lead to redundancy or missing image boundaries, requiring dynamic adjustment of the output size using a minimum bounding rectangle. In this application, the direct linear transformation module 25 accurately models the geometric transformations (such as rotation angles and scaling ratios) of each grid using a local homography matrix, significantly improving registration accuracy for non-planar scenes compared to global transformations. The minimum bounding rectangle mechanism automatically adapts to the actual size of the stitched image, avoiding information cropping or invalid filling caused by fixed-size output in traditional methods. This achieves accurate registration of the entire image, ensuring high alignment of the image in geometric position and structure. This enhances the practicality and flexibility of unsupervised stitching.

[0076] Traditional global homography transformation assumes the image is a rigid planar body, which cannot handle multi-plane or non-rigid deformations (such as curved surfaces). This application decomposes the image into independent local regions through mesh generation and local homography transformation. Each mesh regresses the control point offset (Δx, Δy) through a fully connected layer, supporting flexible transformations such as scaling, rotation, and translation. The direct linear transformation module 25 scales the predicted offset results proportionally according to the pre-recorded scaling ratio (for example, if the scaling ratio is 1 / 8, the Δx and Δy coordinate values ​​are multiplied by 8), thereby accurately mapping the registration parameters from the low-resolution space back to the original high-resolution image space. This strategy ensures that the model effectively captures both global structure and local details, and through a resolution-layered processing mechanism, achieves fast and low-resource-consumption registration of high-resolution images without sacrificing stitching accuracy, successfully solving the performance bottleneck of traditional methods when processing large-size images. In multi-plane stitching scenarios, the registration error (peak signal-to-noise ratio PSNR, structural similarity SSIM) of this application is reduced by 31% and 25% respectively compared with the global homography method, significantly improving the geometric alignment accuracy in complex scenarios.

[0077] In some embodiments, the warped feature map is obtained by the direct linear transformation module 25, and after being fused with a to-be-stitched image, the to-be-stitched image is input into the stitching domain transformation module 26, and after being processed by the stitching domain transformation module 26, the stitching domain transformation module 26 outputs a pair of registered feature maps and a corresponding mask.

[0078] In some embodiments, the stitching domain transformation module 26 is used to calculate the minimum bounding rectangle of the to-be-stitched image, and determine the size and boundary of the to-be-stitched image. The specific steps are as follows: applying the homography matrix Hi of each grid, calculating the coordinates of the four corner points of the grid in the stitching coordinate system; collecting all the corner point coordinates, solving the extreme values of the horizontal and vertical axes (Xmin, Xmax, Ymin, Ymax), and determining the minimum bounding rectangle; mapping each grid region to the minimum bounding rectangle coordinate system through bilinear interpolation, and completing the whole image registration. After being processed by the stitching domain transformation module 26, the stitching domain transformation module 26 outputs a pair of registered feature maps and a corresponding mask. The minimum bounding rectangle boundary is automatically calculated by the stitching domain transformation module 26 layer, which avoids the information clipping or invalid padding caused by the traditional fixed size output, and adapts to the flexible stitching requirements in multiple scenes such as video monitoring, unmanned aerial vehicle inspection, medical microscopy, etc.

[0079] The stitching network is trained by the following loss function. In order to ensure that the matching error in the overlapping area is minimized while constraining the geometric deformation energy in the non-overlapping area. The objective function L of the combination of pixel consistency loss L1 and grid deformation loss L2 is adopted:

[0080] ;

[0081] Wherein, , are the weight coefficients corresponding to different losses, which can be 1 and 5 respectively.

[0082] The pixel consistency loss is expressed as:

[0083] ;

[0084] Wherein, i and j represent the pixel coordinates of the overlapping area, represents the overlapping area of the registered feature map, represents the warping operation, represents the target image, represents the reference image, and N represents the total number of pixels in the overlapping area of the registered feature map. During image registration, one of the images remains unchanged, and the other is warped and registered. The image that remains unchanged is called the reference image, and the image that is used for warping is called the target image.

[0085] The grid deformation loss is expressed as:

[0086] ;

[0087] wherein, and denote the number of horizontal and vertical edges, respectively, 12, 12, and denote the width and height of the registration feature map, respectively, and denote the set of horizontal and vertical edges, respectively, denotes an edge vector in the set of horizontal or vertical edge vectors, and denote the horizontal and vertical unit vectors, respectively.

[0088] denotes a rectified linear unit (Relu) function, P denotes the total number of adjacent edges, is a 0-1 mask, and the non-overlapping region is set to 1, and denote adjacent edges.

[0089] In some embodiments, as shown in Figure 7 , a pair of registration feature maps is input into a fusion network 3, which includes an initial processing module 31, a plurality of encoding modules 32 and a plurality of decoding modules 33 corresponding to the encoding modules 32 connected in turn. The initial processing module 31 is used to fuse the registration feature maps, and the encoding modules 32 are used to halve the size of the registration feature maps to enhance the feature expression of the registration feature maps. The decoding modules 33 are used to increase the size of the registration feature maps in the corresponding decoding and fuse the deep semantic features and shallow detail features in the registration feature maps to obtain a fused image.

[0090] In some embodiments, as shown in Figure 7 , the initial processing module 31 includes a fusion unit and two residual blocks, and the fusion unit fuses the features in a pair of registration feature maps. The registration feature maps are spliced in the channel dimension (such as 512x512x3→512x512x6). The first residual block is a second residual unit, and the second residual block is a first residual unit. The first residual block expands the channel number from 6 to 12 through 1x1 convolution (channel expansion), and further enhances the feature expression through two depth separable convolution units and ECA attention mechanism, and then upgrades to 16 through 1x1 convolution, and outputs after adding the registration feature map (1 / 1, 16). The second residual block keeps the channel number 16 unchanged, and further enhances the feature expression.

[0091] In some embodiments, as shown in Figure 7As shown, the encoding module 32 is provided with four, the encoding module 32 includes a max pooling layer and two residual blocks, the max pooling layer is a 2x2 kernel, used to reduce the size of the registered feature map by half. The first residual block is a second structure residual block, and the second residual block is a first structure residual block. The first residual block expands the channel number from 6 to 12 through 1x1 convolution (channel expansion), strengthens the feature expression through two depth separable convolution units and ECA attention mechanism, and then upgrades to 16 through 1x1 convolution. After adding the registered feature map, the output is (1 / 1, 16). The second residual block keeps the channel number 16 unchanged, and further enhances the feature expression.

[0092] The size and channel number of the input and output of the four encoding modules 32 are shown in Table 1:

[0093] Table 1 Size and channel number of input and output of encoding module

[0094] Encoding module Input size Output size Channel number change 1 1 / 1，16 1 / 2，32 16→32→32 2 1 / 2，32 1 / 4，64 32→64→64 3 1 / 4，64 1 / 8，128 64→128→128 4 1 / 8，128 1 / 16，256 128→256→256

[0095] The four encoding modules 32 realize multi-scale feature capture through stepwise dimension reduction and channel expansion. For example, a 256-channel registered feature map of size 1 / 16 can encode global semantic information (such as object contour), and a 32-channel registered feature map of size 1 / 2 retains local details (such as texture edges).

[0096] Unsupervised learning relies on lightweight models to reduce training difficulty, and traditional convolutional computation has high complexity and is prone to local optimization; image fusion needs to retain both local details (such as pixel-level texture) and global semantics (such as object contour), and needs to capture multi-granularity information through hierarchical feature extraction. Therefore, in the fusion network 3, the down-sampling path is used as the encoding module 32, and the depth separable convolution unit is used for multi-scale feature extraction. This lightweight convolution operation can effectively capture local structural features of the image while reducing computational complexity; then a pooling layer is connected to realize stepwise reduction of spatial resolution, gradually stripping redundant detail information, and finally forming a high-level feature representation containing core semantic information of the registered feature map.

[0097] In some embodiments, as shown in Figure 7 The decoding module 33 includes an up-sampling layer, a fusion layer and two residual blocks connected in turn, the up-sampling layer performs bilinear interpolation, and its magnification is 2 times. Among the two residual blocks, the first residual block is a second structure residual block, which reduces the channel number, and the second residual block is a first structure residual block, which keeps the channel number unchanged. The number of decoding modules 33 corresponds to the number of encoding modules 32, and the last decoding module 33 further includes a convolution layer.

[0098] The traditional deconvolution (transpose convolution) is prone to chessboard effect in unsupervised scenarios, and bilinear interpolation provides smoother up-sampling basis; the details information (such as color gradient) contained in the shallow features need to be gradually recovered through lightweight convolution operations. In the fusion network 3, the up-sampling path is taken as the decoding module 33, and the combination of bilinear interpolation up-sampling and depth separable convolution unit is adopted to realize the layer-by-layer recovery of the spatial dimension. Bilinear interpolation avoids the artifact problem of deconvolution, while supplementing high-frequency details with lightweight operations (reducing the calculation by 40%), so that the pixel-level semantic consistency of the reconstructed overlapping area is improved by 19%. The depth separable convolution unit recovers the size while supplementing the edge, texture and other high-frequency details at low computational cost. The up-sampling operation enlarges the feature map size in proportion, providing a spatial basis for detail reconstruction, and the subsequent depth separable convolution unit performs fine processing on the enlarged feature map to supplement high-frequency detail information.

[0099] The size and channel number of the input and output of the four decoding modules 33 are shown in Table 2:

[0100] Table 2 Size and channel number of input and output of decoding module

[0101] Decoding module Input size Skip connection size Output size Channel number change 1 1 / 16，256 1 / 8，128 1 / 8，128 256→384→128 2 1 / 8，128 1 / 4，64 1 / 4，64 128→192→64 3 1 / 4，64 1 / 2，32 1 / 2，32 64→96→32 4 1 / 2，32 1 / 1，16 1 / 1，16 32→48→16→3

[0102] In this application, the up-sampling layer replaces deconvolution (O(N2)) with bilinear interpolation to eliminate the common chessboard artifacts in unsupervised learning (traditional deconvolution produces jagged artifacts at the edges, while bilinear interpolation maintains smooth transitions).

[0103] In some embodiments, the decoding module 33 adopts a symmetrical structure, gradually recovering the size of the registered feature map through up-sampling and cross-layer jump connection. The decoding module 33 is connected to the encoding module 32 through a jump connection, i.e., the registered feature map of the same size in the encoding stage is connected by channel number, for example, when the size of the decoding is 1 / 8 layer, the size of the encoding output is 1 / 8 layer. The jump connection concatenates and fuses the registered feature maps of the corresponding levels by channel, and the shallow features of the encoding module (high resolution, containing detail information) and the deep features of the decoding module (low resolution, containing semantic information) are concatenated by channel dimension, and the fused feature map is input to the subsequent decoding layer for joint processing.

[0104] In the traditional coding and decoding structure, deep semantic features lack the support of shallow details during decoding, resulting in problems such as edge blur and texture loss; in unsupervised learning, there is a lack of real label supervision, and the feature representation capability needs to be enhanced through cross-layer information complementation. In the present application, the decoding module and the encoding module are connected through a jump connection, the shallow detail features (such as color gradient) of the corresponding level in the encoding module are spliced and fused with the deep semantic features (such as object boundary) of the decoding module to form a composite feature representation containing multi-scale information. This feature fusion strategy effectively solves the problem of loss of detail information in the traditional coding and decoding structure, and significantly enhances the model's ability to recover image edges, textures and other details.

[0105] In some embodiments, the fusion network finally reconstructs the overlapping area for the registered feature map pair. Through the reconstruction result of the generated registered feature map after decoding, which is consistent with the size of the input image to be spliced, the pixel details and semantic consistency of the overlapping area are accurately restored. For non-overlapping areas, the original image area after registration is directly used as the reserved part to avoid errors introduced by the reconstruction process. Finally, through a pixel-level fusion operation, the reserved non-overlapping area and the reconstructed overlapping area are seamlessly spliced to generate a high-quality fusion image with uniform color space and geometric consistency. This method effectively improves the detail fidelity and visual fluency of image fusion while ensuring semantic consistency.

[0106] The overlapping area has multiple image pixel value conflicts (such as exposure difference and viewing role difference), and needs to be reconstructed by the network to achieve semantic consistency; the original pixels of the non-overlapping area already have geometric consistency, and direct reservation can avoid error accumulation introduced by reconstruction.

[0107] In the present application, the overlapping area reconstruction result is generated through the complete size registered feature map after decoding (same size as the input image), that is, a 1x1 convolution layer is used to output a reconstruction image (containing RGB three channels) with the same resolution as the input image, and the reconstruction target focuses on the overlapping area of the registered image pair (located through a mask matrix); the non-overlapping area directly uses the original image area after registration: the determined coordinate mapping relationship is used to extract the non-overlapping original image block; the reconstructed overlapping area and the original non-overlapping area are combined to form the final fusion image.

[0108] Specifically, the fusion network reconstructs the overlapping area and reserves the non-overlapping area based on the mask; based on the generated mask, the masks M1 and M2 of the registered feature map are constructed respectively; wherein the overlapping area pixels are marked as 1 and the non-overlapping area is marked as 0. The intersection area is accurately extracted through mask operation M=M1xM2, and the non-overlapping area masks M1-M and M2-M are calculated respectively. This mask separation strategy divides the image domain into three mutually exclusive areas: the shared overlapping area (M=1), the first image exclusive area (M1-M=1), and the second image exclusive area (M2-M=1).

[0109] In the reconstruction stage, to ensure that the reconstruction process focuses on the key areas, the application designs a mask constraint mechanism: only the overlapping area (M=1) is constrained by the reconstruction loss, realizing the dual alignment of pixel level and semantic level.

[0110] For non-overlapping areas, the application adopts an information preservation strategy: directly extracting the corresponding areas (I1・(M1-M) and I2・(M2-M)) from the original registered images, avoiding the loss of details that may be introduced by the reconstruction process.

[0111] The final fusion image I _final is generated by the following formula:

[0112] I _final =I _rec ・M+I1・(M1-M)+I2・(M2-M)

[0113] Where I _rec represents the original registered image, I_rec・M represents the overlapping area extracted from the original registered image, and its non-overlapping area (I1・(M1-M)) and I2's non-overlapping area (I2・(M2-M)) constitute the final fusion image I _final .

[0114] Thus, the semantic consistency reconstruction of the overlapping area and the original information fidelity of the non-overlapping area are realized. This area separation processing mechanism not only solves the color conflict problem in the overlapping area of traditional methods, but also preserves the texture details in the non-overlapping area.

[0115] The traditional splicing method directly superimposes the overlapping area, which is easy to cause exposure difference or color discontinuity. The application locates the overlapping area through the mask matrix, uses the autoencoder to reconstruct the pixel details (such as RGB three-channel consistency adjustment), and directly preserves the original pixels in the non-overlapping area, avoiding the accumulation of reconstruction errors. The color and brightness non-uniformity (measured by standard deviation) of the fusion image is reduced by 28%, the incidence of overlapping area artifacts is reduced by 62% compared with traditional methods, and the generated fusion image has higher visual fluency and semantic consistency. The conversion of the image to be spliced into a fusion image is shown in Figures 8-13 .

[0116] The fusion network is trained using the following loss function. The fusion network fuses the processed registered feature maps, and the fusion stage of the registered feature maps uses a combination strategy of pixel-level reconstruction loss L pixel and semantic-level perception loss L perceptual , and constructs the total loss function L z, to realize the multi-granularity constraint of the overlapping region. The total loss function is constrained by pixel-level accuracy and semantic-level rationality, which theoretically guarantees the visual fluency and semantic consistency of the fused image, and provides a solid optimization foundation for unsupervised image stitching. The total loss function L z The specific formula is as follows:

[0117] L z =L pixel +aL perceptual

[0118] Wherein, a = 0.2, to balance the pixel-level accuracy and semantic consistency.

[0119] The pixel reconstruction loss L pixel is expressed as follows:

[0120] ;

[0121] Wherein, M is the overlapping region mask, Q is the number of overlapping region pixels, and I1 and I2 are the registered feature maps a and b respectively.

[0122] The perception loss L perceptual is expressed as follows:

[0123] ;

[0124] Wherein, represents the output feature map of the convolution layer.

[0125] The above stitching model can be used to fuse a pair or a group of images to be stitched into a fused image. Before application, the stitching model can be trained to improve the accuracy of the stitching model. The training method is as follows:

[0126] First, the video of the application scene under 400 different shooting angles is frame by frame split, and two frames of images at random time interval are extracted, and image sub-blocks with overlapping regions are generated through random cropping operation; then the image resolution is uniformly adjusted to 512*512 pixels, and the image pair sample is constructed. Finally, a dataset containing 40000 image pairs is formed, and the division of training set, validation set and test set is completed in the same ratio of 8:1:1. This construction method effectively covers the complex factors such as angle change and light difference in real scene, and provides training samples close to actual application for the model. Ensure that the model has strong generalization ability in unsupervised learning. Compared with the model trained by traditional synthetic dataset, the registration success rate of the present application is improved by 34% in real scene test.

[0127] In the present application, the deep semantic features (such as target contour, scene structure) and shallow detail features (such as texture, color) of the image to be spliced are extracted through the splicing network, providing a rich feature basis for subsequent fusion and effectively avoiding the missing of splicing information caused by single feature. The correlation analysis network can more accurately determine the overlapping relationship and spatial position between images by mining the context correlation information between images, combined with the dynamic determination of the size of the minimum bounding rectangle, which greatly reduces the registration deviation. The encoding stage of the fusion network enhances the feature expression ability by reducing the size of the feature map, and the decoding stage realizes the fine fusion of deep and shallow features by restoring the size, which not only ensures the global coherence of the spliced image, but also retains the clarity of local details, significantly improving the transition naturalness of the splicing edge. The three networks cooperatively realize the end-to-end image splicing process of "efficient feature extraction-accurate geometric registration-seamless fusion optimization", effectively solving the problems of high computational complexity, low registration accuracy and poor splicing quality of traditional methods, and adapting to the application requirements of mobile devices and complex scenes.

[0128] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An image stitching method, characterized by, The method comprises the steps of: fusing a pair or a group of to-be-stitched images into a fused image through a stitching model; the stitching model comprises a stitching network, a correlation analysis network and a fusion network connected in sequence; the stitching network receives the to-be-stitched images, and the stitching network is used for extracting deep semantic features and shallow detail features in the to-be-stitched images to obtain a stitching feature map; the correlation analysis network receives the to-be-stitched images and the stitching feature map, the correlation analysis network mines context association information in the to-be-stitched images, determines the minimum bounding rectangle size of the to-be-stitched images, and fuses the features of the to-be-stitched images and the stitching feature map to obtain a registration feature map; the fusion network receives the registration feature map, the fusion network encodes and correspondingly decodes the registration feature map, reduces the size of the registration feature map in the encoding to increase the feature expression of the registration feature map, increases the size of the registration feature map in the corresponding decoding, and fuses the deep semantic features and the shallow detail features in the registration feature map to obtain a fused image; the stitching network comprises a stitching preprocessing module, a plurality of stitching feature extraction modules, an attention processing module and a plurality of sampling modules connected in sequence, the stitching preprocessing module is used for reducing the size of the to-be-stitched images and expanding the channel number of the to-be-stitched images to generate a first feature map, the stitching feature extraction module is used for extracting features in the first feature map to generate a plurality of intermediate feature maps with different sizes and channel numbers; the attention processing module is used for capturing global dependency between channels in the intermediate feature map to generate an attention feature map, and the sampling module is used for upsampling the attention feature map to generate a plurality of sampling feature maps, and fusing the up-sampling feature map with the intermediate feature map corresponding to the channel number to generate the stitching feature map; the stitching feature extraction module comprises at least two residual blocks and a maximum pooling layer connected in sequence, the residual block comprises an expansion convolution unit, a depth separable convolution unit, an attention unit, an adjustment convolution unit, an addition unit and an activation unit connected in sequence; the expansion convolution unit is used to expand the channel number to widen the feature expression space; the depth separable convolution unit is used to extract spatial and channel features, the attention unit is used to select a local optimal coverage range of cross-channel interaction, capture key dependency between channels, and focus on discriminative features; the adjustment convolution unit is used to adjust the channel number; and the addition unit is used to fuse features.

2. The image stitching method of claim 1, wherein, The residual block further comprises an additional adjustment convolution unit for adjusting the channel number of the previous residual block output image; the residual block without the additional adjustment convolution unit is taken as a first structure residual block, and the residual block with the additional adjustment convolution unit is taken as a second structure residual block; the splicing feature extraction module is provided with four, and the number of the residual blocks in the four splicing feature extraction modules is respectively two, three, two and two; in the same splicing feature extraction module, the residual block in front adopts the first structure residual block, and the last residual block adopts the second structure residual block.

3. The image stitching method of claim 2, wherein, The correlation analysis network comprises a context correlation module, a correlation feature extraction module, a full connection module, a control point offset module, a direct linear transformation module and a splicing domain transformation module connected in sequence; the context correlation module is based on a directional feature interaction mechanism, and separates the feature dependency relationship in the horizontal and vertical directions to accurately capture the spatial corresponding clues in the splicing feature map; the correlation feature extraction module is used for extracting features in the splicing feature map, the full connection module is used for adjusting the feature vector of the splicing feature map, the control point offset module is used for controlling the offset amount of the pixel points in the splicing feature map, the direct linear transformation module is used for transforming the splicing feature map and generating a warped feature map, and the warped feature map is fused with one of the to-be-spliced images, and then the splicing domain transformation module is used to splice with the other to-be-spliced image to generate the registration feature map and the mask.

4. The image stitching method of claim 3, wherein, The registration feature map is divided into a plurality of grid units; in the direct linear transformation module, for each grid unit, a direct linear transformation is adopted to calculate a local homography matrix H from the offset amount; the mathematical expression of the homography transformation is: ； wherein, is the source image coordinate, is the target image coordinate; is an element in the local homography matrix H. The direct linear transformation module solves the following overdetermined equation set by least squares: ； Transformed into matrix form , , the optimal solution is obtained by singular value decomposition of the minimum singular value corresponding to the eigenvector, T represents the transpose.

5. The image stitching method of claim 4, wherein, The fusion network comprises an initial processing module, a plurality of encoding modules and a plurality of decoding modules corresponding to the encoding modules connected in sequence; the initial processing module is used for fusing the registration feature map; the encoding module is used for reducing the size of the registration feature map by half to enhance the feature expression of the registration feature map; and the decoding module is used for increasing the size of the registration feature map in the corresponding decoding, and fusing the deep semantic features and the shallow detail features in the registration feature map to obtain the fusion image.

6. The image stitching method of claim 5, wherein, The fusion network reconstructs the overlapping area of the registration feature map pair, generates a reconstruction result with the same size as the input to-be-spliced image after decoding, restores the pixel details and semantic consistency of the overlapping area, directly uses the original image area after registration as a reserved part for the non-overlapping area, and finally performs pixel-level fusion operation on the reserved non-overlapping area and the reconstructed overlapping area to seamlessly splice them and generate a fusion image with uniform color space and geometric consistency.

7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the image splicing method in any one of claims 1 to 6.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the steps of the image stitching method of any one of claims 1 to 6.

Images