Light-weight image fusion method based on Mamba-KAN structure

Abstract

The invention discloses a light-weight image fusion method based on a Mamb-KAN structure, and the method comprises the following steps: obtaining a registration source image, dividing the registration source image into overlapped image blocks, and constructing a Sheaf Laplacian matrix according to an overlapping relation and limited mapping; decomposing the Laplacian spectrum matrix, and projecting image block features into a Sheaf coefficient sequence; a Mama-KAN network is constructed, a Mama branch receives the coefficient sequence to extract fusion features, and a KAN branch calculates a block-level fusion weight according to the coefficient sequence and the fusion features; constructing a loss training network by using the Sheaf consistency energy and the reconstruction error; a lightweight network is obtained according to contribution pruning and quantification of the Mama channel and the KAN base unit to the consistent energy; in the fusion stage, a Sheaf coefficient sequence is generated and input into the lightweight network, and a source image is weighted according to a fusion weight to generate a fusion image. According to the invention, high-quality and low-overhead image fusion is realized.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a lightweight image fusion method based on the Mamba-KAN structure. Background Technology

[0002] In tin smelting production, smelting furnaces, flues, cooling devices, and material yards are constantly exposed to high temperatures, strong radiation, and dust and smoke. Enterprises generally rely on visible light cameras and infrared thermal imagers for process monitoring. Current practices often involve viewing monitoring images from different modalities separately, or using traditional image fusion algorithms such as simple weighting and pyramid decomposition to overlay infrared and visible light images for observing furnace mouth crusting, material level, flame overflow, and equipment hotspots. While these methods are convenient, they are not well-suited to conditions such as smoke obstruction, strong reflections, and localized overexposure. The fused images are prone to edge blurring, contour misalignment, and loss of detail, and the structural information of critical components is not sufficiently stable.

[0003] With the increasing application of deep learning in industrial vision, convolutional networks, encoder-decoder structures, and attention mechanisms are gradually being used for multi-source image fusion in smelting scenarios. By using infrared brightness information and visible light texture information simultaneously through end-to-end training, the rendering effect of furnace opening contours, material surface morphology, and equipment structures has been improved to some extent. The recently emerging state-space models and Mamba structures have also been introduced into image processing tasks to model long-range dependencies with lower computational cost. However, most existing methods simply divide images into patches or feature maps and directly feed them into the network for learning, without establishing clear structural constraints at the level of image patch and modal relationship. They lack targeted control over the shape differences of the same physical part in different modalities. When there is flue gas disturbance and drastic changes in lighting at the smelting site, the fused image is still prone to problems such as misalignment of furnace opening, material surface, and pipeline contours.

[0004] On the other hand, existing deep fusion networks generally have a large number of parameters, with numerous channels, scale branches, and attention units stacked. When deployed on edge computing terminals and industrial control hosts commonly used in smelting workshops, they suffer from high memory consumption and long inference latency. Existing lightweight methods mostly compress the model through convolutional kernel pruning, channel pruning, and low-bit quantization. Pruning is usually based on weight size or overall loss changes, lacking a specific metric for "cross-modal structural consistency," which easily leads to key structural deformations after compression. Existing works also rarely utilize Sheaf Laplacian with constrained mappings to characterize the multimodal consistency relationship between overlapping image patches, and there is no unified design among consistency constraints, sequence modeling networks, and pruning and quantization strategies. This makes it difficult to simultaneously achieve fusion quality and model lightweight requirements in high-interference scenarios such as tin smelting.

[0005] Therefore, how to provide a lightweight image fusion method based on the Mamba-KAN structure is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a lightweight image fusion method based on the Mamba-KAN structure. This invention utilizes Sheaf Laplacian to construct cross-modal structural constraints, achieves structurally consistent fusion of multi-source images through Sheaf spectral basis, Mamba sequence modeling, and KAN nonlinear expression, and completes network pruning and quantization under the guidance of Sheaf consistency index. It has the advantages of accurate fusion structure, small model size, and high deployment efficiency.

[0007] A lightweight image fusion method based on the Mamba-KAN structure according to an embodiment of the present invention includes the following steps:

[0008] Obtain the registration source image, divide it into overlapping image blocks, and calculate the Sheaf Laplacian matrix based on the overlap relationship and constraint mapping.

[0009] The Sheaf spectral basis is obtained by spectral decomposition of the Sheaf Laplacian matrix. Image patch features are extracted, and the image patch features are projected onto the Sheaf spectral basis to generate Sheaf coefficients. The Sheaf coefficients are then arranged in the order of the image patches to form a sequence.

[0010] Construct a Mamba-KAN network, where the Mamba subnetwork receives the Sheaf coefficient sequence and outputs the fused features;

[0011] The KAN subnetwork receives the Sheaf coefficient sequence and fusion features, and outputs the fusion weight parameters of the image patch on the source image.

[0012] The Sheaf consistency energy is calculated based on the Sheaf Laplacian matrix and the Sheaf coefficient sequence. The loss function is constructed using the Sheaf consistency energy and the fusion reconstruction error, and the Mamba-KAN network is trained.

[0013] During training, the structural importance is calculated based on the contribution of Mamba channels and KAN basis functions to the Sheaf consistency energy. Mamba channels and KAN basis functions with structural importance below the threshold are pruned and quantized to obtain a lightweight Mamba-KAN network.

[0014] In the fusion stage, Sheaf coefficients are generated based on the registered source images. The sequence of Sheaf coefficients is input into a lightweight Mamba-KAN network, and the source images are weighted and combined according to the fusion weight parameters to generate a fused image.

[0015] Optionally, the calculation of the Sheaf Laplace matrix specifically includes:

[0016] Read the registration source image under a unified reference coordinate system, record the number of pixels in the row direction and the number of pixels in the column direction of each registration source image, establish the row direction coordinate index and the column direction coordinate index, and add each registration source image to the source image set.

[0017] Set the size parameters of the image patch in the row direction and the size parameters in the column direction, set the step size parameters of the image patch in the row direction and the step size parameters in the column direction, slide the image patch window in the row and column directions of each source image according to the step size parameters, and extract the pixel area covered by the window as the overlapping image patch. Record the starting row index, ending row index, starting column index and ending column index of the overlapping image patch in the source image, and assign an image patch number to each overlapping image patch to form an overlapping image patch set.

[0018] Select any two overlapping image blocks from the set of overlapping image blocks. Calculate the length of the cross interval in the row direction and the length of the cross interval in the column direction based on the starting row index, ending row index, starting column index, and ending column index. When both lengths are greater than zero, it is determined that there is a common pixel region. Record the corresponding image block number pair as the image block overlap relationship to form a set of image block overlap relationships.

[0019] For each pair of image block numbers in the set of overlapping image blocks, the position index of each pixel in the common pixel region is determined according to the row direction intersection interval and the column direction intersection interval. The pixel intensity of the corresponding position in the common pixel region is read from the two overlapping image blocks. The pixel intensity in the complete overlapping image block is arranged in a fixed order to form an image block intensity sequence. The pixel intensity in the common pixel region is arranged in a fixed order to form a common region intensity sequence. The constraint mapping is defined according to the one-to-one correspondence between the two types of sequences, and the constraint mapping is recorded.

[0020] A local feature space is assigned to each overlapping image patch, and a connection operator is assigned to each pair of image patch numbers in the set of overlapping image patch relationships. The connection operator transmits vectors between the two local feature spaces according to the corresponding constraint mapping. A Sheaf Laplacian matrix is ​​constructed based on the connection relationship between the local feature spaces and the connection operators, where the row and column indices of the Sheaf Laplacian matrix correspond to the image patch numbers, the diagonal elements are obtained by combining the connection operators related to the image patch numbers, and the off-diagonal elements are obtained by combining the connection operators connecting the two image patch numbers.

[0021] Optionally, the generation of the Sheaf coefficient sequence specifically includes:

[0022] Eigenvalue decomposition is performed based on the Sheaf Laplacian matrix. The set of eigenvalues ​​and the set of eigenvectors are solved on the Sheaf Laplacian matrix. Each eigenvalue in the set of eigenvalues ​​corresponds one-to-one with an eigenvector in the set of eigenvectors. The row index and column index of the Sheaf Laplacian matrix are consistent with the image patch number.

[0023] The eigenvalue set is arranged in ascending order of eigenvalue values, and the corresponding eigenvector set is arranged in the same order. The eigenvectors are selected sequentially from the starting position of the arranged eigenvector set according to the preset number of spectral bases. All selected eigenvectors together constitute the Sheaf spectral base, and each selected eigenvector is used as a basis vector in the Sheaf spectral base.

[0024] For each overlapping image patch in the set of overlapping image patches, the pixel intensity of each pixel in the overlapping image patch coverage area is collected in the registration source image according to a fixed traversal order in the row and column directions. The collected pixel intensities are arranged according to the traversal order to form an image patch feature vector corresponding to the overlapping image patch number. All image patch feature vectors are arranged according to the image patch number order to form an image patch feature vector sequence.

[0025] According to the image patch numbering order, each image patch feature vector in the image patch feature vector sequence is linearly expanded on the Sheaf spectral basis. The linear expansion includes representing the image patch feature vector as a linear combination of Sheaf spectral basis vectors. The linear combination coefficients constitute the Sheaf coefficients corresponding to the image patch number. All Sheaf coefficients are arranged in order according to the image patch numbering to generate the Sheaf coefficient sequence.

[0026] Optionally, the construction and fusion feature generation of the Mamba-KAN network specifically includes:

[0027] Based on the image block numbering order, each Sheaf coefficient in the Sheaf coefficient sequence is combined with the image block number and the source image modality identifier to form an input feature vector. All input feature vectors are arranged in the image block numbering order to form an input feature vector sequence.

[0028] In the Mamba-KAN network, an input mapping layer is set up. The input mapping layer performs a linear transformation on each input feature vector in the input feature vector sequence, mapping the input feature vector from the original dimension to the preset hidden dimension, so as to obtain the Mamba input sequence that is consistent with the image patch numbering order, and maintain the one-to-one correspondence between the input feature vector and the Mamba input sequence.

[0029] In the Mamba-KAN network, a Mamba subnetwork is set up. The Mamba subnetwork contains Mamba layers arranged in hierarchical order. The first Mamba layer receives the Mamba input sequence, establishes and initializes a state vector for each position in the sequence, and calculates the updated state vector and intermediate feature vector based on the input vector and the state vector at the current position using the state space parameters each time the input vector is processed, generating the first layer intermediate feature vector sequence. Each Mamba layer starting from the second Mamba layer receives the intermediate feature vector sequence of the previous layer as the input sequence, updates the state vector in the same way, and generates a new intermediate feature vector sequence. The last Mamba layer outputs the fused feature vector sequence.

[0030] Arrange the fused feature vector sequence according to the image patch number order, so that each fused feature vector in the fused feature vector sequence corresponds one-to-one with the image patch number in the Sheaf coefficient sequence, thus obtaining the fused feature sequence.

[0031] Optionally, the generation of the fusion weight parameters specifically includes:

[0032] According to the image patch numbering order, the Sheaf coefficient vector corresponding to each image patch is extracted from the Sheaf coefficient sequence, and the fusion feature vector corresponding to the same image patch number is extracted from the fusion feature sequence. The Sheaf coefficient vector is concatenated with the fusion feature vector in the vector dimension to form the KAN input vector. This concatenation operation is performed on all image patches in the order of image patch numbering to obtain the KAN input vector sequence. Each KAN input vector in the KAN input vector sequence corresponds to an image patch number.

[0033] In the Mamba-KAN network, a KAN sub-network is set up. The KAN sub-network contains an input mapping layer, a KAN layer sequence, and an output layer. The input mapping layer performs a linear transformation on each KAN input vector in the KAN input vector sequence, mapping the KAN input vector from the original feature dimension to the preset hidden dimension, and obtaining the KAN initial hidden vector sequence. Each KAN initial hidden vector in the KAN initial hidden vector sequence maintains the correspondence with the corresponding image patch number.

[0034] The first KAN layer in the KAN layer sequence receives the initial KAN hidden vector sequence. For each component of each initial KAN hidden vector in the initial KAN hidden vector sequence, a one-dimensional set of fundamental functions is input. The one-dimensional set of fundamental functions outputs the fundamental function response components. The fundamental function response components are weighted and summed according to preset weight coefficients to form an updated KAN hidden vector, generating the first KAN hidden vector sequence. Each KAN layer starting from the second KAN layer receives the KAN hidden vector sequence output by the previous KAN layer. The one-dimensional fundamental function calculation and weighted summation operation are repeatedly performed on each KAN hidden vector in the KAN hidden vector sequence to obtain a new KAN hidden vector sequence. The KAN hidden representation sequence is obtained in the last KAN layer.

[0035] The output layer receives the KAN hidden representation sequence, performs a linear transformation on each KAN hidden representation vector in the KAN hidden representation sequence to obtain the fusion weight vector. The dimension of the fusion weight vector is consistent with the number of source images, and each component in the fusion weight vector corresponds to a source image. Normalization operation is performed on the fusion weight vector as needed to make the fusion weight vector meet the preset numerical constraints. The normalized fusion weight vector is registered as the fusion weight parameter of the image patch on the source image.

[0036] Optionally, training the Mamba-KAN network specifically includes:

[0037] The Sheaf coefficient sequence is arranged into a Sheaf coefficient matrix according to the image patch number order. Each row of the Sheaf coefficient matrix corresponds to an image patch number, and each column corresponds to a Sheaf spectral basis vector. The Sheaf residual matrix is ​​obtained by performing matrix multiplication operation on the Sheaf Laplacian matrix and the Sheaf coefficient matrix. Each matrix element in the Sheaf residual matrix represents the consistency deviation strength of the corresponding image patch on the Sheaf structure. The Sheaf consistency energy is obtained by squaring all matrix elements of the Sheaf residual matrix and summing them.

[0038] Using the current Mamba-KAN network parameters, forward computation is performed based on the registered source image, Sheaf coefficient sequence, and fusion weight parameters to generate a predicted fused image in a unified reference coordinate system. The pixel intensity difference between the predicted fused image and the target fused image is calculated at each pixel position. The pixel intensity differences at all pixel positions are squared and summed to obtain the fusion reconstruction error.

[0039] Set the weight coefficients for Sheaf consistency energy and fusion reconstruction error, and sum the Sheaf consistency energy and fusion reconstruction error according to their respective weight coefficients to construct a loss function. The loss function reflects both Sheaf structural consistency constraints and pixel-level reconstruction constraints.

[0040] In the training sample set, the registered source image and the target fused image are selected, and the Sheaf coefficient sequence, fusion feature sequence and fusion weight parameters are generated in sequence. The gradient of the trainable parameters in the Mamba subnetwork and KAN subnetwork is calculated according to the loss function. The trainable parameters are updated using a preset optimization algorithm. The training process is repeated until the loss function meets the preset termination condition.

[0041] Optionally, the generation of the lightweight Mamba-KAN network specifically includes:

[0042] During the training phase, Sheaf consistency energy is calculated for each training batch. When calculating Sheaf consistency energy, the absolute value of the gradient of the output of each Mamba channel with respect to Sheaf consistency energy is recorded, as is the absolute value of the gradient of the output of each KAN basis function unit with respect to Sheaf consistency energy. The absolute values ​​of the gradients are averaged over the sample index, image patch number, and feature position within the training batch to obtain the average contribution value of each Mamba channel and the average contribution value of each KAN basis function unit.

[0043] Normalization is performed on the Mamba channel set and the KAN basis function unit set based on the average contribution values. The normalization coefficient is obtained by summing all the average contribution values. The structural importance is obtained by dividing each average contribution value by the normalization coefficient. The obtained structural importance is registered as the structural importance of each Mamba channel and the structural importance of each KAN basis function unit.

[0044] Set a structural importance threshold, and divide the Mamba channel set and KAN basis function unit set according to the structural importance threshold. Mamba channels with structural importance less than the structural importance threshold are marked as pruned channels, and KAN basis function units with structural importance less than the structural importance threshold are marked as pruned units.

[0045] The weight and bias parameters of the pruned channels are removed from the Mamba-KAN network parameter set, and the weight and bias parameters of the pruned units are also removed from the Mamba-KAN network parameter set. The remaining Mamba channel weight parameters and the remaining KAN basis function unit weight parameters are quantized using fixed-point coding with low-bit quantization. The quantized parameters are then loaded into the Mamba-KAN network structure to form a lightweight Mamba-KAN network.

[0046] Optionally, the generation of the fused image specifically includes:

[0047] The registration source image is acquired under a unified reference coordinate system. Overlapping image blocks are divided on the registration source image according to the image block size parameters and image block step size parameters set in the training phase. For each overlapping image block, the pixel intensity within the coverage area is extracted according to a fixed traversal order in the row and column directions and arranged to form an image block feature vector. All image block feature vectors are arranged in the order of image block number to form an image block feature vector sequence. Linear projection operation is performed on each image block feature vector using the Sheaf spectral basis. The linear projection operation includes calculating the inner product between the image block feature vector and each basis vector in the Sheaf spectral basis. All inner product values ​​form a Sheaf coefficient vector. The Sheaf coefficient vector corresponding to each image block is arranged in the order of image block number to generate a Sheaf coefficient sequence.

[0048] The Sheaf coefficient sequence is input into the Mamba subnetwork of the lightweight Mamba-KAN network. In the Mamba subnetwork, the state space update operation is performed on the Sheaf coefficient sequence according to the image patch number order. A state vector is maintained at each sequence position. The updated state vector and output vector are calculated based on the Sheaf coefficient vector and the corresponding state vector. After the state update is completed at all sequence positions, the fused feature vector sequence is obtained. The fused feature vector sequence is arranged according to the image patch number order to form the fused feature sequence.

[0049] According to the image patch numbering order, the Sheaf coefficient vector in the Sheaf coefficient sequence and the fusion feature vector in the fusion feature sequence are concatenated in the feature dimension to generate the KAN input vector sequence. The KAN input vector sequence is then input into the KAN subnetwork of the lightweight Mamba-KAN network. In the KAN subnetwork, basic function operations and linear combination operations are performed on each KAN input vector to output the fusion weight vector. Each component in the fusion weight vector corresponds to a registration source image. The fusion weight vector is normalized so that the sum of the values ​​of each component is one. The normalized fusion weight vector is registered as the fusion weight parameter of the corresponding image patch on the registration source image.

[0050] Under a unified reference coordinate system, the fusion weight parameters corresponding to each fusion weight vector are mapped to the image block coverage area of ​​the registered source image according to the image block number. For each pixel position within the image block coverage area, the pixel intensity of that pixel position is read from all registered source images, and the fusion weight parameters are used to perform a weighted summation to obtain the image block fusion pixel intensity. For pixel positions located within at least two image block coverage areas, the average value of the fusion pixel intensities of all related image blocks is taken. The fusion pixel intensities of all pixel positions are arranged into a two-dimensional array according to the row and column directions to obtain the fusion image.

[0051] The beneficial effects of this invention are:

[0052] This invention constructs a Sheaf Laplacian matrix at the image block level and generates a Sheaf coefficient sequence on the Sheaf spectral basis. This explicitly encodes the spatial overlap and constraint mapping between multimodal images of tin smelting sites into the fusion process, aligning the contours, edges, and structural information of the same physical region under different modalities within a unified structural constraint. Compared to image fusion methods that rely solely on convolutional features or attention weights for implicit alignment, this invention is more stable in preserving furnace mouth crust morphology, material surface undulations, plume boundaries, and high-temperature contours. It maintains high structural consistency and detail resolution even in scenarios with strong smoke, strong reflection, and local overexposure, facilitating accurate identification of critical operating conditions by operators.

[0053] This invention utilizes a Mamba subnetwork to perform state-space modeling of the Sheaf coefficient sequence, combined with block-level fusion weights output from a KAN subnetwork. This integrates the advantages of sequence modeling with the approximation capabilities of piecewise functions, improving the texture representation and brightness hierarchy of the fused image without significantly increasing the number of parameters. By simultaneously introducing Sheaf consistency energy and fusion reconstruction error into the loss function, this invention balances cross-modal structural consistency and pixel-level reconstruction accuracy during the training phase, thereby achieving clearer furnace outlines, more coherent equipment edges, and more reliable representation of temperature anomaly regions in complex smelting scenarios.

[0054] This invention quantifies and evaluates the contributions of Mamba channels and KAN basis function units based on Sheaf consistency energy during the training phase, thereby guiding pruning and low-bit quantization to form a lightweight Mamba-KAN network. Compared with compression based solely on weight norms or general loss, this invention prioritizes retaining channels and basis function units that have a significant impact on cross-modal structural consistency during model compression. While significantly reducing parameter size and computational cost, it maintains a high level of structural representation capability for key regions of tin smelting, making it more suitable for real-time deployment and long-term stable operation on resource-constrained devices such as edge computing terminals and industrial control hosts. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of a lightweight image fusion method based on the Mamba-KAN structure proposed in this invention;

[0057] Figure 2 This is a schematic diagram illustrating the Sheaf Laplacian matrix calculation for a lightweight image fusion method based on the Mamba-KAN structure proposed in this invention. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0059] refer to Figure 1-2 A lightweight image fusion method based on the Mamba-KAN structure includes the following steps:

[0060] Obtain the registration source image, divide it into overlapping image blocks, and calculate the Sheaf Laplacian matrix based on the overlap relationship and constraint mapping.

[0061] The Sheaf spectral basis is obtained by spectral decomposition of the Sheaf Laplacian matrix. Image patch features are extracted, and the image patch features are projected onto the Sheaf spectral basis to generate Sheaf coefficients. The Sheaf coefficients are then arranged in the order of the image patches to form a sequence.

[0062] Construct a Mamba-KAN network, where the Mamba subnetwork receives the Sheaf coefficient sequence and outputs the fused features;

[0063] The KAN subnetwork receives the Sheaf coefficient sequence and fusion features, and outputs the fusion weight parameters of the image patch on the source image.

[0064] The Sheaf consistency energy is calculated based on the Sheaf Laplacian matrix and the Sheaf coefficient sequence. The loss function is constructed using the Sheaf consistency energy and the fusion reconstruction error, and the Mamba-KAN network is trained.

[0065] During training, the structural importance is calculated based on the contribution of Mamba channels and KAN basis functions to the Sheaf consistency energy. Mamba channels and KAN basis functions with structural importance below the threshold are pruned and quantized to obtain a lightweight Mamba-KAN network.

[0066] In the fusion stage, Sheaf coefficients are generated based on the registered source images. The sequence of Sheaf coefficients is input into a lightweight Mamba-KAN network, and the source images are weighted and combined according to the fusion weight parameters to generate a fused image.

[0067] In this embodiment, the calculation of the Sheaf Laplace matrix specifically includes:

[0068] Read the registration source image under a unified reference coordinate system, record the number of pixels in the row direction and the number of pixels in the column direction of each registration source image, establish the row direction coordinate index and the column direction coordinate index, and add each registration source image to the source image set.

[0069] Set the size parameters of the image patch in the row direction and the size parameters in the column direction, set the step size parameters of the image patch in the row direction and the step size parameters in the column direction, slide the image patch window in the row and column directions of each source image according to the step size parameters, and extract the pixel area covered by the window as the overlapping image patch. Record the starting row index, ending row index, starting column index and ending column index of the overlapping image patch in the source image, and assign an image patch number to each overlapping image patch to form an overlapping image patch set.

[0070] Select any two overlapping image blocks from the set of overlapping image blocks. Calculate the length of the cross interval in the row direction and the length of the cross interval in the column direction based on the starting row index, ending row index, starting column index, and ending column index. When both lengths are greater than zero, it is determined that there is a common pixel region. Record the corresponding image block number pair as the image block overlap relationship to form a set of image block overlap relationships.

[0071] For each pair of image block numbers in the set of overlapping image blocks, the position index of each pixel in the common pixel region is determined according to the row direction intersection interval and the column direction intersection interval. The pixel intensity of the corresponding position in the common pixel region is read from the two overlapping image blocks. The pixel intensity in the complete overlapping image block is arranged in a fixed order to form an image block intensity sequence. The pixel intensity in the common pixel region is arranged in a fixed order to form a common region intensity sequence. The constraint mapping is defined according to the one-to-one correspondence between the two types of sequences, and the constraint mapping is recorded.

[0072] A local feature space is assigned to each overlapping image patch, and a connection operator is assigned to each pair of image patch numbers in the set of overlapping image patch relationships. The connection operator transmits vectors between the two local feature spaces according to the corresponding constraint mapping. A Sheaf Laplacian matrix is ​​constructed based on the connection relationship between the local feature spaces and the connection operators, where the row and column indices of the Sheaf Laplacian matrix correspond to the image patch numbers, the diagonal elements are obtained by combining the connection operators related to the image patch numbers, and the off-diagonal elements are obtained by combining the connection operators connecting the two image patch numbers.

[0073] This invention precisely locates the common regions between image blocks by dividing the registered source image into blocks and constructing overlapping relationships under a unified reference coordinate system. It also establishes a constraint mapping based on pixel position correspondence, enabling the spatial adjacency and cross-modal consistency of the image in local regions to be formally expressed as a local feature space and connectivity operators. This allows for the accurate construction of the Sheaf Laplacian matrix. During the construction process, the coupling structure and mutual constraints between image blocks are fully preserved, providing a rigorous topological foundation for subsequent Sheaf spectral basis extraction and fusion network input. This makes the structural constraints more stable and consistent under complex imaging environments, improving the reliability of the fused image in terms of structural alignment and detail preservation.

[0074] In this embodiment, the generation of the Sheaf coefficient sequence specifically includes:

[0075] Eigenvalue decomposition is performed based on the Sheaf Laplacian matrix. The set of eigenvalues ​​and the set of eigenvectors are solved on the Sheaf Laplacian matrix. Each eigenvalue in the set of eigenvalues ​​corresponds one-to-one with an eigenvector in the set of eigenvectors. The row index and column index of the Sheaf Laplacian matrix are consistent with the image patch number.

[0076] The eigenvalue set is arranged in ascending order of eigenvalue values, and the corresponding eigenvector set is arranged in the same order. The eigenvectors are selected sequentially from the starting position of the arranged eigenvector set according to the preset number of spectral bases. All selected eigenvectors together constitute the Sheaf spectral base, and each selected eigenvector is used as a basis vector in the Sheaf spectral base.

[0077] For each overlapping image patch in the set of overlapping image patches, the pixel intensity of each pixel in the overlapping image patch coverage area is collected in the registration source image according to a fixed traversal order in the row and column directions. The collected pixel intensities are arranged according to the traversal order to form an image patch feature vector corresponding to the overlapping image patch number. All image patch feature vectors are arranged according to the image patch number order to form an image patch feature vector sequence.

[0078] According to the image patch numbering order, each image patch feature vector in the image patch feature vector sequence is linearly expanded on the Sheaf spectral basis. The linear expansion includes representing the image patch feature vector as a linear combination of Sheaf spectral basis vectors. The linear combination coefficients constitute the Sheaf coefficients corresponding to the image patch number. All Sheaf coefficients are arranged in order according to the image patch numbering to generate the Sheaf coefficient sequence.

[0079] This invention obtains the Sheaf spectral basis by performing eigenvalue decomposition on the Sheaf Laplacian matrix, enabling the representation of image block-level structural relationships in a unified spectral space. Then, the pixel intensity vector of each image block is linearly expanded on this spectral basis, transforming the local intensity distribution into Sheaf coefficients with structural consistency constraints. This allows cross-modal images to be comparable and fused within the same structural framework. The Sheaf coefficient sequence not only compresses the high-dimensional information of local image blocks but also encodes structural relationships at the coefficient level, thus providing stable input for subsequent Mamba sequence modeling and KAN weight generation. This enables the fusion process to maintain higher boundary alignment, texture preservation, and structural continuity capabilities in complex scenes.

[0080] In this embodiment, the construction and fusion feature generation of the Mamba-KAN network specifically includes:

[0081] Based on the image block numbering order, each Sheaf coefficient in the Sheaf coefficient sequence is combined with the image block number and the source image modality identifier to form an input feature vector. All input feature vectors are arranged in the image block numbering order to form an input feature vector sequence.

[0082] In the Mamba-KAN network, an input mapping layer is set up. The input mapping layer performs a linear transformation on each input feature vector in the input feature vector sequence, mapping the input feature vector from the original dimension to the preset hidden dimension, so as to obtain the Mamba input sequence that is consistent with the image patch numbering order, and maintain the one-to-one correspondence between the input feature vector and the Mamba input sequence.

[0083] In the Mamba-KAN network, a Mamba subnetwork is set up. The Mamba subnetwork contains Mamba layers arranged in hierarchical order. The first Mamba layer receives the Mamba input sequence, establishes and initializes a state vector for each position in the sequence, and calculates the updated state vector and intermediate feature vector based on the input vector and the state vector at the current position using the state space parameters each time the input vector is processed, generating the first layer intermediate feature vector sequence. Each Mamba layer starting from the second Mamba layer receives the intermediate feature vector sequence of the previous layer as the input sequence, updates the state vector in the same way, and generates a new intermediate feature vector sequence. The last Mamba layer outputs the fused feature vector sequence.

[0084] Arrange the fused feature vector sequence according to the image patch number order, so that each fused feature vector in the fused feature vector sequence corresponds one-to-one with the image patch number in the Sheaf coefficient sequence, thus obtaining the fused feature sequence.

[0085] This invention combines Sheaf coefficients, image patch numbers, and modality identifiers into an input feature vector and inputs it into a Mamba-KAN network. This enables the sequence structure to explicitly encode spatial location and source modality information. The Mamba sub-network uses a state-space model to model long-range dependencies of the input sequence, effectively capturing cross-image patch structural associations and modal differences in the sequence dimension. Furthermore, it extracts fusion features reflecting local and global relationships during multi-level state updates. The final output fusion feature sequence has significant advantages in spatial consistency, modality sensitivity, and structural coherence, providing a robust high-level representation for subsequent calculation of fusion weights. This invention enables the fusion model to more accurately maintain boundary, contour, and texture structures in complex scenes, while also possessing higher modeling capabilities and generalization performance.

[0086] In this embodiment, the generation of the fusion weight parameters specifically includes:

[0087] According to the image patch numbering order, the Sheaf coefficient vector corresponding to each image patch is extracted from the Sheaf coefficient sequence, and the fusion feature vector corresponding to the same image patch number is extracted from the fusion feature sequence. The Sheaf coefficient vector is concatenated with the fusion feature vector in the vector dimension to form the KAN input vector. This concatenation operation is performed on all image patches in the order of image patch numbering to obtain the KAN input vector sequence. Each KAN input vector in the KAN input vector sequence corresponds to an image patch number.

[0088] In the Mamba-KAN network, a KAN sub-network is set up. The KAN sub-network contains an input mapping layer, a KAN layer sequence, and an output layer. The input mapping layer performs a linear transformation on each KAN input vector in the KAN input vector sequence, mapping the KAN input vector from the original feature dimension to the preset hidden dimension, and obtaining the KAN initial hidden vector sequence. Each KAN initial hidden vector in the KAN initial hidden vector sequence maintains the correspondence with the corresponding image patch number.

[0089] The first KAN layer in the KAN layer sequence receives the initial KAN hidden vector sequence. For each component of each initial KAN hidden vector in the initial KAN hidden vector sequence, a one-dimensional set of fundamental functions is input. The one-dimensional set of fundamental functions outputs the fundamental function response components. The fundamental function response components are weighted and summed according to preset weight coefficients to form an updated KAN hidden vector, generating the first KAN hidden vector sequence. Each KAN layer starting from the second KAN layer receives the KAN hidden vector sequence output by the previous KAN layer. The one-dimensional fundamental function calculation and weighted summation operation are repeatedly performed on each KAN hidden vector in the KAN hidden vector sequence to obtain a new KAN hidden vector sequence. The KAN hidden representation sequence is obtained in the last KAN layer.

[0090] The output layer receives the KAN hidden representation sequence, performs a linear transformation on each KAN hidden representation vector in the KAN hidden representation sequence to obtain the fusion weight vector. The dimension of the fusion weight vector is consistent with the number of source images, and each component in the fusion weight vector corresponds to a source image. Normalization operation is performed on the fusion weight vector as needed to make the fusion weight vector meet the preset numerical constraints. The normalized fusion weight vector is registered as the fusion weight parameter of the image patch on the source image.

[0091] This invention forms a KAN input vector by concatenating Sheaf coefficients and fusion features along the vector dimension. This allows the local structural information and sequence modeling features of each image patch to simultaneously enter the KAN sub-network for nonlinear expression. The KAN layer performs piecewise fitting of the input features through a set of basic functions, giving the model a stronger function approximation ability. This accurately models the differences and correlations between different modalities at the same spatial location at the image patch level. The resulting fusion weight vector is more refined and stable in cross-modal feature trade-offs, allowing the source image to be reasonably weighted in terms of brightness, texture, and structure. This invention can significantly improve the quality of fused images in high-interference environments, achieving clearer structural presentation and more reliable multimodal information integration.

[0092] In this embodiment, training the Mamba-KAN network specifically includes:

[0093] The Sheaf coefficient sequence is arranged into a Sheaf coefficient matrix according to the image patch number order. Each row of the Sheaf coefficient matrix corresponds to an image patch number, and each column corresponds to a Sheaf spectral basis vector. The Sheaf residual matrix is ​​obtained by performing matrix multiplication operation on the Sheaf Laplacian matrix and the Sheaf coefficient matrix. Each matrix element in the Sheaf residual matrix represents the consistency deviation strength of the corresponding image patch on the Sheaf structure. The Sheaf consistency energy is obtained by squaring all matrix elements of the Sheaf residual matrix and summing them.

[0094] Using the current Mamba-KAN network parameters, forward computation is performed based on the registered source image, Sheaf coefficient sequence, and fusion weight parameters to generate a predicted fused image in a unified reference coordinate system. The pixel intensity difference between the predicted fused image and the target fused image is calculated at each pixel position. The pixel intensity differences at all pixel positions are squared and summed to obtain the fusion reconstruction error.

[0095] Set the weight coefficients for Sheaf consistency energy and fusion reconstruction error, and sum the Sheaf consistency energy and fusion reconstruction error according to their respective weight coefficients to construct a loss function. The loss function reflects both Sheaf structural consistency constraints and pixel-level reconstruction constraints.

[0096] In the training sample set, the registered source image and the target fused image are selected, and the Sheaf coefficient sequence, fusion feature sequence and fusion weight parameters are generated in sequence. The gradient of the trainable parameters in the Mamba subnetwork and KAN subnetwork is calculated according to the loss function. The trainable parameters are updated using a preset optimization algorithm. The training process is repeated until the loss function meets the preset termination condition.

[0097] In this embodiment, the generation of the lightweight Mamba-KAN network specifically includes:

[0098] During the training phase, Sheaf consistency energy is calculated for each training batch. When calculating Sheaf consistency energy, the absolute value of the gradient of the output of each Mamba channel with respect to Sheaf consistency energy is recorded, as is the absolute value of the gradient of the output of each KAN basis function unit with respect to Sheaf consistency energy. The absolute values ​​of the gradients are averaged over the sample index, image patch number, and feature position within the training batch to obtain the average contribution value of each Mamba channel and the average contribution value of each KAN basis function unit.

[0099] Normalization is performed on the Mamba channel set and the KAN basis function unit set based on the average contribution values. The normalization coefficient is obtained by summing all the average contribution values. The structural importance is obtained by dividing each average contribution value by the normalization coefficient. The obtained structural importance is registered as the structural importance of each Mamba channel and the structural importance of each KAN basis function unit.

[0100] Set a structural importance threshold, and divide the Mamba channel set and KAN basis function unit set according to the structural importance threshold. Mamba channels with structural importance less than the structural importance threshold are marked as pruned channels, and KAN basis function units with structural importance less than the structural importance threshold are marked as pruned units.

[0101] The weight and bias parameters of the pruned channels are removed from the Mamba-KAN network parameter set, and the weight and bias parameters of the pruned units are also removed from the Mamba-KAN network parameter set. The remaining Mamba channel weight parameters and the remaining KAN basis function unit weight parameters are quantized using fixed-point coding with low-bit quantization. The quantized parameters are then loaded into the Mamba-KAN network structure to form a lightweight Mamba-KAN network.

[0102] In this embodiment, the generation of the fused image specifically includes:

[0103] The registration source image is acquired under a unified reference coordinate system. Overlapping image blocks are divided on the registration source image according to the image block size parameters and image block step size parameters set in the training phase. For each overlapping image block, the pixel intensity within the coverage area is extracted according to a fixed traversal order in the row and column directions and arranged to form an image block feature vector. All image block feature vectors are arranged in the order of image block number to form an image block feature vector sequence. Linear projection operation is performed on each image block feature vector using the Sheaf spectral basis. The linear projection operation includes calculating the inner product between the image block feature vector and each basis vector in the Sheaf spectral basis. All inner product values ​​form a Sheaf coefficient vector. The Sheaf coefficient vector corresponding to each image block is arranged in the order of image block number to generate a Sheaf coefficient sequence.

[0104] The Sheaf coefficient sequence is input into the Mamba subnetwork of the lightweight Mamba-KAN network. In the Mamba subnetwork, the state space update operation is performed on the Sheaf coefficient sequence according to the image patch number order. A state vector is maintained at each sequence position. The updated state vector and output vector are calculated based on the Sheaf coefficient vector and the corresponding state vector. After the state update is completed at all sequence positions, the fused feature vector sequence is obtained. The fused feature vector sequence is arranged according to the image patch number order to form the fused feature sequence.

[0105] According to the image patch numbering order, the Sheaf coefficient vector in the Sheaf coefficient sequence and the fusion feature vector in the fusion feature sequence are concatenated in the feature dimension to generate the KAN input vector sequence. The KAN input vector sequence is then input into the KAN subnetwork of the lightweight Mamba-KAN network. In the KAN subnetwork, basic function operations and linear combination operations are performed on each KAN input vector to output the fusion weight vector. Each component in the fusion weight vector corresponds to a registration source image. The fusion weight vector is normalized so that the sum of the values ​​of each component is one. The normalized fusion weight vector is registered as the fusion weight parameter of the corresponding image patch on the registration source image.

[0106] Under a unified reference coordinate system, the fusion weight parameters corresponding to each fusion weight vector are mapped to the image block coverage area of ​​the registered source image according to the image block number. For each pixel position within the image block coverage area, the pixel intensity of that pixel position is read from all registered source images, and the fusion weight parameters are used to perform a weighted summation to obtain the image block fusion pixel intensity. For pixel positions located within at least two image block coverage areas, the average value of the fusion pixel intensities of all related image blocks is taken. The fusion pixel intensities of all pixel positions are arranged into a two-dimensional array according to the row and column directions to obtain the fusion image.

[0107] Example 1:

[0108] To verify the feasibility of this invention in practice, it was applied to a furnace opening monitoring and equipment temperature anomaly identification scenario on a tin smelting production line. Smelting furnaces operate in environments with high temperature, high dust, and strong flue gas disturbance. Common visible light imaging is significantly affected by smoke obstruction and interference from incandescent light spots. While infrared thermal imaging can reflect temperature distribution, it suffers from blurred structural boundaries and missing textures. Because the two imaging modalities exhibit significant differences within the same physical area, traditional image fusion methods struggle to obtain a unified image with clear structure and accurate temperature information. This affects furnace opening crust identification, material surface undulation judgment, and equipment overheating area location. This invention is used to construct a fused image with consistent cross-modal structure, enabling the field control system to simultaneously identify temperature hotspots and clear boundary structures within a single image.

[0109] In practical applications, smelters deploy visible light industrial cameras and infrared thermal imagers, with both devices simultaneously imaging the furnace opening area. This invention first maps the two images to a unified reference coordinate system. By fixing the pose and calibration parameters at the acquisition end, the geometric correspondence between modes is ensured. Using a set image block size and step size, the two images are divided into image blocks with overlapping regions. By determining whether there are common pixel regions between the image blocks, the overlap relationship between the image blocks is established. A constraint mapping is constructed based on the pixel position relationships within the common regions, and then the Sheaf Laplacian matrix is ​​calculated. Next, spectral decomposition is performed on the Sheaf Laplacian matrix to obtain the Sheaf spectral basis. Finally, the pixel intensity vector of each image block is projected... The image is projected onto the Sheaf spectral basis to form a Sheaf coefficient sequence. This sequence is then input into the Mamba-KAN network, enabling the Mamba sub-network to model the sequence and generate fusion features. The KAN sub-network calculates the fusion weights of the corresponding image patches in the two modalities based on the fusion features. During the training phase, a loss function is jointly constructed using the Sheaf consistency energy and the fusion reconstruction error to guide the network to maintain structural consistency between modalities while faithfully reconstructing the fused image. In the pruning and quantization phase, the contribution of each Mamba channel and KAN basis function unit to the Sheaf consistency energy is analyzed, and parameter units with low structural importance are filtered out. Finally, a lightweight Mamba-KAN network is obtained, which can be deployed on edge industrial hosts.

[0110] To verify the performance of this invention, its fusion method was compared with three representative comparative algorithms: a weighted average method, a fusion algorithm based on multi-scale decomposition, and a deep fusion method based on convolutional neural networks. The experiment selected 560 sets of continuously running furnace mouth monitoring data, each set containing two modal images, and recorded reference information such as the furnace mouth edge contour, flame overflow area, and equipment hotspot areas manually marked at the smelting site. The data were quantitatively evaluated from four dimensions: clarity, structural consistency, temperature maintenance accuracy, and noise control capability of the fused image.

[0111] In the experiment, visible light image sharpness gradient, infrared temperature distribution error index, modal edge alignment, and PSNR were used as comprehensive evaluation indicators. The comparison results are shown in Table 1:

[0112] Table 1 Performance Comparison of Fusion Methods

[0113] Fusion Method Structural alignment error (pixels) Temperature error (°C) Sharpness gradient (×10⁻³) PSNR (dB) Weighted average method 9.12 7.84 1.21 19.4 Multi-scale decomposition and fusion 6.45 6.30 1.58 21.7 Deep convolutional fusion 4.28 5.02 2.04 23.1 The Mamba-KAN fusion method of this invention 2.31 2.27 3.16 27.8

[0114] As can be seen from the comparative data, the fused image generated by this invention maintains higher consistency in the furnace mouth edge position, material surface boundary, and plume shape, and the structural alignment error is reduced to about 2 pixels. In terms of temperature preservation, due to the joint modeling of Sheaf coefficients and fusion features by the KAN sub-network, this invention effectively reduces the offset and blurring of infrared information, reducing the temperature distribution error to the order of 2°C. The sharpness gradient is greatly improved, indicating that the texture structure of the visible light image is fully preserved in the fusion result. The PSNR is significantly higher than the three comparison methods, indicating that the overall quality of the fused image is higher.

[0115] Furthermore, the lightweight Mamba-KAN network of this invention exhibits higher inference efficiency. On the same hardware platform, the average inference latency of traditional deep convolutional fusion methods is approximately 41ms, and that of multi-scale fusion is 52ms, while the inference latency of the model of this invention, after pruning and quantization, is only 17ms. This enables stable real-time processing in continuous monitoring scenarios, significantly reducing the computational load on industrial hosts.

[0116] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lightweight image fusion method based on the Mamba-KAN structure, characterized in that, Includes the following steps: Obtain the registration source image, divide it into overlapping image blocks, and calculate the Sheaf Laplacian matrix based on the overlap relationship and constraint mapping. The Sheaf spectral basis is obtained by spectral decomposition of the Sheaf Laplacian matrix. Image patch features are extracted, and the image patch features are projected onto the Sheaf spectral basis to generate Sheaf coefficients. The Sheaf coefficients are then arranged in the order of the image patches to form a sequence. Construct a Mamba-KAN network, where the Mamba subnetwork receives the Sheaf coefficient sequence and outputs the fused features; The KAN subnetwork receives the Sheaf coefficient sequence and fusion features, and outputs the fusion weight parameters of the image patch on the source image. The Sheaf consistency energy is calculated based on the Sheaf Laplacian matrix and the Sheaf coefficient sequence. The loss function is constructed using the Sheaf consistency energy and the fusion reconstruction error, and the Mamba-KAN network is trained. During training, the structural importance is calculated based on the contribution of Mamba channels and KAN basis functions to the Sheaf consistency energy. Mamba channels and KAN basis functions with structural importance below the threshold are pruned and quantized to obtain a lightweight Mamba-KAN network. In the fusion stage, Sheaf coefficients are generated based on the registered source images. The sequence of Sheaf coefficients is input into a lightweight Mamba-KAN network, and the source images are weighted and combined according to the fusion weight parameters to generate a fused image.

2. The lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The calculation of the Sheaf Laplace matrix specifically includes: Acquire at least two registration source images under a unified reference coordinate system, and register each registration source image as a source image; Set the image patch size parameters and image patch step size parameters, slide the image patch window on each source image in the row and column directions to generate an overlapping image patch set covering the source image, and register the image patch number and position range in the source image for each overlapping image patch; Based on the location range, determine whether any two overlapping image blocks have a common pixel region. If the common pixel region contains at least one pixel, register the image block overlap relationship to form a set of image block overlap relationships. For each pair of overlapping image blocks in the set of image block overlap relationships, establish a constraint mapping from the pixel position inside the image block to the position in the common pixel region based on the pixel position, and register the constraint mapping; Based on the set of overlapping image patches, the set of image patch overlap relationships, and the constraint mapping, the Sheaf Laplacian matrix is ​​calculated according to the definition of Sheaf Laplacian.

3. The lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The generation of the Sheaf coefficient sequence specifically includes: Eigenvalue decomposition is performed based on the Sheaf Laplacian matrix to obtain a set of eigenvalues ​​and a set of eigenvectors corresponding to each eigenvalue. The eigenvalue set is arranged in ascending order of eigenvalue values. The eigenvector set corresponding to the eigenvalues ​​is arranged in the same order. The eigenvectors are selected sequentially from the arranged eigenvector set according to the preset number of spectral bases to form the Sheaf spectral base. For each overlapping image patch in the set of overlapping image patches, the pixel intensity values ​​of the overlapping image patch coverage area are extracted in the registration source image according to a fixed traversal order in the row and column directions, and the pixel intensity values ​​are arranged into an image patch feature vector corresponding to the image patch number. According to the image patch numbering order, the feature vector of each image patch is linearly expanded on the Sheaf spectral basis to obtain the Sheaf coefficients corresponding to the image patch number. The Sheaf coefficients are then arranged in the order of the image patch numbers to generate the Sheaf coefficient sequence.

4. The lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The construction and fusion feature generation of the Mamba-KAN network specifically include: Based on the image block numbering order, each Sheaf coefficient in the Sheaf coefficient sequence is combined with the image block number and the source image modality identifier to form an input feature vector sequence; In the Mamba-KAN network, an input mapping layer is set up. The input mapping layer performs a linear transformation on each input feature vector in the input feature vector sequence to generate the Mamba input sequence. In the Mamba-KAN network, a Mamba subnetwork is set up. The Mamba subnetwork includes a preset number of Mamba layers. The Mamba layers are connected in hierarchical order. The first Mamba layer receives the Mamba input sequence, maintains a state vector for each input vector according to the sequence position index, and calculates the updated state vector and intermediate feature vector based on the current input vector and the corresponding state vector. Each Mamba layer starting from the second Mamba layer receives the intermediate feature vector sequence output by the previous Mamba layer, updates the state vector according to the same rules, and generates a new intermediate feature vector sequence. The fused feature vector sequence is obtained in the last Mamba layer. The fused feature vector sequence is obtained by arranging the image patch numbers in order.

5. A lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The generation of the fusion weight parameters specifically includes: According to the image patch numbering order, the Sheaf coefficients corresponding to each image patch are obtained from the Sheaf coefficient sequence, and the fusion feature vectors corresponding to the same image patch number are obtained from the fusion feature sequence. The Sheaf coefficients and the fusion feature vectors are concatenated in the feature dimension to form the KAN input vector. All KAN input vectors are arranged according to the image patch numbering order to form the KAN input vector sequence. In the Mamba-KAN network, a KAN subnetwork is set up. The KAN subnetwork contains an input mapping layer, a KAN layer sequence, and an output layer. The input mapping layer receives the KAN input vector sequence and performs a linear transformation on each KAN input vector in the KAN input vector sequence to obtain the initial KAN hidden vector sequence. The KAN layer sequence is determined by the number of KAN layers. The KAN layers are connected in hierarchical order. The first KAN layer receives the initial KAN hidden vector sequence and performs basic function operations and linear combination operations on each initial KAN hidden vector in the initial KAN hidden vector sequence to generate the first KAN hidden vector sequence. Each KAN layer starting from the second KAN layer receives the KAN hidden vector sequence output by the previous KAN layer and performs basic function operations and linear combination operations to generate a new KAN hidden vector sequence. The KAN hidden representation sequence is obtained in the last KAN layer. The output layer receives the KAN hidden representation sequence, performs a linear transformation on each KAN hidden representation vector in the KAN hidden representation sequence to obtain the fusion weight vector. Each component in each fusion weight vector corresponds to a source image. The fusion weight vector is registered as the fusion weight parameter of the image patch on the source image.

6. The lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The training of the Mamba-KAN network specifically includes: The Sheaf coefficient sequence is arranged into a Sheaf coefficient matrix according to the image block numbering order. The Sheaf residual matrix is ​​obtained by performing matrix multiplication operation between the Sheaf Laplacian matrix and the Sheaf coefficient matrix. The Sheaf uniformity energy is obtained by squaring each element of the Sheaf residual matrix and summing them. The Mamba-KAN network is used to generate a predicted fused image based on the registered source image, Sheaf coefficient sequence and fusion weight parameters. The pixel intensity difference between the predicted fused image and the target fused image is calculated at each pixel position under a unified reference coordinate system. The pixel intensity difference is squared and summed to obtain the fusion reconstruction error. Set the weight coefficients for Sheaf consistency energy and fusion reconstruction error, and construct the loss function based on the weighted sum of Sheaf consistency energy and fusion reconstruction error; In the training sample set, the registered source image and the target fused image are selected, and the Sheaf coefficient sequence, fusion feature sequence and fusion weight parameters are generated. The gradient of the trainable parameters in the Mamba subnetwork and KAN subnetwork is calculated according to the loss function. The trainable parameters are updated using the optimization algorithm. The training is repeated until the loss function meets the termination condition.

7. A lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The generation of the lightweight Mamba-KAN network specifically includes: During the training phase, Sheaf consistency energy is calculated for each training batch. When calculating Sheaf consistency energy, the contribution values ​​of each Mamba channel and each KAN basis function unit to Sheaf consistency energy are recorded. The contribution values ​​within the training batch are averaged to obtain the average contribution value of each Mamba channel and the average contribution value of each KAN basis function unit. Normalization is performed on the Mamba channel set and the KAN basis function unit set based on the average contribution value. The normalization results are recorded as the structural importance of each Mamba channel and the structural importance of each KAN basis function unit. Set a structural importance threshold, and divide the Mamba channel set and KAN basis function unit set according to the structural importance threshold. Mamba channels with structural importance less than the structural importance threshold are marked as pruned channels, and KAN basis function units with structural importance less than the structural importance threshold are marked as pruned units. The channel parameters corresponding to the pruned channels are removed from the Mamba-KAN network, and the basis function parameters corresponding to the pruned units are removed from the Mamba-KAN network. Low-bit quantization is then performed on the retained Mamba channel weight parameters and the retained KAN basis function unit weight parameters to obtain a lightweight Mamba-KAN network.

8. A lightweight image fusion method based on the Mamba-KAN structure according to claim 1, characterized in that, The generation of the fused image specifically includes: The registration source image is acquired under a unified reference coordinate system. Overlapping image blocks are divided according to the image block size parameters and image block step size parameters set during the training phase. Pixel intensity is extracted based on the pixel position of the overlapping image block coverage area. Image block feature vector sequence is formed according to the image block numbering order. Linear projection operation is performed on the image block feature vector sequence using the Sheaf spectral basis to generate Sheaf coefficient sequence. The Sheaf coefficient sequence is input into the Mamba subnetwork of the lightweight Mamba-KAN network in the order of image patch numbering, and the fusion feature sequence is calculated according to the preset structure of the Mamba subnetwork. The Sheaf coefficient sequence and the fusion feature sequence are combined in the feature dimension to form the KAN input vector sequence. The KAN input vector sequence is then input into the KAN sub-network in the lightweight Mamba-KAN network to calculate the fusion weight parameters of the image patch on the registration source image. Under a unified reference coordinate system, the fusion weight parameters are mapped to the pixel positions of the registration source images according to the image block number. The pixel intensities of the registration source images within the coverage area of ​​each image block are weighted and summed according to the fusion weight parameters to obtain the image block fusion pixel intensity. All image block fusion pixel intensities are concatenated into a two-dimensional array according to the image block division method, and the fusion image is output.

Images