Cross-view binocular image super-resolution reconstruction method and system based on Mamba

Abstract

The present invention proposes a Mamba-based cross-view binocular image super-resolution reconstruction method and system. The CSOMM deep feature extraction module models the long-range dependencies of binocular images and extracts the contour and texture information of left-view and right-view features. A cross-view selective scanning module uses a cross-multiplication mechanism to establish a robust correspondence between the left and right views, fully utilizing the complementary information between the binocular images and further improving the network's reconstruction accuracy and stability. A pixel shuffling layer is used to upsample and reconstruct the output of the cross-view feature interaction module to obtain a reconstructed binocular image. The present invention's solution not only fully utilizes the correspondence between binocular images, but also refines image texture and structure at the multi-channel and multi-view levels, achieving more accurate and high-quality image reconstruction.

Description

Technical Field

[0001] The present invention belongs to the technical field of image reconstruction, and in particular relates to a Mamba-based cross-view binocular image super-resolution reconstruction method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The vigorous development of binocular imaging technology has promoted the widespread application of binocular images in many fields such as autonomous driving, robot navigation, medical imaging, and film and television production. In these applications, by improving the resolution of binocular images, eye-catching visual effects can be generated, thereby significantly optimizing the performance of these tasks. The core goal of binocular image super-resolution reconstruction is to make full use of the rich information from two perspectives, finely restore the image pixels, and generate high-resolution and detail-rich images. Although binocular images provide richer supplementary information than monocular images, they also pose additional challenges. The main challenge lies in how to accurately establish a reliable binocular correspondence, fully explore and utilize the potential information in the binocular images while ensuring image consistency, so as to achieve excellent visual reconstruction effects.

[0004] When it comes to solving the problem of binocular image super-resolution reconstruction, currently proposed methods primarily rely on convolutional neural networks (CNNs) and Transformers, all of which achieve excellent performance. Due to the local nature of convolution operations, CNN-based methods are computationally efficient when processing images. However, CNNs use local receptive fields to capture image features. Limited by the size of the convolution kernel, the receptive field is also small, making it impossible to establish effective global dependencies. In contrast, Transformer-based methods utilize a global receptive field for global interaction modeling, which can better capture complex global relationships and enable the network to reference more pixels for image reconstruction. However, the Transformer provides a global receptive field at the expense of quadratic computational complexity, which imposes a computational burden on model training. While researchers have attempted to improve model efficiency using effective techniques, such as moving window mechanisms, this has also sacrificed the effective range of the receptive field, potentially resulting in the loss of some pixel information.

[0005] Recently, the selective state-space model (Mamba) has attracted widespread research attention due to its linear computational cost and global receptive field (GRF) capability for long-range modeling. A growing number of researchers have explored the potential of Mamba in various fields and achieved promising results in a variety of downstream tasks. Due to its wide GRF, Mamba is able to exceptionally extract global spatial context. In image reconstruction, this means that anchor pixels in an image can be associated with information about pixels in a wider range. This capability enables anchor pixels to generate more detailed and accurate reconstructed images by referencing information from more surrounding pixels. However, it still has limitations in capturing the interactions between image channels. While the widely used channel attention mechanism can effectively explore the relationships between channels, it lacks explicit consideration of spatial texture content, which can lead to suboptimal pixel representations in local reconstruction and a lack of refined detail processing.

[0006] Therefore, it is crucial to use Mamba to achieve cross-view information interaction and preserve the global contour and texture detail information within a single image as much as possible to achieve high-quality reconstruction. Summary of the Invention

[0007] To overcome the shortcomings of the above-mentioned prior art, the present invention provides a cross-view binocular image super-resolution reconstruction method and system based on Mamba. The present invention fully utilizes the correspondence between binocular images, refines the image texture and structure at the multi-channel and multi-view levels, and achieves more accurate and high-quality image reconstruction.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a cross-view binocular image super-resolution reconstruction method based on Mamba, characterized by comprising:

[0010] Acquire the binocular image to be reconstructed;

[0011] Input the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed binocular image;

[0012] The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0013] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0014] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0015] In a second aspect, the present invention provides a cross-view binocular image super-resolution reconstruction system based on Mamba, comprising:

[0016] An acquisition module is configured to: acquire a binocular image to be reconstructed;

[0017] A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed binocular image;

[0018] The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0019] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0020] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0021] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0023] One or more of the above technical solutions have the following beneficial effects:

[0024] In this invention, the CSOMM deep feature extraction module models the long-range dependencies between binocular images, extracting the contour and texture information of the left and right view features. The cross-view selective scanning module uses a cross-multiplication mechanism to establish a robust correspondence between the left and right views, fully utilizing the complementary information between the binocular images and further improving the network's reconstruction accuracy and stability. This solution not only fully utilizes the correspondence between binocular images but also refines image texture and structure at the multi-channel and multi-view level, achieving more accurate and high-quality image reconstruction.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is a schematic diagram of the overall network reconstruction in Example 1 of the present invention;

[0028] Figure 2 Schematic diagram of the CSRMB module structure in Example 1 of the present invention;

[0029] Figure 3 Schematic diagram of the VSSM module structure in Example 1 of the present invention;

[0030] Figure 4 Schematic diagram of the CRB module structure in Example 1 of the present invention;

[0031] Figure 5 This is a schematic diagram of the CVIM module structure in Example 1 of the present invention;

[0032] Figure 6 This is a schematic diagram of the CVSSM module structure in Example 1 of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0034] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0035] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0036] Example 1

[0037] This embodiment discloses a cross-view binocular image super-resolution reconstruction method based on Mamba, including:

[0038] Acquire the binocular image to be reconstructed;

[0039] Input the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed binocular image;

[0040] The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0041] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0042] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0043] This embodiment attempts to extend Mamba from a channel and cross-view perspective and proposes a cross-view Mamba network for binocular image super-resolution reconstruction. This embodiment designs a Contour-Tex Synergistic Optimization Mamba Module (CSOMM), which not only models long-range dependencies but also deeply optimizes image texture details and contour information. To further explore channel information and obtain more reference pixel information, a Channel Reinforcement Block (CRB) is designed within CSOMM to specifically address the shortcomings of the channel attention mechanism in processing spatial texture. This module significantly improves the expressiveness of image details by accurately capturing and optimizing the most valuable neighborhood features in each channel. The CSOMM module can capture the global structure of the image and more accurately restore the image's detailed texture during reconstruction, thereby achieving higher-quality image reconstruction. To better handle cross-view information interaction, a Cross View Selective Scan Module (CVSSM) is introduced between CSOMM modules. The CVSSM module focuses on the interaction between the left and right views. Through dynamic scanning and selective attention mechanisms, it can more accurately capture and integrate the complementary information between the left and right views, and establish a more robust binocular image correspondence. Subsequently, this rich cross-view information is passed to the CSOMM module for refined fusion processing to further enhance the cross-view interaction effect. The cross-view Mamba network of this embodiment can not only make full use of the correspondence between binocular images, but also refine the image texture and structure at the multi-channel and multi-view levels to achieve more accurate and high-quality image reconstruction.

[0044] like Figure 1As shown in FIG, the reconstruction network proposed in this embodiment consists of a CSOMM deep feature extraction module, a cross-view feature interaction module (CVIM), and a pixel-shuffle layer for reconstruction.

[0045] The proposed reconstruction network uses two branches to extract left and right view features respectively. Given a pair of low-quality left and right input images, and First, a convolutional layer is used to extract shallow features, and then deep features are extracted through the contour detail collaborative optimization Mamba module (CSOMM). After the deep features are extracted by the CSOMM module, the middle CVIM module performs cross-view information interaction, and then it is divided into two branches, which respectively extract deep features through the contour detail collaborative optimization Mamba module (CSOMM), and then the middle CVIM module performs cross-view information interaction; ... and so on; finally, the output of the convolutional layer and the output of the last CSOMM module are connected through the residual to obtain the final feature and input a pixel-shuffle layer for upsampling reconstruction to obtain the reconstructed output image. and

[0046] In this embodiment, the CSOMM architecture consists of a plurality of sequentially connected Contour-Tex Synergistic Reinforcement Mamba Blocks (CSRMB) and a 1×1 convolution.

[0047] CSRMB extracts the image's contour and texture information by modeling long-range dependencies within a single image. Next, 1×1 convolution optimizes local details. The network not only captures the overall contour of the image, but also finely restores texture details, thereby significantly improving the image reconstruction quality. In the CSRMB module, its structure is as follows: Figure 2 As shown. First, the shallow features (The same applies to the right view) After normalization by LayerNorm, the long-distance dependencies within the image are modeled through the Vision State Space Module (VSSM) to capture the contour information of the image; the VSSM output features are then normalized using LayerNorm, and then a layer of convolution is used to compensate for the local texture information. In order to further explore channel information and obtain more reference pixel information, a channel-aware processing module is inserted here. Although the direct introduction of the channel attention mechanism can effectively explore the relationship between channels, the lack of explicit consideration of the spatial texture content may lead to suboptimal pixel representation in the local texture reconstruction process. Therefore, this embodiment designs a channel refinement module Channel Reinforcement Block (CRB), which focuses on restoring the texture details of the image, capturing and refining the most reference neighborhood features in different channels; finally, the final output of CSRMB is obtained through residual connection The above process can be expressed as:

[0048]

[0049] like Figure 3 As shown in Figure 2, the Visual State Space Module (VSSM) has proven its effectiveness in many visual tasks. Therefore, in this embodiment, its state space equation is used to capture long-range dependencies. Input feature X L ∈R H×W×C It will pass through two branches, through a series of linear projections, depth convolutions DWconv, SiLU activation functions, and use the 2D Selective Scan 2D Module (SS2D) to model the long-range spatial relationships in the features. The above process can be expressed as:

[0050]

[0051] Among them, ⊙ represents element-wise dot product and LN represents LayerNorm normalization layer.

[0052] State-space model (SSM): The state-space model captures the dynamic characteristics of data by explicitly modeling the relationship between observed data and the underlying state. Its characteristic is linear time invariance (LTI). The state-space model consists of two linear equations: the state equation and the observation equation. The state equation describes how the implicit state of the system evolves over time. It usually takes a recursive form, that is, the state at the current moment is a linear combination of the state at the previous moment. The observation equation describes the linear relationship between the observed data and the implicit state of the system, and the observation value is a linear combination of the current state. Their definitions are as follows:

[0053] h'(t)=Ah(t)+Bx(t) (6)

[0054] y(t)=Ch(t)+Dx(t) (7)

[0055] Where x(t)∈R, h(t)∈R N , y(t)∈R, respectively represent the input, the invisible state of x and the output, t represents time. A∈R N×N is the state transition matrix, B∈R N×1 is the control input matrix, C∈R 1×N is the observation matrix, describing how the state vector is mapped to the observation value. D∈R is the observation input matrix (this can usually be ignored, N represents the state size). To facilitate processing discrete sequences such as images, SSM uses the zero-order hold (ZOH) discretization method to transform the input sequence.

[0056] The zero-order hold (ZOH) rule is defined as:

[0057]

[0058] Where Δ∈R D represents the time scale parameter.

[0059] The discretized equation is:

[0060]

[0061] In the SS2D module, the 2D image input features are expanded into four 1D sequences and scanned in four different directions: top left to bottom right, bottom right to top left, top right to bottom left, and bottom left to top right. This selective scanning in these four directions yields multi-directional information. The long-range dependencies of each sequence are then constructed using the state-space equations in the discretization formula of the SSM. Finally, the sequences in the four directions are summed and merged to restore the 2D sequence. This selective scanning mechanism enables Mamba to effectively model complex interactions within long sequences.

[0062] In order to solve the shortcomings of the channel attention mechanism in processing spatial textures, a channel refinement module Channel Reinforcement Block (CRB) is designed, which is specifically used to accurately capture and optimize the most valuable neighborhood features in each channel. Figure 4 Specifically, after extracting long-distance dependencies, the input feature f∈R H×W×CFirst, a channel-by-channel attention mechanism is used to learn representations for different channels. Then, average pooling, max pooling, and convolution operations are sequentially performed in the spatial dimension to refine the features of the local region. Next, the two generated feature maps are added together, and the result is converted to a weight between 0 and 1 using a sigmoid function to selectively focus on the key feature regions in each channel that are most conducive to texture reconstruction. Finally, the output features are integrated through element-by-element multiplication to maximize synergy between channels, allowing the network to retain all key information when processing local details, effectively enhancing the network's expressiveness in detail reconstruction.

[0063] f'=Conv(Max(CA(f)))+Conv(Avg(CA(f))) (13)

[0064] f out =Sigmoid(f')⊙CA(f) (14)

[0065] Among them, Max represents the maximum pooling operation, CA represents the channel attention operation, Avg represents the average pooling operation, and ⊙ represents the element-wise dot product.

[0066] Cross-View Interaction Module (CVIM): In order to effectively establish long-distance correspondences between left and right views, the Cross View Selective Scan Module (CVSSM) adopts a cross-multiplication mechanism to achieve selective cross-view modeling. Figure 5 As shown, CVIMM receives two features of left and right views As input, it generates two outputs while keeping the original shape of the features unchanged. The two input features are first processed by the linear layer and depthwise convolution respectively, and then enter the cross-view selection scanning (CVSSM) module; finally, the output features are obtained after the residual connection is performed by the LayerNorm normalization layer and the linear layer Linear.

[0067]

[0068] In order to strengthen the frequent exchange of cross-view information, an information interaction mechanism between multiple selective scanning modules is designed. In the selective scanning process, the matrix C generated by complementary information is introduced. left and C right , which enables SSM to accurately reconstruct the output of the current view from the hidden state under the guidance of another view.

[0069] CVSSM structure Figure 6 As shown, the process can be described as follows:left and C right In CVSSM, based on the selection mechanism of Mamba, the matrix A left ,B left ,C left ,D left and Δ left According to the calculation method of SSM, A left ∈R N×N is the state transition matrix of the left view, B left ∈R N×1 is the left view control input matrix, C left ∈R 1×N is the left view observation matrix, which describes how the state vector is mapped to the observation value. left ∈R is the left view observation input matrix (this item can usually be ignored, N represents the state size.), Δ left ∈R D Represents the time scale parameter of the left view. Matrix A right ,B right ,C right ,D right and Δ right According to the calculation method of SSM, A right ∈R N×N is the state transition matrix of the right view, B right ∈R N×1 is the right view control input matrix, C right ∈R 1×N is the right view observation matrix, which describes how the state vector is mapped to the observation value. right ∈R is the right view observation input matrix (this item can usually be ignored, N represents the state size.), Δ right ∈R D Represents the time scale parameter of the right view. And according to the above state space model discretization formula, the output is and Specifically, the whole process can be expressed as:

[0070]

[0071] in, Represents input features Represents input features t represents time and h represents the input feature hidden state.

[0072] Loss function: The pixel-level L1 distance loss between the super-resolution image and the ground truth stereo image is used as the loss function, which is expressed as:

[0073]

[0074] in, and Represent the super-resolution left and right images generated by the model, and Ground truth high-resolution images representing them.

[0075] This embodiment introduces the Contour-Tex Synergistic Optimization Mamba Module (CSOMM). The network not only models the long-range dependencies between binocular images but also deeply optimizes both the global structure and local details of the images. This modular design ensures that the reconstruction process not only captures the overall image contour but also accurately restores texture details, significantly improving image reconstruction quality.

[0076] The Channel Reinforcement Block (CRB) in the CSOMM module of this embodiment further enhances information interaction between channels, enabling the network to demonstrate greater adaptability and reconstruction capabilities in complex scenarios. CRB is a key component for enhancing image texture detail recovery, aiming to address the limitations of the channel attention mechanism in processing spatial textures.

[0077] This embodiment establishes a robust correspondence between the left and right views through the Cross View Selective Scan Module (CVSSM), which fully utilizes the complementary information between the binocular images and further improves the reconstruction accuracy and stability of the network.

[0078] This embodiment uses peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as quantitative evaluation indicators. These indicators are calculated in the RGB color space between a pair of binocular images (ie, (Left+Right) / 2).

[0079] As shown in Table 1, the method proposed in this embodiment is compared with existing stereo SR methods at a magnification factor of ×4. The following table summarizes the SR comparison results obtained by different stereo SR methods on the public Flickr 1024 dataset. As shown in Table 1, the method of this embodiment achieves superior results.

[0080] Table 1 Comparison of the method in this embodiment with other algorithms

[0081]

[0082]

[0083] Example 2

[0084] The purpose of this embodiment is to provide a cross-view binocular image super-resolution reconstruction system based on Mamba, including:

[0085] An acquisition module is configured to: acquire a binocular image to be reconstructed;

[0086] A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed binocular image;

[0087] The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0088] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0089] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0090] In further embodiments, there is also provided:

[0091] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor. When the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0092] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0093] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0094] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in embodiment 1 is performed.

[0095] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0096] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in embodiment 1 is implemented.

[0097] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions contained in program modules, which are executed in a device on a real or virtual processor of a target to perform the process / method described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided between program modules as needed. The machine-executable instructions for the program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0098] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the computer or other programmable data processing device, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0099] In the context of the present invention, computer program code or related data can be carried by any appropriate carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0100] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A cross-view binocular image super-resolution reconstruction method based on Mamba, characterized by: include: Acquire the binocular image to be reconstructed; Input the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed super-resolution binocular image; The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules; The CSOMM deep feature extraction module is used to model the long-range dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features; wherein the CSOMM deep feature extraction module is composed of a plurality of sequentially connected contour detail collaborative refinement Mamba modules and convolutions; The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

2. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 1, characterized in that: The operation process of the contour detail collaborative refinement Mamba module on the shallow input features is specifically as follows: Use the LayerNorm layer to normalize the shallow input features; The visual state space module is used to model the long-range dependencies within the normalized shallow input features to capture the contour information of the image. After normalizing the output of the visual state space module using LayerNorm, convolution is used to compensate for local texture information; A channel-aware processing module is introduced to capture and refine the most informative neighborhood features in different channels.

3. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 2, characterized in that: The visual state space module models the long-range dependencies within the normalized shallow input features to capture the contour information of the image. Specifically: The input features are processed by the first branch and the second branch in parallel respectively; The input features are sequentially processed by the linear projection layer, the depth convolution layer, the SiLU activation function, and the 2D selective scanning block in the first branch to model the long-range spatial relationship in the input features; The input features are processed sequentially by the linear projection layer and the SiLU activation function in the second branch; The output features of the first branch and the output features of the second branch are fused and normalized to obtain the final output of the visual state space module.

4. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 3, characterized in that: The 2D selection scanning block operates on its 2D image input features by: expanding the 2D image input features into four 1D sequences, and scanning in four different directions: from upper left to lower right, from lower right to upper left, from upper right to lower left, and from lower left to upper right; The long-range dependency of each sequence is constructed according to the state space equation in the discretization formula of SSM; the sequences in the four directions are summed and merged to restore the 2D sequence.

5. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 1, characterized in that: The channel-aware processing module is introduced to capture and refine the most relevant neighborhood features in different channels. Specifically: ， in, is the input feature of the channel perception processing module, represents the channel attention operation, represents the average pooling operation, represents the element-wise dot product, represents the maximum pooling operation, Represents the output of the channel-aware processing module.

6. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 1, characterized in that: The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and right view features processed by the CSOMM deep feature extraction module, specifically: in, are the input features of the left view and right view of the cross-view feature interaction module, is a linear layer, is the depthwise convolution, is the normalization layer, Select the Scan module for the span view.

7. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 6, characterized in that: The cross-view selection scanning module operates on its input features as follows: in, The state transition matrix of the left view, is the left view control input matrix, is the left view observation matrix, which describes how the state vector is mapped to the observation value, is the left view observation input matrix, represents the time scale parameter of the left view; is the state transition matrix of the right view, is the right view control input matrix, is the right view observation matrix, which describes how the state vector is mapped to the observation value, is the right view observation input matrix, represents the time scale parameter of the right view, t represents time, represents the left view input feature, Represents the right view input feature.

8. A cross-view binocular image super-resolution reconstruction system based on Mamba, characterized by: include: An acquisition module is configured to: acquire a binocular image to be reconstructed; A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed super-resolution binocular image; The reconstruction network model includes multiple cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules; The CSOMM deep feature extraction module is used to model the long-range dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features; wherein the CSOMM deep feature extraction module is composed of a plurality of sequentially connected contour detail collaborative refinement Mamba modules and convolutions; The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling on the left view features and the right view features processed by the CSOMM deep feature extraction module.

9. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 7 is completed.

10. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the method according to any one of claims 1 to 7.

Images