Cross-view binocular image super-resolution reconstruction method and system based on Mama

Abstract

The invention provides a Mama-based cross-view binocular image super-resolution reconstruction method and system, and the method comprises the steps: carrying out the modeling of a binocular image long-distance dependence relation through a CSOMM deep feature extraction module, and extracting the contour and texture information of a left view feature and a right view feature; a robust corresponding relation is established between a left view and a right view through a cross-view selection scanning module by adopting a cross multiplication mechanism, so that complementary information between binocular images is fully utilized, and the reconstruction precision and stability of the network are further improved; and the pixel shuffling layer is used for performing up-sampling reconstruction on the output of the cross-view feature interaction module to obtain a reconstructed binocular image. According to the scheme, the corresponding relation between binocular images can be fully utilized, the image texture and structure can be finely processed on the multi-channel and multi-view level, and more accurate and high-quality image reconstruction is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image reconstruction, and in particular relates to a Mamba-based cross-view binocular image super-resolution reconstruction method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The vigorous development of binocular imaging technology has promoted the widespread application of binocular images in many fields such as autonomous driving, robot navigation, medical imaging, and film and television production. In these applications, by improving the resolution of binocular images, eye-catching visual effects can be generated, thereby significantly optimizing the performance of these tasks. The core goal of binocular image super-resolution reconstruction is to make full use of the rich information of two perspectives, finely restore the image pixels, and generate high-resolution and detail-rich images. Although binocular images provide richer supplementary information than monocular images, they also pose additional challenges. The main thing is how to accurately establish a reliable binocular correspondence, while ensuring image consistency, fully explore and utilize the potential information in the binocular image, so as to achieve excellent visual reconstruction effects.

[0004] In solving the problem of binocular image super-resolution reconstruction, the methods proposed currently mainly rely on convolutional neural networks (CNNs) and Transformers, and both have achieved excellent performance. Due to the locality of the convolution operation, the CNN-based method has high computational efficiency when processing images. However, CNN uses local receptive fields to capture image features. Limited by the size of the convolution kernel, the receptive field range is also small, and it is impossible to establish effective global dependencies. In contrast, the Transformer-based method uses the global receptive field for global interaction modeling to better capture complex global relationships, allowing the network to refer to more pixels for image reconstruction. However, the Transformer provides a global receptive field at the cost of quadratic computational complexity, which brings a computational burden to the training of the model. Although researchers have tried to use effective techniques to improve the efficiency of the model, such as the moving window mechanism, the effective range of the receptive field is sacrificed accordingly, which may result in the loss of some pixel information.

[0005] Recently, the selective state space model, Mamba, has attracted widespread attention from researchers due to its linear computational cost and long-distance modeling capabilities of the global receptive field. More and more researchers have explored the potential of Mamba in various fields and achieved promising results in various downstream tasks. Mamba is able to extract global spatial context information excellently due to its wide global receptive field. In the image reconstruction process, this means that anchor pixels in the image can be associated with pixel information in a wider range. This ability enables anchor pixels to generate more detailed and accurate reconstructed images based on more pixel information around them. However, it still has certain limitations in capturing the interactive relationship between image channels. Although the currently widely used channel attention mechanism can effectively explore the relationship between channels, it lacks explicit consideration of spatial texture content, which may lead to suboptimal pixel representation in the local reconstruction process, making the details less refined.

[0006] Therefore, it is crucial to use Mamba to achieve cross-view information interaction and preserve the global contour and texture detail information in a single image as much as possible to achieve high-quality reconstruction. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a cross-view binocular image super-resolution reconstruction method and system based on Mamba. The present invention makes full use of the correspondence between binocular images, refines the image texture and structure at the multi-channel and multi-view levels, and realizes more accurate and high-quality image reconstruction.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a cross-view binocular image super-resolution reconstruction method based on Mamba, characterized in that it includes:

[0010] Acquire a binocular image to be reconstructed;

[0011] Inputting the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed binocular image;

[0012] Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0013] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0014] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0015] In a second aspect, the present invention provides a cross-view binocular image super-resolution reconstruction system based on Mamba, comprising:

[0016] An acquisition module is configured to: acquire a binocular image to be reconstructed;

[0017] A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed binocular image;

[0018] Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0019] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0020] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0021] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0023] One or more of the above technical solutions have the following beneficial effects:

[0024] In the present invention, the long-distance dependency of binocular images is modeled through the CSOMM deep feature extraction module, and the contour and texture information of the left view feature and the right view feature are extracted; a robust correspondence relationship is established between the left and right views by using the cross-multiplication mechanism through the cross-view selection scanning module, so that the complementary information between the binocular images is fully utilized, further improving the reconstruction accuracy and stability of the network. The scheme of the present invention can not only make full use of the correspondence between binocular images, but also refine the image texture and structure at the multi-channel and multi-view level to achieve more accurate and high-quality image reconstruction.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is an overall schematic diagram of the network reconstruction in the first embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of the CSRMB module in the first embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the VSSM module structure in Embodiment 1 of the present invention;

[0030] Figure 4 This is a schematic diagram of the CRB module structure in the first embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the structure of the CVIM module in the first embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the CVSSM module structure in Example 1 of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0034] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0035] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0036] Embodiment 1

[0037] This embodiment discloses a cross-view binocular image super-resolution reconstruction method based on Mamba, comprising:

[0038] Acquire a binocular image to be reconstructed;

[0039] Inputting the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed binocular image;

[0040] Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0041] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0042] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0043] This embodiment attempts to extend Mamba from the perspective of channels and cross-views, and proposes a cross-view Mamba network for binocular image super-resolution reconstruction. This embodiment designs a contour detail collaborative optimization Mamba module Contour-Tex Synergistic Optimization Mamba Module (CSOMM), which is not only used for modeling long-distance dependencies, but also can perform deep optimization on the texture details and contour information of the image. In order to further explore channel information and obtain more reference pixel information, a channel refinement module Channel ReinforcementBlock (CRB) is designed in CSOMM, which is specifically used to solve the shortcomings of the channel attention mechanism in processing spatial textures. This module greatly improves the expressiveness of image details by accurately capturing and optimizing the most valuable neighborhood features in each channel. The CSOMM module can capture the global structure of the image and restore the detailed texture of the image more accurately during the reconstruction process, thereby achieving a higher quality image reconstruction effect. In order to better handle cross-view information interaction, a cross-view selective scanning module Cross View Selective Scan Module (CVSSM) is introduced between CSOMM modules. The CVSSM module focuses on the interaction between the left and right views. Through dynamic scanning and selective attention mechanisms, it can more accurately capture and integrate the complementary information between the left and right views, and establish a more robust binocular image correspondence. Subsequently, these rich cross-view information is passed to the CSOMM module for refined fusion processing to further enhance the cross-view interaction effect. The cross-view Mamba network of this embodiment can not only make full use of the correspondence between binocular images, but also refine the image texture and structure at the multi-channel, multi-view level to achieve more accurate and high-quality image reconstruction.

[0044] like Figure 1As shown, the reconstruction network proposed in this embodiment consists of a CSOMM deep feature extraction module, a cross-view feature interaction module (CVIM), and a pixel-shuffle layer for reconstruction.

[0045] The proposed reconstruction network uses two branches to extract left and right view features respectively. Given a pair of low-quality left and right input images, and First, a convolutional layer is used to extract shallow features, and then the deep features are extracted through the contour detail collaborative optimization Mamba module (CSOMM); after the deep features are extracted by the CSOMM module, the cross-view information interaction is performed by the middle CVIM module, and then it is divided into two branches, respectively, the deep features are extracted by the contour detail collaborative optimization Mamba module (CSOMM), and the cross-view information interaction is performed by the middle CVIM module; ... and so on; finally, the output of the convolutional layer and the output of the last CSOMM module are connected through the residual to obtain the final feature and input into a pixel-shuffle layer for upsampling reconstruction to obtain the reconstructed output image and

[0046] In this embodiment, the CSOMM architecture consists of a plurality of sequentially connected contour-detail collaborative refinement Mamba blocks Contour-Tex Synergistic Reinforcement Mamba Blocks (CSRMB) and a 1×1 convolution.

[0047] CSRMB extracts the contour and texture information of an image by modeling the long-distance dependencies within a single image. Next, 1×1 convolution is used to optimize local details. The network can not only capture the overall contour of the image, but also finely restore the texture details, thereby significantly improving the quality of image reconstruction. In the CSRMB module, its structure is as follows: Figure 2 First, the shallow features (The same applies to the right view) After normalization by LayerNorm, the long-distance dependencies within the image are modeled through the Vision State Space Module (VSSM) to capture the contour information of the image; the VSSM output features are then normalized using LayerNorm, and then a layer of convolution is used to compensate for the local texture information. In order to further explore channel information and obtain more reference pixel information, a channel-aware processing module is inserted here. Although the direct introduction of the channel attention mechanism can effectively explore the relationship between channels, the lack of explicit consideration of the spatial texture content may lead to suboptimal pixel representation in the local texture reconstruction process. Therefore, this embodiment designs a channel refinement module Channel Reinforcement Block (CRB), which focuses on restoring the texture details of the image, capturing and refining the most reference neighborhood features in different channels; finally, the final output of CSRMB is obtained through residual connection. The above process can be expressed as:

[0048]

[0049] like Figure 3 As shown in Fig. 1, the visual state space module (VSSM) has proved its effectiveness in many visual tasks. Therefore, in this embodiment, its state space equation is used to capture long-range dependencies. Input feature X L ∈R H×W×C It will go through two branches, a series of linear projections, deep convolutions DWconv, SiLU activation functions, and use the 2D Selective Scan 2D Module (SS2D) to model the long-range spatial relationship in the features. The above process can be expressed as:

[0050]

[0051] Among them, ⊙ represents element-wise dot product and LN represents LayerNorm normalization layer.

[0052] State Space Model (SSM): The state space model captures the dynamic characteristics of data by explicitly modeling the relationship between observed data and latent states, and its characteristic is linear time invariance (LTI). The state space model consists of two linear equations: the state equation and the observation equation. The state equation describes how the implicit state of the system evolves over time. It usually takes a recursive form, that is, the state at the current moment is a linear combination of the state at the previous moment. The observation equation describes the linear relationship between the observed data and the implicit state of the system, and the observation value is a linear combination of the current state. Their definitions are as follows:

[0053] h'(t)=Ah(t)+Bx(t) (6)

[0054] y(t)=Ch(t)+Dx(t) (7)

[0055] where x(t)∈R, h(t)∈R N , y(t)∈R, respectively representing the input, the invisible state of x and the output, and t represents time. A∈R N×N is the state transition matrix, B∈R N×1 is the control input matrix, C∈R 1×N is the observation matrix, describing how the state vector is mapped to the observation value. D∈R is the observation input matrix (this item can usually be ignored, N represents the state size.) In order to facilitate the processing of discrete sequences such as images, SSM uses the zero-order hold (ZOH) discretization method to transform the input sequence.

[0056] The zero-order hold (ZOH) rule is defined as:

[0057]

[0058] Where Δ∈R D Represents the time scale parameter.

[0059] The discretized equation is:

[0060]

[0061] In the SS2D module, the 2D image input features are expanded into four 1D sequences and scanned in four different directions: from top left to bottom right, bottom right to top left, top right to bottom left, and bottom left to top right. Multi-directional information is obtained through selective scanning in four directions, and then the long-range dependency of each sequence is constructed according to the state space equation in the discretization formula in SSM. Finally, the sequences in the four directions are summed and merged to restore the 2D sequence. Through this scanning selection mechanism, Mamba can effectively model the complex interactive relationships in long sequences.

[0062] In order to solve the shortcomings of the channel attention mechanism in processing spatial textures, a channel refinement module Channel Reinforcement Block (CRB) is designed specifically for this module to accurately capture and optimize the most valuable neighborhood features in each channel. Figure 4 Specifically, after extracting long-distance dependencies, the input feature f∈R H×W×CFirst, the representation of different channels is learned through the channel attention mechanism, and then average pooling, maximum pooling and convolution operations are performed in sequence in the spatial dimension to refine the features of the local area. Next, the two generated feature maps are added together, and the result is converted into a weight between 0 and 1 through the sigmoid function to selectively focus on the key feature areas in each channel that are most helpful for texture reconstruction. Finally, the output features are integrated through element-by-element multiplication to ensure that the synergy between channels is maximized, so that the network can retain all key information when processing local details, effectively enhancing the network's expressiveness in detail reconstruction.

[0063] f'=Conv(Max(CA(f)))+Conv(Avg(CA(f))) (13)

[0064] f out =Sigmoid(f')⊙CA(f) (14)

[0065] Among them, Max represents the maximum pooling operation, CA represents the channel attention operation, Avg represents the average pooling operation, and ⊙ represents the element-wise dot product.

[0066] Cross-View Interaction Module (CVIM): In order to effectively establish the long-distance correspondence between the left and right views, the Cross-View Selective Scan Module (CVSSM) adopts a cross-multiplication mechanism to achieve selective cross-view modeling. Figure 5 As shown, CVIMM receives two features of the left and right views As input, it generates two outputs, and keeps the original shape of the feature unchanged. The two input features are first processed by the linear layer and the depth convolution, and then enter the cross-view selection scanning (CVSSM) module; finally, the output features are obtained after residual connection through the LayerNorm normalization layer and the linear layer Linear

[0067]

[0068] In order to strengthen the frequent exchange of cross-view information, an information interaction mechanism between multiple selective scanning modules is designed. In the selective scanning process, the matrix C generated by complementary information is introduced. left and C right , which enables the SSM to accurately reconstruct the output of the current view from the hidden state under the guidance of the other view.

[0069] CVSSM Structure Figure 6 As shown, the process can be described as follows:left and C right The SSM can effectively fuse the key information from another view in each selective scan, thereby achieving more accurate feature correspondence and information integration in cross-view modeling. In CVSSM, based on the selection mechanism of Mamba, the matrix A left ,B left ,C left ,D left and Δ left According to the calculation method of SSM, A left ∈R N×N is the state transition matrix of the left view, B left ∈R N×1 is the left view control input matrix, C left ∈R 1×N is the left view observation matrix, describing how the state vector is mapped to the observation value. left ∈R is the left view observation input matrix (this item can usually be ignored, N represents the state size.), Δ left ∈R D Represents the time scale parameter of the left view. Matrix A right ,B right ,C right ,D right and Δ right According to the calculation method of SSM, A right ∈R N×N is the state transition matrix of the right view, B right ∈R N×1 is the right view control input matrix, C right ∈R 1×N is the right view observation matrix, describing how the state vector is mapped to the observation value. right ∈R is the right view observation input matrix (this item can usually be ignored, N represents the state size.), Δ right ∈R D Represents the time scale parameter of the right view. And according to the above state space model discretization formula, the output is and Specifically, the whole process can be expressed as:

[0070]

[0071] in, Represents input features Represents input features t represents time and h represents the input feature hidden state.

[0072] Loss function: The pixel-level L1 distance loss between the super-resolution image and the ground truth stereo image is used as the loss function, which is expressed as:

[0073]

[0074] in, and Represent the super-resolution left and right images generated by the model, and Ground truth high-resolution images representing them.

[0075] This embodiment introduces the Contour-Tex Synergistic Optimization Mamba Module (CSOMM). The network of this embodiment not only models the long-distance dependency of binocular images, but also deeply optimizes the global structure and local details of the image. This module design ensures that during the reconstruction process, not only the overall contour of the image can be captured, but also the texture details can be finely restored, thereby significantly improving the reconstruction quality of the image.

[0076] The channel reinforcement block (CRB) in the CSOMM module of this embodiment further enhances the information interaction between channels, making the network more adaptable and reconstructive when dealing with complex scenes. CRB is a key component for enhancing image texture detail recovery, and is designed to address the limitations of the channel attention mechanism when processing spatial textures.

[0077] This embodiment establishes a robust correspondence between the left and right views through the Cross View Selective Scan Module (CVSSM), so that the complementary information between the binocular images is fully utilized, further improving the reconstruction accuracy and stability of the network.

[0078] This embodiment uses peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as quantitative indicators for evaluation, and these indicators are calculated in the RGB color space between a pair of binocular images (ie, (Left+Right) / 2).

[0079] As shown in Table 1, the method proposed in this embodiment is compared with the existing stereo SR method at a magnification factor of ×4. As summarized in the following table, the SR comparison results obtained by different stereo SR methods on the public dataset Flickr 1024 dataset are shown. As can be seen from Table 1, the method of this embodiment achieves better results.

[0080] Table 1 Comparison of the method in this embodiment with other algorithms

[0081]

[0082]

[0083] Embodiment 2

[0084] The purpose of this embodiment is to provide a cross-view binocular image super-resolution reconstruction system based on Mamba, including:

[0085] An acquisition module is configured to: acquire a binocular image to be reconstructed;

[0086] A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed binocular image;

[0087] Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules;

[0088] The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features;

[0089] The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

[0090] In further embodiments, there is also provided:

[0091] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, it will not be described in detail here.

[0092] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0093] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0094] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0095] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0096] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0097] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0098] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.

[0099] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0101] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A cross-view binocular image super-resolution reconstruction method based on Mamba, characterized in that: include: Acquire a binocular image to be reconstructed; Inputting the binocular image to be reconstructed into the trained reconstruction network model to obtain the reconstructed binocular image; Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules; The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features; The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

2. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 1, characterized in that: The CSOMM deep feature extraction module includes a plurality of CSRMB modules and convolutional layers with residual connections; the operation process of the CSRMB module on the shallow input features is specifically as follows: Use the LayerNorm layer to normalize the shallow input features; The visual state space module is used to model the long-distance dependencies within the normalized shallow input features to capture the contour information of the image. After normalizing the output of the visual state space module using LayerNorm, convolution is used to compensate for local texture information; A channel-aware processing module is introduced to capture and refine the most informative neighborhood features in different channels.

3. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 2, characterized in that: The visual state space module models the long-distance dependencies within the normalized shallow input features to capture the contour information of the image, specifically: The input features are processed by the first branch and the second branch in parallel respectively; The input features are processed in sequence by a linear projection layer, a deep convolution layer, a SiLU activation function, and a 2D selective scanning block in the first branch to model long-range spatial relationships in the input features; The input features are processed sequentially by the linear projection layer and the SiLU activation function in the second branch; The output features of the first branch and the output features of the second branch are fused and normalized to obtain the final output of the visual state space module.

4. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 3, characterized in that: The 2D selection scanning block operates and processes its 2D image input features, specifically: expands the 2D image input features into four 1D sequences, and scans in four different directions from upper left to lower right, lower right to upper left, upper right to lower left, and lower left to upper right; The long-range dependency of each sequence is constructed according to the state-space equation in the discretization formula of SSM; the sequences in the four directions are summed and merged to restore the 2D sequence.

5. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 2, characterized in that: The channel-aware processing module is introduced to capture and refine the most informative neighborhood features in different channels, specifically: f'=Conv(Max(CA(f)))+Conv(Avg(CA(f))), f out =Sigmoid(f')⊙CA(f), Among them, f is the input feature of the channel perception processing module, CA represents the channel attention operation, Avg represents the average pooling operation, ⊙ represents the element dot product, Max represents the maximum pooling operation, and f out Represents the output of the channel-aware processing module.

6. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 1, characterized in that: The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module, specifically: in, They are the input features of the left view and right view of the cross-view feature interaction module, Linear is the linear layer, DWConv is the deep convolution, LN is the normalization layer, and CVSSM is the cross-view selection scanning module.

7. The Mamba-based cross-view binocular image super-resolution reconstruction method according to claim 6, characterized in that: The operation process of the cross-view selection scanning module on its input features is as follows: Among them, A left The state transition matrix of the left view, B left is the left view control input matrix, C left is the left view observation matrix, describing how the state vector is mapped to the observation value, D left is the left view observation input matrix (Δ left ∈R D Represents the time scale parameter of the left view; A right is the state transition matrix of the right view, B right is the right view control input matrix, C right is the right view observation matrix, describing how the state vector is mapped to the observation value, D right is the right view observation input matrix, Δ right ∈R D represents the time scale parameter of the right view, t represents time, represents the left view input feature, Represents the right view input feature.

8. A cross-view binocular image super-resolution reconstruction system based on Mamba, characterized in that: include: An acquisition module is configured to: acquire a binocular image to be reconstructed; A reconstruction module is configured to: input the binocular image to be reconstructed into a trained reconstruction network model to obtain a reconstructed binocular image; Wherein, the reconstruction network model includes a plurality of cross-connected CSOMM deep feature extraction modules and cross-view feature interaction modules; The CSOMM deep feature extraction module is used to model the long-distance dependency relationship within the left view features and the right view features, respectively, and extract the contour and texture information of the left view features and the right view features; The cross-view feature interaction module uses a cross-multiplication mechanism to implement cross-view modeling for the left view features and the right view features processed by the CSOMM deep feature extraction module.

9. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 7 is completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 7.

Images