Low-light stereo image enhancement method based on context-guided and structure-awareness
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- COLLEGE OF SCI & TECH NINGBO UNIV
- Filing Date
- 2026-06-24
- Publication Date
- 2026-07-21
Smart Images
Figure CN122434804A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and image processing technology, specifically to a low-light stereo image enhancement method based on context guidance and structure awareness. Background Technology
[0002] With the deployment of vision devices in fields such as autonomous driving and robot navigation, stereo imaging has become an important research direction in computer vision because it can provide geometric information for downstream tasks such as 3D scene reconstruction and depth estimation. However, stereo images acquired in low-light environments are prone to severe noise occlusion and contrast degradation, leading to problems such as increased loss of local details and difficulty in maintaining parallax consistency.
[0003] To improve image quality in low-light environments, existing low-light image enhancement methods are mainly divided into two categories: monocular image enhancement and stereo image enhancement. Traditional monocular image enhancement methods rely on single-view processing, and when directly applied to stereo images, they easily lose the complementary characteristics of binocular vision, making it difficult to maintain the disparity consistency of the reconstructed image. Therefore, existing low-light stereo image enhancement methods typically utilize disparity correlation, using information from the reference view through cross-view interaction to compensate for the loss of detail in the target view due to insufficient lighting.
[0004] While existing cross-view interaction methods have improved restoration quality to some extent, they still have significant limitations in complex low-light degradation scenarios. In real-world low-light and high-noise environments, image degradation leads to a significant decrease in the recognizability of local features in the left and right views. Existing interaction mechanisms typically rely directly on high-precision matching of corresponding point features. This matching method is prone to introducing irrelevant noise and incorrect matching associations when features are contaminated, making it difficult to form stable contextual complementarity guidance in degraded regions. Furthermore, existing stereo feature complementarity methods are mostly strictly limited to point-to-point interaction within a single epipolar line in the horizontal direction, ignoring the two-dimensional geometric continuity of the target feature's neighborhood. When the target view experiences local occlusion, severe texture distortion, or structural damage, relying solely on single-dimensional information in the epipolar direction cannot provide a complete structural reference, easily leading to problems such as local texture damage, blurred edges, or the propagation of incorrect structures in the enhanced image. In addition, existing networks mostly focus on spatial dimension region feature alignment, lacking long-distance dependency perception of feature channel sequences when dealing with global illumination unevenness and color degradation in low-light images, thus limiting the ability to restore the overall brightness and color distribution of the image. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides a low-light stereo image enhancement method based on context guidance and structure awareness, aiming to solve the problems of illuminance degradation, detail loss, and structural inconsistency between viewpoints in stereo images under low-light conditions.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a low-light stereo image enhancement method based on context guidance and structure awareness, comprising the following steps: Acquire low-light stereo image pairs, extract features from the low-light stereo image pairs, and obtain left view features and right view features; The left view features and right view features are used as target view features and reference view features, respectively. Cross-view context features are extracted based on epipolar constraints, and epipolar interaction features are output. Based on epipolar interaction features, a two-dimensional structural representation corresponding to the reference view features is extracted, and the two-dimensional structural representation is injected into the latent state evolution process corresponding to the target view features through gating modulation, and the structural enhancement features are output. Channel-assisted enhancement is performed on the structural enhancement features, the channel-enhanced features are output, and the channel-enhanced features are input into the decoder to perform image reconstruction, outputting enhanced stereo image pairs.
[0007] Preferably, the steps of acquiring low-light stereo image pairs, extracting features from the low-light stereo image pairs, and obtaining left-view features and right-view features include: receiving left-view images and right-view images in a low-light environment acquired by an image acquisition device to form low-light stereo image pairs; extracting basic features from the low-light stereo image pairs through a shallow convolutional network set at the input end of the cross-view state space network to output initial left-view features and initial right-view features; and inputting the initial left-view features and initial right-view features as left-view features and right-view features in parallel into the context-guided and structure-aware cross-view blocks cascaded within the cross-view state space network, and distributing them to the corresponding left-view processing paths and right-view processing paths via the internal dual-branch parallel structure.
[0008] Preferably, the process of extracting cross-view context features based on epipolar constraints includes: collectively referring to the left view features and right view features as input view features; sequentially performing linear mapping operations, depthwise separable convolution operations, and activation function processing on the input view features to generate shallow enhancement features; flattening the shallow enhancement features according to spatial row priority order to construct a one-dimensional feature sequence; for the current scan position index, extracting the center feature corresponding to the target view feature along the horizontal epipolar direction as the query vector, and extracting local window features in the same horizontal row epipolar neighborhood of the one-dimensional feature sequence corresponding to the reference view feature as a candidate set; calculating the normalized similarity between the query vector and the candidate set and obtaining attention weights; and performing weighted aggregation on the candidate set based on the attention weights to obtain cross-view context features.
[0009] In one specific embodiment, the process of outputting epipolar interaction features includes: merging and concatenating the local window features at the corresponding position of the target view feature, the corresponding cross-view context features, the absolute difference between the two, and their element-wise product in the channel dimension; sequentially inputting the multidimensional tensor obtained by concatenation into a convolutional network to output gating coefficients; performing element-wise multiplication operations on the gating coefficients and the cross-view context features, and adding the processed feature data to the local window features at the corresponding position of the target view feature to construct enhanced features; feeding the enhanced features into a parameter mapping network composed of convolutional layers to generate state mapping parameters for the target view state space; inputting the original features corresponding to the target view feature in parallel into an independently set mapping network to generate stride parameters and output matrix parameters; combining the hidden state accumulated at the previous scan position, the state mapping parameters at the current position, and the enhanced features to iteratively calculate the current hidden state and map it to the output result; rearranging the output sequence composed of the output results back into a two-dimensional feature map, and outputting the final feature result as the epipolar interaction feature through residual connections.
[0010] Preferably, the process of extracting the two-dimensional structural representation corresponding to the reference view features includes: feeding the epipolar interaction features into a shallow processing network composed of a linear mapping layer and a deep convolutional layer, performing nonlinear feature enhancement in conjunction with an activation function to obtain the basic input features for state space modeling; after flattening the basic input features into a one-dimensional sequence along the spatial dimension, performing a forward selective scan operation to obtain the corresponding baseline hidden state sequence; performing one-dimensional to two-dimensional reconstruction and restoration on the baseline hidden state sequence, then performing a local neighborhood unfolding operation, and extracting the structural representation of the corresponding view through a spatial channel information mining block, using the structural representation as the two-dimensional structural representation corresponding to the reference view features.
[0011] Preferably, the two-dimensional structural representation is injected into the latent state evolution process corresponding to the target view features through gated modulation, including: Perform the same forward selective scan operation on the target view feature as on the reference view feature, and obtain the center feature of the corresponding target hidden state sequence as the hidden state center feature corresponding to the target view feature; Based on the mapping comparison between the hidden state center features corresponding to the target view features and the two-dimensional structural representation corresponding to the reference view features, a similarity mapping result is generated. The similarity mapping result is then processed by an activation function to generate gating coefficients.
[0012] In one specific embodiment, the process of outputting structural enhancement features includes: during the time-step update of the state space model, multiplying the two-dimensional structural representation corresponding to the reference view feature by a gating coefficient and performing serialization dimensionality reduction and alignment, and then superimposing it as a modulation compensation term onto the hidden state sequence corresponding to the target view feature to generate one-dimensional output sequence response values at each time step; summarizing the one-dimensional output sequence response values at each time step and reconstructing them inversely into a two-dimensional tensor to generate intermediate sequence features; performing projection restoration on the intermediate sequence features using layer normalization and pointwise convolution, and performing residual multiplication in conjunction with the direct path where the input epipolar interaction features are located to output structural enhancement features.
[0013] Preferably, the process of performing channel-assisted enhancement on structural enhancement features and outputting channel-enhanced features includes: using a transpose operation to change the dimensional arrangement logic of the input structural enhancement features, performing axis swapping between the spatial unfolded sequence and the channel dimensions to obtain the transposed tensor; performing a one-dimensional forward selective scan on the transposed tensor to obtain the channel state aggregation sequence; performing an inverse transpose operation to restore the initial spatial and channel arrangement format, and combining the direct jump path with the input structural enhancement features to perform element-wise addition to output the channel-enhanced features.
[0014] Preferably, the process of inputting channel enhancement features into the decoder to perform image reconstruction and outputting enhanced stereo image pairs includes: using a transposed convolutional layer to perform spatial resolution magnification on the channel enhancement features to obtain an upsampled tensor; extracting shallow features in the network that are at the same resolution scale as the current decoding level, and aligning the shallow features and the upsampled tensor in the channel dimension through cross-layer skip connection paths to obtain a stitched tensor; inputting the stitched tensor into the decoding block to perform channel dimensionality reduction and local information recovery, and repeating the above upsampling and stitching fusion operations until the spatial resolution of the feature tensor is restored to the initial input size to generate a decoded feature tensor; applying a mapping convolutional layer to compress the number of channels of the decoded feature tensor to a three-channel specification to generate a residual reconstruction tensor; and using a global residual learning mechanism to superimpose the original input view corresponding to the low-light stereo image pair in a low-light degradation state onto the residual reconstruction tensor to output the enhanced stereo image pair.
[0015] In one specific embodiment, before acquiring low-light stereo image pairs, the method further includes an end-to-end training step using the paired stereo image dataset to train the entire network: constructing a training sample set containing low-light stereo image pairs and corresponding normal exposure reference image pairs; inputting the low-light stereo image pairs into the network, and outputting predicted stereo image pairs through feature extraction, epipolar interaction, structural compensation, and decoding reconstruction processes; using the L1 norm as the loss function to calculate the absolute numerical error between the predicted stereo image pairs and the normal exposure reference image pairs at corresponding pixel coordinate positions; and dynamically updating the mapping weight parameters inside the network based on the calculated error gradient through the backpropagation algorithm until the loss function value converges to a set interval, thus completing the network training process.
[0016] This invention provides a low-light stereo image enhancement method based on context guidance and structure awareness. It has the following beneficial effects: 1. This invention generates cross-view context features by extracting candidate features from the reference view under epipolar constraints and performing similarity-weighted aggregation. It then constructs enhanced features by combining gating coefficients and encodes them as state mapping parameters of the target view. This overcomes the technical defect of reduced discriminability of binocular features due to severe image degradation in low-light environments, reduces the dependence on high-precision matching features, and improves the stability of cross-view information interaction under low-light conditions while suppressing degradation noise interference.
[0017] 2. This invention extracts a two-dimensional structural representation of the reference view by performing local neighborhood expansion on the hidden state sequence, and injects this two-dimensional structural representation as a modulation term into the state evolution process of the target view under the constraint of similarity gating. This overcomes the limitation of traditional cross-view interaction, which is limited to one-dimensional features within the horizontal row, resulting in restricted structural reconstruction. It extends cross-view compensation from the epipolar direction to the local two-dimensional structure, reduces the propagation of erroneous information in areas with disparity shift or local occlusion differences, and achieves effective recovery of damaged edges and texture information.
[0018] 3. After completing cross-view interaction in the spatial dimension, this invention uses transpose operation and one-dimensional forward selective scanning to obtain channel state aggregation sequence, and combines direct jump path to perform auxiliary enhancement of channel dimension, which supplements the model's perception ability in global feature aggregation, synergistically improves the recovery quality of stereo images for global brightness and color distribution, effectively maintains the geometric consistency between binocular images, and reduces the disparity estimation error of stereo images in downstream application scenarios. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the overall network architecture according to an embodiment of the present invention.
[0020] Figure 2This is a schematic diagram of a context-guided cross-view interaction module structure according to an embodiment of the present invention.
[0021] Figure 3 This is a flowchart of the internal processing of a structure-aware state space module according to an embodiment of the present invention.
[0022] Figure 4 A visualization comparison of the augmentation results and MSE plot on the Flickr1024 dataset.
[0023] Figure 5 The image shows the enhanced results on the KITTI2012 and KITTI2015 datasets.
[0024] Figure 6 This is a visual comparison chart of MSE graphs.
[0025] Figure 7 A visualization comparison of the augmented result plot and the MSE plot on the Middlebury dataset.
[0026] Figure 8 A visualization comparison of the predicted disparity maps generated by different methods on the Flickr1024 dataset. Detailed Implementation
[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] See attached document Figure 1 This invention provides a low-light stereo image enhancement method based on context guidance and structure awareness, which is achieved by constructing and running a cross-view state space network.
[0029] Figure 1 middle, The input is a low-light left view image. The input is a low-light right view image. To enhance the output of a high-quality left view image, To enhance the high-quality right-view image output, the CGSA Block (Context-Guided and Structure-Aware Block) is used. This is the core building block of the entire U-Net encoding / decoding architecture; Context-Guided Mamba is the context-guided Mamba module. The first half of the CGSA Block is used to process features along the epipolar direction and perform cross-view information interaction; CGSSM: Context-Guided State Space Module, used to perform epipolar constraint-based context complementarity and state parameter guidance between left and right views; Structure-Aware Mamba, located in the second half of the CGSA Block, is used to perceive the two-dimensional spatial structure and perform compensation; SASSM: Structure-Aware State Space Module, uses the local two-dimensional structural representation across views to guide the state evolution update of the target view; Channel Mamba, located after the two-branch structure, is used to assist in enhancing the feature channel dimension and capture global channel dependencies; LN (LayerNorm): Layer Normalization, used to normalize data before the feature input to the next layer; SSM: State Space Model, the basic unit in the network used for sequential modeling and feature state evolution; FFN: Feed-Forward Network. Typically composed of multilayer perceptrons or convolutions, it is used for nonlinear mapping and further extraction of features; Downsample: represents a downsampling operation, used to reduce the spatial resolution of the feature map and improve the expressive power of the feature channels; Upsample: represents an upsampling operation, used to restore the spatial resolution of the feature map; Element-Wise Addition: represents an element-wise addition operation, used to add and fuse two feature tensors of the same size at corresponding element positions.
[0030] This cross-view state-space network employs a symmetric encoder-decoder architecture. Multiple context-guided and structure-aware cross-view blocks are cascaded within the network at different feature scales. Each cross-view block incorporates a dual-branch parallel structure for simultaneously processing left-view and right-view features.
[0031] Each cross-view block contains three sub-modules in sequence: a context-guided cross-view interaction module, a structure-aware state space module, and a channel state space module.
[0032] The context-guided cross-view interaction module is used to perform cross-view information interaction in the polar direction.
[0033] The structure-aware state space module is used to extract two-dimensional local structures and perform texture compensation.
[0034] The channel state space module is deployed at the tail of the cross-view block to enhance the channel features of the left and right views.
[0035] Based on the above system architecture, the low-light stereo image enhancement method based on context guidance and structure awareness provided in this embodiment of the invention includes the following steps: Step S100: Obtain a pair of low-light stereo images, extract features from the low-light stereo images to obtain left view features and right view features, and input the left view features and right view features into the context-guided and structure-aware cross-view block.
[0036] Step S200: Context-guided epipolar cross-view interaction is executed through the context-guided cross-view interaction module. Cross-view context features are extracted based on epipolar constraints, dynamic state mapping parameters are generated, and epipolar interaction features are output.
[0037] Step S300: Input the epipolar interaction features into the structure-aware state space module to perform local two-dimensional latent state compensation for structure awareness. Extract the two-dimensional structural features of the reference view and inject them into the latent state evolution process of the target view using a gated modulation method, outputting the structure compensation features.
[0038] In step S400, channel-assisted enhancement is performed on the structural compensation features through the channel state space module. The enhanced features are then input into the decoder to perform image reconstruction, and the enhanced stereo image pair is output.
[0039] Based on the overall workflow described above, the specific process of performing image enhancement across view state space networks is explained in detail.
[0040] For the process of acquiring low-light stereo image pairs and performing basic preprocessing, the aforementioned step S100 specifically includes the following sub-steps: Step S101: Receive the left and right view images acquired by the image acquisition device under low-light conditions to form a low-light stereo image pair. The input left and right view images are represented as three-dimensional tensors in data format, with the tensor dimensions corresponding to the number of color channels, height, and width of the image.
[0041] Step S102 involves extracting basic features from the low-light stereo image pair using a shallow convolutional network configured at the input of the cross-view state space network, outputting initial left-view and right-view features. The shallow convolutional network, as the feature extraction front-end of a symmetric encoder-decoder architecture, typically consists of multiple layers of basic two-dimensional convolutional layers stacked together. It maps the input low-light stereo image pair from pixel space to a high-dimensional feature space to expand the feature representation dimension. Regarding the specific implementation principle and operation process of image feature extraction using convolutional layers, those skilled in the art can combine conventional image perception domain partitioning methods to set the convolutional kernel size and stride. The method of using convolutional kernels to slide across the image matrix to extract features is a well-known technique in the field and will not be elaborated here. After processing by the shallow convolutional network, the left-view and right-view images are transformed into high-dimensional feature tensors with expanded channel numbers, thus obtaining the initial left-view and right-view features.
[0042] Step S103 involves inputting the generated initial left-view and right-view features in parallel into the context-guided and structure-aware cross-view block. The cross-view block, as a processing unit on the feature processing path within the encoder-decoder architecture, is deployed at various feature scale levels during the network downsampling and upsampling processes. After entering the cross-view block, the initial left-view and right-view features are distributed to the corresponding left-view and right-view processing paths via its internal dual-branch parallel structure, serving as the basic input data for subsequent epipolar interaction and hidden state evolution modules.
[0043] See attached document Figure 2 The system receives the initial left view features and initial right view features output in step S103, collectively referred to as input view features. The input view features enter the parallel processing branch set up within the cross-view block and perform interactive operations based on epipolar constraints.
[0044] Figure 2 In this context, Linear / DWConv / Conv1×1: Linear mapping layer / Depthly separable convolution / 1×1 convolutional layer; SiLU / Softmax: Non-linear activation function / Normalization function; Forward Scan / Unfold: Forward selective scanning / Local neighborhood unfolding; Similarity / Gate Fusion: Similarity calculation / Gate fusion module.
[0045] Step S201 involves sequentially performing linear mapping, depthwise separable convolution, and activation function processing on the input view features to generate shallow enhanced features. The linear mapping operation performs a linear combination of the channel dimensions on the input tensor, while the depthwise separable convolution operation aggregates spatially neighboring pixel information across independent channels. The specific operational logic follows the shallow feature enhancement formula: ; In the formula, The characteristics of the input view are represented by the dimension. tensor, Represents batch size. Represents the number of channels. Represents height, Represents width. Represents the view identifier. Represents the left view. Represents the right view. This represents the linear mapping layer operation. This represents depthwise separable convolutional layer operations. This represents the SiLU activation function, which is responsible for introducing nonlinear mappings. This represents the shallow enhancement features output after computation.
[0046] Step S202, change the spatial dimension to The shallow enhancement features are flattened according to spatial row priority to construct a one-dimensional feature sequence. The permutation operation transforms the spatial pixel features distributed in the two-dimensional plane into a continuous sequence structure, adapting to the subsequent data reading requirements of truncating epipolar matching windows along the horizontal rows of the image.
[0047] Step S203, for the current scan position index The target view features and reference view candidate features are extracted along the horizontal epipolar line. The center features at the corresponding positions in the left view are extracted as the query vector, and the right view... Figure 1 Local window features within the neighborhood of the same horizontal row epipolar line in the dimensional sequence are extracted to serve as a candidate set. The normalized similarity between the current view's center feature and the reference view's candidate features is calculated to obtain attention weights. Based on these attention weights, the reference view's candidate features are weighted and aggregated to obtain cross-view context features. Feature extraction and aggregation are performed simultaneously in the left and right view branches, with the specific computational logic following the cross-view context feature calculation formula: ; ; In the formula, Represents the current scan position index. Representative at the Local window features of the left view extracted along the epipolar direction at each scan location. This represents the feature of the right view partial window captured at the corresponding position. This represents a similarity mapping function that measures the degree of correlation between features through inner product operations. This represents the Softmax function. This represents element-wise multiplication. This represents the cross-view context feature of the left view side. This represents the cross-view context feature of the right view side.
[0048] Step S204 involves gating and fusing the cross-view context features and the target view features. Specifically, at the channel dimension, the local window features of the target view, the corresponding cross-view context features, the absolute difference between the two, and their element-wise product are merged and concatenated. Multidimensional tensors are sequentially input into the convolutional network, which outputs gating coefficients. The corresponding computational logic follows the feature fusion and gating generation formula: ; ; ; ; In the formula, This represents a channel splicing operation. This represents the absolute value operation. and These represent the multidimensional features resulting from the splicing of the left and right view sides, respectively. Represents a convolutional layer operation, consisting of two consecutive layers responsible for channel compression and feature mapping. This represents the Sigmoid function. and These represent the gating coefficients generated from the left and right viewpoints, respectively.
[0049] Step S205 involves using gating coefficients to filter cross-view context features, suppressing irrelevant edge noise interference caused by low-light environments in the reference view. The truncated feature data is then added to the local window features of the target view to construct enhanced features. The specific computational logic follows the enhanced feature construction formula: ; ; In the formula, and These represent the enhanced input features for the left and right view sides, respectively.
[0050] Step S206: Introduce a state-space model to perform sequential evolution along the epipolar direction. The enhanced features, incorporating reference view information, are fed into a parameter mapping network composed of convolutional layers to generate state mapping parameters for the target view's state space. (Original features of the target view's local area) and A parallel input mapping network with independently configured parameters generates step size and output matrix parameters. The related parameter generation logic follows the state parameter construction formula: ; ; In the formula, Represents the state mapping parameters. This represents a mapping function consisting of two 1×1 convolutional layers and the SiLU activation function. Represents the original features of the current view. This represents the step size parameter. This represents the step size mapping function. This represents the output matrix parameters. This represents the corresponding mapping function consisting of two 1×1 convolutional layers and SiLU activation.
[0051] Step S207, the target view is in the... State updates are performed at each scan position. Combining the accumulated hidden state from the previous scan position, the state mapping parameters at the current position, and the enhanced features, the current hidden state is iteratively calculated and mapped to the output result. The state update follows the formula: , , In the formula, Representing the The hidden state at each scan location. This represents the hidden state at the previous scan position. Represented by step size parameter State transition parameters after discretization of the continuous state matrix. Representing the Output at each scan position. This represents the parameters for skip connections.
[0052] Step S208: Rearrange the output sequence back into a two-dimensional feature map, and output the final feature result through residual connections. The processing logic follows the epipolar interaction output formula: ; In the formula, This represents a two-dimensional feature map rearranged from the output sequence. This represents a convolution operation with a kernel size of 1×1. Representative layer normalization operation. This represents the initial input features. This represents the polar interaction characteristic.
[0053] See attached document Figure 3 , Figure 3In this context, Linear represents a linear mapping layer, DWConv represents a deep convolutional layer, SiLU represents a nonlinear activation function, Forward Scan represents a forward selective scan operation, Unfold represents a local neighborhood unfolding operation, Similarity represents a similarity calculation operation, Sigmoid represents a sigmoid activation function, SSM represents a state-space model, Conv1×1 represents a 1×1 convolutional layer, LN represents a layer normalization operation, and SIMB represents a Space-Channel Information Mining Block, which is used to mine spatial and channel dimensions of information from the two-dimensional features after the local neighborhood is unfolded in order to obtain a two-dimensional structural representation of the corresponding view.
[0054] The epipolar interaction features of the left and right views output in step S200 are received. The channel dimension is The epipolar interaction features are fed into a shallow processing network consisting of linear mapping layers and deep convolutional layers. An activation function is then used to perform nonlinear feature enhancement, yielding the basic input features for state-space modeling. and The feature tensor aggregates local spatial information while expanding the channel dimension, providing a data basis for modeling long program sequences. The specific mapping logic follows the basic feature generation formula: ; In the formula, This represents the epipolar interaction feature input from the previous step. Represents the view identifier. Represents the left view. Represents the right view. This represents a linear mapping operation. This represents a depthwise convolution operation. This represents the SiLU activation function. This represents the basic input features that are generated.
[0055] Based on the generated basic input features and After flattening it into a one-dimensional sequence along the spatial dimension, a preliminary forward selective scan operation is performed to obtain the corresponding baseline hidden state sequence. and The operation follows the implicit state preview formula: ; In the formula, This represents a forward selective scan operation, responsible for initially establishing long-range dependencies along the sequence dimension. This represents the acquired baseline hidden state sequence.
[0056] Because different views often exhibit parallax shifts, local occlusions, and inconsistent degradation due to differences in camera physical positions, relying solely on one-dimensional horizontal parallax direction state evolution is insufficient to maintain the integrity of the image's two-dimensional structure. Therefore, a one-dimensional to two-dimensional reconstruction is performed on the baseline hidden state sequence, followed by a local neighborhood unfolding operation, and the structural representation of the corresponding view is extracted using spatial channel information mining blocks. and The operational logic follows the structure representation extraction formula: ; In the formula, This represents a local neighborhood expansion operation, responsible for extracting a local block of a preset k×k size in two-dimensional space, where k is an odd number greater than or equal to 3, ensuring that the extracted area has a definite center coordinate pixel. In participation Before the operation, the one-dimensional sequence has been restored to a two-dimensional feature map arrangement through a reverse rearrangement operation. The representative spatial channel information mining block consists of layer normalization, multilayer perceptron, large kernel grouping convolution, and channel attention mechanism. This represents the extracted two-dimensional structural representation. Unlike interaction methods that rely on a single corresponding point, the spatial channel information mining block preserves the local two-dimensional structure of the reference view, enabling the network to perceive richer complementary information.
[0057] If structural information from the reference view is injected into the state update process of the target view without constraints, erroneous structural compensation can easily be introduced in occluded or severely degraded regions. Based on the mapping comparison between the latent state center features of the target view and the local neighborhood structural features of the reference view, gating coefficients are generated to dynamically control the injection intensity of structural information. The computational logic follows the gating derivation formula: ; ; In the formula, This represents a similarity mapping function that evaluates the geometric fit between two views at their corresponding spatial locations through feature matching along the channel dimension. This represents the Sigmoid activation function, which is responsible for smoothly mapping the matching results to a continuous interval of (0, 1). The gating coefficient represents the intensity of structural information injection from the left view to the right view. This represents the gating coefficients that control the intensity of structural information injection from the right view to the left view. Each element in the generated gating coefficient matrix precisely corresponds to the matching response weight of the k×k local neighborhood across views.
[0058] After obtaining the gating coefficients and structural representation, the cross-view structural information is integrated into the formal hidden state evolution process of each view in the form of gating modulation. During the time-step update of the state-space model, the two-dimensional structural information of the reference view, after being multiplied by the gating coefficients and subjected to serialization dimensionality reduction and alignment, is superimposed as a modulation compensation term onto the hidden state sequence of the target view. The computational logic follows the gating evolution compensation formula: ; ; ; ; In the formula, Represents the current time step index. and Representing the left and right views respectively in the 1st... The hidden state that is officially generated at any given moment. and Represents the basic input features and After sequence flattening, at the The sequence input components at time t. This represents element-wise multiplication. , , , ( ) represent the discrete state transition matrix, discrete input mapping matrix, discrete output mapping matrix, and direct jump parameter of the state-space model in the corresponding view, respectively. and These represent the current response values of the one-dimensional output sequences generated by the left and right views, respectively. The structural information of the reference view participates in the evolution of the target view state under the constraint of similarity gating, preserving cross-view complementary compensation capabilities while reducing the negative impact caused by the propagation of erroneous structures.
[0059] The one-dimensional output sequence response values at each time step are summarized and inversely reconstructed into a two-dimensional tensor to generate the intermediate sequence features of the left and right views. and Projection restoration is performed using layer normalization and pointwise convolution, combined with the epipolar interaction features input in step S200. The direct path performs residual multiplication, outputting the final updated structural enhancement features. The computational logic follows the feature residual output formula: ; In the formula, The intermediate sequence features represent the complete scan and state evolution output. Representative layer normalization operation. A 1×1 convolution operation representing channel interaction. The structural enhancement features represent the final output. By injecting complementary structural information from the reference view into the feature evolution process of the target view, the compensation is extended from cross-view compensation along the epipolar direction to the local two-dimensional space, improving the model's accuracy in restoring edge details in low-light degradation regions.
[0060] Receive the left view structural enhancement feature output in step S300 Enhanced features of the right view structure The preceding state-space modeling phase focuses on cross-view interaction along the epipolar direction and two-dimensional local structure compensation. To supplement global context information beyond the spatial dimension, the system initiates an auxiliary enhancement processing flow along the channel dimension.
[0061] The structural enhancement features are input into the channel state space module for feature extraction. A transpose operation is used to change the dimensional arrangement of the input tensor, performing axis swapping between the spatial expansion sequence and the channel dimensions, mapping the channel sequence of the feature dimensions to the evolution direction read by the state space model. A one-dimensional forward selective scan is performed on the transposed tensor to capture the dependencies between different channel levels, completing channel state aggregation. An inverse transpose operation is performed to restore the initial spatial and channel arrangement format, and element-wise addition is performed with the direct jump path and the input structural enhancement features to output the channel enhancement features. The channel evolution logic follows the channel state-assisted enhancement formula: ; In the formula, The structural enhancement features represent the output of the aforementioned steps. Represents the view identifier. This represents the tensor dimension transpose operation. This represents a one-dimensional forward selective scan operation. The channel enhancement feature represents the output.
[0062] The channel-enhanced features are input into the decoder module for upsampling and spatial feature decoding. A transposed convolutional layer is used to enlarge the spatial resolution of the channel-enhanced features, obtaining an upsampled tensor. Shallow features at the same resolution scale as the current decoding level are extracted from the encoder network. These shallow features are then concatenated and aligned with the upsampled tensor along the channel dimension via cross-layer skip connections. The concatenated and fused features simultaneously preserve the semantic representations transmitted from the deep network and the high-frequency texture details retained by the shallow network. The concatenated tensor is input into a decoding block consisting of a 3×3 convolutional layer and an activation function to perform channel dimensionality reduction and local information recovery. This upsampling and concatenation / fusion operation is repeated until the spatial resolution of the feature tensor is restored to the initial input size, generating the decoded feature tensor. For channel parameter conversion during feature splicing alignment, those skilled in the art can refer to the cross-layer splicing principle of the standard encoder-decoder architecture. The relevant feature mapping principle is a well-known technology in this field and will not be elaborated here.
[0063] Applying independently configured mapping convolutional layers will decode the feature tensor. The number of channels is compressed to the three-channel specification of a standard color image to generate a residual reconstruction tensor. A global residual learning mechanism is used to superimpose the original input view, which is in a low-light degradation state, onto the corresponding residual reconstruction tensor, outputting a stereo image pair with restored brightness. The reconstruction logic follows the image decoding output formula: ; In the formula, This represents the decoded feature tensor output after multi-level decoding and recovery. This represents a 3×3 convolutional layer responsible for channel mapping output. This represents the initial low-light raw view data received by the system. This represents the stereo image pair that the network will ultimately output.
[0064] During the addition operation execution phase, the original input view data The residual reconstruction tensor output by the mapped convolutional layer maintains strict size alignment in terms of spatial resolution and number of channels. The global residual mechanism limits the network's prediction target to the difference distribution before and after image degradation, avoiding the color distribution shift caused by directly mapping the complete image.
[0065] To ensure the aforementioned system modules possess reliable illumination restoration and structural compensation capabilities, end-to-end training of the entire network is required using paired stereo image datasets before practical inference applications. A training sample set is constructed, comprising pairs of low-light stereo images and corresponding pairs of normally exposed reference images. The low-light stereo image pairs are input into the network, and after the aforementioned feature extraction, epipolar interaction, structural compensation, and decoding reconstruction processes, predicted stereo image pairs are output. The L1 norm is used as the loss function to calculate the absolute numerical error between the predicted stereo image pairs and the normally exposed reference image pairs at corresponding pixel coordinates. Through backpropagation, the mapping weight parameters within the network are dynamically updated based on the calculated error gradient until the loss function converges to a predetermined interval, completing the network training process.
[0066] Specific application examples: I. Test Environment and Evaluation Indicator Settings To verify the effectiveness of the Cross-View State Space Network (CGSA-Net) constructed in this embodiment of the invention in the low-light stereo image enhancement task, standardized tests were performed using four publicly available stereo datasets: Flickr1024, KITTI 2012, KITTI 2015, and Middlebury. The tests used synthetic low-light stereo images as input and corresponding normal-light stereo images as reference images (Ground Truth, GT). Specifically, the Flickr1024 test set contained 112 image pairs, KITTI2012 and KITTI2015 each contained 20 image pairs, and Middlebury contained 5 image pairs.
[0067] Peak signal-to-noise ratio (PSNR), structural similarity (SSIM), learned perceptual patch similarity (LPIPS), and Fréchet Inception distance (FID) were used as objective evaluation metrics. PSNR and SSIM were used to measure the pixel-level fidelity and structural similarity between the reconstructed image and the reference image; higher values indicated better restoration quality. LPIPS and FID were used to evaluate perceptual feature differences and data distribution bias; lower values indicated that the visual perception was closer to the real scene.
[0068] All tests were performed on the hardware platform using the PyTorch framework. Network training employed the Adam optimizer with a batch size of 4. Input stereo images were simultaneously and randomly cropped into 128×128 pixel feature blocks. The initial learning rate was set to 2×10⁻⁶. -4 And a cosine annealing strategy is used for dynamic attenuation adjustment.
[0069] II. Quantitative Comparison and Network Structure Mapping Analysis The method of this invention was quantitatively compared with representative monocular low-light enhancement methods (such as RetinexNet, Zero-DCE, LLFormer, etc.) and binocular low-light enhancement methods (such as DVE-Net, DCI-Net, etc.) under the same conditions. Specific objective comparison data are shown in Table 1. Table 1. Comparison of PSNR and SSIM results for each method on four datasets. Test results, as shown in Table 1, demonstrate that the method of this invention achieves high PSNR and SSIM values in both the left and right views of the four datasets. Monocular methods, lacking a path for reading cross-view information, struggle to leverage the inherent structural complementarity of binocular cameras, resulting in limitations in disparity consistency and edge preservation. While existing binocular enhancement methods introduce disparity interaction, they still tend to exhibit feature alignment biases in complex degradation scenes. Furthermore, by evaluating the computational cost (FLOPs) and parameter count on a 128×128 image patch, the method of this invention achieves performance improvements while maintaining low computational cost and parameter count.
[0070] The performance improvement of the method in this invention is attributed to the intervention of the Context-Guided Cross-View Interaction Module (CGCIM) in the network. During the epipolar interaction phase, the network follows the formula... Cross-view context features were extracted and used to generate gating coefficients ( , Perform dynamic filtering on the reference view information. , This epipolar constraint-based context guidance mechanism suppresses irrelevant edge noise interference caused by low-light environments in the reference view, thereby improving pixel-level fidelity in outdoor scenes with complex lighting. Simultaneously, the method of this invention also achieves low values in both LPIPS and FID, indicating that the generated image closely approximates the normally exposed reference image in terms of deep feature distribution.
[0071] III. Visual Detail Restoration and Error Map Analysis Qualitative analysis was performed by magnifying a local region of the test dataset and extracting a binary map of the mean square error (MSE) between the dataset and the ground truth reference image.
[0072] It should be noted that the appendix Figure 4 To be continued Figure 7 The English labels included are the names of their corresponding test methods or reference benchmarks. Specifically: the label "GT" in the figure represents Ground Truth, i.e., a normally exposed reference image; the labels "LLFormer", "RetinexFormer", "RetinexMamba", "WalMaFa", "HVI-CIDNet", "LatentDark", "DVE-Net", "DCI-Net", "LFE-Net", and "WDCI-Net" respectively represent existing monocular or binocular image enhancement methods participating in the comparative test; the label "Lbaseline-stereo12-struct" corresponds to the enhancement method provided in the embodiments of this invention (i.e., the aforementioned CGSA-Net). Furthermore, the appendix... Figure 4 To be continuedFigure 7 In the image, for each method, the upper and lower parts of the image are shown. The upper part shows the enhanced color output image and the magnified area indicated by the red box in the lower left corner. The lower part shows the binary MSE error map calculated between the enhanced result and the GT reference image. In the MSE map, black areas represent smaller pixel-level errors with the reference image, while white areas represent larger errors. The less white response, the closer the enhanced result is to the reference image in terms of pixel values.
[0073] Combined with appendix Figure 4 The displayed samples feature complex natural textures such as bark and branches, along with... Figure 7 The indoor motorcycle scene shown includes intricate details such as tire textures and wheels. Multiple comparison methods exhibit blurred or indistinct texture edges in magnified areas, with dense white response areas in the corresponding MSE images, indicating significant pixel-level residuals in areas of weak texture. In contrast, the method of this invention (Lbaseline-stereo12-struct) maintains clear circular cross-sectional contours and natural texture transitions in these areas, significantly reducing the white response area in the MSE image.
[0074] Further combine with the appendix Figure 5 With appendix Figure 6 The outdoor road scene shown includes vehicles, road boundaries, and distant fine lines. Some contrast methods exhibit insufficient contrast and unnatural edge transitions in dark areas of the vehicle body, sky areas, or near fine lines, resulting in a significant amount of white response around the road or vegetation in the corresponding MSE image. The method of this invention provides a brighter transition in the magnified area that better matches the GT image, with less white area in the overall MSE image, demonstrating a stable visual restoration effect.
[0075] The aforementioned improvements at the visual level directly correspond to the local hidden state compensation mechanism of the Structure-Aware State-Space Module (SASSM) in the network. This is achieved through the formula... After extracting the two-dimensional local structural representation of the reference view, the system derives the gating coefficients through similarity. And perform dynamic modulation compensation during the state evolution process: This gating evolution compensation mechanism reduces the probability of introducing erroneous structures in occluded or severely degraded regions, enabling the network to maintain high edge recovery accuracy even in scenes with rich detail but significant low-light degradation.
[0076] IV. Ablation Experiment Verification of Core Modules To verify the rationality of the system architecture design, the proposed functional modules were introduced one by one into the basic feature processing network for ablation testing. The corresponding objective indicators of the ablation experiments are shown in Table 2.
[0077] Table 2. Core module ablation experimental results (PSNR / SSIM) on the Flickr1024 and KITTI2015 datasets. Analysis based on the data in Table 2 and the actual visual effects: Pure base network: The overall enhancement result is too dark, with insufficient local contrast in the sky, track, and vehicle areas.
[0078] Baseline+CGCIM: After introducing context-guided epipolar interaction, the model's overall PSNR values on the Flickr1024 and KITTI2015 datasets improved, and the overall scene brightness and visibility were enhanced. This indicates that cross-view epipolar interaction can effectively utilize complementary information between the left and right views to reduce pixel-level brightness errors.
[0079] Baseline+SASSM: After introducing the structure-aware state space module separately, the SSIM index is significantly improved. Visually, the transition of local structures such as track lines, vehicle body edges and distant thin lines is more natural, which verifies the role of local two-dimensional structural feature extraction in preserving structural information.
[0080] The complete model of this invention (CGSA-Net) simultaneously cascades CGCIM and SASSM, achieving maximum gain in objective metrics, and visually, the brightness distribution and local structure are closer to the reference image. Test data confirms that reliable cross-view interaction and local structure perception form a structural complement in low-light stereo image restoration tasks.
[0081] V. Downstream Task Application: Parallax Consistency Maintenance To verify the impact of the enhancement results on the binocular matching task, this embodiment performs disparity estimation tests on the Flickr1024 dataset. The enhanced left and right views from each method are extracted and input into the disparity estimation algorithm to generate a predicted disparity map. The endpoint error (EPE) between this predicted disparity map and the normal illumination ground truth (GT) reference disparity map is then calculated. A lower EPE value indicates better preservation of cross-view geometric consistency in the stereo image. The endpoint errors of disparity estimation after processing by different enhancement methods are shown in Table 3.
[0082] Table 3. Comparison of disparity estimation endpoint errors (EPE) of various methods on the Flickr1024 dataset See attached document Figure 8 It should be noted that the appendix Figure 8 It contains 13 subgraphs from (a) to (m), among which Figure 8The English labels corresponding to sub-images (a) to (j) in the text represent the predicted disparity maps of the output images after enhancement by the contrast methods LLFormer, RetinexFormer, RetinexMamba, WalMaFa, HVI-CIDNet, LatentDark, DVE-Net, DCI-Net, LFE-Net, and WDCI-Net, respectively. Figure 8 In the subgraph (k) label, “CGSA-Net(pro.)” represents the predicted disparity map corresponding to the enhancement method (proposed) proposed in the embodiments of the present invention; Figure 8 In the subplot (l) label, “GT” stands for Ground Truth, which is the reference parallax map under normal lighting conditions. Figure 8 The “RGB(LeftView)” label in the sub-image (m) section represents the color image of the left view input in the test scenario.
[0083] Combining the data in Table 3 with the appendix Figure 8 Observation shows that the predicted disparity maps of multiple contrast methods (such as...) Figure 8 Subgraphs (a) to (j) still exhibit varying degrees of voids, fractures, or local structural discontinuities, resulting in relatively high EPE values. The method of this invention achieves a lower EPE value (3.3617), and as shown in the attached figure... Figure 8 As shown in subgraph (k), the disparity map output by this invention has a smoother grayscale transition and a more continuous structural distribution, compared to... Figure 8 The GT reference disparity map shown in sub-graph (l) is closer. This proves that the image enhancement method provided by the present invention can maintain the original geometric matching relationship between the left and right views while improving the brightness of low-light images and restoring texture details.
Claims
1. A low-light stereo image enhancement method based on context guidance and structure awareness, characterized in that, Includes the following steps: Acquire a pair of low-light stereo images, extract features from the pair of low-light stereo images, and obtain left view features and right view features; The left view features and the right view features are used as target view features and reference view features, respectively. Cross-view context features are extracted based on epipolar constraints, and epipolar interaction features are output. Based on the epipolar interaction features, a two-dimensional structural representation corresponding to the reference view features is extracted, and the two-dimensional structural representation is injected into the hidden state evolution process corresponding to the target view features through a gated modulation method to output structural enhancement features. Channel-assisted enhancement is performed on the structural enhancement features, outputting channel enhancement features. The channel enhancement features are then input into the decoder to perform image reconstruction, outputting enhanced stereo image pairs.
2. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 1, characterized in that, The steps of acquiring a pair of low-light stereo images, extracting features from the pair of low-light stereo images, and obtaining left view features and right view features include: The left and right view images under low-light conditions are received by the image acquisition device to form the low-light stereo image pair; The low-light stereo image pairs are subjected to basic feature extraction by a shallow convolutional network set at the system input, and the initial left view features and the initial right view features are output. The initial left view features and the initial right view features are used as the left view features and the right view features, and are input in parallel into the context guidance and structure awareness cross-view block cascaded within the cross-view state space network. The features are then distributed to the corresponding left view processing path and right view processing path via the internal dual-branch parallel structure.
3. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 1, characterized in that, The method for extracting cross-view context features based on epipolar constraints includes: The left view features and the right view features are collectively referred to as input view features. Linear mapping operation, depthwise separable convolution operation and activation function processing are sequentially performed on the input view features to generate shallow enhancement features. Flatten the shallow enhancement features according to the spatial row priority order to construct a one-dimensional feature sequence; For the current scan position index, extract the center feature of the position corresponding to the target view feature along the horizontal epipolar line direction as the query vector, and extract local window features in the same horizontal row epipolar line neighborhood of the one-dimensional feature sequence corresponding to the reference view feature as the candidate set. Calculate the normalized similarity between the query vector and the candidate set and obtain the attention weight. Based on the attention weight, perform weighted aggregation on the candidate set to obtain the cross-view context features.
4. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 3, characterized in that, The output epipolar interaction features include: In the channel dimension, the center feature at the corresponding position of the target view feature, the corresponding cross-view context feature, the absolute difference between the two and the element-wise product of the two are merged and concatenated, and the multidimensional tensor obtained by concatenation is sequentially input into the convolutional network to output the gating coefficients. Perform element-wise multiplication on the gating coefficient and the cross-view context feature, and add the calculated feature data to the local window feature at the corresponding position of the target view feature to construct the enhanced feature; The enhanced features are fed into a parameter mapping network composed of convolutional layers to generate state mapping parameters for the target view state space. The input features corresponding to the target view features are input into an independently set mapping network in parallel to generate step size parameters and output matrix parameters. By combining the hidden state accumulated at the previous scan position, the state mapping parameters at the current position, and the enhanced features, the current hidden state is iteratively calculated and mapped to the output result. The output sequence composed of the output results is rearranged back into a two-dimensional feature map, and the final feature result is output as the epipolar interaction feature through residual connection.
5. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 1, characterized in that, The extraction of the two-dimensional structural representation corresponding to the features of the reference view includes: The epipolar interaction features are fed into a shallow processing network consisting of a linear mapping layer and a deep convolutional layer, and nonlinear feature enhancement is performed in combination with an activation function to obtain the basic input features for state space modeling. After flattening the basic input features into a one-dimensional sequence along the spatial dimension, a forward selective scan operation is performed to obtain the corresponding baseline hidden state sequence. The baseline hidden state sequence is reconstructed from one dimension to two dimensions, and then a local neighborhood expansion operation is performed. The structural representation of the corresponding view is extracted through the spatial channel information mining block, and the structural representation is used as the two-dimensional structural representation of the reference view feature.
6. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 5, characterized in that, The process of injecting the two-dimensional structural representation into the latent state evolution corresponding to the target view feature through gating modulation includes: Perform the same forward selective scan operation on the target view feature as on the reference view feature, and obtain the center feature of the corresponding target hidden state sequence as the hidden state center feature corresponding to the target view feature; Based on the mapping comparison between the hidden state center features corresponding to the target view features and the two-dimensional structural representation corresponding to the reference view features, a similarity mapping result is generated, and the similarity mapping result is processed by an activation function to generate gating coefficients.
7. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 6, characterized in that, The output structure enhancement features include: During the time-step update of the state-space model, the two-dimensional structural representation corresponding to the reference view feature is multiplied by the gating coefficient and then serialized and dimensionality-reduced and aligned. This representation is then superimposed as a modulation compensation term onto the hidden state sequence corresponding to the target view feature to generate a one-dimensional output sequence response value at each time step. The one-dimensional output sequence response values at each time step are summarized and reconstructed in reverse into a two-dimensional tensor to generate intermediate sequence features; The intermediate sequence features are projected back using layer normalization and pointwise convolution, and residual multiplication is performed in combination with the direct path containing the input epipolar interaction features to output the structural enhancement features.
8. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 1, characterized in that, The process of performing channel-assisted enhancement on the structural enhancement features and outputting channel-enhanced features includes: The transpose operation is used to change the dimensional arrangement logic of the input structural enhancement features. An axis swap is performed between the spatial unfolding sequence and the channel dimension to obtain the transposed tensor. Perform a one-dimensional forward selective scan on the transposed tensor to obtain the channel state aggregation sequence; Perform a reverse transpose operation to restore the initial spatial and channel layout format, combine the direct jump path with the input structural enhancement features to perform element-wise addition, and output the channel enhancement features.
9. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 8, characterized in that, The channel enhancement features are input into the decoder to perform image reconstruction, and the output is an enhanced stereo image pair, including: The spatial resolution of the channel enhancement features is amplified using a transposed convolutional layer to obtain an upsampled tensor. Extract shallow features from the network that are at the same resolution scale as the current decoding level, and perform concatenation and alignment between the shallow features and the upsampled tensor in the channel dimension through cross-layer skip connection paths to obtain the concatenated tensor; The concatenated tensor is input into the decoding block to perform channel dimensionality reduction and local information recovery. The above upsampling and concatenation fusion operations are repeated until the spatial resolution of the feature tensor is restored to the initial input size, and the decoded feature tensor is generated. The number of channels of the decoded feature tensor is compressed to a three-channel specification by applying a mapping convolutional layer to generate a residual reconstruction tensor. A global residual learning mechanism is used to superimpose the original input view corresponding to the low-light stereo image pair in the low-light degradation state onto the residual reconstruction tensor, and output the enhanced stereo image pair.
10. The low-light stereo image enhancement method based on context guidance and structure awareness according to claim 1, characterized in that, Prior to acquiring the low-light stereo image pairs, the process includes an end-to-end training step on the entire network using the paired stereo image dataset: Construct a training sample set containing pairs of low-light stereo images and corresponding pairs of normally exposed reference images; The low-light stereo image pairs are input into the network, and after feature extraction, epipolar interaction, structural compensation and decoding reconstruction, the predicted stereo image pairs are output. Using the L1 norm as the loss function, the absolute numerical error between the predicted stereo image pair and the normally exposed reference image pair at the corresponding pixel coordinate positions is calculated. The backpropagation algorithm dynamically updates the mapping weight parameters inside the network based on the calculated error gradient until the loss function converges to a set interval, thus completing the network training process.