A binocular stereo matching method, system and electronic device

Abstract

The application relates to a binocular stereo matching method and system and electronic equipment, and belongs to the technical field of binocular stereo matching.The application comprises the following steps: after local features and global features are extracted, the global-local feature weighting module based on channel attention is used for weighting, and after weighting, the image shallow layer features are integrated into the weighted global features and local features; the multi-body encoder is used for combining feature bodies containing different matching information; the disparity recovery module is used for recovering initial disparity from the GWC cost volume; the enhanced search unit is used for searching the feature matching information after the multi-body encoder coding according to the current disparity; the searched feature information and the multi-level context features extracted by the high-frequency domain enhancement context feature extractor from the left image are sent into the multi-level gate recurrent unit to iteratively update the current disparity, so that the final disparity is obtained.The application improves the perception ability of image texture and boundaries, and realizes more accurate disparity prediction in the abdominal cavity environment.

Description

Technical Field

[0001] This invention relates to a binocular stereo matching method, system, and electronic device, belonging to the field of binocular stereo matching technology. Background Technology

[0002] The mainstream approach to depth perception in current laparoscopic surgery is to obtain disparity through stereo matching and then analyze depth data based on the disparity. Therefore, the quality of the disparity measurement in the stereo matching algorithm determines the magnitude of the depth perception error. Traditional stereo matching algorithms based on local, global, and semi-global methods are gradually being replaced by deep learning-based methods due to limitations such as reliance on strong texture assumptions, limitations of energy functions, and limitations of path aggregation. In deep learning-based stereo matching methods, DispNet extracts features from the left and right images using a convolutional neural network and constructs a 3D cost volume using feature dot multiplication to calculate disparity. Methods based on this model, such as GC-Net, PSMNet, and GWCNet, improve the feature extraction and cost aggregation steps, achieving increased accuracy. RAFT-Stereo attempts to abandon 3D convolution for cost aggregation and instead uses 2D convolution to iteratively optimize the initial disparity. This method sets all initial disparity values ​​to 0, constructs a fully correlated volume between all features of the left and right images on the same epipolar line, and uses convolutionally gated recurrent units to progressively update the disparity based on matching information retrieved from the fully correlated volume using existing disparity information. Subsequent improvements to this model, such as IGEV, Selective-stereo, and Mocha-stereo, have continuously enhanced accuracy. However, these stereo matching methods are primarily suited for natural scenes and perform poorly in medical laparoscopic scenarios.

[0003] The core task of binocular stereo matching is to determine the positional difference of the same physical imaging point in two images. Existing neural network-based stereo matching methods include stages such as left and right image feature extraction, matching cost volume construction, cost aggregation, and iterative optimization. Among these, the quality of image feature extraction directly affects the accuracy of matching. Compared with natural images, peritoneal images have monotonous colors, uniform textures, and uneven illumination, which brings unique challenges to stereo matching in the peritoneal environment. First, the peritoneal tissue surface has quasi-periodic texture features, leading to a significant increase in feature matching ambiguity. Eliminating matching ambiguity requires fully exploring the global contextual information of the left and right images. However, existing stereo matching methods, in the feature extraction stage, rely solely on deep convolutions to perceive global information, lacking processing of the global contextual features of the left and right images. This fails to fully utilize contextual information to accurately distinguish similar textures within different quasi-periodicities and also results in ineffective matching in areas of strong light reflection. Second, existing methods use multiple layers of convolution continuously in the feature extraction stage, which easily leads to the loss of geometric edge information in the image. In particular, in areas of organ contact and abrupt changes in tissue depth, this can cause excessive parallax smoothing at organ boundaries and in areas of abrupt changes in tissue depth. Existing stereo matching methods have not adequately addressed these issues.

[0004] Therefore, this invention proposes a binocular stereo matching method, system, and electronic device. Summary of the Invention

[0005] The technical problem solved by this invention is that it provides a binocular stereo matching method, system, and electronic device to solve the problems of matching errors in repetitive texture areas, inability to match highly reflective areas, and excessive smoothing of parallax in organ edge areas that existing methods face when performing stereo matching in an abdominal environment.

[0006] The technical solution of the present invention is: a binocular stereo matching method, the method comprising:

[0007] Step 1: The high-frequency domain enhanced context feature extractor uses a hybrid context feature extraction module that includes three sets of Shift-MLP and 2D convolution to extract multi-level context features of the left image;

[0008] The high-frequency domain feature enhancement module is used to enhance the high-frequency information of each level of features;

[0009] Step 2: The global-local feature extractor extracts global features from the left and right images using a Shift-MLP-based global feature extractor.

[0010] Local features are extracted from the left and right images using a local feature extractor based on 2D convolution.

[0011] The extracted local and global features are weighted using a channel attention-based global-local feature weighting module. After weighting, the shallow features of the image are integrated into the weighted global and local features using a shallow feature fusion module with Haar wavelet downsampling.

[0012] Step 3: The multi-body encoder combines three feature bodies containing different matching information: GWC cost body, global APC correlation body, and local APC correlation body.

[0013] Step 4: The parallax recovery module recovers the initial parallax from the GWC cost volume;

[0014] Step 5: The enhanced search unit searches for feature matching information encoded by the multi-body encoder based on the current disparity. The found feature information and the multi-level context features extracted by the high-frequency domain enhanced context feature extractor of the left image are sent together to the multi-level gated loop unit to iteratively update the current disparity and obtain the final disparity.

[0015] Furthermore, in Step 1, the high-frequency domain enhanced context feature extractor includes a hybrid context feature extraction module and a high-frequency domain feature enhancement module;

[0016] The hybrid context feature extraction module uses Shift-MLP and 2D convolution to extract the input from the left image (Figure I). l Feature maps at three scales Where H and W represent the height and width of the input image, respectively, and C i Indicates the number of channels in the feature map;

[0017] The high-frequency feature information of the image is enhanced by using a high-frequency domain feature enhancement module on the feature map at each scale. The calculation formula is:

[0018]

[0019] HFE(f c,i ) = F -1 (F(f c,i )-G low (F(f c,i )))

[0020] Where HFE(.) represents the extraction of high-frequency domain information of the features, F represents the Fourier transform, and F -1 G represents the inverse Fourier transform. low This represents a Gaussian low-pass filter, where weights is a hyperparameter.

[0021] Furthermore, in Step 2, the global feature extractor includes:

[0022] For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16;

[0023] Then, four Shift-MLP modules are used to extract global features at four scales. C i The corresponding feature maps have 48, 64, 128, and 192 channels, respectively.

[0024] Furthermore, in Step 2, the local feature extractor includes:

[0025] For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16;

[0026] Then, initial local features at four scales are extracted using four Conv Blocks based on 2D convolution.

[0027] Additionally, the FreqFusion strategy is used to process the initial local features at four scales. Local features are obtained by fusion. The local feature fusion strategies for the four scales are as follows:

[0028]

[0029] Furthermore, in Step 2, the global-local feature weighting module includes:

[0030] First, group the global features at the same level. With local features The concatenation is performed along the channel dimension, and the attention on the channel is calculated after two layers of 2D convolution.

[0031] Finally, the attention-weighted global features are separated from the local features to obtain the weighted global features. and weighted local features

[0032]

[0033] In the formula, FW(,) represents the global features. With local features Perform attention weighting and separation.

[0034] Furthermore, in Step 2, the shallow feature fusion module includes:

[0035] First, two 3×3 2D convolutional layers are used to extract shallow features from the image. Then, Haar wavelet downsampling is applied to the extracted shallow features to obtain a 1 / 4 resolution feature map. pass Global feature maps weighted by 1 / 4 resolution respectively Weighted local feature maps The weighted feature map is updated by concatenation, and the update process is as follows:

[0036]

[0037] Furthermore, Step 3 includes:

[0038] Using the two input images I (left and right) l(r) Obtained 1 / 4 resolution weighted global feature map Constructing the GWC cost body:

[0039]

[0040] Among them, C gwc () represents a cost body group, d is the disparity index, h is the height index, w is the width index, g represents the cost body group, and N c N represents the number of feature channels. g The number of feature groups is denoted by <·,·>, which represents the inner product. The global feature map is weighted by the left 1 / 4 resolution. Global feature map weighted by resolution in the right 1 / 4;

[0041] Furthermore, following the RAFT-Stereo method, global APC correlation volume A is calculated on the 1 / 4 resolution weighted global feature map. glo and local APC related body A loc :

[0042]

[0043] Where · represents the dot product, w l w r ... The local feature map is weighted by the resolution of the left 1 / 4. The right quarter resolution-weighted local feature map;

[0044] Following the IGEV-Stereo approach, we increase the receptive field of the fully correlated data by using average pooling with a kernel size of 2 and a stride of 2 in the last dimension, thus constructing a combined encoding of the correlated data that includes two receptive fields.

[0045] Furthermore, in Step 5, the enhanced lookup unit is used to determine the current disparity from C. gwc A glo A loc The relevant feature information is searched in the combined encoded related body and fed into the GRU unit for iteration, including:

[0046] First, select the current parallax at C. gwc A glo A loc The corresponding matching features are then selected based on the query radius. Finally, all the searched features are concatenated and fed into the GRU iterative unit along with those extracted from the left image by the high-frequency domain enhanced context feature extractor.

[0047] The terms used in this invention are explained as follows:

[0048] Stereo matching: a technique in computer vision that is primarily used to calculate depth information from images captured by binocular cameras.

[0049] 2D Convolution: A fundamental operation in convolutional neural networks used to extract features from two-dimensional input data (such as images). The process involves sliding a learnable convolutional kernel across the spatial dimension of the input data, calculating the dot product of the kernel and the local region at each position, and accumulating the results to generate a feature map.

[0050] GWC cost body: A matching cost information body composed of a pre-defined maximum disparity range in a stereo matching task.

[0051] APC Correlation Volume: A feature correlation information volume composed of image width in stereo matching tasks.

[0052] Shift-MLP: A neural network architecture based on an improved multilayer perceptron (MLP) that enhances the model's ability to model spatial information by introducing a "shift" operation.

[0053] This invention utilizes a high-frequency domain enhanced contextual feature extractor and a shallow feature fusion module to enhance the capture of image geometric texture and edge information, mitigating the problem of excessive parallax smoothing in areas of organ contact and abrupt tissue depth changes. A global-local feature extractor and a global-local feature weighting module, capable of fusing global features, amplify the feature differences of similar textures by incorporating global information, improving the recognition accuracy of parallax in repetitive textures and highly reflective areas. The dual correlation volume and cost volume utilize a multi-volume encoder and an enhanced search unit to provide more sufficient matching cost information for parallax update decisions, improving the accuracy of parallax measurement. The model GLF-StereoNet and its key modules corresponding to the method described in this invention are the key technical points specifically protected by this method.

[0054] The present invention also provides a stereo matching system for the abdominal cavity environment, the system comprising: a module for performing the aforementioned binocular stereo matching method.

[0055] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the binocular stereo matching method.

[0056] The beneficial effects of this invention are:

[0057] 1. To address the issues of mismatched repetitive texture regions and unmatched highly reflective regions in the abdominal cavity environment, this invention proposes a Global-Local Feature Extractor (GL-FE) to extract image features. Specifically, in GL-FE, a global feature extractor composed of Shift-MLP extracts global features at four levels from the left and right images, while a local feature extractor composed of 2D convolution extracts local features at four levels from the left and right images. The extracted local features and global features are fused using a global-local feature weighting module based on channel attention. By weighting and fusing global and local features, the ability of image features to capture global contextual information is enhanced, thereby improving the feature representation differences of similar textures under different quasi-periods and effectively inferring and judging the disparity of highly reflective regions based on global information.

[0058] 2. To address the issue of excessive parallax smoothing in organ contact areas and tissue depth abrupt change areas caused by similar image colors, this paper proposes two approaches. First, a shallow feature fusion module is constructed using Haar wavelet downsampling. This module integrates shallow texture information from the image into the features weighted by the feature weighting module, thus preserving shallow geometric information. Second, a high-frequency domain enhanced contextual feature extractor is proposed. This module uses a combination of Shift-MLP and 2D convolution to extract left image features, and a high-frequency domain feature enhancement module further enhances the high-frequency information of the features, strengthening the image's boundary and texture information.

[0059] 3. This invention underwent extensive experimentation on two publicly available datasets and subsets, including the SCARED2019 dataset of porcine intraperitoneal images and the dVPN dataset of real human intraperitoneal images. Experimental results show that the network corresponding to the method proposed in this invention, GLF-StereoNet, outperforms state-of-the-art methods in both accuracy and robustness. Compared to existing methods, GLF-StereoNet enhances the feature representation differences of different imaging points in the image through a global-local dual-branch feature extractor, and improves the perception of image texture and boundaries by utilizing frequency domain information, achieving more accurate disparity prediction in the intraperitoneal environment. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the GLF-StereoNet network structure corresponding to the method of the present invention;

[0061] Figure 2 This is a schematic diagram of the hybrid context feature extraction module structure of the present invention;

[0062] Figure 3 This is a schematic diagram illustrating the selection of relevant features of the present invention;

[0063] Figure 4 This is a schematic diagram of the parallax visualization comparison on a subset of the SCARED2019 data in this invention;

[0064] Figure 5 This is a schematic diagram comparing the parallax visualization of different methods on the dVPN dataset of this invention. Detailed Implementation

[0065] Example 1: As Figures 1-5 As shown, a binocular stereo matching method includes:

[0066] Step 1: The high-frequency domain enhanced context feature extractor uses a hybrid context feature extraction module that includes three sets of Shift-MLP and 2D convolution to extract multi-level context features of the left image;

[0067] The high-frequency domain feature enhancement module is used to enhance the high-frequency information of each level of features;

[0068] Furthermore, in Step 1, the high-frequency domain enhanced context feature extractor includes a hybrid context feature extraction module and a high-frequency domain feature enhancement module;

[0069] The hybrid context feature extraction module uses Shift-MLP and 2D convolution to extract the input from the left image (Figure I). l Feature maps at three scales Where H and W represent the height and width of the input image, respectively, and C i Indicates the number of channels in the feature map;

[0070] To preserve more geometric edge information and avoid excessive smoothing of parallax at image edges, a high-frequency domain feature enhancement module is used on the feature maps at each scale to enhance the high-frequency feature information of the image. The calculation formula is:

[0071]

[0072] HFE(f c,i ) = F -1 (F(f c,i )-G low (F(f c,i )))

[0073] Where HFE(.) represents the extraction of high-frequency domain information of the features, F represents the Fourier transform, and F -1 G represents the inverse Fourier transform. low This represents a Gaussian low-pass filter, and weights is a hyperparameter that was set to 0.7 in the experiment.

[0074] Step 2: The global-local feature extractor extracts global features from the left and right images using a Shift-MLP-based global feature extractor.

[0075] Local features are extracted from the left and right images using a local feature extractor based on 2D convolution.

[0076] The extracted local and global features are weighted using a channel attention-based global-local feature weighting module. After weighting, the shallow features of the image are integrated into the weighted global and local features using a shallow feature fusion module with Haar wavelet downsampling.

[0077] Furthermore, in Step 2, the global feature extractor includes:

[0078] For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16;

[0079] Then, four Shift-MLP modules are used to extract global features at four scales. C i The corresponding feature maps have 48, 64, 128, and 192 channels, respectively.

[0080] Furthermore, in Step 2, the local feature extractor includes:

[0081] For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16;

[0082] Then, initial local features at four scales are extracted using four Conv Blocks based on 2D convolution.

[0083] The number of feature channels remains consistent with the globally extracted features. However, unlike the global feature extraction, we additionally employ the FreqFusion strategy to extract initial local features at four scales. Local features are obtained by fusion. The local feature fusion strategies for the four scales are as follows:

[0084]

[0085] Furthermore, in Step 2, the global-local feature weighting module includes:

[0086] First, group the global features at the same level. With local features The concatenation is performed along the channel dimension, and the attention on the channel is calculated after two layers of 2D convolution.

[0087] Finally, the attention-weighted global features are separated from the local features to obtain the weighted global features. and weighted local features

[0088]

[0089] In the formula, FW(,) represents the global features. With local features Perform attention weighting and separation.

[0090] Furthermore, in Step 2, the shallow feature fusion module includes:

[0091] First, two 3×3 2D convolutional layers are used to extract shallow features from the image. Then, Haar wavelet downsampling is applied to the extracted shallow features to obtain a 1 / 4 resolution feature map. pass Global feature maps weighted by 1 / 4 resolution respectively Weighted local feature maps The weighted feature map is updated by concatenation, and the update process is as follows:

[0092]

[0093] Step 3: The multi-body encoder combines three feature bodies containing different matching information: GWC cost body, global APC correlation body, and local APC correlation body.

[0094] Furthermore, Step 3 includes:

[0095] Using the two input images I (left and right) l(r) Obtained 1 / 4 resolution weighted global feature map Constructing the GWC cost body:

[0096]

[0097] Among them, C gwc () represents a cost body group, d is the disparity index, h is the height index, w is the width index, g represents the cost body group, and N c N represents the number of feature channels. g The number of feature groups is denoted by <·,·>, which represents the inner product. The global feature map is weighted by the left 1 / 4 resolution. Global feature map weighted by resolution in the right 1 / 4;

[0098] Furthermore, following the RAFT-Stereo method, global APC correlation volume A is calculated on the 1 / 4 resolution weighted global feature map. glo and local APC related body A loc :

[0099]

[0100] Where · represents the dot product, w l w r ... The local feature map is weighted by the resolution of the left 1 / 4. The right quarter resolution-weighted local feature map;

[0101] Following the IGEV-Stereo approach, we increase the receptive field of the fully correlated data by using average pooling with a kernel size of 2 and a stride of 2 in the last dimension, thus constructing a combined encoding of the correlated data that includes two receptive fields.

[0102] Step 4: The parallax recovery module recovers the initial parallax from the GWC cost volume;

[0103] Step 5: The enhanced search unit searches for feature matching information encoded by the multi-body encoder based on the current disparity. The found feature information and the multi-level context features extracted by the high-frequency domain enhanced context feature extractor of the left image are sent together to the multi-level gated loop unit to iteratively update the current disparity and obtain the final disparity.

[0104] Furthermore, in Step 5, the enhanced lookup unit is used to determine the current disparity from C. gwc A glo A loc Relevant feature information is searched in the combined encoded related body and fed into the GRU unit for iteration. The selection rule is as follows: Figure 3 As shown, it includes:

[0105] First, select the current parallax at C. gwc A glo A loc The corresponding matching features (black pentagram part) are selected, and then the surrounding features (black dot part) are selected according to the query radius (the radius of the legend is 4). Finally, all the searched features are concatenated and sent to the GRU iterative unit together with the features extracted by the high frequency domain enhanced context feature extractor in the left image.

[0106] The present invention also provides a stereo matching system for the abdominal cavity environment, the system comprising: a module for performing the aforementioned binocular stereo matching method.

[0107] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the binocular stereo matching method.

[0108] Through the combined action of the aforementioned modules, the GLF-StereoNet model corresponding to the method of this invention improves the accuracy of stereo matching in the peritoneal environment. To verify the effectiveness of the model, it was evaluated on two dataset subsets from SCARED2019 (Test-19 and Test-Original) and the dVPN dataset. On the Test-19 subset, endpoint error (EPE), root mean square error (RMSE), and percentage error (bad) of n pixels were used. n As evaluation metrics, absolute mean depth error was used as the evaluation metric on the Test-Original subset. Since the dVPN dataset lacks disparity and ground truth depth values, structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) were used as evaluation metrics.

[0109] In Table 1, Cost represents the cost-volume-based method, and Iteration represents the iterative optimization-based method (the meanings are the same in the following tables). As shown in Table 1, GLF-StereoNet achieved disparity metrics of 1.0895px, 2.1796px, 2.6007%, 2.2567%, and 1.5521% for EPE, RMSE, bad2, bad3, and bad5, respectively. Experimental results show that GLF-StereoNet outperforms existing methods in disparity measurement accuracy in the peritoneal environment.

[0110] Table 1 compares the methods in the Test-19 test subset.

[0111]

[0112] from Figure 4 The disparity visualizations show that HybridStereo cannot effectively handle close-range matching, while ICGNet and Select-IGEV cannot accurately calculate disparity in highly reflective areas. The model proposed in this paper maintains accurate disparity in ultra-close-up and reflective areas. Furthermore, the disparity step at tissue overlap boundaries is significant, without excessive smoothing.

[0113] The model was evaluated using mean absolute depth error (in mm) on Test-Original. In the experiment, each keyframe video subset of the Testset 1 and Testset 2 sequences was tested, and the average value of each sequence was obtained. The experimental results are shown in Table 2. GLF-StereoNet achieved mean absolute depth errors of 3.04 mm and 2.07 mm on the two sequences, respectively, and was able to achieve the current minimum error on both sequences.

[0114] Table 2 compares the mean absolute depth error of each method on the Test-Original subset.

[0115]

[0116]

[0117] On the dVPN dataset, the dataset provider does not provide true disparity labels, making it impossible to accurately measure the matching error between predicted and true disparities. In the experiments, the disparities calculated by the model and the right image are first reprojected to obtain the reprojected left image. The structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) (in dB) of the reprojected left image and the original left image are then calculated to roughly measure the accuracy of disparity assessment (all calculated images have been normalized).

[0118] Through experimental comparison and analysis with current state-of-the-art stereo matching benchmark models, detailed experimental results are shown in Table 3. The proposed GLF-StereoNet model consistently maintains a performance advantage on the dVPN dataset, achieving an SSIM of 63.71 and a PSNR of 14.95. This demonstrates that the disparity calculated by the StereoNet model, after reprojecting the right image onto the left image, exhibits smaller image structure and brightness differences compared to other models.

[0119] Table 3 shows the performance comparison of different models on the dVPN dataset.

[0120]

[0121] Parallax visualization results of different methods on the dVPN dataset, as follows Figure 5 As shown, the GLF-StereoNet model proposed in this invention can accurately identify parallax differences at the interface between instruments and human tissues in the human abdominal cavity environment. It also maintains high parallax recognition accuracy for human tissues with complex spatial structures and has excellent generalization ability.

[0122] This invention preserves shallow texture and geometric information of images by utilizing frequency domain information during left and right image feature and context feature extraction, thereby achieving boundary recognition capabilities for organ contact and tissue depth abrupt changes. During left and right image feature extraction, a dual-feature extraction branch (global and local) integrates global feature information into local features, increasing the feature differences between similar textures and reducing matching ambiguity. Simultaneously, during disparity iterative update, a combination encoding of two fully correlated volumes and a cost volume provides rich guidance information for disparity to move in the correct direction, improving the accuracy of disparity measurement. Other optimization algorithms implemented using this technical solution, as summarized above, are also within the scope of this invention.

[0123] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A binocular stereo matching method, characterized in that: The method includes: Step 1: The high-frequency domain enhanced context feature extractor uses a hybrid context feature extraction module that includes three sets of Shift-MLP and 2D convolution to extract multi-level context features of the left image; The high-frequency domain feature enhancement module is used to enhance the high-frequency information of each level of features; Step 2: The global-local feature extractor extracts global features from the left and right images using a Shift-MLP-based global feature extractor. Local features are extracted from the left and right images using a local feature extractor based on 2D convolution. The extracted local and global features are weighted using a channel attention-based global-local feature weighting module. After weighting, the shallow features of the image are integrated into the weighted global and local features using a shallow feature fusion module with Haar wavelet downsampling. Step 3: The multi-body encoder combines three feature bodies containing different matching information: GWC cost body, global APC correlation body, and local APC correlation body. Step 4: The parallax recovery module recovers the initial parallax from the GWC cost volume; Step 5: The enhanced search unit searches for feature matching information encoded by the multi-body encoder based on the current disparity. The found feature information and the multi-level context features extracted by the high-frequency domain enhanced context feature extractor of the left image are sent together to the multi-level gated loop unit to iteratively update the current disparity and obtain the final disparity.

2. The binocular stereo matching method according to claim 1, characterized in that: In Step 1, the high-frequency domain enhanced context feature extractor includes a hybrid context feature extraction module and a high-frequency domain feature enhancement module; The hybrid context feature extraction module uses Shift-MLP and 2D convolution to extract the input from the left image (Figure I). l Feature maps at three scales Where H and W represent the height and width of the input image, respectively, and C i Indicates the number of channels in the feature map; The high-frequency feature information of the image is enhanced by using a high-frequency domain feature enhancement module on the feature map at each scale. The calculation formula is: Where HFE(.) represents the extraction of high-frequency domain information of the features, F represents the Fourier transform, and F -1 G represents the inverse Fourier transform. low This represents a Gaussian low-pass filter, where weights is a hyperparameter.

3. The binocular stereo matching method according to claim 1, characterized in that: In Step 2, the global feature extractor includes: For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16; Then, four Shift-MLP modules are used to extract global features at four scales. C i The corresponding feature maps have 48, 64, 128, and 192 channels, respectively.

4. The binocular stereo matching method according to claim 1, characterized in that: In Step 2, the local feature extractor includes: For the input left and right images First, a 2D convolution with a kernel size of 3×3 is used to expand the image channels to 16; Then, initial local features at four scales are extracted using four Conv Blocks based on 2D convolution. Additionally, the FreqFusion strategy is used to process the initial local features at four scales. Local features are obtained by fusion. The local feature fusion strategies for the four scales are as follows:

5. The binocular stereo matching method according to claim 1, characterized in that: In Step 2, the global-local feature weighting module includes: First, group the global features at the same level. With local features The concatenation is performed along the channel dimension, and the attention on the channel is calculated after two layers of 2D convolution. Finally, the attention-weighted global features are separated from the local features to obtain the weighted global features. and weighted local features In the formula, FW(,) represents the global features. With local features Perform attention weighting and separation.

6. The binocular stereo matching method according to claim 1, characterized in that: In Step 2, the shallow feature fusion module includes: First, two 3×3 2D convolutional layers are used to extract shallow features from the image. Then, Haar wavelet downsampling is applied to the extracted shallow features to obtain a 1 / 4 resolution feature map. pass Global feature maps weighted by 1 / 4 resolution respectively Weighted local feature maps The weighted feature map is updated by concatenation, and the update process is as follows:

7. The binocular stereo matching method according to claim 1, characterized in that: Step 3 includes: Using the two input images I (left and right) l(r) Obtained 1 / 4 resolution weighted global feature map Constructing the GWC cost body: Among them, C gwc () represents a cost body group, d is the disparity index, h is the height index, w is the width index, g represents the cost body group, and N c N represents the number of feature channels. g The number of feature groups is denoted by <·,·>, which represents the inner product. The global feature map is weighted by the left 1 / 4 resolution. Global feature map weighted by resolution in the right 1 / 4; Furthermore, following the RAFT-Stereo method, global APC correlation volume A is calculated on the 1 / 4 resolution weighted global feature map. glo and local APC related body A loc : Where · represents the dot product, w l w r ... The local feature map is weighted by the resolution of the left 1 / 4. The right quarter resolution-weighted local feature map; Following the IGEV-Stereo approach, we increase the receptive field of the fully correlated data by using average pooling with a kernel size of 2 and a stride of 2 in the last dimension, thus constructing a combined encoding of the correlated data that includes two receptive fields.

8. The binocular stereo matching method according to claim 1, characterized in that: In Step 5, the enhanced lookup unit is used to determine the current disparity from C. gwc A glo A loc The relevant feature information is searched in the combined encoded related body and fed into the GRU unit for iteration, including: First, select the current parallax at C. gwc A glo A loc The corresponding matching features are then selected based on the query radius. Finally, all the searched features are concatenated and fed into the GRU iterative unit along with those extracted from the left image by the high-frequency domain enhanced context feature extractor.

9. A stereo matching system for the abdominal cavity environment, characterized in that, The system includes a module for performing a binocular stereo matching method as described in any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a binocular stereo matching method as described in any one of claims 1 to 8.

Images