Parallax estimation method based on uncalibrated binocular image

Abstract

The invention relates to the technical field of image data processing, in particular to a parallax estimation method based on an uncalibrated binocular image. The method comprises the following steps: acquiring an original binocular image pair, and preprocessing the original binocular image pair to obtain standardized binocular image data; generating a center thermodynamic diagram label of the object based on the standardized binocular image data and a preset data set; marking the standardized binocular image data and the corresponding center thermodynamic diagram label as a training sample set; training a preset SAM decoder based on the training sample set to obtain a thermodynamic diagram generation decoder; inputting the original binocular image pair into a thermodynamic diagram generation decoder to obtain a left and right view thermodynamic diagram; performing fixed threshold segmentation on the left and right view thermodynamic diagrams to obtain initial left and right mask images; according to the method, thermodynamic diagram guidance, feature fusion matching and frequency domain analysis strategies are combined, and low-cost, high-precision and high-adaptability target-level depth perception is realized under the condition of weak calibration.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a disparity estimation method based on uncalibrated binocular images. Background Technology

[0002] Traditional binocular disparity estimation methods (such as SGBM) heavily rely on accurate camera calibration parameters, including camera intrinsics, extrinsics, and distortion coefficients. However, in practical applications, many scenarios (such as drones operating under high temperatures or binocular data captured by mobile devices) lack rigorous calibration information, leading to a significant performance degradation of traditional methods. Furthermore, deep learning-based methods typically require large amounts of calibration data for training, resulting in high computational costs and limited generalization capabilities.

[0003] Currently, binocular disparity estimation designs based on uncorrected data face the following problems under the current circumstances: (1) High requirements for calibration accuracy: Traditional disparity estimation algorithms (such as SGBM) rely on strict image calibration, which places extremely high demands on camera parameters, lens distortion correction and lighting conditions. In uncalibrated datasets, these algorithms often cannot be directly applied, resulting in a significant decrease in disparity estimation accuracy.

[0004] (2) Lack of adaptability to uncalibrated scenes: Existing deep learning-based disparity estimation algorithms are mainly optimized for strictly calibrated datasets and are not adaptable to uncalibrated datasets. They cannot effectively handle problems such as unknown camera parameters, lens distortion, lighting changes or scene complexity.

[0005] (3) Overemphasis on high-precision disparity estimation: Most existing methods focus on high-precision disparity estimation, neglecting the important value of coarse but sufficient disparity estimation in practical applications, especially in uncalibrated scenarios. This overemphasis on high precision limits the applicability of existing methods in uncalibrated scenarios.

[0006] (4) Lack of effective use of image features: Existing methods fail to make full use of the geometric characteristics, edge and texture information of images when processing uncalibrated datasets, resulting in insufficient robustness of disparity estimation. Summary of the Invention

[0007] Therefore, it is necessary for the present invention to provide a disparity estimation method based on uncalibrated binocular images to solve at least one of the above-mentioned technical problems.

[0008] To achieve the above objective, a disparity estimation method based on uncalibrated binocular images is provided, comprising the following steps: Step S1: Obtain the original stereo image pairs and preprocess them to obtain standardized stereo image data; generate center heat map labels for objects based on the standardized stereo image data and the preset dataset; denote the standardized stereo image data and the corresponding center heat map labels as the training sample set; Step S2: Train the preset SAM decoder based on the training sample set to obtain the heat map generation decoder; input the original binocular image pairs into the heat map generation decoder to obtain the left and right view heat maps; Step S3: Perform fixed threshold segmentation on the left and right view heatmaps to obtain initial left and right mask images; denoise the initial left and right mask images and extract independent instances through connected component analysis to obtain the left and right mask target regions; Step S4: Calculate the mask size, object feature vector, and mask shape based on the left and right mask target regions to construct multi-feature mask description data; combine and match the left and right mask target regions based on the multi-feature mask description data, and select the best mask pair to obtain the mask pair matching set; Step S5: Calculate the displacement of the mask pair matching set, and perform interpolation and fusion processing to obtain the complete disparity map.

[0009] This invention addresses the problems of decreased accuracy, poor adaptability, high computational cost, and insufficient utilization of image structure in traditional binocular disparity estimation under uncalibrated scenes. It proposes a disparity estimation method based on uncalibrated binocular images, which offers the following advantages: Firstly, by introducing a center heatmap label to achieve prior localization of the salient target region, it avoids reliance on precise camera parameters and spatial epipolar geometry assumptions, improving the robustness of disparity extraction under uncalibrated or weakly calibrated conditions. Secondly, by using a heatmap generation mechanism instead of a traditional end-to-end deep network, it reduces the dependence on large-scale, highly labeled training data, effectively alleviating the problem of insufficient generalization ability of deep learning models on uncalibrated datasets. Thirdly, it employs a fixed threshold segmentation and connected component extraction mechanism to handle complex lighting and background conditions. The system exhibits strong interference suppression capabilities, improving the accuracy of instance-level target separation. By constructing mask description data through multi-dimensional features and introducing a fusion matching mechanism of area, texture, and frequency domain shape, it achieves highly robust matching under weak structural constraints, enhancing the reliability of region correspondence judgment under uncalibrated conditions. By introducing frequency domain phase difference analysis and interpolation fusion strategies, it outputs a disparity map with continuous structure and smooth boundaries while maintaining low computational load. This avoids the impact of traditional pixel-by-pixel stereo matching on issues such as pixel density, occlusion, and distortion sensitivity, improving the usability and engineering adaptability of the disparity map. It is particularly suitable for real-time visual perception tasks in device environments where precise calibration is difficult, such as mobile terminals, drones, and portable AR / VR devices. Attached Figure Description

[0010] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the steps of the disparity estimation method based on uncalibrated binocular images of the present invention; Figure 2 for Figure 1 A detailed flowchart of step S1; Figure 3 for Figure 1 A detailed flowchart of step S3; Figure 4 For calibrated and uncalibrated disparity maps; Figure 5 This is a flowchart illustrating the implementation process of an embodiment of the present invention; Figure 6 This is an image (a) and its corresponding central heat map (b) from an embodiment of the present invention; Figure 7 The images show the left and right heatmaps. Detailed Implementation

[0011] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0012] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0013] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0014] To achieve the above objectives, please refer to Figures 1 to 7This invention provides a disparity estimation method based on uncalibrated binocular images, the method comprising the following steps: Step S1: Obtain the original stereo image pairs and preprocess them to obtain standardized stereo image data; generate center heat map labels for objects based on the standardized stereo image data and the preset dataset; denote the standardized stereo image data and the corresponding center heat map labels as the training sample set; Step S2: Train the preset SAM decoder based on the training sample set to obtain the heat map generation decoder; input the original binocular image pairs into the heat map generation decoder to obtain the left and right view heat maps; Step S3: Perform fixed threshold segmentation on the left and right view heatmaps to obtain initial left and right mask images; denoise the initial left and right mask images and extract independent instances through connected component analysis to obtain the left and right mask target regions; Step S4: Calculate the mask size, object feature vector, and mask shape based on the left and right mask target regions to construct multi-feature mask description data; combine and match the left and right mask target regions based on the multi-feature mask description data, and select the best mask pair to obtain the mask pair matching set; Step S5: Calculate the displacement of the mask pair matching set, and perform interpolation and fusion processing to obtain the complete disparity map.

[0015] Preferably, step S1 includes the following steps: Step S11: Acquire uncalibrated left and right view images using a binocular camera module to obtain the original binocular image pair; Step S12: Scale and resample the original stereo image pairs using nearest neighbor interpolation to unify the image size format and obtain standardized stereo image data; Step S13: Based on the preset dataset, perform center mapping on the target location in the standardized binocular image data to generate a center heat map label, where the heat value is higher in the area closer to the target center and lower in the edge area; Step S14: Pair the standardized binocular image data with the corresponding center heatmap labels, and assign them uniform numbers and bindings to obtain the training sample set.

[0016] In this embodiment of the invention, an integrated binocular camera module is used to acquire raw image data without stereo calibration processing. The binocular camera module consists of two 5-megapixel CMOS image sensors. The optical axis distance between the left and right views is fixed at 6.0 cm, and the sensor sampling frequency is set to 30 frames per second. A hardware synchronous trigger circuit controls the synchronous acquisition of the two-channel image signals to ensure that the left and right view images are acquired in the same time frame, resulting in a raw binocular image pair. The preprocessing module of the image processing server performs a size normalization operation on the raw binocular image pair. First, the image is scaled to 256 pixels along the shorter side while maintaining the aspect ratio. Then, the nearest neighbor interpolation method is used to resample the image to a uniform resolution of 256×256 pixels. This processing method avoids edge blurring caused by bilinear or cubic interpolation, resulting in standardized binocular image data in a uniform format. The object annotation information in the preset dataset COCO2017 is called to standardize the left and right views. The target regions in the image are located. Based on the center point of each extracted target bounding box, a two-dimensional matrix of center heatmap labels is constructed. The gray value of each pixel in the heatmap is calculated based on its Euclidean distance to the center point, with higher values ​​for closer distances. The maximum value is set to 255, and the radius is 15 pixels. Gray values ​​beyond the radius are set to 0. The final generated center heatmap label data maintains a one-to-one correspondence with the standardized binocular image data. The standardized left and right view images and their corresponding center heatmap labels are bound as sample pairs and assigned unique numbers for unified identification. The sample numbers adopt the format "image source_frame number_view label", such as "COCO2017_01523_left". At the same time, the image data and heatmap label data are saved as NumPy array format files, with the naming rules consistent with the numbering, and stored in a high-speed SSD. The path and number index are registered in the sample management index table to form a training sample set.

[0017] This invention, when processing uncalibrated binocular images, achieves effective localization and preprocessing of salient target regions in images without the need for in-camera distortion correction parameters by constructing a training sample set paired with standardized image formats and center heatmap labels. This significantly reduces reliance on precise camera calibration. By unifying image resolution and pixel arrangement, consistency in input dimension and image space is ensured for subsequent feature extraction and heatmap generation operations, improving data processing stability and model input compatibility. The introduction of a target center mapping mechanism, without relying on geometric epipolar constraints, utilizes prior target location information to construct spatial saliency guidance, focusing the training process on high-response regions, thereby improving training efficiency and region recognition accuracy. Sample number binding maintains the structural correspondence between heatmap labels and original images, providing continuous numbering and feature tracking support for subsequent view heatmap generation, mask extraction, and matching, enhancing data organization and traceability throughout the entire process. This method is particularly suitable for scenarios where calibration is difficult, such as portable devices and unmanned systems. While ensuring the visibility of the target structure, it improves the preprocessing quality and task adaptability of the original binocular images in the disparity estimation task.

[0018] Of particular importance, step S13 includes the following steps: Step S131: Load the center point coordinates of the target object based on the preset dataset to obtain the target center coordinate set; Step S132: Mark the target position in the standardized binocular image data according to the target center coordinate set, and generate an initial position marking map; Step S133: Calculate the Euclidean distance from each pixel to the target center based on the initial position marker map to obtain the distance distribution map; Step S134: Apply the Gaussian decay function to the distance distribution map to generate an initial thermal value distribution map; Step S135: Normalize the initial thermal value distribution map, and adjust the thermal value decay rate of the edge region based on the normalization result so that the thermal value of the central region is significantly higher than that of the edge region, and output the central thermal map label.

[0019] In this embodiment of the invention, the labeled target detection information from the publicly available dataset COCO2017 is first called. The rectangular bounding boxes of all target objects in the image are extracted by parsing the "category_id" and "bbox" fields in the annotation file. Coordinate calculation is then performed on each bounding box using the formula... , Obtain the coordinates of the target center point, where and The coordinates of the top left corner of the bounding box. and The width and height are respectively used to form a two-dimensional target center coordinate set. Each item in this coordinate set is stored in a coordinate index matrix in floating-point format for subsequent mapping. The aforementioned target center coordinate set is loaded into the image annotation module. By setting a single-pixel white dot with a value of 255 for each center point coordinate position in the standardized image coordinate system, and setting the values ​​of other pixels to 0, a single-channel binary image is constructed, and an initial position marker image is output. The image size is consistent with the input standardized image, that is, 256×256 pixels. For each pixel coordinate in the initial position marker image ( , ), with the center coordinate point ( , Using this as a reference, calculate the Euclidean distance from this pixel to all target center points using the formula. The distance distribution map of the complete image region is calculated in floating-point form, forming a distance matrix, and stored as a 32-bit floating-point two-dimensional array. This distance distribution map is then input to the image numerical processing module, where a Gaussian decay function is applied pixel-by-pixel to map the distance value of each pixel to a thermal value, using the form... The function, where To fix the standard deviation, set to 5 pixels, an initial heatmap is output. The value range of this heatmap falls between (0,1), with the heatmap value at the center point close to 1, and pixels farther from the center exhibiting exponential decay. Normalization is then performed on the initial heatmap, linearly mapping the heatmap values ​​of all pixels in the entire image to a grayscale range of [0,255]. Maximum value normalization is used, where the normalized maximum value corresponds to grayscale 255. The decay gradient is then reset based on the relationship between heatmap values ​​and boundary distances, ensuring that heatmap values ​​within 15 pixels of the center are greater than 200, and heatmap values ​​greater than 30 pixels of the center are less than 80. The output center heatmap label image maintains the same resolution as the normalized image and serves as the supervision label data for the training phase, stored as a single-channel 8-bit PNG image.

[0020] This invention constructs a spatial thermal distribution mechanism centered on the target center, enabling uncalibrated binocular images to accurately establish spatial guidance for the salient region of the target even without camera parameters and geometric epipolar constraints. This effectively enhances the spatial focusing capability of subsequent heatmap generation and mask extraction. A pixel-level distance mapping is established using the target center point coordinates, combined with a Gaussian decay function to construct a continuous and smooth spatial response map. This achieves a decrease in thermal values ​​from the target core region to the outer region, effectively avoiding the response discontinuity problem caused by traditional hard boundary segmentation. Normalization and edge attenuation adjustment mechanisms further enhance the response contrast between the target region and the background in the image, making subsequent fixed threshold segmentation more robust. It can extract structurally consistent heatmap regions without relying on specific image content, enhancing the robustness of disparity estimation under illumination changes, occlusion interference, and weak texture backgrounds. Furthermore, this scheme is universal across datasets, applicable to standardized heatmap label generation processes for various image sizes, viewpoints, and target types. This significantly reduces data annotation complexity and prior condition dependence, improving the overall system's adaptability and engineering deployment efficiency.

[0021] Preferably, step S2, training the preset SAM decoder based on the training sample set, includes: The standardized binocular image data and corresponding center heatmap labels from the training sample set are loaded sequentially and normalized to obtain normalized input data. Normalized input data is input to a preset SAM decoder, and intermediate feature tensors are extracted and output to the decoder to obtain the decoder input feature tensor. Based on the center heatmap labels corresponding to the training sample set, the decoder input feature tensor is forward computed to obtain the predicted heatmap. The predicted heatmap is compared with the corresponding center heatmap labels in the training sample set at the pixel level, and the loss value is calculated by the mean square error function to obtain the loss function value; The preset SAM decoder is updated based on the loss function value, and multiple training iterations are performed to obtain the heatmap generation decoder.

[0022] In this embodiment of the invention, the training sample set is first imported sequentially into the memory of the image processing server. Each pair of samples consists of a standardized left-view image, a right-view image, and a corresponding center heatmap label. The number of input images in each batch is set to 16. Pixel value normalization is performed on all input images along the channel dimension, transforming the pixel values ​​of the original 8-bit images into [0, 0, ... The normalized input data is obtained by taking the floating-point value range of [1.0]. Then, the normalized left and right view images are input into a set of image feature extraction modules composed of a ViT structure. This module is a preset SAM decoder, mainly composed of multi-layer window-level attention mechanisms and embedded projection layers. After processing by this module, the output is a 256×64×64 intermediate feature tensor, which serves as the input to the decoder. Subsequently, this decoder input feature tensor is input into a deconvolutional decoder structure for feature restoration and upsampling operations. This involves a multi-stage operation including three nearest-neighbor upsampling operations and 1×1 convolutional fusion. Each stage includes batch normalization and activation function processing. The final output is a predicted heatmap of the same size as the original image, with a floating-point two-dimensional image in the range [0.0, 1.0]. To evaluate the error between the predicted heatmap and the true label, the mean square error is calculated at the pixel level for the predicted heatmap and the corresponding center heatmap label in each sample, using the formula: ,in Indicates the first [item] in the predicted heatmap. pixel value, This represents the corresponding pixel value in the label heatmap. The total number of pixels in the image is represented, and a scalar loss function value is output. Based on this loss function value, the parameter update module based on the Adam structure is called. The trainable parameters in the decoder are updated using a fixed learning rate of 1e-4 and a momentum factor of 0.9. All parameters of the SAM decoder are frozen and do not participate in the update. The above operation is repeated for a total of 150 training rounds. Each training round covers the complete training sample set. Finally, after the training error stabilizes and tends to converge, the decoder used to generate the heatmap is obtained.

[0023] This invention achieves precise guided training of a pre-defined SAM decoder in uncalibrated image scenes by introducing a training sample set composed of standardized binocular image data and center heatmap labels, avoiding the dependence of traditional deep networks on complete camera intrinsics and epipolar alignment conditions. During the normalization process, a unified input scale and grayscale distribution are established, effectively reducing training bias caused by differences in illumination and resolution between images. By extracting intermediate feature tensors, semantic feature bridging between the encoder and decoder is achieved, thereby improving the response capability of the decoding stage for spatial localization of the target region. Using the pixel-level mean square error between the labeled heatmap and the prediction results as training feedback, the training process focuses on fitting the thermal response of the real target region, enhancing the decoder's ability to reconstruct the thermal peak of the target's center region. During continuous iterative training, loss values ​​are used to drive parameter updates to ensure joint optimization of the encoder and decoder structures, improving the robustness of the heatmap generation decoder to targets at different viewpoints and scales. The final heatmap generator decoder can efficiently extract target response regions with consistent semantics from the left and right images, providing a stable input foundation for subsequent mask extraction, matching, and disparity estimation, and significantly improving the task adaptability under uncalibrated conditions and the accuracy of structural consistency representation of the target region.

[0024] Preferably, updating the preset SAM decoder based on the loss function value and performing multiple training iterations includes: The loss function value is input into the preset Adam optimizer to calculate the weight gradient and obtain the gradient update data; The parameters of the preset SAM decoder are updated by backpropagation based on the gradient update data to obtain the updated SAM decoder. Repeat the training rounds for updating the SAM decoder until the loss function value meets the preset convergence threshold, and output the heatmap to generate the decoder.

[0025] In this embodiment of the invention, after completing the forward inference and center heatmap prediction for each round of training samples, the pixel-level difference between the predicted heatmap and the corresponding real heatmap label is obtained. The mean squared error loss function value is calculated by averaging the squared errors of each pixel. This loss function value is then used as input to a preset gradient update module. This module performs gradient calculation according to the Adam optimization mechanism, where the initial values ​​of the first-order moment estimation and the second-order moment estimation are set to 0, and the offset correction parameter is set to... , The learning rate is set to 1e-4, and the numerical precision threshold is set to 1e-8. Using these parameters, the partial derivatives of the gradients are calculated for the trainable parameters involved in the current batch, resulting in gradient update data composed of the weight gradients at each parameter position. This gradient update data is input to the parameter adjustment control module, and the gradient values ​​are applied to the preset SAM decoder parameters in the current training process using backpropagation, completing one full weight update operation. The decoder is a deconvolutional structure, and its parameters mainly include kernel weights, bias values, normalization coefficients, and activation channel ratios. All update operations are performed through floating-point multiply-accumulate units with precision controlled within 1e-6. After each weight update, the data is immediately processed from... The next batch of images and labels is loaded into the sample set, and the complete process of input normalization, feature extraction, heatmap prediction, loss calculation, and gradient update is repeated. The entire training process includes 150 training rounds, each round contains 200 batches, and each batch contains 16 pairs of training images and labels, for a total of 480,000 pairs of training samples. During the training process, the trend of training error and validation error is recorded every 5 rounds. When the loss function value decreases by less than 0.001 for 10 consecutive training rounds, it is determined that the preset convergence threshold is met, the training process is terminated, and the heatmap generation decoder in the converged state is finally output. The decoder is then exported as an independent parameter weight file and structure description file.

[0026] This invention, by inputting the loss function value into a preset Adam optimizer, can directly drive the backpropagation correction of the convolution kernel parameters and feature map weights in the SAM decoder, achieving optimal reconstruction of the thermal response of the target region in a high-dimensional feature space. By continuously executing this optimization process in multiple training rounds and using a fixed convergence threshold as the training termination condition, overfitting can be effectively suppressed and generalization ability improved. The resulting heatmap generation decoder has the ability to quickly and accurately identify potential object regions in uncalibrated binocular image pairs, providing a clear and responsive semantic thermal representation for subsequent inter-view matching and disparity extraction.

[0027] Preferably, step S2, inputting the original binocular image pair into the heatmap generation decoder, includes: The heatmap generator decoder is uploaded to the neural network acceleration unit, and the original binocular image pairs are input to the uploaded heatmap generator decoder to decode and generate left and right view heatmaps.

[0028] In this embodiment of the invention, after the heatmap generation training is completed, the network structure description file corresponding to the decoder is exported in JSON format, and the corresponding floating-point parameter weight file is exported in HDF5 format. Both types of files are then uploaded to the neural network acceleration unit and the embedded neural network acceleration unit via hardware interface control commands. A typical embedded acceleration unit is an edge inference module integrated on the image processing motherboard, containing a 256-core parallel processing structure and a floating-point matrix operation engine, supporting FP32 precision inference, and having 2GB of dedicated video memory. Subsequently, the heatmap generation decoder is initialized in the inference engine, and the internal graph computation graph is constructed after loading the structure file and weight file. Afterwards, after the image acquisition module acquires the uncalibrated raw binocular image pair, the left and right view images are transmitted to the embedded acceleration unit via the image buffer module. The input image size is limited to 256×256 pixels, and the image format is a single-channel grayscale image. Pixel values ​​are stored as 8-bit unsigned integers per pixel before preprocessing and converted to floating-point tensors in the range of 0.0 to 1.0 by a normalization processing module before being sent to the decoder. The normalization formula is... ,in These are the original pixel values. After normalization, the normalized image data is input to the starting input of the heatmap generator decoder. The three upsampling modules in the internal structure sequentially restore the spatial size. Each module uses nearest neighbor interpolation to upsample to the next higher size and compresses and fuses the feature map channels through a 1×1 convolutional layer. Finally, a two-dimensional heatmap of the same size as the input image is generated at the output. The gray value of each pixel in the heatmap is represented by a floating-point number, reflecting the response intensity of different regions in the image relative to the target center point. The generated left and right view heatmaps are returned to the main control system storage module through a shared memory channel and cached in the system working directory in NumPy array format.

[0029] This invention uploads the heatmap generator decoder to an embedded neural network acceleration unit, enabling the decoder to run in a dedicated hardware structure. This significantly improves inference efficiency and reduces overall system latency, helping to meet the real-time requirements of edge computing scenarios. The acceleration unit exhibits high parallelism and low power consumption when performing image decoding tasks, maintaining low system resource utilization while ensuring computational stability, thus adapting to the deployment needs of various mobile or embedded platforms. By directly inputting the original binocular image pairs into the deployed heatmap generator decoder, the traditional manual feature extraction and region localization processes can be skipped. Structured left and right view heatmaps are directly output in an end-to-end manner, achieving efficient mapping from the original image to semantic response representation. The high-response regions shown in the left and right view heatmaps accurately locate the object's center region, providing a clear and continuous target candidate region foundation for subsequent mask generation, connected component extraction, and disparity matching processes. This fundamentally improves the robustness and spatial alignment capability of the entire uncalibrated binocular image disparity estimation process.

[0030] Preferably, step S3 includes the following steps: Step S31: Convert the left and right view heatmaps into masks by binarizing with a fixed threshold to obtain the initial left and right mask images; Step S32: Perform an opening operation of erosion and dilation on the initial left and right mask images to eliminate isolated noise and small false target regions, and obtain the opening operation mask image; Step S33: Perform a dilation-erosion closing operation on the opening operation mask image to fill in the small holes at the mask edge and obtain a closed repair mask image; Step S34: Perform pixel connectivity analysis on the closed repair mask image, mark each independent mask region in the image, and obtain a preliminary set of independent mask regions; Step S35: Calculate the pixel area of ​​each mask region in the preliminary independent mask region set, and filter out interference regions based on the preset minimum area threshold to obtain the effective left and right mask target regions.

[0031] In this embodiment of the invention, the left and right view heatmaps output by the heatmap generator decoder are loaded into the image processing module, which is an image processing unit deployed in a GPU-accelerated environment. The module calls a pixel-by-pixel comparison function to perform fixed-threshold binarization processing on each heatmap, setting the threshold to a heatmap value of 128. All pixels in the heatmap with values ​​greater than or equal to 128 are assigned a value of 255, and the rest are assigned a value of 0, generating a single-channel initial left and right mask image. The output image size is the same as the original heatmap. Figure 1The OpenCV image processing library's morphological function interface is called to perform two erosion operations using a 3×3 structuring element on the left and right initial mask images, followed by two dilation operations using the same structuring element, constituting a complete morphological opening operation. This effectively removes isolated noise points and pseudo-target regions with a pixel area less than 25, outputting an opening mask image. The opening mask image is then further processed by closing operations: one dilation operation using a 3×3 structuring element, followed by one erosion operation, closing discontinuous gaps at the mask edges and constructing a complete enclosing region around the target contour, generating a closed repair mask image. Only the closed main contour region is retained in the image. The closed repair is then loaded. The mask image to connected region labeling module uses the four-connectivity rule for pixel connectivity analysis, labels all regions in the image with a value of 255, assigns a unique integer label to each connected region, thus forming a preliminary set of independent mask regions. It also records the bounding box, pixel count, and center coordinates of each mask region, outputting a preliminary mask index table and region set data. For each mask region in the preliminary independent mask region set, a pixel counting operation is performed, calculating the total number of pixels with a value of 255 in each region. Regions with an area threshold of 150 are identified as interference regions and removed from the index table. Valid target regions with areas above the threshold are retained, and valid left and right mask target region data are output.

[0032] This invention achieves rapid conversion from continuous thermal response maps to clear mask maps by performing fixed-threshold binarization on the left and right view heatmaps, avoiding error accumulation caused by complex edge extraction operations. The opening operation, combined with erosion and dilation operations, effectively suppresses background noise and isolated pixel interference, improving the structural integrity of the target region's contour. On this basis, closing operations are performed to seal small gaps and discontinuities in the mask boundaries, making the target contour continuous and separable, facilitating subsequent image region recognition. Furthermore, pixel connectivity analysis accurately distinguishes multiple independent candidate target regions in the image, avoiding region recognition confusion caused by overlapping or adhering multiple targets. By calculating the pixel area of ​​each independent mask region and setting a minimum area threshold, it effectively eliminates false targets, small-region false detections, and other interfering objects, retaining semantically significant and morphologically complete effective mask regions. This provides a spatially accurate and semantically clear target foundation for subsequent feature description and mask matching construction.

[0033] Of particular importance, step S32 includes the following steps: Step S321: Define a 3×3 square structuring element based on the initial left and right mask images to obtain the morphological operation kernel; Step S322: Use morphological operations to perform erosion on the initial left and right mask images to eliminate small noise and obtain the eroded mask image; Step S323: Perform a dilation operation on the erosion mask image using the same kernel to restore the size of the effective target region and obtain the dilated mask image; Step S324: Compare the difference regions between the dilated mask image and the initial left and right mask images, identify the isolated noise points that have been eliminated, and obtain the noise distribution map; Step S325: Correct the edge contour of the dilation mask image according to the noise distribution map to obtain the opening operation mask image.

[0034] In this embodiment of the invention, for the initial left and right mask images, the morphological operation constructor in the image processing module is called to define a 3×3 square structuring element. This structuring element consists of 9 binary units with a value of 1, indicating that 8 adjacent pixels (horizontal, vertical, and diagonal) are considered simultaneously in a local area of ​​the image. This structuring element serves as the convolution kernel for subsequent morphological operations and is named the morphological operation kernel. This kernel is applied to the initial left and right mask images to perform a morphological erosion operation. That is, in the 3×3 neighborhood of each center pixel, the center pixel value is retained as 255 if and only if all adjacent pixel values ​​are 255; otherwise, it is set to 0. After this operation is performed once on the binary image, it can effectively remove isolated edge pixels, thin protruding structures, and scattered areas with an area less than 9 pixels, outputting an eroded mask image. The same kernel is then called to perform a dilation operation, traversing each pixel in the eroded mask image, in each 3×3 region... If any pixel in the image has a value of 255, the corresponding center pixel is set to 255. This operation can restore the compressed effective boundary and reconstruct the target region contour after erosion, outputting an expanded mask image. By comparing the pixel difference map between the expanded mask image and the initial left and right mask images, the positions where the value is 255 in the initial image and 0 in the expanded image are marked as difference regions. These regions are noise regions that have not been restored after erosion, and a noise distribution map is output. This distribution map is a single-channel binary image, where the pixel value of the noise region is set to 255, and the rest are 0. The edge correction module is called to take the noise distribution map as the mask input, and the edge regions of the expanded mask image are scanned sequentially. According to the marked positions in the noise distribution map, the parts connected to the noise in the edge contour are removed or zeroed out to ensure the elimination of edge pseudo-connection phenomenon and avoid noise being mistakenly incorporated into the target contour. Finally, an opening operation mask image is output, which retains the complete target region in its image structure while removing redundant small regions at the boundary.

[0035] This invention effectively removes small, isolated noise points through erosion, enhancing the robustness of the mask image and reducing the false matching rate. The subsequent dilation operation restores the target region's contour size while avoiding the introduction of background interference, thus preserving key object edge information. Further combining difference analysis and contour correction accurately identifies and suppresses false mask regions, improving the geometric consistency and connectivity of the segmentation results. Overall, this processing flow ensures mask clarity while improving the accuracy and stability of subsequent mask matching and disparity estimation, making it particularly suitable for object-level target recognition and registration tasks using uncalibrated binocular images.

[0036] Preferably, step S34 includes the following steps: Step S341: Extract all foreground pixel coordinates based on the closed-in-place mask image to obtain a set of candidate pixels for connected components; Step S342: Perform a four-neighbor traversal search on the candidate pixel set of connected components, mark the pixel region visited for the first time, and obtain the initial connected component labeling map; Step S343: Merge adjacent pixel regions according to the initial connected component labeling map, update the connected component numbers, and obtain the intermediate connected component labeling map; Step S344: Perform cross-region boundary detection on the intermediate connected component labeling graph to eliminate pseudo-connected regions and obtain the optimized connected component labeling graph; Step S345: Based on the optimized connected component labeling graph, count the number of pixels in each connected component and generate a connected component area distribution table; Step S346: Filter the effective independent mask regions according to the connected component area distribution table, and output the preliminary set of independent mask regions.

[0037] In this embodiment of the invention, for a closed-loop repair mask image, the image pixel reading module is called to scan all pixels in the image line by line, extract the coordinates of foreground pixels with a grayscale value of 255, store them in two-dimensional coordinate form, and generate a connected component candidate pixel set. The pixel set data structure is a two-dimensional integer array, the size of which is limited by the image size of 256×256. The connected component candidate pixel set is used as input, and a pixel-by-pixel traversal search process using a four-neighborhood connection strategy is performed. Adjacent pixels in the left, right, top, and bottom directions of the image are selected as connectivity conditions. A stack-based depth-first search is performed on each pixel position, and a unique integer label number is assigned to the first visited pixel region to form an initial connected component label map. In the output labeled image, the pixel value of each region is the same label value, and the remaining background regions are labeled as 0. The number merging module is called to perform region merging processing on the initial connected component label map. For the case where the numbers of adjacent edge regions are inconsistent but share boundaries, different numbers belonging to the same logical region are merged into a unified number through a disjoint-set data structure, and the corresponding pixel values ​​in the entire image are updated. The intermediate connected component labeling map is generated to ensure that all connected regions have consistent numbers and eliminate the problem of overlapping numbers. Cross-region boundary detection is performed on the intermediate connected component labeling map. A 3×3 sliding window is used to scan the entire image sequentially to judge the boundary consistency of the labeled regions. Non-true connected regions caused by image closing operation errors or noise are identified, and pseudo-connected segments with discontinuous numbers but adjacent areas of less than 32 pixels are separated from the original region. The updated numbered labeling map is output as the optimized connected component labeling map. The region statistics module is called to perform pixel counting operations on all non-zero numbered regions in the optimized connected component labeling map, count the number of pixels corresponding to each number, and generate a connected component area distribution table with the number as the index and the area as the value. The area is in pixels and the precision is integer. Conditional filtering is performed on each item in the connected component area distribution table. The area threshold is set to 150 pixels. All numbered regions with a number of pixels less than 150 are removed from the labeling map as invalid regions. The numbers of the remaining regions are kept unchanged. At the same time, the masked region set corresponding to the number is output to form a preliminary independent masked region set.

[0038] This invention effectively extracts and identifies spatially continuous foreground regions in images, ensuring the semantic structural integrity of target regions. Through a refined neighborhood traversal and numbering merging strategy, it enhances the regional consistency of the mask, reducing missegmentation caused by image noise or edge breaks. A cross-region boundary detection mechanism further eliminates pseudo-connected regions, effectively preventing background interference information from being incorrectly identified as target regions. By statistically analyzing and filtering connected component areas, it accurately preserves independent target mask regions with significant scale and physical meaning, providing reliable and structurally clear input for subsequent matching and disparity estimation. This comprehensively improves the accuracy and stability of object-level segmentation tasks, making it particularly suitable for refined instance detection scenarios using uncalibrated binocular images.

[0039] Preferably, in step S4, calculating the mask size, object feature vector, and mask shape based on the left and right mask target regions to construct multi-feature mask description data includes: Pixel counts are performed on the left and right mask target regions, and the area of ​​each mask is calculated to obtain mask area feature data. Feature map embedding is performed on the left and right mask target regions to extract mid-to-high-level semantic features representing shape and texture, and obtain object feature vector data. Fourier transform is performed on the contour boundaries of the left and right mask target regions to extract their amplitude and phase distribution in the frequency domain, thus obtaining the frequency domain feature data of the mask shape. Multi-feature mask description data is constructed using mask area feature data, object feature vector data, and mask shape frequency domain feature data.

[0040] In this embodiment of the invention, firstly, the left and right masked target regions are used as input. The image analysis module is called to perform pixel-by-pixel statistical processing on each masked region. The total number of pixels with a mask value of 255 is used as the result of calculating the region's projected area, and this area is stored in the feature index table in integer form to form mask area feature data. Subsequently, based on the embedded structure generation unit in the image processing device, feature maps are extracted for the corresponding region of each masked region in the standardized image. First, based on the original image, three levels of image texture channels are extracted through convolution processing. Then, the channel features are compressed into a 32-dimensional vector through two average pooling and one-dimensional unfolding operations to describe the mid-to-high-level semantic features such as texture density, edge distribution, and shape orientation of the region, and the object feature vector data is output, where the feature value range is normalized to [−1, 1]. Next, a contour extraction operation is performed on each masked region, and the outermost edge of the region is output using the contour tracking module. The boundary coordinate point sequence is generated, and a boundary signal vector in complex form is constructed for this point set in sequence, where the real part represents the x-coordinate and the imaginary part represents the y-coordinate. Then, the complex sequence is transformed to the frequency domain using the Fast Fourier Transform (FFT) method. The amplitude spectrum and phase spectrum in the frequency domain result are extracted, and the first 32 components with the largest amplitude are retained as the frequency domain shape feature representation of the region, outputting the mask shape frequency domain feature data. Finally, the mask area feature data, object feature vector data, and mask shape frequency domain feature data of each mask region are associated through key-value indexing to construct a unified data dictionary structure. The data is then archived and combined according to the mask number to form multi-feature mask description data. The description data of each mask includes three types of fields in structure: "area", "semantic_vector", and "fft_signature", corresponding to three feature dimensions, and are uniformly stored as a floating-point numerical matrix.

[0041] This invention comprehensively reflects the differences and correspondences between left and right mask target regions in terms of area, structure, and texture, providing a reliable basis for subsequent accurate matching. By combining feature map embedding and frequency domain analysis, it takes into account both the visual semantic information and shape contour details of the object, effectively enhancing the stability and robustness of mask description. Especially in complex scenarios with changes in viewpoint, occlusion, or non-rigid deformation, it can still accurately capture the similarity between targets. At the same time, this multi-feature fusion strategy breaks through the limitations of traditional pixel-level or single-index matching methods, significantly improving the discriminative and fault-tolerant capabilities of mask matching, and laying a solid data foundation for disparity estimation and target alignment in uncalibrated binocular images.

[0042] Preferably, step S4, which involves combining and matching the left and right mask target regions based on multi-feature mask description data and selecting the optimal mask pair, includes: Based on the multi-feature mask description data, left and right mask pairs are filtered according to a preset area ratio threshold to obtain a set of mask pairs with similar areas; For a set of mask pairs with similar areas, the cosine similarity of object feature vectors is calculated to obtain a feature similarity scoring table; Fourier phase correlation matching is performed based on the feature similarity scoring table to obtain the shape similarity scoring table; Based on the shape similarity scoring table, the area, semantics and shape similarity of each mask pair are weighted and fused to obtain a multi-feature mask matching scoring table; The multi-feature mask matching scoring table is sorted by score and filtered by threshold to obtain the mask pair matching set.

[0043] In this embodiment of the invention, the mask area value of each left and right mask target region is first read from the multi-feature mask description data output in the previous stage of step S4. Then, the area ratio of each mask region in the left view to all mask regions in the right view is calculated using the formula... ,in and Let the mask areas of the left and right views be represented respectively. A threshold of 0.75 is set for the area ratio, and only mask pairs with a ratio greater than or equal to this threshold are retained as candidate combinations. A set of mask pairs with similar areas is output. Then, for each of the above mask pairs, the built-in vector similarity calculation module is called. The 32-dimensional object feature vectors of the left and right masks are input into this module, and after normalization, the cosine similarity is calculated using the vector dot product formula. The expression is then used. ,in and The feature vectors of the same numbered masks in the left and right views are calculated, and the results are retained to 4 decimal places. A feature similarity scoring table is output. Next, contour matching processing is performed on mask pairs with a cosine similarity higher than 0.85 in the scoring table. The frequency domain amplitude spectrum and phase spectrum of each mask are used as input. The consistency index between the relative displacement angle and the corresponding point of the main frequency is calculated using a two-dimensional phase difference map. A shape similarity score is generated according to the degree of distribution alignment. The reference metric is set to an error of no more than 15 degrees in the direction of the main axis of the spectrum. A shape similarity scoring table is output. Subsequently, the area similarity (expressed as area ratio) of each mask pair is calculated. Area similarity (represented by cosine score) and shape similarity (represented by phase correlation score) are assigned fixed weights, with area similarity weighted at 0.2, semantic similarity weighted at 0.4, and shape similarity weighted at 0.4. A total score is generated by weighted summation, and a multi-feature mask matching score table is output. Each item in the table records the left mask number, right mask number, and their matching score. Finally, the score table is sorted by score, and a filtering threshold of 0.85 is set. All mask combinations with scores greater than or equal to this threshold are retained as the final mask pair matching set. The output data structure includes the mask number mapping relationship, the corresponding matching score, and various feature indicators.

[0044] The area similarity screening mechanism in this invention can initially eliminate obviously mismatched regions, significantly reducing matching complexity. Secondly, by combining semantic similarity evaluation of object feature vectors and Fourier phase-correlated shape matching, the matching process takes into account both high-level semantic information and low-level contour geometric features, maintaining high matching stability under different viewpoints, scale changes, or occlusion conditions. Furthermore, by constructing a multi-feature scoring table and performing weighted fusion, the transformation of mask matching from a single dimension to a multi-dimensional constraint is effectively realized, enhancing the adaptability to local changes in complex scenes. Finally, the selected high-confidence mask pairs provide accurate and structurally reasonable target correspondences for subsequent disparity estimation, improving the overall accuracy and reliability of the system in uncalibrated binocular image processing tasks.

[0045] Preferably, step S5 includes the following steps: Step S51: Perform image region extraction on the best mask pair matching set to obtain texture image pairs of left and right target regions; Step S52: Perform Fourier transform on the texture image pairs of the left and right target regions to obtain frequency domain representation image pairs; Step S53: Perform phase correlation analysis on the frequency domain image pairs to obtain disparity displacement vector data; Step S54: Combine the disparity displacement vector data with the numbering to generate a preliminary disparity map of the target area; Step S55: Perform boundary interpolation processing on the preliminary target area disparity map to obtain an edge-smoothed disparity map; Step S56: Perform full image fusion processing on the edge-smoothed disparity map to obtain a complete disparity map.

[0046] In this embodiment of the invention, based on the optimal mask pair matching set, the image region extraction module is called to sequentially load the corresponding numbers of each group of left and right mask target regions. A bitwise AND operation is performed in the standardized left and right view images with the mask region as the pixel range, preserving the pixel texture information within the mask region. Pixels in the non-masked region are assigned zero values, generating a pair of texture images of the left and right target regions. The image format is a single-channel 8-bit grayscale image, with a size maintained at 256×256 pixels. The frequency domain processing module is called to perform two-dimensional fast Fourier transforms on the texture images of the left and right target regions respectively. First, the input image is converted to a floating-point matrix format, and after centering preprocessing, the spectrum calculation is accelerated by decomposing the prime factor. The output frequency domain representation image pair includes the real and imaginary parts in complex form, and the number of frequency components is consistent with the image size. Phase correlation analysis is performed on the left and right frequency domain images, calculating their amplitude and phase spectra respectively, and constructing a normalized cross-power spectrum using the formula... ,in and For the left and right figures, the frequency domain complex numbers are... The process involves performing conjugate complex number operations, followed by a two-dimensional inverse transform on the cross-power spectrum to obtain a phase correlation response map. The location of the maximum value is searched within this response map; the corresponding coordinate difference is the disparity displacement vector for that mask pair. The output is a disparity displacement vector containing the mask number and the horizontal displacement value. Next, the region disparity filling module is called to write each disparity displacement value into all pixel positions within the corresponding mask region according to its number, generating a preliminary target region disparity map. All pixel regions not covered by the mask are marked as zero, and the image format is a 32-bit floating-point matrix. Then, the boundary interpolation module is called to perform linear interpolation on the adjacent regions of the mask edges, transitioning the boundary from an abrupt change between disparity values ​​and the background to a continuous gradient. The interpolation method uses a linear weighted average based on the mask edge normal vector direction, with an interpolation width of 5 pixels, outputting a smoothed edge disparity map. Finally, all smoothed disparity maps of the target regions are merged according to their index, and image overlay and fusion processing is performed across the entire image. An average fusion operation is performed on the overlapping areas, and zeros are padded in the uncovered areas, ultimately outputting a complete disparity map with the same image size as the original. Figure 1 The data structure is a 32-bit floating-point depth map.

[0047] This invention significantly reduces the impact of redundant background interference on disparity estimation and improves the accuracy of region localization by extracting the left and right texture regions corresponding to the best matching mask. Fourier transform and phase correlation methods achieve displacement estimation in the frequency domain, effectively avoiding the problems of spatial domain methods being susceptible to occlusion and illumination changes, ensuring the stability and robustness of displacement vector calculation. The disparity map construction process incorporates mask numbering for structural organization, ensuring the independence and consistency among multiple target regions. Boundary interpolation and fusion processing further optimize the edge transition effect of the disparity map, eliminating breakage and abrupt changes, and improving visual continuity and overall readability. The final generated complete disparity map possesses excellent detail preservation and overall structure perception capabilities, providing accurate and reliable depth information support for subsequent tasks such as environmental understanding, target recognition, and 3D reconstruction, and is particularly suitable for real-world applications under uncalibrated binocular image conditions.

[0048] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is not limited by the foregoing description. Thus, all changes falling within the meaning and scope of the equivalents of the application are intended to be included within the scope of the invention.

[0049] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A disparity estimation method based on uncalibrated binocular images, characterized in that, Includes the following steps: Step S1: Obtain the original stereo image pairs and preprocess them to obtain standardized stereo image data; Center heatmap labels for objects are generated based on standardized binocular image data and a pre-set dataset; the standardized binocular image data and the corresponding center heatmap labels are recorded as the training sample set; Step S2: Train the preset SAM decoder based on the training sample set to obtain the heat map generation decoder; input the original binocular image pairs into the heat map generation decoder to obtain the left and right view heat maps; Step S3: Perform fixed threshold segmentation on the left and right view heatmaps to obtain initial left and right mask images; denoise the initial left and right mask images and extract independent instances through connected component analysis to obtain the left and right mask target regions; Step S4: Calculate the mask size, object feature vector, and mask shape based on the left and right mask target regions to construct multi-feature mask description data; combine and match the left and right mask target regions based on the multi-feature mask description data, and select the best mask pair to obtain the mask pair matching set; Step S5: Calculate the displacement of the mask pair matching set, and perform interpolation and fusion processing to obtain the complete disparity map.

2. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Acquire uncalibrated left and right view images using a binocular camera module to obtain the original binocular image pair; Step S12: Scale and resample the original stereo image pairs using nearest neighbor interpolation to unify the image size format and obtain standardized stereo image data; Step S13: Based on the preset dataset, perform center mapping on the target location in the standardized binocular image data to generate a center heat map label, where the heat value is higher in the area closer to the target center and lower in the edge area; Step S14: Pair the standardized binocular image data with the corresponding center heatmap labels, and assign them uniform numbers and bindings to obtain the training sample set.

3. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, Step S2, which involves training a pre-defined SAM decoder based on the training sample set, includes: The standardized binocular image data and corresponding center heatmap labels from the training sample set are loaded sequentially and normalized to obtain normalized input data. Normalized input data is input to a preset SAM decoder, and intermediate feature tensors are extracted and output to the decoder to obtain the decoder input feature tensor. Based on the center heatmap labels corresponding to the training sample set, the decoder input feature tensor is forward computed to obtain the predicted heatmap; The predicted heatmap is compared with the corresponding center heatmap labels in the training sample set at the pixel level, and the loss value is calculated by the mean square error function to obtain the loss function value; The preset SAM decoder is updated based on the loss function value, and multiple training iterations are performed to obtain the heatmap generation decoder.

4. The disparity estimation method based on uncalibrated binocular images according to claim 3, characterized in that, Update the preset SAM decoder based on the loss function value, and perform multiple training iterations including: The loss function value is input into the preset Adam optimizer to calculate the weight gradient and obtain the gradient update data; The parameters of the preset SAM decoder are updated by backpropagation based on the gradient update data to obtain the updated SAM decoder. Repeat the training rounds for updating the SAM decoder until the loss function value meets the preset convergence threshold, and output the heatmap to generate the decoder.

5. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, Step S2, which inputs the original stereo image pair into the heatmap generation decoder, includes: The heatmap generator decoder is uploaded to the neural network acceleration unit, and the original binocular image pairs are input to the uploaded heatmap generator decoder to decode and generate left and right view heatmaps.

6. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Convert the left and right view heatmaps into masks by binarizing with a fixed threshold to obtain the initial left and right mask images; Step S32: Perform an opening operation of erosion and dilation on the initial left and right mask images to eliminate isolated noise and small false target regions, and obtain the opening operation mask image; Step S33: Perform a dilation-erosion closing operation on the opening operation mask image to fill in the small holes at the mask edge and obtain a closed repair mask image; Step S34: Perform pixel connectivity analysis on the closed repair mask image, mark each independent mask region in the image, and obtain a preliminary set of independent mask regions; Step S35: Calculate the pixel area of ​​each mask region in the preliminary independent mask region set, and filter out interference regions based on the preset minimum area threshold to obtain the effective left and right mask target regions.

7. The disparity estimation method based on uncalibrated binocular images according to claim 6, characterized in that, Step S34 includes the following steps: Step S341: Extract all foreground pixel coordinates based on the closed-in-place mask image to obtain a set of candidate pixels for connected components; Step S342: Perform a four-neighbor traversal search on the candidate pixel set of connected components, mark the pixel region visited for the first time, and obtain the initial connected component labeling map; Step S343: Merge adjacent pixel regions according to the initial connected component labeling map, update the connected component numbers, and obtain the intermediate connected component labeling map; Step S344: Perform cross-region boundary detection on the intermediate connected component labeling graph to eliminate pseudo-connected regions and obtain the optimized connected component labeling graph; Step S345: Based on the optimized connected component labeling graph, count the number of pixels in each connected component and generate a connected component area distribution table; Step S346: Filter the effective independent mask regions according to the connected component area distribution table, and output the preliminary set of independent mask regions.

8. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, In step S4, the mask size, object feature vector, and mask shape are calculated based on the left and right mask target regions to construct multi-feature mask description data, including: Pixel counts are performed on the left and right mask target regions, and the area of ​​each mask is calculated to obtain mask area feature data. Feature map embedding is performed on the left and right mask target regions to extract mid-to-high-level semantic features representing shape and texture, and obtain object feature vector data. Fourier transform is performed on the contour boundaries of the left and right mask target regions to extract their amplitude and phase distribution in the frequency domain, thus obtaining the frequency domain feature data of the mask shape. Multi-feature mask description data is constructed using mask area feature data, object feature vector data, and mask shape frequency domain feature data.

9. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, In step S4, the left and right mask target regions are combined and matched based on the multi-feature mask description data, and the best mask pair is selected, including: Based on the multi-feature mask description data, left and right mask pairs are filtered according to a preset area ratio threshold to obtain a set of mask pairs with similar areas; For a set of mask pairs with similar areas, the cosine similarity of object feature vectors is calculated to obtain a feature similarity scoring table; Fourier phase correlation matching is performed based on the feature similarity scoring table to obtain the shape similarity scoring table; Based on the shape similarity scoring table, the area, semantics and shape similarity of each mask pair are weighted and fused to obtain a multi-feature mask matching scoring table; The multi-feature mask matching scoring table is sorted by score and filtered by threshold to obtain the mask pair matching set.

10. The disparity estimation method based on uncalibrated binocular images according to claim 1, characterized in that, Step S5 includes the following steps: Step S51: Perform image region extraction on the best mask pair matching set to obtain texture image pairs of left and right target regions; Step S52: Perform Fourier transform on the texture image pairs of the left and right target regions to obtain frequency domain representation image pairs; Step S53: Perform phase correlation analysis on the frequency domain image pairs to obtain disparity displacement vector data; Step S54: Combine the disparity displacement vector data with the numbering to generate a preliminary disparity map of the target area; Step S55: Perform boundary interpolation processing on the preliminary target area disparity map to obtain an edge-smoothed disparity map; Step S56: Perform full image fusion processing on the edge-smoothed disparity map to obtain a complete disparity map.

Images