A method and system for environment simulation based on binocular stereo vision measurement

Abstract

The application discloses a kind of environment simulation method and system based on binocular stereo vision measurement, it is related to scene simulation technical field, and its technical solution points are: the method includes by constructing the physical imaging model in left and right optical elements in binocular stereo vision camera, and left and right view is obtained based on the physical imaging model;Determine the disparity map between the left and right view, determine scene depth image based on the disparity map;Three and more scene depth images generated by three and more different binocular stereo vision cameras corresponding to the scene are acquired, and the three and more scene depth images are fused, to construct the three-dimensional simulation model corresponding to the scene based on the depth image after fusion.The application can accurately reflect the depth level and spatial relationship of scene, and the three-dimensional simulation model finally constructed is more realistic, and can strongly support virtual reality, three-dimensional measurement and other applications.

Description

Technical Field

[0001] This invention relates to the field of scene simulation technology, and more specifically, to an environment simulation method and system based on binocular stereo vision measurement. Background Technology

[0002] In the field of environmental simulation, accurately constructing virtual scenes that are highly consistent with the real physical world is a core requirement to support applications such as autonomous driving testing, robot navigation training, and industrial digital twins.

[0003] However, current mainstream environmental simulation methods still have significant limitations: traditional LiDAR-based simulation schemes can provide high-precision point clouds, but multi-line LiDAR is expensive and its frame rate is low due to mechanical rotation structure, making it difficult to capture instantaneous changes in dynamic scenes; while in pure vision simulation methods, monocular vision lacks depth information and relies on prior models or strong assumptions, which easily leads to scale drift in complex unstructured scenes. Therefore, there is an urgent need for an environmental simulation method that can effectively support 3D simulation of virtual reality scenes. Summary of the Invention

[0004] The purpose of this invention is to provide an environment simulation method and system based on binocular stereo vision measurement to solve the above-mentioned technical problems.

[0005] The above-mentioned technical objective of this invention is achieved through the following technical solution: an environment simulation method based on binocular stereo vision measurement, comprising the following steps:

[0006] S110. Construct a physical imaging model of the left and right optical elements in the binocular stereo vision camera, and obtain the left and right views based on the physical imaging model;

[0007] S120. Determine the disparity map between the left and right views, and determine the scene depth image based on the disparity map;

[0008] S130. Obtain three or more scene depth images generated by three or more different binocular stereo vision cameras corresponding to the scene, and fuse the three or more scene depth images to construct a three-dimensional simulation model corresponding to the scene based on the fused depth image.

[0009] The present invention is further configured such that the method also includes:

[0010] S210. Determine the corresponding image feature points in the left and right views based on the pixel change intensity in the left and right views;

[0011] S220. Determine the corner points in the left and right views and the corner point response intensity corresponding to the corner points, and optimize the image feature points based on the corner point response intensity to obtain optimized image feature points;

[0012] S230. Update the left and right views based on the optimized image feature points to obtain updated left and right views, and determine the scene depth image based on the updated left and right views.

[0013] The present invention is further configured such that the method also includes:

[0014] S310. After correcting the distortion of the optimized image feature points, transform them to the camera coordinate system to obtain the feature point coordinate information;

[0015] S320. Determine the epipolar equation of the binocular stereo vision camera based on the rotation matrix and translation matrix of the binocular stereo vision camera, and calculate the distance from the optimized image feature points in the right view to the epipolar line based on the epipolar equation.

[0016] S330. If the distance is greater than a preset distance threshold, then the corresponding image feature points in the right view are removed.

[0017] The present invention is further configured such that: in step S120, determining a disparity map between the left and right views and determining a scene depth image based on the disparity map includes:

[0018] (1) A multi-scale cost aggregation strategy is adopted to obtain the disparity search space, and the disparity map between the left and right views is determined based on the disparity search space;

[0019] (2) Obtain the baseline and focal length of the binocular stereo vision camera, and calculate the scene depth image corresponding to the disparity map based on the baseline and the focal length.

[0020] The present invention is further configured such that: in step S120, determining a disparity map between the left and right views and determining a scene depth image based on the disparity map includes:

[0021] 1) Input the left and right views into a preset image processor, the preset image processor including a neural network processor;

[0022] 2) Based on the neural network processor, cost aggregation calculation is performed on the left and right views to determine the disparity map in the left and right views based on the cost aggregation result;

[0023] 3) Determine the depth value corresponding to the disparity in the disparity map based on a preset mapping table, and determine the scene depth image based on the depth value. The preset mapping table includes the mapping relationship between disparity and depth value.

[0024] The present invention is further configured to: fuse three or more scene depth images in S130, including:

[0025] (1) Perform multi-scale decomposition on the three or more scene depth images respectively to obtain three or more levels corresponding to the three or more scene depth images;

[0026] (2) Based on the three or more levels, cross-scale consistency constraints are applied to the three or more scene depth images to obtain cross-scale constraint conditions;

[0027] (3) The depth images of the three or more scenes are fused based on the cross-scale constraints.

[0028] The present invention is further configured to: fuse three or more scene depth images in S130, including:

[0029] 1) Perform temporal alignment on the three or more scene depth images to obtain an aligned depth image sequence;

[0030] 2) Determine the dynamic confidence level corresponding to the three or more scene depth images, and fuse the three or more scene depth images based on the dynamic confidence level and the depth image sequence.

[0031] The present invention is further configured to: fuse three or more scene depth images in S130, including:

[0032] 1. Determine the epipolar constraint equations for any two of the three or more different binocular stereo vision cameras;

[0033] 2° Based on the epipolar constraint equation, perform epipolar matching on the scene depth images corresponding to any two binocular stereo vision cameras to obtain the matching result;

[0034] 3° Based on the matching results, the depth images of the three or more scenes are fused.

[0035] The present invention is further configured such that: in step S130, a three-dimensional simulation model corresponding to the scene is constructed based on the fused depth image, including:

[0036] (1) Based on the fused depth image, a textured dense point cloud is determined, which is used to delineate the outline of the scene;

[0037] (2) Based on the dense point cloud, the Poisson surface of the scene is reconstructed to obtain the mesh model corresponding to the scene;

[0038] (3) Perform multi-view texture mapping on the mesh model to obtain the three-dimensional simulation model corresponding to the scene.

[0039] The present invention further provides an environment simulation system based on binocular stereo vision measurement. The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the above-described environment simulation method based on binocular stereo vision measurement.

[0040] In summary, this invention has the following beneficial effects: By constructing a physical imaging model of the left and right optical elements in a binocular stereo vision camera and generating left and right views accordingly, it accurately simulates the binocular imaging principle of the human eye, enabling highly realistic reproduction of image information of the scene from the left and right perspectives. This provides a reliable foundation for subsequent disparity calculation. By determining the disparity map between the left and right views, a scene depth image is obtained. The disparity map reflects the difference in position of the same object in the left and right views, and this difference is closely related to the distance of the object from the camera. Based on this, the calculated depth image can accurately represent the distance of each object in the scene from the camera, providing crucial depth information for 3D reconstruction. This system can acquire and fuse three or more scene depth images generated by three or more different binocular stereo vision cameras. Different cameras collect scene information from different positions and angles. The fusion process can combine the advantages of each camera, make up for the information loss and errors caused by the limited field of view of a single camera, and effectively improve the integrity and accuracy of the depth images. Finally, a 3D simulation model corresponding to the scene is constructed based on the fused depth image. This model not only closely matches the real scene in geometry, but also accurately reflects the depth level and spatial relationship of the scene. The final 3D simulation model is more realistic and can strongly support applications such as virtual reality and 3D measurement. Attached Figure Description

[0041] Figure 1 This is a flowchart of the environment simulation method based on binocular stereo vision measurement in Embodiment 1 of the present invention.

[0042] Figure 2 This is a schematic diagram of a multi-scale decomposition based on an image pyramid in Embodiment 4 of the present invention;

[0043] Figure 3 This is a block diagram of the environment simulation system based on binocular stereo vision measurement in Embodiment 10 of the present invention. Detailed Implementation

[0044] The following is in conjunction with the appendix Figure 1-3 The present invention will be described in further detail below.

[0045] Description: Binocular stereo vision measurement is a passive 3D perception technology based on the parallax principle. By simulating the stereo vision mechanism of human eyes, it uses two cameras to acquire images of the same scene from different perspectives. The depth information of the scene is reconstructed by calculating the positional differences (parallax) of pixels in the left and right images, ultimately generating a dense 3D point cloud. Due to its low cost, simple structure, and strong adaptability, this technology is widely used in fields such as robot navigation, autonomous driving, industrial inspection, and virtual reality.

[0046] Binocular stereo vision cameras achieve 3D measurement by simulating the stereo perception mechanism of human eyes. Their core principle is based on triangulation: two cameras with identical parameters (optical center distance B) simultaneously acquire images of the same scene from different perspectives. After calibration and correction to align the epipolar lines of the left and right images, for any scene point P, its coordinates in the left image are calculated. (and the corresponding point in the right figure) Horizontal displacement difference (parallax) ), combined with camera focal length Using the relationship of similar triangles Derive the point Depth to the camera Finally, by traversing all pixels, a system containing spatial coordinates is generated. The dense 3D point cloud enables the reconstruction of 3D scenes from 2D images.

[0047] Because existing 3D scene construction technologies have significant shortcomings in terms of accuracy, efficiency, cost, and adaptability, the embodiments of this application propose an environment simulation method based on binocular stereo vision measurement, an environment simulation system based on binocular stereo vision measurement, an electronic device, a computer storage medium, and a computer program product, which will be described in detail below.

[0048] Example 1: As Figure 1 As shown, in this embodiment, an environment simulation method based on binocular stereo vision measurement includes at least steps S110 to S130, which are described in detail below:

[0049] Step S110: Construct physical imaging models of the left and right optical elements in the binocular stereo vision camera, and obtain left and right views based on the physical imaging models.

[0050] For example, in a binocular stereo vision camera, the physical imaging model of the left and right optical elements is constructed based on the biomimetic principle of the human eye: when light reflects off the surface of an object in three-dimensional space, the lens of the left camera focuses this light onto its internal imaging sensor. After refraction by the lens, light from different positions on the same surface of the object forms corresponding pixels on the sensor according to geometric rules. The arrangement of these pixels constitutes the two-dimensional projection of the object from the left camera's perspective, i.e., the left view. At the same time, the right camera works independently on the same principle, but because its optical center is at a fixed horizontal distance from the left camera, light reflected from the same object surface, after being refracted by the right camera's lens, forms a pixel arrangement on its sensor with a horizontal displacement difference compared to the left view. This displacement difference reflects the distance information from different points on the object's surface to the camera, ultimately generating the right view. By recording the projection differences of the same object from different perspectives, the left and right views provide basic data for subsequent calculation of parallax and reconstruction of the three-dimensional structure.

[0051] Specifically, when constructing the physical imaging model of the left and right optical elements of a binocular stereo vision camera, it is necessary to base it on the pinhole imaging principle and lens imaging formula, assuming that the optical centers of the left and right cameras are respectively... and The baseline distance is And the intrinsic parameters of the two cameras, such as focal length Principal point coordinates Consistent, when 3D scene points After being refracted by the left camera lens, the emitted light rays are projected onto the imaging plane as pixels. Its coordinates satisfy , Similarly, the projection point of the right camera satisfy, , (Because the left and right image rows are aligned after epipolar correction), then the parallax of the left and right views can be obtained through this physical imaging model. This generates left and right views, where the left view records... Original projection on the left imaging plane The right view is based on depth Calculate its corresponding position in the right plane. Ultimately, the left and right views together form the basis for parallax calculation, supporting subsequent 3D point cloud reconstruction.

[0052] Step S120: Determine the disparity map between the left and right views, and determine the scene depth image based on the disparity map.

[0053] For example, in a binocular stereo vision system, the process of determining the disparity map between the left and right views and generating a scene depth image is as follows: First, epipolar correction is performed on the left and right views so that corresponding points in the left and right images are located on the same horizontal scan line; then, a matching algorithm (such as block matching or semi-global matching, SGM) is used to search for corresponding points in the right view pixel by pixel in the left view. The difference in horizontal coordinates between the corresponding points in the left and right views is the disparity value. The disparity values ​​of all pixels are combined to form the disparity map; since disparity is inversely proportional to depth (depth... ,in, Baseline distance, Focal length (For parallax), by substituting each pixel value in the parallax map into this relationship, the depth value of the corresponding scene point can be calculated, and finally a depth image is generated, in which the grayscale value or color value of each pixel represents its actual depth from the camera.

[0054] Step S130: Obtain three or more scene depth images generated by three or more different binocular stereo vision cameras corresponding to the scene, and fuse the three or more scene depth images to construct a three-dimensional simulation model corresponding to the scene based on the fused depth image.

[0055] For example, in a multi-view 3D reconstruction task, scene data must first be acquired synchronously by three or more binocular stereo vision cameras deployed at different locations or poses. Each camera independently generates an initial scene depth image based on its own baseline distance and focal length parameters. These depth images are complementary due to differences in camera perspectives. Some areas are missing in a single depth image due to occlusion or perspective limitations, while depth images from other perspectives may contain complete information about that area. Subsequently, when fusing multiple depth images, coarse registration of the multi-view depth images must first be achieved through feature point matching or ICP (Iterative Closest Point) algorithms to unify the depth data from different coordinate systems to the global coordinate system. Then, a weighted average or confidence-based fusion strategy is used to process overlapping areas: for overlapping pixels, weights are assigned according to the measurement accuracy of each depth image (such as disparity matching score and texture richness), prioritizing the use of higher-weighted pixels. The reliability measurement value is used to fill missing regions caused by occlusion or matching failure through hole filling algorithms (such as neighborhood diffusion or plane fitting). The fused depth map needs to be further post-processed, including bilateral filtering to remove noise and morphological operations to optimize edge continuity, so as to obtain a geometrically consistent and complete global depth map. Based on this fused depth map, a 3D model can be constructed by voxel meshing or point cloud generation technology: after back-projecting the depth map into a 3D point cloud, surface meshes are generated by Poisson reconstruction or Delaunay triangulation, or implicit surfaces are constructed by directly using TSDF (truncated sign distance function) to fuse multi-view depth data. The final output is a 3D simulation model containing scene geometry and spatial topology. This model significantly improves the reconstruction accuracy and completeness through multi-view data fusion, and is especially suitable for real-time updates and interactive simulations of large-scale complex scenes or dynamic scenes.

[0056] Optionally, in this embodiment, n binocular stereo vision cameras (with camera index ) are deployed at different positions or orientations. Scene data is collected synchronously. Each camera acquires data based on its own baseline distance. and focal length Through parallax With depth The relationship between the initial scene depth image and the other elements is to generate the image independently.

[0057]

[0058] in, These depth images are complementary due to differences in camera vision, with some areas appearing in a single depth image due to occlusion or viewpoint limitations. The depth map is missing from the middle, while other perspectives are also missing. This may contain complete information about the region. Then, coarse registration of the multi-view depth map is achieved using feature point matching or the ICP (Iterative Nearest Point) algorithm. The goal of the ICP algorithm is to minimize the distance between two point sets. and The error between them is defined by the error function:

[0059]

[0060] in, and They are corresponding point pairs. It represents the number of corresponding point pairs. It is a rotation matrix. It is a translation vector, optimized through iterative processes. and This process unifies depth data from different coordinate systems to a global coordinate system. Then, it uses a weighted average or confidence-based fusion strategy to process overlapping regions. For overlapping pixels... Based on the measurement accuracy of each depth map ( Texture richness Assign weights A common weight allocation method is as follows:

[0061]

[0062] in, and These are parameters that adjust the weights of disparity matching score and texture richness, representing the depth value after fusion. It can be represented as:

[0063]

[0064] Meanwhile, hole-filling algorithms (such as those based on neighborhood diffusion or plane fitting) can be used to repair areas that are actually occluded or have failed to match. Hole-filling based on neighborhood diffusion can be represented as:

[0065]

[0066] in, It is a pixel The neighborhood. N is the number of valid pixels within the neighborhood. The depth values ​​of the effective pixels within the domain are used to obtain the fused depth map. Then, based on this fused depth map, a 3D model can be constructed using voxel meshing or point cloud generation techniques: after back-projecting the depth map into a 3D point cloud, a surface mesh is generated using Poisson reconstruction or Delaunay triangulation, or an implicit surface can be constructed by directly using TSDF (truncated sign distance function) to fuse multi-view depth data, ultimately outputting a 3D simulation model that includes the scene's geometry and spatial topology.

[0067] In this embodiment, a physical imaging model of the left and right optical elements of a binocular stereo vision camera is constructed to generate left and right views, which can accurately simulate real binocular imaging and provide a reliable basis for disparity calculation. The disparity map is determined to obtain a scene depth image, which can accurately obtain the depth information of objects in the scene. By fusing scene depth images generated by three or more different binocular cameras, the advantages of multiple perspectives can be combined to improve the integrity and accuracy of the depth image. The final constructed 3D simulation model is more realistic and can strongly support applications such as virtual reality and 3D measurement.

[0068] Example 2: Based on Example 1, in this example, the specific implementation process of the above-mentioned environment simulation method based on binocular stereo vision measurement may further include steps S210 to S230, which are described in detail below:

[0069] Step S210: Determine the corresponding image feature points in the left and right views based on the intensity of pixel changes in the left and right views;

[0070] Step S220: Determine the corner points in the left and right views and the corner point response intensity corresponding to the corner points, and optimize the image feature points based on the corner point response intensity to obtain optimized image feature points;

[0071] Step S230: Update the left and right views based on the optimized image feature points to obtain updated left and right views, and determine the scene depth image based on the updated left and right views.

[0072] For example, in a binocular stereo vision system, image feature points are first extracted based on the pixel intensity changes in the left and right views: significant change regions within the pixel neighborhood are detected by calculating the image gradient (e.g., using the Sobel operator) or by using algorithms such as FAST and SIFT, and pixels with gradient magnitudes exceeding a threshold or meeting specific invariance conditions are selected as candidate feature points; subsequently, the Harris corner detection algorithm is used to locate corners in the image. This algorithm evaluates the local geometry by calculating the eigenvalues ​​of the pixel autocorrelation matrix, and the corner response intensity is quantified by the ratio of the matrix determinant to the trace (e.g., R=det(M)-k·trace²(M)), and points with response values ​​higher than the threshold are identified as corners; to optimize feature point quality, feature points are filtered based on corner response intensity: the top N% of feature points in terms of response intensity are retained, and low-quality points with response values ​​lower than the local neighborhood mean are removed, while non-maximum suppression (NM) is applied. S) Remove redundant points in dense areas to ensure uniform distribution of feature points; update the left and right views using the optimized feature points, specifically including: performing sub-pixel-level precise localization of feature points (such as interpolation to improve coordinate accuracy), and eliminating mismatched point pairs using the RANSAC algorithm. Then, perform geometric correction on the image based on the feature point coordinates (such as affine transformation or perspective transformation) to ensure that the left and right views strictly satisfy epipolar constraints; finally, calculate the disparity map based on the corrected left and right views: use semi-global matching (SGM) or deep learning models (such as PSMNet) to search for the best matching point in the epipolar direction, obtain a dense disparity map through cost aggregation and disparity optimization, and then convert the disparity into depth values ​​based on the camera baseline distance and focal length parameters to generate a scene depth image. This process significantly improves the matching accuracy and robustness of depth estimation through feature point optimization, and is especially suitable for 3D reconstruction tasks in low-texture or repetitive texture scenes.

[0073] In this embodiment, image feature points are determined based on the intensity of pixel changes in the left and right views, which can quickly capture areas with significant changes in the image and provide a basis for subsequent matching. Determining corner points and response intensity and optimizing feature points accordingly can eliminate unstable and mismatched points and improve the quality of feature points. The optimized feature points are used to update the left and right views and then determine the scene depth image, making the depth calculation more accurate and the resulting depth image can more realistically reflect the spatial information of the scene.

[0074] Example 3: Based on Example 2, in this example, the specific implementation process of the above-mentioned environment simulation method based on binocular stereo vision measurement may further include steps S310 to S330, which are described in detail below:

[0075] Step S310: After the optimized image feature point distortion is corrected, the image is transformed into the camera coordinate system to obtain the feature point coordinate information;

[0076] Step S320: Determine the epipolar equation of the stereo vision camera based on the rotation matrix and translation matrix of the stereo vision camera, and calculate the distance from the optimized image feature points in the right view to the epipolar line based on the epipolar equation.

[0077] Step S330: If the distance is greater than a preset distance threshold, then the corresponding image feature points in the right view are removed.

[0078] For example, when processing image feature points, the optimized image feature points are first subjected to distortion correction. A specific distortion correction algorithm is used to eliminate the influence of lens distortion on the position of the feature points. After correction, these feature points are transformed into the camera coordinate system to obtain their accurate coordinate information. Next, the known rotation and translation matrices are used to determine the epipolar equation of the camera. The rotation matrix describes the rotation relationship of the camera under different viewpoints, and the translation matrix reflects the change of the camera position. Based on these two matrices, the epipolar equation can be derived through specific mathematical derivation. This equation describes the geometric constraint relationship between corresponding feature points in the left and right views. Then, based on the calculated epipolar equation, the distance from the optimized image feature point in the right view to this epipolar line is calculated. This distance reflects the degree of deviation between the feature point and the epipolar line. Finally, the calculated distance is compared with a preset distance threshold. If the distance is greater than the preset distance threshold, it means that the corresponding image feature point in the right view deviates too much from the epipolar line and does not meet the geometric constraints of camera imaging. In this case, the corresponding image feature point in the right view is removed, thereby improving the accuracy and reliability of feature point matching.

[0079] In this embodiment, the optimized image feature points are distorted and transformed to the camera coordinate system, which can eliminate the influence of lens distortion and obtain accurate spatial feature point coordinates. The epipolar equation is determined based on the rotation and translation matrix and the distance from the right view feature points to the epipolar line is calculated. The epipolar line constraint can be used to judge the rationality of the matching. Right view feature points with a distance greater than the threshold are removed, which can effectively remove mismatched points, improve the feature point matching accuracy, and provide more reliable data for subsequent scene depth calculations.

[0080] Example 4: Based on Example 1, in this example, the specific implementation process of determining the disparity map between the left and right views and determining the scene depth image based on the disparity map may further include the following steps, which are detailed below:

[0081] Step (1): A multi-scale cost aggregation strategy is used to obtain the disparity search space, and the disparity map between the left and right views is determined based on the disparity search space;

[0082] Step (2): Obtain the baseline and focal length of the binocular stereo vision camera, and calculate the scene depth image corresponding to the disparity map based on the baseline and the focal length.

[0083] For example, in binocular stereo vision, to obtain a high-precision disparity map, a multi-scale cost aggregation strategy is first used to construct the disparity search space, such as... Figure 2 As shown, by constructing an image pyramid, downsampling is performed on the left and right views at different scales. The matching cost between pixels is calculated at each scale level (e.g., using Census transform or gradient similarity measurement), and cost aggregation is performed within a local window to enhance noise resistance. Then, disparity estimation is progressively optimized from coarse to fine scales. The coarse scale provides global disparity range constraints, while the fine scale refines disparity values ​​through upsampling and local adjustments. Finally, the multi-scale results are fused to form a dense disparity search space, covering all possible disparity ranges in the scene. Based on this search space, the optimal disparity value for each pixel is determined using a winner-take-all (WTA) strategy or a semi-global matching (SGM) algorithm: the WTA strategy selects the disparity by comparing the minimum value after cost aggregation, while SGM optimizes the energy function in three or more directions through dynamic programming, balancing smoothness and matching accuracy, thereby generating a pixel-level disparity map between the left and right views. Subsequently, based on the calibration parameters of the binocular camera (baseline distance...), ... and focal length Convert the disparity map to a depth map. Due to disparity... With depth satisfy The formula is used to calculate the corresponding 3D spatial depth for each pixel value in the disparity map, and finally a depth image with the same resolution as the disparity map is generated. The grayscale or color value of each pixel directly reflects the distance from the scene point to the camera, providing accurate geometric information for subsequent 3D reconstruction or scene understanding.

[0084] Optionally, a multi-scale cost aggregation strategy can be employed to construct a disparity search space from coarse to fine, and the initial disparity map can be computed at a low-resolution scale. Upsampling to high resolution via bilinear interpolation; at high resolution scale, using To guide the search, the parallax search range is limited to... ,in A preset threshold is set. Then, sub-pixel-level optimization is applied to the aggregated cost volume, and the disparity plane is fitted using a quadratic function:

[0085]

[0086] in, Integer disparity determined for the winner-take-all algorithm. For pixels Parallax at location The cost of aggregation.

[0087] Then, based on the subpixel disparity map Combined with the baseline of the binocular camera and focal length Calculate the depth value:

[0088]

[0089] in, For disparity map quantization factor, This is the disparity offset, which is then used for multi-scale guidance and sub-pixel optimization to reduce disparity mismatches and improve depth edge accuracy.

[0090] In this embodiment, a multi-scale cost aggregation strategy is adopted to obtain the disparity search space. This strategy can integrate information from different scales, effectively address complex situations such as texture repetition and occlusion in the image, improve the accuracy and robustness of disparity estimation, and thus obtain a more accurate disparity map. By using the baseline and focal length of the binocular stereo vision camera to calculate the scene depth image, a rapid conversion from disparity to depth can be achieved. The calculation process is simple and efficient, and it can quickly generate a depth image that closely matches the actual scene depth, providing reliable data for subsequent applications such as 3D reconstruction.

[0091] Example 5: Based on Example 1, in this example, the specific implementation process of determining the disparity map between the left and right views and determining the scene depth image based on the disparity map may further include the following steps, which are detailed below:

[0092] Step 1), input the left and right views to a preset image processor, the preset image processor including a neural network processor.

[0093] Step 2), perform cost aggregation calculation on the left and right views based on the neural network processor, and determine the disparity map in the left and right views based on the cost aggregation result;

[0094] Step 3) Determine the depth value corresponding to the disparity in the disparity map based on a preset mapping table, and determine the scene depth image based on the depth value. The preset mapping table includes the mapping relationship between disparity and depth value.

[0095] For example, the acquired left and right views are input into a preset image processor, which integrates a neural network processor with powerful image data processing and computing capabilities. Once the left and right views enter the neural network processor, a cost aggregation calculation is performed on them based on its complex internal algorithm model. This cost aggregation calculation process can be understood as accumulating or integrating the matching costs of corresponding pixels in the left and right views under different disparity assumptions through a certain functional relationship. It can generally be expressed by the following formula:

[0096]

[0097] in, Represents pixels At parallax The cost of aggregation at that time It is a pixel. In the field, pixels within the domain At parallax The initial matching cost is calculated, and then after cost aggregation, the disparity map in the left and right views is determined by a winner-take-all strategy based on the calculation results. That is, the disparity with the smallest aggregation cost is selected as the disparity of each pixel.

[0098] After obtaining the disparity maps of the left and right views, a preset mapping table can be used. This preset mapping table records in detail the mapping relationship between disparity and depth values. For example, in a parallel binocular vision system, the depth value... Parallax Camera focal length and baseline distance Existing formula The relationship between the left and right views is established, and the depth value corresponding to each disparity in the disparity map can be quickly determined by querying the preset mapping table. Finally, a scene depth image is constructed based on these determined depth values, thus completely realizing the conversion process from left and right views to scene depth images.

[0099] In this embodiment, the left and right views are input into a preset image processor containing a neural network processor. The powerful feature extraction and learning capabilities of the neural network can efficiently process image information, perform cost aggregation calculations more accurately, and significantly improve the accuracy of disparity map determination. Then, with the help of a preset mapping table, the disparity is quickly converted into depth values ​​to generate a scene depth image. The whole process is efficient and accurate, and can quickly obtain high-quality scene depth information, providing a solid and reliable data foundation for applications such as 3D reconstruction.

[0100] Example 6: Based on Example 1, in this example, the specific implementation process of fusing the three or more scene depth images described above may further include the following steps, which are detailed below:

[0101] Step (1): Perform multi-scale decomposition on the three or more scene depth images respectively to obtain three or more levels corresponding to the three or more scene depth images;

[0102] Step (2): Based on the three or more levels, apply cross-scale consistency constraints to the three or more scene depth images to obtain cross-scale constraint conditions;

[0103] Step (3): The depth images of the three or more scenes are fused based on the cross-scale constraints.

[0104] For example, multi-scale decomposition can typically employ classic methods such as Gaussian pyramid decomposition. Taking Gaussian pyramid decomposition as an example, for the original scene depth image... Image layers of different scales are constructed by continuously applying Gaussian smoothing and downsampling. Assume the original image is layer 0. ,go through After the second downsampling, the first... Layer Image The size of the image decreases with increasing downsampling times, thus decomposing each scene depth image into three or more corresponding levels. These levels contain image information at different resolutions, from coarse to fine, comprehensively representing the depth features of the scene. Next, cross-scale consistency constraints are applied to the three or more scene depth images based on the decomposed levels. The core of this step is to ensure that image information at different scales maintains logical consistency during the fusion process. For example, at adjacent scales… and Assuming a smooth transition relationship exists between them, and implementing this constraint by constructing an energy function, specifically, assuming... and Given two depth images at adjacent scales, their energy function can be expressed as:

[0105]

[0106] in, Image pixel position, This indicates an upsampling operation, which restores a low-resolution image to the same size as a high-resolution image to facilitate pixel-level comparison. These are weighting coefficients used to balance the influence of different terms in the energy function. As a regularization term, a smoothing prior, such as the norm of the gradient, is typically used to maintain the smoothness of the image and avoid unreasonable abrupt changes in depth. By minimizing this energy function, cross-scale constraints can be obtained, providing important optimization criteria for subsequent fusion.

[0107] After fulfilling the cross-scale consistency constraints, the crucial step of fusion involving overlapping regions begins. Since three or more scene depth images may overlap, these overlapping regions require special handling during fusion to ensure the continuity and consistency of the fusion result. For overlapping regions, we determine the fusion weights based on the cross-scale constraints and the reliability of each scene depth image at different levels. Reliability can be measured by various metrics at different scales, such as image sharpness and the richness of edge information. Generally, regions with high sharpness and rich edge information have higher reliability and should be given greater weight during fusion. For example, for the first... The scene depth image at the ... The overlapping areas of layers can have their weights set to [value]. .

[0108] In the fusion process, cross-scale constraints are incorporated as part of the optimization objective, comprehensively considering image information at different scales and the correlation between depth images of different scenes. Fusion can be performed using methods such as weighted averaging or optimization solutions. If a weighted averaging fusion method is used, the fused depth image... At pixel position The depth value at that location can be expressed as:

[0109]

[0110] in, The number of scene depth images, The number of levels in the multi-scale decomposition. For the first The scene depth image at the ... Layer pixel position The depth value is obtained by fully considering the information of different scene depth images at different scales and locations. At the same time, combined with the processing of overlapping areas, the scene depth images are effectively fused to obtain high-quality, continuous and consistent scene depth images.

[0111] In this embodiment, multi-scale decomposition of three or more scene depth images can obtain detailed information at different levels, comprehensively depicting scene features from coarse to fine. Based on multi-level cross-scale consistency constraints, the consistency and complementarity of image information at different scales can be ensured, avoiding errors caused by scale mismatch during fusion. By fusing images according to cross-scale constraints, the advantages of each image can be fully utilized, effectively improving the fusion quality and obtaining more accurate, complete, and detailed scene depth images.

[0112] Example 7: Based on Example 1, in one of the exemplary embodiments provided in this application, the specific implementation process of fusing the three or more scene depth images described above may further include the following steps, which are detailed below:

[0113] Step 1), perform temporal alignment on the three or more scene depth images to obtain an aligned depth image sequence;

[0114] Step 2), determine the dynamic confidence level corresponding to the three or more scene depth images, and fuse the three or more scene depth images based on the dynamic confidence level and the depth image sequence.

[0115] For example, when processing three or more scene depth images, a time alignment operation is first performed. Since scene depth images acquired at different times may have temporal misalignments due to factors such as the movement of the acquisition device and time differences, time alignment aims to eliminate these misalignments and ensure that the images correspond consistently in the time dimension. Common time alignment methods include feature point matching or optical flow. For instance, a transformation matrix can be obtained through a corresponding time alignment algorithm. (For the first) (a scene depth image), then the original depth image After transformation, we obtain:

[0116]

[0117] The resulting aligned depth image sequence is represented as follows:

[0118]

[0119] Next, the dynamic confidence scores for three or more scene depth images are determined. Dynamic confidence score represents the reliability of each scene depth image at different times and in different regions. Its calculation can comprehensively consider multiple factors, such as image sharpness, richness of edge information, and consistency with neighboring images. Taking image sharpness as an example, it can be measured by calculating the gradient magnitude of the image. Specifically, assuming the image... In pixels The gradient at is:

[0120]

[0121] Then its gradient magnitude is:

[0122]

[0123] Then the sharpness-related confidence component at that pixel and The confidence level is directly proportional to the depth of the scene image at each pixel location. Then, by combining three or more factors, the dynamic confidence level at each pixel location is obtained. ( Indicates the first (3 or more images); Finally, based on dynamic confidence and the depth image sequence, the depth images of three or more scenes are fused. A weighted fusion method can be used. For the fused depth image It can be represented as:

[0124]

[0125] in The number of scene depth images is calculated using this formula. After aligning different scene depth images, they are weighted, summed, and normalized according to their dynamic confidence levels to obtain a high-quality fused scene depth image. This method can make full use of the useful information of each image while reducing the influence of unreliable information, thus improving the accuracy and reliability of the fusion result.

[0126] In this embodiment, time alignment of three or more scene depth images can eliminate image time deviations caused by different acquisition times, ensuring that subsequent processing is based on an accurate time sequence. Determining dynamic confidence can measure the reliability of each depth image at different times. By combining the aligned depth image sequence with dynamic confidence, the role of reliable images can be fully utilized and the influence of unreliable images can be reduced, thereby obtaining a more accurate, stable, and dynamic fused depth image that reflects scene changes.

[0127] Example 8: Based on Example 1, in one of the exemplary embodiments provided in this application, the specific implementation process of fusing the three or more scene depth images described above may further include the following steps, which are detailed below:

[0128] Step 1°: Determine the epipolar constraint equations for any two of the three or more different binocular stereo vision cameras.

[0129] Step 2°: Based on the epipolar constraint equation, perform epipolar matching on the scene depth images corresponding to any two binocular stereo vision cameras to obtain the matching result;

[0130] Step 3°: Based on the matching results, fuse the depth images of the three or more scenes.

[0131] For example, in a scenario involving three or more different binocular stereo vision cameras, it is first necessary to determine the epipolar constraint equations for any two of the binocular stereo vision cameras. For two binocular stereo vision cameras, let their projection matrices be respectively... and A point in space The projection point on the first camera imaging plane is: The projection point on the second camera's imaging plane is: According to the principle of polar geometry, there exists a The basic matrix , making The fundamental matrix This can be calculated using the projection matrices of the two cameras. Here are the epipolar constraint equations for the first and second cameras.

[0132] Next, based on the epipolar constraint equation, epipolar matching is performed on the scene depth images corresponding to the two stereo vision cameras, and a feature point is selected in the scene depth image of the first camera. According to the epipolar constraint equation, in the scene depth image of the second camera, with Matching It must be located by On a defined polar line, by searching along the polar line... The points with the most similar features (similarity can be measured using feature descriptors such as SIFT and SURF) are the matching point pairs. , Performing this operation on all feature points in the first camera scene depth image yields a complete matching result. Finally, based on the matching result, three or more scene depth images are fused. Since scene depth images acquired by different stereo vision cameras may have overlapping regions, the matching point pairs obtained through epipolar matching can determine the correspondence between these overlapping regions. For overlapping regions, weights can be assigned to each scene depth image based on indicators such as matching accuracy and distance. For example, regions corresponding to matching point pairs with high matching accuracy and reasonable distances are assigned higher weights. Let the first... The weights of the scene depth images in the overlapping region are: The fused scene depth image At pixel The value at this point can be calculated using a weighted average:

[0133]

[0134] in It is the number of scene depth images. It is the first A scene depth image at pixels The depth value at that location allows for the effective fusion of depth images from three or more scenes by fully utilizing information collected by different binocular stereo vision cameras.

[0135] Finally, based on the matching results obtained above, three or more scene depth images are fused. Since the shooting angles and ranges of different binocular stereo vision cameras may differ, their acquired scene depth images will have some overlapping areas. The matching results provide precise correspondences for these overlapping areas. By analyzing the position and feature similarity of matching point pairs, the reliability and importance of each scene depth image in the overlapping areas can be evaluated. For overlapping areas, corresponding weights can be assigned to different images based on these evaluation results: image regions with high reliability and good feature matching are given larger weights, and vice versa. Then, a weighted fusion method is used to merge three or more scene depth images, so that the fused image can combine the advantages of each image, preserving detailed information from different viewpoints while avoiding conflicts and inconsistencies caused by image differences, thus obtaining a high-quality, complete scene depth image.

[0136] In this embodiment, determining the epipolar constraint equations for any pairwise correspondences between different binocular stereo vision cameras can strictly constrain the image matching process using geometric relationships, effectively reducing the probability of mismatches. Epipolar matching based on the epipolar constraint equations can quickly and accurately find corresponding points in the depth images of different cameras, obtaining reliable matching results. By fusing three or more scene depth images based on the matching results, multi-camera information can be fully integrated, improving the accuracy and completeness of the fused depth image and more realistically and accurately restoring the 3D information of the scene.

[0137] Example 9: Based on Example 1, in one of the exemplary embodiments provided in this application, the specific implementation process of constructing the 3D simulation model corresponding to the scene based on the fused depth image may further include the following steps, which are described in detail below:

[0138] Step (1): Determine a textured dense point cloud based on the fused depth image, the dense point cloud being used to characterize the outline of the scene;

[0139] Step (2): Reconstruct the Poisson surface of the scene based on the dense point cloud to obtain the mesh model corresponding to the scene;

[0140] Step (3): Perform multi-view texture mapping on the mesh model to obtain the three-dimensional simulation model corresponding to the scene.

[0141] For example, after fusing three or more scene depth images to obtain a fused depth image, a textured dense point cloud can be determined to characterize the scene outline. The fused depth image contains distance information from the surfaces of various objects in the scene to the camera. By mapping each pixel in the depth image to three-dimensional space, and combining camera intrinsic parameters and pose information, the three-dimensional spatial coordinates corresponding to each pixel can be determined. Simultaneously, since the original scene image contains rich color and texture information, associating each three-dimensional point with the corresponding pixel color in the original image allows each three-dimensional point to be assigned a texture attribute, thereby generating a textured dense point cloud. This dense point cloud can present the shape, position, and surface texture features of objects in the scene in considerable detail, providing rich basic data for subsequent scene reconstruction. After obtaining the textured dense point cloud, Poisson surface reconstruction of the scene is performed using this point cloud data to obtain the corresponding mesh model of the scene. Poisson surface reconstruction is a reconstruction method based on implicit functions. It treats point cloud data as a sample of an indicator function, utilizing... The indicator function is solved using the Poisson equation, and then the surface mesh of the scene is obtained through isosurface extraction. This method can effectively handle noise and unevenly sampled point cloud data, generating a smooth and continuous mesh model. This mesh model can accurately describe the surface shape and topology of objects in the scene and is an important representation of 3D scenes. Finally, multi-view texture mapping is performed on the obtained mesh model to construct a corresponding 3D simulation model of the scene. Multi-view texture mapping requires using images of the original scene taken from different perspectives. By analyzing the visibility of each facet in the mesh model in images from different perspectives, the most suitable image texture is mapped onto the corresponding facet. In this process, factors such as perspective changes, lighting effects, and texture splicing and fusion need to be considered to ensure the accuracy and naturalness of the texture mapping. After multi-view texture mapping, the mesh model not only has accurate geometry but also realistic surface texture, thus generating a 3D simulation model that is highly similar to the real scene. This model can be widely used in virtual reality, augmented reality, 3D visualization and other fields.

[0142] For example, after generating a 3D simulation model highly similar to the real scene, Gaussian sputtering technology can be used to further process the model to better apply it to fields such as virtual reality, augmented reality, and 3D visualization. Gaussian sputtering is an image processing method based on the Gaussian function, which can effectively smooth the model surface and enhance the model's visual effect. The specific implementation process is as follows: First, the 3D simulation model is preprocessed, including noise removal and edge continuity optimization, to ensure the quality of the model. Then, the model surface is divided into several small regions, and the pixels in each small region are assigned different weights based on their distance from the center point. This weight is determined by the Gaussian function. The Gaussian function has central symmetry; pixels closer to the center point have a larger weight, and pixels farther away have a smaller weight. Next, a weighted average is performed on the pixels in each small region to obtain a new pixel value. This new pixel value not only considers the original value of the current pixel but also the influence of surrounding pixels, thus achieving smoothing of the model. During the smoothing process, the parameters of the Gaussian function, such as the standard deviation, can be adjusted as needed to control the degree of smoothing. A larger standard deviation results in a more pronounced smoothing effect, but may lead to a loss of model details; a smaller standard deviation results in a weaker smoothing effect, but better preserves model details. Finally, the processed pixel values ​​are recombined to form a complete model surface, resulting in a 3D simulation model after Gaussian sputtering. This model has a smoother surface, better visual effects, and can better meet the needs of virtual reality, augmented reality, and 3D visualization fields.

[0143] In this embodiment, a textured dense point cloud is generated based on the fused depth image, which can finely depict the scene outline and retain rich details; the dense point cloud is converted into a mesh model through Poisson surface reconstruction, which can efficiently construct the geometric structure of the scene; multi-view texture mapping is performed on the mesh model to give the model realistic texture. The final 3D simulation model is highly realistic in terms of geometry and appearance, providing high-quality, immersive 3D scene data for applications such as virtual reality and scene analysis.

[0144] Example 10: As Figure 3 As shown, in this embodiment, an environment simulation system based on binocular stereo vision measurement is provided. The system includes a processor, an input device, an output device, and a memory. The processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the environment simulation method based on binocular stereo vision measurement as described in any of the above embodiments.

[0145] It should be noted that the environment simulation system based on binocular stereo vision measurement provided in this embodiment and the environment simulation method based on binocular stereo vision measurement provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments and will not be repeated here. In practical applications, the environment simulation system based on binocular stereo vision measurement provided in this embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0146] Example 11: This example provides an electronic device, including: one or three or more processors; and a storage device for storing one or three or more programs, which, when executed by the one or three or more processors, enable the electronic device to implement the environment simulation method based on binocular stereo vision measurement provided in the above examples.

[0147] It should be noted that the computer-readable medium shown in this embodiment can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or three or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0148] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units described in this embodiment can be implemented in software or hardware, and can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0150] Example 12: This example provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the environment simulation method based on binocular stereo vision measurement as described above. This computer-readable storage medium may be included in the electronic device described in the above examples, or it may exist independently and not be assembled into the electronic device.

[0151] Example 13: This example provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the environment simulation method based on binocular stereo vision measurement provided in the above embodiments.

[0152] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.

Claims

1. An environment simulation method based on binocular stereo vision measurement, characterized by: Includes the following steps: S110. Construct a physical imaging model of the left and right optical elements in the binocular stereo vision camera, and obtain the left and right views based on the physical imaging model; S120. Determine the disparity map between the left and right views, and determine the scene depth image based on the disparity map; S130. Obtain three or more scene depth images generated by three or more different binocular stereo vision cameras corresponding to the scene, and fuse the three or more scene depth images to construct a three-dimensional simulation model corresponding to the scene based on the fused depth image.

2. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: The method further includes: S210. Determine the corresponding image feature points in the left and right views based on the pixel change intensity in the left and right views; S220. Determine the corner points in the left and right views and the corner point response intensity corresponding to the corner points, and optimize the image feature points based on the corner point response intensity to obtain optimized image feature points; S230. Update the left and right views based on the optimized image feature points to obtain updated left and right views, and determine the scene depth image based on the updated left and right views.

3. The environmental simulation method based on binocular stereo vision measurement according to claim 2, characterized in that: The method further includes: S310. After correcting the distortion of the optimized image feature points, transform them to the camera coordinate system to obtain the feature point coordinate information; S320. Determine the epipolar equation of the binocular stereo vision camera based on the rotation matrix and translation matrix of the binocular stereo vision camera, and calculate the distance from the optimized image feature points in the right view to the epipolar line based on the epipolar equation. S330. If the distance is greater than a preset distance threshold, then the corresponding image feature points in the right view are removed.

4. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: In step S120, determining the disparity map between the left and right views and determining the scene depth image based on the disparity map includes: (1) A multi-scale cost aggregation strategy is adopted to obtain the disparity search space, and the disparity map between the left and right views is determined based on the disparity search space; (2) Obtain the baseline and focal length of the binocular stereo vision camera, and calculate the scene depth image corresponding to the disparity map based on the baseline and the focal length.

5. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: In step S120, determining the disparity map between the left and right views and determining the scene depth image based on the disparity map includes: 1) Input the left and right views into a preset image processor, the preset image processor including a neural network processor; 2) Based on the neural network processor, cost aggregation calculation is performed on the left and right views to determine the disparity map in the left and right views based on the cost aggregation result; 3) Determine the depth value corresponding to the disparity in the disparity map based on a preset mapping table, and determine the scene depth image based on the depth value. The preset mapping table includes the mapping relationship between disparity and depth value.

6. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: The fusion of three or more scene depth images in S130 includes: (1) Perform multi-scale decomposition on the three or more scene depth images respectively to obtain three or more levels corresponding to the three or more scene depth images; (2) Based on the three or more levels, cross-scale consistency constraints are applied to the three or more scene depth images to obtain cross-scale constraint conditions; (3) The depth images of the three or more scenes are fused based on the cross-scale constraints.

7. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: The fusion of three or more scene depth images in S130 includes: 1) Perform temporal alignment on the three or more scene depth images to obtain an aligned depth image sequence; 2) Determine the dynamic confidence level corresponding to the three or more scene depth images, and fuse the three or more scene depth images based on the dynamic confidence level and the depth image sequence.

8. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: The fusion of three or more scene depth images in S130 includes:

1. Determine the epipolar constraint equations for any two of the three or more different binocular stereo vision cameras; 2° Based on the epipolar constraint equation, perform epipolar matching on the scene depth images corresponding to any two binocular stereo vision cameras to obtain the matching result; 3° Based on the matching results, the depth images of the three or more scenes are fused.

9. The environmental simulation method based on binocular stereo vision measurement according to claim 1, characterized in that: The step S130, which involves constructing a 3D simulation model of the scene based on the fused depth image, includes: (1) Based on the fused depth image, a textured dense point cloud is determined, which is used to delineate the outline of the scene; (2) Based on the dense point cloud, the Poisson surface of the scene is reconstructed to obtain the mesh model corresponding to the scene; (3) Perform multi-view texture mapping on the mesh model to obtain the three-dimensional simulation model corresponding to the scene.

10. An environmental simulation system based on binocular stereo vision measurement, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the environment simulation method based on binocular stereo vision measurement as described in any one of claims 1 to 9.

Images