Visual dense depth optimization method and device based on local texture adaptive weighting

Abstract

The application provides a visual dense depth optimization method and device based on local texture adaptive weighting, and belongs to the technical field of visual-inertial odometer and visual SLAM, and comprises the following steps: acquiring synchronous binocular images, generating a dense disparity map by using a stereo matching algorithm, and converting the dense disparity map into a dense depth map; performing Laplace transform on a left eye image of the synchronous binocular images to locate a texture-rich region, extracting high-response Harris corner points in the texture-rich region as high-quality feature points, and calculating sparse depth of the high-quality feature points by triangulation; defining a neighborhood window with each high-quality feature point as the center, calculating a Gaussian weight according to the distance between pixels in the neighborhood window and the high-quality feature points, weighting and fusing the sparse depth and the original dense depth, and updating and outputting the optimized dense depth map pixel by pixel. The application can greatly improve the depth estimation accuracy and the image definition under the harsh working conditions of shaking and weak texture without changing the original hardware and algorithm framework.

Description

Technical Field

[0001] This invention belongs to the field of visual-inertial odometry and visual SLAM technology, specifically relating to a visual density depth optimization method and device based on local texture adaptive weighting. Background Technology

[0002] To estimate the 3D positions of feature points, VIO (Visual-Inertial Odometry) systems typically generate a dense disparity map from binocular (or multi-view) images. The depth of triangulated sparse feature points is then projected onto this disparity map for optimization by a sliding window backend. This process achieves sufficient accuracy under stable lighting, rich textures, and slow camera movement. However, in scenarios with rapid motion, blurred images, or simple textures, stereo matching errors surge, and the dense disparity map becomes highly noisy. Post-processing with global filtering or deep learning models becomes too computationally and memory-intensive for real-time operation on mobile devices. Furthermore, while the feature point depth obtained through sparse triangulation is accurate, its sparse distribution fails to directly cover the entire dense map and is insufficient to guide local correction of noisy regions.

[0003] The prior art disclosed in CN116402978A is a dense 3D reconstruction method based on the structural characteristics of binocular vision, belonging to the field of computer vision technology. The method involves: calibrating a binocular camera to obtain intrinsic and extrinsic parameters; acquiring binocular camera images; using intrinsic parameters to correct image distortion; extracting image features; performing image matching to retrieve image pairs containing the same scene region; performing image feature matching; generating sparse point clouds using an incremental SfM algorithm based on the structural characteristics of binocular vision to recover the environment structure from image sequences; densely reconstructing the scene and fusing it with the scene data; generating a mesh using the 3DDelaunay algorithm and performing mesh filtering; generating UV texture maps and fusing the texture information with the mesh to obtain the final result of dense 3D reconstruction. This method is complex, requires significant computational power to achieve the desired effect, and performs poorly on images acquired under shaky conditions.

[0004] Therefore, how to perform local quality adaptive weighted optimization of dense disparity maps on resource-constrained mobile platforms in a low-latency, low-computation manner using sparse but high-precision feature point depth, fully preserve and enhance reliable depth in areas with clear textures, and suppress noise and fill holes in blurry or texture-deficient areas, thereby improving the overall depth estimation accuracy and VIO system robustness without sacrificing real-time performance, is an urgent problem to be solved. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a visual dense depth optimization method and apparatus based on local texture adaptive weighting. This method performs dense depth calculation and feature point quality assessment, while utilizing the texture response of binocular images to enhance the robustness of high-quality region detection. Simultaneously, it performs depth weighting optimization based on image quality, combining distance weights and feature point response weights to adaptively fuse sparse depth information.

[0006] To achieve the above objectives, this invention discloses a visual density depth optimization method based on local texture adaptive weighting, comprising the following steps: Step 1: Use a stereo matching algorithm to convert the synchronized binocular images into dense disparity maps, and further obtain dense depth maps; Step 2: Perform two-step texture processing on the left eye image of the synchronized stereo image: first, filter the texture region by Laplacian transform, and then perform Harris corner detection within the filtered region; calculate the sparse depth of high-quality feature points. Step 3: Delineate a neighborhood window centered on each high-quality feature point. Calculate the Gaussian weight of each pixel based on the distance between the pixel in the neighborhood window and the high-quality feature point. Weight the sparse depth map with the depth value of the corresponding pixel position in the original dense depth map and update and output the optimized dense depth map pixel by pixel.

[0007] Further, in step 1, the synchronized stereo images are used to generate a dense disparity map using a stereo matching algorithm, and then converted into a dense depth map; the formula for converting the synchronized stereo images into a dense depth map is: , in, The focal length of the camera. The length of the three-dimensional baseline. For pixels disparity value at that location, in pixels Pixel coordinates for synchronizing stereo images.

[0008] Furthermore, step 2 includes: Perform a Laplacian transform on the left eye image and calculate the absolute value of the second derivative of the grayscale change at each pixel; A threshold is set based on the variance of the Laplacian transform, and only pixel regions with values ​​above the threshold are retained as texture-rich regions. Within texture-rich regions, the response value of each candidate point is calculated using Harris corner detection. The response values ​​of all candidate points are normalized to obtain normalized weights. Candidate points with normalized weights greater than a preset threshold are regarded as high-quality feature points. The normalization formula is: To obtain the normalized weights :in, This represents the Harris corner response value for each candidate corner. This represents the Harris corner response value of the indexed candidate corner. For the traversal index in the summation symbol; This represents the total number of high-quality feature points in the current frame. High-quality feature points are projected onto a dense depth map. By combining the disparity matching results with the triangulation principle, the 3D sparse depth value and its pixel coordinates corresponding to each high-quality feature point are obtained.

[0009] Furthermore, step 3 includes: defining a neighborhood window around each high-quality feature point, calculating the distance between each pixel in the neighborhood window and the high-quality feature point, and introducing a Gaussian weighting function based on the distance to calculate the Gaussian weight corresponding to each pixel; The Gaussian weighting function is: , in, This represents the Euclidean distance between a pixel within the neighborhood window and the high-quality feature point. , The standard deviation is a Gaussian distribution used to control the range of influence of pixels within the neighborhood window on the depth of high-quality feature points. This represents an exponential function.

[0010] Furthermore, the sparse depth of the i-th high-quality feature point is... Compared with the original dense depth map The depth values ​​at corresponding pixel locations are weighted and fused to update the depth values ​​of each pixel within the neighborhood window pixel by pixel. The specific process is as follows: Using the formula: Calculate pixels Optimized depth value ,in, This is the mixing coefficient, used to balance the contributions of the original depth and the depth of high-quality feature points, with a value of 0.4. is the normalized response weight for the i-th high-quality feature point.

[0011] Furthermore, if a pixel falls into multiple neighborhood windows, the arithmetic mean of the multiple updated depth values ​​obtained for that pixel is taken, or a weighted average is taken based on the Gaussian weights calculated at each update, and finally a smooth and consistent optimized dense depth map is output.

[0012] Furthermore, the optimized dense depth map is directly used as the depth prior input for the visual inertial odometry backend.

[0013] An apparatus for a visual density depth optimization method based on local texture adaptive weighting includes a transformation unit, a computation unit, and an output unit connected in sequence, wherein: The conversion unit takes the acquired synchronous stereo images, generates a dense disparity map using a stereo matching algorithm, and converts it into a dense depth map. The computing unit performs a Laplacian transform on the left eye image of the synchronized stereo image to locate texture-rich regions, extracts high-response Harris corner points as high-quality feature points in the texture-rich regions, and calculates their sparse depth through triangulation. The output unit defines a neighborhood window centered on each high-quality feature point, calculates Gaussian weights based on the distance between pixels within the neighborhood window and the high-quality feature points, and weights and fuses the sparse depth with the original dense depth, updating and outputting the optimized dense depth map pixel by pixel.

[0014] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a visual density depth optimization method based on local texture adaptive weighting.

[0015] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a visual density depth optimization method based on local texture adaptive weighting.

[0016] Beneficial effects: 1. This invention combines dense parallax and texture region feature depth joint optimization, which not only preserves density but also utilizes reliable feature point depths in texture regions to correct the overall depth estimation, and is more robust to rapid motion and jitter. 2. This invention introduces dense parallax into the global depth constraint, while texture region feature points participate in the optimization, avoiding error propagation caused by relying solely on the depth of a small number of feature points. Under realistic and harsh conditions such as jitter and weak textures, this invention, without changing the original hardware and algorithm framework, improves depth estimation accuracy by 18.9% through a post-processing method of "texture selection of high-quality feature points + Gaussian distance weighted fusion," significantly improving image imaging quality. Attached Figure Description

[0017] Figure 1 This is a flowchart of the visual density depth optimization method based on local texture adaptive weighting according to the present invention.

[0018] Figure 2 This is a schematic diagram of the visual density depth optimization device based on local texture adaptive weighting of the present invention.

[0019] Figure 3 This refers to the room1 sequence set collected in this embodiment of the invention.

[0020] Figure 4 This refers to the room2 sequence set collected in this embodiment of the invention.

[0021] Figure 5This refers to the room3 sequence set collected in this embodiment of the invention.

[0022] Figure 6 This refers to the room4 sequence set collected in this embodiment of the invention.

[0023] Figure 7 These are sample images captured under conditions of camera shake and changes in indoor texture in an embodiment of the present invention.

[0024] Figure 8 This is a sample image of high-quality feature point extraction and spatial distribution in an embodiment of the present invention. The red dots in the image are the high-quality feature points that have been marked.

[0025] Figure 9 This is the original depth sample image obtained in an embodiment of the present invention.

[0026] Figure 10 This is an unoptimized depth residual sample image in an embodiment of the present invention.

[0027] Figure 11 The images show a comparison of depth errors before and after optimization for each acquisition sequence set in this embodiment of the invention; where (a) is a comparison of test results for the room1 sequence set, (b) is a comparison of test results for the room2 sequence set, (c) is a comparison of test results for the room3 sequence set, and (d) is a comparison of test results for the room4 sequence set; ori represents the depth error of the original dense depth map before optimization, and opt represents the depth error sequence of the dense depth map after optimization by the method of this invention. Detailed Implementation

[0028] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0029] like Figure 1 As shown, the present invention discloses a visual density depth optimization method based on local texture adaptive weighting, which includes the following steps: Step 1: Use a stereo matching algorithm to convert the synchronized binocular images into dense disparity maps, and then further convert them into dense depth maps; Step 2: Perform two-step texture processing on the left eye image of the synchronized stereo image: first, filter the texture region by Laplacian transform, and then perform Harris corner detection within the filtered region; calculate the sparse depth of high-quality feature points. Step 3: Delineate a neighborhood window centered on each high-quality feature point. Calculate the Gaussian weight of each pixel based on the distance between the pixel in the neighborhood window and the high-quality feature point. Weight the sparse depth map with the depth value of the corresponding pixel position in the original dense depth map and update and output the optimized dense depth map pixel by pixel.

[0030] Specifically, step 1 includes: Step 1.1: Acquire synchronized left and right stereo images: Left eye image And right eye image ,in These are pixel coordinates.

[0031] Step 1.2: Apply stereo matching algorithms, including Semi-Global Matching and Block Matching methods, to the synchronized left and right stereo images to calculate the dense disparity map. ;in, A function representing the selected stereo matching algorithm; For pixels disparity value at that location, This is an abbreviation for the left and right eye images.

[0032] Step 1.3: Based on the disparity map Calculate a coarse dense depth map : ; in, This refers to the camera's focal length, expressed in pixels. The length of the stereo baseline is in meters; the parallax value is converted into the three-dimensional depth of the corresponding pixel using the above formula.

[0033] Specifically, step 2 includes: Step 2.1: Perform "two-step texture processing" on the left eye image: select the most reliable and representative region from the image and extract high-quality feature points, which are used as anchor points to optimize the entire depth map, while the right eye image is used only as a reference image for stereo matching.

[0034] Use the following formula to analyze the left eye image. Perform a Laplacian operator transform to obtain the absolute value response map of the second derivative of the grayscale image. : ; in, This represents the Laplacian operator, used to measure the degree of local grayscale change.

[0035] Step 2.2: Threshold filtering is performed by calculating the variance of the Laplace transform, based on the absolute value response map of the second derivative of the grayscale image. Pixels in areas with rich texture are selected from the data, and the filtering threshold is chosen based on the documentation of the open-source vision library.

[0036] Step 2.3: Within the selected texture-rich areas, analyze the grayscale of the left eye image. Harris corner detection is performed, and the Harris corner response value of each candidate corner is calculated using the following formula. ; ; in, Let be the gradient covariance matrix at the candidate corner point. It is the determinant of a matrix. The trace of a matrix, a constant The value ranges from 0.04 to 0.06, and this value is an empirical value obtained from multiple experiments.

[0037] Step 2.4: Apply Harris response values ​​to all candidate corner points. Perform normalization and calculate the normalization weights. : ; in, The traversal index in the summation symbol (summing all candidate corner points within the same frame); This represents the total number of high-quality feature points in the current frame.

[0038] Due to Harris response value As a floating-point real number, its absolute value is greatly affected by the local contrast of the image, making it incomparable between different images. This is achieved by considering all candidate corner points within the same frame. Value calculation of normalized weights This can be converted into relative quality weights within the [0,1] interval, which can then be used with normalized weights. Sort all candidate corner points and retain only their weights. Feature points that exceed a preset empirical threshold are considered high-quality feature points.

[0039] Projecting high-quality feature points onto a dense depth map In the middle, combined with the disparity matching results Using triangulation to calculate the sparse depth of high-quality feature points And record the corresponding pixel coordinates. .

[0040] Specifically, step 3 includes: Step 3.1, for the first High-quality feature points Define its neighborhood window The center of the window is The radius is set to Pixel.

[0041] Step 3.2, at the i-th high-quality feature point neighborhood window Inside, calculate dense depth map All pixels With Gao Di High-quality feature points Euclidean distance : ; Step 3.3: Based on Euclidean distance Introducing Gaussian weighting function Calculate the Gaussian weights for each pixel: ; in, The standard deviation is a Gaussian distribution used to control the range of influence of pixels within the neighborhood window on the depth of high-quality feature points. This represents an exponential function.

[0042] Step 3.4, place the first High-quality feature points sparse depth Compared with the original dense depth map The depth values ​​at corresponding pixel locations are weighted, fused, and updated to obtain the optimized depth values. : ; in, This is the mixing coefficient, used to balance the contributions of the original depth and the depth of high-quality feature points, with a value of 0.4. For the first Normalized response weights for high-quality feature points; For the first The sparse depth of high-quality feature points.

[0043] Update the depth value of each pixel within the neighborhood window on a pixel-by-pixel basis.

[0044] Step 3.5: For all high-quality feature points Repeat the above operation, updating the depth values ​​in its neighborhood sequentially, until all high-quality neighborhoods of the dense depth map have been merged and updated, resulting in an optimized dense disparity depth map. If the same pixel is updated repeatedly in multiple neighborhood windows, the results of multiple updates are weighted and accumulated or weighted and averaged to ensure a smooth and consistent depth map.

[0045] Step 3.6: The final optimized dense disparity depth map is used as a sparse depth prior or dense depth prior for high-quality feature points in the back-end sliding window nonlinear optimization (Visual-Inertial Bundle Adjustment, V-BA), which significantly improves the depth estimation accuracy and system robustness in dynamic scenes (such as pedestrian jitter, high-speed movement, etc.).

[0046] Example: To verify the improvement effect of the method of the present invention on the accuracy and stability of depth estimation, this example collects multiple sets of synchronous binocular image data in an office scene. During the acquisition process, the operator moves the camera indoors while actively shaking it significantly in the left and right directions to simulate camera pose disturbances, motion blur, and insufficient local texture in real-world applications. The experiment first uses a conventional stereo matching method to generate an original dense depth map from the left and right eye images. Then, the original dense depth map is optimized using the local texture adaptive weighting method proposed in this invention. During the evaluation process, the depth information output by the camera's built-in depth lens is used as a reference depth. The differences between the depth results before and after optimization and the reference depth are calculated, and the magnitude of the difference is used as the evaluation criterion for depth estimation error. The original data is collected in an office indoor scene using an Intel RealSense D455 depth camera. During the acquisition process, the operator moves the camera along an indoor route while artificially applying large left and right and small up and down movements to simulate camera pose disturbances, motion blur, and local observation instability in real-world applications. The D455 camera simultaneously acquires image and depth data, with the depth data serving as a reference depth to evaluate the error change in depth estimation results before and after optimization. This acquisition method verifies the improvement in stability and accuracy of dense depth estimation under conditions of camera shake and indoor texture variations.

[0047] Figure 3 , Figure 4 , Figure 5 and Figure 6 These represent the actual acquisition results of the room1, room2, room3, and room4 sequence sets, respectively. The room1, room2, room3, and room4 sequence sets were selected according to the principle of "same acquisition device, same indoor application scenario, different texture complexity, and different camera perturbation levels," ensuring that each sequence set can respectively cover the typical problems to be solved by this invention. Specifically, Figure 3 The images in the room1 sequence set are a richly textured baseline sequence set captured by a D455 camera during variable-speed movement, used to verify that the invention does not destroy the original depth structure in ordinary indoor scenes. Figure 3 The first and second images are two consecutive indoor reference images taken with a D455 camera under conditions of relatively small shaking and high-speed movement; the third and fourth images are two consecutive indoor reference images taken with a D455 camera under conditions of relatively small shaking and low-speed movement; the fifth and sixth images are two consecutive indoor reference images taken with a D455 camera under conditions of large shaking and high-speed movement, with a 1-second interval between the consecutive images. Figure 4The images in the room2 sequence set contain weakly textured areas such as white walls and tabletops, and exhibit localized repetitive textures and low-contrast regions. These features can easily lead to instability in binocular parallax matching and localized depth holes. This was used to verify the effectiveness of Laplacian texture filtering and high-quality feature points in suppressing depth noise in weakly textured regions. Figure 4 Six sample images taken at a rate of one frame per second in an office area while the operator holds a D455 camera and maintains the same speed of movement and waving. Figure 5 The images in the room3 sequence set contain repetitive textures, local occlusions, and abrupt depth boundary changes, used to verify the corrective effect of neighborhood window Gaussian weighted fusion on local mismatches and edge depth anomalies. Figure 5 Six sample images were taken at a rate of one frame per second, with the operator holding a D455 camera and pointing it towards a door or obvious structural boundary area, making slight left-right and up-down movements in the direction directly facing the target to create a visual disturbance. Figure 6 The room4 sequence contains images with significant hand-held shaking and viewpoint changes to verify the robustness of the invention under motion blur and observational instability conditions. Figure 6 The six sample images were taken at a rate of one frame per second, with the operator holding a D455 camera and rotating and translating it slightly around a fixed workstation scene while applying significant left-right shaking.

[0048] Figure 7 Images captured under conditions of camera shake and changes in indoor texture. Figure 8 For high-quality feature point extraction and spatial distribution mapping, Figure 9 and Figure 10 These are weakly textured scenes and their corresponding unoptimized depth residual maps.

[0049] Figure 11 The results comparing the depth error before and after optimization are presented for four sets of office data acquisition sequences (room1, room2, room3, and room4 sequence sets). Figure 11 The horizontal axis 's' represents the unit 'second', and the vertical axis 'm' represents the unit 'meter'; where, Figure 11 (a) is a comparison of the depth error before and after optimization of the room1 sequence. Figure 11 (b) is a comparison of the depth error before and after optimization of the room2 sequence. Figure 11 (c) is a comparison of the depth error before and after optimization of the room3 sequence. Figure 11 Figure (d) shows a comparison of the depth error before and after optimization of the room4 sequence. In the figure, "ori" represents the error of the original dense depth result obtained by conventional stereo matching methods relative to the reference depth, and "opt" represents the error of the dense depth result optimized by the method of this invention relative to the reference depth. The vertical axis is in meters. Figure 11It can be seen that the overall fluctuation amplitude of the optimized error curve is reduced, and the peak values ​​of local abnormal errors are suppressed, indicating that the present invention can improve the stability of depth estimation under conditions of camera shake and insufficient local texture. According to the statistics of existing experimental data, the overall depth estimation accuracy is improved by 18.9% after adopting the method of the present invention compared with that before optimization.

[0050] like Figure 2 As shown, the present invention also provides a visual density depth optimization device based on local texture adaptive weighting for implementing the above method, comprising the following units: The conversion unit acquires synchronized stereo images, generates a dense disparity map using a stereo matching algorithm, and converts it into a dense depth map; The computing unit performs Laplacian transform on the left eye image of the synchronized stereo image to locate texture-rich regions, extracts high-response Harris corner points as high-quality feature points in the texture-rich regions, and calculates their sparse depth through triangulation. The output unit defines a neighborhood window centered on each high-quality feature point, calculates Gaussian weights based on the distance between pixels within the neighborhood window and the high-quality feature points, and weights and fuses the sparse depth with the original dense depth, updating and outputting the optimized dense depth map pixel by pixel.

[0051] In this embodiment, the original dense depth map directly generated by the conventional binocular stereo matching method is used as a comparison result with the dense depth map output after local texture adaptive weighting according to the present invention. The conventional binocular stereo matching method mainly calculates depth based on the disparity relationship between the left and right eye images. When there are local weak textures, repeated textures, or camera shake in the scene, unstable matching and increased local depth errors are likely to occur. The present invention, based on the conventional dense depth results, introduces Laplacian texture response, high-quality Harris corner sparse depth, and neighborhood window Gaussian weighted fusion to adaptively correct the depth values ​​near reliable texture areas, thereby improving local errors and enhancing the consistency of the depth map.

[0052] from Figure 11 The comparative results show that the "ori" curve of existing conventional methods exhibits significant error fluctuations at certain times, while the "opt" curve corresponding to the method of this invention is more stable overall, especially in suppressing local abnormal peaks. Based on existing experimental statistics, the method of this invention improves the depth estimation accuracy by 18.9% compared to the dense depth results directly output by conventional binocular stereo matching, demonstrating that this invention can improve the accuracy and stability of dense depth estimation without changing the basic framework of binocular depth estimation.

[0053] like Figure 7As shown, the original dense depth map is first generated using the semi-global matching (SGM) algorithm, and then the depth residual is calculated to obtain the following result. Figure 10 The residual distribution is used to filter out textured regions by Laplacian response for areas with large errors. Harris corner points are then re-extracted from these textured regions, and normalized weights are calculated based on the corner response values. Sparse depth values ​​are obtained through triangulation. An optimization mask is constructed based on the residual distribution, redistributing high-quality feature points only in the region to be optimized. Figure 8 As shown in the figure. Subsequently, Gaussian distance weights and feature response weights are used to complete local depth fusion, and a weighted average is applied to the regions updated multiple times to obtain the final depth map. Finally, as shown in the figure... Figure 7 The results were validated in weak texture and high dynamic range scenes, demonstrating that the present invention can effectively suppress the propagation of local errors and improve the continuity and stability of depth maps. Figure 11 As shown, the overall depth estimation accuracy is improved by 18.9%.

[0054] The above description is merely one embodiment of the present invention and is not intended to limit the present invention. Any minor modifications, equivalent substitutions, and improvements made to the above embodiment based on the technical essence of the present invention should be included within the protection scope of the present invention.

Claims

1. A visual density depth optimization method based on local texture adaptive weighting, characterized in that, Includes the following steps: Step 1: Use a stereo matching algorithm to convert the synchronized binocular images into a dense disparity map, and further obtain a dense depth map; Step 2: Perform two-step texture processing on the left eye image of the synchronized stereo image: first, filter the texture region by Laplacian transform, and then perform Harris corner detection within the filtered region; calculate the sparse depth of high-quality feature points. Perform a Laplacian transform on the left eye image and calculate the absolute value of the second derivative of the grayscale change at each pixel; A threshold is set based on the variance of the Laplacian transform, and only pixel regions with values ​​above the threshold are retained as texture-rich regions. Within texture-rich regions, Harris corner detection is used to calculate the response value of each candidate point; The response values ​​of all candidate points are normalized to obtain normalized weights. Candidate points with normalized weights greater than a preset threshold are regarded as high-quality feature points. The normalization formula is: To obtain the normalized weights :in, This represents the Harris corner response value for each candidate corner. This represents the Harris corner response value of the indexed candidate corner. For the traversal index in the summation symbol; This represents the total number of all candidate corner points in the current frame. High-quality feature points are projected onto a dense depth map, and the three-dimensional sparse depth value and pixel coordinates corresponding to each high-quality feature point are obtained by combining the disparity matching results and the triangulation principle. Step 3: Delineate a neighborhood window centered on each high-quality feature point. Calculate the Gaussian weight of each pixel based on the distance between the pixel in the neighborhood window and the high-quality feature point. Weight the sparse depth map with the depth value of the corresponding pixel position in the original dense depth map and update and output the optimized dense depth map pixel by pixel.

2. The visual density depth optimization method based on local texture adaptive weighting according to claim 1, characterized in that, In step 1, the synchronized stereo images are used to generate a dense disparity map using a stereo matching algorithm, and then converted into a dense depth map; the formula for converting the synchronized stereo images into a dense depth map is as follows: ， in, The focal length of the camera. The length of the three-dimensional baseline. For pixels disparity value at that location, in pixels Pixel coordinates for synchronizing stereo images.

3. The visual density depth optimization method based on local texture adaptive weighting according to claim 2, characterized in that, Step 3 includes: defining a neighborhood window around each high-quality feature point, calculating the distance between each pixel in the neighborhood window and the high-quality feature point, and introducing a Gaussian weighting function based on the distance to calculate the Gaussian weight corresponding to each pixel. The Gaussian weighting function is: , in, This represents the Euclidean distance between a pixel within the neighborhood window and the high-quality feature point. , The standard deviation is a Gaussian distribution used to control the range of influence of pixels within the neighborhood window on the depth of high-quality feature points. This represents an exponential function.

4. The visual density depth optimization method based on local texture adaptive weighting according to claim 3, characterized in that, The sparse depth of the i-th high-quality feature point Compared with the original dense depth map The depth values ​​at corresponding pixel locations are weighted and fused to update the depth values ​​of each pixel within the neighborhood window pixel by pixel. The specific process is as follows: Using the formula: Calculate pixels Optimized depth value ,in, This is the mixing coefficient, used to balance the contributions of the original depth and the depth of high-quality feature points; is the normalized response weight for the i-th high-quality feature point.

5. The visual density depth optimization method based on local texture adaptive weighting according to claim 4, characterized in that, If a pixel falls into multiple neighborhood windows, the results of its multiple updates are then weighted and averaged to finally output a smooth and consistent optimized dense depth map.

6. The visual density depth optimization method based on local texture adaptive weighting according to claim 5, characterized in that, The optimized dense depth map is directly used as the depth prior input for the visual inertial odometry backend.

7. An apparatus for use in the visual density depth optimization method based on local texture adaptive weighting as described in any one of claims 1-6, characterized in that, It includes a sequentially connected conversion unit, calculation unit, and output unit, wherein: The conversion unit takes the acquired synchronous stereo images, generates a dense disparity map using a stereo matching algorithm, and converts it into a dense depth map. The computing unit performs a Laplacian transform on the left eye image of the synchronized stereo image to locate texture-rich regions, extracts high-response Harris corner points as high-quality feature points in the texture-rich regions, and calculates their sparse depth through triangulation. The output unit defines a neighborhood window centered on each high-quality feature point, calculates Gaussian weights based on the distance between pixels within the neighborhood window and the high-quality feature points, and weights and fuses the sparse depth with the original dense depth, updating and outputting the optimized dense depth map pixel by pixel.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the visual density depth optimization method based on local texture adaptive weighting as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the visual density depth optimization method based on local texture adaptive weighting as described in any one of claims 1-6.

Images