Single-frame high-precision multi-view reconstruction system and method based on curvature perception adaptive window

Abstract

The invention belongs to the technical field of optical reconstruction, and particularly provides a single-frame high-precision multi-view reconstruction system and method based on a curvature perception adaptive window, and the system comprises a multi-camera collection module which is used for collecting a single-frame image of a detected workpiece through a multi-view camera; the camera calibration module is used for carrying out distortion correction on the single-frame image; the multi-view matching module is used for obtaining a pixel-by-pixel hypothesis inclined plane of each single-frame image; the curvature sensing window size selection module is used for obtaining a comprehensive score of curvature sensing adaptive window size selection based on the local plane fitting error, the local hypothesis inclined plane normal vector consistency and the multi-view aggregation matching cost; and the multi-view reconstruction module is used for allocating the sizes of the matched windows based on the comprehensive score, and performing iteration of a multi-view three-dimensional pipeline algorithm by utilizing an allocation result of the sizes of the matched windows, so as to obtain a depth map and complete multi-view reconstruction. According to the technical scheme of the invention, single-frame high-precision and high-integrity measurement of the measured object is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of optical reconstruction, and particularly relates to a single-frame high-precision multi-view reconstruction system and method based on curvature perception adaptive window. BACKGROUND

[0002] Optical measurement is a key technology in geometric measurement research and industrial applications, including biomedical devices, shape measurement, and industrial manufacturing. In optical reconstruction tasks, the goal is to achieve high precision and high integrity of point clouds, and to ensure the adaptability of the reconstruction algorithm. Existing multi-view matching methods mostly focus on high-precision and high-integrity reconstruction of large-scale scenes, and direct matching based on object surface texture can significantly reduce reconstruction accuracy, making it difficult to be used for high-precision measurement in industrial sites. The existing speckle projection profilometry topography reconstruction method is inefficient, and the digital image correlation algorithm with a small matching window can cause significant errors. In areas with large curvature, the digital image correlation algorithm with a large matching window can cause large reconstruction errors. Therefore, a new matching method and a curvature perception adaptive window size selection algorithm need to be developed to achieve high-precision measurement. Many scholars have studied cost aggregation methods and window size selection techniques. Most methods rely on using auxiliary information in the image to empirically select the matching window size, and few methods are developed based on the surface morphology characteristics of the object, which limits the improvement of accuracy.

[0003] In application fields such as workpiece measurement and reverse engineering in industrial sites, where measurement accuracy is very high, existing methods have shortcomings in terms of surface topography measurement accuracy and integrity. SUMMARY

[0004] To solve the problems existing in the prior art, the application provides a single-frame high-precision multi-view reconstruction system and method based on curvature perception adaptive window, which realizes single-frame high-precision and high-integrity measurement.

[0005] To achieve the above-mentioned purpose, the application provides the following solutions: The single-frame high-precision multi-view reconstruction system based on curvature perception adaptive window comprises: A multi-camera acquisition module is configured to acquire single-frame images of a measured workpiece by using a multi-view camera. A camera calibration module is configured to obtain internal and external parameters of the multi-view camera and correct distortion of the single-frame images. A multi-view matching module is configured to calculate a multi-view aggregated matching cost of the single-frame images after distortion correction by using a multi-view stereo pipeline algorithm, and obtain a pixel-by-pixel hypothesis plane for each single-frame image. a curvature-aware window size selection module configured to obtain a comprehensive score of curvature-aware adaptive window size selection based on the local plane fitting error, the local hypothesis slant plane normal consistency, and the multi-view aggregated matching cost; a multi-view reconstruction module configured to allocate a matching window size based on the comprehensive score, and perform iteration of a multi-view stereo pipeline algorithm using the allocation result of the matching window size to obtain a depth map and complete multi-view reconstruction.

[0006] Preferably, the camera calibration module comprises: a monocular camera calibration unit configured to capture calibration board images of a monocular camera, and obtain intrinsic parameters of the monocular camera and initial radial and tangential distortion coefficients using a monocular camera calibration algorithm; a multi-camera calibration unit configured to construct a multi-camera group using the monocular camera, and synchronously capture multiple frames of calibration board images of the multi-camera group, and obtain extrinsic parameters of the multi-camera group, and then perform joint optimization of the intrinsic parameters, the extrinsic parameters, and the initial radial and tangential distortion coefficients of the monocular camera in the multi-camera group using a bundle adjustment algorithm to obtain an optimal solution; a distortion correction unit configured to perform distortion correction on the single frame images based on the optimal solution.

[0007] Preferably, the multi-view matching module comprises: an initialization unit configured to randomly initialize an initial hypothesis slant plane with a normal vector and a depth value in a preset depth range and a normal direction range for each pixel of the distortion-corrected single frame images; a matching cost calculation unit configured to take one single frame image as a reference image, and take the remaining single frame images as source images, and calculate a homography matrix between a current pixel of the reference image and a pixel of a source image using parameters of the initial hypothesis slant plane to obtain a multi-view aggregated matching cost; an optimization unit configured to optimize the multi-view aggregated matching cost using geometric consistency between the reference image and the source images, and take a hypothesis slant plane with the minimum multi-view aggregated matching cost as a final hypothesis slant plane.

[0008] Preferably, in the matching cost calculation unit, the process of calculating the multi-view aggregated matching cost comprises: obtaining a calculation weight of each source image based on a weight obtained from a binocular matching cost between a pixel point with the optimal matching cost in a multi-neighborhood around a pixel point of the reference image and a corresponding pixel point in each source image, and a preset good view selection set and a pixel index set in the source image that satisfy a preset matching cost; obtaining the multi-view aggregated matching cost based on the calculation weight of each source image and the binocular matching cost between each source image and the reference image at the current reference pixel point.

[0009] Preferably, the curvature-aware window size selection module comprises: a sampling unit configured to locally adaptively sample black pixels of the reference image by using an eight-direction ring sequence sampling strategy to obtain a local neighborhood spatial point set; a score calculation unit configured to calculate a comprehensive score of curvature-aware adaptive window size selection by using a fitting plane error of the local neighborhood spatial point set, an average normal vector angle between a normal vector of a hypothetical inclined plane corresponding to a neighborhood pixel and a normal vector of a hypothetical inclined plane corresponding to a center pixel, and a difference between a minimum and a maximum multi-view aggregated matching cost.

[0010] Preferably, the score calculation unit comprises: a first score sub-unit configured to perform spatial plane fitting by using the local neighborhood spatial point set, calculate a fitting standard deviation of the spatial plane, and obtain a standard deviation-based window size score by using the fitting standard deviation; a second score sub-unit configured to smooth a normal vector of a sampling point by using a normal vector of a hypothetical inclined plane around the sampling point, calculate an average normal vector angle between the smoothed normal vector of the sampling point and the normal vector of the hypothetical inclined plane corresponding to the center pixel, and calculate a normal vector angle-based window size score by using the average normal vector angle; a third score sub-unit configured to obtain a matching cost-based window size score by using a multi-view aggregated matching cost of a predefined minimum matching window and a predefined maximum matching window; a comprehensive score calculation sub-unit configured to obtain the comprehensive score of curvature-aware adaptive window size selection by using the standard deviation-based window size score, the normal vector angle-based window size score, and the matching cost-based window size score.

[0011] The application further provides a single-frame high-precision multi-view reconstruction method based on curvature-aware adaptive window, which is used to implement the system and comprises the following steps: acquiring a single-frame image of a measured workpiece by using a multi-view camera; obtaining internal and external parameters of the multi-view camera, and rectifying distortion of the single-frame image; calculating a multi-view aggregated matching cost of the rectified single-frame image by using a multi-view stereo pipeline algorithm, and obtaining a pixel-by-pixel hypothetical inclined plane of each single-frame image; obtaining a comprehensive score of curvature-aware adaptive window size selection based on a local plane fitting error, a local hypothetical inclined plane normal vector consistency, and the multi-view aggregated matching cost; allocating a matching window size based on the comprehensive score, and performing iteration of the multi-view stereo pipeline algorithm by using an allocation result of the matching window size to obtain a depth map and complete multi-view reconstruction.

[0012] Preferably, the method for rectifying the distortion of the single frame image comprises: capturing a calibration board image of a monocular camera, obtaining the intrinsic parameters of the monocular camera and initial radial and tangential distortion coefficients by using a monocular camera calibration algorithm; constructing a multi-view camera group by using the monocular camera, synchronously capturing multiple calibration board images of the multi-view camera group, obtaining the extrinsic parameters of the multi-view camera group, and then jointly optimizing the intrinsic parameters, extrinsic parameters and initial radial and tangential distortion coefficients of the monocular camera in the multi-view camera group by using a bundle adjustment algorithm to obtain an optimal solution; based on the optimal solution, rectifying the distortion of the single frame image.

[0013] Compared with the prior art, the beneficial effects of the present application are: the technical scheme of the present application realizes non-contact measurement, and has high precision, fast speed, good versatility, can realize the shape curvature sensing and single frame measurement of the measured object, realizes the effect of self-adaptation to achieve higher measurement precision in high and low curvature areas which is not achieved by the existing method, and is more robust to occlusion and shadow (multi-view vision). In a field of view with a physical resolution of 4 pixels / mm, high integrity measurement with a precision of about 0.06mm is realized. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical scheme of the present application, the following briefly introduces the drawings needed to be used in the embodiments of the present application. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0015] Figure 1 The schematic diagram of the inclined plane for the embodiments of the present application is assumed; Figure 2 The schematic diagram of the double-color checkerboard division and iteration mode for the embodiments of the present application is assumed; Figure 3 The schematic diagram of the local neighborhood adaptive sampling for the embodiments of the present application is assumed; Figure 4 The schematic diagram of the single frame high-precision multi-view reconstruction system structure based on curvature sensing adaptive window for the embodiments of the present application is assumed. DETAILED DESCRIPTION

[0016] The technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] Example 1: like Figure 4 As shown, the single-frame high-precision multi-view reconstruction system based on curvature-aware adaptive window includes: a multi-camera acquisition module, a camera calibration module, a multi-view matching module, a curvature-aware window size selection module, and a multi-view reconstruction module.

[0019] The multi-camera acquisition module is used to acquire single-frame images of the workpiece under test using multiple cameras. Specifically, the camera's exposure time is adjusted to a reasonable value, and a single-frame image of the workpiece under test is acquired using the multi-camera acquisition module with photolithographic digital speckle projection (the number of cameras is variable, this is a general method).

[0020] The camera calibration module is used to acquire the intrinsic and extrinsic parameters of the multi-view camera and to perform distortion correction on the single-frame image.

[0021] A further embodiment includes a camera calibration module comprising: The monocular camera calibration unit is used to capture calibration plate images of the monocular camera and obtain the intrinsic parameters and initial radial and tangential distortion coefficients of each monocular camera using a monocular camera calibration algorithm. In this embodiment, 30 calibration plate images are captured for each camera, covering various imaging positions and working distances.

[0022] The specific calibration algorithm for a monocular camera is as follows: The calibration board is either a checkerboard or a circular spot calibration board. In the acquired images, the black and white corner points of the checkerboard or the center points of the circular spots are obtained using a corner detection algorithm (checkerboard) and an ellipse edge extraction algorithm (circular spots), respectively. This point set is of sub-pixel accuracy. A world coordinate system (xyz) is constructed using the predefined corner size information on the calibration board. Calibration is performed using the same world coordinate system for all poses. The world coordinates of the corner points and the detected sub-pixel coordinates of the image are jointly input into the `calibrateCamera` function in OpenCV for monocular calibration to obtain the camera's intrinsic parameters. With distortion coefficient .

[0023] The multi-view camera calibration unit is used to construct a multi-view camera group by using a monocular camera and capture multiple calibration board images of each multi-view camera group, similar to monocular camera calibration, a calibration board world coordinate system is established, and a set of sub-pixel points of the markers in the calibration board in the multi-camera imaging image is extracted, and one-to-one correspondence of the sub-pixel points between the cameras is performed. One of the cameras is regarded as a reference camera, and the stereoCalibration algorithm in opencv is used to calculate the binocular extrinsic parameters of the remaining cameras and the reference camera (the intrinsic parameters of the monocular camera and the initial radial and tangential distortion coefficients are used as input initial values), and the bundle adjustment algorithm is used to jointly optimize the intrinsic parameters, extrinsic parameters and initial radial and tangential distortion coefficients of the multi-view cameras in the camera group, and the optimal solution is obtained. The specific cost function formula of the bundle adjustment algorithm is as follows: , By continuously iterating the camera intrinsic parameters, extrinsic parameters and distortion parameter optimization to solve the above cost function, the comprehensive re-projection error is minimized, so as to realize bundle adjustment, wherein the spatial coordinate points are regarded as constants (unchanged) because the manufacturing precision of the calibration board is high enough. is the re-projection pixel point, is the detected pixel point, is the number of calibration board poses, and n is the number of cameras, represents a set of rotation matrices of each camera in the world coordinate system, represents a set of translation vectors of each camera in the world coordinate system, represents the spatial coordinates of the marker points reconstructed by the calibration board at the i-th position, represents the component in represents the component in represents the component in .

[0024] Specifically, a plurality of cameras are regarded as a group, and the method in the monocular camera calibration unit is used to capture 30 groups (determined by the camera arrangement form) of calibration board pose images (the calibration board is at least in the common field of view of two cameras).

[0025] The distortion correction unit is used to correct the distortion of a single frame image based on the optimal solution.

[0026] The multi-view matching module is used to calculate the multi-view aggregated matching cost of the single frame image after distortion correction by using a multi-view stereo pipeline algorithm, and obtain a pixel-by-pixel hypothesis inclined plane for each single frame image.

[0027] Further embodiments are that the multi-view matching module comprises: The initialization unit is used to initialize an initial hypothesis inclined plane with a normal vector and a depth value in a preset depth range and a normal vector direction range for each pixel of the single frame image after distortion correction; specifically, a pixel-by-pixel random spatial plane initialization is performed, such asFigure 1 as shown.

[0028] The matching cost calculation unit is configured to take one single-frame image as a reference image and the rest single-frame images as source images, calculate the homography matrix between the current pixel of the reference image and the pixels of the source images by using the parameters of the initial hypothesis plane to achieve sub-pixel matching block correspondence, and obtain the multi-view aggregated matching cost. In a further embodiment, the process of calculating the multi-view aggregated matching cost in the matching cost calculation unit includes: obtaining the calculation weight of each source image based on the pixel points with the optimal matching cost in the multi-neighborhood around the pixel point of the reference image, the weight obtained from the binocular matching cost between the corresponding pixel points in each source image, and the pre-set good view selection set and the pixel index set in the source image satisfying the pre-set matching cost.

[0029] obtaining the multi-view aggregated matching cost based on the calculation weight of each source image and the binocular matching cost between each source image and the reference image at the current reference pixel point wherein n-1 is the number of source images, represents the source image, is the calculation weight of each source image, which can be expressed as: wherein represents the pixel point with the optimal matching cost in the multi-neighborhood around the reference pixel point and the weight obtained from the binocular matching cost between each source image, represents the good view selection set, the good view being the pixel point set in each neighborhood with the optimal matching cost, which satisfies more than matching costs less than and less than matching costs greater than (the smaller the matching cost, the more accurate the matching), and is a pre-set cost threshold constant. represents the total number of matching costs less than , and represents the pixel index set with the lower matching cost in the jth source image, is the binocular matching cost between each reference image and the source image at the current reference pixel point, which is realized by using the bilinear normalized cross-correlation algorithm. Finally, the hypothesis plane parameters of the center pixel are updated to the plane parameters with the lowest multi-view aggregated matching cost. As shown. Figure 2

[0030] ​​​The optimization unit is configured to optimize the multi-view aggregated matching cost by using geometric consistency between the reference image and the source image, and to take the hypothesis inclined plane with the minimum multi-view aggregated matching cost as the final hypothesis inclined plane.

[0031] Specifically, after multiple iterations, geometric consistency between the reference image and the source image is used to improve consistency between the multiple views, as shown in the following formula: , wherein is a reprojection error between the reference image and the source image at the reference pixel, is a balance constant of geometric consistency and photometric consistency in space. Finally, more multi-view aggregated matching costs are calculated by using the plane parameter jitter method, and more accurate inclined plane parameters are obtained in a larger solution space. Finally, the inclined plane parameter with the minimum multi-view aggregated matching cost is taken as the final corresponding hypothesis inclined plane of the pixel point.

[0032] The curvature-aware window size selection module is configured to obtain a comprehensive score of curvature-aware adaptive window size selection based on the hypothesis inclined plane and the multi-view aggregated matching cost.

[0033] Further embodiments are that the curvature-aware window size selection module comprises: The sampling unit is configured to perform local neighborhood adaptive sampling on the black pixels of the reference image by using an eight-direction ring sequential sampling strategy to obtain a local neighborhood spatial point set. Specifically, the eight-direction ring sequential sampling strategy is used, as shown in the following formula: Figure 3 Specifically, starting from the first black pixel at the top, the black pixels are sequentially sampled in eight directions in a clockwise order. High-confidence spatial points are obtained by using the following formula condition to achieve high-confidence matching: Reconstruction: , wherein represents the total number of the reference image plus the source image satisfying the reprojection error threshold condition at the sampling point. represents a 3x3 matrix composed of the first three columns of the kth camera projection matrix, represents a 3x1 matrix composed of the last column of the kth camera projection matrix, represents the depth value of the kth image at the pixel satisfying the condition that is greater than 1 will be recorded, and the pixels not satisfying the condition will be discarded. When the number of valid recorded sampling pixels reaches the threshold , the search stops. By using the above formula, the corresponding spatial coordinates of the pixels including the center point are calculated, and the spatial coordinates use the reference camera coordinate system. This The spatial coordinates represent a structured region of the neighborhood around the center point, and are fitted into a spatial inclined plane by least square method.

[0034] The score calculation unit is configured to calculate a comprehensive score of the curvature-aware adaptive window size selection by using the fitting plane error of the local neighborhood spatial point set, the average normal vector angle between the normal vector of the assumed inclined plane corresponding to each pixel in the neighborhood and the normal vector of the assumed inclined plane corresponding to the center pixel, and the difference between the minimum and maximum multi-view aggregated matching cost of the matching window.

[0035] In a further implementation, the score calculation unit comprises: A first score sub-unit is configured to perform spatial plane fitting by using the local neighborhood spatial point set, and calculate the fitting standard deviation of the spatial plane, and obtain the window size score based on the standard deviation by using the fitting standard deviation; specifically, the fitting standard deviation of the plane is denoted as , and the window size score derived from the standard deviation of the plane fitting is denoted as The window size score based on the standard deviation of the plane fitting is calculated as follows: , wherein is the physical resolution of the camera under the current field of view, and is a pre-defined constant coefficient.

[0036] A second score sub-unit is configured to smooth the normal vector of the sampling point by using the normal vector of the assumed inclined plane around the sampling point, and calculate the average normal vector angle between the smoothed normal vector of the sampling point and the normal vector of the assumed inclined plane corresponding to the center pixel, in order to reduce the influence of single-point noise on the calculation of the normal vector angle; the The angle values are arranged in ascending order, and the window size score based on the average normal vector angle is calculated by using the average normal vector angle.

[0037] The average normal vector angle is given by the following formula: , wherein is the angle between the smoothed normal vector of the o-th sampling point and the normal vector of the assumed inclined plane corresponding to the center pixel, and is an radian value. On this basis, the window size score based on the average normal vector angle can be represented by the following formula: , wherein is a constant coefficient.

[0038] A third score sub-unit is configured to obtain the window size score based on the matching cost by using the multi-view aggregated matching cost of the pre-defined minimum matching window and maximum matching window: ,​ wherein and are the multi-view aggregated matching cost calculated with the predefined minimum matching window and maximum matching window respectively. and is a predefined constant.

[0039] The integrated score calculation sub-unit is configured to obtain an integrated score of the curvature-aware adaptive window size selection by using the standard deviation-based window size score, the average normal vector angle-based window size score, and the matching cost-based window size score. After the three scores are calculated, the integrated score formula can be expressed as follows: , represents the preset constant weight of the first score sub-unit, represents the preset constant weight of the second score sub-unit, represents the preset constant weight of the third score sub-unit.

[0040] When and one of which is higher than a preset matching cost threshold constant , the integrated score formula is rewritten as follows: , represents the preset constant weight of the first score sub-unit in this case, represents the preset constant weight of the second score sub-unit in this case.

[0041] The multi-view reconstruction module is configured to allocate the matching window size based on the integrated score, and perform iteration of the multi-view stereo pipeline algorithm by using the allocation result of the matching window size to obtain a depth map and complete multi-view reconstruction.

[0042] Specifically, the minimum matching window size and the maximum matching window size are set, and a predefined number of window size levels is set, and the score lower limit and the score upper limit are set. When the score is less than the score lower limit, the minimum matching window size is allocated to the pixel point, and when the score is greater than the score upper limit, the maximum matching window size is allocated to the pixel point. When the score is between the two, the window size level is proportional to the curvature-aware integrated score.

[0043] After one or two iterations of the multi-view stereo pipeline algorithm using the adaptive window size distribution obtained by the curvature-aware score, a plurality of depth maps are obtained, corresponding to a plurality of images. The reference view can be used to implement depth map fusion at each pixel point using a consistency test method including reprojection error, relative depth value difference, and normal vector consistency of each source image, to obtain a high-precision and high-completeness point cloud.

[0044] The technical scheme of the present application realizes non-contact measurement, has high precision, fast speed, good versatility, can perform topography curvature perception and single-frame measurement on the object to be measured, realizes the effect of adaptive high measurement precision in high and low curvature regions which is not achieved by existing methods, and is robust to occlusion and shadow (multi-view vision). In a field of view with a physical resolution of 4 pixels / mm, high-completeness measurement with a precision of about 0.06 mm is realized.

[0045] Embodiment two The single-frame high-precision multi-view reconstruction method based on curvature-aware adaptive window is used for the system of embodiment one, and comprises the following steps: A single-frame image of the workpiece to be measured is captured by using a multi-view camera.

[0046] The internal and external parameters of the multi-view camera are obtained, and the single-frame image is rectified.

[0047] The multi-view aggregated matching cost of the rectified single-frame image is calculated by using a multi-view stereo pipeline algorithm, and a pixel-by-pixel hypothesis slant plane of each single-frame image is obtained.

[0048] Based on the local plane fitting error, the local hypothesis slant plane normal vector consistency, and the multi-view aggregated matching cost, a comprehensive score for curvature-aware adaptive window size selection is obtained.

[0049] The matching window size is allocated based on the comprehensive score, and the iteration of the multi-view stereo pipeline algorithm is performed using the allocation result of the matching window size, to obtain a depth map and complete multi-view reconstruction.

[0050] A further embodiment is that the method for rectifying the single-frame image comprises the following steps: A calibration plate image of a monocular camera is captured, and the internal parameters of the monocular camera and the initial radial and tangential distortion coefficients are obtained by using a monocular camera calibration algorithm.

[0051] A multi-view camera group is constructed by using a monocular camera, and a plurality of calibration plate images of the multi-view camera group are synchronously captured, the external parameters of the multi-view camera group are obtained by calibration, and then the internal parameters, the external parameters, and the initial radial and tangential distortion coefficients of the monocular cameras in the multi-view camera group are jointly optimized by using a bundle adjustment algorithm to obtain an optimal solution.

[0052] Based on the optimal solution, distortion correction is performed on the single frame image.

[0053] The above-described embodiments are merely intended to describe the preferred modes of the present application, and are not intended to limit the scope of the present application. Various modifications and improvements to the present application made by those skilled in the art without departing from the design spirit of the present application shall fall within the scope of the present application as defined by the claims.

Claims

1. A single-frame high-precision multi-view reconstruction system based on curvature-aware adaptive windowing, characterized in that, The method comprises the following steps: A multi-camera acquisition module is used to acquire a single frame image of a measured workpiece by using a multi-view camera; A camera calibration module is used to obtain internal and external parameters of the multi-view camera, and to correct distortion of the single frame image; A multi-view matching module is used to calculate a multi-view aggregated matching cost of the single frame image after distortion correction by using a multi-view stereo pipeline algorithm, and to obtain a pixel-by-pixel hypothesis slant plane of each single frame image; A curvature-aware window size selection module is used to obtain a comprehensive score of curvature-aware adaptive window size selection based on local plane fitting error, local hypothesis slant plane normal vector consistency and the multi-view aggregated matching cost; A multi-view reconstruction module is used to allocate a matching window size based on the comprehensive score, and to perform iteration of the multi-view stereo pipeline algorithm by using the allocation result of the matching window size, so as to obtain a depth map and complete multi-view reconstruction.

2. The system of claim 1, wherein, The camera calibration module comprises: A monocular camera calibration unit is used to capture a calibration board image of a monocular camera, and to obtain internal parameters and initial radial and tangential distortion coefficients of the monocular camera by using a monocular camera calibration algorithm; A multi-view camera calibration unit is used to construct a multi-view camera group by using a monocular camera, and to synchronously capture multiple calibration board images of the multi-view camera group, so as to obtain external parameters of the multi-view camera group, and then to perform joint optimization on the internal parameters, the external parameters and the initial radial and tangential distortion coefficients of the monocular camera in the multi-view camera group by using a bundle adjustment algorithm, so as to obtain an optimal solution; A distortion correction unit is used to correct distortion of the single frame image based on the optimal solution.

3. The system of claim 1, wherein, The multi-view matching module comprises: An initialization unit is used to randomly initialize an initial hypothesis slant plane with a normal vector and a depth value in a preset depth range and a normal vector direction range for each pixel of the single frame image after distortion correction; A matching cost calculation unit is used to take one single frame image as a reference image, and the remaining single frame images as source images, to calculate a homography matrix between a current pixel of the reference image and a source image pixel by using parameters of the initial hypothesis slant plane, and to obtain a multi-view aggregated matching cost; An optimization unit is used to optimize the multi-view aggregated matching cost by using geometric consistency between the reference image and the source images, and to take a hypothesis slant plane with the minimum multi-view aggregated matching cost as a final hypothesis slant plane.

4. The system of claim 3, wherein, In the matching cost calculation unit, the process of calculating the multi-view aggregated matching cost comprises: Based on the weight obtained from the binocular matching cost between the pixel point with the optimal matching cost in the multi-neighborhood around the reference image pixel point and the corresponding pixel point in each source image, and a preset good view selection set and a pixel index set in the source image that satisfies a preset matching cost, a calculation weight of each source image is obtained; Based on the calculation weight of each source image and the binocular matching cost between each source image and the reference image at the current reference pixel point, the multi-view aggregated matching cost is obtained.

5. The system of claim 3, wherein, The curvature-aware window size selection module comprises: A sampling unit is used to perform local neighborhood adaptive sampling on black pixels of the reference image by using an eight-direction ring sequential sampling strategy, so as to obtain a local neighborhood spatial point set; The score calculation unit is configured to calculate a comprehensive score of curvature-aware adaptive window size selection by using a fitting plane error of the local neighborhood spatial point set, an average normal vector angle between a normal vector of a center pixel and a normal vector of a hypothetical inclined plane corresponding to each pixel in the neighborhood, and a difference between a minimum and a maximum multi-view aggregated matching cost.

6. The system of claim 5, wherein, The score calculation unit comprises: A first score sub-unit configured to perform spatial plane fitting on the local neighborhood spatial point set, calculate a fitting standard deviation of the spatial plane, and obtain a standard deviation-based window size score by using the fitting standard deviation; A second score sub-unit configured to smooth the normal vector of the sampling point by using the normal vector of the hypothetical inclined plane around the sampling point, calculate an average normal vector angle between the smoothed normal vector of the sampling point and the normal vector of the center pixel, and obtain an average normal vector angle-based window size score by using the average normal vector angle; A third score sub-unit configured to obtain a matching cost-based window size score by using the multi-view aggregated matching cost of the predefined minimum and maximum matching windows; A comprehensive score calculation sub-unit configured to obtain the comprehensive score of curvature-aware adaptive window size selection by using the standard deviation-based window size score, the average normal vector angle-based window size score, and the matching cost-based window size score.

7. A single frame high-precision multi-view reconstruction method based on curvature-aware adaptive windowing for implementing the system of any of claims 1-6, characterized in that, The method comprises: capturing a single-frame image of a measured workpiece by using a multi-view camera; obtaining internal and external parameters of the multi-view camera, and rectifying the single-frame image; calculating a multi-view aggregated matching cost of the rectified single-frame image by using a multi-view stereo pipeline algorithm, and obtaining a hypothetical inclined plane of each pixel in the single-frame image; obtaining a comprehensive score of curvature-aware adaptive window size selection based on a local plane fitting error, local normal vector consistency of the hypothetical inclined plane, and the multi-view aggregated matching cost; allocating a matching window size based on the comprehensive score, and performing iteration of the multi-view stereo pipeline algorithm by using the allocation result of the matching window size to obtain a depth map and complete multi-view reconstruction.

8. The method of claim 7, wherein, The method for rectifying the single-frame image comprises: capturing a calibration board image of a monocular camera, and obtaining internal parameters and initial radial and tangential distortion coefficients of the monocular camera by using a monocular camera calibration algorithm; constructing a multi-view camera group by using the monocular camera, synchronously capturing multiple calibration board images of the multi-view camera group, obtaining external parameters of the multi-view camera group, and then performing joint optimization on the internal parameters, the external parameters, and the initial radial and tangential distortion coefficients of the monocular camera in the multi-view camera group by using a bundle adjustment algorithm to obtain an optimal solution; rectifying the single-frame image based on the optimal solution.

Images