Three-dimensional modeling method based on binocular vision

Abstract

The present invention provides a three-dimensional modeling method based on binocular vision, including selecting a positioning point, using a binocular camera to shoot in six directions to obtain six sets of image pairs, calculating parallax, using camera internal and external parameters and parallax to calculate the three-dimensional coordinates of the object through triangulation, and integrating and constructing a three-dimensional model. The present invention can achieve all-round and high-precision three-dimensional modeling of the target object, and improve the efficiency and accuracy of modeling.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and more specifically, to a three-dimensional modeling method based on binocular vision. Background Art

[0002] In the field of 3D modeling technology, especially in the application of 3D information reconstruction using binocular vision systems, existing technologies mainly rely on the principle of stereo vision, that is, collecting images of the target through two or more cameras and reconstructing 3D information or achieving 3D measurement based on the difference between pixels. This method is widely used in industrial fields such as 3D navigation, positioning, target tracking and robotics research.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing binocular vision system may encounter problems such as inaccurate parallax calculation and low matching efficiency when processing fast-moving objects or complex scenes, and in the three-dimensional modeling of large-scale scenes, a large amount of manpower and material resources are required for on-site surveys, which is inefficient. In addition, the existing three-dimensional modeling technology has limitations when dealing with situations where blind spots or tops of buildings cannot be photographed, resulting in incomplete model effects. Summary of the invention

[0004] The present invention provides a three-dimensional modeling method based on binocular vision, comprising:

[0005] Select a positioning point, take the positioning point as the center, use the binocular camera to shoot in six directions in sequence, and obtain six sets of image pairs;

[0006] Calculate disparity based on matched feature points;

[0007] Using the camera's intrinsic matrix, extrinsic matrix and parallax, the three-dimensional coordinates of the target object in six directions are calculated through the principle of triangulation;

[0008] Integrate the three-dimensional coordinates in six directions to construct a three-dimensional model of the target object.

[0009] Furthermore, the feature point extraction includes:

[0010] The scale space is constructed by convolving the Gaussian kernels of different scales with the image, and the local extreme points are detected in different scale spaces. The scale space formula is:

[0011]

[0012] in, For images, is the scale factor;

[0013] Locate the detected extreme points and remove low-contrast points and unstable edge response points;

[0014] Assign a main direction to each key point, which is determined based on the gradient direction distribution of the pixels in the key point neighborhood.

[0015] The gradient amplitude formula is:

[0016]

[0017] The gradient direction formula is:

[0018]

[0019] Generate key point descriptors for feature point matching.

[0020] Furthermore, the feature point matching adopts a nearest neighbor matching algorithm based on Euclidean distance, including:

[0021] For a feature point, calculate the Euclidean distance between it and all feature points in another image. The Euclidean distance formula is:

[0022]

[0023] in, and are the coordinates of the two feature points;

[0024] Find the nearest neighbor and the next nearest neighbor feature point. If the ratio of the nearest neighbor distance to the next nearest neighbor distance is less than the preset value, the two feature points are matched successfully.

[0025] Furthermore, the disparity calculation adopts a block matching algorithm, and the calculation formula is:

[0026]

[0027] in represents the left image, represents the right image, is the pixel coordinate, For parallax.

[0028] Furthermore, the calculation formula of the triangulation principle is:

[0029]

[0030] in is the three-dimensional coordinate of the target object, is the baseline distance of the binocular camera, is the camera focal length, is parallax, is the image height, is the pixel coordinate.

[0031] Furthermore, after calculating the three-dimensional coordinates of the target object in six directions, the step of optimizing the three-dimensional coordinates is also included, specifically:

[0032] An error function based on the least squares method is established, the three-dimensional coordinate data is substituted into it, and the error function is converged to the minimum value through iterative calculation, thereby obtaining the optimized three-dimensional coordinates.

[0033] Furthermore, the optimization adopts the least square method, and the calculation process is as follows: Assume that the measured three-dimensional coordinates are The corresponding theoretical value is , error function:

[0034]

[0035] The parameter correction value that minimizes E is found through mathematical methods such as derivation, and the original three-dimensional coordinates are corrected.

[0036] Furthermore, before shooting in six directions in sequence, the method also includes a step of calibrating the binocular camera to obtain the camera's intrinsic parameter matrix and extrinsic parameter matrix. The specific steps are as follows:

[0037] Set up the calibration board, collect images of the calibration board at different angles and positions, extract the coordinates of the feature points on the calibration board in the image, combine the known coordinates of the feature points in the world coordinate system, and calculate the camera's intrinsic parameter matrix and extrinsic parameter matrix through a specific mathematical model and optimization algorithm.

[0038] Furthermore, before feature point extraction and matching, a step of preprocessing each set of image pairs is also included, and the specific steps are as follows:

[0039] Gaussian filtering algorithm is used to denoise the image, and image enhancement is performed by adjusting the contrast and brightness of the image.

[0040] Furthermore, the preprocessing includes image denoising and image enhancement. Image denoising adopts Gaussian filtering algorithm, and the filtering formula is:

[0041]

[0042] in, is the standard deviation, and the noise is removed by convolution operation with the image pixels; image enhancement adjusts the contrast and brightness of the image through linear transformation, and the transformation formula is:

[0043]

[0044] in is the contrast adjustment factor, is the brightness adjustment coefficient, is the original image pixel value, is the pixel value of the enhanced image.

[0045] The above-mentioned embodiments of the present invention have at least the following beneficial effects: the three-dimensional modeling method based on binocular vision described in the present invention can improve the accuracy and efficiency of three-dimensional modeling. By selecting a positioning point and shooting in six directions around the point, combined with feature point matching and parallax calculation, the method can fully capture the three-dimensional information of the target object, thereby constructing a more accurate three-dimensional model. In addition, the method can further optimize the accuracy of the three-dimensional coordinates through the triangulation principle and the precise calculation of the internal and external parameters of the camera, so that the constructed model is richer and more realistic in details.

[0046] In addition, the present invention also includes a calibration step for the binocular camera, as well as the application of image preprocessing and optimization algorithms, which can effectively improve the stability of feature point matching and the accuracy of parallax calculation. By optimizing the three-dimensional coordinates by the least squares method, the error can be further reduced and the accuracy of the model can be improved. Overall, the method of the present invention can improve the quality of three-dimensional modeling while reducing the demand for human and material resources, making the three-dimensional modeling process more efficient and economical. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0048] Figure 1 A schematic flow chart of a three-dimensional modeling method based on binocular vision provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0050] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0051] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0052] Reference below Figure 1 , Figure 1 The following is a flow chart of a three-dimensional modeling method based on binocular vision provided by an embodiment of the present invention. Figure 1 As shown, a three-dimensional modeling method 100 based on binocular vision includes:

[0053] Step 101, select a positioning point, take the positioning point as the center, use a binocular camera to shoot in six directions in sequence, and obtain six sets of image pairs;

[0054] Step 102, calculating the disparity according to the matched feature points;

[0055] Step 103, using the camera's intrinsic parameter matrix, extrinsic parameter matrix and parallax, calculate the three-dimensional coordinates of the target object in six directions through the principle of triangulation;

[0056] Step 104 , integrating the three-dimensional coordinates in six directions to construct a three-dimensional model of the target object.

[0057] It should be noted that this embodiment relates to a 3D modeling method based on binocular vision, which first requires selecting a positioning point as the center point of shooting. The positioning point refers to a reference point pre-set on the target object, which is used to determine the relative position and direction of the binocular camera when shooting. The binocular camera refers to a visual system composed of two cameras, which simulate human eyes and obtain depth information by capturing images of the same scene from two different perspectives.

[0058] Specifically, after selecting the positioning point, the binocular camera is used to shoot the six directions of the target object in turn to obtain six sets of image pairs. The six directions can be the front, back, left, right, top, and bottom of the object, or other directions customized according to actual needs. Each time the binocular camera is shot, the two cameras of the binocular camera will simultaneously capture images in the corresponding directions to form a set of image pairs. These image pairs will be used for subsequent feature point matching and disparity calculation. In actual operation, the camera's intrinsic parameter matrix and extrinsic parameter matrix need to be obtained in advance through a calibration process to ensure that the captured image can accurately reflect the actual position and posture of the object.

[0059] Preferably, in order to improve the accuracy and efficiency of 3D modeling, the binocular camera can be accurately calibrated before shooting to obtain its intrinsic parameter matrix and extrinsic parameter matrix. The intrinsic parameter matrix contains the camera's focal length, principal point coordinates and other intrinsic parameters, while the extrinsic parameter matrix describes the camera's position and posture relative to the world coordinate system. These parameters are crucial for subsequent parallax calculations and accurate reconstruction of 3D coordinates.

[0060] Furthermore, during the calibration process, a calibration plate with a known geometric pattern can be used. By collecting images of the calibration plate at different angles and positions, the coordinates of the feature points on the calibration plate in the image are extracted, and combined with the known coordinates of the feature points in the world coordinate system, the intrinsic and extrinsic matrix of the camera is calculated through a specific mathematical model and optimization algorithm. In addition, in order to improve the accuracy of feature point matching, a key point extraction method based on scale space and gradient direction distribution and a nearest neighbor matching algorithm based on Euclidean distance can be used for feature point matching.

[0061] In some embodiments, the feature point extraction includes:

[0062] The scale space is constructed by convolving the Gaussian kernels of different scales with the image, and the local extreme points are detected in different scale spaces. The scale space formula is:

[0063]

[0064] in, For images, is the scale factor;

[0065] Locate the detected extreme points and remove low-contrast points and unstable edge response points;

[0066] Assign a main direction to each key point, which is determined based on the gradient direction distribution of the pixels in the key point neighborhood.

[0067] The gradient amplitude formula is:

[0068]

[0069] The gradient direction formula is:

[0070]

[0071] Generate key point descriptors for feature point matching.

[0072] It should be noted that this embodiment involves a process of feature point extraction, which includes convolving an image with Gaussian kernels of different scales to construct a scale space and detecting local extreme points in different scale spaces. Gaussian kernel is a commonly used image filter used to smooth an image and enhance local features of the image. Scale space is an image representation method that captures image features by observing the image at different scales.

[0073] Specifically, feature point extraction first involves constructing a scale space, which is achieved by convolving the image with Gaussian kernels of different scales. The formula for the Gaussian kernel is:

[0074]

[0075] in, is a scale factor used to control the width of the Gaussian kernel. In practical applications, The value of is used to capture feature points of different scales. The detection of local extreme points is to find points with maximum or minimum intensity values ​​in the local area in the constructed scale space. These points often correspond to feature points in the image.

[0076] Preferably, in order to improve the accuracy and robustness of feature point extraction, the detected extreme points can be further screened to remove low-contrast points and unstable edge response points. The contrast can be determined by calculating the gradient amplitude of the image. The calculation formula of the gradient amplitude is:

[0077]

[0078] in, Represents an image, and Represents pixel coordinates.

[0079] Furthermore, a main direction is assigned to each key point, which is determined based on the gradient direction distribution of the pixels in the neighborhood of the key point. The calculation formula of the gradient direction is:

[0080]

[0081] Through these steps, the descriptor of the key point can be generated for subsequent feature point matching.

[0082] In some embodiments, the feature point matching adopts a nearest neighbor matching algorithm based on Euclidean distance, including:

[0083] For a feature point, calculate the Euclidean distance between it and all feature points in another image. The Euclidean distance formula is:

[0084]

[0085] in, and are the coordinates of the two feature points;

[0086] Find the nearest neighbor and the next nearest neighbor feature point. If the ratio of the nearest neighbor distance to the next nearest neighbor distance is less than the preset value, the two feature points are matched successfully.

[0087] It should be noted that this embodiment describes the process of feature point matching, which uses the nearest neighbor matching algorithm based on Euclidean distance. Euclidean distance is the straight-line distance between two points in Euclidean space, and is a common method for measuring the distance between two feature points in space. The nearest neighbor matching algorithm is a method for finding the nearest feature point of a given feature point in another image, which is crucial for establishing the correspondence between images.

[0088] Specifically, in the feature point matching process, we first need to calculate the Euclidean distance between a feature point and all feature points in another image. The calculation formula of the Euclidean distance is:

[0089]

[0090] in, and are the coordinates of the two feature points. Then, for each feature point, find the closest and second closest feature points. If the ratio of the nearest neighbor distance to the second closest neighbor distance is less than a preset threshold, the two feature points are considered to be matched successfully. The setting of this threshold has an important impact on the accuracy of the matching, and usually needs to be determined based on the actual application scenario and data characteristics.

[0091] Preferably, in order to improve the accuracy and robustness of the matching, the above process can be optimized. For example, a more complex distance metric method can be introduced, such as Mahalanobis distance, which takes into account the covariance structure of the data.

[0092] Furthermore, machine learning methods, such as support vector machines (SVM) or neural networks, can be used to classify feature points to distinguish between successful and unsuccessful matching points. These methods can further improve the accuracy of feature point matching, especially when the feature point distribution is complex or the noise is large. Through these optimization measures, the feature point matching process can be ensured to be more reliable, providing a solid foundation for subsequent 3D modeling.

[0093] In some embodiments, the disparity calculation adopts a block matching algorithm, and the calculation formula is:

[0094]

[0095] in represents the left image, represents the right image, is the pixel coordinate, For parallax.

[0096] It should be noted that this embodiment describes the process of disparity calculation, in which a block matching algorithm is used to calculate the disparity. The block matching algorithm is a commonly used technique in image processing, which is used to estimate the disparity between two images, that is, the horizontal or vertical displacement of the same scene in two images with different viewing angles. Disparity is a key parameter for reconstructing the depth information of a three-dimensional scene in a binocular vision system.

[0097] Specifically, during the disparity calculation process, for each pixel in the left image, the algorithm will search for a matching pixel in the right image. This process involves calculating the difference in pixel values ​​of corresponding blocks in the left and right images. The calculation formula of the block matching algorithm is:

[0098]

[0099] in, represents the left image, represents the right image, is the pixel coordinate, is the parallax. In practical applications, the parallax The search range and step size are important parameters of the algorithm and need to be set according to the specific application scenario and image resolution. Usually, the disparity search range is set to a reasonable value to ensure the accuracy and efficiency of the disparity calculation.

[0100] Preferably, in order to improve the accuracy and robustness of the disparity calculation, the block matching algorithm may be optimized. For example, a multi-scale block matching algorithm may be used to first quickly estimate the disparity at a coarser scale and then perform precise calculation at a finer scale.

[0101] Furthermore, parallax smoothing constraints can be introduced to reduce noise and discontinuity in parallax calculation. These optimization measures help improve the accuracy of parallax estimation, thereby providing more reliable depth information for subsequent 3D reconstruction. In practical applications, the parallax search range and step size can be dynamically adjusted according to the complexity of the image content and the dynamic changes of the scene to achieve the best parallax calculation effect.

[0102] In some embodiments, the calculation formula of the triangulation principle is:

[0103]

[0104] in is the three-dimensional coordinate of the target object, is the baseline distance of the binocular camera, is the camera focal length, is parallax, is the image height, is the pixel coordinate.

[0105] It should be noted that this embodiment describes in detail the process of calculating the three-dimensional coordinates of the target object based on the principle of triangulation. Triangulation is a method that uses geometric principles to determine the position of a point by measuring the viewing angles of a point at both ends of a known baseline. In this embodiment, by combining the baseline distance, focal length, parallax of the binocular camera and the coordinates of the feature points on the image, the accurate position of the target object in three-dimensional space can be calculated.

[0106] Specifically, the calculation of triangulation involves several key parameters: baseline distance (the distance between the binocular cameras), focal length (the distance from the camera lens to the imaging plane), parallax (the horizontal pixel difference between corresponding points in the left and right images), and image height. These parameters together determine the calculation of the three-dimensional coordinates of the target object. The calculation formula is as follows:

[0107]

[0108]

[0109]

[0110] in, , , are the three-dimensional coordinates of the object, is the camera focal length, is parallax, , is the coordinate of the feature point on the image, is the image height, is the baseline distance of the camera. The specific settings of these parameters need to be determined according to the physical characteristics of the camera and the actual shooting conditions.

[0111] Preferably, in order to improve the accuracy of the three-dimensional coordinate calculation, some optimization measures can be adopted. For example, the accuracy of the model can be improved by increasing the number of shooting directions, or a higher-precision camera and calibration equipment can be used to reduce system errors.

[0112] Furthermore, compensation for environmental factors can be introduced, such as the impact of temperature and humidity on camera parameters, and the impact of changes in atmospheric refractive index on parallax. These optimization measures help improve the accuracy of triangulation and thus obtain more accurate three-dimensional coordinates.

[0113] In some embodiments, after calculating the three-dimensional coordinates of the target object in six directions, the step of optimizing the three-dimensional coordinates is further included, specifically:

[0114] An error function based on the least squares method is established, the three-dimensional coordinate data is substituted into it, and the error function is converged to the minimum value through iterative calculation, thereby obtaining the optimized three-dimensional coordinates.

[0115] It should be noted that this embodiment involves the step of optimizing the three-dimensional coordinates after calculating the three-dimensional coordinates of the target object in six directions. Optimization here refers to adjusting the three-dimensional coordinates obtained by preliminary calculation by mathematical methods to improve the accuracy of the model. The least squares method is used in the optimization process, which is a commonly used mathematical optimization technique for finding the best function match for data given a certain error.

[0116] Specifically, in the optimization step, an error function based on the least squares method is established, which measures the difference between the measured value and the theoretical value. The calculation formula of the error function is:

[0117]

[0118] in, , , are the measured three-dimensional coordinates, and , , is the corresponding theoretical value. Through iterative calculation, find the error function The parameter correction value that reaches the minimum value is used to correct the original three-dimensional coordinates.

[0119] Preferably, in order to further improve the efficiency and accuracy of the optimization process, some advanced numerical optimization algorithms such as gradient descent, Newton method or Gauss-Newton method can be used. These methods can converge to the minimum value of the error function faster, especially when dealing with large-scale data sets.

[0120] Furthermore, regularization terms can be introduced to avoid overfitting, or robust optimization methods can be used to reduce the impact of outliers. These alternatives can provide more flexibility and robustness to adapt to different application scenarios and data characteristics.

[0121] In some embodiments, the optimization adopts the least square method, and the calculation process is: let the measured three-dimensional coordinates be The corresponding theoretical value is , error function:

[0122]

[0123] The parameter correction value that minimizes E is found through mathematical methods such as derivation, and the original three-dimensional coordinates are corrected.

[0124] It should be noted that this embodiment describes in detail the three-dimensional coordinate optimization process based on the least squares method. The least squares method is a mathematical optimization technique that finds the best function match for data by minimizing the sum of squares of errors. In this embodiment, the least squares method is used to adjust and optimize the three-dimensional coordinate data obtained from the preliminary measurement to improve the accuracy of the model.

[0125] Specifically, the optimization process first sets the measured three-dimensional coordinates as The corresponding theoretical value is . Error function It is defined as the sum of the squared errors in each coordinate dimension, that is . Through mathematical methods, such as derivation, find The minimum parameter correction value is used to correct the original three-dimensional coordinates. In practical applications, these parameter correction values ​​are calculated through an iterative algorithm to ensure that the error function can converge to the global or local minimum.

[0126] Preferably, in order to improve the efficiency and accuracy of the optimization process, some advanced numerical optimization techniques can be used. For example, a regularization term can be introduced to avoid overfitting, or a robust optimization method can be used to reduce the impact of outliers.

[0127] Furthermore, it is also possible to consider using nonlinear least squares to handle more complex error models, or adopting global optimization algorithms such as genetic algorithms and particle swarm optimization algorithms to find the global optimal solution. These alternatives can provide more flexibility and robustness to adapt to different application scenarios and data characteristics, thereby ensuring that the optimized three-dimensional coordinates more accurately reflect the actual position and shape of the target object.

[0128] In some embodiments, before shooting in six directions in sequence, a step of calibrating the binocular camera is also included to obtain the camera's intrinsic parameter matrix and extrinsic parameter matrix. The specific steps are:

[0129] Set up the calibration board, collect images of the calibration board at different angles and positions, extract the coordinates of the feature points on the calibration board in the image, combine the known coordinates of the feature points in the world coordinate system, and calculate the camera's intrinsic parameter matrix and extrinsic parameter matrix through a specific mathematical model and optimization algorithm.

[0130] It should be noted that this embodiment describes the steps of calibrating a binocular camera to obtain the camera's intrinsic parameter matrix and extrinsic parameter matrix. Camera calibration is an important process in the field of computer vision, which involves determining the camera's intrinsic parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as the camera's position and posture relative to the world coordinate system). The intrinsic and extrinsic matrices are crucial for subsequent image processing and three-dimensional reconstruction because they provide the necessary information for converting image coordinates into world coordinates.

[0131] Specifically, during the calibration process, a calibration plate is first set up, which usually has a known geometric pattern, such as a checkerboard or a dot grid. Then, images of the calibration plate are collected at different angles and positions, and the coordinates of the feature points on the calibration plate in the image are extracted. These coordinates are combined with the known coordinates of the feature points in the world coordinate system, and the intrinsic and extrinsic matrix of the camera is calculated through a specific mathematical model and optimization algorithm.

[0132] More specifically, in actual operation, calibration methods such as Zhang Zhengyou calibration method can be used, which can provide accurate camera parameter estimation. Parameter setting includes selecting a suitable optimization algorithm (such as the least squares method) and adjusting the initial conditions of the algorithm to ensure the accuracy and stability of the calibration process.

[0133] Preferably, in order to improve the accuracy and robustness of calibration, a multi-scale calibration or a dynamic calibration method may be used. Multi-scale calibration takes into account the imaging characteristics of the camera at different distances, while dynamic calibration takes into account the parameter changes of the camera during motion.

[0134] Furthermore, adaptive algorithms can be introduced to automatically adjust the parameters in the calibration process to adapt to different environmental conditions and camera characteristics. These alternatives can provide more flexibility and robustness to adapt to different application scenarios and camera characteristics, thereby ensuring the accuracy and reliability of the calibration results.

[0135] In some embodiments, before feature point extraction and matching, a step of preprocessing each set of image pairs is also included, and the specific steps are:

[0136] Gaussian filtering algorithm is used to denoise the image, and image enhancement is performed by adjusting the contrast and brightness of the image.

[0137] It should be noted that this embodiment involves the step of preprocessing the image captured by the binocular camera to improve the accuracy of subsequent feature point extraction and matching. Preprocessing is a key step in image processing, which includes two main links: image denoising and image enhancement. Image denoising aims to reduce random noise in the image, while image enhancement aims to improve the visual effect of the image, such as contrast and brightness, making the feature points more obvious.

[0138] Specifically, image denoising uses the Gaussian filter algorithm, which is a linear filtering technique that smoothes the image by convolution with a Gaussian kernel. The standard deviation σ of the Gaussian kernel is a key parameter of the algorithm, which determines the degree of filtering. The larger the σ value, the more obvious the filtering effect, but it may cause the loss of image details. Image enhancement adjusts the contrast and brightness of the image through linear transformation. The transformation formula is:

[0139]

[0140] in, is the original image pixel value, is the contrast adjustment factor, is the brightness adjustment coefficient, To enhance the pixel value of the image, the contrast and brightness adjustment needs to be set according to the characteristics and requirements of the actual image to achieve the best visual effect.

[0141] Preferably, in order to further improve the effect of preprocessing, an adaptive Gaussian filter may be used, and its standard deviation σ may be dynamically adjusted according to local statistical characteristics of the image to better preserve image details.

[0142] Furthermore, histogram equalization technology can be introduced into the image enhancement step to automatically adjust the contrast of the image so that the histogram distribution of the image is more uniform, thereby improving the recognition of feature points in the image. These alternatives can provide more flexibility and adaptability to adapt to different image characteristics and application requirements.

[0143] In some embodiments, the preprocessing includes image denoising and image enhancement. Image denoising uses a Gaussian filtering algorithm, and the filtering formula is:

[0144]

[0145] in, is the standard deviation, and the noise is removed by convolution operation with the image pixels; image enhancement adjusts the contrast and brightness of the image through linear transformation, and the transformation formula is:

[0146]

[0147] in is the contrast adjustment factor, is the brightness adjustment coefficient, is the original image pixel value, is the pixel value of the enhanced image.

[0148] It should be noted that this embodiment describes the steps of preprocessing the image before feature point extraction and matching, including image denoising and image enhancement. Image denoising refers to reducing or eliminating unnecessary noise in the image, while image enhancement improves the visual effect of the image, such as increasing the contrast and adjusting the brightness to make the feature points more obvious. Gaussian filtering is a commonly used image denoising method, which smoothes the image by convolving the Gaussian kernel with the image; while image enhancement can adjust the contrast and brightness of the image through linear transformation.

[0149] Specifically, the image denoising adopts the Gaussian filtering algorithm, and its formula is:

[0150]

[0151] in, is the standard deviation, which controls the width of the Gaussian kernel. In practical applications, The value needs to be set according to the specific situation of the image. The value will make the filtering effect more obvious, but may cause the loss of image details. Image enhancement adjusts the contrast and brightness of the image through linear transformation. The transformation formula is:

[0152]

[0153] in, is the contrast adjustment factor, is the brightness adjustment coefficient, is the original image pixel value, To enhance the pixel value of the image, the contrast and brightness adjustment needs to be set according to the characteristics and requirements of the actual image to achieve the best visual effect.

[0154] Preferably, in order to further improve the effect of preprocessing, an adaptive Gaussian filter can be used, and its standard deviation It can be dynamically adjusted according to the local statistical characteristics of the image to better preserve image details.

[0155] Furthermore, histogram equalization technology can be introduced into the image enhancement step to automatically adjust the contrast of the image so that the histogram distribution of the image is more uniform, thereby improving the recognition of feature points in the image. These alternatives can provide more flexibility and adaptability to adapt to different image characteristics and application requirements.

[0156] The above-mentioned embodiments of the present invention have the following beneficial effects: The three-dimensional modeling method based on binocular vision described in the present invention can provide an efficient and accurate means of acquiring three-dimensional information. By systematically photographing the target object in six directions, combined with the principles of parallax calculation and triangulation, the method can accurately reconstruct the three-dimensional coordinates of the object, thereby constructing a complete three-dimensional model. This method can be widely used in industrial inspection, virtual reality, augmented reality and other fields, providing high-quality three-dimensional data support for these fields.

[0157] In addition, the steps of preprocessing, feature point extraction and matching, disparity calculation, and three-dimensional coordinate optimization in this method can improve the robustness and accuracy of the modeling process. In particular, the use of Gaussian kernels of different scales and the nearest neighbor matching algorithm based on Euclidean distance in feature point extraction can enhance the stability of feature points and the accuracy of matching. The block matching algorithm used in disparity calculation can further ensure the reliability of disparity estimation. And by optimizing the three-dimensional coordinates through the least squares method, the error can be reduced and the accuracy of the model can be improved. The combined use of these technologies can not only improve the efficiency of three-dimensional modeling, but also ensure that high-quality three-dimensional models can be obtained even in complex environments.

[0158] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0159] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A three-dimensional modeling method based on binocular vision, characterized in that: The following steps are involved: Select a positioning point, take the positioning point as the center, use the binocular camera to shoot in six directions in sequence, and obtain six sets of image pairs; Calculate disparity based on matched feature points; Using the camera's intrinsic matrix, extrinsic matrix and parallax, the three-dimensional coordinates of the target object in six directions are calculated through the principle of triangulation; Integrate the three-dimensional coordinates in six directions to construct a three-dimensional model of the target object.

2. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: The feature point extraction comprises: The scale space is constructed by convolving the Gaussian kernels of different scales with the image, and the local extreme points are detected in different scale spaces. The scale space formula is: ； in, For images, is the scale factor; Locate the detected extreme points and remove low-contrast points and unstable edge response points; Each key point is assigned a main direction, which is determined based on the gradient direction distribution of the pixels in the key point neighborhood. The gradient amplitude formula is: ； The gradient direction formula is: ； Generate key point descriptors for feature point matching.

3. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: The feature point matching adopts the nearest neighbor matching algorithm based on Euclidean distance, including: For a feature point, calculate the Euclidean distance between it and all feature points in another image. The Euclidean distance formula is: ； in, and are the coordinates of the two feature points; Find the nearest neighbor and the next nearest neighbor feature point. If the ratio of the nearest neighbor distance to the next nearest neighbor distance is less than the preset value, the two feature points are matched successfully.

4. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: The disparity is calculated based on the matched feature points. The calculation formula is: ； in, is the left image, For the right image, is the pixel coordinate, For parallax.

5. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: The three-dimensional coordinates of the target object in six directions are calculated by the triangulation principle. The calculation formula is: ； in is the three-dimensional coordinate of the target object, is the baseline distance of the binocular camera, is the camera focal length, is parallax, is the image height, is the pixel coordinate.

6. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: After calculating the three-dimensional coordinates of the target object in six directions, the three-dimensional coordinates are also optimized, including: An error function based on the least squares method is established, the three-dimensional coordinate data is substituted into it, and the error function is converged to the minimum value through iterative calculation to obtain the optimized three-dimensional coordinates.

7. The three-dimensional modeling method based on binocular vision according to claim 6, characterized in that: An error function based on the least squares method is established, and the three-dimensional coordinate data is substituted into it. The error function is converged to the minimum value through iterative calculation, and the optimized three-dimensional coordinates are obtained, including: Assume the measured three-dimensional coordinates are The corresponding theoretical value is , the error function is: ； The parameter correction value that minimizes the error E is found by derivation, and the original three-dimensional coordinates are corrected.

8. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: Before shooting in six directions in sequence, the binocular camera is calibrated to obtain the camera's intrinsic and extrinsic matrix, including: Set up the calibration board, collect images of the calibration board at different angles and positions, extract the coordinates of the feature points on the calibration board in the image, and calculate the intrinsic parameter matrix and extrinsic parameter matrix of the camera based on the known coordinates of the feature points in the world coordinate system.

9. The three-dimensional modeling method based on binocular vision according to claim 1, characterized in that: The method further comprises: preprocessing each set of image pairs, comprising: Gaussian filtering algorithm is used to denoise the image, and image enhancement is performed by adjusting the contrast and brightness of the image.

10. The three-dimensional modeling method based on binocular vision according to claim 9, characterized in that: The Gaussian filtering algorithm is used for image denoising, and the image is enhanced by adjusting the contrast and brightness of the image. The filtering formula is: ； in, is the standard deviation, and the noise is removed by convolution operation with the image pixels; image enhancement adjusts the contrast and brightness of the image through linear transformation, and the transformation formula is: ； in is the contrast adjustment factor, is the brightness adjustment coefficient, is the original image pixel value, is the pixel value of the enhanced image.

Images