A Machine Learning-Based Method for Estimating Array Encoding Depth

Abstract

This invention relates to a machine learning-based array coding depth estimation method, belonging to the fields of machine learning and computer vision. Based on the fundamental principles of array coding, this invention designs array coding patterns and formulates a method for accurately representing the sub-pixel coordinates of primitive patterns. It classifies and predicts primitive patterns using a decision tree, training the decision tree with a white balance board through machine learning to obtain a satisfactory decision tree. The decision tree is used to obtain the classification of primitive patterns in the actual acquired image, thereby obtaining the sub-pixel positioning coordinates of the primitive patterns. Depth information is solved based on the principle of triangulation, and finally, the obtained depth information is converted into a 3D point cloud information map. This invention is simple to implement, has high accuracy, and meets the depth estimation requirements for dynamic targets.

Description

Technical Field

[0001] This invention belongs to the fields of machine learning and computer vision, and specifically relates to a machine learning-based array coding depth estimation method. Background Technology

[0002] With the development of machine learning and computer vision, many feature matching algorithms have been proposed for depth estimation. When the target is stationary, the array-coded primitive patterns are relatively clear in the camera's field of view. However, due to the defocusing effect, the array-coded primitive patterns in local areas are blurred. When the moving target moves slowly relative to the camera, local areas will experience line-drawing errors. Blur caused by defocusing or motion will lead to a deviation in the center position of the image during decoding, reducing the accuracy of depth estimation. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] The technical problem to be solved by this invention is how to provide an array coding depth estimation method based on machine learning to solve the problem that blur caused by defocusing or motion will cause the center position of the image to deviate during decoding, thus reducing the accuracy of depth estimation.

[0005] (II) Technical Solution

[0006] To address the aforementioned technical problems, this invention proposes a machine learning-based array encoding depth estimation method, which includes the following steps:

[0007] S1. Based on the basic principle of array coding, design an array coding pattern and formulate a method for representing the sub-pixel coordinates of the primitive pattern position accurately.

[0008] S2. Based on the decision tree, classify and predict primitive patterns. The decision tree is trained using a white balance board through machine learning to obtain a decision tree that meets the requirements.

[0009] S3. Use a decision tree to obtain the classification of primitive patterns in the actual acquired images, and then obtain the sub-pixel positioning coordinates of the primitive patterns.

[0010] S4. Encoding Depth Estimation: Based on the principle of triangulation, depth information is solved, and finally the obtained depth information is transformed into a three-dimensional point cloud information map.

[0011] (III) Beneficial Effects

[0012] This invention proposes a machine learning-based array coding depth estimation method. The accuracy of the depth estimation algorithm is tested using plane fitting with a white balance board. Results show high accuracy and low error. Experimental results demonstrate that this encoding and decoding algorithm achieves a 0.5mm accuracy error in dynamic target measurement, which is excellent. It basically meets the depth estimation requirements for dynamic targets. This invention is simple to implement, highly accurate, and meets application requirements. Attached Figure Description

[0013] Figure 1 This is the array coding pattern of the present invention;

[0014] Figure 2 This is a subpixel segmentation diagram of the center of the basic pattern of the present invention;

[0015] Figure 3 This is a subpixel positioning map of the basic unit center of this invention. Detailed Implementation

[0016] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0017] This invention proposes an algorithm based on machine learning decision tree classification to calculate the center coordinates of array-coded primitive patterns. Utilizing the fuzzy features of array-coded primitive patterns, this invention employs a decision tree classification registration method to design a sub-pixel decoding method for array-coded primitive patterns in fuzzy states. Image feature matching is a method for searching for similar image targets by analyzing the correspondence, similarity, and consistency of image features, structures, relationships, textures, and grayscale values. In the array-coded primitive pattern feature matching algorithm, sub-pixel features of array primitives are obtained based on a decision tree (DT), transforming the problem of calculating the center coordinates of the feature matching primitives into a feature classification problem. The proposed machine learning-based sparse depth estimation method for array coding can be used for both sparse depth estimation of moving targets and optimization of the coding pattern design for stationary targets.

[0018] This invention primarily focuses on a machine learning-based array coding depth estimation method. Specifically, it investigates array coding methods for depth estimation of the target object, designing array coding patterns based on the fundamental principles of array coding. The invention also proposes a sub-pixel feature analysis method for array coding primitive patterns and a training data generation method for sub-pixel feature matching of primitive patterns. Finally, it utilizes decision trees to perform sub-pixel feature decoding calculations for array primitives. For moving targets, the depth estimation method proposed in this invention demonstrates superior accuracy compared to other methods, while also exhibiting considerable speed. It effectively addresses the issue of image center position deviation caused by blurring due to defocusing or motion, which reduces depth estimation accuracy.

[0019] The machine learning-based array encoding depth estimation method of the present invention specifically includes:

[0020] S1. Based on the basic principle of array coding, design an array coding pattern and formulate a method for representing the sub-pixel coordinates of the primitive pattern position accurately.

[0021] S2. Based on the decision tree, classify and predict primitive patterns. The decision tree is trained using a white balance board through machine learning to obtain a decision tree that meets the requirements.

[0022] S3. Use a decision tree to obtain the classification of primitive patterns in the actual acquired images, and then obtain the sub-pixel positioning coordinates of the primitive patterns.

[0023] S4. Encoding Depth Estimation: Based on the principle of triangulation, depth information is solved, and finally the obtained depth information is transformed into a three-dimensional point cloud information map.

[0024] This invention designs an array coding pattern and a sub-pixel value coordinate representation method to improve the accuracy of depth estimation. Specifically, the array coding depth estimation is based on machine learning. The first step is to create and generate machine learning training data based on the designed sub-pixel feature primitive pattern; the second step is sub-pixel feature primitive pattern matching based on the decision tree method; and the third step is to obtain the depth estimate. Example

[0025] Figure 1This is the array encoding diagram of the present invention. In the array encoding pattern design, each primitive pattern has different characteristics. The primitive pattern in the array encoding pattern of the present invention adopts a four-directional hourglass encoding pattern. The primitive pattern is designed based on a single color, resulting in small decoding errors. The four-directional hourglass encoding pattern has directionality, providing obvious matching features. The four-directional hourglass encoding pattern includes four types: the first type is a vertical hourglass, corresponding to the encoded number 0; the second type is an hourglass rotated 45 degrees clockwise, corresponding to the encoded number 1; the third type is an hourglass rotated 90 degrees clockwise, corresponding to the encoded number 2; and the fourth type is an hourglass rotated 135 degrees clockwise, corresponding to the encoded number 3.

[0026] Two images captured by two cameras targeting the same object can be obtained after preprocessing. Figure 1 The array encoding pattern is only related to subsequent processing in this invention, and does not involve the preprocessing process.

[0027] Figure 2 This is a subpixel segmentation map of the center of the primitive pattern in this invention. In the process of matching the similarity of two images of the same object captured by two cameras, the coordinates of the center of the primitive pattern are used as the basis for matching calculation. Therefore, each primitive pattern can be uniquely represented, and its machine learning matching label can be represented as... When the horizontal and vertical coordinates of the center of the primitive pattern and When known, its pixel coordinates can be expressed as Therefore, high-precision primitive center localization is the foundation of high-precision depth sensing. This invention defines the coordinates of primitives as the sub-pixel value of the primitive's center rather than an integer value, thus calculating more accurate depth information. For example... Figure 2 As shown, the left side is a pixel-level primitive pattern, and the right side is a sub-pixel-level primitive pattern.

[0028] Therefore, this invention subdivides the x and y coordinates of a pixel into 10 parts each. Using the above method, sub-pixel coordinates accurately representing the position of the primitive pattern can be obtained. An offset range can be set based on the offset of the primitive pattern, which reduces computational variables. Therefore, it is necessary to set the maximum value of the primitive pattern's offset in the x-direction. Figure 2 In this embodiment, the offset is set to 7 pixels. In other words, the maximum offset is 7 pixels in the epipolar direction of the same matching point corresponding to two acquired images. Therefore, based on pixel-level positioning of array-coded pattern primitive pattern matching, each pixel is further subdivided into 10×10 subpixels within the 7×7 pixel neighborhood of the center coordinate of each primitive pattern positioning, that is, divided into 70×70 parts.

[0029] A decision tree is a deep classifier that can predict each attribute S of a primitive pattern, including position and orientation. Therefore, this invention constructs a primitive pattern row vector classifier based on the row vectors of the primitive pattern to classify the primitive patterns. Since decision trees perform well in training mixed targets, the inputs in the proposed method are... , and ,in Given a set of primitive patterns in the input image. This is a collection of primitive patterns; here, it represents four types of hourglass images. Let be the set of classification targets, here 0, 1, 2, 3. Therefore, the learning objective of the decision tree is... arrive The mapping relationship. The probability distribution function in the decision tree can be expressed as follows:

[0030]

[0031] Therefore, the attribute prediction classification function of the primitive pattern can be expressed as follows:

[0032] (1)

[0033] in, Represents a primitive pattern, which is usually a set of primitive patterns from the input image. Basic pattern Here I am Length in the axial direction, Basic pattern In the y-axis direction, θ is the direction of the primitive pattern, and x and y are the pixel-level offsets of the image. L , R It is a classifier for decision trees, which is a predictive model of an attribute structure. When designing a decision tree: determine the category according to formula (1), and judge one category at a time. First, determine whether it is class 0. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 1. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 2. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 3. If it is, it is located on the left branch L. If not, it is another abnormal class that does not belong to class 0-3. Each class corresponds to a different primitive pattern direction θ.

[0034] In one embodiment of the present invention, the classification target is refined into 100 parts, which is a 10×10 subpixel subdivision of a single pixel, thus achieving a positioning accuracy of 0.1 pixels. Once the mapping relationship between the primitive pattern and the classification target is obtained, the subpixel center coordinates of each primitive pattern can be calculated. Therefore, the learning parameters of the decision tree can be expressed as...

[0035]

[0036] Where the coordinates x and y represent the pixel-level offset of the image, and the offset is related to the parameter Related.

[0037] Figure 3 This is the subpixel localization map of the primitive center in this invention. Training the decision tree using a white balance board via machine learning includes: training and measuring using a moving white balance board; preprocessing two images acquired by two cameras to obtain an array-coded pattern; then obtaining the type and subpixel localization of the primitive pattern based on the decision tree; comparing this subpixel localization with the pre-designed subpixel localization of the primitive pattern center on the white balance board and modifying the decision tree parameters accordingly until a satisfactory decision tree is obtained. Figure 3 The image shows the experimental results of subpixel positioning of the center of the white balance board's primitive pattern. As can be seen from the image, the calculated center position matches the pre-designed actual center position of the white balance board. Figure 3 As shown, the center position of the first type of hourglass is indicated by a green dot, the center position of the second type of hourglass is indicated by a yellow dot, the center position of the third type of hourglass is indicated by a red dot, and the center position of the fourth type of hourglass is indicated by a black dot.

[0038] Based on machine learning, primitive pattern localization within an array-coded pattern can achieve depth estimation for a slowly moving target. To test the accuracy of depth estimation using a decision tree based on the array-coded pattern, this invention trains and measures using a moving white balance board. The accuracy of the depth estimation is determined by the measured plane fitting error of the white balance board. Specifically, first, the type and sub-pixel localization of the primitive pattern are obtained based on the decision tree. Then, depth information is solved based on the triangulation principle. Finally, the obtained depth information is converted into a 3D point cloud image, and the accuracy of the depth estimation is determined by the measured plane fitting error of the white balance board.

[0039] This invention employs the array-encoded depth estimation method based on machine learning. In terms of depth estimation accuracy experiments, the algorithm's precision was tested using plane fitting with a white balance board. Results show high accuracy and low error. Experimental results demonstrate that this encoding and decoding algorithm achieves a 0.5mm accuracy error in dynamic target measurement, demonstrating excellent performance. It essentially meets the depth estimation requirements for dynamic targets. This invention is simple to implement, highly accurate, and meets application requirements.

[0040] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A machine learning-based method for estimating the depth of array encoding, characterized in that, The method includes the following steps: S1. Based on the basic principle of array coding, design an array coding pattern and formulate a method for representing the sub-pixel coordinates of the primitive pattern position accurately. S2. Based on the decision tree, classify and predict primitive patterns. The decision tree is trained using a white balance board through machine learning to obtain a decision tree that meets the requirements. S3. Use a decision tree to obtain the classification of primitive patterns in the actual acquired images, and then obtain the sub-pixel positioning coordinates of the primitive patterns. S4. Encoding Depth Estimation: Based on the principle of triangulation, depth information is solved, and finally the obtained depth information is converted into a three-dimensional point cloud information map; in, In step S1, a method for accurately representing the subpixel coordinates of the primitive pattern position is established: the x and y coordinates of a pixel are each subdivided into 10 parts, and the subpixel coordinates accurately representing the primitive pattern position are obtained through the above method; an offset range is set according to the offset of the primitive pattern. Setting the offset range can reduce the calculation variables, so it is necessary to set the maximum value of the primitive pattern offset in the x direction; in the epipolar direction of the same matching point corresponding to two acquired images, the maximum offset is 7 pixels. Therefore, based on the pixel-level positioning of primitive pattern matching based on array-coded pattern, within the 7×7 pixel neighborhood of the center coordinate of each primitive pattern positioning, each pixel is further subdivided into 10×10 subpixels, that is, divided into 70×70 parts.

2. The array encoding depth estimation method based on machine learning as described in claim 1, characterized in that, In S1, the primitive pattern in the array coding pattern adopts the four-way hourglass coding pattern. The four-way hourglass coding pattern has directionality and provides obvious matching features. The four-way hourglass coding pattern includes four types: the first type is a vertical hourglass, and the corresponding coding number is 0; the second type is an hourglass rotated 45 degrees clockwise, and the corresponding coding number is 1; the third type is an hourglass rotated 90 degrees clockwise, and the corresponding coding number is 2; the fourth type is an hourglass rotated 135 degrees clockwise, and the corresponding coding number is 3.

3. The array encoding depth estimation method based on machine learning as described in claim 2, characterized in that, The basic pattern is designed based on a single color.

4. The array encoding depth estimation method based on machine learning as described in claim 2, characterized in that, Each primitive pattern is uniquely represented, and its machine learning matching label is represented as p0 = (x, y). When the horizontal and vertical coordinates x′ and y′ of the center of the primitive pattern are known, its pixel coordinates are represented as p′0 = (x′, y′).

5. The array encoding depth estimation method based on machine learning as described in claim 2, characterized in that, In step S2, the decision tree is a deep classifier used to predict each attribute S of the primitive pattern. Attribute S includes: position and orientation. A primitive pattern row vector classifier is constructed using the row vectors of the primitive pattern as units to classify the primitive pattern. The inputs to the decision tree are I and s, respectively. c and s p Where I is the set of primitive patterns in the input image, and s c This is a collection of primitive patterns; here, it represents four types of hourglass images, s p Let be the set of classification targets, here 0, 1, 2, 3; therefore, the learning objective of the decision tree is s. c to s p The mapping relationship.

6. The array encoding depth estimation method based on machine learning as described in claim 2, characterized in that, The attribute prediction classification function for primitive patterns is expressed as follows: Where I represents the primitive pattern, which is the set of primitive patterns of the input image; X(s) c ) is the basic pattern s c The length of Y(s) along the x-axis direction c ) is the basic pattern s c In the y-axis direction, θ is the direction of the primitive pattern, x and y are the pixel-level offsets of the image; L and R are the classifiers of the decision tree, which is a prediction model of an attribute structure. When designing the decision tree: determine the category according to formula (1), and judge one category at a time. First, determine whether it is class 0. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 1. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 2. If it is, it is located on the left branch L. If not, it is another class and is located on the right branch R. Then determine whether it is class 3. If it is, it is located on the left branch L. If not, it is another abnormal class that does not belong to class 0-3. Each class corresponds to a different primitive pattern direction θ.

7. The array encoding depth estimation method based on machine learning as described in claim 6, characterized in that, The learning parameters of a decision tree are represented as follows: S = (x, y, θ) The coordinates x and y represent the pixel-level offset of the image, and the offset is related to the parameter θ.

8. The array encoding depth estimation method based on machine learning as described in claim 7, characterized in that, In step S2, training the decision tree using a white balance board through machine learning includes: training and measuring using a moving white balance board through machine learning; acquiring two images from two cameras and preprocessing them to obtain an array-coded pattern; then obtaining the type and sub-pixel positioning of the primitive pattern based on the decision tree; comparing it with the sub-pixel positioning of the center of the primitive pattern pre-designed by the white balance board and providing feedback to modify the decision tree parameters until a decision tree that meets the requirements is obtained.

9. The array encoding depth estimation method based on machine learning as described in claim 8, characterized in that, The method also includes testing the depth estimation accuracy based on array-coded pattern decision trees: training and measurement are performed using a moving white balance board, and the accuracy of the depth estimation is judged by the measured plane fitting error of the white balance board. Specifically, firstly, the type of primitive pattern and sub-pixel localization are obtained based on the decision tree, then the depth information is solved based on the triangulation principle, and finally the obtained depth information is converted into a three-dimensional point cloud information map, and the accuracy of the depth estimation is judged by the measured plane fitting error of the white balance board.

Images