Array coding depth estimation method based on machine learning

Abstract

The invention relates to an array coding depth estimation method based on machine learning, and belongs to the field of machine learning and computer vision. On the basis of the basic principle of array coding, an array coding pattern is designed, and a sub-pixel value coordinate representation method for accurately representing the position of an element pattern is formulated; performing classification prediction on the primitive patterns based on a decision tree, and training the decision tree by using a white balance plate in a machine learning mode to obtain a decision tree meeting requirements; acquiring classification of primitive patterns in an actually acquired image by using a decision tree, and further acquiring sub-pixel positioning coordinates of the primitive patterns; and depth information is solved based on a triangulation principle, and finally the obtained depth information is converted into a three-dimensional point cloud information graph. The method is easy to implement and high in precision, and meets the depth estimation requirement of the dynamic target.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning and computer vision, and in particular relates to an array coding depth estimation method based on machine learning. Background Art

[0002] Due to the development of machine learning and computer vision, many feature matching algorithms have been proposed to estimate depth. When the target is stationary, the primitive patterns of the array code are relatively clear in the camera field of view, but due to the defocus effect, the primitive patterns of the array code in the local area are blurred. When the moving target moves slowly relative to the camera, a wire drawing error will occur in the local area. The blur caused by defocus or motion will cause the deviation of the center position of the image in decoding, reducing the accuracy of depth estimation. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is how to provide an array coding depth estimation method based on machine learning to solve the problem that blur caused by defocus or motion will cause deviation of the center position of the image during decoding, thereby reducing the accuracy of depth estimation.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention proposes an array coding depth estimation method based on machine learning, which comprises the following steps:

[0007] S1. Based on the basic principle of array coding, design array coding patterns and formulate a sub-pixel value coordinate representation method that accurately represents the position of the primitive pattern;

[0008] S2. Classify and predict the primitive pattern based on the decision tree, and train the decision tree using the white balance board through machine learning to obtain a decision tree that meets the requirements;

[0009] S3, using a decision tree to obtain the classification of the primitive pattern in the actually collected image, and then obtaining the sub-pixel positioning coordinates of the primitive pattern;

[0010] S4. Encoded depth estimation: Solve the depth information based on the principle of triangulation, and finally convert the obtained depth information into a three-dimensional point cloud information map.

[0011] (III) Beneficial effects

[0012] The present invention proposes an array coding depth estimation method based on machine learning. The array coding depth estimation method based on machine learning of the present invention is used. In the accuracy experiment of depth estimation, the plane fitting of the white balance board is used to test the accuracy of the depth estimation algorithm. The results show that the accuracy is high and the error is small. According to the experimental results, after adopting the encoding and decoding algorithm, an accuracy error of 0.5mm can be achieved in dynamic target measurement, and the effect is very good. The depth estimation requirements of dynamic targets are basically met. The present invention is simple to implement and has high accuracy, meeting the application requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The array coding pattern of the present invention;

[0014] Figure 2 It is the central sub-pixel segmentation map of the primitive pattern of the present invention;

[0015] Figure 3 It is a sub-pixel positioning map of the center of the primitive of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0017] The present invention proposes an algorithm based on machine learning decision tree classification to calculate the center coordinates of the primitive pattern of array coding. The present invention utilizes the fuzzy features of the primitive pattern of array coding and adopts the decision tree classification and registration method to design a sub-pixel decoding method of the primitive pattern of array coding in a fuzzy state. Image feature matching is a method of searching for similar image targets by analyzing the correspondence, similarity and consistency of image features, structures, relationships, textures and grayscale. In the primitive pattern feature matching algorithm of array coding, the sub-pixel features of the array primitives are obtained based on the decision tree (Decisiontree, DT), and the feature matching calculation center problem is converted into a feature classification problem of the center coordinates. The sparse depth estimation method of array coding based on machine learning proposed by the present invention can be used for sparse depth estimation of moving targets and can also optimize the coding pattern design of stationary targets.

[0018] The present invention mainly studies the array coding depth estimation method based on machine learning. The present invention studies the array coding method for depth estimation of the measured target, designs the array coding pattern based on the basic principle of array coding; studies the array coding depth estimation based on machine learning, proposes a sub-pixel feature analysis method for array coding primitive patterns, and proposes training data generation for sub-pixel feature matching of primitive patterns, and implements sub-pixel feature decoding calculation of array primitives based on decision trees. For moving measured targets, the depth estimation method proposed by the present invention is more accurate than the calculation accuracy of other methods, and its speed is also considerable. It can effectively solve the defect that the blur caused by defocus or motion will cause the deviation of the center position of the image in decoding, reducing the accuracy of depth estimation.

[0019] The array coding depth estimation method based on machine learning of the present invention specifically includes:

[0020] S1. Based on the basic principle of array coding, design array coding patterns and formulate a sub-pixel value coordinate representation method that accurately represents the position of the primitive pattern;

[0021] S2. Classify and predict the primitive pattern based on the decision tree, and train the decision tree using the white balance board through machine learning to obtain a decision tree that meets the requirements;

[0022] S3, using a decision tree to obtain the classification of the primitive pattern in the actually collected image, and then obtaining the sub-pixel positioning coordinates of the primitive pattern;

[0023] S4. Encoded depth estimation: Solve the depth information based on the principle of triangulation, and finally convert the obtained depth information into a three-dimensional point cloud information map.

[0024] The present invention designs an array coding pattern and a sub-pixel value coordinate representation method to improve the accuracy of depth estimation. Among them, array coding depth estimation is performed based on machine learning. The first step is to produce and generate machine learning training data based on the designed sub-pixel feature primitive pattern; the second step is sub-pixel feature primitive pattern matching based on the decision tree method; and the third step is to obtain depth estimation.

[0025] Embodiment 1:

[0026] Figure 1It is the array coding diagram of the present invention; in the array coding pattern design, each primitive pattern has different characteristics, and the primitive pattern in the array coding pattern of the present invention adopts a four-direction hourglass coding pattern. Among them, the primitive pattern is designed based on a single color, and the decoding error is small; the four-direction hourglass coding pattern has directionality and provides obvious matching features. The four-direction hourglass coding pattern includes four types, the first type is an hourglass in the vertical direction, and the corresponding coding number is 0, the second type is the first type of hourglass rotated 45 degrees clockwise, and the corresponding coding number is 1, the third type is the first type of hourglass rotated 90 degrees clockwise, and the corresponding coding number is 2, and the fourth type is the first type of hourglass rotated 135 degrees clockwise, and the corresponding coding number is 3.

[0027] Two images of the same object captured by two cameras can be obtained after preliminary preprocessing. Figure 1 The present invention only focuses on the subsequent processing and does not involve the pre-processing process.

[0028] Figure 2 It is the sub-pixel segmentation map of the center of the primitive pattern of the present invention; in the process of matching similarity of two images captured by two cameras for the same object, the coordinates of the center of the primitive pattern are used as the basis for matching calculation. Therefore, each primitive pattern can be expressed uniquely, and its machine learning matching label can be expressed as p0=(x,y). When the horizontal and vertical coordinates x′ and y′ of the center of the primitive pattern are known, its pixel coordinates can be expressed as p′0=(x′,y′). Therefore, high-precision primitive center positioning is the basis of high-precision depth perception. The present invention defines the coordinates of the primitive as the central sub-pixel value of the primitive rather than an integer value, so that the calculated depth information is more accurate. Figure 2 As shown, (a) is a pixel-level primitive pattern, and (b) is a sub-pixel-level primitive pattern.

[0029] Therefore, the present invention divides the x and y coordinates of a pixel into 10 parts each. Through the above method, the sub-pixel value coordinates that accurately represent the position of the primitive pattern can be obtained. An offset range can be set according to the offset of the primitive pattern, and setting the offset range can reduce the calculation variables. Therefore, it is necessary to set the maximum value of the offset of the primitive pattern in the x direction. Figure 2 In the embodiment of , it is set to 7 pixels. In other words, in the epipolar direction of the same matching point corresponding to the two collected images, the maximum offset is 7 pixels. Therefore, based on the pixel-level positioning based on the primitive pattern matching of the array coding pattern, within the 7×7 pixel neighborhood of the center coordinates of each primitive pattern positioning, each pixel is further subdivided into 10×10 sub-pixels, that is, divided into 70×70 parts.

[0030] The decision tree is a deep classifier that can predict each attribute S of the primitive pattern, including position and direction. Therefore, the present invention uses the row vector of the primitive pattern as a unit to construct a primitive pattern row vector classifier to achieve primitive pattern classification. Since the decision tree has a good performance in the training of mixed targets, the inputs in the method proposed by the present invention are I, s c and p , where I is the set of primitive patterns in the input image, s c is a set of primitive patterns, here are four types of hourglass images, s p is the set of classification targets, here 0, 1, 2, 3. Therefore, the learning target of the decision tree is s c to p The probability distribution function in the decision tree can be expressed as follows:

[0031] p(s p |s c ,I)

[0032] Therefore, the attribute prediction classification function of the primitive pattern can be expressed as follows:

[0033]

[0034] Where I represents a primitive pattern, which is usually a set of primitive patterns of the input image. c ) is the primitive pattern s c The length in the x-axis direction, Y(s c ) is the primitive pattern s c The length in the y-axis direction, θ is the direction of the primitive pattern, and x, y are the pixel-level offsets of the image. L, R are the classification setters of the decision tree, which is a prediction model of an attribute structure. When designing the decision tree: determine the category according to formula (1), and judge one category at a time. First, determine whether it is category 0. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 1. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 2. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 3. If so, it is located on the left branch L. If not, it is other abnormal categories that do not belong to category 0-3. Each category corresponds to a different primitive pattern direction θ.

[0035] In one embodiment of the present invention, the classification target is subdivided into 100 parts, that is, 10×10 sub-pixel subdivisions of a single pixel, so the positioning accuracy is 0.1 pixel. After the mapping relationship between the primitive pattern and the classification target is obtained, the sub-pixel center coordinates of each primitive pattern can be calculated. Therefore, the learning parameters of the decision tree can be expressed as

[0036] S=(x,y,θ)

[0037] The coordinates x, y are the pixel-level offsets of the image, and the offset is related to the parameter θ.

[0038] Figure 3 The sub-pixel positioning diagram of the center of the primitive of the present invention. The decision tree is trained by using a white balance board in a machine learning manner, including: training and measuring with a moving white balance board in a machine learning manner, two cameras collect two images and perform pre-processing to obtain an array coding pattern, and then obtain the type and sub-pixel positioning of the primitive pattern based on the decision tree, and then compare it with the sub-pixel positioning of the center of the primitive pattern pre-designed by the white balance board and feedback to modify the decision tree parameters until a decision tree that meets the requirements is obtained. Figure 3 The figure shows the experimental results of sub-pixel positioning of the center of the primitive pattern of the white balance board. It can be seen from the figure that the center position obtained by calculation is consistent with the actual center position pre-designed for the white balance board. Figure 3 As shown, for the first type of hourglass, the center position is represented by a green dot, for the second type of hourglass, the center position is represented by a yellow dot, for the third type of hourglass, the center position is represented by a red dot, and for the fourth type of hourglass, the center position is represented by a black dot.

[0039] Based on machine learning, the primitive pattern positioning in the array coding pattern can realize the depth estimation of the slow-moving target. In order to test the depth estimation accuracy based on the array coding pattern decision tree, the present invention uses a moving white balance board for training and measurement, and the accuracy of the depth estimation can be judged by the measured white balance board plane fitting error. Specifically, the type and sub-pixel positioning of the primitive pattern are first obtained based on the decision tree, and then the depth information is solved based on the triangulation principle. Finally, the obtained depth information is converted into a three-dimensional point cloud information map, and the accuracy of the depth estimation is judged by the measured white balance board plane fitting error.

[0040] The present invention adopts the array coding depth estimation method of machine learning. In the accuracy experiment of depth estimation, the plane fitting of the white balance board is used to test the accuracy of the depth estimation algorithm. The results show that the accuracy is high and the error is small. From the results of the experiment, after adopting the encoding and decoding algorithm, an accuracy error of 0.5mm can be achieved in dynamic target measurement, and the effect is very good. The depth estimation requirements of dynamic targets are basically met. The present invention is simple to implement and has high accuracy, meeting the application requirements.

[0041] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for estimating array coding depth based on machine learning, characterized in that: The method comprises the following steps: S1. Based on the basic principle of array coding, design array coding patterns and formulate a sub-pixel value coordinate representation method that accurately represents the position of the primitive pattern; S2. Classify and predict the primitive pattern based on the decision tree, and train the decision tree using the white balance board through machine learning to obtain a decision tree that meets the requirements; S3, using a decision tree to obtain the classification of the primitive pattern in the actually collected image, and then obtaining the sub-pixel positioning coordinates of the primitive pattern; S4. Encoded depth estimation: Solve the depth information based on the principle of triangulation, and finally convert the obtained depth information into a three-dimensional point cloud information map.

2. The array coding depth estimation method based on machine learning according to claim 1, characterized in that: In the S1, the primitive pattern in the array coding pattern adopts a four-directional hourglass coding pattern. The four-directional hourglass coding pattern is directional and provides obvious matching features. The four-directional hourglass coding pattern includes four types: the first type is an hourglass in the vertical direction, and the corresponding coding number is 0; the second type is the first type of hourglass rotated 45 degrees clockwise, and the corresponding coding number is 1; the third type is the first type of hourglass rotated 90 degrees clockwise, and the corresponding coding number is 2; the fourth type is the first type of hourglass rotated 135 degrees clockwise, and the corresponding coding number is 3.

3. The array coding depth estimation method based on machine learning as claimed in claim 2, characterized in that: Primitive patterns are designed based on a single color.

4. The array coding depth estimation method based on machine learning as claimed in claim 2, characterized in that: Each primitive pattern is uniquely expressed, and its machine learning matching label is represented by p0=(x,y). When the horizontal and vertical coordinates x′ and y′ of the center of the primitive pattern are known, its pixel coordinates are represented by p′0=(x′,y′.

5. The array coding depth estimation method based on machine learning as claimed in claim 2, characterized in that: In the S1, a sub-pixel value coordinate representation method for accurately representing the position of the primitive pattern is formulated: the x and y coordinates of a pixel are each subdivided into 10 parts, and the sub-pixel value coordinates that accurately represent the position of the primitive pattern are obtained through the above method; an offset range is set according to the offset of the primitive pattern. Setting the offset range can reduce the calculation variables, so it is necessary to set the maximum value of the offset of the primitive pattern in the x direction; in the polar line direction of the same matching point corresponding to the two acquired images, the maximum offset is 7 pixels. Therefore, on the basis of pixel-level positioning based on array coding pattern primitive pattern matching, each pixel is further subdivided into 10×10 sub-pixels within the 7×7 pixel neighborhood of the center coordinate of each primitive pattern positioning, that is, divided into 70×70 parts.

6. The array coding depth estimation method based on machine learning as claimed in claim 5, characterized in that: In S2, the decision tree is a deep classifier used to predict each attribute S of the primitive pattern, and the attribute S includes: position, direction; the primitive pattern row vector classifier is constructed with the row vector of the primitive pattern as the unit to realize the classification of the primitive pattern; the inputs of the decision tree are I, s c and p , where I is the set of primitive patterns in the input image, s c is a set of primitive patterns, here are four types of hourglass images, s p is the classification target set, here is 0, 1, 2, 3; therefore, the learning target of the decision tree is s c to p The mapping relationship.

7. The array coding depth estimation method based on machine learning as claimed in claim 5, characterized in that: The attribute prediction classification function of the primitive pattern is expressed as follows: Where I represents the primitive pattern, which is the primitive pattern set of the input image; X(s c ) is the primitive pattern s c The length in the x-axis direction, Y(s c ) is the primitive pattern s c In the y-axis direction, θ is the direction of the primitive pattern, and x, y are the pixel-level offsets of the image; L, R are the classification ensembles of the decision tree, which is a prediction model of an attribute structure. When designing the decision tree: determine the category according to formula (1), and judge one category at a time. First, determine whether it is category 0. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 1. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 2. If so, it is located on the left branch L. If not, it is other categories and is located on the right branch R; then determine whether it is category 3. If so, it is located on the left branch L. If not, it is other abnormal categories that do not belong to categories 0-3, where each category corresponds to a different primitive pattern direction θ.

8. The array coding depth estimation method based on machine learning according to claim 7, characterized in that: The learning parameters of the decision tree are expressed as S = (x, y, θ) The coordinates x, y are the pixel-level offsets of the image, and the offset is related to the parameter θ.

9. The array coding depth estimation method based on machine learning as claimed in claim 8, characterized in that: In S2, training the decision tree using the white balance board by machine learning includes: using a moving white balance board for training and measurement by machine learning, two cameras capture two images and perform preprocessing to obtain an array coding pattern, and then obtaining the type and sub-pixel positioning of the primitive pattern based on the decision tree, and then comparing with the sub-pixel positioning of the center of the primitive pattern pre-designed by the white balance board and providing feedback to modify the decision tree parameters until a decision tree that meets the requirements is obtained.

10. The array coding depth estimation method based on machine learning according to claim 9, characterized in that: The method also includes testing the depth estimation accuracy based on the array coding pattern decision tree: training and measuring through a moving white balance board, and judging the accuracy of the depth estimation by measuring the plane fitting error of the white balance board. Specifically, firstly, the type and sub-pixel positioning of the primitive pattern are obtained based on the decision tree, and then the depth information is solved based on the triangulation principle. Finally, the obtained depth information is converted into a three-dimensional point cloud information map, and the accuracy of the depth estimation is judged by measuring the plane fitting error of the white balance board.

Images