A tea leaf picking point detection method based on a light field camera

Abstract

The application provides a tea leaf picking point detection method based on a light field camera. It relates to the field of agricultural intelligence. The specific steps are as follows: step (1): obtaining 4D light field data by using a light field camera; step (2): realizing coarse positioning of a tea leaf bud area under low resolution; step (3): digital refocusing of a tea leaf tender shoot part small depth of field; step (4): applying a super-resolution algorithm to the refocused part to obtain a high-resolution tea leaf tender shoot image; step (5): detecting a tea stem part by using a rotating target detection algorithm based on an improved IoU calculation method; step (6): three-dimensional reconstruction of the tea stem; and step (7): obtaining a tea leaf tender shoot picking point by matching the stem point cloud based on a PPF algorithm. The application uses a light field camera as a data acquisition device, utilizes the characteristic that the light field camera can be digitally refocused, combines a two-dimensional target detection method and a three-dimensional point cloud matching method, and realizes accurate positioning of a tea leaf picking point.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of agricultural intelligence, and particularly relates to a tea leaf picking point detection method based on a light field camera. BACKGROUND

[0002] Intelligent picking of tea leaves is of great significance for reducing the workload of tea leaves. At present, the research on tea bud detection mainly focuses on the two-dimensional field. Mainly two-dimensional image combined with traditional algorithm and two-dimensional image combined with deep learning algorithm. In terms of traditional algorithm, the features of tea buds are extracted, and then a threshold based on super green features is used to segment the tea bud image, and finally the whole tea bud is recognized. In the deep learning algorithm, the general target detection algorithm is used to classify and locate the tea bud.

[0003] However, the above technology still has the following defects: ①Due to the growth characteristics of tea, the tea bud detection method based on two-dimensional image has poor detection effect on the occluded target in target detection, and cannot detect the occluded bud. ②Traditional algorithm and deep learning algorithm perform poorly when processing unordered point cloud data, and cannot obtain good detection accuracy. SUMMARY

[0004] The present application proposes a tea bud detection method based on a light field camera to solve the problem that traditional neural networks have a certain upper limit in target detection accuracy.

[0005] The present application proposes a tea leaf picking point detection method based on a light field camera, comprising the following steps:

[0006] Step (1): obtaining 4D light field data using a light field camera;

[0007] Step (2): realizing coarse positioning of tea bud area at low resolution;

[0008] Step (3): digital refocusing of tea bud part small scene depth;

[0009] Step (4): applying super-resolution algorithm to the refocused part to obtain high-resolution tea bud image;

[0010] Step (5): using a rotating target detection algorithm based on an improved IoU calculation method to detect tea stem part;

[0011] Step (6): three-dimensional reconstruction of tea stem;

[0012] Step (7): PPF algorithm-based stem point cloud matching to obtain tea bud picking point.

[0013] The specific steps are as follows:

[0014] Step (1): Obtain 4D light field data using a light field camera.

[0015] A light field camera was used to capture the tea tree at an angle of 45° to obtain 4D light field data of the tea tree, and a low-resolution large-depth-of-field tea tree two-dimensional RGB image and the corresponding depth image were synthesized based on the obtained 4D light field data of the tea tree.

[0016] Step (2): Coarse positioning of the tea leaf bud area at low resolution.

[0017] A low-resolution large-depth-of-field tea tree two-dimensional RGB image was used in combination with the Nvidia rotated box object detection toolbox to achieve rapid coarse positioning of the object area. Nvidia provides an open-source rotated object detection toolbox that uses RetinaNet as the backbone network for rotated object detection. The toolbox integrates five modules: mixed-precision training, data loading, highly optimized inference engine, intelligent video analysis solution, and a static image solution, making it convenient for rotated object detection. Here, we implemented tea leaf bud area positioning at low resolution as the result of preliminary target detection.

[0018] Step (3): Small-depth-of-field digital refocusing of the tea leaf shoot part.

[0019] Light field digital refocusing is the process of projecting the collected light field to a new image plane for integration. Taking the two-dimensional case as an example, L(u,s) is the collected light field, U and S represent the planes where the main lens aperture and the microlens array are located, respectively, and the distance between the two planes is L. A new focusing plane S' is selected, with a distance of L' from the U plane, L' = α × L, and the image formed on the S' plane is equal to the integration of the light field between U and S', i.e.:

[0020] I(s') = ∫L'(u,s')du, and for the same light ray, there should be L(u,s) = L'(u,s'), and according to the coordinates of the intersection of the light ray with each plane, we can get: Let l' = al, and after transformation we get: Substituting the integral formula gives: If extended to the four-dimensional case, the refocusing formula can be obtained: Refocusing is the process of shifting the light field in the position dimension and integrating it in the direction dimension.

[0021] Step (4): Apply super-resolution algorithm to the refocused part to obtain high-resolution tea leaf shoot images.

[0022] This invention proposes a learning-based method using residual convolutional networks to reconstruct light fields with higher spatial resolution. A view image in a light field is first grouped into different image stacks with consistent sub-pixel offsets and fed into different network branches to implicitly learn intrinsic correspondences. Residual information in different spatial directions is then computed from each branch and further integrated to supplement high-frequency details of the view image.

[0023] Step (5): Use a rotating target detection algorithm based on an improved IoU calculation method to detect the tea stem parts.

[0024] This invention improves the accuracy of rotating target detection by refining the IoU calculation method. Let A = [A1, A2, A3, A4] and B = [B1, B2, B3, B4] be two rectangles, and P = A ∩ B. If A and B overlap, then P is an m-sided polygon, where m is an integer in the set {0, 1, ..., 8}. If m = 0, P is a point; if m = 1, P is a line segment; and if m ≥ 3, P is an m-sided polygon with a non-zero surface. In the latter case, we use L = {I1, I2, ..., I...} m Let} represent the vertices of P. The elements of L can be divided into two subsequences. L1 is the first sublist containing all intersections of an edge of A with an edge of B, which are different from the vertices of A or B. L2 is the second sublist containing vertices of A within the region of B, and vertices of B located inside the region of A. We define the following concept: if an element of L is both a vertex of A or B and an intersection of an edge of A with an edge of B, then it is called a multi-order vertex. An element of L is called a single vertex if it is an intersection of an edge of A with an edge of B, and not a vertex of A or B. Assume I k The coordinates of k = 1, ..., m in the Cartesian coordinate system B = (O, i, j) of the two-dimensional Euclidean affine plane are (x... k ,y k ) t .

[0025] To find the vertices L = {I1, I2, ..., I...} of the convex polygon P, we need to define the vertices L = {I1, I2, ..., I...}. m To perform counter-clockwise sorting, we first consider the centroid G of set L, then calculate the polar angle of each vertex relative to centroid G. Finally, sort the vertices according to I1,...,I... m The vertices are arranged in ascending order of their polar angles. We then denote this as J. k =(x (k) ,y (k) ) t Let k = 1, ..., m, be the counterclockwise arrangement of the ordered vertices of P = A ∩ B. And let (x... (0) ,y (0)(x (m) ,y (m) ) and (x (m+1) ,y (m+1) ) = (x (1) ,y (1) ). The intersection region is calculated as and the final IoU value is calculated by . In the aspect of rotated object detection, we use the R3Det algorithm as the backbone network to predict the object frame.

[0026] Step (6): three-dimensional reconstruction of tea stems.

[0027] The application uses a sub-pixel multi-view stereo matching algorithm based on phase shift to realize depth estimation of a light field. The core of the algorithm is the use of phase shift theory, that is, a small displacement in the spatial domain is the product of the frequency domain expression of the original signal and the exponential power of the displacement in the frequency domain, that is, the following formula: Therefore, the image after displacement can be represented as: The idea of phase shift can realize sub-pixel accuracy matching, and to some extent, solve the problem of short baseline. In order to enable the matching between sub-view images, two different cost quantities, SAD and GRAD, are designed, and finally the final matching quantity C is obtained through weighting, which is a function of point x and loss number l, and the specific form is shown in the following formula: C(x,l) = alpha C A (x,l)+(1-alpha)C G (x,l), wherein alpha [0,1] represents the weight between SAD loss quantity C A and SGD loss quantity C G . Meanwhile, C A is defined as follows: Wherein R x represents a rectangular region in the neighborhood of x point; tau1 is the truncation value of the cost; V represents the remaining view angles except the center view angle u c . The above formula compares the difference between the center view angle image I(u c ,x) and the remaining view angles I(u,x) to construct the loss quantity, specifically, by constantly moving a small distance around the point x in a certain view I(u i ,x) and subtracting the center view; repeat this process until all view angles are compared. The small distance mentioned above is Delta x in the formula, which is defined as: Delta x(u,l) = lk(u-u c ), wherein k represents the unit (pixel) of depth / disparity layer, and Delta x will increase linearly with the increase of the distance between any view angle and the center view angle. Similarly, the second matching cost quantity SGD can be constructed, and the basic form is as follows:

[0028] where Diff x denotes the gradient of the sub-aperture image in x direction; β(u) controls the weight of the cost volume in two directions, which is represented by the relative distance between the arbitrary view and the center view:

[0029]

[0030] So far, the cost function is constructed. Then, the edge-preserving filter is used to aggregate the loss for the cost function, and the optimized cost volume is obtained. Then, an iterative optimization model is constructed to optimize the depth map.

[0031] After obtaining the depth map, it is input into the open3d library to restore the three-dimensional point cloud of the tea stem.

[0032] Step (7): Stem point cloud matching based on PPF algorithm.

[0033] In the global modeling stage, the model point information is first used for sampling preprocessing, because the points in the model may not be needed. Then, the discrete PPF features are used as indexes to establish a four-dimensional lookup table for storing model pairs. This table stores the four-dimensional features of many pairs of points in the model, which will provide the discrete PPF features obtained in the subsequent scene pairs for searching corresponding reference points and rotation angles. In the matching stage, the point cloud of the scene is also preprocessed by sampling. Then, two points in the scene are randomly taken to form PPF features, which are discretized and used as indexes, and the four-dimensional lookup table established in the global modeling stage is used for searching. After searching, a group of model point pairs and all possible candidate scene point pairs corresponding to the model point pairs are obtained. Then, a voting method similar to the generalized Hough transform is used. The Hough transform is used to detect shapes that can be precisely defined, and the data can be used to find the location and direction of these shapes in the image. The generalized Hough transform is an adjustment based on the Hough transform according to the principle of template matching, which can detect any shape. In the voting process, the best local matching point pair in the scene candidate point pair is found, and after processing all the model point pairs, the global rigid motion can be calculated from the coordinates of the best local point pair, thereby finding the position and direction of the target object. In this method, the middle section of the model is defined as the picking part of the tea tender shoots when the model is constructed.

[0034] The above-mentioned light field digital refocusing realizes the effect of precise positioning of the tea stem part by shifting and superimposing the multi-view sub-images according to the principle that the corresponding parallax of objects at different depths in the multi-view sub-images is different, and finally realizing the effect of "small depth of field" with a large equivalent aperture.

[0035] The light field super-resolution algorithm uses a learning-based method to reconstruct high spatial resolution light field using residual convolutional network. Firstly, the multi-view images in a light field are grouped into different image stacks with consistent sub-pixel offsets, and are sent into different network branches to implicitly learn the inherent correspondence. Then the residual information in different spatial directions is calculated from each branch, and further integrated to supplement the high-frequency details of the view images. The recognition efficiency and detection accuracy of tea stems are further improved by obtaining high-resolution images.

[0036] The present application uses a light field camera as an image acquisition device, utilizes the refocusing characteristics of the light field camera, first performs coarse positioning of tea tender shoots at low resolution, and then performs accurate rotation target detection on the tea stem part through digital refocusing and light field high resolution method. And on this basis, the three-dimensional point cloud of the tea stem part is restored by combining the obtained depth image, and the final picking point is obtained through point cloud matching. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 Technical route of the present application

[0038] Figure 2 Light field refocusing principle involved in the present application; (a) is the superposition of light field sub-images after joint displacement; (b) is the superposition of light field sub-images; (c) is a schematic diagram of light rays imaging in different planes;

[0039] Figure 3 Point cloud matching method based on global feature modeling and local matching involved in the present application; (a) is a global modeling schematic diagram; (b) is a local matching schematic diagram; DETAILED DESCRIPTION

[0040] The embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0041] Step (1): acquiring 4D light field data using a light field camera.

[0042] A light field camera is used to shoot tea trees at an angle of 45 degrees to obtain 4D light field data of tea trees, and low-resolution large-depth-of-field tea tree two-dimensional RGB images and corresponding depth images are synthesized according to the obtained 4D light field data of tea trees.

[0043] Step (2): coarse positioning of tea bud area at low resolution.

[0044] The low-resolution large depth-of-field tea tree two-dimensional RGB image is combined with the Nvidia rotated box object detection toolbox to realize the rapid coarse positioning of the object area. Nvidia provides an open source rotated object detection toolbox which uses RetinaNet as the backbone network for rotated object detection. The toolbox integrates five modules of mixed precision training, data loading, highly optimized inference engine, intelligent video analysis solution and static image solution, and can conveniently perform rotated object detection. Here, the tea bud area positioning under low resolution is realized as the preliminary object detection result.

[0045] Step (3): tea tender shoot part small depth-of-field digital refocusing.

[0046] Light field digital refocusing is to project the collected light field to a new image plane for integration. Taking the two-dimensional case as an example, L(u,s) is the collected light field, U and S represent the planes where the main lens aperture and the microlens array are located respectively, and the distance between the two planes is L. A new focusing plane S' is selected, and the distance from U plane is L', L' = a x L. The image formed on the S' plane is equal to the integration of the light field between U-S', that is:

[0047] I(s') = ∫L'(u,s')du, and for the same light ray, there should be L(u,s) = L'(u,s'), and according to the coordinates of the intersection of the light ray and each plane, we can get: Let l' = al, and after transformation, we get: Substitute the integral formula to get: If extended to the four-dimensional case, the refocusing formula can be obtained: Refocusing is the process of translating the light field in the position dimension and integrating in the direction dimension.

[0048] Step (4): applying super-resolution algorithm to the refocusing part to obtain high-resolution tea tender shoot image.

[0049] The present application proposes a learning-based method using residual convolutional network to reconstruct light field with higher spatial resolution. The view images in a light field are first grouped into different image stacks with consistent sub-pixel offsets, and fed into different network branches to implicitly learn the intrinsic correspondence. Then the residual information in different spatial directions is calculated from each branch and further integrated to supplement the high-frequency details of the view images.

[0050] Step (5): using the rotated object detection algorithm based on the improved IoU calculation method to detect the tea stem part.

[0051] This invention improves the accuracy of rotating target detection by refining the IoU calculation method. Let A = [A1, A2, A3, A4] and B = [B1, B2, B3, B4] be two rectangles, and P = A ∩ B. If A and B overlap, then P is an m-sided polygon, where m is an integer in the set {0, 1, ..., 8}. If m = 0, P is a point; if m = 1, P is a line segment; and if m ≥ 3, P is an m-sided polygon with a non-zero surface. In the latter case, we use L = {I1, I2, ..., I...} m Let} represent the vertices of P. The elements of L can be divided into two subsequences. L1 is the first sublist containing all intersections of an edge of A with an edge of B, which are different from the vertices of A or B. L2 is the second sublist containing vertices of A within the region of B, and vertices of B located inside the region of A. We define the following concept: if an element of L is both a vertex of A or B and an intersection of an edge of A with an edge of B, then it is called a multi-order vertex. An element of L is called a single vertex if it is an intersection of an edge of A with an edge of B, and not a vertex of A or B. Assume I k The coordinates of k = 1, ..., m in the Cartesian coordinate system B = (O, i, j) of the two-dimensional Euclidean affine plane are (x... k ,y k ) t .

[0052] To find the vertices L = {I1, I2, ..., I...} of the convex polygon P, we need to define the vertices L = {I1, I2, ..., I...}. m To perform counter-clockwise sorting, we first consider the centroid G of set L, then calculate the polar angle of each vertex relative to centroid G. Finally, sort the vertices according to I1,...,I... m The vertices are arranged in ascending order of their polar angles. We then denote this as J. k =(x (k) ,y (k) ) t Let k = 1, ..., m, be the counterclockwise arrangement of the ordered vertices of P = A ∩ B. And let (x... (0) ,y (0) )=(x (m) ,y (m) ), (x (m+1) ,y (m+1) )=(x (1) ,y (1) The intersecting region was calculated as follows: and through The final IoU value is calculated. For rotated object detection, we use the R3Det algorithm as the backbone network for bounding box prediction.

[0053] Step (6): Three-dimensional reconstruction of tea stems.

[0054] The present application uses a phase shift based sub-pixel multi-view stereo matching algorithm to realize depth estimation of light field. The core of the algorithm is to use the phase shift theory, that is, a small displacement in the spatial domain is the product of the frequency domain expression of the original signal and the exponent of the displacement, that is, the following formula: Therefore, the image after displacement can be represented as:

[0055]

[0056] The idea of phase shift can realize sub-pixel precision matching, and to some extent solve the problem of short baseline. In order to enable the matching between sub-view images, two different cost quantities, SAD and GRAD, are designed, and finally the final matching quantity C is obtained through weighted manner, which is a function of point x and loss number l, and the specific form is shown in the following formula: C(x, l) = αC A (x, l) + (1-α)C G (x, l), where α∈[0,1] represents the weight between SAD loss quantity C A and SGD loss quantity C G . At the same time, C A in it is defined as the following form:

[0057]

[0058] R x in it represents the rectangular region in the neighborhood of x point; τ1 is the truncation value of the cost; V represents the remaining views except the center view u c . The above formula constructs the loss quantity by comparing the difference between the center view image I(u c , x) and the remaining views I(u, x), specifically, by constantly moving a small distance around the point x in a certain view I(u i , x) and subtracting the center view; repeat this process until all views are compared. The small distance mentioned above is Δx in the formula, which is defined as: Δx(u, l) = lk(u-u c ), where k represents the unit (pixel) of depth / disparity layer, and Δx will increase linearly with the increase of the distance between any view and the center view. Similarly, the second matching cost quantity SGD can be constructed, and its basic form is as follows:

[0059]

[0060] Diff xdenotes the gradient of the sub-aperture image in x direction; β(u) controls the weight of the cost volume in two directions, which is represented by the relative distance between the arbitrary view and the center view:

[0061]

[0062] So far, the cost function is constructed. Then, loss aggregation is performed on the cost function using an edge-preserving filter to obtain an optimized cost volume. Then, an iterative optimization model is constructed to optimize the depth map. After obtaining the depth map, it is input into the open3d library to restore the three-dimensional point cloud of the tea stem.

[0063] Step (7): Stem point cloud matching based on PPF algorithm.

[0064] In the global modeling stage, the model point information is used for sampling preprocessing. Then, the discrete PPF features are used as indexes to establish a four-dimensional lookup table for storing model pairs. This table stores the four-dimensional features of many pairs of points in the model, which will provide the corresponding reference points and rotation angles for the discrete PPF features obtained in the subsequent scene pairs. In the matching stage, the scene point cloud is also sampled for preprocessing. Then, two points in the scene are randomly selected to form PPF features. These PPF features are discretized and used as indexes to search the four-dimensional lookup table established in the global modeling stage. After searching, a group of model point pairs and all possible candidate scene point pairs corresponding to the model point pairs are obtained. Then, a voting method similar to the generalized Hough transform is used. The Hough transform is used to detect shapes that can be precisely defined, and the generalized Hough transform is a modification of the Hough transform based on template matching. It can detect any shape. In the voting process, the best local matching point pair in the scene candidate point pair is found. After processing all the model point pairs, the global rigid motion can be calculated from the coordinates of the best local point pair, thereby finding the position and direction of the target object. In this method, the middle section of the model is defined as the picking site of the tea tender shoots when the model is constructed.

[0065] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the exemplary description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0066] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since the scope of the application will be defined with respect to the claims and their equivalents.

Claims

1. A method for detecting tea picking points based on a light field camera, characterized in that, Includes the following steps: Step (1): Acquire 4D light field data using a light field camera; Step (2) achieves coarse localization of the tea bud area at low resolution; Step (3), digital refocusing on the tender tea shoots with shallow depth of field; Step (4): Apply a super-resolution algorithm to the refocused area to obtain a high-resolution image of the tea shoots; Step (5): Use a rotating target detection algorithm based on an improved IoU calculation method to detect the tea stem parts; Step (6), 3D reconstruction of tea stems; Step (7): The picking points of tea shoots are obtained by matching the stem point cloud based on the PPF algorithm; Step (3) involves the following process: Digital refocusing of the light field involves reprojecting the acquired light field onto a new image plane for integration, using... Representing the light field in four dimensions, we obtain the refocusing formula: ; The specific process of step (5) is as follows: To find the vertices of the convex polygon P To perform counter-clockwise sorting, first consider the centroid G of set L, then calculate the polar angle of each vertex relative to the centroid G, and finally sort by vertex. Arrange the vertices in ascending order of their polar angles, and then denote them as follows: ,in For different vertices, Let the coordinates of the corresponding vertex be used as... The ordered vertices are arranged counterclockwise, and let , ; The intersecting region was calculated as follows and through The final IoU value is calculated. For rotating target detection, the R3Det algorithm is used as the backbone network to predict the target box. The specific process of step (7) is as follows: In the global modeling stage, the model point information is first sampled and preprocessed. Discrete PPF features are used as indexes to build a four-dimensional lookup table to store model pairs. In the matching stage, the point cloud of the scene is sampled and preprocessed. Then, any two points in the scene are randomly selected to form PPF features. The four-dimensional lookup table established during the global modeling stage is used to search for model point pairs and candidate scene point pairs. After obtaining the model point pairs and candidate scene point pairs, the generalized Hough transform is used for voting. During the voting process, the best local matching point pair among the candidate scene point pairs is found. The global rigid motion is calculated from the coordinates of the best local point pair to find the position and orientation of the target object.

2. The method for detecting tea picking points based on a light field camera according to claim 1, characterized in that, In step (1), a light field camera is used to take pictures of the tea tree at a 45° angle to obtain the 4D light field data of the tea tree, and a low-resolution, large-depth-of-field two-dimensional RGB image of the tea tree and the corresponding depth image are synthesized based on the obtained 4D light field data of the tea tree.

3. The method for detecting tea picking points based on a light field camera according to claim 1, characterized in that, Step (2) is as follows: use a low-resolution, large-depth-of-field two-dimensional RGB image of tea trees combined with the NVIDIA Rotating Box Object Detection Toolbox to achieve fast coarse localization of the object region, and use RetinaNet as the backbone network for rotating object detection.

4. The method for detecting tea picking points based on a light field camera according to claim 1, characterized in that, In step (4), a learning-based method using residual convolutional networks is proposed to reconstruct light fields with higher spatial resolution. The view images in a light field are first grouped into different image stacks with consistent sub-pixel offsets and fed into different network branches to implicitly learn the intrinsic correspondences. Then, residual information in different spatial directions is computed from each branch and further integrated to supplement the high-frequency details of the view images.

5. The method for detecting tea picking points based on a light field camera according to claim 1, characterized in that, The specific process of step (6) is as follows: In order to enable matching between sub-view images, two different cost quantities are designed: Sum of Absolute Differences (SAD) and Sum of Gradient Differences (GRAD). Finally, the final matching quantity is obtained by weighting them together. loci and loss number The function is shown in the following formula: ,in Indicates SAD loss amount and the loss of stochastic gradient descent (SGD) The weights between them, and the weights among them It is defined in the following form: , among them Indicates in A rectangular region in the neighborhood of a point; It is the cutoff value of the cost; In addition to the central perspective Other perspectives besides those mentioned above; The above formula compares images from the center viewpoint. Compared to other perspectives The loss is constructed by varying the values ​​of the elements, specifically by continuously adjusting the values ​​from a certain perspective. On point Move the viewpoint a small distance around the center viewpoint and compare the differences; repeat this process until all viewpoints have been compared. The small distance mentioned here is the one in the formula. , is defined as: Where k represents the unit pixel of the depth / parallax layer, The cost increases linearly with the distance between any viewpoint and the central viewpoint. Similarly, a second matching cost, SGD, is constructed, with the following basic form: , in Indicates in A rectangular region in the neighborhood of a point; , It is the cutoff value of the cost; In addition to the central perspective Other perspectives besides those mentioned above; This represents the gradient of the sub-aperture image in the x-direction; The weights controlling the cost in both directions are represented by the relative distance between any viewpoint and the central viewpoint: ; In the formula, , The center of different views is used, and then the cost function is aggregated using an edge-preserving filter to obtain the optimized cost amount. Then, an iterative optimization model is constructed to optimize the depth map. After obtaining the depth map, it is input into the Open3D library to recover the 3D point cloud of the tea stem.

Images