A Method for Extracting 3D Parameters of Banana Fruit Comb Based on RGB-D Image Segmentation

Abstract

This invention discloses a method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation. By combining depth and color images, multi-target finger segmentation is performed on the banana fruit comb, acquiring the depth image of each finger and converting it into a point cloud for arc length and width parameter extraction. A regression model is then constructed to estimate volume and weight parameters. This invention can effectively extract the three-dimensional parameters of banana fruit combs with complex shapes, providing a data foundation for fruit sorting.

Description

Technical Field

[0001] This invention relates to the technical field of fruit morphological feature parameter extraction, and in particular to a method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation. Background Technology

[0002] In agriculture, the three-dimensional parameters of fruits, such as size, volume, and weight, have received widespread attention due to their important role in fruit classification. These parameters help enhance the market value and quality of agricultural products. Currently, the three-dimensional parameters of fruits are mainly extracted using machine vision technology. By analyzing two-dimensional images, the size parameters of the fruit are calculated, and the fruit is fitted into a sphere or other regularly shaped three-dimensional model for volume calculation. To improve the accuracy of parameter extraction, the fruit is also reconstructed in three dimensions, using three-dimensional techniques to extract the three-dimensional parameters of the fruit in three-dimensional space.

[0003] Significant progress has been made in extracting fruit parameters, but existing methods primarily target individual, regularly shaped fruits. Banana fruit combs consist of two rows of fingers, with gaps of varying sizes between them creating an uneven surface, resulting in a highly irregular overall shape. Bananas are typically sold as whole fruit combs, and the morphological differences in these combs significantly impact sales and processing. Different dimensions, volumes, and weights influence consumer purchasing choices and significantly affect the grading of banana fruit combs. Currently, this sorting work is mainly done manually, which is inefficient and relies heavily on subjective judgment. Therefore, existing methods for extracting the three-dimensional parameters of fruits are not suitable for banana fruit combs, necessitating the design of a method specifically designed for their extraction. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a method for extracting three-dimensional parameters of banana combs based on RGB-D image segmentation. This method can effectively extract the three-dimensional parameters of banana combs with complex shapes, providing a data foundation for fruit sorting of banana combs.

[0005] To achieve the above objectives, the technical solution provided by this invention is: a method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation, comprising the following steps:

[0006] S1: Acquire depth and color images of banana combs, and use image alignment methods to map the pixels of the depth and color images to the same coordinate system to obtain depth and color images with the same resolution;

[0007] S2: Preprocess the color image to obtain a binarized image containing only the banana comb. Mask the depth image and color image with the binarized image to obtain a depth image and a color image containing only the area where the banana comb is located.

[0008] S3: Combining depth images and color images containing only the region where the banana comb is located, a multi-target banana finger segmentation strategy based on region growing was designed. The operation is as follows: a portion of the internal region of each finger in the image is automatically located, and pixels on the edge of this region are selected as seed points. The difference in depth values ​​between the seed points and growth points in the depth image is used as the growth criterion. The seed points gradually grow from the edge of the internal region of the finger along the outer region. If the growth point meets the growth criterion, it is classified as a new seed point. This process continues until all points in the outer region have grown. The regions containing all seed points are then merged with the originally identified internal regions to obtain the complete region of a single finger in the image. The above operation is performed on all fingers to obtain the depth image of the region where each finger is located.

[0009] S4: Perform point cloud conversion on the depth image of the region corresponding to a single finger and preprocess the point cloud to obtain a finger point cloud suitable for subsequent parameter extraction.

[0010] S5: Combining the preprocessed single finger point cloud, the pose coordinate system of the banana finger is corrected using principal component analysis. Projections are made in different directions within the coordinate system. Based on the finger arc length and width, specific boundary points of the projected point cloud are obtained. Then, curve fitting is performed on the boundary points. The finger arc length parameter is obtained by calculating the length of the fitted curve, while the finger width parameter is obtained by slicing the fitted curve into point clouds and performing corresponding calculations.

[0011] S6: Combining the arc length and width parameters of the banana fingers, a multivariate multinomial regression model is constructed. The multinomial regression model is used to estimate the volume parameters of the banana fingers, and a multivariate linear regression model is constructed to extract the weight parameters of the banana fingers. The weight parameters of all fingers are summed up, and the fruit index per kilogram of the banana comb is further calculated.

[0012] Furthermore, in step S1, a data acquisition device is constructed using a computer, a Femto Bolt depth camera, a camera bracket, and a darkroom. The Femto Bolt depth camera is fixed on the camera bracket and positioned above the banana comb, with the lens of the Femto Bolt depth camera facing the banana comb. The computer calls the Orbbec SDK interface to control the Femto Bolt depth camera to photograph the banana comb in the darkroom. The pixel points of the depth image and the color image are mapped to the same coordinate system using an image alignment method to obtain depth images and color images with the same resolution.

[0013] Further, in step S2, the color image of the banana comb is converted to a grayscale image, and an appropriate grayscale threshold is set. A threshold segmentation method is used to obtain a binarized image containing only the banana comb. The pixel value of the foreground region where the banana comb is located is 255, and the pixel value of other regions is 0. The above binarized image is masked with the original depth image and color image to obtain a depth image and a color image containing only the region where the banana comb is located.

[0014] Furthermore, in step S3, the specific operation steps of the multi-target banana finger segmentation strategy are as follows:

[0015] S31: Automatic localization of the inner region of the banana finger. The operation is as follows: Morphological processing of the depth image is performed using gradient operation, combined with threshold segmentation, to obtain the outer region of the banana finger in the depth image; for the color image, the edge of the banana finger is detected using the Canny algorithm, and morphological processing of dilation operation is performed to obtain the outer region of the banana finger in the color image; the outer regions of the depth image and the color image are superimposed to obtain the closed outer region of the banana finger; connected component analysis is performed on the image of the closed outer region, the area of ​​all connected components is counted, the connected components with small areas are removed, and the remaining connected components are the inner regions of each finger. By calculating the average position of the pixels in each region, the inner region of each finger can be automatically located.

[0016] S32: Seed point selection optimization. The operation is as follows: In order to reduce the amount of computation for region growth, the internal regions of each fruit finger obtained in step S31 are not directly used as seed points. Instead, the edges of the internal regions of each fruit finger are detected by the Canny detection algorithm, and the pixels at the location of the edge are used as the initial seed points for banana fruit finger region growth and segmentation.

[0017] S33: Design of the termination criterion for the predetermined growth range. The operation is as follows: To improve the segmentation speed and accuracy of the fruit finger, the outer region of the fruit finger obtained in step S31 is taken as the predetermined growth range. Within this range, the initial seed point in step S32 starts to grow from the edge point of the inner region of the fruit finger. The growth process is as follows: examine the eight adjacent pixels of the seed point in turn, calculate the depth difference between the growth point and the depth image position of the seed point. If it is less than the predetermined threshold, it is determined that the growth point belongs to a part of the fruit finger and is taken as a new seed point. Repeat the above growth process until all seed points have finished growing. Then merge the region where all seed points are located with the originally identified inner region to obtain the complete region of a single fruit finger in the image. Perform the above operation on all fruit fingers to obtain the depth image of the region where each fruit finger is located.

[0018] Furthermore, in step S4, based on the imaging principle of the camera model, the world coordinate system and the camera coordinate system are aligned, and the formula for converting the two-dimensional depth image into three-dimensional point cloud data is:

[0019]

[0020]

[0021] Z w =Z c

[0022] In the formula, X w Y w Z w These represent the three-dimensional coordinates of each point in the point cloud along the x, y, and z axes in the world coordinate system, respectively. u and v represent the row and column numbers of each pixel in the depth image, respectively. Z c f represents the value of each pixel in the depth image. x f y The focal length represents the camera's intrinsic parameters, and u0 and v0 represent the coordinates of the optical center point of the camera's intrinsic parameters.

[0023] After performing point cloud transformation on the banana fingers obtained in step S3, point clouds of each banana finger are obtained. The point clouds of the fingers are preprocessed using voxel downsampling and statistical filtering methods respectively to obtain point clouds of each finger suitable for subsequent parameter extraction.

[0024] Furthermore, the specific operation steps of step S5 are as follows:

[0025] S51: The coordinate system of the banana finger point cloud is corrected by performing PCA principal component analysis. The principal directions of the obtained banana finger point cloud are used as the pose coordinate system, where the x, y, and z axes represent the width, height, and length of the banana finger, respectively. The banana finger point cloud is projected onto the yoz plane to obtain the projected point cloud. The Alpha Shapes algorithm is used to obtain the concave boundary point set of this projected point cloud. Boundary points of the upper region of the concave boundary point set are sequentially selected along the z-axis, and NURBS curve fitting is performed on the boundary points to obtain the set of fitted points {(y1,z1),(y2,z2),...,(y...z2)}. i ,z i ),(y i+1 ,z i+1 ),...,(y n ,z n )}, where (y i ,z i ) represents the i-th point in the fitted curve, y i The z-axis value represents the value of the i-th point. i The value of the z-axis represents the i-th point, n represents the total number of fitted points, and the arc length s of the banana finger is:

[0026]

[0027] S52: Project the banana finger point cloud onto the xoz plane to obtain the projected point cloud. Use the Alpha Shapes algorithm to obtain the concave boundary point set of the projected point cloud. Use the NURBS curve algorithm to fit the closed contour curve of the concave boundary points, and slice along the z-axis to obtain the sliced ​​point cloud. Take the maximum x-axis value of all points in the sliced ​​point cloud in sequence. max and minimum value x min The absolute value of the difference between the two is the width w of the banana finger where the slice point cloud is located.

[0028] w = |x max -x min |

[0029] Furthermore, in step S6, the width of each slice point cloud of the fruit finger in step S5 is taken as the diameter of an approximate cylinder. According to the formula for calculating the volume of a cylinder, the width of the fruit finger has a quadratic relationship with its volume, and the arc length of the fruit finger has a linear relationship with its volume. The arc length vector S = [s1, s2, s3, ..., s j ] T And the width vector W = [w1, w2, w3, ..., w j ] T , where s j The representative fruit refers to the arc length of the j-th slice point cloud obtained according to step S51, w j Based on the width obtained in step S52, the j-th segment point cloud representing the fruit finger is used to construct the following multivariate multinomial regression model for fruit finger volume extraction:

[0030] V finger =α+βS+γW 2 +ε

[0031] In the formula, V finger W represents the volume vector of the point cloud slice of the fruit finger. 2 ε represents the inner product of the width vector itself, α, β and γ represent the regression coefficient vector of the multivariate multinomial regression model, and ε represents the error term vector of the multivariate multinomial regression model.

[0032] According to the volume-density formula, there is a linear relationship between the weight and volume of the banana comb. The weight vector of the banana comb is M = [m1, m2, m3, ..., m i ,…,m k ] T m k Let the weight of the k-th banana finger be the basis for the following multiple linear regression model to extract the comb weight:

[0033] M=λ+μV hand +σ

[0034] In the formula, Vhand σ represents the volume vector of the resultant, λ and μ represent the regression coefficient vectors of the multiple linear regression model, and σ represents the error term vector of the multiple linear regression model.

[0035] For the banana fruit comb's inner and outer row fruit finger images, after removing small areas in step S31, the number of all connected components is counted. This yields the fruit indexes n1 and n2 for the inner and outer rows of the banana fruit comb, respectively. Based on the calculated fruit comb weight m, the fruit index N per kilogram of banana fruit comb is:

[0036]

[0037]

[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0039] 1. The method of the present invention is simple and reliable. It does not require complex three-dimensional reconstruction of the fruit. It only requires taking images of the outer and inner rows of banana fruit combs, and the three-dimensional parameters of the banana fruit comb can be directly extracted through a limited number of steps.

[0040] 2. This invention is low-cost, requiring only the use of Orbbec's Femto Bolt depth camera to acquire data, without the need for expensive equipment such as laser scanning.

[0041] 3. This invention realizes multi-target fruit finger segmentation of banana fruit comb, which can automatically locate the region of each fruit finger in the image and automatically segment the banana fruit fingers based on the shape characteristics of the banana fruit comb.

[0042] 4. This invention realizes the three-dimensional parameter extraction of banana fruit combs with complex shapes, can extract parameters of multiple target fruit fingers, and constructs a regression model to reduce the influence of various factors on parameter extraction, and can accurately extract three-dimensional parameters. Attached Figure Description

[0043] Figure 1 This is a flowchart of the method of the present invention.

[0044] Figure 2 This is a schematic diagram of the data acquisition device.

[0045] Figure 3 This is a schematic diagram of the fruit finger areas for automatic positioning.

[0046] Figure 4 A schematic diagram illustrating the optimization of seed point selection and automatic extraction results.

[0047] Figure 5 This is a schematic diagram of the results of multi-target fruit finger segmentation.

[0048] Figure 6 This is a point cloud diagram of banana fruit finger transformation.

[0049] Figure 7 The fitted curve is the arc length extracted after projecting the finger cloud of a banana fruit.

[0050] Figure 8 The fitted curve is the width extracted after projecting the finger cloud of a banana fruit. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0052] This embodiment discloses a method for extracting three-dimensional parameters of banana combs based on RGB-D image segmentation. By combining depth and color images, multi-target finger segmentation is performed on the banana comb. The depth image of each finger is obtained and converted into a point cloud for arc length and width parameter extraction. A regression model is then constructed to estimate volume and weight parameters. Figure 1 As shown, the specific details are as follows:

[0053] 1) Construct a data acquisition device using a computer, a Femto Bolt depth camera, a camera mount, and a darkroom, such as... Figure 2 As shown, the Femto Bolt depth camera is fixed on a camera bracket and positioned above the banana comb. The lens of the Femto Bolt depth camera is kept facing the banana comb. The computer calls the Orbbec SDK interface to control the Femto Bolt depth camera to photograph the banana comb in the dark box. The image alignment method is used to map the pixels of the depth image and the color image to the same coordinate system to obtain depth images and color images with the same resolution.

[0054] 2) Preprocess the color image by converting the color image of the banana comb to a grayscale image and setting an appropriate grayscale threshold. Use the threshold segmentation method to obtain a binarized image containing only the banana comb. The pixel value of the foreground region where the banana comb is located is 255, and the pixel value of other regions is 0. Perform masking processing on the above binarized image, the original depth image, and the color image to obtain a depth image and a color image containing only the region where the banana comb is located.

[0055] 3) Combining depth and color images containing only the region where the banana fruit comb is located, a multi-target banana finger segmentation strategy based on region growing was designed to segment the banana fruit comb into multiple targets, as follows:

[0056] 3.1) Automatic localization of the banana finger's inner region: The operation is as follows: Using a 9th-order rectangular convolution kernel, morphological processing of gradient operations is performed on the depth image, combined with threshold segmentation, to obtain the outer region of the banana finger in the depth image. The pixels in this region are 0, while the pixels in other regions are 255. For the color image, the Canny algorithm is used to detect the edges of the banana finger, and morphological processing of dilation operations is performed to thicken the edges of the finger. The region in the image where the edges are located is the outer region of the banana finger in the color image. The pixels in this region are 0, while the pixels in other regions are 255. To close the outer region of the banana finger, the outer regions of the depth image and the color image are superimposed. On a blank image where all pixels have a value of 255, the pixels corresponding to the outer regions of the banana fingers in both the depth and color images are set to 0 in the blank image, resulting in the closed outer region of the banana fingers. Connected component analysis is performed on this closed outer region. Larger connected components represent the interior regions of the target fingers requiring parameter extraction, while smaller connected components represent the interior regions of non-target fingers that do not require parameter extraction. To remove non-target fingers, the area of ​​all connected components is first calculated, then the smaller connected components are removed. The remaining connected components represent the interior regions of each target finger. By calculating the average position of pixels within each region, the interior regions of each finger can be automatically located. Figure 3 As shown.

[0057] 3.2) Seed point selection optimization: To reduce the computational load of region growth and retain as many of the obtained fruit fingers as possible, instead of directly using the internal regions of each obtained fruit finger as seed points, the edge portions of the seed regions are selected as seed points for actual region growth. The edges of the internal regions of each obtained fruit finger are detected using the Canny detection algorithm, and the pixels at the locations of these edges are used as the initial seed points for banana fruit finger region growth and segmentation. Figure 4 As shown.

[0058] 3.3) Design of the termination criterion for the predetermined growth range: To improve the segmentation speed and accuracy of the finger, the acquired closed outer region of the finger is used as the predetermined growth range. Within this range, the initial seed point grows from the edge point of the inner region of the finger. The growth process is as follows: Eight adjacent pixels of the seed point are examined sequentially, and the depth difference between the growth point and the depth image position of the seed point is calculated. If the difference is less than a predetermined threshold, the growth point is determined to be part of the finger and is used as a new seed point. This growth process is repeated until all seed points have grown. Finally, the regions containing all seed points are merged with the originally identified inner regions. Figure 5 As shown, the complete region of a single finger in the image is obtained; the above operation is performed on all fingers, and the corresponding region of the depth map of each finger is obtained, thus obtaining the depth image of the region where each finger is located.

[0059] 4) Based on the imaging principle of the camera model, the world coordinate system and the camera coordinate system are aligned. The formula for converting a two-dimensional depth image into three-dimensional point cloud data is as follows:

[0060]

[0061]

[0062] Z = Z

[0063] In the formula, X w Y w Z w These represent the three-dimensional coordinates of each point in the point cloud along the x, y, and z axes in the world coordinate system, respectively. u and v represent the row and column numbers of each pixel in the depth image, respectively. Z c f represents the value of each pixel in the depth image. x f y The focal length represents the camera's intrinsic parameters, and u0 and v0 represent the coordinates of the optical center point of the camera's intrinsic parameters.

[0064] After performing point cloud transformation on the depth images of the segmented banana fingers, point clouds of each banana finger are obtained. Voxel downsampling and statistical filtering methods are then used to preprocess the finger point clouds to obtain point clouds suitable for subsequent parameter extraction, such as... Figure 6 As shown.

[0065] 5) Extract the three-dimensional parameters of the arc length and width of the fruit finger, as follows:

[0066] 5.1) Extracting the arc length parameters of each banana finger: Principal component analysis (PCA) is performed on the banana finger point cloud to correct its coordinate system. The principal directions of the obtained banana finger point cloud are used as the pose coordinate system, where the x, y, and z axes represent the width, height, and length of the banana finger, respectively. The banana finger point cloud is projected onto the yoz plane to obtain the projected point cloud. The Alpha Shapes algorithm is used to obtain the concave boundary point set of this projected point cloud. Boundary points of the upper region of the concave boundary point set are sequentially selected along the z-axis, and NURBS curve fitting is performed on the boundary points, such as... Figure 7 As shown, the set of fitted points {(y1,z1),(y2,z2),...,(y...} is obtained. i ,z i ),(y i+1 ,z i+1 ),...,(y n ,z n )}, where (y i ,z i ) represents the i-th point in the fitted curve, y i The z-axis value represents the value of the i-th point.i The value of the z-axis represents the i-th point, n represents the total number of fitted points, and the arc length s of the banana finger is:

[0067]

[0068] 5.2) Extract the width parameters of each finger: Project the banana finger point cloud onto the xoz plane to obtain the projected point cloud. Use the Alpha Shapes algorithm to obtain the concave boundary point set of this projected point cloud; use the NURBS curve algorithm to fit the closed contour curve of the concave boundary points, such as... Figure 8 As shown, the points are sliced ​​along the z-axis to obtain a sliced ​​point cloud. The maximum x-axis value of all points in the sliced ​​point cloud is then taken sequentially. max and minimum value x min The absolute value of the difference between the two is the width w of the banana finger where the slice point cloud is located.

[0069] w = |x max -x min |

[0070] 6) Extract the three-dimensional parameters of volume, weight, and fruit index per kilogram, as follows:

[0071] 6.1) Extracting the volume parameters of each fruit finger: The width of the point cloud of each slice of the fruit finger is taken as the diameter of an approximate cylinder. According to the formula for calculating the volume of a cylinder, the width and volume of the fruit finger have a quadratic relationship, and the arc length of the fruit finger has a linear relationship with the volume. The arc length vector S = [s1, s2, s3, ..., s j ] T And the width vector W = [w1, w2, w3, ..., w j ] T , where s j w represents the arc length of the point cloud of the j-th slice. j Let represent the width of the point cloud of the j-th segment of the fruit finger. To reduce errors caused by mutual occlusion between fruit fingers, different positions of fruit fingers in the fruit comb, differences in the external morphology of fruit fingers, the segmentation accuracy of fruit fingers, and the shooting conditions of depth cameras on the calculation of the arc length and width of fruit fingers, and to improve the volume extraction accuracy of banana fruit fingers, the following multivariate multinomial regression model is constructed for fruit finger volume extraction:

[0072] V finger =α+βS+γW 2 +ε

[0073] In the formula, V finger W represents the volume vector of the point cloud slice of the fruit finger. 2 ε represents the inner product of the width vector itself, α, β and γ represent the regression coefficient vectors of the multivariate multinomial regression model, and ε represents the error term vector of the multivariate multinomial regression model.

[0074] 6.2) Extracting the weight parameters of the fruit comb: According to the volume-density formula, there is a linear relationship between the weight and volume of the fruit comb. The weight vector of the banana fruit comb is M = [m1, m2, m3, ..., m i ,…,m k ] T m k Let the weight of the k-th banana finger be the basis of the comb weight extraction. The following multiple linear regression model is constructed to extract the comb weight:

[0075] M=λ+μV hand +σ

[0076] In the formula, V hand σ represents the volume vector of the resultant, λ and μ represent the regression coefficient vectors of the multiple linear regression model, and σ represents the error term vector of the multiple linear regression model.

[0077] 6.3) Extracting the fruit index per kilogram: For the fruit index images of the inner and outer rows of banana combs, count the number of all connected components after removing small areas. This yields the fruit indices n1 and n2 for the inner and outer rows of the banana comb, respectively. Based on the calculated comb weight m, the fruit index N per kilogram of the banana comb is:

[0078]

[0079]

[0080] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation, characterized in that, Includes the following steps: S1: Acquire depth and color images of banana combs, and use image alignment methods to map the pixels of the depth and color images to the same coordinate system to obtain depth and color images with the same resolution; S2: Preprocess the color image to obtain a binarized image containing only the banana comb. Mask the depth image and color image with the binarized image to obtain a depth image and a color image containing only the area where the banana comb is located. S3: Combining depth images and color images containing only the region where the banana comb is located, a multi-target banana finger segmentation strategy based on region growing was designed. The operation is as follows: a portion of the internal region of each finger in the image is automatically located, and pixels on the edge of this region are selected as seed points. The difference in depth values ​​between the seed points and growth points in the depth image is used as the growth criterion. The seed points gradually grow from the edge of the internal region of the finger along the outer region. If the growth point meets the growth criterion, it is classified as a new seed point. This process continues until all points in the outer region have grown. The regions containing all seed points are then merged with the originally identified internal regions to obtain the complete region of a single finger in the image. The above operation is performed on all fingers to obtain the depth image of the region where each finger is located. S4: Perform point cloud conversion on the depth image of the region corresponding to a single finger and preprocess the point cloud to obtain a finger point cloud suitable for subsequent parameter extraction. S5: Combining the preprocessed single finger point cloud, the pose coordinate system of the banana finger is corrected using principal component analysis. Projections are made in different directions within the coordinate system. Based on the finger arc length and width, specific boundary points of the projected point cloud are obtained. Then, curve fitting is performed on the boundary points. The finger arc length parameter is obtained by calculating the length of the fitted curve, while the finger width parameter is obtained by slicing the fitted curve into point clouds and performing corresponding calculations. S6: Combining the arc length and width parameters of the banana fingers, a multivariate multinomial regression model is constructed. The multinomial regression model is used to estimate the volume parameters of the banana fingers, and a multivariate linear regression model is constructed to extract the weight parameters of the banana fingers. The weight parameters of all fingers are summed up, and the fruit index per kilogram of the banana comb is further calculated.

2. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 1, characterized in that, In step S1, a data acquisition device is constructed using a computer, a Femto Bolt depth camera, a camera bracket, and a dark box. The Femto Bolt depth camera is fixed on the camera bracket and positioned above the banana comb, with the lens of the Femto Bolt depth camera facing the banana comb. The computer uses the Orbbec SDK interface to control the Femto Bolt depth camera to photograph the banana comb in the dark box. The pixel points of the depth image and the color image are mapped to the same coordinate system using an image alignment method to obtain depth images and color images with the same resolution.

3. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 1, characterized in that, In step S2, the color image of the banana comb is converted to a grayscale image, and an appropriate grayscale threshold is set. A threshold segmentation method is used to obtain a binarized image containing only the banana comb. The pixel value of the foreground region where the banana comb is located is 255, and the pixel value of other regions is 0. The above binarized image is masked with the original depth image and color image to obtain a depth image and a color image containing only the region where the banana comb is located.

4. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 1, characterized in that, In step S3, the specific operation steps of the multi-target banana finger segmentation strategy are as follows: S31: Automatic localization of the inner region of the banana finger. The operation is: performing morphological processing of gradient operation on the depth image and combining it with threshold segmentation to obtain the outer region of the banana finger in the depth image. For color images, the Canny algorithm is used to detect the edges of banana fingers, and morphological processing with dilation is performed to obtain the outer region of banana fingers in the color image. The outer regions of the depth image and the color image are superimposed to obtain the closed outer region of banana fingers. Connectivity analysis is performed on the image of the closed outer region, the area of ​​all connected regions is counted, and the connected regions with small areas are removed. The remaining connected regions are the internal regions of each finger. By calculating the average position of the pixels in each region, the internal regions of each finger can be automatically located. S32: Seed point selection optimization. The operation is as follows: In order to reduce the amount of computation for region growth, the internal regions of each fruit finger obtained in step S31 are not directly used as seed points. Instead, the edges of the internal regions of each fruit finger are detected by the Canny detection algorithm, and the pixels at the location of the edge are used as the initial seed points for banana fruit finger region growth and segmentation. S33: Design of the termination criterion for the predetermined growth range. The operation is as follows: To improve the segmentation speed and accuracy of the fruit finger, the outer region of the fruit finger obtained in step S31 is taken as the predetermined growth range. Within this range, the initial seed point in step S32 starts to grow from the edge point of the inner region of the fruit finger. The growth process is as follows: examine the eight adjacent pixels of the seed point in turn, calculate the depth difference between the growth point and the depth image position of the seed point. If it is less than the predetermined threshold, it is determined that the growth point belongs to a part of the fruit finger and is taken as a new seed point. Repeat the above growth process until all seed points have finished growing. Then merge the region where all seed points are located with the originally identified inner region to obtain the complete region of a single fruit finger in the image. Perform the above operation on all fruit fingers to obtain the depth image of the region where each fruit finger is located.

5. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 4, characterized in that, In step S4, based on the imaging principle of the camera model, the world coordinate system and the camera coordinate system are aligned. The formula for converting a two-dimensional depth image into three-dimensional point cloud data is as follows: WITH w =Z c In the formula, X w Y w Z w These represent the three-dimensional coordinates of each point in the point cloud along the x, y, and z axes in the world coordinate system, respectively. u and v represent the row and column numbers of each pixel in the depth image, respectively. Z c f represents the value of each pixel in the depth image. x f y The focal length represents the camera's intrinsic parameters, and u0 and v0 represent the coordinates of the optical center point of the camera's intrinsic parameters. After performing point cloud transformation on the banana fingers obtained in step S3, point clouds of each banana finger are obtained. The point clouds of the fingers are preprocessed using voxel downsampling and statistical filtering methods respectively to obtain point clouds of each finger suitable for subsequent parameter extraction.

6. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 5, characterized in that, The specific steps of step S5 are as follows: S51: The coordinate system of the banana finger point cloud is corrected by performing PCA principal component analysis. The principal directions of the obtained banana finger point cloud are used as the pose coordinate system, where the x, y, and z axes represent the width, height, and length of the banana finger, respectively. The banana finger point cloud is projected onto the yoz plane to obtain the projected point cloud. The Alpha Shapes algorithm is used to obtain the concave boundary point set of this projected point cloud. Boundary points of the upper region of the concave boundary point set are sequentially selected along the z-axis, and NURBS curve fitting is performed on the boundary points to obtain the set of fitted points {(y1,z1),(y2,z2),...,(y...z2)}. i ,z i ),(y i+1 ,z i+1 ),...,(y n ,z n )}, where (y i ,z i ) represents the i-th point in the fitted curve, y i The z-axis value represents the value of the i-th point. i The value of the z-axis represents the i-th point, n represents the total number of fitted points, and the arc length s of the banana finger is: S52: Project the banana finger point cloud onto the xoz plane to obtain the projected point cloud. Use the Alpha Shapes algorithm to obtain the concave boundary point set of the projected point cloud. Use the NURBS curve algorithm to fit the closed contour curve of the concave boundary points, and slice along the z-axis to obtain the sliced ​​point cloud. Take the maximum x-axis value of all points in the sliced ​​point cloud in sequence. max and minimum value x min The absolute value of the difference between the two is the width w of the banana finger where the slice point cloud is located. w=|x max -x min |。 7. The method for extracting three-dimensional parameters of banana fruit combs based on RGB-D image segmentation according to claim 6, characterized in that, In step S6, the width of each slice point cloud of the fruit finger in step S5 is taken as the diameter of an approximate cylinder. According to the formula for calculating the volume of a cylinder, the width of the fruit finger has a quadratic relationship with its volume, and the arc length of the fruit finger has a linear relationship with its volume. The arc length vector S = [s1, s2, s3, ..., s j ] T And the width vector W = [w1, w2, w3, ..., w j ] T , where s j The representative fruit refers to the arc length of the j-th slice point cloud obtained according to step S51, w j Based on the width obtained in step S52, the j-th segment point cloud representing the fruit finger is used to construct the following multivariate multinomial regression model for fruit finger volume extraction: V finger =α+βS+γW 2 +e In the formula, V finger W represents the volume vector of the point cloud slice of the fruit finger. 2 ε represents the inner product of the width vector itself, α, β and γ represent the regression coefficient vector of the multivariate multinomial regression model, and ε represents the error term vector of the multivariate multinomial regression model. According to the volume-density formula, there is a linear relationship between the weight and volume of the banana comb. The weight vector of the banana comb is M = [m1, m2, m3, ..., m i ,…,m k ] T m k Let the weight of the k-th banana finger be the basis of the comb weight extraction. The following multiple linear regression model is constructed to extract the comb weight: M=λ+μV hand +s In the formula, V hand σ represents the volume vector of the resultant, λ and μ represent the regression coefficient vectors of the multiple linear regression model, and σ represents the error term vector of the multiple linear regression model. For the banana fruit comb's inner and outer row fruit finger images, after removing small areas in step S31, the number of all connected components is counted. This yields the fruit indexes n1 and n2 for the inner and outer rows of the banana fruit comb, respectively. Based on the calculated fruit comb weight m, the fruit index N per kilogram of banana fruit comb is:

Images