Navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm

Abstract

This invention discloses a method for estimating the grasping pose of navel oranges based on multi-feature segmentation and a visual hedgehog algorithm, belonging to the field of computer vision technology. The method includes: acquiring color and depth images of the target navel oranges and performing spatial registration to generate a 3D point cloud; performing sphere fitting on the point clouds of individual navel orange instances, outputting the center coordinates and radius parameters of each orange, and using these parameters as the geometric data of the orange; performing collision detection and visibility analysis on the candidate grasping pose set to generate collision cost and visibility cost; comprehensively optimizing the collision cost and visibility cost through a multi-objective optimization function to generate the optimal grasping pose; and driving a robotic arm to perform the grasping operation based on the optimal grasping pose. This invention achieves accurate estimation of the grasping pose of navel oranges by combining multi-feature segmentation and the visual hedgehog algorithm.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for estimating the pose of navel orange grasping based on multi-feature segmentation and the visual hedgehog algorithm. Background Technology

[0002] With the continuous development of artificial intelligence, machine vision and agricultural automation technologies, fruit recognition and automatic grasping methods based on computer vision have become an important research direction for modern intelligent fruit picking equipment. In recent years, automatic fruit grasping technology combining visual sensing and robotic arm motion control has gradually emerged. Convolutional neural networks are usually used to identify and detect fruits, and methods such as color space segmentation and region growing are used to extract fruit regions, and then three-dimensional modeling and spatial positioning are achieved based on point cloud data.

[0003] However, existing methods still have the following shortcomings: In the fruit segmentation stage, color space segmentation methods mostly use the RGB model, but it is difficult to achieve accurate segmentation in complex natural scenes. Although deep learning-based instance segmentation methods can improve robustness, they require a large amount of labeled data and high-performance computing resources. In terms of 3D point cloud fitting and grasping posture estimation, existing sphere fitting methods are prone to fitting bias when there are many noisy points. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm to solve the problem that existing sphere fitting methods are prone to fitting deviation when there are many noise points.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a method for estimating the pose of a navel orange grasping object based on multi-feature segmentation and a visual hedgehog algorithm, comprising:

[0008] Acquire color and depth images of the target navel orange, perform spatial registration, and generate a 3D point cloud;

[0009] The 3D point cloud is projected onto the CIELAB color space and the navel orange region is initially segmented to generate a five-dimensional feature vector. The five-dimensional feature vector is then input into a hierarchical density clustering model. The cross-distance between the five-dimensional feature vectors is calculated and a hierarchical clustering tree is constructed to output the point cloud of independent navel orange instances.

[0010] Perform sphere fitting on the point cloud of an independent navel orange instance, and output the center coordinates and radius parameters of the sphere for each navel orange. Use the center coordinates and radius parameters of the sphere for each navel orange as the geometric data of the navel orange.

[0011] Spatial pose calculation is performed on the geometric data of navel oranges to construct a grasping coordinate system. Uniform sampling is performed on the surface of the navel oranges using the grasping coordinate system to generate a candidate set of grasping poses.

[0012] Collision detection and visibility analysis are performed on the candidate set of grasping poses to generate collision cost and visibility cost. The collision cost and visibility cost are comprehensively optimized by a multi-objective optimization function to generate the optimal grasping pose. Based on the optimal grasping pose, the robotic arm is driven to perform the grasping operation.

[0013] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for generating the 3D point cloud are as follows:

[0014] The color and depth images of the target navel orange are acquired using an RGB-D vision sensor, and image time synchronization is performed. The intrinsic parameter matrix of the RGB-D vision sensor is used to spatially register the synchronized color and depth images to generate pixel depth values.

[0015] Perform a back-projection transformation on the pixel depth values ​​to generate three-dimensional spatial coordinates;

[0016] A point set is constructed from the three-dimensional spatial coordinates to generate a three-dimensional point cloud.

[0017] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for generating the five-dimensional feature vector are as follows:

[0018] The color information in the 3D point cloud is converted to the CIELAB color space using a color space conversion method, and the luminance component and chrominance component are separated.

[0019] A color threshold is set based on the color distribution range in the chromaticity component. Color information is then filtered using the color threshold to generate a point cloud of the navel orange color feature region.

[0020] A preliminary point cloud of the navel orange region is generated by performing connected component analysis on the point cloud of the color feature region of the navel orange using the luminance component.

[0021] Spatial coordinates, curvature, normals, and normal change rates are extracted from the point cloud of the initial navel orange region using the point cloud geometric feature extraction method and combined into a five-dimensional feature vector.

[0022] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for outputting independent navel orange instance point clouds are as follows:

[0023] The five-dimensional feature vectors are input into the hierarchical density clustering model. The cross distance between the five-dimensional feature vectors is calculated. The cross distance is then used for single-link hierarchical clustering, and a hierarchical clustering tree is constructed.

[0024] Perform cluster stability determination on the hierarchical clustering tree and output the point cloud of independent navel orange instances.

[0025] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps of using the center coordinates and radius parameters of each navel orange as its geometric data are as follows:

[0026] A random sampling consensus algorithm is executed on the point cloud of an independent navel orange instance to generate initial parameters for the sphere.

[0027] The initial parameters of the sphere are iteratively solved by least squares optimization to generate the coordinates of the sphere's center and the radius parameters, which are then defined as the geometric data of the navel orange.

[0028] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the construction of the grasping coordinate system refers to obtaining the normal direction of the navel orange surface, constructing an orthogonal basis between the normal direction and the center coordinate of the sphere through the direction basis vector of the world coordinate system, and outputting the grasping coordinate system.

[0029] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for generating the grasping pose candidate set are as follows:

[0030] Using the capture coordinate system as the reference coordinate system, uniform sampling is performed on the surface of the navel orange to generate multiple distribution points;

[0031] The visual hedgehog algorithm is used to calculate the gripping posture of each distribution point by the normal direction and the gripper opening and closing direction, and generate multiple gripping direction combinations.

[0032] Multiple distribution points are filtered by the spatial distribution range of the navel orange surface. Distribution points located in the shading area are removed, and the valid distribution points are output.

[0033] The effective distribution points are combined with multiple grasping directions and encapsulated into a grasping pose candidate set.

[0034] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for generating collision cost and visibility cost are as follows:

[0035] The collision cost is generated by performing a segmented penalty transformation on the candidate set of captured poses using a collision detection method.

[0036] The candidate set of grab poses is cosine mapped using the gaze analysis method to generate a visibility cost.

[0037] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the specific steps for generating the optimal grasping pose are as follows:

[0038] The collision cost and visibility cost are input into a multi-objective optimization function to perform iterative optimization and generate a comprehensive cost.

[0039] The optimal grasping pose is selected from the candidate grasping pose set, which has the lowest overall cost.

[0040] As a preferred embodiment of the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm described in this invention, the method of driving the robotic arm to perform grasping operation refers to converting the optimal grasping pose into the robotic arm base coordinate system through the hand-eye calibration matrix, extracting the position and attitude from the converted optimal grasping pose, generating six-dimensional grasping parameters, and driving the robotic arm to perform grasping operation through the six-dimensional grasping parameters.

[0041] The beneficial effects of this invention are as follows: By performing time synchronization and spatial registration operations, color information and depth information are ensured to correspond in the same coordinate system; by combining random sampling consensus algorithm with least squares optimization, the accuracy and stability of grasping position calculation are improved; by calculating spatial posture and defining the opening and closing direction of the gripper, a combination of grasping directions is generated, realizing diversified expression of grasping directions; by calculating the minimum distance between the gripper envelope and the environmental point cloud using collision detection method, the grasping safety is quantitatively assessed; by extracting three-dimensional position and three-dimensional posture information through posture calculation, the robotic arm can achieve high-precision grasping along the optimal path; and by combining multi-feature segmentation with the visual hedgehog algorithm, the grasping posture of the navel orange is accurately calculated. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart of a navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm.

[0044] Figure 2 A flowchart for generating a 3D point cloud.

[0045] Figure 3 A flowchart for generating five-dimensional feature vectors.

[0046] Figure 4 The flowchart for generating a candidate set of poses. Detailed Implementation

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0050] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for estimating the pose of a navel orange grasping based on multi-feature segmentation and a visual hedgehog algorithm, including the following steps:

[0051] S1: Acquire color and depth images of the target navel orange, perform spatial registration, and generate a 3D point cloud.

[0052] S1.1: Acquire color and depth images of the target navel orange using an RGB-D vision sensor, perform image time synchronization, and spatially register the synchronized color and depth images using the intrinsic parameter matrix of the RGB-D vision sensor to generate pixel depth values.

[0053] Color and depth images of the target navel orange are acquired using an RGB-D vision sensor. The sampling time signal of the RGB-D vision sensor is synchronized and calibrated so that the color and depth images correspond to the same field of view at the same timestamp, resulting in time-synchronized color and depth images. Index matching is performed on the time-synchronized color and depth images to match the pixels of the color image with the pixels of the depth image one-to-one, resulting in synchronized color and depth images. Spatial registration processing is performed on the synchronized color and depth images using the intrinsic parameter matrix of the RGB-D vision sensor. Specifically, the pixels of the depth image are geometrically corrected using the focal length and principal point offset parameters in the intrinsic parameter matrix to align the pixels of the color image with the pixels of the depth image in spatial position. The pixels of the depth image corresponding to the pixels of the color image are read and associated to form a pixel set with depth attributes. All pixel sets are processed to remove pixels with invalid or abnormal depth values, generating pixel depth values.

[0054] The expression for the intrinsic parameter matrix is:

[0055] ;

[0056] in, This is the intrinsic parameter matrix. This refers to the focal length parameter in the horizontal direction. This refers to the focal length parameter in the vertical direction. Horizontal principal point offset parameter, This is the principal point offset parameter in the vertical direction.

[0057] The expression for the pixel depth value is:

[0058] ;

[0059] in, The pixel coordinates in the color image are The pixel depth value, This represents the pixel depth value in the depth image that corresponds to the pixel in the color image.

[0060] S1.2: Perform a back projection transformation on the pixel depth value to generate three-dimensional spatial coordinates.

[0061] The depth values ​​of all pixels are obtained from the synchronized color image and depth image. All pixel depth values ​​are paired with the two-dimensional coordinates in the color image. Using the focal length parameter and principal point offset parameter in the intrinsic parameter matrix, the two-dimensional coordinates in the paired color image and all pixel depth values ​​are back-projected to map the two-dimensional coordinates in each color image to three-dimensional space to obtain the corresponding three-dimensional space coordinates.

[0062] S1.3: Construct a point set from the three-dimensional spatial coordinates to generate a three-dimensional point cloud.

[0063] All 3D spatial coordinates are paired with their corresponding pixel depth values ​​to generate a set of 3D coordinate points. This set is then sorted and filtered by spatial location to remove outliers and incomplete 3D spatial coordinates, such as those with invalid, out-of-limit, or discontinuous pixel depth values. Valid 3D spatial coordinates are generated and organized using density clustering based on their spatial relationships. Specifically, a distance threshold is set based on the Euclidean distance between valid 3D spatial coordinates, typically ranging from 0.5cm to 5cm, with the threshold value determined by the size and shape of the navel orange. The range of values, selected as 0.5cm-5cm, effectively distinguishes different parts of the navel orange, avoiding incorrect clustering results due to noise interference and errors. The distance between adjacent valid 3D spatial coordinates is obtained. Adjacent valid 3D spatial coordinates with a distance less than the threshold are grouped into the same cluster, while adjacent valid 3D spatial coordinates with a distance greater than the threshold are separated. Adjacent valid 3D spatial coordinates with distances greater than and less than the threshold are divided into two clusters. Through homogenization processing, the valid 3D spatial coordinates within each cluster are spatially optimized, redundant points are removed, sparse regions are filled, and a representative 3D point cloud is generated.

[0064] The distance expression between adjacent valid three-dimensional spatial coordinates is:

[0065] ;

[0066] in, The distance between adjacent valid three-dimensional spatial coordinates. For the first The effective three-dimensional spatial coordinates are located on the x-axis in three-dimensional space. For the first The effective three-dimensional spatial coordinates are located at the y-axis position in three-dimensional space. For the first The effective three-dimensional spatial coordinates are located at the z-axis position in three-dimensional space. In order to be with the first The effective three-dimensional spatial coordinates of each adjacent coordinate are located on the x-axis in three-dimensional space. In order to be with the first The effective 3D spatial coordinates of each adjacent coordinate are located on the y-axis in 3D space. In order to be with the first The effective three-dimensional spatial coordinates of each adjacent coordinate are located on the z-axis in three-dimensional space.

[0067] S2: Project the 3D point cloud onto the CIELAB color space and perform preliminary navel orange region segmentation to generate a five-dimensional feature vector. Input the five-dimensional feature vector into the hierarchical density clustering model, calculate the cross-distance between the five-dimensional feature vector and construct the hierarchical clustering tree, and output the point cloud of independent navel orange instances.

[0068] S2.1: The color information in the 3D point cloud is converted to the CIELAB color space through the color space conversion method, and the luminance component and chrominance component are separated.

[0069] The color information, such as RGB color values, of each 3D coordinate point is extracted from the 3D point cloud using a color extraction method. The color information of each 3D coordinate point is then paired with its corresponding 3D spatial coordinates to form a 3D point cloud with color information. Using a standard color space conversion method, the color information in each 3D point cloud is converted into the luminance and chrominance components in the CIELAB color space.

[0070] S2.2: Set a color threshold based on the color distribution range in the chromaticity component, filter the color information using the color threshold, and generate a point cloud of the navel orange color feature region; perform connected component analysis on the point cloud of the navel orange color feature region using the luminance component to generate a preliminary point cloud of the navel orange region.

[0071] A color threshold is set based on the color distribution range of the navel orange in the chromaticity component, typically ranging from -60 to 60. This range effectively distinguishes different parts of the navel orange, avoiding incorrect clustering results due to noise interference and errors. It effectively filters out the color feature regions of the navel orange, avoids background noise interference, and improves accuracy. The color threshold is used to filter the color information in the 3D point cloud, removing 3D coordinate points whose color information does not meet the threshold, thus generating a point cloud of the navel orange's color feature regions.

[0072] Connectivity analysis is performed on the changes in brightness components. This involves comparing the spatial distance and brightness error between adjacent 3D coordinate points to determine whether they belong to the same connected region. Specifically, a brightness threshold is set based on the brightness variation characteristics of the navel orange surface. The typical value range for the brightness threshold is 2 to 10. Using a value range of 2 to 10 can effectively distinguish between normal brightness variations and areas of abnormal illumination on the navel orange surface, avoiding interference from sudden brightness changes caused by reflections or shadows on the connected region analysis results. This maintains the continuity and integrity of the navel orange region, thereby improving the accuracy of the initial point cloud extraction of the navel orange region. If adjacent 3D coordinates... If the spatial distance between points is less than a set distance threshold and the brightness error is lower than a set brightness threshold, then consecutive three-dimensional coordinate points are grouped into the same connected region. Conversely, if the spatial distance between adjacent three-dimensional coordinate points is greater than the distance threshold and the brightness error is higher than the brightness threshold, then adjacent three-dimensional coordinate points are determined to belong to different connected regions, and region merging is not performed. All three-dimensional coordinate points in the navel orange color feature region point cloud are traversed, and all interconnected connected regions are gradually merged to generate a preliminary navel orange region point cloud containing the main navel orange region. The preliminary navel orange region point cloud contains the main navel orange region, and surrounding noise points and irrelevant points have been removed.

[0073] S2.3: Extract spatial coordinates, curvature, normals, and normal change rates from the initial point cloud of the navel orange region using the point cloud geometric feature extraction method, and combine them into a five-dimensional feature vector.

[0074] The point cloud of the initial navel orange region is analyzed using a point cloud coordinate extraction method. The spatial coordinates of each 3D coordinate point are read and integrated into a 3D coordinate point set. A neighborhood search method is used to determine the neighborhood range of each 3D coordinate point in the set. Using each 3D coordinate point as the center, adjacent 3D coordinate points are extracted with a fixed search radius to generate a local point set. The fixed search radius is set based on point cloud density, object size, and noise levels, typically ranging from 0.5cm to 10cm. This range is determined according to the point cloud sampling resolution and the surface feature scale of the navel orange. Using a range of 0.5cm to 10cm preserves the geometric shape during the extraction of normals, curvature, and rate of change of normals. To ensure the stability and continuity of features, avoiding insufficient neighborhood points due to excessively small radii and boundary ambiguity caused by excessively large radii, a least-squares plane fitting method is used to fit the spatial distribution of neighborhood points in the local point set. This generates a normal direction vector corresponding to each 3D coordinate point. Specifically, each 3D coordinate point in the local point set is used as the center point, and adjacent 3D coordinate points within a fixed search radius are selected as neighborhood points. The spatial coordinate data of the neighborhood points are extracted and combined to form a neighborhood coordinate set. Least-squares fitting is performed on the neighborhood coordinate set to fit a planar shape that can represent local geometric features. The plane normal direction is extracted based on the geometric orientation of the planar shape, and the normal direction is used as the normal direction vector of the center point. The normal direction vector is normalized using a vector normalization method to ensure that the normal direction is independent of spatial scale, generating a normal direction vector corresponding to each 3D coordinate point. This normal direction vector is then associated with its corresponding 3D coordinate point to generate a set of 3D coordinate points with normals. A curvature extraction method is used to perform eigenvalue decomposition on the neighborhood points of each 3D coordinate point, generating a set of 3D coordinate points with normals and curvature. Specifically, for each 3D coordinate point, neighborhood points within a fixed search radius are selected, and principal component analysis is used to extract the neighborhood points, outputting principal direction features. The difference ratio between the principal direction features is calculated, and the curvature value of the local surface is output. The difference ratio of the principal direction features is used to characterize the neighborhood. The degree of change of a point in the spatial distribution direction is considered; the greater the degree of change, the higher the curvature value. The curvature value of the local surface is associated with the corresponding 3D coordinate point and normal direction vector to generate a set of 3D coordinate points with normal and curvature. An angle difference analysis is performed on the normal direction of adjacent 3D coordinate points by extracting the normal change rate. Specifically, the normal direction vectors of adjacent 3D coordinate points are read and the change range of the angle between the normal direction vectors is obtained. The change range of the angle is used as the normal change rate to characterize the degree of change of the normal direction of the local surface. A set of 3D coordinate points with normal change rate, normal and curvature is generated. The spatial coordinates, curvature, normal and normal change rate of each 3D coordinate point are combined to generate a five-dimensional feature vector.

[0075] The expression for the five-dimensional eigenvector is:

[0076] ;

[0077] in, It is a five-dimensional feature vector. For the first The characteristic coordinate components of a three-dimensional coordinate point along the x-axis in three-dimensional space. For the first The characteristic coordinate components of a three-dimensional coordinate point along the y-axis in three-dimensional space. For the first The characteristic coordinate components of a three-dimensional coordinate point along the y-axis in three-dimensional space. For the first The curvature of a three-dimensional coordinate point For the first Rate of change of the normal at each three-dimensional coordinate point.

[0078] The expression for the rate of change of the normal is:

[0079] ;

[0080] in, For the first The normal direction vector of a three-dimensional spatial coordinate. In order to be with the first The normal direction vectors of three-dimensional spatial coordinates adjacent to each other.

[0081] S2.4: Input the five-dimensional feature vectors into the hierarchical density clustering model, calculate the cross-distance between the five-dimensional feature vectors, perform single-link hierarchical clustering on the cross-distance, and construct a hierarchical clustering tree.

[0082] All five-dimensional feature vectors are input into a hierarchical density clustering model, which consists of a cross-distance calculation layer, a density aggregation layer, a single-link layer, and a clustering tree generation layer. In the cross-distance calculation layer, the feature distance between any pair of five-dimensional feature vectors is calculated using a cross-distance calculation method. Specifically, the feature distance between the five-dimensional feature vectors is calculated using the Euclidean distance formula; a neighborhood search is performed on each five-dimensional feature vector using a nearest neighbor search method to generate a list of core distances corresponding to all five-dimensional feature vectors; and the maximum value of the feature distance between each pair of five-dimensional feature vectors and the corresponding core distances at both ends is combined to generate the cross-distance between each pair of five-dimensional feature vectors. In the density aggregation layer, all cross-distances are... The system fills the matrix with symmetric elements based on index positions and sets the diagonals to zero. It then performs density aggregation on the five-dimensional feature vectors to generate density-connected clusters. In the single-link layer, it performs layer-by-layer merging on these density-connected clusters using a single-link hierarchical clustering method. It merges the closest five-dimensional feature vector clusters according to the minimum reach distance and records the clustering level information for each merging operation, generating hierarchical clustering data containing multi-level aggregation relationships. In the clustering tree generation layer, it hierarchically organizes the hierarchical clustering data using a hierarchical structure construction method. It constructs a hierarchical clustering tree according to the merging order of the clusters, connecting the upper-level cluster nodes generated by each merge with the corresponding lower-level cluster nodes, thus generating a hierarchical clustering tree containing the hierarchical relationships of the five-dimensional feature vectors.

[0083] S2.5: Perform cluster stability determination on the hierarchical clustering tree and output the point cloud of independent navel orange instances.

[0084] All clustering nodes in the hierarchical clustering tree are traversed using a hierarchical traversal method. The five-dimensional feature vector corresponding to each clustering node is extracted to form a clustering node set. The stability of each clustering node in the set is evaluated using a cluster stability criterion. Specifically, the stability value of each clustering node in the hierarchical structure is calculated by the rate of change of the number of five-dimensional feature vectors and the rate of change of cluster hierarchy density in each clustering node. All clustering node stability values ​​are integrated to form a stability dataset. A stability threshold is set based on the rate of change of the number of samples and the rate of change of density of each clustering node in the hierarchical clustering tree, typically ranging from 0.5 to 0.9. The value range is set based on the density distribution of the point cloud on the navel orange surface and the stability of the cluster hierarchy, using a range of 0.5 to 0.9. The value range of 9 can effectively distinguish between stable navel orange clusters and unstable noise clusters, avoiding over-segmentation and under-segmentation, and improving the recognition accuracy and spatial coherence of independent navel orange instance point clouds. The stability dataset is filtered by a stability threshold. When the stability value of a cluster node in the stability dataset is higher than the stability threshold, the cluster node is determined to be a stable cluster. When the stability value of a cluster node is lower than the stability threshold, the cluster node is determined to be an unstable cluster and is removed. The output is a stability dataset containing all stable clusters. The five-dimensional feature vector corresponding to each stable cluster in the stability dataset is mapped to three-dimensional coordinate points through a clustering assignment method to generate navel orange instance point clouds corresponding to spatial locations. The navel orange instance point clouds corresponding to all stable clusters are merged to output independent navel orange instance point clouds that do not overlap.

[0085] The expression for the stability value of cluster nodes is:

[0086] ;

[0087] in, For the first Stability value of each cluster node For the first Rate of change of the number of five-dimensional feature vectors in each cluster node For the first The rate of change of cluster hierarchy density of each cluster node The weighting parameter is the rate of change of quantity. This is a weighting parameter for the rate of change of cluster hierarchy density, used to adjust the influence of the rate of change of number and the rate of change of cluster hierarchy density on the stability value of cluster nodes.

[0088] S3: Perform spherical fitting on the point cloud of independent navel orange instances, output the center coordinates and radius parameters of each navel orange, and use the center coordinates and radius parameters of each navel orange as the geometric data of the navel orange.

[0089] S3.1: Execute the random sampling consensus algorithm on the point cloud of independent navel orange instances to generate the initial parameters of the sphere.

[0090] Multiple 3D coordinate point samples were randomly selected from the point cloud of independent navel orange instances using a point cloud random sampling method, and combined to form a candidate sample point set. Four non-coplanar 3D coordinate points were randomly selected from the candidate sample point set using a minimum sample set fitting method. The spatial relationship between the four non-coplanar 3D coordinate points was geometrically analyzed. Specifically, the spatial distance from each of the four non-coplanar 3D coordinate points to the center of the sphere was calculated using the Euclidean distance calculation method, generating four distance expressions. The four distance expressions were then constrained with the radius parameter using an equidistant condition definition method, generating four distance equality constraints. These four distance equality constraints were then standardized using equation normalization. After sorting, four spatial constraint equations are obtained. These equations are then parameter-normalized to ensure consistency of unknown parameters. Using an equation assembly method, the four equations are combined according to the order of their corresponding unknown parameters, forming a linear system of equations containing the coordinates of the sphere's center and the radius parameter. This linear system simultaneously satisfies the geometric condition that four three-dimensional coordinate points are equidistant from the sphere's center. Solving this linear system using a linear solution method determines the optimal parameter solution that satisfies the four spatial constraint equations, yielding the sphere's center coordinates and radius parameter. Substituting these parameters into the standard form of the spherical equation, the sphere's coordinates and radius parameter are obtained. The four coefficient parameters of the surface equation characterize the initial shape of the currently fitted sphere, corresponding to the three components of the sphere's center coordinates and a constant term. The position coordinates of the sphere's center in three-dimensional space are determined by using geometric inverse calculation relationships through the equation coefficients. The sphere's radius parameter is obtained based on the geometric relationship between the sphere's center position and the constant term. The sphere's center coordinates and radius parameter are used as parameters of the spherical surface equation to form the sphere assumption parameters. The geometric distance from the remaining three-dimensional coordinate points in the point cloud of the independent navel orange instance to the sphere is calculated using a point cloud consistency judgment method. The geometric distance threshold is set based on the geometric residual distribution of the fitting results from each three-dimensional coordinate point in the three-dimensional point cloud to the sphere, and the value range is usually 0. The value range of 0.5mm to 2.0mm is set based on the spatial resolution of the navel orange point cloud and the statistical results of the fitting error. The range of 0.5mm to 2.0mm can effectively distinguish between spherical structure points and abnormal noise points, and can achieve a balance between fitting accuracy and fault tolerance. Three-dimensional coordinate points with a geometric distance less than the geometric distance threshold are determined as interior points, and three-dimensional coordinate points with a geometric distance greater than the geometric distance threshold are determined as exterior points. All interior points are combined to form an interior point set. The number of interior points in each sphere fitting is counted by the interior point number statistics method, and the corresponding sphere assumption parameters are recorded. The set of sphere assumption parameters with the most interior points is determined as the initial parameters of the sphere.

[0091] The expression for the spatial constraint equation is:

[0092] ;

[0093] in, This represents the position coordinates of the center of the navel orange ball along the x-axis in three-dimensional space. This represents the position coordinates of the center of the navel orange ball along the y-axis in three-dimensional space. This represents the position coordinates of the center of the navel orange ball along the z-axis in three-dimensional space. Let be the radius of the navel orange.

[0094] S3.2: The initial parameters of the sphere are iteratively solved by least squares optimization to generate the coordinates of the sphere's center and the radius parameters, and the coordinates of the sphere's center and the radius parameters are defined as the geometric data of the navel orange.

[0095] The least squares optimization method is used with the center coordinates and radius parameters as optimization variables. The error of the geometric deviation of each interior point relative to the surface of the sphere is calculated, and the errors of all interior points are squared and accumulated to generate the overall error. The center coordinates and radius parameters are continuously updated through the iterative solution process. In each iteration, the parameters are continuously adjusted by the overall error to gradually reduce the overall error. When the overall error no longer decreases in multiple iterations (e.g., 10 iterations) and the maximum number of iterations (e.g., 50 iterations) is reached, the iteration is terminated. The center coordinates and radius parameters at the time of termination are taken as the final optimization result. The changes in the center coordinates generated during the continuous iteration are smoothed and corrected by the parameter smoothing method. The center coordinates and radius parameters are output and defined as the geometric data of the navel orange.

[0096] S4: Calculate the spatial pose of the navel orange geometric data, construct a grasping coordinate system, and uniformly sample the navel orange surface using the grasping coordinate system to generate a candidate set of grasping poses.

[0097] S4.1: Obtain the normal direction of the navel orange surface, construct an orthogonal basis between the normal direction and the sphere center coordinates using the direction basis vector of the world coordinate system, and output the grab coordinate system.

[0098] The system reads the center coordinates and radius parameters from the navel orange's geometric data, establishes a spherical geometric coordinate relationship using the center coordinates as a reference coordinate system, and obtains the normal direction of any sampling point on the navel orange surface through the surface normal analysis method. Specifically, the line connecting the center coordinates and the spatial coordinates of the sampling point on the navel orange surface is used as the normal direction vector. Each normal direction vector is normalized using the direction vector normalization method to ensure that the normal direction is independent of the spatial scale, generating a unit normal direction vector. The unit normal direction vector is then orthogonalized to the direction basis vectors of the world coordinate system using the vector orthogonal calculation method, calculating two direction basis vectors perpendicular to the unit normal direction vector, generating three mutually orthogonal direction vectors. The center coordinates of the navel orange are used as the origin, the unit normal direction vector is used as the approach direction of the grasping coordinate system, and the two direction basis vectors orthogonal to the unit normal direction vector are used as the attitude basis vectors of the grasping coordinate system, forming a three-dimensional grasping coordinate system.

[0099] S4.2: Using the capture coordinate system as the reference coordinate system, perform uniform sampling on the surface of the navel orange to generate multiple distribution points.

[0100] The spherical parametric sampling method is used to parametrically expand the coordinates of the sphere's center and the radius parameters. With the origin of the grasping coordinate system as the center and the radius of the sphere as the radius range, the surface of the navel orange is mapped into a parametric region composed of meridians and parallels. A fixed angular interval is set in the meridian and parallel directions using the equal-angle interval sampling method. The spatial position coordinates in the grasping coordinate system are calculated for each intersection point of the angular interval to generate an initial distribution point set. The spatial distance constraint method is used to check the spatial distance between adjacent distribution points in the initial distribution point set. Specifically, when the distance between adjacent distribution points is less than the preset interval, point merging is performed; when the distance between adjacent distribution points is greater than the preset interval, point interpolation is performed to generate a distribution point set with uniform spacing. The preset interval is a three-dimensional spatial sampling distance set based on the radius of the navel orange and the curvature of the sphere to ensure that the sampling points are evenly distributed in three-dimensional space. The local coordinates of each distribution point in the evenly spaced distribution point set are transformed to the three-dimensional spatial coordinates of the grasping coordinate system through coordinate transformation. Multiple distribution points aligned with the grasping coordinate system are output, so that all distribution points maintain a spatial correspondence with the grasping coordinate system.

[0101] S4.3: The visual hedgehog algorithm is used to calculate the gripping posture of the normal direction and the gripper opening and closing direction of each distribution point, and generate multiple gripping direction combinations.

[0102] The normal direction of each distribution point is analyzed using the surface normal analysis method. The direction of the line connecting the center coordinates of the navel orange sphere and the spatial coordinates of the distribution point is taken as the normal direction of the distribution point. The normal direction of the distribution point is normalized using a direction vector normalization method to generate a unit normal direction vector. A hedgehog-like grasping direction field is then formed on the sphere using the unit normal direction vectors of all distribution points. The spatial attitude parameters of each distribution point are calculated using an attitude angle estimation method. Specifically, the initial opening and closing direction of the gripper is obtained by combining the unit normal direction vector with the attitude basis vectors of the grasping coordinate system, generating the initial attitude angle of each distribution point. This is then processed using RGB-D visual transmission. The sensor's field-of-view constraint information and visibility correction method are used to adjust the initial attitude angle of each distribution point, ensuring visibility and feasibility while maintaining normal consistency. A corrected attitude angle is generated for each distribution point, and spatial constraints are applied to this corrected attitude angle using an attitude angle constraint method. This limits the angle range between the gripper's opening / closing direction and the normal direction, ensuring the physical feasibility of the grasping posture. Attitude angle parameters that meet mechanical grasping requirements are generated. The unit normal direction vector of each distribution point is combined with the attitude angle parameters to generate multiple grasping direction combinations. The visual hedgehog algorithm is then used to optimize the global uniformity and local visibility of the grasping posture.

[0103] S4.4: Filter multiple distribution points based on the spatial distribution range of the navel orange surface, remove distribution points located in the shading area, and output the valid distribution points.

[0104] The spatial boundary range of the navel orange surface in the grasping coordinate system is determined by the spatial boundary calculation method. Specifically, with the center coordinates of the navel orange as the center and the radius of the navel orange as the radius, an outer spherical region is formed in three-dimensional space by combining the boundary offset coefficient. This region is used to limit the effective spatial range of the distribution points. The spatial range screening method is used to determine the spatial position of multiple distribution points. Distribution points that exceed the outer spherical region are determined as invalid distribution points, while distribution points that are located within the outer spherical region are retained as candidate distribution points. The occlusion detection method is used to determine whether each distribution point has spatial occlusion. Specifically, with the origin of the grasping coordinate system as the visual reference point, a detection ray is emitted along the normal direction of each distribution point. When the detection ray intersects with the point cloud of other objects before reaching the distribution point, it is determined as an occlusion point. The occlusion elimination method is used to remove all distribution points determined as occlusion points from the candidate distribution points, generating effective distribution points that only contain distribution points in the unoccluded area.

[0105] It should be noted that the boundary offset coefficient refers to the geometric compensation coefficient used to correct the edge radius error of the navel orange point cloud. It is used to perform outward calculation based on the spherical radius to construct an outer spherical region containing all effective points on the navel orange surface. The boundary offset coefficient is obtained by the radius deviation ratio between the radius parameter and the actual point cloud boundary radius. The value range is usually 1.05 to 1.15. Using 1.05 to 1.15 can effectively compensate for the problem of the boundary radius being too small due to noise, occlusion or reflection in the point cloud scanning.

[0106] The radius expression for the outer spherical region is:

[0107] ;

[0108] in, Let be the radius of the spherical region. This is the boundary offset coefficient.

[0109] The expression for the bounding spherical region is:

[0110] ;

[0111] in, This represents the position coordinates of any point in three-dimensional space along the x-axis. This represents the position coordinates of any point in three-dimensional space along the y-axis. This represents the position coordinates of any point in three-dimensional space along the z-axis.

[0112] S4.5: Combine the effective distribution points with multiple grasping directions and encapsulate them into a grasping pose candidate set.

[0113] Using a spatial pose matching method, the spatial coordinates of each effective distribution point are used as the position reference, the corresponding normal direction as the approach direction, and the gripper opening and closing direction as the pose reference direction. The position parameters and direction parameters are matched one by one to generate the initial grasping pose corresponding to each effective distribution point. The pose parameter normalization method is used to standardize the pose angles of all grasping directions in the initial grasping pose to ensure that the rotation angle range of the grasping pose is consistent and that the grasping direction maintains a unified reference direction in three-dimensional space, generating a normalized grasping pose set. Using a pose encapsulation method, the spatial position parameters and corresponding pose parameters of each effective distribution point in the normalized grasping pose set are structurally combined to generate a grasping pose parameter set containing position and direction information. Using a candidate set construction method, all grasping pose parameter sets are indexed, sorted, and encapsulated to form a grasping pose candidate set with a multi-pose parallel structure.

[0114] S5: Perform collision detection and visibility analysis on the candidate set of grasping poses to generate collision cost and visibility cost. Optimize the collision cost and visibility cost through a multi-objective optimization function to generate the optimal grasping pose. Based on the optimal grasping pose, drive the robotic arm to perform the grasping operation.

[0115] S5.1: The candidate set of captured poses is transformed by segmentation penalty using collision detection method to generate collision cost.

[0116] The system reads the spatial position and attitude parameters of each grasping pose candidate set and loads the point cloud data of the surrounding environment of the navel orange. Using a spatial bounding volume construction method, a spatial bounding volume of the gripper is constructed with the position parameters of each grasping pose as the center point and based on the gripper size and attitude parameters. Using a collision detection method, the minimum spatial distance between each gripper spatial bounding volume and the environmental point cloud is calculated. Using a piecewise penalty transformation method, the minimum spatial distance is mapped to a cost. When the spatial distance increases, the penalty value approaches zero. As the distance decreases, the penalty value increases proportionally. When the gripper contacts the obstacle, the penalty value reaches its maximum, forming a continuous mapping relationship between distance and penalty value. The penalty value corresponding to each grasping pose is used as the collision cost weight. Weighted statistics are performed on all grasping poses in the grasping pose candidate set to generate a collision cost value that includes the safety risk level of each grasping pose.

[0117] S5.2: Cosine mapping is performed on the candidate set of grab poses using the gaze analysis method to generate a visibility cost.

[0118] The angle between the proximity direction vector of each grasping pose in the candidate grasping pose set and the gaze direction vector of the visual sensor is read using gaze analysis. The angle is then quantified for visibility using a cosine mapping method, converting the angle into a view similarity. Specifically, view similarity increases when the angle decreases and gradually decreases when the angle increases. A visibility cost calculation method is used, taking view similarity as input, and jointly evaluating view similarity and the spatial occlusion degree of the grasping pose. Specifically, a high visibility cost is assigned when either a decrease in view similarity or the presence of occlusion is met, and a low visibility cost is assigned when view similarity increases and there is no occlusion. This generates a visibility cost for each grasping pose. The visibility costs for all grasping poses are then aggregated to generate a visibility cost that includes visibility information for all grasping poses.

[0119] S5.3: Input the collision cost and visibility cost into the multi-objective optimization function to perform iterative optimization and generate a comprehensive cost.

[0120] Collision cost and visibility cost are jointly optimized using a multi-objective optimization function, with weight parameters defined. These weight parameters include collision weight parameters and visibility weight parameters, which are set using statistical methods based on sampled data. The weight parameters adjust the proportion of the impact of collision cost and visibility cost in the comprehensive cost function. The average and standard deviation of collision cost and visibility cost in the grasping pose set are calculated separately. The weight ratio is determined based on the ratio of the standard deviations of collision cost and visibility cost. When the standard deviation of collision cost is greater than that of visibility cost, the collision weight parameter is increased; when the standard deviation of visibility cost is greater than that of collision cost, the visibility weight parameter is increased. When the grasping environment has dense obstacles, the value of the collision weight parameter is 0. The visibility weight parameter ranges from 0.2 to 0.4. When the grasping environment is open, the collision weight parameter ranges from 0.3 to 0.5, and the visibility weight parameter ranges from 0.5 to 0.7. The range of weight parameters is set according to the scene complexity of the grasping task. The collision cost and visibility cost are combined into a comprehensive cost function. An iterative optimization method is used to gradually optimize the collision cost and visibility cost of each grasping pose. The parameters of each grasping pose are adjusted by the weighted sum of the comprehensive cost function so that the total cost value gradually decreases. When the change in the comprehensive cost satisfies either the condition of not changing for multiple consecutive rounds or reaching the maximum number of iterations, the optimization process is terminated, and the current optimal comprehensive cost is output.

[0121] S5.4: Select the grasping pose with the minimum overall cost from the grasping pose candidate set as the optimal grasping pose.

[0122] Read the comprehensive cost value corresponding to each grasping pose, sort the comprehensive costs of all grasping poses, and select the grasping pose with the smallest comprehensive cost value as the priority option in the candidate set as the optimal grasping pose, ensuring that the optimal grasping pose can achieve a balance between collision cost and visibility cost.

[0123] S5.5: The optimal grasping pose is converted into the robot arm base coordinate system through the hand-eye calibration matrix. The position and attitude are extracted from the converted optimal grasping pose to generate six-dimensional grasping parameters. The robot arm is driven to perform grasping operations through the six-dimensional grasping parameters.

[0124] The spatial correspondence between the visual coordinate system and the robotic arm base coordinate system is obtained through a hand-eye calibration matrix. The hand-eye calibration matrix consists of a rotation matrix and a translation vector, which is used to describe the pose transformation relationship between the visual sensor coordinate system and the robotic arm base coordinate system. Through coordinate transformation methods, the optimal grasping pose is transformed using the hand-eye calibration matrix, converting the optimal grasping pose in the visual coordinate system into the grasping pose in the robotic arm base coordinate system, generating the transformed optimal grasping pose. The position and attitude are extracted from the transformed optimal grasping pose using an attitude calculation method. The three-dimensional spatial position coordinates of the transformed grasping pose are used as position parameters, and the pitch angle, yaw angle, and roll angle of the grasping pose are used as attitude parameters. The position and attitude are combined to generate six-dimensional grasping parameters, which are then input into the robotic arm's control program. The robotic arm's control program executes trajectory planning and motion control, causing the robotic arm's end effector to move along the path indicated by the six-dimensional grasping parameters and complete the grasping operation.

[0125] In the experiment, the traditional single-feature segmentation method and the method of the present invention were used to test the grasping of the same batch of navel orange samples. The results showed that the present invention improved the integrity and boundary accuracy of navel orange region recognition through multi-feature fusion segmentation, which can effectively avoid the problem of false detection due to fruit adhesion. At the same time, the visual hedgehog algorithm realizes global uniform optimization of grasping posture and visual constraint correction, making the grasping path smoother and the distribution of gripper contact points more stable. The comprehensive statistical results show that the method of the present invention has achieved performance improvement in grasping accuracy and grasping stability compared with the traditional method, and can realize high-precision and high-robust grasping operation of navel oranges in complex orchard environments.

[0126] This embodiment also provides a computer device applicable to the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm proposed in the above embodiment.

[0127] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0128] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the orange grasping pose estimation method based on multi-feature segmentation and the visual hedgehog algorithm proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0129] In summary, this invention ensures that color and depth information correspond in the same coordinate system through time synchronization and spatial registration operations. It improves the accuracy and stability of grasping position calculation by combining a random sampling consensus algorithm with least squares optimization. It generates grasping direction combinations through spatial pose calculation and gripper opening / closing direction definition, enabling diverse expressions of grasping directions. It calculates the minimum distance between the gripper envelope and the environmental point cloud using collision detection, achieving a quantitative assessment of grasping safety. It extracts three-dimensional position and pose information through pose calculation, enabling the robotic arm to achieve high-precision grasping along the optimal path. Finally, it combines multi-feature segmentation with the visual hedgehog algorithm to accurately calculate the grasping pose of the navel orange.

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A navel orange grasping pose estimation method based on multi-feature segmentation and visual hedgehog algorithm, characterized in that: include, Acquire color and depth images of the target navel orange, perform spatial registration, and generate a 3D point cloud; The 3D point cloud is projected onto the CIELAB color space and the navel orange region is initially segmented to generate a five-dimensional feature vector. The five-dimensional feature vector is then input into a hierarchical density clustering model. The cross-distance between the five-dimensional feature vectors is calculated and a hierarchical clustering tree is constructed to output the point cloud of independent navel orange instances. The specific steps for generating the five-dimensional feature vector are as follows: The color information in the 3D point cloud is converted to the CIELAB color space using a color space conversion method, and the luminance component and chrominance component are separated. A color threshold is set based on the color distribution range in the chromaticity component. Color information is then filtered using the color threshold to generate a point cloud of the navel orange color feature region. A preliminary point cloud of the navel orange region is generated by performing connected component analysis on the point cloud of the color feature region of the navel orange using the luminance component. Spatial coordinates, curvature, normals, and normal change rates are extracted from the point cloud of the initial navel orange region using the point cloud geometric feature extraction method and combined into a five-dimensional feature vector. Perform sphere fitting on the point cloud of an independent navel orange instance, and output the center coordinates and radius parameters of the sphere for each navel orange. Use the center coordinates and radius parameters of the sphere for each navel orange as the geometric data of the navel orange. Spatial pose calculation is performed on the geometric data of navel oranges to construct a grasping coordinate system. Uniform sampling is performed on the surface of the navel oranges using the grasping coordinate system to generate a candidate set of grasping poses. The specific steps for generating the candidate set of grasping poses are as follows: Using the capture coordinate system as the reference coordinate system, uniform sampling is performed on the surface of the navel orange to generate multiple distribution points; The visual hedgehog algorithm is used to calculate the gripping posture of each distribution point by the normal direction and the gripper opening and closing direction, and generate multiple gripping direction combinations. Multiple distribution points are filtered by the spatial distribution range of the navel orange surface, and distribution points located in the shading area are removed to output the valid distribution points. The effective distribution points are combined with multiple grasping directions and encapsulated into a grasping pose candidate set; Collision detection and visibility analysis are performed on the candidate set of grasping poses to generate collision cost and visibility cost. The collision cost and visibility cost are comprehensively optimized by a multi-objective optimization function to generate the optimal grasping pose. Based on the optimal grasping pose, the robotic arm is driven to perform the grasping operation.

2. The navel orange grasp pose estimation method based on multi-feature segmentation and visual hedgehog algorithm according to claim 1, wherein: The specific steps for generating the 3D point cloud are as follows. The color and depth images of the target navel orange are acquired using an RGB-D vision sensor, and image time synchronization is performed. The intrinsic parameter matrix of the RGB-D vision sensor is used to spatially register the synchronized color and depth images to generate pixel depth values. Perform a back-projection transformation on the pixel depth values ​​to generate three-dimensional spatial coordinates; A point set is constructed from the three-dimensional spatial coordinates to generate a three-dimensional point cloud.

3. The navel orange grasp pose estimation method based on multi-feature segmentation and visual hedgehog algorithm according to claim 1, characterized in that: The specific steps for outputting independent navel orange instance point clouds are as follows. Input the five-dimensional feature vectors into the hierarchical density clustering model, calculate the cross distance between the five-dimensional feature vectors, perform single-link hierarchical clustering on the cross distances, and construct a hierarchical clustering tree. Perform cluster stability determination on the hierarchical clustering tree and output the point cloud of independent navel orange instances.

4. The method for estimating the pose of navel orange grasping based on multi-feature segmentation and visual hedgehog algorithm as described in claim 1, characterized in that: The specific steps for using the center coordinates and radius parameters of each navel orange as its geometric data are as follows. A random sampling consensus algorithm is executed on the point cloud of an independent navel orange instance to generate initial parameters for the sphere. The initial parameters of the sphere are iteratively solved by least squares optimization to generate the center coordinates and radius parameters, which are then defined as the geometric data of the navel orange.

5. The method for estimating the pose of navel orange grasping based on multi-feature segmentation and visual hedgehog algorithm as described in claim 1, characterized in that: The construction of the grasping coordinate system refers to obtaining the normal direction of the navel orange surface, constructing an orthogonal basis between the normal direction and the sphere center coordinates using the direction basis vector of the world coordinate system, and outputting the grasping coordinate system.

6. The method for estimating the pose of navel orange grasping based on multi-feature segmentation and visual hedgehog algorithm as described in claim 1, characterized in that: The specific steps for generating collision costs and visibility costs are as follows. The collision cost is generated by performing a segmented penalty transformation on the candidate set of captured poses using a collision detection method. The candidate set of grab poses is cosine mapped using the gaze analysis method to generate a visibility cost.

7. The method for estimating the pose of navel orange grasping based on multi-feature segmentation and visual hedgehog algorithm as described in claim 1, characterized in that: The specific steps for generating the optimal grasping pose are as follows. The collision cost and visibility cost are input into a multi-objective optimization function to perform iterative optimization and generate a comprehensive cost. The optimal grasping pose is selected from the candidate grasping pose set, which has the lowest overall cost.

8. The method for estimating the pose of navel orange grasping based on multi-feature segmentation and visual hedgehog algorithm as described in claim 1, characterized in that: The process of driving the robotic arm to perform a grasping operation refers to converting the optimal grasping pose into the robotic arm's base coordinate system through a hand-eye calibration matrix, extracting the position and attitude from the converted optimal grasping pose, generating six-dimensional grasping parameters, and driving the robotic arm to perform the grasping operation through the six-dimensional grasping parameters.

Images