A method for extracting grafting robot operation information based on semantic and geometric co-processing

Abstract

This invention provides a method for extracting grafting robot operation information based on semantic and geometric co-processing, comprising: extracting semantic and geometric features from seedling point clouds; performing bidirectional attention fusion on the semantic and geometric features to generate a complete point cloud with category labels; dividing the complete point cloud with category labels into a main stem point set and a stem node set; extracting main stem target parameters based on the main stem point set, and extracting stem node target parameters based on the stem node set. This invention avoids the cascading errors and misclassification problems caused by the mutual constraints of accuracy between complex perceptual information in serial step-by-step processing by simultaneously extracting semantic and geometric features and performing bidirectional attention fusion on them to generate a complete point cloud with category labels. This avoids the mutual interference of missing point clouds and poor point cloud segmentation results, making the subsequently divided main stem point set and stem node set more accurate, and greatly improving the accuracy of seedling operation information extraction.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for extracting grafting robot operation information based on semantic and geometric co-processing. Background Technology

[0002] With the development of agricultural automation, fully automatic grafting machines have effectively solved the reliance on manual labor in the seedling industry. In grafting operations, accurate perception of seedling information is crucial for the precise clamping and cutting by the actuator, typically achieved by acquiring 3D point cloud images of the seedlings. However, when acquiring these point cloud images, occlusion by the plant itself and between surrounding seedlings is inevitable, resulting in missing point cloud data. This leads to inaccurate identification of key seedling parts, making it difficult to effectively extract critical operational information.

[0003] To address the challenge of information extraction caused by missing point clouds, deep learning-based point cloud processing techniques are commonly employed. Existing conventional methods involve first using a point cloud completion network to fill in missing and occluded point clouds, followed by a point cloud segmentation network to classify different parts; or first performing point cloud segmentation, and then performing point cloud completion on each part. By using these step-by-step processing methods to obtain point clouds of different parts of the seedling, the required seedling operation parameters can be extracted.

[0004] However, the aforementioned existing technologies have significant drawbacks: in the step-by-step processing, completing before segmentation highly depends on the completion effect, while segmenting before completion highly depends on the segmentation effect. If the previous step in the sequential processing is ineffective, the error will be directly propagated and amplified, leading to misclassification or completion into the wrong category, resulting in larger errors when extracting parameters related to the main stem and nodes. Therefore, to improve the accuracy of seedling information extraction, how to avoid cascading errors in the point cloud segmentation and completion process to accurately extract seedling operation information has become an urgent problem to be solved in this field. Summary of the Invention

[0005] This invention provides a method for extracting grafting robot operation information based on semantic and geometric co-processing, which solves the technical problem of how to avoid cascade errors in the point cloud segmentation and completion process in order to accurately extract seedling operation information.

[0006] This invention provides a method for extracting grafting robot operation information based on semantic and geometric co-processing, including: Extract semantic and geometric features from seedling point clouds; The semantic features and the geometric features are fused by bidirectional attention to generate a complete point cloud with category labels. The complete point cloud with category labels is divided into a main stem point set and a stem node set; The target parameters of the main stem are extracted based on the main stem point set, and the target parameters of the stem nodes are extracted based on the stem node set.

[0007] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, which extracts semantic and geometric features of seedling point clouds, including: The seedling point cloud is downsampled and its features are aggregated layer by layer to obtain features at each level and a global feature vector; The semantic features are obtained by upsampling the features at each level layer by layer through feature propagation. The global feature vector is resized and mapped to coordinates to obtain the geometric features.

[0008] According to the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the present invention, the seedling point cloud is downsampled and feature aggregated layer by layer to obtain features at each level and a global feature vector, including: Local reference points are selected in the seedling point cloud, and the neighborhood data around the local reference points are extracted for feature encoding, and the features of each level are output. Max pooling is performed on the last layer of features in each layer to obtain the global feature vector.

[0009] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, which performs bidirectional attention fusion on the semantic features and the geometric features to generate a complete point cloud with category labels, including: Calculate the bidirectional attention weights between the semantic features and the geometric features to obtain the fused pointwise semantic features and the fused complete geometric features; The fused completed geometric features are upsampled and then concatenated with the fused point-by-point semantic features to obtain joint features; The joint features are encoded and mapped to three-dimensional coordinate data and class probabilities respectively. The class corresponding to the highest class probability is assigned as the class label to the three-dimensional coordinate data to generate the complete point cloud with class label.

[0010] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, wherein the joint features are feature-encoded and mapped to three-dimensional coordinate data and class probabilities respectively, including: During the training phase of the collaborative network, a completion segmentation fusion loss and a feature consistency loss are constructed based on the completion coordinate features and predicted segmentation features output by the collaborative network. The total loss is obtained by weighted summing of the completion segmentation fusion loss, the feature consistency loss, the segmentation loss, and the completion loss. The network parameters of the cooperative network are updated based on the total loss. The trained collaborative network maps the joint features to the 3D coordinate data and the class probabilities.

[0011] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, which constructs a completion segmentation fusion loss and a feature consistency loss based on the completion coordinate features and predicted segmentation features output by the co-processing network, including: Traverse the real point cloud and the completed point cloud corresponding to the completed coordinate features, and calculate the sum of squares of the minimum point pair distances in both directions as the completed segmentation fusion loss; The negative exponent of the Euclidean distance between any two predicted coordinates is calculated as the geometric similarity, and the cosine similarity of the predicted segmentation features between any two points is calculated as the semantic similarity. For each pair of points, the square of the difference between the geometric similarity and the semantic similarity is calculated, and the arithmetic mean of the squared differences of all point pairs is taken to obtain the feature consistency loss.

[0012] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, wherein the target parameters of the main stem include diameter and uprightness; Extracting target parameters of the main stem based on the main stem point set includes: The main stem point set is projected onto a horizontal plane to obtain horizontal projection points; Calculate the Gaussian weight vector of the horizontal projection point; Construct a minimum area bounding rectangle for the horizontal projection point based on the Gaussian weight vector; The diameter is determined to be the width of the minimum area circumscribed rectangle, and the verticality is the angle between the spatial vector of the long side of the minimum area circumscribed rectangle and the preset vertical coordinate axis.

[0013] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, which calculates the Gaussian weight vector of the horizontal projection point, including: Calculate the mean center coordinates of the horizontal projection points; Calculate the squared distance from each of the horizontal projection points to the mean coordinate of the center; The negative exponential mapping value obtained by dividing the squared distance by the variance parameter is used as the Gaussian weight vector.

[0014] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, which constructs a minimum area bounding rectangle for the horizontal projection point based on the Gaussian weight vector, including: Calculate the convex hull of the horizontal projection points to obtain a sequence of boundary vertices arranged in order; A local coordinate system is constructed along each convex hull edge of the boundary vertex sequence, and the weighted projection mean and weighted projection standard deviation of the horizontal projection point in the corresponding convex hull edge normal and tangential directions are calculated in combination with the Gaussian weight vector. The candidate rectangle boundaries are determined based on the weighted projection mean and the weighted projection standard deviation, and the candidate rectangle boundary with the smallest area is taken as the minimum area circumscribed rectangle.

[0015] According to the present invention, a method for extracting grafting robot operation information based on semantic and geometric co-processing is provided, wherein the target parameter of the stem segment includes the central spatial position; Based on the aforementioned stem node set, target parameters for stem nodes are extracted, including: If the total number of points in the stem node set is less than or equal to a preset minimum number of points threshold, then the average coordinate of each point in the stem node set is taken as the center spatial position. If the total number of points is greater than the minimum number of points threshold, a point cloud clustering algorithm is used to divide the stem node set into at least one main cluster point set; the target main cluster point set containing the most points is determined, and the average coordinate of the target main cluster point set is used as the center spatial position.

[0016] The grafting robot operation information extraction method provided by this invention extracts semantic and geometric features simultaneously and performs bidirectional attention fusion to generate a complete point cloud with category labels. This avoids the error cascading and misclassification problems caused by the mutual constraints of the accuracy between complex perceptual information in serial step-by-step processing. This avoids the mutual interference of missing point clouds and poor point cloud segmentation, making the subsequent division of the main stem point set and stem node set more accurate. This greatly improves the accuracy of seedling operation information extraction and directly provides a reliable basis for the precise clamping and cutting of fully automatic grafting machines. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1This is a flowchart illustrating the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of the collaborative network provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the distributed multi-task information extraction process provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the process for extracting grafting robot operation information based on semantic and geometric co-processing provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the grafting robot operation information extraction device based on semantic and geometric co-processing provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Fully automated grafting machines can precisely, efficiently, and stably solve the problem of heavy reliance on skilled grafting workers in the traditional seedling industry, realizing intelligent, standardized, and large-scale grafting operations. Accurate seedling perception is a crucial step in achieving fully automated grafting, and the extracted key information provides important references for the execution mechanism. A visual acquisition system distributed along the conveyor belt of the rootstock / scion seedling trays acquires point cloud information of each row of seedlings in real time, extracting information such as the uprightness, diameter, and spatial position of each stem node. The system calculates the cutting position based on this key information and then executes the cutting. However, low accuracy in perceiving key information about the grafted seedlings leads to inaccurate clamping and seedling damage, severely impacting grafting efficiency and seedling survival rates. Therefore, effectively extracting accurate information about key seedling parts and employing different processing methods for different types is crucial for the success rate of rootstock and scion cutting and grafting.

[0026] When acquiring point cloud images of grafted seedlings of cucurbits and solanaceous fruits on the conveyor belt of a fully automated grafting machine, self-occlusion and mutual occlusion between surrounding seedlings are unavoidable, resulting in missing point clouds. This is the main reason for inaccurate seedling identification and difficulty in extracting key information. To address the problem of missing point clouds, deep learning point cloud completion methods are typically used to fill in the missing and occluded points. For different parts of the seedling, point cloud segmentation networks are used to achieve different categories of point cloud segmentation. However, a critical problem arises in point cloud processing: the process of completing point cloud images before segmentation is highly dependent on the effectiveness of the completion; poor completion directly affects the segmentation results. Similarly, segmenting before completion is highly dependent on the effectiveness of the segmentation; poor segmentation or misclassified points will result in incorrect segmentation. Furthermore, after acquiring the main stem and node clouds, noise and low-correlation points interfere, requiring targeted acquisition of useful information to obtain accurate information such as diameter, uprightness, and spatial location of nodes. Existing methods for extracting key information have large errors and cannot meet the requirements of fully automated grafting machines.

[0027] The following is combined with Figures 1 to 6 This invention describes a grafting robot operation information extraction method based on semantic and geometric co-processing.

[0028] Figure 1 This is a flowchart illustrating the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the present invention, as shown below. Figure 1 As shown, the method includes, but is not limited to, steps S1, S2, S3 and S4.

[0029] Step S1: Extract the semantic and geometric features of the seedling point cloud.

[0030] Seedlings can be various forms of plants, such as grafted seedlings of cucurbits and grafted seedlings of solanaceous plants, specifically encompassing different parts such as rootstocks or scions. Point clouds are discrete data points representing the three-dimensional spatial distribution of the seedling surface, acquired through visual acquisition systems (such as 3D cameras, LiDAR, etc.) using methods such as side-viewing.

[0031] To construct the dataset, this invention sets up an input seedling point cloud containing N points. A complete point cloud with a total number of points M. The number of categories is set to C; simultaneously, the true value category label of each point in the input seedling point cloud is labeled. And the truth value category label for each point in the complete point cloud. .

[0032] Semantic features are high-dimensional features that reflect the local or global category attributes and functional tendencies of each data point within a point cloud. Geometric features are three-dimensional structural features that reflect the spatial morphology, distribution contours, and overall size of the point cloud.

[0033] It can be done as follows Figure 2 The point cloud semantic and geometric collaborative network structure shown extracts semantic and geometric features from seedling point clouds. For example, by feeding the original seedling point cloud, which contains points missing due to self-occlusion or mutual occlusion, into the encoder for multi-layer abstraction, semantic features reflecting the category tendency of each point can be extracted, while geometric features representing the overall outline of the seedling can also be extracted.

[0034] Step S1 can initially analyze the three-dimensional structural base and category potential of seedlings from the original messy and incomplete visual data, providing basic data feature support for subsequent synchronous completion and category segmentation.

[0035] Step S2: Perform bidirectional attention fusion on semantic features and geometric features to generate a complete point cloud with category labels.

[0036] Bidirectional attention fusion refers to the bidirectional computation and application of attention weights between the semantic segmentation branch and the geometric completion branch, enabling features to reference and correct each other. Category labels are identifiers given to each point in the point cloud, representing its specific part type, such as main stem category, stem node category, etc. A complete point cloud refers to a set of point cloud data that has been supplemented by spatial coordinate generation or supplementation operations, filling in the parts that were previously missing due to occlusion.

[0037] Matrix multiplication and weighted summation can be performed using self-attention or cross-attention mechanisms, allowing the input semantic features and the coarsely completed geometric features to undergo mutual attention weighting. This enables the completion process to reference class boundary information, and the segmentation process to reference complete 3D shape information. Subsequently, the final result of the collaborative network is output through a classifier and coordinate mapping layer: the segmentation result of the input seedling point cloud. Complete the point cloud results And the segmentation results of completing the point cloud. (i.e., a complete point cloud with category labels).

[0038] Step S2 can break the limitation of serial execution in existing point cloud processing, realize the collaborative processing of semantic understanding and geometric construction, and avoid error amplification caused by single dependence between different perception tasks.

[0039] Step S3: Divide the complete point cloud with category labels into a main stem point set and a stem node set.

[0040] The main stem point set is a subset of point cloud data whose category labels have been determined and assigned to the main supporting stem positions of the seedling. The stem node set is a subset of point cloud data whose category labels have been determined and assigned to key intersection points such as stem segments, leaf primordia, or scion growth.

[0041] By traversing each point in the complete point cloud, the category label value of that point can be extracted. If the value represents the main stem, the point is stored in the main stem point set array; if the value represents the stem node, it is stored in the stem node set array.

[0042] Step S3 can scientifically separate the massive grafted seedling point cloud, which has been completed and classified, into subsets of different morphological and functional parts, so that the optimal post-processing algorithm can be used to analyze each morphological part.

[0043] Step S4: Extract the target parameters of the main stem based on the main stem point set, and extract the target parameters of the stem nodes based on the stem node set.

[0044] The target parameters for the main stem can include parameters such as the diameter and uprightness of the main stem. The target parameters for the stem nodes can include parameters such as the coordinates of the spatial center position of the stem nodes.

[0045] The target parameters of the main stem and stem nodes can be obtained through calculation methods such as geometric fitting algorithms, statistical analysis algorithms, or clustering algorithms. For example, the diameter and uprightness parameters of the main stem can be obtained by applying two-dimensional projection and Gaussian weighted rectangle fitting, or the rotating caliper method to the set of main stem points; at the same time, density clustering algorithms can be applied to the scattered set of stem nodes to analyze the coordinates of the center position of its core.

[0046] Step S4 can directly obtain the specific numerical references necessary for the fully automatic grafting machine to clamp, cut and attach seedlings, supporting the precise and smooth operation of the equipment.

[0047] As described above, the grafting robot operation information extraction method based on semantic and geometric co-processing of the present invention extracts semantic features and geometric features simultaneously and performs bidirectional attention fusion on the two to generate a complete point cloud with category labels. This avoids the error cascading and misclassification problems caused by the mutual constraints of the accuracy between complex perceptual information in serial step-by-step processing, thereby avoiding the mutual interference of missing point cloud and poor point cloud segmentation effect. This makes the subsequent division of the main stem point set and stem node set more accurate, greatly improving the accuracy of seedling operation information (such as main stem and stem node related parameters), and directly providing a reliable basis for the precise clamping and cutting of fully automatic grafting machines.

[0048] In one embodiment, step S1 may further include: The seedling point cloud is downsampled and its features are aggregated layer by layer to obtain features at each level and a global feature vector. Semantic features are obtained by upsampling features at each level layer by layer through feature propagation. The global feature vector is resized and mapped to coordinate representation to obtain geometric features.

[0049] Layer-by-layer downsampling and feature aggregation are performed on the seedling point cloud to obtain features at each level and a global feature vector. Layer-by-layer downsampling gradually reduces the number of points in the cloud and expands the receptive field through a step-by-step sampling algorithm. Feature aggregation fuses the features of multiple scattered points in the local spatial neighborhood into a representative feature map. The features at each level are data feature maps with different resolution scales obtained after each downsampling and aggregation operation. The global feature vector is a one-dimensional or high-dimensional global feature descriptor that represents the macroscopic attributes of the entire seedling point cloud after final abstraction.

[0050] Can be used Figure 2 The pooling and sampling functions in the PointNet++ ensemble abstraction layers (SA1, SA2, SA3, etc.) are used to implement downsampling and feature aggregation. For example, in the point cloud encoder, representative points are selected layer by layer and their neighboring points are searched for convolutional fusion. This continuously reduces the size of the point cloud while increasing the feature dimension, thereby outputting features at each level. Finally, the deepest layer features are compressed and calculated into a global feature vector.

[0051] Semantic features are obtained by upsampling features at each level layer by layer through feature propagation. Feature propagation (FP) maps low-resolution deep features and interpolates them back to the high-resolution shallow original point cloud space. Layer-by-layer upsampling restores the original number of points by gradually increasing the density of the point set through interpolation algorithms.

[0052] Upsampling can be achieved by combining inverse distance weighted interpolation, nearest neighbor interpolation, and other calculations with skip connections. For example, using... Figure 2 The split branches (such as FP3, FP2, FP1) in the formula are used to... (X represents the input point cloud,) , For each level of features, n1 and n2 are the number of points after downsampling layer by layer, d1, d2, and d3 are the feature channel dimensions extracted from the corresponding layer, and Sfp is the semantic feature. The features compressed by the encoder are interpolated and amplified layer by layer. The FP operation is passed layer by layer with SA, so as to obtain semantic features containing rich point-level context information.

[0053] The global feature vectors are resized and mapped to coordinates to obtain geometric features. Reshaping changes the shape and dimensions of the feature matrix through tensor operations without altering its data content. The coordinate representation is a set of (x, y, z) spatial coordinates representing the complete three-dimensional morphology of the seedling, predicted by network regression.

[0054] It can be done Figure 2 The multilayer perceptron (MLP) used in the mid-head reconstruction performs matrix multiplication and nonlinear activation to achieve size reshaping and mapping, as shown in the formula. Where g is the global feature vector, This represents the MLP operation during the completion process. The Reshape operation reshapes the feature size, mapping the global feature vector to a complete set of point cloud coordinates as the coarse completion geometric feature Y. c .

[0055] This invention employs layer-by-layer upsampling through feature propagation, enabling semantic features to retain rich fine-grained structure and multi-scale spatial contextual information. Through global feature vector size reshaping mapping, it rapidly and robustly constructs macroscopic coordinate geometric features with overall morphological consistency. This dual-track feature extraction mechanism of "local interpolation for fine semantics + global reshaping for macroscopic geometry" allows the model to comprehensively capture individual seedling morphological differences, providing a highly representative and complementary high-quality feature source for subsequent bidirectional fusion.

[0056] In one embodiment, downsampling and feature aggregation are performed on the seedling point cloud layer by layer to obtain features at each level and a global feature vector, which may further include: Local reference points are selected in the seedling point cloud, and the neighborhood data around the local reference points are extracted for feature encoding, and the features of each level are output. A global feature vector is generated based on the last layer of features in each level.

[0057] Local reference points are selected in the seedling point cloud, and the surrounding neighborhood data of the local reference points are extracted for feature encoding, outputting features at each level. The local reference point is the set of backbone points selected as the neighborhood search center in the current level point cloud. The neighborhood data is the set of surrounding points within a certain spatial distance from the local reference point (such as a spherical region of a set radius).

[0058] Local reference points can be selected using algorithms such as farthest-point sampling. Feature encoding can be performed by constructing a Set Abstraction (SA) layer to extract neighborhood data around the local reference points. For example, for an input seedling point cloud... The encoder consists of multiple ensemble abstraction layers (SAs). Each SA obtains local reference points by performing farthest point sampling, acquires neighborhood data through neighborhood queries, and uses multilayer perceptron (MLP) encoding and max pooling to progressively extract features to output features at each level. ; A global feature vector is generated based on the last layer of features from each level. The last layer of features is the deep feature set with the largest receptive field and lowest resolution output by the encoder after all downsampling operations. The global feature vector is represented as: ; This invention obtains local reference points by sampling from the farthest point, which can uniformly and effectively cover the three-dimensional spatial morphology of the entire seedling. Combined with targeted encoding of local neighborhoods and max pooling processing, noise interference is effectively eliminated, so that the final global feature vector has permutation invariance and excellent noise robustness, which greatly improves the model's ability to generalize perception of morphologically varied seedlings of cucurbits and solanaceous fruits.

[0059] In one embodiment, step S2 may further include: Calculate the bidirectional attention weights between semantic features and geometric features to obtain the fused pointwise semantic features and the fused complete geometric features; The fused completed geometric features are upsampled and then concatenated with the fused point-by-point semantic features to obtain joint features; The joint features are encoded and mapped to 3D coordinate data and class probabilities respectively. The class corresponding to the highest class probability is assigned as the class label to the 3D coordinate data to generate a complete point cloud with class labels.

[0060] The bidirectional attention weights between semantic and geometric features are calculated to obtain the fused pointwise semantic features and the fused completed geometric features. The bidirectional attention weights are correlation coefficient matrices used to measure the mutual importance of semantic and geometric information at corresponding spatial positions. The fused pointwise semantic features and the fused completed geometric features are new feature representations carrying auxiliary information from each other's modalities after attention interaction correction.

[0061] By constructing a cross-task fusion module, mathematical computations such as self-attention, global pooling, and linear transformations can be used to obtain the fused pointwise semantic features and the fused completed geometric features. For example, the semantic features S output by the segmentation branch... fp and the geometric features Y of the completed branch output c Perform bidirectional attention fusion (Fuse): ; In the formula, This represents the pointwise semantic features after attention fusion. This represents the completed geometric features after attention fusion. Fuse represents the bidirectional attention fusion operation, including computations such as global pooling, attention-weighted features, and linear transformations.

[0062] The fused completed geometric features are upsampled and then concatenated with the fused pointwise semantic features to obtain joint features. Joint features are highly integrated feature vectors formed by merging aligned multimodal features along the channel dimension.

[0063] Feature stitching can be achieved by combining the interpolation function (Interp) with the tensor concatenation (Concat) operation. First, complete the geometric features. Upsampling to pointwise semantic features Number of similarities N: ; Then, the features from both are concatenated to obtain the joint feature Z: ; The joint features are encoded and mapped to 3D coordinate data and class probabilities respectively. The class corresponding to the highest class probability is assigned as the class label to the 3D coordinate data, generating a complete point cloud with class labels. The class probability is the numerical value of the likelihood of belonging to each part class after processing by a normalization function.

[0064] They can be accessed separately after being encoded using a shared multilayer perceptron (MLP). Figure 2 The segmentation heads on the right side of the image undergo matrix transformation calculations to generate a complete point cloud with category labels. Specifically, the joint feature Z is first encoded using a shared MLP: ;d joint For joint feature dimensions.

[0065] Three segmentation headers are set for different task types: For segmentation prediction, the point cloud segmentation head is represented as: Used only for network training.

[0066] For coordinate generation, the completed point cloud coordinate header is represented as follows: ; in The coordinates are upsampled and then downsampled (e.g., nearest neighbor downsampling) back to M points.

[0067] For complete segmentation, the complete point cloud segmentation head is represented as: ; in Then, the M points are back through downsampling (such as farthest point sampling or nearest neighbor sampling).

[0068] This invention achieves true cross-task collaboration by introducing a bidirectional attention weight calculation and feature upsampling and splicing mechanism. The bidirectional attention mechanism allows the macroscopic geometric distribution of the point cloud and the local microscopic semantic boundaries to supervise and correct each other; by mapping the joint features formed by splicing the two, the final completion coordinate prediction and segmentation classification decisions are based on a completely consistent multimodal feature basis. This fundamentally eliminates the risk of error propagation amplification in a single serial network, ensuring that the final grafted seedling complete point cloud has extremely high accuracy in both shape repair and category segmentation.

[0069] In one embodiment, encoding the joint features and mapping them to three-dimensional coordinate data and class probabilities respectively may further include: During the training phase of the collaborative network, the completion segmentation fusion loss and feature consistency loss are constructed based on the completion coordinate features and predicted segmentation features output by the collaborative network. The total loss is obtained by weighted summation of the completion segmentation fusion loss, feature consistency loss, segmentation loss, and completion loss. Update the network parameters of the cooperative network based on the total loss; The trained collaborative network maps joint features into 3D coordinate data and class probabilities.

[0070] During the training phase of the collaborative network, completion-segmentation fusion loss and feature consistency loss are constructed based on the completed coordinate features and predicted segmentation features output by the collaborative network. The training phase is a learning process that iteratively optimizes the network weights using sample data with ground truth labels. The collaborative network is the point cloud semantic and geometric collaborative processing network constructed in this invention. The completed coordinate features and predicted segmentation features are the geometric shapes and class probabilities predicted by the corresponding branches of the network. The completion-segmentation fusion loss and feature consistency loss are mathematical penalty terms used to constrain the degree of matching between the completed features and segmentation features at the spatial and semantic levels, and to measure the difference between the predicted values ​​and physical rules.

[0071] Various losses can be constructed by writing objective functions based on tensor computation frameworks and inputting predictor variables and ground truth values. For example, the above-mentioned fusion loss calculation rules can be designed for the network output simultaneously, so that the loss value can reflect the deviation between morphological reconstruction and category prediction.

[0072] The total loss is obtained by weighted summing of the segmentation / completeness fusion loss, feature consistency loss, segmentation loss, and completeness loss. The segmentation loss and completeness loss are the fundamental penalty terms for measuring the prediction error in a single segmentation task and a single coordinate completeness task, respectively. This represents the total loss of the network. It can be represented as: ; Where α, β, γ, and δ are the segmentation losses, respectively. , make up for the loss Completing the segmentation and fusion loss and feature consistency loss The weighting coefficients.

[0073] The segmentation loss can be achieved using the cross-entropy loss function: ; Input seedling point cloud truth value category label The nth tag in; Completion loss can also be achieved using cross-entropy loss: ; Complete point cloud truth value category label The nth tag in; in These are the predicted probability distributions; , , This represents multi-scale feature generation and upsampling, which are then concatenated and mapped to the number of categories. .

[0074] The network parameters of the collaborative network are updated based on the total loss. The network parameters consist of all learnable weight matrices and bias terms in the neural network. End-to-end training can be achieved by using the backpropagation algorithm combined with an optimizer to calculate the gradient of the total loss with respect to each parameter and adjusting the values ​​along the negative gradient direction.

[0075] The trained collaborative network maps joint features into 3D coordinate data and class probabilities. Specifically, the point cloud of the grafted seedlings sampled is input into the feedforward network with fixed parameters, and a series of matrix operations are performed to directly output the results.

[0076] This invention innovatively introduces completion segmentation fusion loss and feature consistency loss with weighted supervision during the training phase, powerfully guiding the collaborative network to simultaneously ensure accurate generation of geometric coordinates and consistent determination of semantic categories during parameter optimization. This multi-dimensional end-to-end joint loss-supervised training enables the network's joint features to achieve a more discriminative and robust manifold distribution in the feature space, resulting in highly stable and accurate 3D coordinate data and class probabilities output after training.

[0077] In one embodiment, constructing the completion segmentation fusion loss and feature consistency loss based on the completion coordinate features and predicted segmentation features output by the collaborative network may further include: Traverse the real point cloud and the completed point cloud corresponding to the completed coordinate features, and calculate the sum of squares of the minimum point pair distances in both directions as the completion segmentation fusion loss; The negative exponent of the Euclidean distance between the predicted coordinates of any two points is calculated as the geometric similarity, and the cosine similarity of the predicted segmentation features between any two points is calculated as the semantic similarity. For each pair of points, the square of the difference between the geometric similarity and the semantic similarity is calculated, and the arithmetic mean of the squared differences of all point pairs is taken to obtain the feature consistency loss.

[0078] The algorithm iterates through the ground truth point cloud and the completed point cloud corresponding to the completed coordinate features, calculating the sum of squared minimum point pair distances in both directions as the completion segmentation fusion loss. The ground truth point cloud is the complete and unmissing ground truth point cloud data of the grafted seedling that exists in the dataset as the standard answer, i.e. The minimum point-to-point distance is the Euclidean distance between the two nearest neighbors in the two point clouds.

[0079] The complete segmentation fusion loss can be obtained by introducing a chamfer distance algorithm for double-layer loop traversal or by tensor quantization for parallel distance calculation and extremum calculation. Complete segmentation fusion loss It can be represented as: ; In the formula, for The m-th point in for The nth point in the array.

[0080] The negative exponent of the Euclidean distance between the predicted coordinates of any two points is calculated as the geometric similarity, and the cosine similarity of the predicted segmentation features between any two points is calculated as the semantic similarity. For each pair of points, the square of the difference between the geometric and semantic similarities is calculated. The arithmetic mean of the squared differences for all point pairs is then obtained to obtain the feature consistency loss. (Consistency Loss) It can be represented as: ; In the formula, y i y j for The predicted coordinates of two different points in the equation, where σ is the standard deviation parameter, and s i s j Represents the predicted segmentation features of points i and j .

[0081] This invention uses the chamfered form of the bidirectional minimum point-pair distance as the completion fusion loss, strictly constraining the fitting degree between the completed point cloud and the actual anatomical structure of the grafted seedling. Simultaneously, it constructs a feature consistency loss by combining the negative exponent of the Euclidean distance with the feature inner product to calculate the similarity deviation, directly enforcing the physical law of "spatial proximity equals semantic consistency" at the loss level. This rigorous mathematical loss constraint effectively smooths the segmentation boundary, avoiding interference from isolated misclassified noise points in the grafted seedling stem.

[0082] In one embodiment, the target parameters for the main stem may include diameter and uprightness; Combination Figure 3 In step S4, extracting the target parameters of the main stem based on the main stem point set can further include: Projecting the main stem point set onto the horizontal plane yields the horizontal projection point; Calculate the Gaussian weight vector of the horizontal projection points; Construct a minimum area bounding rectangle for the horizontal projection points based on the Gaussian weight vector; Determine the angle between the spatial vector of the minimum area circumscribed rectangle (whose diameter is the width of the rectangle) and the long side of the minimum area circumscribed rectangle (whose verticality is the angle between the vector and the preset vertical coordinate axis).

[0083] Projecting the main stem point set onto a horizontal plane yields horizontal projection points. The horizontal plane is a two-dimensional reference plane (such as the XY plane) perpendicular to the natural growth direction of the grafted seedling. The horizontal projection points are a set of two-dimensional scattered points on the plane after removing the height axis coordinates. Specifically, the main stem category point set in three-dimensional space can be... Projected onto the XY plane to obtain M c The number of main stem points for category c.

[0084] Calculate the Gaussian weight vector for the horizontal projection points. The Gaussian weight vector is a set of anti-interference weight coefficients assigned to each projection point using a Gaussian distribution function, based on the data point distribution density and spatial distance. It can be obtained by extracting the distribution center of the point set and then mapping the distance using an exponential decay function.

[0085] A minimum-area bounding rectangle is constructed based on Gaussian weight vectors for the horizontal projection points. The minimum-area bounding rectangle is a two-dimensional rectangle that can encompass all projection points and has the smallest possible area. This can be constructed by introducing geometric calculations such as convex hull construction and polygonal rotation caliper methods, combined with weighted statistics using assigned Gaussian weights. For example, by using Gaussian weights as a reference to offset the projection variances in each direction, the coordinate system is continuously rotated to find the minimum-area bounding rectangle enclosed by the weighted boundary.

[0086] Determine the angle between the spatial vectors whose diameter is the width of the minimum area circumscribed rectangle and whose verticality is the angle between the long side of the minimum area circumscribed rectangle and the preset vertical coordinate axis. The preset vertical coordinate axis is the absolute vertical reference axis set by the system (such as the Y-axis or Z-axis).

[0087] Specifically, extract the maximum and minimum values ​​t of the rectangular boundary range. max t min s max s min , t min t max With s min s maxLet be the minimum and maximum values ​​of the optimal edge in its weighted confidence intervals along its tangent and normal directions, respectively, and let the length of the longer side be: ; Width is the diameter of the main stem of the corresponding category: ; Furthermore, based on the spatial vector of the longer side... Calculate the angle with the y-axis and extract the uprightness of the fitted main stem point cloud. : ; This invention addresses the problem of inaccurate cross-sectional extraction of grafted seedling main stems due to occlusion by residual leaves and system noise interference during actual collection. It introduces a horizontal plane projection processing mechanism based on Gaussian weight vectors. This mechanism reliably transforms complex and irregular 3D point clouds into regular geometric parameter expressions by constructing a minimum-area bounding rectangle. By adaptively weakening the interference of scattered edge points using Gaussian weights, it significantly improves the anti-interference capability and accuracy of main stem diameter and uprightness calculations under complex seedling conditions, ensuring precise control of the robotic arm's gripping position and force.

[0088] In one embodiment, calculating the Gaussian weight vector of the horizontal projection points may further include: Calculate the mean center coordinates of the horizontal projection points; Calculate the squared distance from each horizontal projection point to the center mean coordinate; The negative exponential mapping value obtained by dividing the squared distance by the variance parameter is used as the Gaussian weight vector.

[0089] Calculate the mean coordinates of the center of the horizontal projection points. The mean coordinates are two-dimensional coordinates representing the centroid of the projected point distribution. Let... This is expressed as the mean of the projection points on the XY plane: ; Calculate the squared distance from each horizontal projection point to the central mean coordinate: ; The negative exponential mapping value, obtained by dividing the squared distance by the variance parameter, is used as the Gaussian weight vector: ; In the formula, σ is the set standard deviation.

[0090] This forms the Gaussian weight vector. .

[0091] This invention establishes a Gaussian weight vector based on the idea of ​​Gaussian distribution distance decay. By accurately solving the center mean coordinates and the exponential decay operation of the squared distance, it effectively assigns extremely high confidence weights to the core points of the trunk region, greatly reducing the influence weight of residual noise points at the edge (such as small side branches that are not fully pruned) when participating in the fitting calculation. From the underlying mathematical logic, it ensures that the fitting of the outer rectangle boundary is not affected by distortion noise.

[0092] In one embodiment, constructing a minimum-area bounding rectangle for the horizontal projection points based on the Gaussian weight vector may further include: Calculate the convex hull of the horizontal projection points to obtain the sequentially arranged boundary vertex sequence; A local coordinate system is constructed along each convex hull edge of the boundary vertex sequence. The weighted projection mean and weighted projection standard deviation of the horizontal projection point in the corresponding convex hull edge normal and tangential directions are calculated by combining the Gaussian weight vector. The candidate rectangle boundaries are determined based on the weighted projection mean and weighted projection standard deviation, and the candidate rectangle boundary with the smallest area is taken as the minimum area circumscribed rectangle.

[0093] Calculate the convex hull of the horizontally projected points to obtain the ordered sequence of boundary vertices. The convex hull is the smallest circumscribed convex polygon containing all given scattered points and where all interior angles are less than 180 degrees. The sequence of boundary vertices is an ordered array of points that constitute the outer contour of this polygon.

[0094] Specifically, calculate the projection set The convex hull of the vertex is used to obtain a vertex sequence arranged in counterclockwise order. ,in Here, x is the total number of vertices of the convex hull, and v j These are the vertices of the convex hull.

[0095] A local coordinate system is constructed along each convex hull edge of the boundary vertex sequence. Using a Gaussian weight vector, the weighted mean and standard deviation of the horizontal projection points on the corresponding convex hull edge's normal and tangential directions are calculated. The local coordinate system is a new reference system with the current convex hull edge direction and its perpendicular direction as its axes. The normal and tangential directions correspond to the vector directions perpendicular to and parallel to the edge, respectively.

[0096] Specifically, for each edge of the convex hull Calculate the unit vector in the direction of this side. and normal vector . Let point q i Corresponding projection point (x) i y i ), calculate the mean of the tangential weighted projection and standard deviation : ; ; Among them, the mean of the projection of the normal direction and standard deviation Similarly, the calculation can be performed.

[0097] Candidate rectangle boundaries are determined based on the weighted projection mean and weighted projection standard deviation, and the candidate rectangle boundary with the smallest area is selected as the minimum area bounding rectangle. The candidate rectangle boundary is an imaginary bounding rectangle constructed based on statistical confidence intervals in the current direction of investigation.

[0098] Specifically, based on the rectangular boundary as well as Calculate the area of ​​the candidate rectangle: ; Then iterate through all edges and select the index of the edge with the smallest area. As the final circumscribed rectangle of the main stem, the center point of the rectangle is: z0 is the main stem category point set s c Mean coordinates along the Z-axis (height) direction.

[0099] This invention innovatively combines the classic convex hull algorithm and rotating caliper method with Gaussian weighted variance statistics. It utilizes the convex hull to filter interior points, reducing computational complexity, and establishes confidence interval boundaries using the weighted projection mean and weighted projection standard deviation, replacing the traditional absolute extreme value rigid bounding box calculation method which is susceptible to extreme value noise. This algorithm can highly adapt to the irregular elliptical characteristics and noise disturbances of seedling cross-sections, robustly obtaining a rectangular bounding box that closely matches the actual transverse section of the main stem, ensuring that the output diameter and tilt angle possess extremely high industrial-grade measurement accuracy.

[0100] In one embodiment, the target parameter for the stem node may include the central spatial location; In step S4, extracting the target parameters of the stem nodes based on the stem node set may further include: If the total number of points in the stem node set is less than or equal to the preset minimum number of points threshold, then the average coordinate of each point in the stem node set is taken as the center spatial position. If the total number of points is greater than the minimum number of points threshold, a point cloud clustering algorithm is used to divide the stem node set into at least one main cluster point set; the target main cluster point set containing the most points is determined, and the average coordinate of the target main cluster point set is used as the center spatial position.

[0101] Set neighborhood radius And the minimum point count threshold o, for the stem segment category point set obtained using point cloud semantic and geometric collaborative networks. Its total number of points is M c .

[0102] like This indicates that the number of extracted stem node clouds is too small to form an effective cluster. Therefore, the average value of these points is directly calculated as the center spatial location. ;like Then, point cloud clustering algorithms are used to obtain the main cluster labels of stem nodes. Taking the DBSCAN clustering algorithm as an example: ; This will lead to the gathering of points and clouds. Classified as main cluster and noise set By defining a label function Remove noise to avoid interference from isolated or extremely sparse sets of invalid points caused by flying points or misidentification.

[0103] The target principal cluster set containing the most points is identified, and the average coordinates of this set are used as the center spatial location. This target principal cluster set represents the largest and most reliable core cluster among all valid settlements. Specifically, the size of each principal cluster is calculated: L(s) is the label function. l Cluster label; Take the tag that appears most frequently. The primary cluster label is: ; If it does not exist Then the target principal cluster point set Otherwise, the target principal cluster point set .

[0104] Calculate the central spatial location of the stem node based on the target principal cluster point set: ; This invention addresses the common interference from scattered, misaligned leaves and branch noise in the extracted grafted seedling stem node clusters by introducing a main cluster selection mechanism based on point size judgment and density clustering. This mechanism can directly determine the location in special cases where the point cloud is severely missing (very few points), while filtering isolated noisy scattered points through clustering when the number of points is sufficient. This greatly improves the robustness and consistency of spatial positioning of seedling cutting nodes under occlusion and noise environments, ensuring the absolute reliability of the cutting height estimation.

[0105] Figure 4The implementation process of this invention is as follows: First, a visual acquisition system is constructed to obtain point cloud information of the grafted seedling rootstock / scion, and a dataset with completion and semantic labels is constructed using point cloud annotation software. Then, the point cloud semantic and geometric collaborative network is trained and tested using the dataset, and the output includes the completed semantic point cloud. Further, the point cloud completion and segmentation results are divided into multiple tasks such as main stem and stem node cloud, and multi-task information extraction is achieved through main stem diameter and uprightness extraction module and stem node spatial information fitting module, respectively. Finally, the diameter, uprightness, and stem node spatial position information are compared and verified with manually measured information, and the information extraction effect is verified through the grafting results of a fully automated grafting robot.

[0106] The grafting robot operation information extraction device based on semantic and geometric co-processing provided by the present invention will be described below. The grafting robot operation information extraction device based on semantic and geometric co-processing described below can be referred to in correspondence with the grafting robot operation information extraction method based on semantic and geometric co-processing described above.

[0107] like Figure 5 As shown, the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the present invention includes: The feature extraction module is used to extract the semantic and geometric features of the seedling point cloud; The feature fusion module is used to perform bidirectional attention fusion of semantic and geometric features to generate a complete point cloud with category labels; The point cloud classification module is used to divide a complete point cloud with category labels into a main stem point set and a stem node set; The parameter extraction module is used to extract target parameters of the main stem based on the main stem point set, and to extract target parameters of the stem nodes based on the stem node set.

[0108] Figure 6 A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor, a communication interface, memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory to execute a grafting robot task information extraction method based on semantic and geometric co-processing.

[0109] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the above methods.

[0111] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the grafting robot operation information extraction method based on semantic and geometric co-processing provided by the above methods.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting operational information of grafting robots based on semantic and geometric co-processing, characterized in that, include: Extract semantic and geometric features from seedling point clouds; The semantic features and the geometric features are fused by bidirectional attention to generate a complete point cloud with category labels. The complete point cloud with category labels is divided into a main stem point set and a stem node set; The target parameters of the main stem are extracted based on the main stem point set, and the target parameters of the stem nodes are extracted based on the stem node set.

2. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 1, characterized in that, Extract semantic and geometric features from the seedling point cloud, including: The seedling point cloud is downsampled and its features are aggregated layer by layer to obtain features at each level and a global feature vector; The semantic features are obtained by upsampling the features at each level layer by layer through feature propagation. The global feature vector is resized and mapped to coordinates to obtain the geometric features.

3. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 2, characterized in that, The seedling point cloud is downsampled and its features are aggregated layer by layer to obtain features at each level and a global feature vector, including: Local reference points are selected in the seedling point cloud, and the neighborhood data around the local reference points are extracted for feature encoding, and the features of each level are output. The global feature vector is generated based on the last layer of features in each layer.

4. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 1, characterized in that, The semantic features and the geometric features are fused by bidirectional attention to generate a complete point cloud with category labels, including: Calculate the bidirectional attention weights between the semantic features and the geometric features to obtain the fused pointwise semantic features and the fused complete geometric features; The fused completed geometric features are upsampled and then concatenated with the fused point-by-point semantic features to obtain joint features; The joint features are encoded and mapped to three-dimensional coordinate data and class probabilities respectively. The class corresponding to the highest class probability is assigned as the class label to the three-dimensional coordinate data to generate the complete point cloud with class label.

5. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 4, characterized in that, The joint features are feature-encoded and mapped to three-dimensional coordinate data and class probabilities, respectively, including: During the training phase of the collaborative network, a completion segmentation fusion loss and a feature consistency loss are constructed based on the completion coordinate features and predicted segmentation features output by the collaborative network. The total loss is obtained by weighted summing of the completion segmentation fusion loss, the feature consistency loss, the segmentation loss, and the completion loss. The network parameters of the cooperative network are updated based on the total loss. The trained collaborative network maps the joint features to the 3D coordinate data and the class probabilities.

6. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 5, characterized in that, Based on the completed coordinate features and predicted segmentation features output by the collaborative network, a completion segmentation fusion loss and a feature consistency loss are constructed, including: Traverse the real point cloud and the completed point cloud corresponding to the completed coordinate features, and calculate the sum of squares of the minimum point pair distances in both directions as the completed segmentation fusion loss; The negative exponent of the Euclidean distance between any two predicted coordinates is calculated as the geometric similarity, and the cosine similarity of the predicted segmentation features between any two points is calculated as the semantic similarity. For each pair of points, the square of the difference between the geometric similarity and the semantic similarity is calculated, and the arithmetic mean of the squared differences of all point pairs is taken to obtain the feature consistency loss.

7. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 1, characterized in that, The target parameters for the main stem include diameter and uprightness; Extracting target parameters of the main stem based on the main stem point set includes: The main stem point set is projected onto a horizontal plane to obtain horizontal projection points; Calculate the Gaussian weight vector of the horizontal projection point; Construct a minimum area bounding rectangle for the horizontal projection point based on the Gaussian weight vector; The diameter is determined to be the width of the minimum area circumscribed rectangle, and the verticality is the angle between the spatial vector of the long side of the minimum area circumscribed rectangle and the preset vertical coordinate axis.

8. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 7, characterized in that, Calculating the Gaussian weight vector of the horizontal projection point includes: Calculate the mean center coordinates of the horizontal projection points; Calculate the squared distance from each of the horizontal projection points to the mean coordinate of the center; The negative exponential mapping value obtained by dividing the squared distance by the variance parameter is used as the Gaussian weight vector.

9. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 7, characterized in that, Constructing a minimum area bounding rectangle for the horizontal projection point based on the Gaussian weight vector includes: Calculate the convex hull of the horizontal projection points to obtain a sequence of boundary vertices arranged in order; A local coordinate system is constructed along each convex hull edge of the boundary vertex sequence, and the weighted projection mean and weighted projection standard deviation of the horizontal projection point in the corresponding convex hull edge normal and tangential directions are calculated in combination with the Gaussian weight vector. The candidate rectangle boundaries are determined based on the weighted projection mean and the weighted projection standard deviation, and the candidate rectangle boundary with the smallest area is taken as the minimum area circumscribed rectangle.

10. The grafting robot operation information extraction method based on semantic and geometric co-processing according to claim 1, characterized in that, The target parameters for the stem segment include the central spatial location; Based on the aforementioned stem node set, target parameters for stem nodes are extracted, including: If the total number of points in the stem node set is less than or equal to a preset minimum number of points threshold, then the average coordinate of each point in the stem node set is taken as the center spatial position. If the total number of points is greater than the minimum number of points threshold, a point cloud clustering algorithm is used to divide the stem node set into at least one main cluster point set; the target main cluster point set containing the most points is determined, and the average coordinate of the target main cluster point set is used as the center spatial position.

Images