Collaborative arm motion planning method and system based on image analysis

Abstract

The invention provides a collaborative arm motion planning method and system based on image analysis, and relates to the technical field of collaborative arm motion, and the method comprises the steps: obtaining an image of a to-be-pruned fruit forest; determining distance information between the plurality of pruned fruit tree images and the pruned fruit trees by using a fruit tree determination model based on the to-be-pruned fruit tree forest image; constructing a fruit tree pruning map; processing the pruned fruit tree map based on a graph neural network to determine a plurality of preliminarily selected pruned fruit trees; determining a cooperative arm control scheme for pruning branches and trunks of each primary carefully-selected pruned fruit tree based on the plurality of primary carefully-selected pruned fruit tree images; obtaining a fruit tree forest image after the first pruning operation is completed; determining a cooperative arm control scheme for pruning the remaining pruned fruit tree branches based on the fruit tree forest image after the first pruning operation is completed; and controlling the cooperative arm to perform second pruning operation based on the cooperative arm control scheme for pruning the remaining pruned fruit tree branches. According to the method, the cooperative arm control scheme for pruning the fruit tree can be accurately determined.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of collaborative arm motion, in particular to a collaborative arm motion planning method and system based on image analysis. BACKGROUND

[0002] Pruning of fruit tree forest is a key agricultural operation to ensure the healthy growth of fruit trees and improve the yield and quality of fruits. Pruning of fruit tree forest needs to be accurately carried out based on the growth state, spatial distribution and fruit cultivation demand of fruit trees. Traditional pruning of fruit tree forest is completed by manual operation, which has many pain points. Manual identification of fruit trees that need to be pruned is highly subjective and is easily influenced by the subjective experience of technical personnel, which may lead to missed pruning and mispruning. In addition, manual pruning of fruit tree forest has low efficiency and is difficult to adapt to the pruning demand of large-scale fruit tree forest. Furthermore, manual operation is limited by terrain and fruit tree height, and there are safety risks and uneven pruning quality. Collaborative arms have the characteristics of precise operation and adaptation to complex environments, and have significant potential value in solving the pain points of manual pruning. However, traditional collaborative arms lack targeted adaptation in pruning of fruit tree forest, have insufficient accuracy in identifying the shape of fruit trees, and lack flexibility in operation, which has made it difficult to meet the requirements of precision agriculture development.

[0003] Therefore, how to accurately determine the collaborative arm control scheme for pruning fruit trees is a problem to be solved at present. SUMMARY

[0004] The technical problem solved by the present application is how to accurately determine the collaborative arm control scheme for pruning fruit trees.

[0005] According to a first aspect, the present application provides a collaborative arm motion planning method based on image analysis, comprising: acquiring a fruit tree forest image to be pruned; determining distance information between a plurality of pruned fruit tree images and pruned fruit trees based on the fruit tree forest image to be pruned using a fruit tree determination model; constructing a pruned fruit tree graph, the pruned fruit tree graph comprising a plurality of pruned fruit tree nodes and a plurality of edges between the pruned fruit tree nodes, the node feature of each pruned fruit tree node being each pruned fruit tree image, and the edges between the pruned fruit tree nodes being distance information between the pruned fruit trees; processing the pruned fruit tree graph based on a graph neural network to determine a plurality of preliminary selected pruned fruit trees; determining a collaborative arm control scheme for pruning of the branch of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images; controlling the collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for pruning of the branch of each preliminary selected pruned fruit tree; acquiring a fruit tree forest image after the first pruning operation is completed; determining a collaborative arm control scheme for pruning of the remaining pruned fruit tree branch based on the fruit tree forest image after the first pruning operation is completed; and controlling the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning of the remaining pruned fruit tree branch.

[0006] In one possible implementation, the collaborative arm control scheme for determining the pruning of each preliminary selected pruning fruit tree branch based on multiple preliminary selected pruning fruit tree images includes: determining the estimated fruit-bearing area of ​​each preliminary selected pruning fruit tree based on multiple preliminary selected pruning fruit tree images; determining the high-importance branches and low-importance branches of each preliminary selected pruning fruit tree based on the multiple preliminary selected pruning fruit tree images and the estimated fruit-bearing area of ​​each preliminary selected pruning fruit tree; and determining the collaborative arm control scheme for pruning each preliminary selected pruning fruit tree branch based on the multiple preliminary selected pruning fruit tree images, the high-importance branches and the low-importance branches of each preliminary selected pruning fruit tree.

[0007] In one possible implementation, determining the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree based on multiple pre-selected pruned fruit tree images includes: determining the information of each green leaf of each pre-selected pruned fruit tree based on multiple pre-selected pruned fruit tree images; obtaining K clusters using the K-means clustering algorithm based on the information of each green leaf of each pre-selected pruned fruit tree; and generating the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree using a generative adversarial network based on the K clusters.

[0008] In one possible implementation, the fruit tree determination model is a convolutional neural network model.

[0009] According to a second aspect, the present invention provides a collaborative arm motion planning system based on image analysis, comprising: an image acquisition module for acquiring images of an orchard to be pruned; an orchard information determination module for determining distance information between multiple pruned orchard images and pruned orchards based on the images of the orchard to be pruned using an orchard determination model; a construction module for constructing a pruned orchard atlas, the pruned orchard atlas including multiple pruned orchard nodes and multiple edges between the multiple pruned orchard nodes, the node features of each pruned orchard node being each pruned orchard image, and the edges between the pruned orchard nodes being the distance information between the pruned orchards; and a preliminary selection module for processing the pruned orchard atlas based on a graph neural network to determine multiple preliminary selected orchards. Pruning fruit trees; a first control scheme determination module, used to determine a collaborative arm control scheme for pruning the branches of each preliminarily selected pruned fruit tree based on multiple preliminarily selected pruned fruit tree images; a first pruning control module, used to control the collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for each preliminarily selected pruned fruit tree branch; a second image acquisition module, used to acquire an image of the orchard after the first pruning operation is completed; a second control scheme determination module, used to determine a collaborative arm control scheme for pruning the remaining branches of the pruned fruit tree based on the image of the orchard after the first pruning operation is completed; a second pruning control module, used to control the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning the remaining branches of the pruned fruit tree.

[0010] In one possible implementation, the first control scheme determination module is further configured to: determine the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree based on multiple images of pre-selected pruned fruit trees; determine the high-importance branches and low-importance branches of each pre-selected pruned fruit tree based on the multiple images of pre-selected pruned fruit trees and the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree; and determine a collaborative arm control scheme for pruning the branches of each pre-selected pruned fruit tree based on the multiple images of pre-selected pruned fruit trees, the high-importance branches and low-importance branches of each pre-selected pruned fruit tree.

[0011] In one possible implementation, determining the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree based on multiple pre-selected pruned fruit tree images includes: determining the information of each green leaf of each pre-selected pruned fruit tree based on multiple pre-selected pruned fruit tree images; obtaining K clusters using the K-means clustering algorithm based on the information of each green leaf of each pre-selected pruned fruit tree; and generating the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree using a generative adversarial network based on the K clusters.

[0012] In one possible implementation, the fruit tree determination model is a convolutional neural network model.

[0013] According to a third aspect, embodiments of the present invention provide an electronic device, including: a processor; a memory; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method as described above, the method including: acquiring an image of an orchard to be pruned; determining distance information between multiple images of pruned fruit trees and pruned fruit trees based on the image of the orchard to be pruned using a fruit tree determination model; constructing a pruned fruit tree atlas, the pruned fruit tree atlas including multiple pruned fruit tree nodes and multiple edges between the multiple pruned fruit tree nodes, each pruned fruit tree node having node features of each pruned fruit tree image, and the edges between the pruned fruit tree nodes... To obtain distance information between fruit trees during pruning; to process the pruning tree map using a graph neural network to determine multiple preliminary selected pruning trees; to determine a collaborative arm control scheme for pruning the branches of each preliminary selected pruning tree based on the multiple preliminary selected pruning tree images; to control the collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for each preliminary selected pruning tree branch; to acquire an image of the orchard after the first pruning operation; to determine a collaborative arm control scheme for pruning the remaining branches of the pruning tree based on the image of the orchard after the first pruning operation; and to control the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning the remaining branches of the pruning tree.

[0014] According to the fourth aspect, this embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the aforementioned collaborative arm motion planning method based on image analysis. The method includes: acquiring an image of an orchard to be pruned; determining distance information between multiple images of pruned fruit trees and pruned fruit trees using a fruit tree determination model based on the image of the orchard to be pruned; constructing a pruned fruit tree atlas, the pruned fruit tree atlas including multiple pruned fruit tree nodes and multiple edges between the multiple pruned fruit tree nodes, the node feature of each pruned fruit tree node being each pruned fruit tree image, and the edges between pruned fruit tree nodes being the edges between pruned fruit trees. The distance information is obtained; the pruning tree map is processed based on a graph neural network to determine multiple preliminary selected pruning trees; a cooperative arm control scheme for pruning the branches of each preliminary selected pruning tree is determined based on the multiple preliminary selected pruning tree images; the cooperative arm is controlled to perform a first pruning operation based on the cooperative arm control scheme for each preliminary selected pruning tree; an image of the orchard after the first pruning operation is obtained; a cooperative arm control scheme for pruning the remaining branches of the pruning tree is determined based on the image of the orchard after the first pruning operation; and the cooperative arm is controlled to perform a second pruning operation based on the cooperative arm control scheme for pruning the remaining branches of the pruning tree.

[0015] This invention provides a collaborative arm motion planning method and system based on image analysis. The method includes: acquiring an image of an orchard to be pruned; determining distance information between multiple pruned fruit tree images and pruned fruit trees using a fruit tree identification model based on the image of the orchard to be pruned; constructing a pruned fruit tree atlas, which includes multiple pruned fruit tree nodes and multiple edges between the multiple pruned fruit tree nodes, where the node features of each pruned fruit tree node are each pruned fruit tree image, and the edges between pruned fruit tree nodes are the distance information between pruned fruit trees; processing the pruned fruit tree atlas based on a graph neural network to determine multiple preliminary selected pruned fruit trees; determining a collaborative arm control scheme for pruning the branches of each preliminary selected pruned fruit tree image; controlling the collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for pruning the branches of each preliminary selected pruned fruit tree; acquiring an orchard image after the first pruning operation; determining a collaborative arm control scheme for pruning the remaining branches of the pruned fruit trees based on the orchard image after the first pruning operation; and controlling the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning the remaining branches of the pruned fruit trees. This method can accurately determine the collaborative arm control scheme for pruning fruit trees. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a collaborative arm motion planning method based on image analysis provided in an embodiment of the present invention;

[0017] Figure 2A schematic diagram of an orchard provided for an embodiment of the present invention;

[0018] Figure 3 A flowchart illustrating a collaborative arm control scheme for determining the pruning of each preliminary selected fruit tree branch, provided in an embodiment of the present invention.

[0019] Figure 4 A schematic diagram of a process for determining the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree, provided in an embodiment of the present invention;

[0020] Figure 5 A schematic diagram of a mechanical collaborative arm provided in an embodiment of the present invention;

[0021] Figure 6 This is a schematic diagram of a collaborative arm motion planning system based on image analysis, provided as an embodiment of the present invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0023] In this embodiment of the invention, the following are provided: Figure 1 The image analysis-based collaborative arm motion planning method shown includes steps S1 to S9:

[0024] Step S1: Obtain an image of the orchard to be pruned.

[0025] An orchard is a forest area formed by planting a large number of fruit trees in a concentrated manner according to a certain plan, encompassing the fruit trees, branches, leaves, and the space between the trees. Figure 2 This is a schematic diagram of an orchard provided in an embodiment of the present invention.

[0026] The images of the orchard to be pruned are panoramic static digital images of the entire orchard work area, captured by a high-definition camera visual acquisition device mounted on a drone.

[0027] Images of orchards to be pruned can record the overall distribution pattern of fruit trees in the orchard, the growth posture of individual fruit trees, the density of branches and leaves, and the environmental background around the fruit trees.

[0028] Step S2: Based on the image of the orchard to be pruned, use the fruit tree determination model to determine the distance information between multiple images of pruned fruit trees and the pruned fruit trees.

[0029] The fruit tree identification model is a convolutional neural network model. The input of the fruit tree identification model is an image of the orchard to be pruned, and the output of the fruit tree identification model is multiple images of pruned fruit trees and distance information between the pruned fruit trees.

[0030] Convolutional Neural Network (CNN) models are a type of artificial neural network capable of processing data with a grid-like structure. A CNN can contain convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract local features by sliding operations on the input data using convolutional kernels; pooling layers reduce the dimensionality of the data while preserving key information; and fully connected layers map the extracted features to a labeled sample space. CNNs can automatically learn and extract features from images, ranging from low-level edge textures to high-level semantic concepts, thereby achieving accurate recognition and localization of image content.

[0031] The image of a pruned fruit tree is an image of the fruit trees that need to be pruned, which is obtained by analyzing the image of the orchard to be pruned using a fruit tree identification model.

[0032] Each image of a pruned fruit tree can show the distribution of the crown, branches, and leaves of each tree that needs pruning.

[0033] The distance information between pruned fruit trees is the spatial distance data between every two fruit trees that need to be pruned, output by the fruit tree determination model. This distance information includes the actual physical straight-line distance and the actual physical horizontal distance between every two fruit trees that need to be pruned.

[0034] Convolutional neural networks (CNNs) can perform deep analysis of images of orchards to be pruned through multi-layer convolutional operations. Shallow convolutional kernels in a CNN can capture low-level visual features such as edges and color spots in the image, and identify the green texture of branches and leaves and the brown lines of the trunk. As the network depth increases, the CNN can aggregate these low-level features layer by layer to form high-level semantic representations of the canopy outline, branch orientation, leaf density, and spatial occlusion relationships. The CNN can analyze the canopy compactness, branch-leaf crossing frequency, internal gap ratio, and overlap with neighboring trees of each tree, then quantify the structural disorder index of the tree and assess leaf color uniformity and edge clarity to determine the presence of abnormal growth or weak branches. When the structural disorder index of a tree exceeds a preset threshold, or when significant redundancy in branches and leaves, poor ventilation and light transmission, or other typical pruning characteristics are detected, the model identifies the tree as needing pruning. Subsequently, the convolutional neural network generates corresponding bounding boxes on the feature map for the fruit trees identified as needing pruning, and crops out their respective region images, thus forming multiple images of pruned fruit trees. The model can utilize perspective relationships, occlusion relationships, and monocular depth estimation cues in the images of the fruit trees to be pruned to analyze the differences in pixel coordinates and scale variations of different pruned fruit trees on the image plane. Through the learned spatial mapping relationships, the convolutional neural network can convert pixel distances on the image plane into actual physical distances, ultimately outputting the actual physical straight-line distance and actual physical horizontal distance between each pair of fruit trees that need pruning, forming distance information between the pruned fruit trees.

[0035] In some embodiments, determining the distance information between multiple images of pruned fruit trees and the pruned fruit trees using a fruit tree determination model based on the image of the orchard to be pruned includes steps S21 to S23:

[0036] Step S21: Based on the image of the orchard to be pruned, determine the crown outline information of each fruit tree, the branch direction of each fruit tree, and the leaf distribution information.

[0037] In some embodiments, convolutional neural networks can be used to determine the crown outline information, branch orientation, and leaf distribution information of each fruit tree.

[0038] The crown outline information for each fruit tree is a quantitative description of the crown edge morphology, boundary, and overall geometry of a single fruit tree, output by a convolutional neural network. The crown outline information for each fruit tree includes the coordinate sequence of the crown outline and the pixel area of ​​the region enclosed by the outline.

[0039] The branch orientation and leaf distribution information of each fruit tree are quantitative information output by a convolutional neural network, describing the growth direction and extension trend of the branches of a single fruit tree, as well as the arrangement and distribution of leaves on the branches.

[0040] Branch orientation information includes the starting and ending pixel coordinates of the branch, tilt angle, fork node coordinates, and fork direction.

[0041] Leaf distribution information includes the pixel range of leaf clusters, the number of leaf pixels in a single region, the spacing between leaves on branches, and the overall distribution uniformity.

[0042] Convolutional neural networks (CNNs) can efficiently extract multi-layer features from images through their local receptive fields and weight sharing characteristics. Shallow convolutions can capture basic information such as edges and textures to accurately outline tree canopies. Deep networks can aggregate features to distinguish pixel differences between branches and leaves, and can capture branch direction, branching nodes, and leaf distribution density. CNNs possess powerful feature discrimination and detail capture capabilities, enabling them to adapt to the visual feature differences of various parts of fruit trees in orchard images, thereby accurately determining the canopy outline, branch direction, and leaf distribution information of each fruit tree.

[0043] Step S22: Based on the crown outline information of each fruit tree, the branch orientation and leaf distribution information of each fruit tree, determine the structural disorder characteristics of each fruit tree, the disease and weakness characteristics of each fruit tree, the branch and leaf overlap distribution map within the fruit tree forest area, and the abnormal growth status distribution map of fruit trees within the fruit tree forest area.

[0044] In some embodiments, convolutional neural networks can be used to determine the structural disorder features of each fruit tree, the disease and weakness features of each fruit tree, the distribution map of branch and leaf overlap within the fruit orchard area, and the distribution map of abnormal growth status of fruit trees within the fruit orchard area.

[0045] The structural disorder characteristics of each fruit tree are features output by a convolutional neural network that reflect the degree of disorder in the overall growth structure of a single fruit tree. These structural disorder characteristics include the canopy compactness, branch crossing frequency, and the proportion of internal voids in the canopy.

[0046] The disease and weakness characteristics of each fruit tree are output by a convolutional neural network, reflecting the poor growth status and disease presence of individual trees. These characteristics include: abnormal green saturation values ​​for leaves; pixel percentage of yellowing and mottled areas; the ratio of pixel area of ​​damaged leaf regions to the total pixel area of ​​the leaf; pixel value of thin and withered branch diameter to the percentage of withered areas; and the ratio of the pixel length of withered leaf edges to the total pixel length of the leaf edge.

[0047] The distribution map of branch and leaf overlap within the orchard area is generated by using a convolutional neural network to quantify and label the density of branch and leaf stacking in different areas of the entire orchard.

[0048] The distribution map of abnormal growth status of fruit trees in the orchard area is formed by using a convolutional neural network to quantify and label the type, degree, and spatial location of abnormal growth of each fruit tree in the orchard.

[0049] Abnormal growth types include redundant branches and leaves, disordered structure, diseased and weak type, excessive growth type, and leaf damage type.

[0050] The redundant branch and leaf type is characterized by excessively dense branch and leaf growth, with the proportion of voids inside the canopy being lower than a preset threshold.

[0051] The disordered structure type is characterized by excessively high frequency of branch crossings, disordered growth direction, or abnormal branch forking angles.

[0052] The diseased and declining type is characterized by insufficient green saturation of leaves, excessive yellowing and mottling, and an excessively high proportion of withered areas on branches and trunks.

[0053] The excessive growth type is characterized by some branches growing too fast, exceeding the normal range of the tree canopy, and competing fiercely for space with surrounding branches.

[0054] The leaf defect type is characterized by the defective area of ​​the leaf having a pixel area ratio exceeding the set standard, indicating that the integrity of the leaf is compromised.

[0055] Convolutional neural networks (CNNs) can perform high-order feature fusion and quantification analysis on basic information about tree canopy outlines, branches, and leaves. By learning the structural and state characteristics of normal fruit tree growth, the model can accurately identify structural disorder indicators such as branch crossing frequency and canopy gap ratio, as well as disease and weakness characteristics such as leaf saturation and withering rate. CNNs can aggregate information from the entire fruit tree domain to quantify the density of branch and leaf overlap in different areas and label the location and type of abnormal fruit trees. Through feature mapping capabilities, CNNs can efficiently generate target feature information and distribution maps.

[0056] Step S23: Based on the structural disorder characteristics of each fruit tree, the disease and weakness characteristics of each fruit tree, the branch and leaf overlap distribution map within the fruit tree forest area, and the abnormal growth status distribution map of the fruit trees within the fruit tree forest area, determine the distance information between multiple pruned fruit tree images and pruned fruit trees.

[0057] In some embodiments, a convolutional neural network can be used to determine the distance information between multiple images of pruned fruit trees and the pruned fruit trees.

[0058] Convolutional neural networks (CNNs) can correlate structural disorder, diseased and weak features with two types of distribution data, and select candidate fruit trees for pruning through multi-dimensional feature weighting evaluation. Simultaneously, CNNs can model based on the spatial location information of fruit trees, thereby quantifying the pixel distance between candidate fruit trees and converting it into actual spacing, and simultaneously cropping the corresponding fruit tree images. With its multi-feature fusion and spatial computing capabilities, CNNs can efficiently output multiple images of pruned fruit trees and information on the distances between them.

[0059] Step S3: Construct a pruning tree map. The pruning tree map includes multiple pruning tree nodes and multiple edges between the pruning tree nodes. The node features of each pruning tree node are each pruning tree image, and the edges between pruning tree nodes are the distance information between pruning tree nodes.

[0060] A pruned fruit tree graph is a graphical data structure used to describe the individual fruit trees in an orchard that require pruning and their spatial relationships. A pruned fruit tree graph consists of multiple pruned fruit tree nodes and multiple edges connecting these nodes.

[0061] Multiple pruned tree nodes are the basic building blocks of a pruned tree atlas, with each node representing a single fruit tree in the orchard awaiting pruning. The node characteristics of each pruned tree node define the characteristics of each pruned tree image.

[0062] The edges between the nodes of the pruned fruit trees are the lines connecting different nodes, and the edges represent the distance information between the pruned fruit trees.

[0063] Step S4: Based on the graph neural network, the pruning tree map is processed to determine a number of preliminary selected pruning trees.

[0064] Graph Neural Networks (GNNs) are deep learning models specifically designed for processing graphs. Through message passing mechanisms, GNNs allow nodes in a graph to aggregate information from their neighbors to update their feature representations. GNNs can simultaneously utilize the intrinsic attributes of nodes and the topological features of the graph, thereby capturing complex non-Euclidean spatial dependencies in the data. Through multi-layer aggregation operations, GNNs can fuse local and global structural information into the embedding vector of each node, enabling the classification or prediction of nodes based on the context of the entire graph. The input to the GNN is the pruned fruit tree graph, and the output is a number of preliminarily selected pruned fruit trees.

[0065] Several preliminary selected fruit trees were those with high pruning priority, screened from the pruning tree atlas using a graph neural network.

[0066] By constructing a pruning tree atlas, the spatial topological relationships between multiple pruning tree nodes can be clearly reflected. Pruning operations on adjacent trees often involve spatial interference, operational efficiency coupling, and branch occlusion, among other factors. Therefore, using each pruning tree image as a node feature and the distance information between pruning trees as edge features between nodes allows for a more comprehensive integration of information from both the individual tree's visual state and the group's spatial layout. This helps the graph neural network more accurately understand the relative position, density, and operational accessibility of each tree requiring pruning in the orchard, thereby identifying higher-priority pruning targets. Processing the pruning tree atlas using a graph neural network effectively aggregates features from adjacent nodes and propagates global structural information, enabling a comprehensive evaluation of each pruning tree within the group context. Compared to methods that rely solely on individual pruning tree images for independent judgment, graph neural networks demonstrate stronger relationship modeling and context awareness capabilities when processing pruning tree atlases with clear spatial relationships.

[0067] Graph neural networks (Graph Neural Networks) encode the features of each pruned tree node, transforming image features into a computable vector representation. Through message passing, each node aggregates feature information from its neighbors and updates its own feature representation by incorporating the weights of edges between nodes. During aggregation, closer pruned tree nodes exhibit stronger feature interactions. The model captures the distribution characteristics of pruned tree clusters and, through iterative learning across multiple layers, uncovers key features influencing pruning priority, such as the tree's growth status and its importance within the orchard. Finally, the model scores and ranks all pruned tree nodes based on the learned node features, selecting the trees corresponding to the highest-scoring nodes as a preliminary selection for pruning.

[0068] Step S5: Determine the collaborative arm control scheme for pruning the branches of each preliminary selected pruning fruit tree based on multiple preliminary selected pruning fruit tree images.

[0069] In some embodiments, Figure 3 This is a flowchart illustrating a collaborative arm control scheme for determining each preliminary selected pruning fruit tree branch, as provided in an embodiment of the present invention. The collaborative arm control scheme for determining each preliminary selected pruning fruit tree branch includes steps S51 to S53:

[0070] Step S51: Determine the estimated fruit-bearing area for each pre-selected pruned fruit tree based on multiple images of pre-selected pruned fruit trees.

[0071] In some embodiments, Figure 4This is a schematic diagram of a process for determining the estimated fruit-bearing area of ​​each preliminarily selected pruned fruit tree according to an embodiment of the present invention. The determination of the estimated fruit-bearing area of ​​each preliminarily selected pruned fruit tree includes steps S511-S513:

[0072] Step S511: Determine the information of each green leaf of each preliminary selected pruned fruit tree based on multiple preliminary selected pruned fruit tree images.

[0073] Multiple preliminary selection images of pruned fruit trees are a collection of images focused on the details of each preliminary selection of pruned fruit trees, separated from multiple images of pruned fruit trees.

[0074] In some embodiments, a leaf analysis model can be used to determine the information of each leaf in each pre-selected pruned fruit tree. The leaf analysis model is a convolutional neural network model. The input to the leaf analysis model is multiple images of pre-selected pruned fruit trees, and the output of the leaf analysis model is the information of each leaf in each pre-selected pruned fruit tree.

[0075] The information for each leaf of each preliminarily selected pruned fruit tree consists of specific feature data about a single leaf in each tree, extracted from multiple images of preliminarily selected pruned fruit trees using a leaf analysis model. This information includes the shape, size, color, and location coordinates of each leaf.

[0076] Convolutional neural networks (CNNs) can leverage their powerful feature extraction capabilities to perform pixel-by-pixel semantic segmentation on multiple preliminary selection images of pruned fruit trees. The model first uses a shallow network to identify green patches in the preliminary selection images and distinguish the color boundaries between leaves and branches. Next, a deeper network further analyzes the edge contours of the leaves. The model calculates the geometric center of each identified leaf region as its location coordinates and counts the number of pixels covered by the leaf region as its area, then extracts the average color value within the region as color information. Simultaneously, the model quantifies texture density by analyzing the grayscale variation rate of the leaf surface. Finally, the model integrates these extracted feature parameters to output precise information on every green leaf of each preliminary selection image of a fruit tree.

[0077] Step S512: Based on the information of each green leaf of each pre-selected pruned fruit tree, K clusters are obtained using the K-means clustering algorithm.

[0078] K-means clustering is an iterative clustering analysis algorithm. It divides a dataset into K non-overlapping clusters, minimizing the sum of squared distances from each data point to the centroid of its cluster. Data points within each cluster have high similarity, while data points between different clusters have low similarity. The value of K in K-means clustering can be preset. K-means clustering effectively identifies dense regions in the spatial distribution of data.

[0079] The K clusters are K independent groups of leaf features obtained by clustering all the green leaf information contained in a single pre-selected pruned fruit tree using the K-means clustering algorithm. Each pre-selected pruned fruit tree corresponds to an independent group of K clusters. Each cluster contains only a portion of the green leaf information of that pre-selected pruned fruit tree, and these green leaves are highly consistent in terms of location, shape, size, color, and other characteristics. There are significant differences in the green leaf information features between different clusters.

[0080] The process of processing each leaf information of each initially selected pruned fruit tree using the K-means clustering algorithm is as follows: First, K initial points are randomly selected as centers in the leaf coordinate space of each initially selected pruned fruit tree. Then, the position coordinates of each leaf are traversed and the Euclidean distance between it and these K centers is calculated. Each leaf is then classified into the group containing the nearest center. After classification, the average coordinates of all leaves in each group are calculated, and this average value is used as the new center point. This process is iterated continuously, eventually causing the center point to gradually move towards the area with the highest leaf density. Through continuous iterative optimization, the originally discretely distributed leaves in the canopy can be divided into K sets with clear boundaries. These sets are the K clusters.

[0081] K-means clustering can group green leaves that are similar in shape, size, color, and location into the same cluster, forming groups of leaves with similar characteristics. These clusters represent different distribution patterns of leaves on fruit trees, such as new leaves, mature leaves, and old leaves.

[0082] Step S513: Based on the K clusters, a generative adversarial network is used to generate the estimated fruit-bearing area for each preliminarily selected pruned fruit tree.

[0083] Generative Adversarial Networks (GANs) are deep learning models that include a generator and a discriminator. The generator is responsible for capturing the distribution of real data and generating new data samples, while the discriminator is responsible for distinguishing whether the input data comes from real samples or fake samples generated by the generator. The generator and discriminator are continuously optimized through adversarial training, eventually reaching a Nash equilibrium, which enables the generator to generate highly realistic data. GANs have a strong ability to fit potential distributions in the field of data prediction. The input of the GAN is the K clusters, and the output of the GAN is the estimated fruit-bearing area for each pre-selected pruned fruit tree.

[0084] The predicted fruit-bearing area for each pre-selected pruned fruit tree is the spatial region with the highest future fruit yield and is predicted by processing K clusters through a generative adversarial network.

[0085] The leaf feature groups contained in the K clusters represent the distribution pattern of leaves on the fruit tree and provide necessary structured information for the Generative Adversarial Network (GAN), enabling the model to predict the fruit-bearing area based on the leaf distribution. The GAN uses a generator to convert the feature information of the K clusters into a latent representation. The generator network learns the mapping relationship between the K cluster features and the fruit-bearing area through multiple nonlinear transformations. Then, the discriminator evaluates the realism of the generated fruit-bearing area, determining whether it conforms to the fruit distribution characteristics of a real fruit tree. During training, the generator and discriminator continuously optimize through an adversarial process. The generator attempts to generate more realistic fruit-bearing areas, while the discriminator attempts to more accurately distinguish between real and generated areas. After multiple rounds of adversarial training, the generator can accurately generate the estimated fruit-bearing area for each initially selected pruned fruit tree from the feature information of the K clusters. During the generation process, the generator considers the distribution pattern of leaf clusters, fruit development patterns, and fruit tree growth characteristics to ensure that the generated fruit-bearing area conforms to the actual growth patterns of the fruit tree. For example, if the K clusters contain a large number of new leaves and leaflets, the generator will predict that the fruit-bearing area will mainly be on the outer edge of the canopy and on new shoots. If the K clusters contain a large number of mature leaves and large leaves, the generator will predict that the fruit-bearing area will mainly be on the middle of the canopy and on mature shoots. In this way, the generative adversarial network can generate accurate estimates of the fruit-bearing area based on leaf cluster information.

[0086] Step S52: Based on the multiple preliminary selected pruned fruit tree images and the estimated fruit harvest area of ​​each preliminary selected pruned fruit tree, determine the high importance branches and low importance branches of each preliminary selected pruned fruit tree.

[0087] In some embodiments, a branch analysis model can be used to determine the high-importance branches and low-importance branches of each pre-selected pruned fruit tree. The branch analysis model is a deep neural network. The input to the branch analysis model is the plurality of images of the pre-selected pruned fruit trees and the estimated fruit-bearing area of ​​each pre-selected pruned fruit tree; the output of the branch analysis model is the high-importance branches and low-importance branches of each pre-selected pruned fruit tree.

[0088] A deep neural network (DNN) is a neural network structure with multiple hidden layers. DNNs transform and abstract features from input data through stacked neurons. Each neuron in a layer receives the output of the previous layer, processes it through weighted summation and a non-linear activation function, and then passes it to the next layer. Deep neural networks can establish extremely complex functional mappings between inputs and outputs.

[0089] Each highly important branch of a fruit tree selected in the initial pruning is a branch that makes a significant contribution to a good fruit harvest, as determined by a branch analysis model.

[0090] High-importance branches are those branches on fruit trees that are located within the projected high-yield area, are in good growth condition, and have a positive impact on the fruit yield. High-importance branches should be retained during pruning.

[0091] Each low-importance branch of a fruit tree selected in the initial pruning process is a branch analysis model that identifies branches with minimal impact on fruit yield.

[0092] Low-importance branches can be those that contribute little to the area expected to yield a large harvest, or those that are poorly positioned, such as vigorous shoots, inner branches, or branches that compete for space with high-importance branches. Low-importance branches are the primary targets for pruning.

[0093] Multiple preliminary selected images of pruned fruit trees provide visual morphology and connectivity of branches. The estimated fruit-bearing area for each preliminary selected pruned tree can be marked with spatial regions requiring focused protection. A deep neural network can extract the skeletal structure of each preliminary selected pruned tree from these images, identifying the growth path, thickness, and branching nodes of each branch. The deep neural network can then perform spatial overlay analysis with the extracted branch skeleton and the estimated fruit-bearing area for each preliminary selected pruned tree. Through logical operations in the hidden layers, the deep neural network can determine the connectivity and spatial distance between each branch and the estimated fruit-bearing area. If the model identifies a branch that extends directly into the estimated fruit-bearing area, or if the branch is the main path leading to the estimated fruit-bearing area, the model will activate specific neurons to classify it as a high-importance branch. If a branch is far from the fruit-bearing area, or although close but exhibits characteristics such as weakness or interference, the model can determine that the branch contributes little or even is harmful to the fruit-bearing area, thus classifying it as a low-importance branch.

[0094] Step S53: Based on the multiple preliminary selected pruning fruit tree images, the high importance branches of each preliminary selected pruning fruit tree, and the low importance branches of each preliminary selected pruning fruit tree, determine the collaborative arm control scheme for pruning the branches of each preliminary selected pruning fruit tree.

[0095] In some embodiments, a control scheme determination model can be used to determine the cooperative arm control scheme for pruning each preliminary selected fruit tree branch. The control scheme determination model is a deep neural network model. The inputs to the control scheme determination model are the plurality of preliminary selected fruit tree images, the high-importance branches of each preliminary selected fruit tree, and the low-importance branches of each preliminary selected fruit tree; the output of the control scheme determination model is the cooperative arm control scheme for pruning each preliminary selected fruit tree branch.

[0096] Each preliminary selected collaborative arm control scheme for pruning fruit tree branches and trunks is a sequence of instructions output by the control scheme determination model to control the mechanical collaborative arm to perform pruning operations. Figure 5 This is a schematic diagram of a mechanical collaborative arm provided in an embodiment of the present invention.

[0097] The collaborative arm control scheme includes the collaborative arm's motion path, trimming force, trimming position, and trimming sequence.

[0098] Multiple preliminary selected images of the pruned fruit trees provide information on the overall morphology and spatial distribution of branches. For each preliminary selected pruned tree, high-importance branches are identified as those requiring special protection and to avoid accidental pruning, while low-importance branches are identified as the target branches to be pruned. Deep neural networks possess multi-source data fusion capabilities, complex feature deep mining capabilities, and accurate mapping modeling capabilities. Deep neural networks can map the visual image space to the motion configuration space of the collaborative arm. The model first sets the position coordinates of each low-importance branch of each preliminary selected pruned tree as the pruning target point, and the position coordinates of each high-importance branch as the obstacle avoidance constraint region. Through learning from a large amount of robotic arm kinematic data, the deep neural network can plan an optimal path from the current posture to the target pruning point. During the planning process, the model calculates the inverse kinematic solution for each step to ensure that the collaborative arm does not collide with high-importance branches during movement, while ensuring that the end effector approaches the low-importance branches at the optimal cutting angle. The model also optimizes the motion sequence, such as arranging the pruning path from the outside to the inside and from top to bottom, to reduce the wasted travel of the robotic arm. Ultimately, the model can output each preliminary, carefully selected collaborative arm control scheme for pruning fruit tree branches, containing precise motion paths, pruning positions, force, and sequence parameters.

[0099] Step S6: Based on the control scheme of the cooperative arm for each preliminarily selected fruit tree branch pruning, control the cooperative arm to perform the first pruning operation.

[0100] Once a control scheme for the collaborative arm of the preliminary selected fruit tree branch pruning is determined, the collaborative arm is controlled based on the control scheme for each preliminary selected fruit tree branch pruning to perform the first pruning operation on each preliminary selected fruit tree.

[0101] The first pruning operation is a physical cutting operation performed by the collaborative arm on the low-importance branches of each pre-selected pruning fruit tree, based on the instruction sequence in the collaborative arm control scheme for each pre-selected pruning fruit tree branch.

[0102] Step S7: Obtain an image of the orchard after the first pruning operation is completed.

[0103] The orchard image after the first pruning operation is a panoramic static digital image of the entire orchard operation area, taken by a high-definition camera mounted on a drone after the robotic arm has completed the initial pruning of the selected fruit trees.

[0104] Images of orchards after the first pruning operation record the new state of the orchards after some branches have been pruned. These images show the changes in light conditions, the spatial distribution of the remaining trees, and the alteration of branch shading relationships after removing the low-importance branches from the initially selected pruned trees.

[0105] Step S8: Determine the collaborative arm control scheme for pruning the remaining branches of the fruit orchard based on the orchard image after the first pruning operation is completed.

[0106] In some embodiments, a second scheme determination model can be used to determine the cooperative arm control scheme for pruning the remaining branches of the fruit orchard. The second scheme determination model is a deep neural network model. The input to the second scheme determination model is an image of the orchard after the first pruning operation is completed, and the output of the second scheme determination model is the cooperative arm control scheme for pruning the remaining branches of the fruit orchard.

[0107] The collaborative arm control scheme for pruning remaining fruit tree branches is determined by the second scheme, which outputs a sequence of instructions to control the mechanical collaborative arm to prune the remaining fruit tree branches after the first pruning operation. The collaborative arm control scheme for pruning remaining fruit tree branches includes the collaborative arm's motion path, pruning force, pruning position, and pruning sequence.

[0108] The orchard image after the first pruning operation realistically reflects the physical environment of the orchard after the initial pruning. The image not only includes the visual morphology of the remaining trees to be pruned, but also shows the new space freed up by removing high-priority branches, as well as the altered lighting and shading conditions. This image provides the model with updated environmental constraints and target object features, enabling it to fully utilize the cleared space when planning the pruning of the remaining trees, avoiding secondary damage to already pruned trees, and accurately identifying the branch structure of the remaining trees.

[0109] Deep neural networks (DNNs) leverage their powerful feature extraction and spatial awareness capabilities to perform in-depth analysis of orchard images after the first pruning operation. The model first identifies the location and shape of the remaining trees to be pruned after the first pruning operation and extracts the skeletal structure of their branches. The DNN analyzes the spatial environment surrounding the remaining trees, identifying the space freed up by the first pruning operation as potential movement paths for the robotic arm. Through internal neuron connection weight calculations and in conjunction with the kinematic constraints of the robotic arm, the DNN can plan a collision-free optimal path from the current position to the pruning point of the remaining trees. Based on the thickness and growth angle of the remaining tree branches, the model calculates the optimal pruning force, cutting angle, and pruning position, and arranges the pruning sequence according to the principle of optimal work efficiency. Finally, the model generates a collaborative arm control scheme for pruning the remaining tree branches, including precise motion paths, pruning positions, force, and sequence parameters.

[0110] Step S9: Based on the collaborative arm control scheme for pruning the remaining fruit tree branches, control the collaborative arm to perform the second pruning operation.

[0111] Once the control scheme for the collaborative arm in pruning the remaining fruit tree branches is determined, the collaborative arm is controlled based on the control scheme to perform a second pruning operation on the remaining fruit trees that need pruning after the first pruning operation.

[0112] The second pruning operation is a physical cutting operation performed by the collaborative arm on the remaining fruit tree branches that need to be pruned after the first pruning operation, based on the instruction sequence in the collaborative arm control scheme for pruning the remaining fruit tree branches.

[0113] Based on the same inventive concept Figure 6 This is a schematic diagram of an image analysis-based collaborative arm motion planning system provided in an embodiment of the present invention. The image analysis-based collaborative arm motion planning system includes:

[0114] Image acquisition module 61 is used to acquire images of the orchard to be pruned;

[0115] Fruit tree information determination module 62 is used to determine the distance information between multiple images of fruit trees to be pruned and the fruit trees to be pruned based on the image of the orchard to be pruned using a fruit tree determination model;

[0116] Module 63 is used to construct a pruned fruit tree map. The pruned fruit tree map includes multiple pruned fruit tree nodes and multiple edges between the multiple pruned fruit tree nodes. The node features of each pruned fruit tree node are each pruned fruit tree image, and the edges between pruned fruit tree nodes are the distance information between pruned fruit trees.

[0117] The initial selection module 64 is used to process the pruned fruit tree map based on a graph neural network to determine a number of preliminary selected pruned fruit trees;

[0118] The first control scheme determination module 65 is used to determine the collaborative arm control scheme for pruning the branches of each preliminary selected pruning fruit tree based on multiple preliminary selected pruning fruit tree images.

[0119] The first pruning control module 66 is used to control the collaborative arm to perform the first pruning operation based on the collaborative arm control scheme for each of the preliminary selected pruning fruit tree branches and trunks.

[0120] The second image acquisition module 67 is used to acquire images of the orchard after the first pruning operation is completed;

[0121] The second control scheme determination module 68 is used to determine the collaborative arm control scheme for pruning the remaining branches of the fruit orchard based on the orchard image after the first pruning operation is completed.

[0122] The second pruning control module 69 is used to control the collaborative arm to perform the second pruning operation based on the collaborative arm control scheme for pruning the remaining fruit tree branches.

[0123] It should be noted that, in order to simplify the descriptions disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments of this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0124] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method of collaborative arm motion planning based on image analysis, characterized in that, The method comprises the following steps: obtaining an image of a fruit tree forest to be pruned; determining a plurality of pruned fruit tree images and distance information between pruned fruit trees based on the image of the fruit tree forest to be pruned using a fruit tree determination model; constructing a pruned fruit tree graph, the pruned fruit tree graph comprising a plurality of pruned fruit tree nodes and a plurality of edges between the plurality of pruned fruit tree nodes, a node feature of each pruned fruit tree node being each pruned fruit tree image, and the edges between the pruned fruit tree nodes being distance information between pruned fruit trees; processing the pruned fruit tree graph based on a graph neural network to determine a plurality of preliminary selected pruned fruit trees; determining a collaborative arm control scheme for pruning of a branch of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images; controlling a collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for pruning of the branch of each preliminary selected pruned fruit tree; obtaining an image of the fruit tree forest after the first pruning operation is completed; determining a collaborative arm control scheme for pruning of a remaining branch based on the image of the fruit tree forest after the first pruning operation is completed; controlling the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning of the remaining branch.

2. The image analysis based collaborative arm motion planning method of claim 1, wherein, The method for determining the collaborative arm control scheme for pruning of the branch of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images comprises the following steps: determining an estimated fruit harvest area of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images; determining a high importance branch of each preliminary selected pruned fruit tree and a low importance branch of each preliminary selected pruned fruit tree based on the plurality of preliminary selected pruned fruit tree images and the estimated fruit harvest area of each preliminary selected pruned fruit tree; determining a collaborative arm control scheme for pruning of a branch of each preliminary selected pruned fruit tree based on the plurality of preliminary selected pruned fruit tree images, the high importance branch of each preliminary selected pruned fruit tree, and the low importance branch of each preliminary selected pruned fruit tree.

3. The image analysis based collaborative arm motion planning method of claim 2, wherein, The method for determining the estimated fruit harvest area of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images comprises the following steps: determining leaf information of each preliminary selected pruned fruit tree based on a plurality of preliminary selected pruned fruit tree images; obtaining K clusters based on the leaf information of each preliminary selected pruned fruit tree using a K-means clustering algorithm; generating an estimated fruit harvest area of each preliminary selected pruned fruit tree based on the K clusters using a generative adversarial network.

4. The image analysis based collaborative arm motion planning method of claim 1, wherein, The fruit tree determination model is a convolutional neural network model.

5. An image analysis based collaborative arm motion planning system, characterized by, The method comprises the following steps: an image acquisition module for acquiring an image of a fruit tree forest to be pruned; a fruit tree information determination module for determining a plurality of pruned fruit tree images and distance information between pruned fruit trees based on the image of the fruit tree forest to be pruned using a fruit tree determination model; a construction module for constructing a pruned fruit tree graph, the pruned fruit tree graph comprising a plurality of pruned fruit tree nodes and a plurality of edges between the plurality of pruned fruit tree nodes, a node feature of each pruned fruit tree node being each pruned fruit tree image, and the edges between the pruned fruit tree nodes being distance information between pruned fruit trees; a preliminary selection module for processing the pruned fruit tree graph based on a graph neural network to determine a plurality of preliminary selected pruned fruit trees; The first control scheme determination module is configured to determine, based on the plurality of preliminary selected pruning fruit tree images, a collaborative arm control scheme for pruning each of the preliminary selected fruit trees; The first pruning control module is configured to control the collaborative arm to perform a first pruning operation based on the collaborative arm control scheme for pruning each of the preliminary selected fruit trees; The second image acquisition module is configured to acquire a fruit tree forest image after the first pruning operation is completed; The second control scheme determination module is configured to determine, based on the fruit tree forest image after the first pruning operation is completed, a collaborative arm control scheme for pruning remaining fruit trees; The second pruning control module is configured to control the collaborative arm to perform a second pruning operation based on the collaborative arm control scheme for pruning the remaining fruit trees.

6. The image analysis based cooperative arm motion planning system of claim 5, wherein, The first control scheme determination module is further configured to: determine, based on the plurality of preliminary selected pruning fruit tree images, a predicted fruit yield area of each of the preliminary selected fruit trees; determine, based on the plurality of preliminary selected pruning fruit tree images, high importance branches of each of the preliminary selected fruit trees, and low importance branches of each of the preliminary selected fruit trees; determine, based on the plurality of preliminary selected pruning fruit tree images, the high importance branches of each of the preliminary selected fruit trees, and the low importance branches of each of the preliminary selected fruit trees, a collaborative arm control scheme for pruning each of the preliminary selected fruit trees.

7. The image analysis based cooperative arm motion planning system of claim 5, wherein, The determination of the predicted fruit yield area of each of the preliminary selected fruit trees based on the plurality of preliminary selected pruning fruit tree images comprises: determining, based on the plurality of preliminary selected pruning fruit tree images, leaf information of each of the preliminary selected fruit trees; using a K-means clustering algorithm to obtain K clusters based on the leaf information of each of the preliminary selected fruit trees; generating, based on the K clusters, a predicted fruit yield area of each of the preliminary selected fruit trees using a generative adversarial network.

8. The image analysis based cooperative arm motion planning system of claim 5, wherein, The fruit tree determination model is a convolutional neural network model.

9. An electronic device, comprising: The method comprises: a processor; a memory; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the image analysis based collaborative arm motion planning method according to any one of claims 1 to 4.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the image analysis based collaborative arm motion planning method according to any one of claims 1 to 4.

Images