An integrated method for the cultivation and picking of annonas based on machine vision

Abstract

The present invention discloses an integrated method for the cultivation and picking of annonas based on machine vision. First, image data of annonas is obtained by a drone, and a three-dimensional environmental model of the annonas is constructed in combination with a multi-view three-dimensional reconstruction algorithm, and the spatial data of the annonas is extracted. Then, the annonas are clustered according to the spatial distribution of the planting areas, and a robot equipped with a robotic arm is dispatched to each cluster for cultivation and picking operations. Next, the drone is manipulated to count the number of annonas in each cluster, and path planning is performed for the robots in each cluster. Finally, based on fuzzy mathematics, a robotic arm force control scheme, a moving speed control scheme, and a supplementary light amount control scheme for the robots in each cluster are generated, and the robots in each cluster perform cultivation and picking of the annonas according to the generated schemes. The present invention combines machine vision and fuzzy control technologies to realize the integrated service of annona cultivation and picking, improve the efficiency and yield of annona cultivation and picking, and reduce the bad fruit rate and damage rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing, and particularly relates to an integrated method for the cultivation and picking of annonas based on machine vision. Background Art

[0002] As a typical tropical and subtropical berry fruit, the pulp of annonas is rich in vitamin C, antioxidant polyphenols and essential amino acids for the human body, so it is very popular in the fruit market. However, its picking is not an easy task. Traditional manual picking requires operators to complete a series of operations such as visual positioning, manual lifting, and fruit stalk breaking at a height of several meters. There are problems such as low picking efficiency and high labor intensity. At the same time, due to the too high frequency of touching and pressing the fruit during picking, it is easy to cause invisible cell-level damage to the fruit. These hidden defects will cause abnormal ethylene release within 24 hours after picking, accelerating fruit spoilage and resulting in a serious bad fruit rate. Therefore, the intelligent and automated annona picking technology has gradually become a research hotspot. By combining technologies such as unmanned aerial vehicles, robotic arms, visual recognition, and path planning, precise picking can be achieved, the picking efficiency can be improved, and at the same time, the damage to fruits and branches can be reduced, providing technical support for the development of smart agriculture.

[0003] Most of the existing annona (sugar apple) pickings adopt automated mechanical devices to improve the picking efficiency. However, when the annona enters the fully ripe stage, the protopectin in the cell wall is hydrolyzed in large amounts under the action of polygalacturonase, resulting in a decrease in the pulp hardness. At this time, the geometric stress concentration effect formed by the scale-like protrusions on the fruit skin exacerbates the damage risk of mechanical picking. And most mechanical devices only involve picking and rarely involve the cultivation of annonas (such as pruning, pest control, etc.). Therefore, there is a lack of an integrated picking method, and the control of the operating force of the robotic arm, the moving speed of the equipment, and the required light supplement amount during the cultivation and picking process are all ignored, resulting in different degrees of damage to the fruits during the cultivation and picking process, and thus reducing the yield. Therefore, the present invention aims at the shortcomings of the existing patents and proposes an integrated method for the cultivation and picking of annonas based on machine vision. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology, and provide an integrated method for the cultivation and picking of annonas based on machine vision, which combines the fuzzy control idea and machine vision technology to realize the fuzzy control of the operating force, moving speed and required light supplement amount of the robotic arm of the robot, and improve the cultivation and picking efficiency of annonas.

[0005] To achieve the above object, on the one hand, the present invention provides an integrated method for the cultivation and picking of annonas based on machine vision, including the following steps:

[0006] Obtain the image data of sugar apples in the planting area through a drone, construct a three-dimensional environment model of the sugar apples in combination with a multi-view three-dimensional reconstruction algorithm, and extract the spatial data of the sugar apples; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed, and light sensitivity.

[0007] Cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environment model of the sugar apples, and dispatch a robot equipped with a robotic arm to each cluster for cultivation and picking operations.

[0008] Manipulate the drone to count the number of sugar apples in each cluster, and plan the path of the robot for each cluster based on the number of sugar apples in each cluster.

[0009] When the robots in each cluster are operating, generate a control scheme for the robotic arm force, moving speed, and light supplement amount of the robots in each cluster based on fuzzy mathematics, and the robots in each cluster cultivate and pick the sugar apples according to the control scheme for the robotic arm force, moving speed, and light supplement amount.

[0010] As a preferred technical solution, the drone cruises and shoots below the sugar apples in a low-altitude flight mode, and the camera faces the sky during shooting; a neutral density filter is installed on the camera.

[0011] The drone is also equipped with a light-sensitive element for obtaining the light sensitivity of the sugar apples.

[0012] Both the drone and the robot are equipped with an instance segmentation model for identifying sugar apples.

[0013] The instance segmentation model is trained on an artificially labeled dataset.

[0014] As a preferred technical solution, the construction of the three-dimensional environment model of the sugar apples and the extraction of the spatial data of the sugar apples are specifically as follows:

[0015] Deploy a drone in the planting area of the sugar apples, and regularly fly at low altitude from different heights and angles to shoot high-definition images of the sugar apples and their surrounding environment; the flight path of the drone is planned through preset waypoints or automatically.

[0016] After obtaining the sugar apple image data, use image registration technology to splice and align the two-dimensional images taken by the drone, and then calculate the depth information of each image pixel through image-based stereo vision technology to generate a high-precision depth map.

[0017] Apply a multi-view three-dimensional reconstruction algorithm to convert the depth map into a three-dimensional point cloud model, and perform noise removal, point cloud registration, and surface reconstruction through point cloud processing technology to generate a three-dimensional environment model of the sugar apples.

[0018] Based on the three-dimensional environmental model of annona squamosa, spatial data of annona squamosa are extracted using geometric analysis and volume calculation algorithms.

[0019] As a preferred technical solution, when clustering according to the spatial distribution of the planting areas, first obtain the spatial coordinates of annona squamosa through the three-dimensional environmental model of annona squamosa, and use the elbow method and silhouette coefficient method to determine the optimal number of clustering clusters; then use the clustering algorithm to perform spatial clustering on annona squamosa according to the optimal number of clustering clusters and the spatial distribution of the planting areas to obtain multiple clusters.

[0020] As a preferred technical solution, when performing path planning, first control the unmanned aerial vehicle to count the number of annona squamosa in each cluster and set a quantity threshold; then judge the number of annona squamosa in each cluster and the quantity threshold. If the number of annona squamosa in a certain cluster is less than or equal to the quantity threshold, use the heuristic algorithm to perform path planning for the robot in that cluster; if the number of annona squamosa in a certain cluster is greater than the quantity threshold, use the heuristic search algorithm to perform path planning for the robot in that cluster.

[0021] As a preferred technical solution, the mechanical arm force control scheme, moving speed control scheme and supplementary light quantity control scheme of each cluster of robots are generated based on fuzzy mathematics, specifically:

[0022] Calculate the operation feasibility and operation execution ability index of each cluster of robots, respectively construct the fuzzy relationship matrix between the mechanical arm operation force of the robot and the operation feasibility and operation execution ability index, and use the maximum membership degree method to obtain the mechanical arm force control scheme of each cluster of robots;

[0023] Calculate the equipment movement stability of each cluster of robots, respectively construct the fuzzy relationship matrix between the moving speed of the robot and the operation feasibility and equipment movement stability, and use the maximum membership degree method to obtain the moving speed control scheme of each cluster of robots;

[0024] Calculate the operation environment light efficiency index of each cluster of robots, construct the fuzzy relationship matrix between the photosensitive quantity of the robot and the operation environment light efficiency index, calculate the supplementary light quantity required for annona squamosa picking, and use the maximum membership degree method to obtain the supplementary light quantity control scheme of each cluster of robots.

[0025] As a preferred technical solution, the specific method for obtaining the mechanical arm force control scheme of each cluster of robots using the maximum membership degree method is as follows:

[0026] Define the operation feasibility and operation execution ability index, expressed as:

[0027] ,

[0028] ,

[0029] Among them,TP is the job feasibility, ranging from [0, 10], and the larger the value, the lower the feasibility; M is the maturity, ranging from [0, 1]; F is the wind speed, ranging from [0, 1]; H is the humidity, ranging from [0, 1]; Te is the temperature, with the unit of degree Celsius; Te opt is the optimal picking temperature of annona squamosa, w 1, w 2, w 3, w 4 are the weights of maturity, wind speed, humidity, and temperature respectively; a 1, b 2, c 3, d 4 are the adjustment coefficients of maturity, wind speed, humidity, and temperature respectively; OPI is the job execution ability index; E is the working efficiency of the operator; D is the status of the equipment or tool, with the value of 0 or 1; C is the complexity of the job area; T ' is the operation time; A is the adaptability and flexibility of the job; w 5, w 6, w 7, w 8, w 9 are the weights of working efficiency, status, complexity, operation time, and the adaptability and flexibility of the job respectively; α 2, α 3, α 4, α 5, α 6 are the non - linear weighting coefficients of working efficiency, status, complexity, operation time, and the adaptability and flexibility of the job respectively, β 1, β 2, β 3, β 5, β 6 are the adjustment coefficients of working efficiency, status, complexity, operation time, and the adaptability and flexibility of the job respectively;

[0030] Define the fuzzy matrix domain and membership function of the robot's job feasibility, and construct the fuzzy matrix of the robot's job feasibility P matrix :

[0031] ,

[0032] Among them, ELPRepresent high feasibility, LP Represent relatively high feasibility, MP Represent medium feasibility, HP Represent relatively low feasibility, EHP Represent low feasibility;

[0033] Define the fuzzy matrix domain and its membership function of the operation execution ability index of the robot, and construct the fuzzy matrix of the operation execution ability index of the robot C o :

[0034] ,

[0035] Among them, ELC Represent extremely low execution ability, LC Represent relatively low execution ability, MC Represent medium execution ability, HC Represent relatively high execution ability, EHC Represent high execution ability;

[0036] Define the fuzzy matrix domain and its membership function of the manipulator operation force of the robot, and construct the fuzzy matrix of the manipulator operation force of the robot S matrix :

[0037] ,

[0038] Among them, ELS Represent low force, LS Represent relatively low force, MS Represent medium force, HS Represent relatively high force, EHS Represent high force;

[0039] Construct the fuzzy relation matrix of the operation feasibility of the robot and the manipulator operation force M 1 and the fuzzy relation matrix of the operation execution ability index of the robot and the manipulator operation force M 2:

[0040] ,

[0041] ,

[0042] Among them, T Is the matrix transpose operation;

[0043] Couple the fuzzy relation matrix M 1, M 2, update the manipulator operation force of the robot and its membership function:

[0044] ,

[0045] Among them, is the fuzzy matrix of the manipulator operation force of the updated robot.

[0046] As an optimal technical solution, the maximum membership degree method is used to obtain the movement speed control scheme of each cluster of robots, specifically:

[0047] Define the equipment movement stability:

[0048] ,

[0049] Among them, TAI is the equipment movement stability; Sl is the slope, Ls is the land stability, w 10 、 w 11 are the weights of the slope and the land stability respectively, α 7, α 8 are the nonlinear coefficients for controlling the slope and the land stability respectively, β 7, β 8 are the adjustment coefficients of the slope and the land stability respectively;

[0050] Define the fuzzy matrix universe and its membership function of the equipment movement stability of the robot, and construct the fuzzy matrix of the equipment movement stability TAI matrix :

[0051] ,

[0052] Among them, ELT represents high stability, LT represents relatively high stability, MT represents medium stability, HT represents relatively low stability, EHT represents low stability;

[0053] Define the fuzzy matrix universe and its membership function of the movement speed of the robot, and construct the fuzzy matrix of the movement speed V matrix :

[0054] ,

[0055] Among them, ELV represents stationary, LV represents low-speed movement, MV represents medium-speed movement, HV represents relatively high-speed movement, EHV represents high-speed movement;

[0056] Construct the fuzzy relation matrix of the device movement stability and movement speed of the robot M 3 and the fuzzy relation matrix of the operation feasibility and movement speed of the robot M 4:

[0057] ,

[0058] ,

[0059] Among them, P matrix is the fuzzy matrix of operation feasibility, T is the matrix transpose operation;

[0060] Coupled fuzzy relation matrix M 3, M 4, update the fuzzy matrix of the movement speed of the robot and its membership function:

[0061] ,

[0062] Among them, is the fuzzy matrix of the updated movement speed of the robot.

[0063] As a preferred technical solution, the method for obtaining the fill light amount control scheme for each cluster of robots using the maximum membership degree method is specifically:

[0064] Define the light efficiency index of the operation environment:

[0065] ELEI = L × e -αH ,

[0066] Among them, L is the photosensitive amount of the annona corresponding to the robot during operation, H is the air humidity, α is the adjustment constant for adjusting the influence of humidity on the light efficiency index of the operation environment;

[0067] Define the fuzzy matrix domain and its membership function of the light efficiency index of the robot's operation environment, and construct the fuzzy matrix of the light efficiency index of the operation environment ELEI matrix :

[0068] ,

[0069] Among them, ELE represents a low light efficiency index, LE represents a relatively low light efficiency index, ME represents a medium light efficiency index, HE represents a relatively high light efficiency index,EHE Represents a high photosynthetic efficiency index;

[0070] Define the fuzzy matrix domain and its membership function for the light sensing amount of the robot, and construct the fuzzy matrix of the light sensing amount of the robot L matrix :

[0071] ,

[0072] Among them, ELL Represents a low light sensing amount, LL Represents a relatively low light sensing amount, ML Represents a medium light sensing amount, HL Represents a relatively high light sensing amount, EHL Represents a high light sensing amount;

[0073] Construct the fuzzy relationship matrix between the light efficiency index and the light sensing amount of the robot's working environment M 5:

[0074] ,

[0075] Among them, T Is the matrix transpose operation;

[0076] Based on the inverse relationship between the light sensing amount and the supplementary light amount, according to the fuzzy relationship matrix between the light efficiency index and the light sensing amount of the working environment M 5 Calculate and update the fuzzy matrix and its membership function of the supplementary light amount of the robot:

[0077] ,

[0078] Among them, Is the updated fuzzy matrix of the supplementary light amount of the robot.

[0079] On the other hand, the present invention also provides an integrated system for cultivating and harvesting annonas based on machine vision, which is applied to the above-mentioned integrated method for cultivating and harvesting annonas based on machine vision. The system includes a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module, and a main control module;

[0080] The data acquisition module is used to obtain the image data of annonas in the planting area through an unmanned aerial vehicle, and construct a three-dimensional environment model of annonas in combination with a multi-view three-dimensional reconstruction algorithm and extract the spatial data of annonas; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed, and light sensing amount;

[0081] The cloud computing module is used to cluster annonas according to the spatial distribution of the planting area based on the three-dimensional environment model of annonas, and dispatch a robot equipped with a robotic arm for each cluster to perform cultivation and harvesting operations;

[0082] The visual module is used to manipulate the drone to count the number of annonas in each cluster, and plan the paths of the robots in each cluster according to the number of annonas in each cluster;

[0083] The individual calculation module is used to generate control schemes for the robotic arm force, moving speed, and supplementary light amount of the robots in each cluster based on fuzzy mathematics when the robots in each cluster are operating;

[0084] The main control module is used to cultivate and pick annonas by the robots in each cluster according to the control schemes for the robotic arm force, moving speed, and supplementary light amount;

[0085] The communication module is used for encrypted transmission of data between modules;

[0086] The control module is responsible for task scheduling, data management, and logic control between modules.

[0087] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0088] 1. The present invention proposes an integrated method for cultivating and picking annonas based on machine vision. Based on the fuzzy control idea and machine vision technology, a three-dimensional environmental model of annonas is constructed by using a drone combined with multi-view vision reconstruction technology, and the path of the picking device is planned according to the three-dimensional coordinates. At the same time, fuzzy control is performed on the operating force of the robotic arm, moving speed, and required supplementary light amount of the annona picking device, so as to improve the efficiency of cultivating and picking annonas, reduce the bad fruit rate and damage rate, and increase the yield of annonas.

[0089] 2. The present invention uses a drone to fly below the annonas and the camera faces the sky for shooting to avoid the influence of sunlight exposure, and reduces light interference by installing a neutral density filter; at the same time, the drone is also equipped with a photosensitive component, which is used to obtain the photosensitive amount when the drone is positioned directly below the annonas, so as to prepare for the fuzzy control of the required supplementary light amount. In addition, the drone and the robot are also equipped with an instance segmentation model, which can not only identify the target annonas, but also accurately segment the pixel-level area of each target, further providing the fine contour information of the annonas, improving the accuracy of annona recognition, and ensuring the accuracy of cultivation and picking.

[0090] 3. When the present invention performs clustering, it first uses the elbow method and the silhouette coefficient method to determine the optimal number of clustering clusters, which can effectively determine the optimal number of clustering clusters, avoiding redundancy or insufficiency of operations caused by excessive or insufficient clustering. Then, it uses the clustering algorithm to perform spatial clustering on the sugar apples, which helps to improve the efficiency and accuracy of the cultivation and picking operations of sugar apples. Then, a robot equipped with a robotic arm is dispatched to each cluster for cultivation and picking operations, so that each cluster of sugar apples can be individually cultivated and picked by a dedicated robot, optimizing the task allocation of the robot operations, making the working range of each robot more concentrated, reducing duplication and conflicts in the operations, and further enhancing the automation and accuracy of the cultivation and picking operations, thus significantly improving the overall operation efficiency.

[0091] 4. In the method of the present invention, when the robot equipped with a robotic arm performs cultivation and picking operations, based on the idea of fuzzy mathematics, considering factors such as operation feasibility, operation execution ability index, equipment movement stability, and operation environment light effect index, the robot is systematically controlled, enabling the robot to adjust and adapt to changes in the outside world, facilitating making the optimal decision during the cultivation and picking of sugar apples and improving the efficiency of the cultivation and picking of sugar apples. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0093] Figure 1 It is the overall flowchart of the integrated method for sugar apple cultivation and picking based on machine vision in the embodiments of the present invention.

[0094] Figure 2 It is the flowchart of constructing the three-dimensional environment model of sugar apples and extracting the spatial data of sugar apples in the embodiments of the present invention.

[0095] Figure 3 It is the flowchart of generating the robotic arm force control scheme in the embodiments of the present invention.

[0096] Figure 4 It is the flowchart of generating the moving speed control scheme in the embodiments of the present invention.

[0097] Figure 5 It is the flowchart of generating the supplementary light amount control scheme in the embodiments of the present invention.

[0098] Figure 6 It is the structural schematic diagram of the integrated system for sugar apple cultivation and picking based on machine vision in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0099] In order to enable those skilled in the art of this technology to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts belong to the scope of protection of this application.

[0100] When "embodiment" is mentioned in this application, it means that the specific features, structures or characteristics described in combination with the embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0101] Embodiment 1

[0102] As Figure 1 shown, this embodiment provides an integrated method for the cultivation and picking of annonas based on machine vision, and the steps are as follows:

[0103] S1. Obtain the image data of annonas in the planting area through a drone, construct a three-dimensional environment model of the annonas in combination with a multi-view three-dimensional reconstruction algorithm, and extract the spatial data of the annonas; wherein, the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed, light sensitivity, etc.

[0104] Specifically, the drone cruises and shoots below the annonas in a low-altitude flight mode, and the camera faces the sky during shooting to avoid the influence of sunlight exposure. To reduce the interference of light, a neutral density filter (ND filter) (such as ND4, ND8, ND16, etc.) is installed on the camera to reduce the light and avoid overexposure of sunlight. At the same time, the drone is also equipped with a light-sensitive element (such as a photodiode, CCD / CMOS sensor, etc.) for obtaining the light sensitivity of the annonas, and further providing accurate data for evaluating the growth state of the annonas.

[0105] In addition, both the drone and the robot are equipped with an instance segmentation model for identifying sugar apples to improve accuracy; the instance segmentation model is trained using a manually labeled dataset to improve its accuracy. In this embodiment, the manually labeled dataset is made based on the sugar apple image data obtained by the drone, but this application is not limited thereto, and datasets made by other methods can also be used to train the instance segmentation model as long as they can identify sugar apples. In this embodiment, the YOLOv8-seg model is used as the instance segmentation model, and other deep learning models that can achieve the same function can also be regarded as the instance segmentation model in this application and also fall within the protection scope of this application.

[0106] Further, as Figure 2 shown, the steps for this application to construct a three-dimensional environment model of sugar apples and extract the spatial data of sugar apples are as follows:

[0107] First, deploy a drone equipped with a high-precision camera and sensors in the sugar apple planting area, and fly low at different heights and angles regularly to take high-definition images of sugar apples and their surrounding environment; among them, the flight path of the drone is planned through preset waypoints or automatically to achieve full coverage of the planting area and ensure obtaining the image data of sugar apples from multiple angles and all-round.

[0108] After obtaining the sugar apple image data, use image registration technology to splice and align the two-dimensional images taken by the drone to eliminate the influence caused by the change of camera position and the difference in viewing angles; then calculate the depth information of each image pixel through image-based stereo vision technology to generate a high-precision depth map.

[0109] Next, apply a multi-view three-dimensional reconstruction algorithm to convert the depth map into a three-dimensional point cloud model, and perform noise removal, point cloud registration, and surface reconstruction through point cloud processing technology to generate a three-dimensional environment model of sugar apples.

[0110] Finally, based on the three-dimensional environment model of sugar apples, use geometric analysis and volume calculation algorithms to extract the spatial data of sugar apples, such as spatial coordinates, shape, size, volume, temperature, humidity, wind speed, and light exposure amount, etc.

[0111] S2. Cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environment model of sugar apples, and dispatch a robot equipped with a robotic arm for each cluster to perform cultivation and harvesting operations.

[0112] Further, when clustering according to the spatial distribution of the planting areas, first obtain the spatial coordinates of the sugar apples through the three-dimensional environment model of the sugar apples, and use the elbow method and the silhouette coefficient method to determine the optimal number of clustering clusters to avoid redundancy or insufficiency of operations caused by excessive or insufficient clustering. Then, perform spatial clustering on the sugar apples using the clustering algorithm according to the optimal number of clustering clusters and the spatial distribution of the planting areas to obtain multiple clusters, which helps to improve the efficiency and accuracy of the cultivation and picking operations of the sugar apples. For each cluster, dispatch a robot equipped with a robotic arm to perform cultivation and picking operations, so that there is a dedicated robot in each cluster to perform cultivation and picking operations (such as pruning branches and leaves, pest control, applying pesticides, fruit picking, etc.), ensuring that the working range of the robot is more concentrated, reducing repetition and conflicts in the operations, and improving the operation efficiency.

[0113] In this embodiment, the K-means clustering algorithm is used as the clustering algorithm. Of course, other clustering algorithms such as DBSCAN and hierarchical clustering can also be used. As long as the spatial clustering of the sugar apples can be achieved, it is considered to fall within the protection scope of this application.

[0114] S3. Operate the drone to count the number of sugar apples in each cluster, and plan the path of the robot for each cluster according to the number of sugar apples in each cluster.

[0115] Specifically, when performing path planning, first control the drone to count the number of sugar apples in each cluster and set a quantity threshold. Then, judge the number of sugar apples in each cluster and the quantity threshold. If the number of sugar apples in a certain cluster is less than or equal to the quantity threshold, use a heuristic algorithm (such as the greedy algorithm) to plan the path of the robot in this cluster; if the number of sugar apples in a certain cluster is greater than the quantity threshold, use a heuristic search algorithm (such as the A* algorithm) to plan the path of the robot in this cluster, ensuring that the cultivation and picking process is efficient, the traveling path is the shortest, and obstacle interference is avoided.

[0116] In this embodiment, the quantity threshold is set to 300. For the robots with the number of sugar apples in each cluster greater than or equal to 300, the greedy algorithm is used to plan their paths, otherwise the A* algorithm is used to plan the paths of the robots in this cluster. Of course, other algorithms can also be selected for path planning according to actual needs. For example, the heuristic algorithm can be a simulated annealing algorithm, a genetic algorithm, a tabu search algorithm, etc., and the heuristic search algorithm can use the best-first search algorithm, etc. In addition, path planning algorithms such as the Dijkstra algorithm, the Rapidly-exploring Random Tree algorithm (RTT), and the Dynamic Window Approach (DWA) can also be used for path planning to achieve the same purpose as this application.

[0117] S4. When each cluster of robots is working, a mechanical arm strength control plan, a moving speed control plan and a fill light control plan are generated based on fuzzy mathematics. Each cluster of robots cultivates and picks the sugar apples according to the mechanical arm strength control plan, the moving speed control plan and the fill light control plan.

[0118] Specifically, in the prior art, the control of the robot arm's operating force, movement speed and fill light amount during the picking process has been neglected. First, inaccurate control of the robot arm's operating force may cause the sugar apple to be damaged by excessive force during the picking process, affecting the quality of the fruit and increasing the rate of bad fruit; secondly, improper control of the movement speed may cause the robot arm to operate unsmoothly and be unable to adapt to different environments and fruit conditions, thereby reducing operating efficiency and accuracy; finally, insufficient or unbalanced fill light amount control will affect the lighting conditions of the sugar apple during the cultivation process, and then affect the growth and maturity of the fruit, resulting in unsatisfactory picking results and affecting yields; therefore, based on these three considerations, this application generates a robot arm force control scheme, a movement speed control scheme and a fill light amount control scheme for each cluster of robots based on fuzzy mathematics, and the steps are:

[0119] S4.1. First, calculate the operation feasibility and operation execution capability index of each cluster of robots, respectively construct the fuzzy relationship matrix between the robot's mechanical arm operation force and the operation feasibility and operation execution capability index, and use the maximum membership method to obtain the mechanical arm force control plan of each cluster of robots.

[0120] Specifically, Figure 3 As shown in the figure, the steps for generating the robot arm force control solution are:

[0121] S4.1.1 First, define the task feasibility and task execution capability index; among them, task feasibility (Taskfeasibility, TP ) is used to evaluate the impact of external environment (such as wind speed, humidity, temperature, etc.) and the maturity of sugar apple on the cultivation and picking operations. The larger the value, the more difficult the operation is. Operational Performance Index (OPI) OPI ) measures the impact of factors such as operators, equipment, and complexity of the operating area on the operating effect in actual operations, which are expressed as:

[0122] ,

[0123] ,

[0124] in, TP is the feasibility of the operation, ranging from [0,10], the larger the value, the lower the feasibility; Mis the maturity level, ranging from [0, 1] (immature to fully mature); F is the wind speed, ranging from [0, 1] (calm to strong wind); H is the humidity, ranging from [0, 1] (low humidity to high humidity); Te is the temperature, in degrees Celsius; Te opt is the optimal picking temperature of annona squamosa (e.g., 25°C), w 1, w 2, w 3, w 4 are the weights of maturity level, wind speed, humidity, and temperature respectively; a 1, b 2, c 3, d 4 are the adjustment coefficients of maturity level, wind speed, humidity, and temperature respectively, used to adjust the sensitivity of each factor to risk; OPI is the operation execution ability index; E is the work efficiency of the operator, D is the state of the robot, taking values of 0 (not feasible) or 1 (feasible); C is the complexity of the operation area; T ' is the operation time; A is the adaptability and flexibility of the operation (such as whether it can flexibly respond to emergencies); w 5, w 6, w 7, w 8, w 9 are the weights of work efficiency, state, complexity, operation time, and adaptability and flexibility of the operation respectively; α 2, α 3, α 4, α 5, α 6 are the non - linear weighting coefficients of work efficiency, state, complexity, operation time, and adaptability and flexibility of the operation respectively, β 1, β 2, β 3, β 5, β 6 are the adjustment coefficients of work efficiency, state, complexity, operation time, and adaptability and flexibility of the operation respectively.

[0125] In this embodiment, the maturity level MObtained by spectral detection technology; during the process of detecting the maturity of annonas using spectroscopy, a threshold wavelength can be defined to distinguish different spectral characteristics for identifying maturity. By detecting the pigment content on the surface of annonas in the visible light band (400 - 700 nm), the color change of the fruit peel can be reflected, thus preliminarily judging its external maturity. At the same time, by detecting the water, sugar, and other chemical components inside the annonas in the near-infrared band (700 - 2500 nm), its internal maturity can be evaluated. Combining the data in the visible light and near-infrared bands and using the threshold wavelength for comprehensive evaluation, the overall maturity of annonas can be accurately judged. The complexity of the operation area C Refers to the spatial area complexity of each cluster, measured by the average slope of the area. The adaptability and flexibility of the operation A Quantified by the foliage coverage of the annona area when using the robotic arm for operation.

[0126] S4.1.2. Define the fuzzy matrix domain and membership function of the operation feasibility of the robot, and construct the fuzzy matrix of the operation feasibility of the robot P matrix :

[0127] ,

[0128] Among them, ELP Represents high feasibility, LP Represents relatively high feasibility, MP Represents medium feasibility, HP Represents relatively low feasibility, EHP Represents low feasibility.

[0129] S4.1.3. Define the fuzzy matrix domain and membership function of the operation execution ability index of the robot, and construct the fuzzy matrix of the operation execution ability index of the robot C o :

[0130] ,

[0131] Among them, ELC Represents extremely low execution ability, LC Represents relatively low execution ability, MC Represents medium execution ability, HC Represents relatively high execution ability, EHC Represents high execution ability.

[0132] S4.1.4. Define the fuzzy matrix domain and membership function of the robotic arm operation force of the robot, and construct the fuzzy matrix of the robotic arm operation force of the robot S matrix :

[0133] ,

[0134] Among them, ELS represents a very low intensity, LS represents a relatively low intensity, MS represents a medium intensity, HS represents a relatively high intensity, EHS represents a high intensity.

[0135] S4.1.5. Construct the fuzzy relation matrix between the operation feasibility of the robot and the operation force of the robotic arm M 1 and the fuzzy relation matrix between the operation execution ability index of the robot and the operation force of the robotic arm M 2:

[0136] ,

[0137] ,

[0138] Among them, T is the matrix transpose operation;

[0139] S4.1.6. Couple the fuzzy relation matrix M 1, M 2, and update the operation force of the robotic arm of the robot and its membership function:

[0140] ,

[0141] Among them, is the fuzzy matrix of the updated operation force of the robotic arm of the robot.

[0142] In this embodiment, the domain of the fuzzy matrix of the operation feasibility of the robot is set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, and its corresponding membership function is:

[0143] ,

[0144] The domain of the fuzzy matrix of the operation execution ability index of the robot is set to {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}, and its corresponding membership function is:

[0145] ,

[0146] The domain of the fuzzy matrix of the operation force of the robotic arm of the robot is set to {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}, and its corresponding membership function is:

[0147] ,

[0148] Among them,S is the current robot arm operating force value, then the updated robot arm operating force fuzzy matrix calculated by the above steps is The membership function is expressed as:

[0149] ,

[0150] Control the robot's arm in the cluster based on the updated robot's arm operating force:

[0151] when S' When (operation force value) <3, it means that the robot arm needs to use low force during the cultivation and picking process, and the cultivation and picking actions should be performed gently to avoid damage to the fruit and ensure that the fruit quality during the cultivation and picking process is not affected;

[0152] When 3 ≤ S' When it is <5, it means that the mechanical arm needs a lower force during the cultivation and picking process. The force can be increased appropriately during operation, but care should still be taken to protect the surface of the fruit to avoid excessive force that may damage the peel;

[0153] When 5 ≤ S' <7, it means that the robot arm needs medium force during cultivation and picking. At this time, stronger force can be used for cultivation and picking, but the stability of the robot arm must still be ensured to avoid damaging the internal structure of the fruit;

[0154] When 7 ≤ S' <9: It means that the robot arm needs a higher force during the cultivation and picking process. A stronger force can be used during cultivation and picking, but it needs to be operated with extra care to ensure that the robot arm does not cause external damage to the fruit;

[0155] When 9 ≤ S' When ≤ 11, it means that the robot arm needs high force during the cultivation and picking process. At this time, it should be ensured that the robot arm can fully withstand the large force to ensure that the cultivation and picking process is rapid without causing too much pressure on the fruit.

[0156] S4.2. Calculate the equipment movement stability of each cluster of robots, construct the fuzzy relationship matrix between the robot's movement speed and the operation feasibility and equipment movement stability, and use the maximum membership method to obtain the movement speed control plan of each cluster of robots.

[0157] Specifically, Figure 4 As shown, the steps for generating the mobile speed control scheme are:

[0158] S4.2.1. Define device mobility stability, which is used to measure the mobility stability of the robot under different environmental conditions, especially the response to slope and land stability, expressed as:

[0159] ,

[0160] Among them, TAI is the equipment movement stability; Sl is the slope, Ls is the land stability, w 10 and w 11 are the weights of the slope and land stability respectively, representing the influence degree on the equipment movement stability; α 7, α 8 are the non - linear coefficients for controlling the slope and land stability respectively, β 7, β 8 are the adjustment coefficients of the slope and land stability respectively. Among them, Sl the larger the value, the larger the slope and the lower the equipment movement stability; Ls the larger the value, the lower the land stability and the lower the equipment movement stability. The slope ( Sl ) and the land stability ( Ls ) are important factors affecting the equipment stability, TAI the calculation of

[0161] S4.2.2. Define the fuzzy matrix domain and its membership function of the equipment movement stability of the robot, and construct the fuzzy matrix of the equipment movement stability TAI matrix :

[0162] ,

[0163] Among them, ELT represents high stability, LT represents relatively high stability, MT represents medium stability, HT represents relatively low stability, EHT represents low stability.

[0164] S4.2.3. Define the fuzzy matrix domain and its membership function of the movement speed of the robot, and construct the fuzzy matrix of the movement speed V matrix :

[0165] ,

[0166] Among them, ELV represents moving at a relatively low speed, LV represents moving at a low speed, MV represents moving at a medium speed,HV Represents a higher speed movement, EHV Represents a high speed movement.

[0167] S4.2.4. Construct the fuzzy relation matrix of the equipment movement stability and movement speed of the robot M 3 and the fuzzy relation matrix of the operation feasibility and movement speed of the robot M 4:

[0168] ,

[0169] ,

[0170] Among them, P matrix is the fuzzy matrix of operation feasibility, T is the matrix transpose operation.

[0171] S4.2.5. Coupled fuzzy relation matrix M 3, M 4, update the fuzzy matrix of the movement speed of the robot and its membership function:

[0172] ,

[0173] Among them, is the fuzzy matrix of the updated movement speed of the robot.

[0174] In this embodiment, the domain of the fuzzy matrix of the equipment movement stability of the robot is set to {0, 1, 2, 3, 4, 5}, and its corresponding membership function is:

[0175] ,

[0176] The domain of the fuzzy matrix of the movement speed of the robot is set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}, and its corresponding membership function is:

[0177] ,

[0178] Among them, V is the current movement speed of the robot. After the above steps, the membership function of the updated fuzzy matrix of the movement speed of the robot is expressed as:

[0179] ,

[0180] Based on the updated movement speed of the robot, perform movement control on the robots within the cluster. According to the movement speed value ( V'Depending on the difference of

[0181] When V' = 0, it means that the robot is in a stationary state, suitable for delicate cultivation and picking operations or waiting for instructions. For example, conducting detailed inspections on the branches, leaves, and fruits of sugar apples (such as monitoring growth status), waiting for picking instructions, etc.

[0182] When 3 ≤ V' < 6, it means that the robot is in a low-speed moving state, suitable for relatively delicate operations, such as detecting the growth of fruits and proper pruning of branches and leaves.

[0183] When 6 ≤ V' < 9, it means that the robot is in a medium-speed moving state, suitable for moving at a general speed, which can ensure work efficiency and maintain stability. For example, conducting daily inspections along the planned path, identifying early diseases, and implementing intelligent water and fertilizer management (such as automatic irrigation and precise fertilization).

[0184] When 9 ≤ V' < 12, it means that the robot is in a relatively high-speed moving state, suitable for large-scale cruising and task execution in a larger range. For example, cruising in a large area within a cluster of orchards, detecting the maturity of sugar apples to generate a picking task list, pre-scanning and path planning before picking, etc.

[0185] When 12 ≤ V' ≤ 15, it means that the robot is in a high-speed moving state, suitable for quickly covering a large area for picking or other operations. For example, quickly reaching the target area, executing automated picking tasks, classifying and collecting fruits, etc.

[0186] S4.3. Calculate the light efficiency index of the working environment for each cluster of robots, construct a fuzzy relationship matrix between the light-sensitive amount of the robot and the light efficiency index of the working environment, calculate the amount of supplementary light required for sugar apple picking, and use the maximum membership degree method to obtain the supplementary light control scheme for each cluster of robots.

[0187] Specifically, as Figure 5 shown, the steps for generating the supplementary light control scheme are as follows:

[0188] S4.3.1. Define the light efficiency index of the working environment, which is used to measure the impact of the required light and humidity for sugar apples on the operation efficiency during the robot's cultivation and picking process, expressed as:

[0189] ELEI = L × e -αH ,

[0190] where L is the light-sensitive amount of the sugar apple corresponding to the robot during operation, reflecting the light intensity;H is the air humidity, and the unit can be percentage (0 - 100%); α is the adjustment constant, which is used to adjust the influence of humidity on the light efficiency index of the working environment. As the humidity increases, the lighting effect will weaken, thereby affecting the efficiency of cultivation and picking operations; by calculating the ELEI, the current environmental lighting conditions can be understood, which helps to optimize the robot's operation strategy and improve the cultivation and picking efficiency.

[0191] S4.3.2. Define the fuzzy matrix domain and its membership function of the light efficiency index of the robot's working environment, and construct the fuzzy matrix of the light efficiency index of the working environment ELEI matrix :

[0192] ,

[0193] wherein, ELE represents a low light efficiency index, LE represents a relatively low light efficiency index, ME represents a medium light efficiency index, HE represents a relatively high light efficiency index, EHE represents a high light efficiency index.

[0194] S4.3.3. Define the fuzzy matrix domain and its membership function of the light sensing amount of the robot, and construct the fuzzy matrix of the light sensing amount of the robot L matrix :

[0195] ,

[0196] wherein, ELL represents a low light sensing amount, LL represents a relatively low light sensing amount, ML represents a medium light sensing amount, HL represents a relatively high light sensing amount, EHL represents a high light sensing amount.

[0197] S4.3.4. Construct the fuzzy relation matrix of the light efficiency index and the light sensing amount of the robot M 5:

[0198] ,

[0199] wherein, T is the matrix transpose operation.

[0200] S4.3.5. Based on the inverse relationship between the light sensing amount and the supplementary light amount, according to the fuzzy relation matrix M 5 of the light efficiency index and the light sensing amount of the working environment, calculate and update the fuzzy matrix and its membership function of the supplementary light amount of the robot:

[0201] ,

[0202] Among them, is the fuzzy matrix of the supplementary light amount of the updated robot.

[0203] In this embodiment, the universe of discourse of the fuzzy matrix of the light efficiency index of the robot's working environment is set to {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}, and its corresponding membership function is:

[0204] ,

[0205] The universe of discourse of the fuzzy matrix of the light-sensitive amount of the robot is set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, and its corresponding membership function is:

[0206] ,

[0207] After the above steps of calculation, the membership function of the supplementary light amount fuzzy matrix of the updated robot is:

[0208] ,

[0209] According to the updated supplementary light amount, the robot trims the branches and leaves of the annona squamosa, and the strategy is as follows: when L ’ (the value of the supplementary light amount) < 2, it means that the annona squamosa needs a high supplementary light amount, and large-scale pruning should be carried out to remove the dense branches and leaves blocking the fruits and improve the light penetration rate; when 2 ≤ L ’ < 4, it means that the annona squamosa needs a relatively high supplementary light amount, and some branches and leaves can be appropriately pruned to reduce the blockage but maintain moderate photosynthesis; when 4 ≤ L ’ < 6, it means that the annona squamosa needs a medium supplementary light amount, and only the overlapping branches and leaves are lightly pruned to optimize the internal light transmittance; when 6 ≤ L ’ < 8, it means that the annona squamosa needs a relatively low supplementary light amount, and only the branches and leaves need to be slightly adjusted to avoid over-pruning affecting the fruit growth; when 8 ≤ L ’ ≤ 10, it means that the annona squamosa needs a low supplementary light amount. At this time, the light is sufficient, and pruning can be reduced to maintain the crown structure to promote the balanced ripening of the fruits.

[0210] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously.

[0211] Example 2

[0212] As Figure 6 shown, another embodiment of the present invention provides an integrated system for the cultivation and picking of annonas based on machine vision, including a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module, and a main control module;

[0213] Among them, the data acquisition module is used to obtain the image data of annonas in the planting area through a drone, construct a three-dimensional environment model of the annonas in combination with a multi-view three-dimensional reconstruction algorithm, and extract the spatial data of the annonas; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed, light exposure amount, etc.;

[0214] The cloud computing module is used to cluster the annonas according to the spatial distribution of the planting area based on the three-dimensional environment model of the annonas, and dispatch a robot equipped with a robotic arm for each cluster to perform operations;

[0215] The vision module is used to manipulate the drone to count the number of annonas in each cluster, and plan the path of the robot for each cluster according to the number of annonas in each cluster;

[0216] The individual computing module is used to generate a control scheme for the robotic arm force, a control scheme for the moving speed, and a control scheme for the light supplement amount of the robot for each cluster based on fuzzy mathematics when the robot for each cluster is operating;

[0217] The main control module is used for the robot for each cluster to prune the branches and leaves or pick the fruits of the annonas according to the control scheme for the robotic arm force, the control scheme for the moving speed, and the control scheme for the light supplement amount;

[0218] The communication module is used for the encrypted transmission of data between modules to ensure that the data is not stolen or tampered with during the transmission process. In this embodiment, the communication module uses the RSA algorithm for encryption and decryption, that is: the message is encrypted using the public key of the recipient, and only the private key of the recipient can decrypt it, so as to ensure the confidentiality of the data. In addition, the message authentication in the communication module uses the SHA-256 hash algorithm to generate a digest of the message, and combines an appropriate signature or authentication mechanism to ensure the integrity of the message and the authenticity of the source. Of course, in addition to the RSA algorithm mentioned for encryption and decryption, other encryption methods (such as AES, DES, SHA-3, ECC, etc.) can also be used, and they are not listed one by one in this application; the same is true for message authentication, and other authentication methods can also be used for message authentication (such as HMAC, CBC-MAC, Kerberos authentication protocol, etc.).

[0219] The control module is responsible for task scheduling, data management, and logic control among various modules. In this embodiment, the control chip of the control module uses STM32. Of course, other chips can also be used, which will not be elaborated in this application.

[0220] It should be noted that in the implementation manner of the above-mentioned integrated system for annona squamosa cultivation and picking based on machine vision, the logical division of each program module is only for illustrative purposes. In actual applications, according to needs, for example, due to the configuration requirements of corresponding hardware or the convenience of software implementation, the above functions can be assigned to different program modules to complete, that is, the internal structure of the above-mentioned integrated system for annona squamosa cultivation and picking based on machine vision can be divided into different program modules to complete all or part of the functions described above.

[0221] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0222] The above embodiments are preferred implementation manners of the present invention, but the implementation manners of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement manners and are all included in the protection scope of the present invention.

Claims

1. A machine vision-based integrated method for cultivating and picking sugar apples, characterized in that: The steps include: The image data of the sugar apple in the planting area is obtained by using a drone, and a three-dimensional environmental model of the sugar apple is constructed by combining a multi-view three-dimensional reconstruction algorithm to extract the spatial data of the sugar apple; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; Based on the three-dimensional environmental model of sugar apples, sugar apples are clustered according to the spatial distribution of the planting area, and a robot equipped with a mechanical arm is dispatched to each cluster to carry out cultivation and picking operations; Operate the drone to count the number of sugar apples in each cluster, and plan the path for the robot in each cluster based on the number of sugar apples in each cluster; When each cluster of robots is working, a mechanical arm strength control scheme, a moving speed control scheme, and a fill light control scheme are generated for each cluster of robots based on fuzzy mathematics, and each cluster of robots cultivates and picks the sugar apples according to the mechanical arm strength control scheme, the moving speed control scheme, and the fill light control scheme. The mechanical arm strength control scheme, moving speed control scheme and fill light control scheme of each cluster of robots generated based on fuzzy mathematics are specifically: Calculate the operation feasibility and operation execution capability index of each cluster of robots, construct the fuzzy relationship matrix between the robot's mechanical arm operation force and the operation feasibility and operation execution capability index, and use the maximum membership method to obtain the mechanical arm force control plan of each cluster of robots; The equipment movement stability of each cluster of robots is calculated, and the fuzzy relationship matrix between the robot's movement speed, operation feasibility and equipment movement stability is constructed respectively. The maximum membership method is used to obtain the movement speed control scheme of each cluster of robots. The light efficiency index of the working environment of each cluster of robots was calculated, and the fuzzy relationship matrix between the robot's light sensitivity and the light efficiency index of the working environment was constructed. The amount of supplementary light required for picking sugar apples was calculated, and the maximum membership method was used to obtain the supplementary light control plan for each cluster of robots.

2. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The drone cruises and films under the sugar apple in a low-altitude flight mode, with the camera facing the sky during filming; the camera is equipped with a neutral density filter; The drone is also equipped with a photosensitive element for obtaining the light sensitivity of the sugar apple; The drone and the robot are both equipped with an instance segmentation model for identifying sugar apples; The instance segmentation model is trained on a manually annotated dataset.

3. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The three-dimensional environment model of the sugar apple is constructed and the spatial data of the sugar apple is extracted, specifically: A drone is deployed in the sugar apple planting area to regularly take high-definition images of the sugar apple and its surrounding environment from low altitudes and angles; the flight path of the drone passes through preset waypoints or is automatically planned; After acquiring the sugar apple image data, the two-dimensional images taken by the drone are stitched and aligned using image registration technology. Then, the depth information of each image pixel is calculated using image-based stereo vision technology to generate a high-precision depth map. A multi-view 3D reconstruction algorithm was applied to convert the depth map into a 3D point cloud model, and point cloud processing technology was used to perform noise removal, point cloud registration, and surface reconstruction to generate a 3D environment model of the sugar apple. Based on the three-dimensional environmental model of sugar apple, the spatial data of sugar apple is extracted using geometric analysis and volume calculation algorithm.

4. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: When clustering is performed according to the spatial distribution of the planting area, the spatial coordinates of the sugar apple are first obtained through the three-dimensional environmental model of the sugar apple, and the optimal number of clusters is determined using the elbow rule and the silhouette coefficient method; then, the sugar apples are spatially clustered using a clustering algorithm according to the optimal number of clusters and the spatial distribution of the planting area to obtain multiple clusters.

5. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: When performing the path planning, the drone is first controlled to count the number of sugar apples in each cluster and set a number threshold; Then the number of sugar-apples in each cluster and the quantity threshold are judged. If the number of sugar-apples in a cluster is less than or equal to the quantity threshold, the heuristic algorithm is used to plan the path of the cluster robot; if the number of sugar-apples in a cluster is greater than the quantity threshold, the heuristic search algorithm is used to plan the path of the cluster robot.

6. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The maximum membership method is used to obtain the mechanical arm strength control scheme of each cluster of robots, specifically: Define the operation feasibility and operation execution capability index, expressed as: ， ， in, TP is the feasibility of the operation, ranging from [0,10], the larger the value, the lower the feasibility; M is the maturity, ranging from [0,1]; F is the wind speed, ranging from [0,1]; H is humidity, ranging from [0,1]; Te is the temperature in degrees Celsius; Te opt This is the best temperature for picking sugar apples. w 1. w 2. w 3. w 4 are the weights of maturity, wind speed, humidity, and temperature respectively; a 1. b 2. c 3. d 4 are the adjustment coefficients of maturity, wind speed, humidity, and temperature respectively; OPI is the job execution capability index; E It is the efficiency of the operator; D is the status of the device or tool, with a value of 0 or 1; C is the complexity of the operating area; T ' is the operation time; A It is the adaptability and flexibility of the operation; w 5. w 6. w 7. w 8. w 9 are the weights of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation; α 2. α 3. α 4. α 5. α 6 are the nonlinear weighting coefficients of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation, respectively. β 1. β 2. β 3. β 5. β 6 are adjustment coefficients for work efficiency, status, complexity, operation time, and adaptability and flexibility of operations; Define the fuzzy matrix domain and membership function of the robot's operation feasibility, and construct the fuzzy matrix of the robot's operation feasibility P matrix : ， in, ELP Represents high feasibility, LP Represents higher feasibility, MP Represents medium feasibility, HP Represents lower feasibility, EHP stands for low feasibility; Define the fuzzy matrix domain and membership function of the robot's task execution capability index, and construct the fuzzy matrix of the robot's task execution capability index C o : ， in, ELC Represents extremely low execution capability. LC Represents lower execution capability. MC Represents medium executive ability. HC Represents higher execution capability. EHC Represents high executive ability; Define the fuzzy matrix domain and membership function of the robot's mechanical arm operating force, and construct the fuzzy matrix of the robot's mechanical arm operating force S matrix : ， in, ELS Represents low intensity, LS Represents lower strength, MS Represents medium strength. HS Represents a higher intensity. EHS Represents high intensity; Constructing the fuzzy relationship matrix between the robot's operational feasibility and the robot's operational strength M 1 and the fuzzy relationship matrix between the robot's operation execution capability index and the robot's operation strength M 2: ， ， in, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 1. M 2. Update the robot's arm operating force and its membership function: ， in, It is the fuzzy matrix of the robot's mechanical arm operating force after update.

7. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The maximum membership method is used to obtain the moving speed control scheme of each cluster of robots, specifically: Define device movement stability: ， in, TAI Provides stability for device movement; Sl is the slope, LS is land stability, w 10 , w 11 are the weights of slope and land stability, α 7. α 8 are the nonlinear coefficients controlling the slope and land stability, β 7. β 8 are the adjustment coefficients for slope and land stability respectively; Define the fuzzy matrix domain and its membership function of the robot's equipment movement stability, and construct the fuzzy matrix of equipment movement stability TAI matrix : ， in, ELT Represents high stability, LT Represents higher stability, MT Represents medium stability. HT Represents lower stability, EHT Represents low stability; Define the fuzzy matrix domain and membership function of the robot's moving speed, and construct the fuzzy matrix of the moving speed V matrix : ， in, ELV Represents stillness, LV Represents slow movement, MV Represents medium-speed movement. HV Represents higher speed movement, EHV Represents high-speed movement; Constructing the fuzzy relationship matrix between the robot's equipment movement stability and movement speed M 3 and the fuzzy relationship matrix between the robot's operation feasibility and movement speed M 4: ， ， in, P matrix is the fuzzy matrix of job feasibility, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 3. M 4. Update the fuzzy matrix and membership function of the robot's moving speed: ， in, is the fuzzy matrix of the robot's moving speed after update.

8. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The maximum membership method is used to obtain the fill light control scheme for each cluster of robots, specifically: Define the working environment light efficiency index: ELEI = L × e -αH ， in, L is the light sensitivity of the sugar apple when the robot is working. H is the air humidity, α It is the adjustment constant used to adjust the effect of humidity on the light efficiency index of the working environment; Define the fuzzy matrix domain and membership function of the robot's working environment light efficiency index, and construct the fuzzy matrix of the working environment light efficiency index ELEI matrix : ， in, ELE Represents low light efficiency index, LE Represents a lower light efficiency index, ME Represents the medium light efficiency index. HE Represents a higher light efficiency index, EHE Represents high light efficiency index; Define the fuzzy matrix domain and membership function of the robot's light sensitivity, and construct the fuzzy matrix of the robot's light sensitivity L matrix : ， in, ELL Represents low light sensitivity. LL Represents lower light sensitivity. ML Represents medium sensitivity. HL Represents a higher sensitivity. EHL Represents high sensitivity; Constructing the fuzzy relationship matrix between the light efficiency index and light sensitivity of the robot's working environment M 5: ， in, T It is the matrix transpose operation; Based on the inverse relationship between light sensitivity and fill light, the fuzzy relationship matrix between the light efficiency index of the working environment and the light sensitivity is constructed. M 5 Calculate and update the fuzzy matrix and membership function of the robot's fill light: ， in, It is the fuzzy matrix of the robot's fill light after update.

9. A machine vision-based integrated system for cultivating and harvesting sugar apples, characterized in that: The machine vision-based integrated method for cultivating and picking sugar apples as described in any one of claims 1 to 8, wherein the system comprises a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module and a main control module; The data acquisition module is used to obtain image data of sugar apples in the planting area through a drone, build a three-dimensional environmental model of sugar apples in combination with a multi-view three-dimensional reconstruction algorithm, and extract spatial data of sugar apples; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; The cloud computing module is used to cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environmental model of the sugar apples, and dispatch a robot equipped with a mechanical arm to each cluster for cultivation and picking operations; The visual module is used to control the drone to count the number of sugar apples in each cluster, and to plan the path of the robot in each cluster according to the number of sugar apples in each cluster; The individual calculation module is used to generate a mechanical arm force control scheme, a moving speed control scheme and a fill light control scheme for each cluster of robots based on fuzzy mathematics when each cluster of robots is working; The main control module is used for each cluster of robots to cultivate and pick sugar apples according to the mechanical arm strength control scheme, the moving speed control scheme and the fill light control scheme; The communication module is used for encrypted transmission of data between modules; The control module is responsible for task scheduling, data management and logic control among modules.

Images