Annona squamosa cultivating and picking integrated method based on machine vision

Abstract

The invention discloses a machine vision-based annona squamosa cultivating and picking integrated method, which comprises the following steps of: firstly, acquiring image data of annona squamosa through an unmanned aerial vehicle, constructing a three-dimensional environment model of the annona squamosa by combining a multi-view three-dimensional reconstruction algorithm, and extracting spatial data of the annona squamosa; then, the annona squamosa is clustered according to spatial distribution of the planting area, and a robot carrying a mechanical arm is dispatched for each cluster to conduct cultivation and picking operation; then the unmanned aerial vehicle is controlled to count the number of annona squamosa in each cluster, and path planning is conducted on the robot in each cluster; and finally, generating a mechanical arm force control scheme, a moving speed control scheme and a light supplementing quantity control scheme of each cluster of robots based on fuzzy mathematics, and cultivating and picking the annona squamosa by each cluster of robots according to the generated schemes. The machine vision and fuzzy control technology is combined, the annona squamosa cultivation and picking integrated service is achieved, the annona squamosa cultivation and picking efficiency and yield are improved, and the bad fruit rate and the damage rate are reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of digital image processing, and in particular relates to an integrated method for cultivating and picking sugar apples based on machine vision. Background Art

[0002] As a typical tropical and subtropical berry fruit, sugar apple is highly favored in the fruit market because its flesh is rich in vitamin C, antioxidant polyphenols and essential amino acids for the human body. However, picking sugar apples is not easy. Traditional manual picking requires operators to complete a series of operations such as visual positioning, manual lifting, and fruit stalk pinching on branches several meters high. There are problems such as low picking efficiency and high labor intensity. At the same time, due to the high frequency of touching and pressing the fruit during picking, the fruit is prone to cellular damage that is invisible to the naked eye. These hidden defects will cause abnormal release of ethylene within 24 hours after picking, accelerating fruit corruption and leading to a serious bad fruit rate. Therefore, intelligent and automated sugar apple picking technology has gradually become a research hotspot. By combining technologies such as drones, robotic arms, visual recognition and path planning, accurate picking can be achieved, picking efficiency can be improved, and damage to fruits and branches and leaves can be reduced, providing technical support for the development of smart agriculture.

[0003] Most of the existing sugar apple (custard apple) picking uses automated mechanical devices to improve picking efficiency; however, when the sugar apple enters the fully ripe stage, the original pectin in the cell wall is hydrolyzed in large quantities under the action of polygalacturonase, resulting in a decrease in the hardness of the pulp. At this time, the geometric stress concentration effect formed by the scale-like protrusions of the peel exacerbates the risk of damage during mechanical picking; and most mechanical devices only involve picking, and rarely involve the cultivation of sugar apples (such as pruning, pest control, etc.), so there is a lack of integrated picking methods, and the control of the operating force of the mechanical arm during the cultivation and picking process, the control of the moving speed of the equipment, and the control of the amount of supplementary light required for the sugar apple are all neglected, resulting in different degrees of damage to the fruit during the cultivation and picking process, thereby reducing the yield. Therefore, the present invention aims at the shortcomings of the existing patents and proposes an integrated method for cultivating and picking sugar apples based on machine vision. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide an integrated method for cultivating and picking sugar apples based on machine vision, which combines fuzzy control ideas with machine vision technology to achieve fuzzy control of the robot's mechanical arm's operating force, moving speed, and required fill light amount, thereby improving the efficiency of cultivating and picking sugar apples.

[0005] In order to achieve the above object, the present invention provides, on one hand, a method for integrating cultivation and harvesting of sugar apples based on machine vision, comprising the following steps: The image data of the sugar apple in the planting area is obtained by using a drone, and a three-dimensional environmental model of the sugar apple is constructed by combining a multi-view three-dimensional reconstruction algorithm to extract the spatial data of the sugar apple; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; Based on the three-dimensional environmental model of sugar apples, sugar apples are clustered according to the spatial distribution of the planting area, and a robot equipped with a mechanical arm is dispatched to each cluster to carry out cultivation and picking operations; Operate the drone to count the number of sugar apples in each cluster, and plan the path for the robot in each cluster based on the number of sugar apples in each cluster; When each cluster of robots is working, a robotic arm force control plan, a moving speed control plan and a fill light control plan are generated based on fuzzy mathematics. Each cluster of robots cultivates and picks sugar apples according to the robotic arm force control plan, the moving speed control plan and the fill light control plan.

[0006] As a preferred technical solution, the drone cruises and films below the sugar apple in a low-altitude flight mode, with the camera facing the sky during filming; the camera is equipped with a neutral density filter; The drone is also equipped with a photosensitive element for obtaining the light sensitivity of the sugar apple; The drone and the robot are both equipped with an instance segmentation model for identifying sugar apples; The instance segmentation model is trained on a manually annotated dataset.

[0007] As a preferred technical solution, the three-dimensional environmental model of the sugar apple is constructed and the spatial data of the sugar apple is extracted, specifically: A drone is deployed in the sugar apple planting area to regularly take high-definition images of the sugar apple and its surrounding environment from low altitudes and angles; the flight path of the drone passes through preset waypoints or is automatically planned; After acquiring the sugar apple image data, the two-dimensional images taken by the drone are stitched and aligned using image registration technology. Then, the depth information of each image pixel is calculated using image-based stereo vision technology to generate a high-precision depth map. A multi-view 3D reconstruction algorithm was applied to convert the depth map into a 3D point cloud model, and point cloud processing technology was used to perform noise removal, point cloud registration, and surface reconstruction to generate a 3D environment model of the sugar apple. Based on the three-dimensional environmental model of sugar apple, the spatial data of sugar apple is extracted using geometric analysis and volume calculation algorithm.

[0008] As a preferred technical solution, when clustering according to the spatial distribution of the planting area, the spatial coordinates of the sugar apple are first obtained through the three-dimensional environmental model of the sugar apple, and the optimal number of clusters is determined using the elbow rule and the silhouette coefficient method; then, the sugar apple is spatially clustered using a clustering algorithm according to the optimal number of clusters and the spatial distribution of the planting area to obtain multiple clusters.

[0009] As a preferred technical solution, when performing path planning, the drone is first controlled to count the number of sugar-apples in each cluster and set a quantity threshold; then the number of sugar-apples in each cluster and the quantity threshold are judged. If the number of sugar-apples in a cluster is less than or equal to the quantity threshold, a heuristic algorithm is used to perform path planning for the cluster robot; if the number of sugar-apples in a cluster is greater than the quantity threshold, a heuristic search algorithm is used to perform path planning for the cluster robot.

[0010] As a preferred technical solution, the mechanical arm strength control scheme, moving speed control scheme and fill light control scheme of each cluster of robots generated based on fuzzy mathematics are specifically as follows: Calculate the operation feasibility and operation execution capability index of each cluster of robots, construct the fuzzy relationship matrix between the robot's mechanical arm operation force and the operation feasibility and operation execution capability index, and use the maximum membership method to obtain the mechanical arm force control plan of each cluster of robots; The equipment movement stability of each cluster of robots is calculated, and the fuzzy relationship matrix between the robot's movement speed, operation feasibility and equipment movement stability is constructed respectively. The maximum membership method is used to obtain the movement speed control scheme of each cluster of robots. The light efficiency index of the working environment of each cluster of robots was calculated, and the fuzzy relationship matrix between the robot's light sensitivity and the light efficiency index of the working environment was constructed. The amount of supplementary light required for picking sugar apples was calculated, and the maximum membership method was used to obtain the supplementary light control plan for each cluster of robots.

[0011] As a preferred technical solution, the maximum membership method is used to obtain the mechanical arm strength control scheme of each cluster of robots, specifically: Define the operation feasibility and operation execution capability index, expressed as: , , in, TP is the feasibility of the operation, ranging from [0,10], the larger the value, the lower the feasibility; M is the maturity, ranging from [0,1]; F is the wind speed, ranging from [0,1]; H is humidity, ranging from [0,1]; Te is the temperature in degrees Celsius; Teopt This is the best temperature for picking sugar apples. w 1. w 2. w 3. w 4 are the weights of maturity, wind speed, humidity, and temperature respectively; a 1. b 2. c 3. d 4 are the adjustment coefficients of maturity, wind speed, humidity, and temperature respectively; OPI is the job execution capability index; E It is the efficiency of the operator; D is the status of the device or tool, with a value of 0 or 1; C is the complexity of the operating area; T ' is the operation time; A It is the adaptability and flexibility of the operation; w 5. w 6. w 7. w 8. w 9 are the weights of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation; α 2. α 3. α 4. α 5. α 6 are the nonlinear weighting coefficients of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation, respectively. β 1. β 2. β 3. β 5. β 6 are adjustment coefficients for work efficiency, status, complexity, operation time, and adaptability and flexibility of operations; Define the fuzzy matrix domain and membership function of the robot's operation feasibility, and construct the fuzzy matrix of the robot's operation feasibility P matrix : , in, ELP Represents high feasibility, LP Represents higher feasibility, MP Represents medium feasibility, HP Represents lower feasibility, EHP stands for low feasibility; Define the fuzzy matrix domain and membership function of the robot's task execution capability index, and construct the fuzzy matrix of the robot's task execution capability index C o : , in, ELC Represents extremely low execution capability. LC Represents lower execution capability. MC Represents medium executive ability. HC Represents higher execution capability. EHC Represents high executive ability; Define the fuzzy matrix domain and membership function of the robot's mechanical arm operating force, and construct the fuzzy matrix of the robot's mechanical arm operating force S matrix : , in, ELS Represents low intensity, LS Represents lower strength, MS Represents medium strength. HS Represents a higher intensity. EHS Represents high intensity; Constructing the fuzzy relationship matrix between the robot's operational feasibility and the robot's operational strength M 1 and the fuzzy relationship matrix between the robot's operation execution capability index and the robot's operation strength M 2: , , in, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 1. M 2. Update the robot's arm operation force and its membership function: , in, It is the fuzzy matrix of the robot's mechanical arm operating force after update.

[0012] As a preferred technical solution, the maximum membership method is used to obtain the moving speed control scheme of each cluster of robots, specifically: Define device movement stability: , in, TAI Provides stability for device movement; Sl is the slope, LS is land stability, w 10 , w 11 are the weights of slope and land stability, α 7. α8 are the nonlinear coefficients controlling the slope and land stability, β 7. β 8 are the adjustment coefficients for slope and land stability respectively; Define the fuzzy matrix domain and its membership function of the robot's equipment movement stability, and construct the fuzzy matrix of equipment movement stability TAI matrix : , in, ELT Represents high stability, LT Represents higher stability, MT Represents medium stability. HT Represents lower stability, EHT Represents low stability; Define the fuzzy matrix domain and membership function of the robot's moving speed, and construct the fuzzy matrix of the moving speed V matrix : , in, ELV Represents stillness, LV Represents slow movement, MV Represents medium-speed movement. HV Represents higher speed movement, EHV Represents high-speed movement; Constructing the fuzzy relationship matrix between the robot's equipment movement stability and movement speed M 3 and the fuzzy relationship matrix between the robot's operation feasibility and movement speed M 4: , , in, P matrix is the fuzzy matrix of job feasibility, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 3. M 4. Update the fuzzy matrix and membership function of the robot's moving speed: , in, is the fuzzy matrix of the robot's moving speed after update.

[0013] As a preferred technical solution, the maximum membership method is used to obtain the fill light control scheme for each cluster of robots, specifically: Define the working environment light efficiency index: ELEI =L × e -αH , in, L is the light sensitivity of the sugar apple when the robot is working. H is the air humidity, α It is the adjustment constant used to adjust the effect of humidity on the light efficiency index of the working environment; Define the fuzzy matrix domain and membership function of the robot's working environment light efficiency index, and construct the fuzzy matrix of the working environment light efficiency index ELEI matrix : , in, ELE Represents low light efficiency index, LE Represents a lower light efficiency index, ME Represents the medium light efficiency index. HE Represents a higher light efficiency index, EHE Represents high light efficiency index; Define the fuzzy matrix domain and membership function of the robot's light sensitivity, and construct the fuzzy matrix of the robot's light sensitivity L matrix : , in, ELL Represents low light sensitivity. LL Represents lower light sensitivity. ML Represents medium sensitivity. HL Represents a higher sensitivity. EHL Represents high sensitivity; Constructing the fuzzy relationship matrix between the light efficiency index and light sensitivity of the robot's working environment M 5: , in, T It is the matrix transpose operation; Based on the inverse relationship between light sensitivity and fill light, the fuzzy relationship matrix between the light efficiency index of the working environment and the light sensitivity is constructed. M 5 Calculate and update the fuzzy matrix and membership function of the robot's fill light: , in, It is the fuzzy matrix of the robot's fill light after update.

[0014] On the other hand, the present invention also provides an integrated system for cultivating and picking sugar apples based on machine vision, which is applied to the integrated method for cultivating and picking sugar apples based on machine vision. The system includes a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module and a main control module; The data acquisition module is used to obtain image data of sugar apples in the planting area through a drone, build a three-dimensional environmental model of sugar apples in combination with a multi-view three-dimensional reconstruction algorithm, and extract spatial data of sugar apples; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; The cloud computing module is used to cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environmental model of the sugar apples, and dispatch a robot equipped with a mechanical arm to each cluster for cultivation and picking operations; The visual module is used to control the drone to count the number of sugar apples in each cluster, and to plan the path of the robot in each cluster according to the number of sugar apples in each cluster; The individual calculation module is used to generate a mechanical arm force control scheme, a moving speed control scheme and a fill light control scheme for each cluster of robots based on fuzzy mathematics when each cluster of robots is working; The main control module is used for each cluster of robots to cultivate and pick sugar apples according to the mechanical arm strength control scheme, the moving speed control scheme and the fill light control scheme; The communication module is used for encrypted transmission of data between modules; The control module is responsible for task scheduling, data management and logic control among modules.

[0015] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. The present invention proposes an integrated method for cultivating and picking sugar apples based on machine vision. Based on the fuzzy control concept and machine vision technology, an unmanned aerial vehicle combined with multi-view visual reconstruction technology is used to construct a three-dimensional environmental model of sugar apples, and a picking device is path planned according to three-dimensional coordinates. At the same time, fuzzy control is performed on the operating force, moving speed and required fill light amount of the mechanical arm of the sugar apple picking device, thereby improving the efficiency of sugar apple cultivation and picking, reducing the bad fruit rate and damage rate, and increasing the yield of sugar apples.

[0016] 2. The present invention uses a drone to fly under the sugar apple, with the camera facing the sky to shoot, to avoid the influence of sunlight exposure, and to reduce light interference by installing a neutral density filter; at the same time, the drone is also equipped with a photosensitive component, which is used to obtain the light sensitivity when the drone is positioned directly under the sugar apple, in preparation for the fuzzy control of the required fill light amount. In addition, the drone and the robot are also equipped with an instance segmentation model, which can not only identify the target sugar apple, but also accurately segment the pixel-level area of ​​each target, further provide the fine contour information of the sugar apple, improve the accuracy of sugar apple identification, and ensure the accuracy of cultivation and picking.

[0017] 3. When the present invention performs clustering, the elbow rule and the silhouette coefficient method are first used to determine the optimal number of clustering clusters, which can effectively determine the optimal number of clustering clusters and avoid redundant or insufficient operations caused by too many or too few clusters; then the clustering algorithm is used to spatially cluster the sugar apples, which helps to improve the efficiency and accuracy of sugar apple cultivation and picking operations; then a robot equipped with a mechanical arm is dispatched to each cluster to perform cultivation and picking operations, so that each cluster of sugar apples can be cultivated and picked separately by a dedicated robot, which optimizes the task allocation of robot operations, makes the working range of each robot more concentrated, reduces duplication and conflict in operations, and further improves the automation and accuracy of cultivation and picking operations, thereby significantly improving the overall operation efficiency.

[0018] 4. In the method of the present invention, when the robot equipped with a mechanical arm performs cultivation and picking operations, based on the idea of ​​fuzzy mathematics, factors such as operation feasibility, operation execution capability index, equipment movement stability and operating environment light efficiency index are considered to systematically control the robot so that the robot can adjust and adapt to external changes, thereby facilitating the optimal decision-making when cultivating and picking sugar apples, and improving the efficiency of cultivating and picking sugar apples. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 The present invention is an overall flow chart of the integrated method for cultivating and picking sugar apples based on machine vision in an embodiment of the present invention.

[0021] Figure 2 The present invention is a flowchart of constructing a three-dimensional environment model of sugar apple and extracting spatial data of sugar apple in an embodiment of the present invention.

[0022] Figure 3This is a flow chart of generating a robot arm force control solution in an embodiment of the present invention.

[0023] Figure 4 The figure is a flow chart of generating a moving speed control solution in an embodiment of the present invention.

[0024] Figure 5 The figure is a flow chart of generating a fill light amount control solution in an embodiment of the present invention.

[0025] Figure 6 It is a structural schematic diagram of an integrated system for cultivating and picking sugar apples based on machine vision in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0027] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0028] Example 1 like Figure 1 As shown, this embodiment provides an integrated method for cultivating and picking sugar apples based on machine vision, the steps are: S1. Obtain image data of sugar apples in the planting area through drones, build a three-dimensional environmental model of sugar apples and extract spatial data of sugar apples by combining a multi-view three-dimensional reconstruction algorithm; wherein the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity, etc.

[0029] Specifically, the drone cruised and filmed below the sugar apple at a low altitude, with the camera facing the sky to avoid exposure to sunlight. To reduce light interference, the camera is equipped with a neutral density filter (ND filter) (such as ND4, ND8, ND16, etc.) to reduce light and avoid overexposure to sunlight. At the same time, the drone is also equipped with photosensitive elements (such as photodiodes, CCD\CMOS sensors, etc.) to obtain the light sensitivity of the sugar apple, and further provide accurate data for evaluating the growth status of the sugar apple.

[0030] In addition, both the drone and the robot are equipped with an instance segmentation model for identifying sugar apples to improve accuracy; the instance segmentation model is trained using an artificially annotated data set to improve its accuracy. In the present embodiment, the artificially annotated data set is made based on the sugar apple image data obtained by the drone, but the present application is not limited thereto, and data sets made in other ways can also be used to train the instance segmentation model, as long as the sugar apple can be identified. In the present embodiment, the instance segmentation model uses the YOLOv8-seg model, and other deep learning models that can achieve the same function can also be regarded as the instance segmentation model in the present application, and also belong to the protection scope of the present application.

[0031] Further, such as Figure 2 As shown, the steps of constructing a three-dimensional environmental model of sugar apple and extracting spatial data of sugar apple in this application are: First, a drone equipped with a high-precision camera and sensor is deployed in the sugar-apple planting area. It regularly flies low at different heights and angles to capture high-definition images of the sugar-apple and its surrounding environment. The flight path of the drone is achieved through preset waypoints or automatic path planning to achieve full coverage of the planting area, ensuring multi-angle and all-round acquisition of sugar-apple image data.

[0032] After acquiring the sugar apple image data, the two-dimensional images taken by the drone are stitched and aligned using image registration technology to eliminate the impact of camera position changes and perspective differences; then, the depth information of each image pixel is calculated using image-based stereo vision technology to generate a high-precision depth map.

[0033] Next, a multi-view 3D reconstruction algorithm is applied to convert the depth map into a 3D point cloud model, and point cloud processing technology is used to perform noise removal, point cloud registration and surface reconstruction to generate a 3D environmental model of the sugar apple.

[0034] Finally, based on the three-dimensional environmental model of the sugar apple, geometric analysis and volume calculation algorithms were used to extract the spatial data of the sugar apple, such as spatial coordinates, shape, size, volume, temperature, humidity, wind speed, and photosensitivity.

[0035] S2. Based on the three-dimensional environmental model of sugar apples, sugar apples are clustered according to the spatial distribution of the planting area, and a robot equipped with a robotic arm is dispatched to each cluster to carry out cultivation and picking operations.

[0036] Furthermore, when clustering according to the spatial distribution of the planting area, the spatial coordinates of the sugar apple are first obtained through the three-dimensional environmental model of the sugar apple, and the elbow rule and silhouette coefficient method are used to determine the optimal number of clusters to avoid redundant or insufficient operations caused by too many or too few clusters; then the sugar apples are spatially clustered according to the optimal number of clusters and the spatial distribution of the planting area using a clustering algorithm to obtain multiple clusters, which helps to improve the efficiency and accuracy of sugar apple cultivation and picking operations. For each cluster, a robot equipped with a robotic arm is dispatched to perform cultivation and picking operations, so that each cluster has a dedicated robot for cultivation and picking operations (such as pruning branches and leaves, pest control, pesticide application, fruit picking, etc.), ensuring that the robot's working range is more concentrated, reducing duplication and conflict in operations, and improving work efficiency.

[0037] In this embodiment, the clustering algorithm adopts the K-means clustering algorithm. Of course, other clustering algorithms may also be used, such as DBSCAN, hierarchical clustering, etc. As long as the spatial clustering of the sugar apple can be achieved, it is considered to fall within the protection scope of this application.

[0038] S3. Operate the drone to count the number of sugar-apples in each cluster, and plan the path of the robot in each cluster according to the number of sugar-apples in each cluster.

[0039] Specifically, when performing path planning, first control the drone to count the number of sugar-apples in each cluster and set a quantity threshold; then judge the number of sugar-apples in each cluster and the quantity threshold. If the number of sugar-apples in a cluster is less than or equal to the quantity threshold, use a heuristic algorithm (such as a greedy algorithm) to plan the path of the cluster robot; if the number of sugar-apples in a cluster is greater than the quantity threshold, use a heuristic search algorithm (such as an A* algorithm) to plan the path of the cluster robot, so as to ensure that the cultivation and picking process is efficient, the travel path is the shortest, and obstacle interference is avoided.

[0040] In this embodiment, the quantity threshold is set to 300. For robots with more than or equal to 300 sugar apples in each cluster, the greedy algorithm is used for path planning. Otherwise, the A* algorithm is used for path planning of the cluster robot. Of course, other algorithms can also be selected for path planning according to actual needs. For example, the heuristic algorithm can be a simulated annealing algorithm, a genetic algorithm, a taboo search algorithm, etc. The heuristic search algorithm can use a best-first search algorithm, etc.; in addition, the Dijkstra algorithm (Dijkstra algorithm), the rapid exploration random tree algorithm (RTT), the dynamic window method (DWA) and other path planning algorithms can also be used for path planning to achieve the same purpose as the present application.

[0041] S4. When each cluster of robots is working, a mechanical arm strength control plan, a moving speed control plan and a fill light control plan are generated based on fuzzy mathematics. Each cluster of robots cultivates and picks the sugar apples according to the mechanical arm strength control plan, the moving speed control plan and the fill light control plan.

[0042] Specifically, in the prior art, the control of the robot arm's operating force, movement speed and fill light amount during the picking process has been neglected. First, inaccurate control of the robot arm's operating force may cause the sugar apple to be damaged by excessive force during the picking process, affecting the quality of the fruit and increasing the rate of bad fruit; secondly, improper control of the movement speed may cause the robot arm to operate unsmoothly and be unable to adapt to different environments and fruit conditions, thereby reducing operating efficiency and accuracy; finally, insufficient or unbalanced fill light amount control will affect the lighting conditions of the sugar apple during the cultivation process, and then affect the growth and maturity of the fruit, resulting in unsatisfactory picking results and affecting yields; therefore, based on these three considerations, this application generates a robot arm force control scheme, a movement speed control scheme and a fill light amount control scheme for each cluster of robots based on fuzzy mathematics, and the steps are: S4.1. First, calculate the operation feasibility and operation execution capability index of each cluster of robots, respectively construct the fuzzy relationship matrix between the robot's mechanical arm operation force and the operation feasibility and operation execution capability index, and use the maximum membership method to obtain the mechanical arm force control plan of each cluster of robots.

[0043] Specifically, Figure 3 As shown in the figure, the steps for generating the robot arm force control solution are: S4.1.1 First, define the task feasibility and task execution capability index; among them, task feasibility (Taskfeasibility, TP ) is used to evaluate the impact of external environment (such as wind speed, humidity, temperature, etc.) and the maturity of sugar apple on the cultivation and picking operations. The larger the value, the more difficult the operation is. Operational Performance Index (OPI) OPI ) measures the impact of factors such as operators, equipment, and complexity of the operating area on the operating effect in actual operations, which are expressed as: , , in, TP is the feasibility of the operation, ranging from [0,10], the larger the value, the lower the feasibility; M is the maturity, ranging from [0,1] (immature to fully mature); F is the wind speed, ranging from [0,1] (calm to strong wind); His the humidity, ranging from [0,1] (low humidity to high humidity); Te is the temperature in degrees Celsius; Te opt is the best picking temperature for sugar apple (e.g. 25℃). w 1. w 2. w 3. w 4 are the weights of maturity, wind speed, humidity, and temperature respectively; a 1. b 2. c 3. d 4 are the adjustment coefficients of maturity, wind speed, humidity, and temperature, which are used to adjust the sensitivity of each factor to risk; OPI is the job execution capability index; E is the operator's work efficiency, D is the state of the robot, which can be 0 (not feasible) or 1 (feasible); C is the complexity of the operating area; T ' is the operation time; A The adaptability and flexibility of the operation (e.g., whether it can flexibly respond to unexpected situations); w 5. w 6. w 7. w 8. w 9 are the weights of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation; α 2. α 3. α 4. α 5. α 6 are the nonlinear weighting coefficients of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation, respectively. β 1. β 2. β 3. β 5. β 6 are the adjustment coefficients for work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation.

[0044] In this embodiment, the maturity MObtained using spectral detection technology; in the process of spectral detection of the maturity of sugar apple, a threshold wavelength can be defined to distinguish different spectral characteristics to identify maturity. Detecting the pigment content on the surface of sugar apple through the visible light band (400-700nm) can reflect the color change of the peel, thereby preliminarily judging its external maturity. At the same time, the moisture, sugar and other chemical components inside the sugar apple are detected through the near-infrared band (700-2500nm) to evaluate its internal maturity. Combining the data from the visible light and near-infrared bands and using the threshold wavelength for comprehensive evaluation, the overall maturity of the sugar apple can be accurately judged. Complexity of the operating area C Refers to the spatial complexity of each cluster, measured by the average slope of the region. Adaptability and flexibility of operations A The leaf and branch coverage of the sugar apple area was quantified when the robotic arm was used.

[0045] S4.1.2. Define the fuzzy matrix domain and membership function of the robot's operation feasibility, and construct the fuzzy matrix of the robot's operation feasibility P matrix : , in, ELP Represents high feasibility, LP Represents higher feasibility, MP Represents medium feasibility, HP Represents lower feasibility, EHP Represents low feasibility.

[0046] S4.1.3. Define the fuzzy matrix domain and membership function of the robot's task execution capability index, and construct the fuzzy matrix of the robot's task execution capability index. C o : , in, ELC Represents extremely low execution capability. LC Represents lower execution capability. MC Represents medium executive ability. HC Represents higher execution capability. EHC Represents high execution ability.

[0047] S4.1.4. Define the fuzzy matrix domain and membership function of the robot's mechanical arm operating force, and construct the fuzzy matrix of the robot's mechanical arm operating force S matrix : , in, ELS Represents very low intensity. LS Represents lower strength, MS Represents medium strength. HS Represents a higher intensity. EHS Represents high intensity.

[0048] S4.1.5. Construct the fuzzy relationship matrix between the robot's operational feasibility and the robot's operational strength M 1 and the fuzzy relationship matrix between the robot's operation execution capability index and the robot's operation strength M 2: , , in, T It is the matrix transpose operation; S4.1.6. Coupled fuzzy relationship matrix M 1. M 2. Update the robot's arm operation force and its membership function: , in, It is the fuzzy matrix of the robot's mechanical arm operating force after update.

[0049] In this embodiment, the domain of the robot's operation feasibility fuzzy matrix is ​​set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, and its corresponding membership function is: , The fuzzy matrix domain of the robot's task execution capability index is set to {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}, and its corresponding membership function is: , The fuzzy matrix domain of the robot's mechanical arm operating force is set to {1,2,3,4,5,6,7,8,9,10,11}, and its corresponding membership function is: , in, S is the current robot arm operating force value, then the updated robot arm operating force fuzzy matrix calculated by the above steps is The membership function is expressed as: , Control the robot's arm in the cluster based on the updated robot's arm operating force: when S'When (operation force value) <3, it means that the robot arm needs to use low force during the cultivation and picking process, and the cultivation and picking actions should be performed gently to avoid damage to the fruit and ensure that the fruit quality during the cultivation and picking process is not affected; When 3 ≤ S' When it is <5, it means that the mechanical arm needs a lower force during the cultivation and picking process. The force can be increased appropriately during operation, but care should still be taken to protect the surface of the fruit to avoid excessive force that may damage the peel; When 5 ≤ S' <7, it means that the robot arm needs medium force during cultivation and picking. At this time, stronger force can be used for cultivation and picking, but the stability of the robot arm must still be ensured to avoid damaging the internal structure of the fruit; When 7 ≤ S' <9: It means that the robot arm needs a higher force during the cultivation and picking process. A stronger force can be used during cultivation and picking, but it needs to be operated with extra care to ensure that the robot arm does not cause external damage to the fruit; When 9 ≤ S' When ≤ 11, it means that the robot arm needs high force during the cultivation and picking process. At this time, it should be ensured that the robot arm can fully withstand the large force to ensure that the cultivation and picking process is rapid without causing too much pressure on the fruit.

[0050] S4.2. Calculate the equipment movement stability of each cluster of robots, construct the fuzzy relationship matrix between the robot's movement speed and the operation feasibility and equipment movement stability, and use the maximum membership method to obtain the movement speed control plan of each cluster of robots.

[0051] Specifically, Figure 4 As shown, the steps for generating the mobile speed control scheme are: S4.2.1. Define device mobility stability, which is used to measure the mobility stability of the robot under different environmental conditions, especially the response to slope and land stability, expressed as: , in, TAI Provides stability for device movement; Sl is the slope, LS is land stability, w 10 , w 11 are the weights of slope and land stability, respectively, indicating the degree of influence on the stability of equipment movement; α 7. α 8 are the nonlinear coefficients controlling the slope and land stability, β 7. β 8 are the adjustment coefficients for slope and land stability respectively. SlThe larger the value, the greater the slope and the lower the stability of the equipment movement; LS The larger the value, the lower the stability of the land and the lower the stability of the equipment movement. Sl ) and land stability ( LS ) is an important factor affecting the stability of the equipment. TAI The calculation takes into account the impact of slope and land stability on the stability of equipment movement. The greater the slope, the worse the stability of the equipment and the more difficult it is to move; the lower the land stability, the worse the stability of the equipment on the ground and the more likely it is to become unstable.

[0052] S4.2.2. Define the fuzzy matrix domain and its membership function of the robot's equipment movement stability, and construct the fuzzy matrix of equipment movement stability TAI matrix : , in, ELT Represents high stability, LT Represents higher stability, MT Represents medium stability. HT Represents lower stability, EHT Represents low stability.

[0053] S4.2.3. Define the fuzzy matrix domain and membership function of the robot's moving speed and construct the fuzzy matrix of the moving speed V matrix : , in, ELV Represents lower speed movement, LV Represents low speed movement, MV Represents medium speed movement. HV Represents higher speed movement, EHV Represents high-speed movement.

[0054] S4.2.4. Construct the fuzzy relationship matrix between the robot's equipment movement stability and movement speed M 3 and the fuzzy relationship matrix between the robot's operation feasibility and movement speed M 4: , , in, P matrix is the fuzzy matrix of job feasibility, T Performs a matrix transpose operation.

[0055] S4.2.5. Coupled fuzzy relationship matrix M 3. M4. Update the fuzzy matrix and membership function of the robot's moving speed: , in, is the fuzzy matrix of the robot's moving speed after update.

[0056] In this embodiment, the fuzzy matrix domain of the robot's device movement stability is set to {0, 1, 2, 3, 4, 5}, and its corresponding membership function is: , The fuzzy matrix domain of the robot's moving speed is set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10, 11, 12, 13, 14, 15}, and its corresponding membership function is: , in, V is the current moving speed of the robot. After the above steps, the fuzzy matrix of the moving speed of the robot is updated The membership function is expressed as: , Based on the updated movement speed of the robot, the movement of the robots in the cluster is controlled. V' ), the robot moves as follows: when V' =0, it indicates that the robot is in a stationary state, which is suitable for fine cultivation and picking operations or waiting for instructions, for example: detailed inspection of the branches, leaves, and fruits of the sugar apple (such as growth status monitoring, etc.), waiting for picking instructions, etc.

[0057] When 3 ≤ V' When <6, it indicates that the robot is moving at a low speed, which is suitable for more delicate operations, such as detecting fruit growth and properly pruning branches and leaves.

[0058] When 6 ≤ V' When the value is <9, it indicates that the robot is moving at a medium speed, which is suitable for moving at a normal speed, ensuring both work efficiency and stability; for example, conducting daily inspections along the planned path, identifying early-stage diseases, and implementing intelligent water and fertilizer management (such as automatic irrigation and precision fertilization).

[0059] When 9 ≤ V' <12:00, it means the robot is in a high-speed moving state, which is suitable for large-scale cruising and task execution; for example: large-scale orchard cruising within the cluster, detecting the maturity of sugar apples to generate a picking task list, pre-scanning and path planning before picking, etc.

[0060] When 12 ≤V' When ≤ 15, it indicates that the robot is in a high-speed moving state, which is suitable for quickly covering a large area for picking or other operations; for example: quickly reaching the target area, performing automated picking tasks, and collecting and sorting fruits.

[0061] S4.3. Calculate the light efficiency index of the working environment of each cluster of robots, construct the fuzzy relationship matrix between the robot's light sensitivity and the light efficiency index of the working environment, calculate the amount of supplementary light required for picking sugar apples, and use the maximum membership method to obtain the supplementary light control plan for each cluster of robots.

[0062] Specifically, Figure 5 As shown, the steps for generating the fill light control solution are: S4.3.1. Define the light efficiency index of the working environment, which is used to measure the effect of the light and humidity required for the robot to cultivate and harvest sugar apples on the working efficiency, expressed as: ELEI = L × e -αH , in, L It is the light sensitivity of the sugar apple corresponding to the robot operation, reflecting the light intensity; H is the air humidity, the unit can be percentage (0-100%); α It is a constant used to adjust the effect of humidity on the light efficiency index of the working environment. As humidity increases, the lighting effect will weaken, which will in turn affect the efficiency of cultivation and picking operations. By calculating ELEI, we can understand the current ambient light conditions, help optimize the robot's operating strategy, and improve the efficiency of cultivation and picking.

[0063] S4.3.2. Define the fuzzy matrix domain and membership function of the robot's working environment light efficiency index, and construct the fuzzy matrix of the working environment light efficiency index ELEI matrix : , in, ELE Represents low light efficiency index, LE Represents a lower light efficiency index, ME Represents the medium light efficiency index. HE Represents a higher light efficiency index, EHE Stands for high light efficiency index.

[0064] S4.3.3. Define the fuzzy matrix domain and membership function of the robot's light sensitivity and construct the fuzzy matrix of the robot's light sensitivity L matrix : , in, ELLRepresents low light sensitivity. LL Represents lower light sensitivity. ML Represents medium sensitivity. HL Represents a higher sensitivity. EHL Stands for high sensitivity.

[0065] S4.3.4. Construct the fuzzy relationship matrix between the light efficiency index and light sensitivity of the robot's working environment M 5: , in, T Performs a matrix transpose operation.

[0066] S4.3.5, based on the inverse relationship between light sensitivity and fill light, the fuzzy relationship matrix between the light efficiency index of the working environment and light sensitivity M 5 Calculate and update the fuzzy matrix and membership function of the robot's fill light: , in, It is the fuzzy matrix of the robot's fill light after update.

[0067] In this embodiment, the fuzzy matrix domain of the robot's working environment light efficiency index is set to {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1}, and its corresponding membership function is: , The fuzzy matrix domain of the robot's light sensitivity is set to {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}, and its corresponding membership function is: , After the above steps, the robot's fill light fuzzy matrix is ​​updated. The membership function of is: , According to the updated amount of supplementary light, the robot prunes the branches and leaves of the sugar apple with the following strategy: L ’ When (light supplement value) <2, it means that the sugar apple needs high light supplement and should be pruned more significantly to remove the dense branches and leaves that block the fruit and increase the light penetration rate; when 2 ≤ L ’ <4, it means that the sugar apple needs a higher amount of supplementary light, and some branches and leaves can be properly pruned to reduce shading but maintain moderate photosynthesis; when 4≤ L ’ <6, it means that the sugar apple needs a moderate amount of supplementary light, and only light pruning of overlapping branches and leaves is performed to optimize the internal light transmittance; when 6 ≤L ’ <8, it means that the sugar apple needs a lower amount of supplementary light, and only needs to fine-tune the branches and leaves to avoid excessive pruning that affects fruit growth; when 8 ≤ L ’ When ≤ 10, it means that the sugar apple needs low supplementary light. At this time, the light is sufficient, pruning can be reduced, and the crown structure can be maintained to promote balanced ripening of the fruit.

[0068] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0069] Example 2 like Figure 6 As shown, another embodiment of the present invention provides an integrated system for cultivating and picking sugar apples based on machine vision, including a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module and a main control module; The data acquisition module is used to obtain the image data of sugar apples in the planting area through drones, and build a three-dimensional environmental model of sugar apples and extract the spatial data of sugar apples by combining a multi-view three-dimensional reconstruction algorithm; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed, and light sensitivity, etc. The cloud computing module is used to cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environmental model of the sugar apples, and dispatch a robot equipped with a mechanical arm to each cluster to perform operations; The vision module is used to control the drone to count the number of sugar apples in each cluster, and to plan the path of the robot in each cluster based on the number of sugar apples in each cluster; The individual calculation module is used to generate the mechanical arm force control scheme, moving speed control scheme and fill light control scheme of each cluster of robots based on fuzzy mathematics when each cluster of robots is working; The main control module is used for each cluster of robots to prune the branches and leaves or pick the fruits of the sugar apple according to the mechanical arm force control scheme, the moving speed control scheme and the fill light control scheme; The communication module is used for encrypted transmission of data between modules to ensure that the data is not stolen or tampered with during the transmission process. In this embodiment, the communication module uses the RSA algorithm for encryption and decryption, that is, the public key of the recipient is used to encrypt the message, and only the private key of the recipient can decrypt it, thereby ensuring the confidentiality of the data. In addition, the message authentication in the communication module uses the SHA-256 hash algorithm to generate a summary of the message, and combines appropriate signatures or authentication mechanisms to ensure the integrity of the message and the authenticity of the source. Of course, in addition to the RSA algorithm mentioned, the encryption and decryption algorithm can also use other encryption methods (such as AES, DES, SHA-3, ECC, etc.), which are not listed one by one in this application; similarly, other authentication methods can also be used for message authentication (such as HMAC, CBC-MAC, Kerberos authentication protocol, etc.).

[0070] The control module is responsible for task scheduling, data management and logic control between modules. In this embodiment, the control chip of the control module adopts STM32, and other chips can also be used, which will not be described in detail in this application.

[0071] It should be noted that, in the implementation of the integrated system for cultivating and picking sugar apples based on machine vision in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the integrated system for cultivating and picking sugar apples based on machine vision is divided into different program modules to complete all or part of the functions described above.

[0072] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A machine vision-based integrated method for cultivating and picking sugar apples, characterized in that: The steps include: The image data of the sugar apple in the planting area is obtained by using a drone, and a three-dimensional environmental model of the sugar apple is constructed by combining a multi-view three-dimensional reconstruction algorithm to extract the spatial data of the sugar apple; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; Based on the three-dimensional environmental model of sugar apples, sugar apples are clustered according to the spatial distribution of the planting area, and a robot equipped with a mechanical arm is dispatched to each cluster to carry out cultivation and picking operations; Operate the drone to count the number of sugar apples in each cluster, and plan the path for the robot in each cluster based on the number of sugar apples in each cluster; When each cluster of robots is working, a robotic arm force control plan, a moving speed control plan and a fill light control plan are generated based on fuzzy mathematics. Each cluster of robots cultivates and picks sugar apples according to the robotic arm force control plan, the moving speed control plan and the fill light control plan.

2. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The drone cruises and films under the sugar apple in a low-altitude flight mode, with the camera facing the sky during filming; the camera is equipped with a neutral density filter; The drone is also equipped with a photosensitive element for obtaining the light sensitivity of the sugar apple; The drone and the robot are both equipped with an instance segmentation model for identifying sugar apples; The instance segmentation model is trained on a manually annotated dataset.

3. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The three-dimensional environment model of the sugar apple is constructed and the spatial data of the sugar apple is extracted, specifically: A drone is deployed in the sugar apple planting area to regularly take high-definition images of the sugar apple and its surrounding environment from low altitudes and angles; the flight path of the drone passes through preset waypoints or is automatically planned; After acquiring the sugar apple image data, the two-dimensional images taken by the drone are stitched and aligned using image registration technology. Then, the depth information of each image pixel is calculated using image-based stereo vision technology to generate a high-precision depth map. A multi-view 3D reconstruction algorithm was applied to convert the depth map into a 3D point cloud model, and point cloud processing technology was used to perform noise removal, point cloud registration, and surface reconstruction to generate a 3D environment model of the sugar apple. Based on the three-dimensional environmental model of sugar apple, the spatial data of sugar apple is extracted using geometric analysis and volume calculation algorithm.

4. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: When clustering is performed according to the spatial distribution of the planting area, the spatial coordinates of the sugar apple are first obtained through the three-dimensional environmental model of the sugar apple, and the optimal number of clusters is determined using the elbow rule and the silhouette coefficient method; then, the sugar apples are spatially clustered using a clustering algorithm according to the optimal number of clusters and the spatial distribution of the planting area to obtain multiple clusters.

5. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: When performing the path planning, the drone is first controlled to count the number of sugar apples in each cluster and set a number threshold; Then the number of sugar-apples in each cluster and the quantity threshold are judged. If the number of sugar-apples in a cluster is less than or equal to the quantity threshold, the heuristic algorithm is used to plan the path of the cluster robot; if the number of sugar-apples in a cluster is greater than the quantity threshold, the heuristic search algorithm is used to plan the path of the cluster robot.

6. The integrated method for cultivating and picking sugar apples according to claim 1, characterized in that: The mechanical arm strength control scheme, moving speed control scheme and fill light control scheme of each cluster of robots generated based on fuzzy mathematics are specifically: Calculate the operation feasibility and operation execution capability index of each cluster of robots, construct the fuzzy relationship matrix between the robot's mechanical arm operation force and the operation feasibility and operation execution capability index, and use the maximum membership method to obtain the mechanical arm force control plan of each cluster of robots; The equipment movement stability of each cluster of robots is calculated, and the fuzzy relationship matrix between the robot's movement speed, operation feasibility and equipment movement stability is constructed respectively. The maximum membership method is used to obtain the movement speed control scheme of each cluster of robots. The light efficiency index of the working environment of each cluster of robots was calculated, and the fuzzy relationship matrix between the robot's light sensitivity and the light efficiency index of the working environment was constructed. The amount of supplementary light required for picking sugar apples was calculated, and the maximum membership method was used to obtain the supplementary light control plan for each cluster of robots.

7. The integrated method for cultivating and picking sugar apples according to claim 6, characterized in that: The maximum membership method is used to obtain the mechanical arm strength control scheme of each cluster of robots, specifically: Define the operation feasibility and operation execution capability index, expressed as: ， ， in, TP is the feasibility of the operation, ranging from [0,10], the larger the value, the lower the feasibility; M is the maturity, ranging from [0,1]; F is the wind speed, ranging from [0,1]; H is humidity, ranging from [0,1]; Te is the temperature in degrees Celsius; Te opt This is the best temperature for picking sugar apples. w 1. w 2. w 3. w 4 are the weights of maturity, wind speed, humidity, and temperature respectively; a 1. b 2. c 3. d 4 are the adjustment coefficients of maturity, wind speed, humidity, and temperature respectively; OPI is the job execution capability index; E It is the work efficiency of the operator; D is the status of the device or tool, with a value of 0 or 1; C is the complexity of the operating area; T ' is the operation time; A It is the adaptability and flexibility of the operation; w 5. w 6. w 7. w 8. w 9 are the weights of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation; α 2. α 3. α 4. α 5. α 6 are the nonlinear weighting coefficients of work efficiency, status, complexity, operation time, and adaptability and flexibility of the operation, respectively. β 1. β 2. β 3. β 5. β 6 are adjustment coefficients for work efficiency, status, complexity, operation time, and adaptability and flexibility of operations; Define the fuzzy matrix domain and membership function of the robot's operation feasibility, and construct the fuzzy matrix of the robot's operation feasibility P matrix : ， in, ELP Represents high feasibility, LP Represents higher feasibility, MP Represents medium feasibility, HP Represents lower feasibility, EHP represents low feasibility; Define the fuzzy matrix domain and membership function of the robot's task execution capability index, and construct the fuzzy matrix of the robot's task execution capability index C o : ， in, ELC Represents extremely low execution capability. LC Represents lower execution capability. MC Represents medium executive ability. HC Represents higher execution capability. EHC Represents high executive ability; Define the fuzzy matrix domain and membership function of the robot's mechanical arm operating force, and construct the fuzzy matrix of the robot's mechanical arm operating force S matrix : ， in, ELS Represents low intensity, LS Represents lower strength, MS Represents medium strength. HS Represents a higher intensity. EHS Represents high intensity; Constructing the fuzzy relationship matrix between the robot's operational feasibility and the robot's operational strength M 1 and the fuzzy relationship matrix between the robot's operation execution capability index and the robot's operation strength M 2: ， ， in, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 1. M 2. Update the robot's arm operation force and its membership function: ， in, It is the fuzzy matrix of the robot's mechanical arm operating force after update.

8. The integrated method for cultivating and picking sugar apples according to claim 6, characterized in that: The maximum membership method is used to obtain the moving speed control scheme of each cluster of robots, specifically: Define device movement stability: ， in, TAI Provides stability for device movement; Sl is the slope, LS is land stability, w 10 , w 11 are the weights of slope and land stability, α 7. α 8 are the nonlinear coefficients controlling the slope and land stability, β 7. β 8 are the adjustment coefficients for slope and land stability respectively; Define the fuzzy matrix domain and its membership function of the robot's equipment movement stability, and construct the fuzzy matrix of equipment movement stability TAI matrix : ， in, ELT Represents high stability, LT Represents higher stability, MT Represents medium stability. HT Represents lower stability, EHT Represents low stability; Define the fuzzy matrix domain and membership function of the robot's moving speed, and construct the fuzzy matrix of the moving speed V matrix : ， in, ELV Represents stillness, LV Represents slow movement, MV Represents medium-speed movement. HV Represents higher speed movement. EHV Represents high-speed movement; Constructing the fuzzy relationship matrix between the robot's equipment movement stability and movement speed M 3 and the fuzzy relationship matrix between the robot's operation feasibility and movement speed M 4: ， ， in, P matrix is the fuzzy matrix of job feasibility, T It is the matrix transpose operation; Coupled fuzzy relationship matrix M 3. M 4. Update the fuzzy matrix and membership function of the robot's moving speed: ， in, is the fuzzy matrix of the robot's moving speed after update.

9. The integrated method for cultivating and picking sugar apples according to claim 6, characterized in that: The maximum membership method is used to obtain the fill light control scheme for each cluster of robots, specifically: Define the working environment light efficiency index: ELEI = L × e -αH ， in, L is the light sensitivity of the sugar apple when the robot is working. H is the air humidity, α It is the adjustment constant used to adjust the effect of humidity on the light efficiency index of the working environment; Define the fuzzy matrix domain and membership function of the robot's working environment light efficiency index, and construct the fuzzy matrix of the working environment light efficiency index ELEI matrix : ， in, ELE Represents low light efficiency index, LE Represents a lower light efficiency index, ME Represents the medium light efficiency index. HE Represents a higher light efficiency index, EHE Represents high light efficiency index; Define the fuzzy matrix domain and membership function of the robot's light sensitivity, and construct the fuzzy matrix of the robot's light sensitivity L matrix : ， in, ELL Represents low light sensitivity. LL Represents lower light sensitivity. ML Represents medium sensitivity. HL Represents a higher sensitivity. EHL Represents high sensitivity; Constructing the fuzzy relationship matrix between the light efficiency index and light sensitivity of the robot's working environment M 5: ， in, T It is the matrix transpose operation; Based on the inverse relationship between light sensitivity and fill light, the fuzzy relationship matrix between the light efficiency index of the working environment and the light sensitivity is constructed. M 5 Calculate and update the fuzzy matrix and membership function of the robot's fill light: ， in, It is the fuzzy matrix of the robot's fill light after update.

10. A machine vision-based integrated system for cultivating and picking sugar apples, characterized in that: The machine vision-based integrated method for cultivating and picking sugar apples as described in any one of claims 1 to 9, wherein the system comprises a data acquisition module, a cloud computing module, a control module, a communication module, a vision module, an individual computing module and a main control module; The data acquisition module is used to obtain image data of sugar apples in the planting area through a drone, build a three-dimensional environmental model of sugar apples in combination with a multi-view three-dimensional reconstruction algorithm, and extract spatial data of sugar apples; the spatial data includes spatial coordinates, shape, size, volume, temperature, humidity, wind speed and light sensitivity; The cloud computing module is used to cluster the sugar apples according to the spatial distribution of the planting area based on the three-dimensional environmental model of the sugar apples, and dispatch a robot equipped with a mechanical arm to each cluster for cultivation and picking operations; The visual module is used to control the drone to count the number of sugar apples in each cluster, and to plan the path of the robot in each cluster according to the number of sugar apples in each cluster; The individual calculation module is used to generate a mechanical arm force control scheme, a moving speed control scheme and a fill light control scheme for each cluster of robots based on fuzzy mathematics when each cluster of robots is working; The main control module is used for each cluster of robots to cultivate and pick sugar apples according to the mechanical arm strength control scheme, the moving speed control scheme and the fill light control scheme; The communication module is used for encrypted transmission of data between modules; The control module is responsible for task scheduling, data management and logic control among modules.

Images