Papaya full-automatic maturity identification and picking method and system based on image identification

Abstract

The invention belongs to the technical field of image recognition, and particularly relates to a papaya full-automatic maturity recognition and picking method and system based on image recognition, which is provided with a track facility, can enable a picking robot to carry out periodic inspection tour picking according to an accurate set path, has higher picking precision and inspection tour efficiency, runs stably, and is suitable for large-scale popularization and application. Collision in the transportation process of the fruits is reduced; the suction cup is selected as a papaya tail end picking actuator and can be attached to the surface of papaya to a large extent, and extrusion damage to papaya caused by a traditional grabbing and clamping device is reduced. A six-axis mechanical arm is used for driving a tail end picking actuator to move to a designated position, excellent flexibility is achieved, the robot can run beside papayas to be picked strictly according to a path set by a computer, and blocking of leaves, branches and immature papayas to the actuator is reduced; a depth-of-field camera is combined with an image processing algorithm to carry out position judgment and system establishment on papaya fruits, and efficiency and cost are both considered.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a method and system for fully automatic ripening identification and picking of papayas based on image recognition. Background Art

[0002] Papaya is a large, perennial herbaceous fruit tree, typically 3-10 meters tall with a crown width of approximately 3-4 meters. Commercial cultivation typically requires a plant spacing of approximately 2-2.2 x 2.5 meters. The trunk is typically unbranched, and the leaves are large, palmately divided, and arranged in a spiral pattern. As the tree grows, the lower leaves gradually age and fall off, and by the time the fruit is ripe and ready for harvest, the leaves are concentrated at the top of the tree, forming an umbrella shape. Papaya is widely cultivated in tropical and subtropical regions. It boasts rich nutritional value, a unique flavor, a wide range of uses, a short growing cycle, and high economic returns with rapid results. According to FAO statistics, the world's papaya planting area is approximately 7 million mu (approximately 16,000 hectares), with a production of 13 million tons. my country is a major producing region, with a planting area of ​​approximately 300,000 mu (approximately 16,000 hectares) and a production of approximately 1.5 million tons.

[0003] Papaya fruit develops from flowers, which do not open simultaneously. Papaya is a species that blooms and bears fruit year-round. The fruits from flowers that open early mature first, while those from flowers that open later mature later. This results in significant differences in the ripening time of different fruits on the same tree, and across an entire plantation, the individual differences are even more pronounced. Therefore, in current traditional papaya cultivation, a large amount of manpower and material resources are required for frequent, periodic inspections to identify papaya fruits that meet maturity standards. Furthermore, the characteristics of papayas that meet the "suitability for picking" criteria are not obvious, and manual identification accuracy is low, further limiting the economic benefits of large-scale papaya cultivation. Summary of the Invention

[0004] According to a first aspect of the present invention, the present invention claims protection for a fully automatic ripe papaya identification and picking method based on image recognition, comprising:

[0005] Get the papaya picking object processing request started by the real-time level linked list;

[0006] Determining the papaya picking object level of the real-time level linked list;

[0007] Determine the primary and backup picking robotic arms adapted to the real-time grade linked list based on the grade of the papaya picking object; wherein the primary and backup picking robotic arms include a primary picking robotic arm and multiple backup picking robotic arms;

[0008] The candidate picking robots are scheduled for the real-time ranking list based on the primary and backup picking robotic arms.

[0009] Furthermore, the step of determining the papaya picking object level of the real-time level linked list further includes:

[0010] Selecting a standard lighting environment, and obtaining photos of the papaya picking subject before and after the suitable picking stage;

[0011] Finding the characteristics of the ripeness of the papaya picking object through manual judgment;

[0012] Tracks are laid between papaya trees covering the plantation. Picking robots travel along fixed tracks, equipped with light sensors that monitor ambient light intensity in real time.

[0013] When circling the field, the picking robot's camera is directed toward the papaya trees on one side of the track, and the depth-of-field camera acquires multiple depth-of-field images and ordinary color images at different angles on one side of each plant;

[0014] Traversing each pixel point of the multiple depth of field images and ordinary color images, calculating the normal vector and curvature of the adjacent area, and judging whether they are located on the same surface based on their change trend. If so, continue traversing the surrounding points and count them into the point set;

[0015] Substitute each point in this point set into the ellipsoid error function to determine the first level of the papaya fruit picking object to which the surface of the point set belongs;

[0016] Projecting the area on the surface of the point set in two dimensions onto a color image captured simultaneously, binarizing the image based on pixel RGB values ​​to determine the ripening area, and obtaining a second level of picking objects;

[0017] The papaya picking object level of the real-time level chain table is determined based on the first picking object level and the second picking object level.

[0018] Furthermore, the method further comprises:

[0019] Determining the suction cup operation time for each level of picking robotic arms based on the picking robotic arm identifiers of the picking robotic arms of each level;

[0020] The suction cup operation time is used to determine the suction cup rotation time of the level picking robotic arm as a backup picking robotic arm.

[0021] Furthermore, the step of determining the master and standby picking robot arms adapted to the real-time grade linked list based on the grade of the papaya picking object includes:

[0022] Based on the papaya picking object level and the picking robot arm identification of each level of picking robot arm, a candidate picking robot arm identification is obtained, and the picking robot arm of the level corresponding to the candidate picking robot arm identification is used as the main picking robot arm;

[0023] A backup picking robotic arm is determined from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy.

[0024] Furthermore, the step of determining a backup picking robotic arm from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy includes:

[0025] In an environment where the papaya picking object level is determined to be in the first level state, the second level picking robotic arm and the third level picking robotic arm are determined as the backup picking robotic arms;

[0026] In an environment where the papaya picking object level is determined to be in the second level state, the third level picking robotic arm is determined as the first backup picking robotic arm, and the first level picking robotic arm is determined as the second backup picking robotic arm;

[0027] In an environment where the papaya picking object level is determined to be in the third level state, the second-level picking robot arm is determined as the backup picking robot arm.

[0028] Furthermore, the step of determining a backup picking robotic arm from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy includes:

[0029] Obtaining the harvested papaya scheduling data of each level of harvesting robotic arms for the level of the papaya harvesting object;

[0030] Determining an idle picking robot arm that has picked fruits from the other picking robot arms based on the picked scheduling data;

[0031] The idle picking robotic arm that has already picked fruits is determined as the backup picking robotic arm.

[0032] Furthermore, the scheduling of candidate picking robots for the real-time ranking list based on the primary and backup picking robotic arms includes:

[0033] In an environment where the picking robots included in the main picking robot arm meet the suction cup requirements of the real-time grade chain table, determining the picking robots in the main picking robot arm as the candidate picking robots;

[0034] In an environment where the picking robots included in the main picking robotic arm do not meet the suction cup requirements of the real-time grade chain list, the picking robot in the main picking robotic arm is determined as the first candidate picking robot, and the picking robot in the backup picking robotic arm is determined as the second candidate picking robot; wherein, the first candidate picking robot and the second candidate picking robot together constitute the candidate picking robot.

[0035] Furthermore, determining the picking robot in the backup picking robot arm as the second candidate picking robot includes:

[0036] In an environment where it is determined that there is only one backup picking robot arm, determining the picking robot in the backup picking robot arm as the second candidate picking robot;

[0037] In an environment where it is determined that there are multiple backup picking robotic arms, a candidate backup picking robotic arm is determined from each of the backup picking robotic arms, and the picking robot in the candidate backup picking robotic arm is determined as the second candidate picking robot.

[0038] Furthermore, the method further comprises:

[0039] In an environment where it is determined that the candidate picking robots include a picking robot with a backup picking robot arm, obtaining a suction cup operation time of the backup picking robot arm;

[0040] In an environment where it is determined that the suction cup operation time reaches the suction cup operation time, or the picking robot of the main picking robot arm meets the suction cup requirement of the real-time grade chain table, the picking robot of the backup picking robot arm included in the candidate picking robots is rotated;

[0041] The main picking robot arm and the backup picking robot arm are a positioning program pool, and the candidate picking robot is a positioning program suction cup.

[0042] According to a second aspect of the present invention, the present invention claims protection for a fully automatic papaya ripening identification and picking system based on image recognition, which is used to perform the fully automatic papaya ripening identification and picking method based on image recognition, comprising:

[0043] The six-axis robotic arm is fixed on the chassis, with a first-axis drive motor 3, a second-axis drive motor 4, a third-axis drive motor 5, a fourth-axis drive motor 6, a fifth-axis drive motor 7, and a sixth-axis drive motor 8, as well as an actuator 9 at the end thereof, including a suction cup 11, a depth of field camera 12, a light sensor 13, and a lighting lamp 14, and the sixth-axis drive motor 8.

[0044] The technical effects of the present invention are as follows:

[0045] (1) Reduce the labor cost of papaya picking and increase the market competitiveness of papaya production. The system can efficiently complete the papaya maturity identification, papaya picking and papaya transportation within the plantation, greatly reducing the plantation's demand for human resources and completing the picking of mature papayas in a timely manner.

[0046] (2) This technical solution has track facilities that enable the unmanned picking vehicle to perform periodic inspections and picking according to the accurate set path, with high picking accuracy and inspection efficiency. The operation is smooth, reducing the bumps and collisions during the transportation of fruits.

[0047] (3) The suction cup is used as the end-picking actuator of papaya, which can fit the surface of papaya to a greater extent and reduce the squeezing damage to papaya caused by traditional gripping devices.

[0048] (4) A six-axis robotic arm is used to drive the end-picking actuator to the designated position. It has excellent flexibility and can strictly follow the path planned by the computer to move to the papaya to be picked, reducing the obstruction of leaves, branches and unripe papayas to the actuator.

[0049] (5) Use a depth-of-field camera combined with an image processing algorithm to determine the position and establish a system for papaya fruits, taking into account both efficiency and cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a method for fully automatic ripe papaya identification and picking based on image recognition as claimed in an embodiment of the present invention;

[0051] Figure 2 A schematic structural diagram of a picking robot in a fully automatic papaya ripening identification and picking system based on image recognition as claimed in an embodiment of the present invention;

[0052] Figure 3 A second structural schematic diagram of a picking robot of a fully automatic papaya ripening identification and picking system based on image recognition as claimed in an embodiment of the present invention;

[0053] Figure 4 This is a second flow chart of a fully automatic papaya ripening identification and picking method based on image recognition claimed in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] The terms "first", "second" and "third" in this application are used only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second" and "third" may explicitly or implicitly include multiple such features. In the description of this application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of this application (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, motion environment, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0056] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in multiple embodiments of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0057] Figure 1 This is a flowchart of a fully automatic papaya ripening identification and picking method based on image recognition provided by an embodiment of the present application. This embodiment is applicable to an environment where a primary and secondary picking robot arm is used to process papaya picking objects of a server. The method can be executed by a suction cup scheduling device, which can be implemented by software and / or hardware and can generally be integrated into a computer device. The computer device can serve as a server to execute various levels of papaya picking objects. Accordingly, Figure 1 As shown, the method includes the following operations:

[0058] S110: Obtain a papaya picking object processing request initiated by a real-time level linked list.

[0059] The real-time level linked list may be a calculated papaya picking object in a certain level papaya picking object linked list being processed by the server in real time. The papaya picking object processing request may be initiated by the real-time level linked list to request execution of the request of the real-time level linked list.

[0060] There may be multiple papaya picking objects of different levels to be processed in the server. When the concurrency of papaya picking objects is high, papaya picking objects of each level will be cached in the corresponding level chain list, waiting to be executed by the server. For example, when the papaya picking object status that the server can process protects high-level papaya picking objects, medium-level papaya picking objects, and low-level papaya picking objects, after the server obtains each papaya picking object to be processed, it can add it to each level chain list based on the level status corresponding to each papaya picking object to be processed, forming a high-level chain list, a medium-level chain list, and a low-level chain list. In each level chain list, a certain number of papaya picking objects can be stored, or the papaya picking objects to be processed in some level chain lists can also be empty, indicating that the real-time server does not need to process papaya picking objects of that level status. Optionally, the level status of each papaya picking object to be processed can be determined based on factors such as the timeliness and importance of the papaya picking object, and added to the corresponding level chain list.

[0061] Accordingly, each hierarchical linked list can send a papaya picking object processing request to the server based on the needs of each papaya picking object to be processed, requesting the server to process the papaya picking object to be processed. Accordingly, the server can obtain the papaya picking object processing request initiated by the real-time hierarchical linked list.

[0062] S120: Determine the papaya picking object level in the real-time level linked list.

[0063] The papaya picking object level, i.e., the level status of the real-time level linked list, can be determined based on the level status supported by the server. For example, when the level status supported by the server includes four levels: emergency level, high level, medium level, and low level, the papaya picking object level can be one of the emergency level, high level, medium level, and low level. When the level status supported by the server includes three levels: high level, medium level, and low level, the papaya picking object level can be one of the high level, medium level, and low level.

[0064] The papaya grade is determined by analyzing the following characteristic parameters using an image recognition algorithm:

[0065] Color features (HSV color space threshold):

[0066] High level (mature): H (hue) ∈ [20, 30], S (saturation) ≥ 0.7, V (value) ≥ 0.8 (orange-red and bright);

[0067] Medium (half-cooked): H∈[30,50], S∈[0.5,0.7], V∈[0.6,0.8] (yellow-orange);

[0068] Low grade (unripe): H∈[50,70], S≤0.5, V≤0.6 (turquoise).

[0069] Texture features (local binary pattern LBP algorithm):

[0070] The surface texture of ripe papaya is smooth (LBP variance ≤ 0.1), while that of unripe papaya is rough (LBP variance > 0.3).

[0071] Shape / Size (minimum circumscribed circle diameter):

[0072] High grade: diameter ≥10cm; medium grade: 8-10cm; low grade: <8cm.

[0073] The camera captures papaya images and extracts HSV, LBP and size data after preprocessing;

[0074] According to the above threshold classification level, the results are output to the control system.

[0075] The robot arm grades are classified according to their motion accuracy, load capacity and end effector type:

[0076] High-grade robotic arm: repeatability accuracy ±0.1mm, load ≥5kg, equipped with flexible grippers (to prevent damage to ripe fruits);

[0077] Intermediate robotic arm: accuracy ±0.5mm, load 3-5kg, equipped with standard grippers;

[0078] Low-level robotic arm: accuracy ±1mm, load <3kg, equipped with universal grippers.

[0079] High-grade papaya (fragile) → high-grade robotic arm (high precision + flexible gripper);

[0080] Intermediate Papaya → Intermediate Robotic Arm;

[0081] Low-level papaya (high hardness) → low-level robotic arm.

[0082] The system receives the papaya level signal and calls the corresponding robotic arm;

[0083] The high-grade robotic arm picks fruits with a constant clamping force of 5N, and the path planning avoids the sensitive areas of the fruit stems.

[0084] Image recognition of a papaya with H = 25, S = 0.75, V = 0.9, diameter 12 cm, and LBP variance 0.08 → judged as high grade;

[0085] Dispatching high-grade robotic arms (accuracy ±0.1mm, flexible grippers);

[0086] Based on 3D positioning information, the robotic arm contacts the fruit stem in an arc trajectory, and the clamping force feedback control ensures no damage.

[0087] S130. Determine the primary and backup picking robotic arms adapted to the real-time grade linked list based on the grade of the papaya picking object; wherein the primary and backup picking robotic arms include a primary picking robotic arm and multiple backup picking robotic arms.

[0088] The primary and backup picking robotic arms can be harvesting robotic arms that the server divides into different levels of papaya harvesting objects based on the computing suction cups they hold. The primary and backup harvesting robotic arms include a single primary harvesting robotic arm and multiple backup harvesting robotic arms. The primary harvesting robotic arm can be a harvesting robotic arm dedicated to harvesting papayas of a specific level, while the backup harvesting robotic arms can be harvesting robotic arms that serve papayas of other levels.

[0089] In an embodiment of the present application, after the server determines the papaya picking object level corresponding to the real-time level linked list, it can further determine the adapted primary and backup picking robotic arms based on the papaya picking object level in the real-time level linked list.

[0090] It can be understood that the main picking robot arm and the backup picking robot arm are relative concepts and need to be determined based on the level status of the papaya picking object. For example, when the real-time level list is a high-level papaya picking object, the corresponding main and backup picking robot arms can be specifically as follows: the main picking robot arm is a picking robot arm that handles high-level papaya picking objects, and the backup picking robot arm is two picking robot arms that handle medium-level papaya picking objects and low-level papaya picking objects. When the real-time level list is a medium-level papaya picking object, the corresponding main and backup picking robot arms can be specifically as follows: the main picking robot arm is a picking robot arm that handles medium-level papaya picking objects, and the backup picking robot arm is a picking robot arm that handles medium and low-level papaya picking objects.

[0091] S140: Scheduling candidate picking robots for the real-time ranking list based on the primary and backup picking robotic arms.

[0092] Accordingly, after determining the master and backup picking robot arms that are compatible with the real-time level list, the server can schedule candidate picking robots for the real-time level list based on the determined master and backup picking robot arms to process the real-time level list based on the candidate picking robots.

[0093] Optionally, the server may first use the computing sucker included in the main picking robot arm as a candidate picking robot to process the real-time ranking list. If the computing sucker included in the main picking robot arm does not meet the processing requirements of the real-time ranking list, the server may further use the idle computing sucker included in the backup picking robot arm as a candidate picking robot to assist in processing the real-time ranking list.

[0094] In an optional embodiment of the present application, the main picking robot arm and the backup picking robot arm may be a positioning program pool, and correspondingly, the candidate picking robot may be a positioning program suction cup.

[0095] Furthermore, the step of determining the papaya picking object level of the real-time level linked list further includes:

[0096] Selecting a standard lighting environment, and obtaining photos of the papaya picking subject before and after the suitable picking stage;

[0097] Finding the characteristics of the ripeness of the papaya picking object through manual judgment;

[0098] Tracks are laid between papaya trees covering the plantation. Picking robots travel along fixed tracks, equipped with light sensors that monitor ambient light intensity in real time.

[0099] When circling the field, the picking robot's camera is directed toward the papaya trees on one side of the track, and the depth-of-field camera acquires multiple depth-of-field images and ordinary color images at different angles on one side of each plant;

[0100] Traversing each pixel point of the multiple depth of field images and ordinary color images, calculating the normal vector and curvature of the adjacent area, and judging whether they are located on the same surface based on their change trend. If so, continue traversing the surrounding points and count them into the point set;

[0101] Substitute each point in this point set into the ellipsoid error function to determine the first level of the papaya fruit picking object to which the surface of the point set belongs;

[0102] Projecting the area on the surface of the point set in two dimensions onto a color image captured simultaneously, binarizing the image based on pixel RGB values ​​to determine the ripening area, and obtaining a second level of picking objects;

[0103] The papaya picking object level of the real-time level chain table is determined based on the first picking object level and the second picking object level.

[0104] In this embodiment, the characteristics of the ripeness of the papaya variety are found by manual judgment, specifically including: 1. The RGB value of a certain orange on the surface of the ripe papaya fruit and its proportion are set to a floating range of 5%, recorded as (R min ,G min ,B min ) and (Rmax ,G max ,B max ).

[0105] Laying tracks between papaya trees covering plantations, e.g. Figure 2 The picking robot of the fully automatic papaya ripening identification and picking system based on image recognition in this embodiment includes:

[0106] A six-axis robotic arm, mounted on a chassis, is equipped with a first-axis drive motor 3, a second-axis drive motor 4, a third-axis drive motor 5, a fourth-axis drive motor 6, a fifth-axis drive motor 7, and a sixth-axis drive motor 8. The actuator 9 at its end includes a suction cup 11, a depth-of-field camera 12, a light sensor 13, and a light 14. The sixth-axis drive motor 8 is equipped in the harvesting robot to perform a daily round-the-field routine. The harvesting robot travels along a fixed track, and its onboard light sensor monitors ambient light intensity in real time. The light sensor extracts light information and sends it to a central computer. A brightness compensation program then drives the light to compensate for the brightness, ensuring uniform and appropriate illumination of the papayas.

[0107] When circling the field, the picking robot's camera faces the papaya trees on one side of the track. The depth-of-field camera acquires multiple depth-of-field images and normal color images at different angles on each side of each plant. The rotation matrix of each image relative to the first image is Where θ is the rotation angle of the camera relative to the first position; the displacement vector of each image relative to the first image is Where ΔX, ΔY, and ΔZ are the world coordinate changes of the camera relative to the first position. Use the formula P = P i R i (θ)+T i The pixel coordinate information of each position is unified into a coordinate system, where P i is the pixel coordinate matrix of each position.

[0108] Traverse each point and calculate the normal vector and curvature of the adjacent area. Use the trend of their change to determine whether they are on the same surface. If so, continue traversing the surrounding points and add them to the point set. Next, solve the ellipsoid parameters of the fruit and prevent overlapping fruits.

[0109] Divide this point set into multiple subsets, and randomly select 6 points from a subset and substitute them into the matrix equation:

[0110]

[0111] Solve [ABCDEFG] T Then get the coordinates of the center of the sphere: and semi-axis length

[0112] Each point of this subset (X i , Y i , Z i )Substitute into the ellipsoid error function If the function value is close to 0, then the points in this subset can be determined to belong to the first papaya fruit. Keeping the center and semi-axis lengths unchanged, substitute the ellipsoid error function E into the remaining subsets one by one. If the function value is close to 0, then this subset is also included in the fruit surface area.

[0113] Project the surface area two-dimensionally onto the color image taken at the same time. For the color corresponding to this surface, binarize the image based on the pixel RGB value, with the mature area as 1 and the other areas as 0. The formula is as follows:

[0114]

[0115] Use the Mat data structure in the OpenCV library to process the image and generate a binary image. Calculate the percentage of the mature area. If it is greater than 80%, the fruit is considered to meet the picking criteria and the following picking procedure is executed:

[0116] The positioning program drives the six-axis robotic arm, aligning the suction cup with the papaya's ellipsoid pole and creating negative pressure suction through the air pump. The sixth axis drives the suction cup to apply torque τ, rotating the fruit back and forth, and recording the rotation angle θ and angular velocity. The fruit stem system can be simplified into an elastic rod structure with low fatigue strength, and the equivalent damping can be calculated using the following formula: When C suddenly drops to near 0, the robotic arm moves backward to pull the fruit off. Where K is the stalk stiffness of the papaya variety.

[0117] If no papaya meeting the picking criteria is identified, the operation continues;

[0118] The number of papayas that the picking robot can accommodate is preset based on the average size and weight of the papaya varieties in the orchard. When the number of picked papayas reaches the preset number, the picking stops and the papayas are unloaded manually;

[0119] It can be seen from this that the fully automatic papaya maturity identification and picking method based on image recognition provided by the embodiment of the present application is a combination of dynamic and static methods. Each level chain list can be dispatched to a corresponding main picking robot arm. When the main picking robot arm does not meet the processing requirements of the real-time level chain list, the backup picking robot arm that processes papaya picking objects of other levels can be dispatched to calculate the suction cup to jointly process the real-time level chain list. This suction cup scheduling method can not only meet the processing requirements of the real-time level chain list and improve the execution efficiency and execution quality of the papaya picking objects, but also maximize the use of idle suction cups to centrally process papaya picking objects, thereby improving the suction cup utilization rate of the server.

[0120] After the embodiment of the present application obtains the papaya picking object processing request initiated by the real-time level linked list, the papaya picking object level of the real-time level linked list is determined, and the master and backup picking robotic arms adapted to the real-time level linked list, including a master picking robotic arm and multiple backup picking robotic arms, are determined based on the papaya picking object level of the real-time level linked list. The candidate picking robots are scheduled for the real-time level linked list based on the determined master and backup picking robotic arms, thereby solving the problems of low suction cup utilization and poor papaya picking object execution efficiency and execution quality when the existing server schedules suction cups, thereby improving the server's suction cup utilization, and the execution efficiency and execution quality of the papaya picking objects.

[0121] In one example, Figure 3 This is a flowchart of a fully automatic papaya maturity identification and picking method based on image recognition provided by an embodiment of the present application. The embodiment of the present application has been optimized and improved on the basis of the technical solutions of the above embodiments, and provides a plurality of specific optional implementation methods for determining the master and standby picking robotic arms adapted to the real-time level list based on the papaya picking object level, scheduling candidate picking robots for the real-time level list based on the master and standby picking robotic arms, and performing operations before obtaining the papaya picking object processing request initiated by the real-time level list.

[0122] like Figure 4 The method for fully automatic ripe papaya identification and picking based on image recognition is shown, comprising:

[0123] S210: Obtain the level status of the picking robot and the papaya picking object that meets the picking conditions.

[0124] The picking robots may be all picking robots held by the server, such as all positioning program suckers or memory suckers, etc. The level status of the papaya picking object is the level status of the papaya picking object supported by the server.

[0125] Before dispatching suction cups to the papaya picking object being processed, the server first needs to divide the picking robot arms and determine the primary and backup picking robot arms based on the divided picking robot arms. Accordingly, the server needs to obtain the level status of the picking robot and the papaya picking object that meets the required picking conditions.

[0126] Taking the positioning program suction cup as an example of a picking robot, the server first confirms the total amount of the positioning program, and at the same time sets the papaya picking object level status that the system can support. Among them, the picking robot is determined by the hardware structure of the server, and the papaya picking object level status can be configured by the server based on business needs. For example, the server can set the supported papaya picking object level status to include four states: emergency level, high level, medium level and low level, or three states: high level, medium level and low level. The embodiment of the present application does not limit the specific content of the suction cup status, the total amount of suction cups, and the papaya picking object level status of the picking robot.

[0127] S220: Divide the picking robot into multiple levels of picking robotic arms based on the level status of the papaya picking objects.

[0128] Among them, one of the hierarchical picking robotic arms includes multiple unit picking robots.

[0129] A hierarchical picking robot arm is a picking robot arm composed of a plurality of picking robots. A unit picking robot is a unit of picking robots. For example, when a positioning program suction cup is dispatched as a picking robot, the unit picking robot can be a positioning program.

[0130] After the server obtains data such as the picking robot and the grade status of the papaya picking object that meets the picking conditions, the picking robot can be divided into multiple grade picking robotic arms based on the grade status of the papaya picking object. Each grade status of the papaya picking object can correspond to a grade picking robotic arm as the main picking robotic arm. Each grade picking robotic arm can include multiple unit picking robots. Under normal circumstances, each grade picking robotic arm includes multiple unit picking robots. Each grade picking robotic arm gives priority to serving the papaya picking object of its adapted grade status and can serve as the main picking robotic arm of the papaya picking object of its adapted grade status. When the grade picking robotic arm includes an idle suction cup, it can serve other papaya picking objects of unsuitable grade status and can serve as a backup picking robotic arm for the papaya picking object of its unsuitable grade status.

[0131] In an optional embodiment of the present application, the dividing the picking robots into multiple levels of picking robotic arms based on the papaya picking object level status may include: in an environment where the papaya picking object level status is determined to be a first level status, dividing a first number of picking robots from the picking robots to construct a first level picking robotic arm; in an environment where the papaya picking object level status is determined to be a second level status, dividing a second number of picking robots from other picking robots to construct a second level picking robotic arm; in an environment where the papaya picking object level status is determined to be a first level status, dividing a third number of picking robots from other picking robots to construct a third level picking robotic arm; wherein, the first number is greater than the second number, and the second number is greater than the third number.

[0132] The first level state is higher than the second level state, and the second level state is higher than the third level state. The first level picking robot arm may be the main picking robot arm for the papaya picking object in the first level state, the second level picking robot arm may be the main picking robot arm for the papaya picking object in the second level state, and the third level picking robot arm may be the main picking robot arm for the papaya picking object in the third level state.

[0133] Accordingly, the server can classify the picking robots into multiple levels of picking robotic arms based on the order of the papaya picking object's status from high to low. Considering that the first level status is the highest, the most picking robots can be dispatched for papaya picking objects in the first level status. Similarly, the second level status is the second highest, so more picking robots can be dispatched for papaya picking objects in the second level status. The third level status is the highest, so the fewest picking robots can be dispatched for papaya picking objects in the third level status.

[0134] It should be noted that the server can also expand or reduce the papaya picking object level status based on actual business needs. For example, a lower level fourth level status can be set, and fewer picking robots can be dispatched for papaya picking objects in the fourth level status. It should also be noted that the server can also use big data technology to analyze the papaya picking object data that has been picked and processed to dynamically adjust the supported papaya picking object level status, and at the same time dynamically adjust the number of suction cups in the level picking robot arm corresponding to the papaya picking object in each papaya picking object level status. This embodiment of the present application is not limited to this.

[0135] S230: Determine the picking robot arm identifier of each level of picking robot arm.

[0136] The picking robot arm identifier is used to determine the papaya picking object of the grade state to be executed by the picking robot in any of the graded picking robot arms.

[0137] Accordingly, after the server obtains multiple levels of picking robotic arms, it can determine the corresponding picking robotic arm identifier based on the level status of the papaya picking object adapted by the picking robotic arms of each level.

[0138] The above technical solution realizes the static division of picking robots based on business needs by pre-dividing papaya picking objects with corresponding grade picking robotic arms for each papaya picking object grade status. Fixed suction cups can be pre-dispatched for papaya picking objects with different papaya picking object grade status to ensure that papaya picking objects with each papaya picking object grade status can be reasonably dispatched to a certain number of picking robots to meet their papaya picking object processing needs.

[0139] In an optional embodiment of the present application, the fully automatic ripening identification and picking method of papaya based on image recognition may also include: determining the suction cup running time for each level of picking robotic arms based on the picking robotic arm identification of each level of picking robotic arms; wherein, the suction cup running time is used to determine the suction cup rotation time of the picking robotic arm of the level as a backup picking robotic arm.

[0140] The suction cup rotation time may be the time during which the suction cup is rotated.

[0141] In an embodiment of the present application, in order to avoid the backup picking robot arm being continuously occupied by the real-time level list, the server can also determine the suction cup operation time for each level of picking robot arm to clarify the available time for the level picking robot arm to be dispatched to other levels of papaya picking objects. It should be noted that the suction cup operation time of level picking robot arms in different level states can be the same or different. For example, the first-level picking robot arm mainly serves high-level papaya picking objects, and its corresponding suction cup operation time can be the shortest, that is, when the first-level picking robot arm is used as a backup picking robot arm, its corresponding callable time is the shortest. The third-level picking robot arm mainly serves low-level papaya picking objects, and its corresponding suction cup operation time can be the longest, that is, when the third-level picking robot arm is used as a backup picking robot arm, its corresponding callable time is the longest.

[0142] S240: Obtain a papaya picking object processing request initiated by a real-time level linked list, and determine the papaya picking object level of the real-time level linked list.

[0143] S250: Based on the papaya picking object level and the picking robot arm identifiers of the picking robot arms of each level, a candidate picking robot arm identifier is obtained, and the picking robot arm of the level corresponding to the candidate picking robot arm identifier is used as the main picking robot arm.

[0144] The candidate picking robot arm identifier may be a picking robot arm identifier that is compatible with the papaya picking object level state in the real-time level chain table.

[0145] Specifically, after determining the papaya picking object level in the real-time level list, the server can adapt the papaya picking object level in the real-time level list to the picking robot arm identification of each level of picking robot arm to obtain the candidate picking robot arm identification, and use the level picking robot arm corresponding to the candidate picking robot arm identification as the main picking robot arm.

[0146] For example, assuming that the papaya picking object level in the real-time level list is high, and the picking robot arm identifications of the level picking robot arms include high-level picking robot arms, medium-level picking robot arms and low-level picking robot arms, the papaya picking object level in the real-time level list is adapted to the picking robot arm identifications of the picking robot arms of each level, and the high-level picking robot arm is obtained as the candidate picking robot arm identification, and the level picking robot arm corresponding to the high-level picking robot arm is used as the main picking robot arm.

[0147] S260: Determine a backup picking robot arm from other picking robot arms based on the papaya picking object level and the backup picking robot arm determination strategy, and schedule candidate picking robots for the real-time level linked list based on the primary and backup picking robot arms.

[0148] The other picking robot arms may be picking robot arms of other levels except the main picking robot arm. The backup picking robot arm determination strategy may be a strategy for determining a backup picking robot arm.

[0149] Accordingly, the server can determine a backup picking robot arm from the other level picking robot arms except the main picking robot arm based on the papaya picking object level and the backup picking robot arm determination strategy. For example, assuming that the papaya picking object level in the real-time level list is high, and its corresponding main picking robot arm is a high-level picking robot arm, then the other picking robot arms can include a medium-level picking robot arm and a low-level picking robot arm. Accordingly, the server can select a picking robot arm from the medium-level picking robot arm and the low-level picking robot arm based on the backup picking robot arm determination strategy as the backup picking robot arm.

[0150] The above technical solution, by determining the graded picking robot arm adapted to the grade of the papaya picking object as the main picking robot arm, and determining the backup picking robot arm based on the grade of the papaya picking object and the backup picking robot arm determination strategy, can achieve the determination of different main and backup picking robot arms for the grade of papaya picking objects in different states, thereby enriching the configuration methods of the main and backup picking robot arms.

[0151] In an optional embodiment of the present application, the strategy for determining a backup picking robotic arm from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm may include: in an environment where the papaya picking object level is determined to be in the first level state, determining the second level picking robotic arm and the third level picking robotic arm as the backup picking robotic arm; in an environment where the papaya picking object level is determined to be in the second level state, determining the third level picking robotic arm as the first backup picking robotic arm, and determining the first level picking robotic arm as the second backup picking robotic arm; in an environment where the papaya picking object level is determined to be in the third level state, determining the second level picking robotic arm as the backup picking robotic arm.

[0152] Among them, the first backup picking robot arm can be the backup picking robot arm that is scheduled with priority, and the second backup picking robot arm can be the backup picking robot arm that is scheduled suboptimally. For example, assuming that the suction cups in the first backup picking robot arm and the second backup picking robot arm are both in an idle state, the first backup picking robot arm can be scheduled with priority to serve the real-time level chain list. If the first backup picking robot arm still does not meet the business processing requirements of the real-time level chain list, the second backup picking robot arm can continue to be scheduled to serve the real-time level chain list.

[0153] Accordingly, if the server determines that the papaya picking object level in the real-time level chain list is in the first level state, since the first level state has the highest level, the second level picking robot arm and the third level picking robot arm can be simultaneously determined as the backup picking robot arms in the real-time level chain list. If the server determines that the papaya picking object level in the real-time level chain list is in the second level state, since the second level state has the second highest level, the first level picking robot arm and the third level picking robot arm can be simultaneously determined as the backup picking robot arms in the real-time level chain list. However, considering that the first level picking robot arm is the main picking robot arm of the papaya picking object in the first level state, the third level picking robot arm with a lower level can be preferentially scheduled as the first backup picking robot arm. When the first backup picking robot arm does not meet the business processing requirements of the real-time level chain list, the first level picking robot arm with the highest level can be scheduled as the second backup picking robot arm. If the server determines that the papaya picking object in the real-time level list is in the third level state, since the papaya picking object in the third level state has the lowest level, the first level picking robot arm with the highest level will usually not be dispatched to serve it, then the second level picking robot arm can be determined as the backup picking robot arm in the real-time level list. It should be noted that if the business requirements stipulate that the papaya picking object in the third level state can use the suction cup of the first level picking robot arm, the second level picking robot arm with a lower level can be preferentially dispatched as the first backup picking robot arm. When the first backup picking robot arm does not meet the business processing requirements of the real-time level list, the first level picking robot arm with the highest level can be dispatched as the second backup picking robot arm.

[0154] The above technical solution enriches the configuration mode of the backup picking robotic arms and improves the utilization rate of the picking robots by determining the backup picking robotic arms from other picking robotic arms based on the different level states of the papaya picking object levels and business needs.

[0155] In an optional embodiment of the present application, the strategy for determining a backup picking robot arm from other picking robot arms based on the papaya picking object level and the backup picking robot arm may include: obtaining the picked scheduling data of the picking robot arms of each level for the papaya picking object level; determining the picked idle picking robot arm from the other picking robot arms based on the picked scheduling data; and determining the picked idle picking robot arm as the backup picking robot arm.

[0156] The harvested scheduling data may be the scheduling status data of the backup harvesting robot arms in all processed papaya harvesting objects corresponding to the papaya harvesting object level in the real-time level linked table. The harvested idle harvesting robot arms may be harvesting robot arms that are expected to be idle based on analysis of the harvested scheduling data.

[0157] Optionally, the server can also use the scheduling status data of the backup picking robot arms in the processed papaya picking objects of different papaya picking object levels to determine the slave suction cups of the real-time level chained list. Specifically, the server can obtain the picked scheduling data of the picking robot arms of each level for the papaya picking object level. For example, among the high-level papaya picking objects that have been processed, the server schedules the medium-level picking robot arms as backup picking robot arms for 90% of the high-level papaya picking objects. Accordingly, if the real-time level chained list is a high-level papaya picking object, the server can determine from other picking robot arms that the medium-level picking robot arm is an estimated idle picking robot arm based on the picked scheduling data of the high-level papaya picking object, and determine the idle picking robot arm, that is, the medium-level picking robot arm, as the backup picking robot arm of the real-time level chained list.

[0158] The above technical solution determines the backup picking robot arm from other picking robot arms based on the picking scheduling environment of the grade picking robot arm corresponding to the grade of the papaya picking object, thereby enriching the configuration method of the backup picking robot arm and improving the utilization rate of the picking robot.

[0159] It should be noted that when the level status of the papaya picking objects that the server can support is determined, the server can pre-set the master and backup picking robotic arms for papaya picking objects with different level statuses of papaya picking objects, or it can dynamically set the master and backup picking robotic arms in real time for the level linked list currently being processed. The embodiment of the present application does not limit the setting method of the master and backup picking robotic arms.

[0160] In a specific example, suppose the server schedules positioning program suckers for papaya picking objects of various levels. Instead of scheduling a fixed number of positioning programs for each level of papaya picking object, each level of papaya picking object schedules a master-slave pool of multiple master and backup positioning programs. For example, the master picking robot in the high-level linked list is from the high-level positioning program pool, while the backup picking robots are from the mid-level positioning program pool and the low-level positioning program pool. Similarly, the master picking robot in the mid-level linked list is from the mid-level positioning program pool, while the backup picking robots are from the high-level positioning program pool and the low-level positioning program pool. The master picking robot in the low-level linked list is from the low-level positioning program pool, while the backup picking robots are from the high-level positioning program pool and the mid-level positioning program pool. This sucker scheduling method can be pre-configured by the server. When the server receives a papaya picking object processing request initiated by the real-time level linked list and determines the level of the papaya picking object in the real-time level linked list, it directly determines the master and backup picking robot for the real-time level linked list based on the sucker configuration information and the level of the papaya picking object in the real-time level linked list. Alternatively, the server may also dynamically determine the primary and backup picking robotic arms in real time based on the papaya picking object levels in the real-time level chain table using the above-mentioned method for determining the primary and backup picking robotic arms.

[0161] In an optional embodiment of the present application, scheduling candidate picking robots for the real-time level chain list based on the main and backup picking robotic arms may include: in an environment where the picking robots included in the main picking robotic arm meet the suction cup requirements of the real-time level chain list, determining the picking robots in the main picking robotic arm as the candidate picking robots; in an environment where the picking robots included in the main picking robotic arm do not meet the suction cup requirements of the real-time level chain list, determining the picking robots in the main picking robotic arm as the first candidate picking robot, and determining the picking robots in the backup picking robotic arm as the second candidate picking robot; wherein, the first candidate picking robot and the second candidate picking robot together constitute the candidate picking robots.

[0162] The first candidate picking robot may be a picking robot included in a main picking robot arm, and the second candidate picking robot may be a picking robot in a backup picking robot arm.

[0163] In an embodiment of the present application, the server prioritizes scheduling the main picking robot arm for the real-time level chain list to process it. If the server determines that the picking robot included in the main picking robot arm meets the suction cup requirements of the real-time level chain list, that is, the picking robot included in the main picking robot arm meets the business processing requirements of the real-time level chain list, then the picking robot in the main picking robot arm can be directly determined as a candidate picking robot, and there is no need to schedule a backup picking robot arm for the real-time level chain list; otherwise, the server needs to determine the picking robot in the main picking robot arm as the first candidate picking robot, and at the same time, it needs to schedule the picking robot in the backup picking robot arm as the second candidate picking robot for the real-time level chain list to meet the suction cup requirements of the real-time level chain list.

[0164] The above technical solution schedules candidate picking robots for the real-time ranking list based on whether the picking robot of the main picking robot arm meets the suction cup requirements of the real-time ranking list. This not only meets the suction cup requirements of the real-time ranking list, ensuring the normal processing of the real-time ranking list, improving the efficiency and quality of papaya picking, but also avoids waste of picking robots, thereby improving the suction cup utilization rate of the server.

[0165] In an optional embodiment of the present application, determining the picking robot in the backup picking robotic arm as the second candidate picking robot may include: in an environment where it is determined that there is only one backup picking robotic arm, determining the picking robot in the backup picking robotic arm as the second candidate picking robot; in an environment where it is determined that there are multiple backup picking robotic arms, determining a candidate backup picking robotic arm from each of the backup picking robotic arms, and determining the picking robot in the candidate backup picking robotic arm as the second candidate picking robot.

[0166] The candidate backup picking robot arm may be a backup picking robot arm that provides a picking robot for the real-time grade linked list.

[0167] In an embodiment of the present application, in order to further avoid waste of picking robots and improve the suction cup utilization rate of the server, the server needs to further schedule the picking robots of the backup picking robotic arms based on the number of backup picking robotic arms. Optionally, if there is only one backup picking robotic arm, the picking robot in the backup picking robotic arm can be determined as the second candidate picking robot. If there are multiple backup picking robotic arms, one of the picking robotic arms can be selected from the picking robotic arms of each level as the candidate picking robotic arm, and the picking robot in the candidate backup picking robotic arm can be determined as the second candidate picking robot.

[0168] It is understandable that the second candidate picking robot that the server schedules to the real-time ranking list based on the backup picking robot arm is an idle picking robot. If the picking robot in the backup picking robot arm is occupied, the server may refuse to schedule the occupied picking robot in the backup picking robot arm to the real-time ranking list.

[0169] In an optional embodiment of the present application, the fully automatic ripe papaya identification and picking method based on image recognition may also include: in an environment where it is determined that the candidate picking robots include a picking robot with a backup picking robotic arm, obtaining the suction cup running time of the backup picking robotic arm; in an environment where it is determined that the suction cup running time reaches the suction cup running time, or the picking robot of the main picking robotic arm meets the suction cup requirements of the real-time grade list, rotating the picking robot with the backup picking robotic arm included in the candidate picking robots.

[0170] The suction cup operation time may be the cumulative time of the picking robot using the backup picking robotic arm in the real-time ranking list.

[0171] In an embodiment of the present application, the server can count the usage time of the picking robot of the backup picking robot arm among the candidate picking robots in real time as the suction cup operation time. Since each backup picking robot arm is configured with a corresponding suction cup operation time, when the server determines that the suction cup operation time reaches the suction cup operation time, the picking robot of the backup picking robot arm included in the candidate picking robots needs to be rotated. Alternatively, before the server determines that the suction cup operation time reaches the suction cup operation time, if the picking robot of the main picking robot arm is able to meet the suction cup requirements of the real-time level chain list, the picking robot of the backup picking robot arm included in the candidate picking robots also needs to be rotated in real time.

[0172] The above technical solution can avoid long-term occupation of the picking robot of the backup picking robot arm by timely rotating the picking robot of the backup picking robot arm included in the candidate picking robots based on the suction cup operation time, the suction cup operation time and the use environment of the picking robot of the main picking robot arm, resulting in the occupied backup picking robot arm being unable to serve the papaya picking object of its adapted level status in time, thereby avoiding the papaya picking object of the adapted level status of the backup picking robot arm being unable to be processed and executed as scheduled.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0174] In addition, the functional units in the various embodiments of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0175] The above detailed description of the specific embodiments of the invention is intended only as an example, and the present application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions of the invention are also within the scope of the present application. Therefore, equivalent changes, modifications, and improvements made without departing from the spirit and scope of the present application should be included within the scope of the present application.

Claims

1. A fully automatic papaya ripening identification and picking method based on image recognition, characterized in that: include: Get the papaya picking object processing request started by the real-time level linked list; Determining the papaya picking object level of the real-time level linked list; Determine the primary and backup picking robotic arms adapted to the real-time grade linked list based on the grade of the papaya picking object; wherein the primary and backup picking robotic arms include a primary picking robotic arm and multiple backup picking robotic arms; The candidate picking robots are scheduled for the real-time ranking list based on the primary and backup picking robotic arms.

2. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 1, characterized in that: Determining the papaya picking object level of the real-time level chain table further includes: Selecting a standard lighting environment, and obtaining photos of the papaya picking subject before and after the suitable picking stage; Finding the characteristics of the ripeness of the papaya picking object through manual judgment; Tracks are laid between papaya trees covering the plantation. Picking robots travel along fixed tracks, equipped with light sensors that monitor ambient light intensity in real time. When circling the field, the picking robot's camera is directed toward the papaya trees on one side of the track, and the depth-of-field camera acquires multiple depth-of-field images and ordinary color images at different angles on one side of each plant; Traversing each pixel point of the multiple depth of field images and ordinary color images, calculating the normal vector and curvature of the adjacent area, and judging whether they are located on the same surface based on their change trend, if so, continue to traverse the surrounding points and count them into the point set; Substitute each point in this point set into the ellipsoid error function to determine the first level of the papaya fruit picking object to which the surface of the point set belongs; Projecting the area on the surface of the point set in two dimensions onto a color image captured simultaneously, binarizing the image based on pixel RGB values ​​to determine the ripening area, and obtaining a second level of picking objects; The papaya picking object level of the real-time level chain table is determined based on the first picking object level and the second picking object level.

3. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 2, characterized in that: Also includes: Determining the suction cup operation time for each level of picking robotic arms based on the picking robotic arm identifiers of the picking robotic arms of each level; The suction cup operation time is used to determine the suction cup rotation time of the level picking robotic arm as a backup picking robotic arm.

4. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 1, characterized in that: The method of determining the master and standby picking robot arms adapted to the real-time grade linked list based on the grade of the papaya picking object includes: Based on the papaya picking object level and the picking robot arm identification of each level of picking robot arm, a candidate picking robot arm identification is obtained, and the picking robot arm of the level corresponding to the candidate picking robot arm identification is used as the main picking robot arm; A backup picking robotic arm is determined from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy.

5. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 4, characterized in that: in, The step of determining a backup picking robotic arm from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy includes: In an environment where the papaya picking object level is determined to be in the first level state, the second level picking robotic arm and the third level picking robotic arm are determined as the backup picking robotic arms; In an environment where the papaya picking object level is determined to be in the second level state, the third level picking robotic arm is determined as the first backup picking robotic arm, and the first level picking robotic arm is determined as the second backup picking robotic arm; In an environment where the papaya picking object level is determined to be in the third level state, the second level picking robot arm is determined as the backup picking robot arm.

6. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 5, characterized in that: in, The step of determining a backup picking robotic arm from other picking robotic arms based on the papaya picking object level and the backup picking robotic arm determination strategy includes: Obtaining the harvested papaya scheduling data of each level of harvesting robotic arms for the level of the papaya harvesting object; Determining an idle picking robot arm that has picked fruits from the other picking robot arms based on the picked scheduling data; The idle picking robotic arm that has already picked fruits is determined as the backup picking robotic arm.

7. The method for fully automatic ripe papaya identification and picking based on image recognition according to claim 1, characterized in that: in, The scheduling of candidate picking robots for the real-time ranking list based on the primary and backup picking robotic arms includes: In an environment where the picking robots included in the main picking robot arm meet the suction cup requirements of the real-time grade chain table, determining the picking robots in the main picking robot arm as the candidate picking robots; In an environment where the picking robots included in the main picking robotic arm do not meet the suction cup requirements of the real-time grade chain list, the picking robot in the main picking robotic arm is determined as the first candidate picking robot, and the picking robot in the backup picking robotic arm is determined as the second candidate picking robot; wherein, the first candidate picking robot and the second candidate picking robot together constitute the candidate picking robot.

8. The method for fully automatic ripe papaya identification and picking based on image recognition as claimed in claim 7, characterized in that: in, The step of determining the picking robot in the backup picking robot arm as the second candidate picking robot includes: In an environment where it is determined that there is only one backup picking robot arm, determining the picking robot in the backup picking robot arm as the second candidate picking robot; In an environment where it is determined that there are multiple backup picking robotic arms, a candidate backup picking robotic arm is determined from each of the backup picking robotic arms, and the picking robot in the candidate backup picking robotic arm is determined as the second candidate picking robot.

9. The method for fully automatic ripe papaya identification and picking based on image recognition according to claim 1, characterized in that: Also includes: In an environment where it is determined that the candidate picking robots include a picking robot with a backup picking robot arm, obtaining a suction cup operation time of the backup picking robot arm; In an environment where it is determined that the suction cup operation time reaches the suction cup operation time, or the picking robot of the main picking robot arm meets the suction cup requirement of the real-time grade chain table, the picking robot of the backup picking robot arm included in the candidate picking robots is rotated; The main picking robot arm and the backup picking robot arm are a positioning program pool, and the candidate picking robot is a positioning program suction cup.

10. A papaya full-automatic ripening identification and picking system based on image recognition, used to execute the papaya full-automatic ripening identification and picking method based on image recognition according to any one of claims 1 to 9, characterized in that: include: A six-axis robotic arm fixed on a chassis, a first-axis drive motor (3), a second-axis drive motor (4), a third-axis drive motor (5), a fourth-axis drive motor (6), a fifth-axis drive motor (7), a sixth-axis drive motor (8), and an actuator (9) at the end thereof, including a suction cup (11), a depth-of-field camera (12), a light sensor (13), an illuminating lamp (14), and the sixth-axis drive motor (8).

Images