An automated method and device for picking shiitake mushrooms

Abstract

The present invention discloses a method and device for automatic picking of shiitake mushrooms. The visual module is used to identify the shiitake mushrooms to be picked and obtain the three-dimensional information of the shiitake mushrooms to be picked; the three-dimensional information is compared with the preset picking standard, and the pickable shiitake mushrooms are screened out from the shiitake mushrooms to be picked according to the comparison result; according to the three-dimensional information of the pickable shiitake mushrooms, the clamping jaws are controlled to clamp and bend the mushroom stalks of the pickable shiitake mushrooms, and the pickable shiitake mushrooms are taken away from the mushroom stick substrate to complete a single picking; step S3 is executed cyclically until all the pickable shiitake mushrooms are picked. The present invention uses the visual module to automatically identify and obtain the three-dimensional information of shiitake mushrooms, so as to judge whether the size of shiitake mushrooms is suitable for picking. When picking, it bends and picks from the mushroom stalks of the pickable shiitake mushrooms, imitating the manual picking action, effectively solving problems such as mis-picking small shiitake mushrooms and damaging the shiitake mushroom body, making the picking process have an intelligent decision-making function, and realizing automatic identification of shiitake mushrooms and efficient and accurate picking of shiitake mushrooms.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent agriculture, and particularly relates to a method and device for automatic picking of shiitake mushrooms. Background Art

[0002] The ground cultivation mode is a cultivation method in a mushroom shed. First, mushroom farmers tidy up the ridge beds in the mushroom shed, and then arrange the shiitake mushroom sticks neatly in a certain direction on the ridge beds, which is called the mushroom stick ridge bed. The general shape of the mushroom stick ridge bed is a rectangular strip-shaped raised ridge bed, with a width between 1.5 and 1.8 m; the length is determined according to the actual mushroom shed and is usually more than 5 m; the mushroom stick ridge beds are usually separated by ditches with a width of 20 - 30 cm to maintain a certain humidity environment.

[0003] Due to the uneven depth and muddy softness of the isolation ditches, the process of picking shiitake mushrooms requires frequent walking. Moreover, the characteristics of shiitake mushroom fruiting are uneven batches, and the sizes of mushroom caps are different at the same moment. It is necessary to decide whether to pick or retain according to the harvesting standard, and it is impossible to harvest neatly at one time; and the entire fruiting cycle is long, about 5 months. The growth characteristics of shiitake mushrooms result in a huge labor volume for mushroom farmers to harvest shiitake mushrooms (the harvesting task needs to be carried out almost every day during the fruiting cycle), with a long duration and a heavy burden.

[0004] Currently, some companies and technicians have made some technical explorations on the picking link of shiitake mushrooms. For example, the patent with the patent number CN109463203A discloses an automatic integrated machine for shiitake mushroom planting and picking. However, it only elaborates on the mechanical operation process of shiitake mushroom collection and does not specifically describe how to achieve the automatic picking link of shiitake mushrooms; the patent with the patent number CN112889592B discloses a control system for a mushroom picking device, which only describes the conventional process of machine vision guiding the robotic arm to perform target grasping and does not specifically address the specific process of mushroom picking. In particular, it does not further study problems such as mis-picking small mushrooms and damaging the mushroom body, and it is difficult to truly achieve accurate automatic picking of shiitake mushrooms. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: to provide a method and device for automatic picking of shiitake mushrooms, which can automatically identify shiitake mushrooms and efficiently and accurately complete the picking.

[0006] To solve the above technical problem, the technical solution adopted by the present invention is:

[0007] A method for automatic picking of shiitake mushrooms includes the following steps:

[0008] S1. Identify the shiitake mushrooms to be picked through a vision module and obtain the three-dimensional information of the shiitake mushrooms to be picked;

[0009] S2. Compare the three-dimensional information with a preset picking standard, and screen out pickable shiitake mushrooms from the to-be-picked shiitake mushrooms according to the comparison result;

[0010] S3. Control the gripper to grip and bend the stalks of the pickable shiitake mushrooms according to the three-dimensional information of the pickable shiitake mushrooms, and take the pickable shiitake mushrooms away from the mushroom stick substrate to complete a single picking;

[0011] S4. Loop and execute step S3 until all the pickable shiitake mushrooms are picked.

[0012] To solve the above technical problems, another technical solution adopted by the present invention is:

[0013] An automatic shiitake mushroom picking device includes a mobile chassis, and a main control device, a robotic arm, and a shiitake mushroom storage component arranged on the mobile chassis;

[0014] The robotic arm is provided with a gripper, and the main control device is electrically connected to the mobile chassis, the robotic arm, the gripper, and the shiitake mushroom storage component respectively;

[0015] The main control device is used to execute the above-mentioned automatic shiitake mushroom picking method.

[0016] The beneficial effects of the present invention are as follows: It provides an automatic shiitake mushroom picking method and device, which uses a vision module to automatically identify shiitake mushrooms within the visual range, and then obtains the three-dimensional information of the shiitake mushrooms and compares it with a preset picking standard, so as to judge whether the size of the shiitake mushrooms is suitable for picking. When picking, it bends and picks from the stalks of the pickable shiitake mushrooms, imitating the manual picking action, effectively solving problems such as mis-picking small shiitake mushrooms and damaging the shiitake mushroom bodies, enabling the picking process to have an intelligent decision-making function, and realizing automatic identification of shiitake mushrooms and efficient and accurate picking of shiitake mushrooms. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a step schematic diagram of an automatic shiitake mushroom picking method of the present invention;

[0018] Figure 2 It is a step schematic diagram of a single picking of an automatic shiitake mushroom picking method of the present invention;

[0019] Figure 3 It is a schematic diagram of an automatic shiitake mushroom picking device of the present invention performing a picking operation on a mushroom stick substrate;

[0020] Figure 4 It is a schematic structural diagram of an automatic shiitake mushroom picking device of the present invention;

[0021] Figure 5 It is a system block diagram of an automatic shiitake mushroom picking device of the present invention.

[0022] Description of reference numerals:

[0023] 1. Mobile chassis; 2. Main control device; 3. Robot arm; 4. Gripper; 5. Temporary storage box; 6. Inversion and dumping mechanism; 7. Backup storage box; 8. Vision module; 9. Communication module; 10. Mushroom stick substrate; 11. Power module. Specific implementation mode

[0024] To describe in detail the technical content, achieved objectives and effects of the present invention, the following is described in conjunction with the implementation modes and with reference to the accompanying drawings.

[0025] Please refer to Figure 1 and Figure 2 , an automatic mushroom picking method, including the following steps:

[0026] S1. Identify the mushrooms to be picked through the vision module 8 and obtain the three-dimensional information of the mushrooms to be picked;

[0027] S2. Compare the three-dimensional information with the preset picking standards, and screen out the pickable mushrooms from the mushrooms to be picked according to the comparison results;

[0028] S3. Control the gripper 4 to grip and bend the mushroom stalks of the pickable mushrooms according to the three-dimensional information of the pickable mushrooms, and take the pickable mushrooms away from the mushroom stick substrate 10 to complete a single picking;

[0029] S4. Loop and execute the step S3 until all the pickable mushrooms are picked.

[0030] As can be seen from the above description, the beneficial effects of the present invention are as follows: An automatic mushroom picking method is provided, which uses the vision module 8 to automatically identify the mushrooms within the visual range, and then obtains the three-dimensional information of the mushrooms and compares it with the preset picking standards, so as to judge whether the size of the mushrooms is suitable for picking, etc. When picking, the mushrooms are bent and picked from the mushroom stalks of the pickable mushrooms, imitating the manual picking action, effectively solving problems such as mis-picking small mushrooms and damaging the mushroom bodies, making the picking process have an intelligent decision-making function, and realizing automatic identification of mushrooms and efficient and accurate picking of mushrooms.

[0031] Further, the step S3 specifically includes:

[0032] S31. According to the three-dimensional information of the pickable mushrooms, control the gripper 4 to move down from above the pickable mushrooms to the position corresponding to the mushroom stalks of the pickable mushrooms, and grip the mushroom stalks of the pickable mushrooms;

[0033] S32. Control the gripper 4 to tilt, so that the pickable mushrooms tilt on the mushroom stick substrate 10 at a preset angle;

[0034] S33. Control the gripper 4 to remove the pickable shiitake mushroom from the mushroom stick substrate 10.

[0035] As can be seen from the above description, during picking, the gripper 4 is controlled to grip the stalk of the shiitake mushroom, and the gripper 4 is tilted at a certain angle so that a certain-sized crack is formed between the stalk of the shiitake mushroom and the mushroom stick substrate 10, making it easier for the two to separate. Then, the gripper 4 is controlled to pick up the pickable shiitake mushroom; the entire picking process mimics the manual picking action, effectively improving the picking success rate and avoiding damage to the mushroom cap at the same time.

[0036] Further, after the step S31 and before the step S32, it also includes:

[0037] Obtain and judge whether the load value of the load detector on the gripper 4 is greater than the first preset value. If so, execute the step S32; otherwise, open the gripper 4, adjust the position of the gripper 4, close the gripper 4 again and judge whether the load value is greater than the first preset value;

[0038] Record and judge whether the number of times the gripper 4 is closed again is equal to the second preset value. If so, terminate this picking.

[0039] As can be seen from the above description, a load detector is set and the load value is inspected to judge whether the gripper 4 has successfully gripped the stalk of the shiitake mushroom. If the stalk of the shiitake mushroom is not clamped, the position of the gripper 4 is finely adjusted, the load value is obtained again and judged, so as to improve the clamping success rate. At the same time, the number of times the gripper 4 is closed again is limited to avoid the gripper 4 falling into a loop of continuously fine-tuning the position due to not clamping the stalk, and avoid wasting too much time in ineffective picking actions.

[0040] Further, the step S1 is specifically:

[0041] S11. Obtain the external environment image and depth map data through the vision module 8;

[0042] S12. Use a rectangular frame to enclose the to-be-picked shiitake mushroom in the external environment image, and use the upper left corner coordinates, width, and length of the rectangular frame corresponding to the to-be-picked shiitake mushroom as the first output data;

[0043] S13. Obtain the three-dimensional information of the corresponding to-be-picked shiitake mushroom according to the first output data and the depth map data.

[0044] As can be seen from the above description, the establishment of the three-dimensional information of the to-be-picked shiitake mushroom is determined based on the external environment image and depth map data obtained by the vision module 8. Specifically, the to-be-picked shiitake mushroom in the two-dimensional external environment image is first enclosed by a rectangular frame, and then combined with the depth map data to make the entire rectangular frame three-dimensional, so as to estimate the three-dimensional data of the shiitake mushroom.

[0045] Further, step S13 specifically includes:

[0046] S131. Obtain the center coordinates of the rectangular frame corresponding to the to-be-picked shiitake mushroom, the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side according to the first output data;

[0047] S132. According to the depth map data, convert the center coordinates, the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side from a plane into three-dimensional coordinates in a three-dimensional coordinate system;

[0048] S133. Calculate the shiitake mushroom diameter in the three-dimensional information of the to-be-picked shiitake mushroom according to the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side in the three-dimensional coordinate system. Its expression is:

[0049] D = (abs(Xw1 - Xw3) + abs(Yw2 - Yw4)) / 2;

[0050] Wherein, D represents the shiitake mushroom diameter, Xw1 and Xw3 respectively represent the X coordinates of the midpoints of the first side and the third side that are opposite sides in the horizontal direction of the rectangular frame, and Yw2 and Yw4 respectively represent the Y coordinates of the midpoints of the second side and the fourth side that are opposite sides in the vertical direction of the rectangular frame;

[0051] S134. Calculate the average depth value of the to-be-picked shiitake mushroom according to the depth map data, the shiitake mushroom diameter, and the center coordinates of each side in the three-dimensional coordinate system, and obtain the mushroom cap edge height value of the to-be-picked shiitake mushroom in the three-dimensional coordinate system;

[0052] S135. Calculate the flatness in the three-dimensional information of the to-be-picked shiitake mushroom according to the mushroom cap edge height value, the shiitake mushroom diameter, and the center coordinates in the three-dimensional coordinate system. Its expression is:

[0053] P = abs(Zw5 - Zw0) / D;

[0054] Wherein, P represents the flatness, Zw5 represents the mushroom cap edge height value, and Zw0 represents the Z coordinate of the center coordinates in the three-dimensional coordinate system.

[0055] As can be seen from the above description, using the center coordinates of the rectangular frame and the coordinates of the midpoints of the corresponding four sides, the position and size of the to-be-picked shiitake mushroom are roughly described. Furthermore, by combining the algorithm of converting a plane into a three-dimensional coordinate system, the estimation calculation of the shiitake mushroom diameter and flatness of the to-be-picked shiitake mushroom is realized, and the general shape of the shiitake mushroom is mastered, so as to facilitate the subsequent automatic judgment of whether it can be picked.

[0056] Furthermore, step S2 specifically includes:

[0057] S21. Determine whether the central coordinate of the to-be-picked shiitake mushroom in the three-dimensional coordinate system falls within the moving and clamping range of the clamping jaw 4. If so, execute step S22; otherwise, execute step S21 for another to-be-picked shiitake mushroom.

[0058] S22. Determine whether the diameter of the to-be-picked shiitake mushroom falls within the preset picking range. If so, regard the to-be-picked shiitake mushroom as the pickable shiitake mushroom; otherwise, execute step S23.

[0059] S23. Determine whether the flatness of the to-be-picked shiitake mushroom is less than or equal to the preset flatness. If so, regard the to-be-picked shiitake mushroom as the pickable shiitake mushroom; otherwise, execute step S21 for another to-be-picked shiitake mushroom.

[0060] As can be seen from the above description, during picking, according to the central coordinate of the to-be-picked shiitake mushroom in the three-dimensional coordinate system, it is judged whether the position of the to-be-picked shiitake mushroom can be clamped by the clamping jaw 4. Furthermore, according to the diameter and flatness of the shiitake mushroom, it is judged whether the to-be-picked shiitake mushroom is suitable for picking at present. By means of multiple judgment logics and reference data, the to-be-picked mushrooms are screened layer by layer, improving the picking efficiency and optimizing the quality of the picked shiitake mushrooms.

[0061] Furthermore, step S3 further includes:

[0062] Control the clamping jaw 4 to place the pickable shiitake mushroom into the temporary storage box 5.

[0063] As can be seen from the above description, by setting the temporary storage box 5 to store the picked shiitake mushrooms, the collection speed of the shiitake mushrooms in the same area can be effectively improved.

[0064] Furthermore, before step S1, it further includes:

[0065] S0. Determine whether the number of pickable shiitake mushrooms in the temporary storage box 5 is equal to the preset number. If so, pour all the pickable shiitake mushrooms in the temporary storage box 5 into the backup storage box 7; otherwise, execute step S1.

[0066] As can be seen from the above description, by using the temporary storage box 5 and the backup storage box 7 in cooperation, the picking process has both front-end collection caching and back-end storage, improving the convenience of the picking process.

[0067] Please refer to Figures 3 to 5 , an automatic shiitake mushroom picking device, including a mobile chassis 1, a main control device 2, a robotic arm 3, and a shiitake mushroom storage component provided on the mobile chassis 1;

[0068] A gripper 4 is provided on the robotic arm 3, and the main control device 2 is electrically connected to the mobile chassis 1, the robotic arm 3, the gripper 4, and the shiitake mushroom storage assembly respectively;

[0069] While controlling the mobile chassis 1 to move on the mushroom stick substrate 10, the main control device 2 is used to execute the above-mentioned automatic shiitake mushroom picking method.

[0070] As can be seen from the above description, the beneficial effect of the present invention is to provide an automatic shiitake mushroom picking device, which uses the mobile chassis 1 as the moving basis of the device. The main control device 2 controls the gripper 4 through the robotic arm 3 to pick shiitake mushrooms, and cooperates with the shiitake mushroom storage assembly for storing shiitake mushrooms, realizing automatic picking operations in the mushroom shed, improving the picking efficiency and reducing the labor cost.

[0071] Further, the shiitake mushroom storage assembly includes a temporary storage box 5, a tipping mechanism 6, and a backup storage box 7;

[0072] The tipping mechanism 6 and the backup storage box 7 are distributed on the mobile chassis 1, and the main control device 2 is electrically connected to the tipping mechanism 6;

[0073] The temporary storage box 5 is arranged on the tipping mechanism 6, and the tipping mechanism 6 is used to pour the shiitake mushrooms in the temporary storage box 5 into the backup storage box 7.

[0074] As can be seen from the above description, the use of the temporary storage box 5 and the backup storage box 7 in cooperation enables the picking process to have both front-end collection and caching and back-end storage, improving the convenience of the picking process.

[0075] Please refer to Figures 1 to 3 , the first embodiment of the present invention is:

[0076] An automatic shiitake mushroom picking method includes the following steps:

[0077] S0. Determine whether the number of pickable shiitake mushrooms in the temporary storage box 5 is equal to the preset number. If so, pour all the pickable shiitake mushrooms in the temporary storage box 5 into the backup storage box 7; otherwise, execute step S1.

[0078] In this embodiment, the automatic picking of shiitake mushrooms is mainly controlled by a machine to operate the gripper 4 to complete the picking. The shiitake mushrooms picked each time are placed in the temporary storage box 5; the number of pickable shiitake mushrooms in the temporary storage box 5 can be judged not only by the number of times the gripper 4 successfully picks up shiitake mushrooms, but also by data such as the total sum of the shiitake mushroom diameters of all the shiitake mushrooms picked within a preset time period.

[0079] S1. Identify the shiitake mushroom to be picked through the vision module 8 and obtain the three-dimensional information of the shiitake mushroom to be picked;

[0080] In this embodiment, step S1 is specifically as follows:

[0081] S11. Obtain the external environment image and depth map data through the vision module 8;

[0082] In this embodiment, the vision module 8 includes: a color image sensor for obtaining a two-dimensional color image of the external environment, that is, the external environment image; a depth image sensor for obtaining a three-dimensional point cloud map of the external environment, that is, the depth map data; the vision module 8 uses an image recognition algorithm, such as the YoloV5 algorithm, to identify each shiitake mushroom in the picture. During the recognition process, the number of shiitake mushrooms can be denoted as N. If N is 0, it means that there are no shiitake mushrooms within the acquisition range of the current vision module 8, and then the viewing angle needs to be re-converted to obtain the external environment image and depth map data and re-identify the shiitake mushrooms.

[0083] S12. Use a rectangular box to frame the shiitake mushrooms to be picked in the external environment image, and take the upper left corner coordinates, width, and length of the rectangular box corresponding to the shiitake mushrooms to be picked as the first output data;

[0084] The process of identifying the shiitake mushrooms with the above rectangular box can use traditional image processing techniques, such as OpenCV; or it can be artificial intelligence inference techniques, such as TensorFlow, PyTorch, Caffe, etc.

[0085] S13. Obtain the three-dimensional information of the corresponding shiitake mushrooms to be picked according to the first output data and the depth map data.

[0086] Among them, step S13 specifically includes:

[0087] S131. Obtain the center coordinates (Xc0, Yc0) of the rectangular box corresponding to the shiitake mushrooms to be picked, the midpoint coordinates (Xc1, Yc1) of the first side, the midpoint coordinates (Xc2, Yc2) of the second side, the midpoint coordinates (Xc3, Yc3) of the third side, and the midpoint coordinates (Xc4, Yc4) of the fourth side according to the first output data;

[0088] S132. According to the depth map data, convert the center coordinates, the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side from a plane to three-dimensional coordinates in a three-dimensional coordinate system, that is, obtain the center coordinates (Xw0, Yw0, Zw0), the midpoint coordinates of the first side (Xw1, Yw1, Zw1), the midpoint coordinates of the second side (Xw2, Yw2, Zw2), the midpoint coordinates of the third side (Xw3, Yw3, Zw3), and the midpoint coordinates of the fourth side (Xw4, Yw4, Zw4);

[0089] S133. Calculate the diameter of the mushroom to be picked in the three-dimensional information of the mushroom to be picked according to the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side in the three-dimensional coordinate system. The expression is as follows:

[0090] D = (abs(Xw1 - Xw3) + abs(Yw2 - Yw4)) / 2;

[0091] Where D represents the diameter of the mushroom, Xw1 and Xw3 respectively represent the X coordinates of the midpoints of the first side and the third side that are opposite sides in the horizontal direction of the rectangular frame, and Yw2 and Yw4 respectively represent the Y coordinates of the midpoints of the second side and the fourth side that are opposite sides in the vertical direction of the rectangular frame; abs is a calculation function that returns the absolute value.

[0092] S134. Calculate the average depth value of the mushroom to be picked according to the depth map data, the diameter of the mushroom, and the center coordinates in the three-dimensional coordinate system, and obtain the height value of the mushroom edge of the mushroom to be picked in the three-dimensional coordinate system;

[0093] S135. Calculate the flatness in the three-dimensional information of the mushroom to be picked according to the height value of the mushroom edge, the diameter of the mushroom, and the center coordinates in the three-dimensional coordinate system. The expression is as follows:

[0094] P = abs(Zw5 - Zw0) / D;

[0095] Where P represents the flatness, Zw5 represents the height value of the mushroom edge, and Zw0 represents the Z coordinate of the center coordinates in the three-dimensional coordinate system.

[0096] In this embodiment, the flatness can be represented by the ratio of the height and diameter of the mushroom to be picked, which represents the degree of umbrella opening of the mushroom. If the flatness is too small, it means that the mushroom has opened its umbrella, and the mushroom needs to be picked regardless of its diameter.

[0097] S2. Compare the three-dimensional information with the preset picking standard, and screen out the mushrooms that can be picked from the mushrooms to be picked according to the comparison result. The specific comparison process is as follows:

[0098] S21. Determine whether the center coordinates of the mushroom to be picked in the three-dimensional coordinate system fall within the moving clamping range of the clamping jaw 4. If so, execute step S22; otherwise, execute step S21 for another mushroom to be picked;

[0099] S22. Determine whether the diameter of the mushroom to be picked falls within the preset picking range. If so, regard the mushroom to be picked as a mushroom that can be picked; otherwise, execute step S23;

[0100] S23, determining whether the flatness of the mushrooms to be picked is less than or equal to a preset flatness, if so, treating the mushrooms to be picked as pickable mushrooms, otherwise executing step S21 for another mushroom to be picked.

[0101] In the above comparison process, the preset picking standards include the maximum picking diameter Dmax of 10 cm, the minimum picking diameter Dmin of 5 cm and the picking flatness Pmax of 0.3, as shown below:

[0102] The three-dimensional coordinates of the center point of the surface of the i-th mushroom to be picked in the robot arm 3 coordinate system are recorded as (Xwi, Ywi, Zwi), the diameter of the i-th mushroom to be picked is recorded as Di, and the flatness of the i-th mushroom to be picked, that is, the ratio of the height to the diameter of the mushroom to be picked, is recorded as Pi; wherein the range of i is 1≤i≤N, and i is an integer;

[0103] When selecting the mushrooms to be picked from the mushrooms to be picked, on the one hand, the mushrooms to be picked must satisfy the three-dimensional coordinates (Xwi, Ywi, Zwi) of the center point of the mushrooms to be picked within the working space of the gripper 4; on the other hand, the mushrooms to be picked must satisfy at least one of Dmin≤Di≤Dmax and Pi≤Pmax. The data of the mushrooms to be picked that do not meet the above conditions will be deleted after being rejected.

[0104] S3, according to the three-dimensional information of the pickable mushrooms, the gripper 4 is controlled to clamp and bend the mushroom feet of the pickable mushrooms, and the pickable mushrooms are taken away from the mushroom stick matrix 10, and the gripper 4 is controlled to place the pickable mushrooms in the temporary storage box 5, so as to complete a single picking;

[0105] In this embodiment, step S3 specifically includes:

[0106] S31, according to the three-dimensional information of the pickable mushrooms, control the clamping claw 4 to move downward from the upper side of the pickable mushrooms to the position corresponding to the mushroom feet of the pickable mushrooms, and clamp the mushroom feet of the pickable mushrooms;

[0107] When clamping the mushroom feet of the pickable mushrooms, obtain and determine whether the load value of the load detector on the clamping jaw 4 is greater than the first preset value. If so, execute step S32; otherwise, open the clamping jaw 4, adjust the position of the clamping jaw 4, reclose the clamping jaw 4 and determine whether the load value is greater than the first preset value; at the same time, record and determine whether the number of times the clamping jaw 4 is reclosed is equal to the second preset value. If so, terminate the picking.

[0108] The load value may be displayed in the form of a percentage, 100% means the clamp 4 is clamped, and the first preset value ranges from 25% to 35%, preferably 30%; the second preset value ranges from 3 to 5 times, preferably 3 times.

[0109] S32. Control the gripper 4 to tilt so that the pickable shiitake mushrooms tilt on the mushroom stick substrate 10 at a preset angle.

[0110] Among them, the pickable shiitake mushrooms tilt on the mushroom stick substrate 10 at a preset angle to create a certain gap with the mushroom stick substrate 10, facilitating the subsequent action of picking up the shiitake mushrooms. The value range of the preset angle is 30° - 40°, preferably 30°.

[0111] S33. Control the gripper 4 to take the pickable shiitake mushrooms away from the mushroom stick substrate 10.

[0112] S4. Loop and execute step S3 until all pickable shiitake mushrooms are picked.

[0113] In this embodiment, the picked shiitake mushrooms are those corresponding to the external environment image. Then, control the vision module 8 to obtain the image data of other positions of the mushroom stick substrate 10, so as to continue to identify and pick shiitake mushrooms until the entire automated picking operation is completed.

[0114] Please refer to Figures 3 to 5 , Embodiment 2 of the present invention is as follows:

[0115] An automated shiitake mushroom picking device includes a mobile chassis 1 and a main control device 2, a robotic arm 3, a power module 11, and a shiitake mushroom storage component arranged on the mobile chassis 1; a gripper 4 is provided on the robotic arm 3, and the main control device 2 is electrically connected to the mobile chassis 1, the robotic arm 3, the gripper 4, and the shiitake mushroom storage component respectively; while controlling the mobile chassis 1 to move on the mushroom stick substrate 10, the main control device 2 is used to execute an automated shiitake mushroom picking method according to Embodiment 1. The power module 11 can be selected as a lithium battery to provide long-term independent power supply for the entire device.

[0116] In this real-time example, the main control device 2 is preferably with AI processing capabilities, such as RisingMicro RK3568, NVIDIA Jetson TX2 NX, etc., which can effectively improve the image recognition speed and accuracy; the communication interface can be RS485, RS232, TTL serial port, USB, and Ethernet, etc.

[0117] The mobile chassis 1 includes: a chassis panel for carrying and connecting other components, which can be made of materials such as aluminum alloy or stainless steel by CNC machining; a traveling mechanism, one end of which is connected to the chassis panel and the other end is in contact with a fixed surface, enabling the chassis panel to have relative movement with the fixed surface, and can be a wheeled, caterpillar, or legged mechanism, etc.; a motor, the output shaft of the motor is connected to one end of the traveling mechanism through a transmission mechanism to make the traveling mechanism move and provide power for the movement of the entire mobile chassis 1.

[0118] When performing the picking operation, the main control device 2 controls the traveling mechanism to move or stop on the surface of the mushroom stick beds, avoiding the complex and unpredictable working environment in the field during travel, which provides sufficient possibilities for the automation and autonomy of the shiitake mushroom picking device; the main control device 2 obtains the operating status of the mobile chassis 1, such as whether it is moving or stopped, the degree of inclination, whether it is stuck, etc., which facilitates the main control device 2 to make decisions and protect the entire device.

[0119] In this embodiment, the end gripper 4 includes: a gripper 4 base, one end of which is connected to the robotic arm 3; at least one power source, one end of the power source is connected to the gripper 4 base, and the output end of the power source generates a rotational or translational motion; at least two fingers, one end of the fingers is connected to the output end of the power source, or one end of the fingers is connected to the gripper 4 base, and the other ends of all the fingers can approach or move away from each other; at least one load detector, used to obtain the contact degree between the end of the finger and the object to be gripped; a communication interface, connected to and communicating with the main control device 2, and can use methods such as RS485, RS232, and TTL serial ports.

[0120] Among them, the power source can be a serial servo, which incorporates a motor, a reducer, an encoder, and a current detector (equivalent to a load detector). The main control device 2 can control the output shaft of the serial servo to rotate to a specified angle through serial commands, and at the same time, feedback data of the load value can be obtained, which can conveniently control the opening and closing of the fingers and sense the clamping degree.

[0121] The main control device 2 is equipped with a vision module 8. The vision module 8 includes: a color image sensor, used to obtain a two-dimensional color image of the external environment; a depth image sensor, used to obtain a three-dimensional point cloud map of the external environment; a communication interface, connected to and communicating with the main control device 2; a fill light, used to improve the image quality of the color image sensor; the vision module 8 transmits external environment image and depth map data to the main control device 2. The vision module 8 can specifically include an RGBD camera, which can simultaneously transmit external environment image and depth map data to the main control device 2 through USB or Ethernet signals; since the light in the mushroom shed is insufficient, the fill light is necessary and can effectively improve the image quality of the external environment image. Usually, an LED fill light is used.

[0122] In this embodiment, the robotic arm 3 includes: a fixed base fixedly connected to the chassis panel of the mobile chassis 1; at least three moving axis rods serially connected in sequence from the fixed base, and the moving axis rod at the outermost end is called the end axis; a power module, where each moving axis rod is individually driven by at least one power module; a communication interface connected and communicating with the central controller, which can use methods such as RS485, RS232, CAN bus, and Ethernet; the end of the end axis is fixedly connected to the jaw base of the end jaw 4; among them, the robotic arm 3 can specifically be a SCARA robotic arm 3, a six-axis robotic arm 3, etc., as long as it is within the required moving and gripping range and can move the jaw 4 at the end of the robotic arm 3 from one point to another according to the instruction requirements.

[0123] In addition, the automated picking device also has a remote control function, connecting the main control device 2 and external devices through the communication module 9, and sending control instructions from the external devices to the main control device 2. One end of the communication module 9 and the main control device 2 can be connected by means such as USB, serial port, and Ethernet, while the end connected to the external device can be connected by one or more of communication methods such as 4G, WIFI, Bluetooth, serial port, and Ethernet.

[0124] In this embodiment, the mushroom storage component includes a temporary storage box 5, a tipping mechanism 6, and a backup storage box 7; the tipping mechanism 6 and the backup storage box 7 are distributed on the mobile chassis 1, and the main control device 2 is electrically connected to the tipping mechanism 6; the temporary storage box 5 is arranged on the tipping mechanism 6, and the tipping mechanism 6 is used to pour the mushrooms in the temporary storage box 5 into the backup storage box 7.

[0125] In summary, the mushroom automated picking method and device provided by the present invention use the vision module to automatically identify the mushrooms within the visual range, and then obtain the three-dimensional information of the mushrooms and compare it with the preset picking standards to judge whether the size of the mushrooms is suitable for picking. When picking, it bends and picks from the mushroom stalks of the pickable mushrooms, imitating the manual picking action, effectively solving problems such as mis-picking small mushrooms and damaging the mushroom body, enabling the picking process to have an intelligent decision-making function, and realizing automatic identification of mushrooms and efficient and accurate picking of mushrooms.

[0126] Moreover, an automated mushroom picking device can directly move on the mushroom stick ridges, enabling the device to have the ability of autonomous cruising, greatly reducing the degree of manual intervention; due to continuous cruising, the size of the picked mushrooms can be more uniform, improving the quality of mushrooms and increasing the income of mushroom farmers; combined with the aforementioned method, it can independently complete the tasks of identification, picking, and collection, greatly liberating the productivity; using a temporary storage box to collect mushrooms can effectively shorten the movement trajectory length of the robotic arm, save time, and improve work efficiency; after being filled, it will automatically pour the mushrooms, reducing the degree of manual intervention.

[0127] The above are only embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in the relevant technical fields, shall similarly be included in the patent protection scope of the present invention.

Claims

1. An automated method for picking shiitake mushrooms, characterized in that, It includes the following steps: S11. Obtain the external environment image and depth map data through the vision module; S12. Use a rectangular box to frame the shiitake mushrooms to be picked in the external environment image, and use the upper left corner coordinates, width, and length of the rectangular box corresponding to the shiitake mushrooms to be picked as the first output data; S13. Obtain the three-dimensional information of the corresponding shiitake mushrooms to be picked according to the first output data and the depth map data; The specific steps of step S13 include: S131. Obtain the center coordinates of the rectangular box corresponding to the shiitake mushrooms to be picked, the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side according to the first output data; S132. According to the depth map data, convert the center coordinates, the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side from a plane into three-dimensional coordinates in a three-dimensional coordinate system; S133. Calculate the shiitake mushroom diameter in the three-dimensional information of the shiitake mushrooms to be picked according to the midpoint coordinates of the first side, the midpoint coordinates of the second side, the midpoint coordinates of the third side, and the midpoint coordinates of the fourth side in the three-dimensional coordinate system; S134. Calculate the average depth value of the shiitake mushrooms to be picked according to the depth map data, the shiitake mushroom diameter, and the center coordinates in the three-dimensional coordinate system, and obtain the mushroom cap edge height value of the shiitake mushrooms to be picked in the three-dimensional coordinate system; S135. Calculate the flatness in the three-dimensional information of the shiitake mushrooms to be picked according to the mushroom cap edge height value, the shiitake mushroom diameter, and the center coordinates in the three-dimensional coordinate system; S2. Compare the three-dimensional information with the preset picking standard, and screen out the shiitake mushrooms that can be picked from the shiitake mushrooms to be picked according to the comparison result; S31. According to the three-dimensional information of the shiitake mushrooms that can be picked, control the gripper to move down from above the shiitake mushrooms that can be picked to the position corresponding to the mushroom stalks of the shiitake mushrooms that can be picked, and grip the mushroom stalks of the shiitake mushrooms that can be picked; S32. Control the gripper to tilt so that the shiitake mushrooms that can be picked are tilted at a preset angle on the mushroom stick substrate; S33. Control the gripper to take the shiitake mushrooms that can be picked away from the mushroom stick substrate to complete a single picking; Before step S32 and after step S31, it also includes: Obtain and judge whether the load value of the load detector on the gripper is greater than a first preset value. If so, execute step S32; otherwise, open the gripper, adjust the position of the gripper, close the gripper again, and judge whether the load value is greater than the first preset value; Record and judge whether the number of times the gripper is closed again is equal to a second preset value. If so, terminate this picking; S4. Loop through steps S31 to S33 until all the shiitake mushrooms that can be picked are picked.

2. The automated lentinula edodes picking method according to claim 1, characterized in that The expression for calculating the shiitake mushroom diameter in the three-dimensional information of the shiitake mushrooms to be picked is: D=(abs(Xw1-Xw3)+abs(Yw2-Yw4)) / 2; Where D represents the diameter of the shiitake mushroom, Xw1 and Xw3 respectively represent the X coordinates of the midpoints of the first side and the third side that are opposite to each other in the horizontal direction of the rectangular frame, and Yw2 and Yw4 respectively represent the Y coordinates of the midpoints of the second side and the fourth side that are opposite to each other in the vertical direction of the rectangular frame; The expression for calculating the flatness in the three-dimensional information of the shiitake mushroom to be picked is: P = abs(Zw5 - Zw0) / D; Where P represents the flatness, Zw5 represents the edge height value of the mushroom cap, and Zw0 represents the Z coordinate of the central coordinate in the three-dimensional coordinate system.

3. The automated shiitake mushroom picking method according to claim 2, characterized in that, The specific steps of step S2 include: S21. Determine whether the central coordinate of the shiitake mushroom to be picked in the three-dimensional coordinate system falls within the moving and clamping range of the gripper. If so, execute step S22; otherwise, execute step S21 for another shiitake mushroom to be picked. S22. Determine whether the diameter of the shiitake mushroom to be picked falls within the preset picking range. If so, regard the shiitake mushroom to be picked as the pickable shiitake mushroom; otherwise, execute step S23. S23. Determine whether the flatness of the shiitake mushroom to be picked is less than or equal to the preset flatness. If so, regard the shiitake mushroom to be picked as the pickable shiitake mushroom; otherwise, execute step S21 for another shiitake mushroom to be picked.

4. The automated shiitake mushroom picking method according to claim 1, characterized in that, Step S33 further includes: Control the gripper to place the pickable shiitake mushroom in the temporary storage box.

5. The automated shiitake mushroom picking method according to claim 4, characterized in that, Before step S1, it further includes: S0. Determine whether the number of pickable shiitake mushrooms in the temporary storage box is equal to the preset number. If so, pour all the pickable shiitake mushrooms in the temporary storage box into the backup storage box; otherwise, execute step S1.

6. An automatic mushroom picking device, characterized in that, It includes a mobile chassis and a main control device, a robotic arm, and a shiitake mushroom storage component arranged on the mobile chassis; The robotic arm is provided with a gripper, and the main control device is electrically connected to the mobile chassis, the robotic arm, the gripper, and the shiitake mushroom storage component respectively; The main control device controls the mobile chassis to move on the mushroom stick substrate and is used to execute any one of the methods for automatic picking of shiitake mushrooms according to claims 1 to 5.

7. The automatic mushroom picking device according to claim 6, characterized in that, The shiitake mushroom storage component includes a temporary storage box, a tipping mechanism, and a backup storage box; The tipping mechanism and the backup storage box are respectively arranged on the mobile chassis, and the main control device is electrically connected to the tipping mechanism; The temporary storage box is arranged on the tipping mechanism, and the tipping mechanism is used to pour the shiitake mushrooms in the temporary storage box into the backup storage box.

Images