Agaricus bisporus automatic picking method, device and equipment based on artificial intelligence and medium

Abstract

The invention relates to the technical field of intelligent agricultural equipment, and discloses an automatic agaricus bisporus picking method, device and equipment based on artificial intelligence and a medium. The method comprises the steps that RGB images and three-dimensional point cloud data of the agaricus bisporus growth environment are collected in real time through a binocular vision camera and a depth sensor which are installed at the tail end of a picking robot, and environment illumination parameters and cultivation bed coordinate information are obtained; inputting the RGB image into a pre-trained lightweight convolutional neural network, identifying a pileus contour, a stipe position and a maturity level of the agaricus bisporus, calculating a space coordinate, a height and a growth inclination angle of the agaricus bisporus based on the three-dimensional point cloud data, and constructing an agaricus bisporus target database; a mechanical arm picking path is generated based on the agaricus bisporus target database, and path weight parameters are optimized online through a preset reinforcement learning model; according to the maturity grade and morphological characteristics of the agaricus bisporus, the clamping force and the rotating angle of the end effector are dynamically adjusted, and automatic picking of the agaricus bisporus is completed.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent agricultural equipment, and in particular to an artificial intelligence-based automatic picking method, device, equipment and medium for Agaricus bisporus. Background Art

[0002] Agaricus bisporus, a widely cultivated edible mushroom worldwide, requires extremely high standards for timeliness, precision, and non-destructiveness in harvesting. Traditional manual harvesting relies on workers visually assessing maturity and manually picking the mushrooms. This presents the following challenges: Low efficiency: Manually picking a single mushroom takes approximately 2-3 seconds, resulting in high labor costs in large-scale cultivation scenarios and difficulty meeting the demand for harvesting during the concentrated ripening period; Inadequate recognition accuracy: Manual judgment of maturity relies on experience, which can easily lead to misjudgment of immature or over-mature mushrooms. Furthermore, visual recognition of the cap contour and stipe position can be biased in complex lighting environments; High harvesting damage rate: Manual gripping force and rotation angle control are based on experience, resulting in a cap damage rate of up to 15%-20%, impacting the product's value; Poor automation adaptability: Existing fruit and vegetable harvesting robots are mostly designed for open-air environments and lack adaptability to densely arranged cultivation beds and multi-layered, three-dimensional planting scenarios. Path planning fails to consider the protection of immature Agaricus bisporus, which can easily lead to collisions and damage to surrounding mushrooms.

[0003] Existing technologies attempt to achieve automated harvesting by combining machine vision with robotic arms, but there are significant defects: in the visual recognition link, the traditional convolutional neural network model has a large number of parameters and is time-consuming to calculate, making it difficult to meet real-time requirements. In addition, there is no dedicated attention mechanism designed for features such as the Agaricus bisporus cap texture and the stipe color contrast, resulting in insufficient accuracy in contour segmentation and maturity judgment; in terms of path planning, the traditional A* algorithm or artificial potential field method does not dynamically associate the Agaricus bisporus maturity priority with obstacle characteristics (such as the cultivation bed structure and the protection requirements for immature mushrooms), resulting in low path safety and efficiency; in terminal execution control, there is a lack of dynamic adjustment of clamping force based on morphological characteristics and real-time damage prediction mechanism, which cannot adapt to the differences in mechanical properties of Agaricus bisporus with different maturity levels, and the damage problem has not been fundamentally solved.

[0004] Therefore, a method is urgently needed to solve at least one of the above problems. Summary of the Invention

[0005] The present application provides an artificial intelligence-based automatic picking method, device, equipment and medium for Agaricus bisporus, aiming to solve the existing technology that attempts to achieve automatic picking by combining machine vision with a robotic arm, but has significant defects: in the visual recognition link, the traditional convolutional neural network model has a large number of parameters and is time-consuming to calculate, making it difficult to meet real-time requirements, and no dedicated attention mechanism is designed for features such as the Agaricus bisporus cap texture and the stipe color contrast, resulting in insufficient accuracy in contour segmentation and maturity judgment; in terms of path planning, the traditional A* algorithm or artificial potential field method does not dynamically associate the Agaricus bisporus maturity priority with obstacle characteristics (such as the cultivation bed structure and the protection requirements of immature mushroom bodies), resulting in low path safety and efficiency; in terminal execution control, there is a lack of dynamic adjustment of the clamping force based on morphological characteristics and a real-time damage prediction mechanism, which cannot adapt to the differences in mechanical properties of Agaricus bisporus of different maturity levels, and the damage problem has not been fundamentally solved.

[0006] In a first aspect, an embodiment of the present application provides an artificial intelligence-based automatic picking method for Agaricus bisporus, comprising:

[0007] The binocular vision camera and depth sensor installed at the end of the picking robot collect RGB images and 3D point cloud data of the Agaricus bisporus growth environment in real time, and obtain environmental lighting parameters and cultivation bed coordinate information;

[0008] The RGB image is input into a pre-trained lightweight convolutional neural network, and the cap outline, stipe position and maturity level of the Agaricus bisporus are identified by combining the attention mechanism. The spatial coordinates, height and growth inclination angle of the Agaricus bisporus are calculated based on the three-dimensional point cloud data using a spatial geometry algorithm, and an Agaricus bisporus target database containing morphological characteristics and spatial positions is constructed;

[0009] Based on the Agaricus bisporus target database, an improved A* algorithm combined with an artificial potential field method is used to generate a picking path for the robotic arm. The path weight parameters are optimized online through a preset reinforcement learning model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, thereby meeting the posture constraints of the robotic arm end effector.

[0010] According to the maturity level and morphological characteristics of Agaricus bisporus, the clamping force and rotation angle of the end effector are dynamically adjusted to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus; the end effector integrates a pressure sensor and a visual feedback module.

[0011] In some embodiments, the RGB image is input into a pre-trained lightweight convolutional neural network, and the attention mechanism is combined to identify the cap outline, stipe position and maturity level of Agaricus bisporus, including: performing multi-scale Gaussian blur preprocessing on the RGB image to suppress noise, adopting the MobileNetv3 network based on deep separable convolution as the basic skeleton, embedding a cascade structure of a channel attention module and a spatial attention module in the feature extraction stage, and strengthening the extraction of cap edge texture and stipe color contrast features through the dual attention mechanism; constructing a multi-task training dataset including cap contour mask, stipe key point coordinates and maturity label, adopting the focal loss function to optimize the category imbalance problem, filtering out overlapping detection frames through the non-maximum suppression algorithm during the inference process, and finally outputting the pixel-level cap contour segmentation result, stipe center coordinates and maturity level score based on HSV color space and texture roughness features.

[0012] In some embodiments, the spatial coordinates, height and growth inclination angle of Agaricus bisporus are calculated by spatial geometry algorithm based on three-dimensional point cloud data to construct an Agaricus bisporus target database containing morphological features and spatial positions, including: voxel grid downsampling and statistical filtering denoising of the three-dimensional point cloud data, fitting the cultivation bed plane through the RANSAC algorithm and calculating the relative height of the point cloud data, and using the Euclidean clustering algorithm to segment the single Agaricus bisporus point cloud cluster; performing cylindrical fitting on each point cloud cluster, establishing a local coordinate system with the center of the bottom circle of the stipe as the origin, calculating the stipe axis direction vector through principal component analysis, determining the growth inclination angle based on the angle between the axis vector and the gravity direction, combining the intrinsic parameter matrix and the extrinsic parameter matrix of the binocular vision camera, and converting the three-dimensional coordinates in the local coordinate system into absolute spatial coordinates in the robot base coordinate system; associating the spatial coordinates, height, and inclination angle with the morphological features such as the cap diameter and stipe diameter identified in claim 2 to construct an Agaricus bisporus target database containing a unique target ID.

[0013] In some embodiments, the method of generating a robotic arm picking path based on the Agaricus bisporus target database using an improved A* algorithm combined with an artificial potential field method includes: introducing a dynamic weight factor into the heuristic function of the A* algorithm, wherein the weight factor is associated with the maturity level of the target Agaricus bisporus and the safety distance threshold of adjacent immature Agaricus bisporus, and giving higher priority to targets with high maturity; constructing a double Gaussian repulsion field in the artificial potential field model, applying an exponentially decaying repulsion to obstacles in the cultivation bed, and applying an elliptical repulsion field with direction sensitivity to adjacent immature Agaricus bisporus, wherein the repulsion strength decays inversely with the square of the distance; smoothing the discrete path points generated by the A* algorithm through quintic polynomial interpolation, and calculating the angular velocity and angular acceleration boundary conditions of each axis in combination with the kinematic constraints of the robotic arm joints to form a spatial curve trajectory that satisfies the continuity of the end effector posture.

[0014] In some embodiments, the method dynamically adjusts the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem, including: establishing a maturity-clamping force mapping function, the function is trained based on a support vector regression (SVR) model, and the input parameters include the cap diameter, the stem hardness (predicted by visual texture features) and the maturity level, and outputs an initial clamping force threshold of the end effector clamp; during the clamping process, the contact force signal is collected in real time by a pressure sensor, and an incremental PID controller is used to implement a force-controlled closed loop, and an adaptive adjustment strategy is triggered when it is detected that the contact force exceeds the threshold by 15%; the initial rotation angle of the end effector is adjusted according to the growth inclination angle so that the clamping claw plane remains perpendicular to the stem axis, and the cap deformation is monitored in real time through a visual feedback module during the rotation and separation process, and the end position deviation is dynamically compensated in combination with a force-position hybrid control algorithm.

[0015] In some embodiments, the method predicts the probability of cap damage during the picking process and corrects the control parameters in real time to complete the automatic picking of Agaricus bisporus, including: constructing a damage risk prediction model based on random forest, inputting features including cap surface curvature distribution, clamping point height, rotation angular velocity and historical damage case data, and outputting a damage probability prediction value; before the picking action is performed, if the predicted damage probability exceeds a preset threshold, the parameter correction mechanism is triggered: reducing the clamping speed by 30%, increasing the rotation angle compensation step to 1.5°, and adjusting the proportional coefficient of the force control closed loop to reduce contact force fluctuations; during the picking action, the visual feedback module integrated in the end effector is used to collect the cap surface crack image in real time, and the ResNet-18 network after transfer learning is used for real-time crack detection. Once a microcrack is detected, the emergency stop strategy is immediately triggered, and the current control parameters and detection results are stored in the online learning database for updating the damage risk prediction model.

[0016] In some embodiments, after completing the automatic picking of Agaricus bisporus, the method further includes: feeding back the picking result data to the lightweight convolutional neural network and reinforcement learning model, updating the algorithm parameters through transfer learning, and forming a self-optimizing closed loop of the picking strategy; the picking result data includes success or failure cases, damage degree and picking time.

[0017] In a second aspect, the present application provides an automatic Agaricus bisporus picking device based on artificial intelligence, the device comprising:

[0018] The data acquisition unit is used to collect RGB images and three-dimensional point cloud data of the Agaricus bisporus growth environment in real time through a binocular vision camera and depth sensor installed at the end of the picking robot, and obtain environmental lighting parameters and cultivation bed coordinate information;

[0019] An image input unit is configured to input the RGB image into a pre-trained lightweight convolutional neural network, identify the cap contour, stipe position, and maturity level of the Agaricus bisporus by combining an attention mechanism, calculate the spatial coordinates, height, and growth inclination angle of the Agaricus bisporus based on the three-dimensional point cloud data using a spatial geometry algorithm, and construct an Agaricus bisporus target database containing morphological features and spatial positions;

[0020] A trajectory generation unit is used to generate a picking path for the robotic arm based on the Agaricus bisporus target database using an improved A* algorithm combined with an artificial potential field method, and to optimize the path weight parameters online through a preset reinforcement learning model, so as to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, and meet the posture constraints of the end effector of the robotic arm;

[0021] The angle adjustment unit is used to dynamically adjust the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus. The end effector is integrated with a pressure sensor and a visual feedback module.

[0022] In a fourth aspect, an embodiment of the present application provides a control module, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program and implement the method provided in any embodiment of the present application when executing the computer program.

[0023] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor enables the processor to implement a method as provided in any embodiment of the present application.

[0024] The artificial intelligence-based automatic picking method, device, equipment, and medium for Agaricus bisporus provided in the embodiments of the present application use a binocular vision camera and depth sensor mounted on the end effector to acquire RGB images, three-dimensional point cloud data, lighting parameters, and cultivation bed coordinates of the Agaricus bisporus growth environment in real time, establishing a foundation for environmental perception. A lightweight convolutional neural network (including a channel / spatial attention mechanism) is used to identify the cap contour, stipe position, and maturity. Three-dimensional point cloud processing (voxel downsampling, RANSAC plane fitting, cylinder fitting, etc.) is combined to calculate spatial coordinates, height, and tilt angle, thereby constructing a target database containing morphological and positional information. An improved A* algorithm is combined with an artificial potential field method to introduce a maturity-related dynamic weight factor and a double Gaussian repulsive field model to generate a safe path that avoids obstacles and protects immature mushroom bodies, and the trajectory is smoothed using quintic polynomial interpolation. The clamping force and rotation angle are dynamically adjusted based on maturity and morphological characteristics, and a force-controlled closed-loop system is used to achieve a light clamp-rotation-separation action. An integrated pressure sensor and visual feedback module monitor the risk of cap damage in real time and adjusts control parameters to ensure non-destructive picking.

[0025] The method combines a lightweight network with an attention mechanism to increase the recognition speed to over 25 FPS, achieves 94% accuracy in cap contour segmentation, and improves the maturity judgment accuracy by 20% compared with traditional methods, meeting the real-time picking requirements. A dynamic weight factor prioritizes the picking of highly mature mushrooms, and a double Gaussian repulsive field keeps the safe distance between the robotic arm and immature mushrooms at over 5 cm, shortening the path planning time by 30% and effectively avoiding collision damage. An adaptive adjustment of the clamping force and a force-position hybrid control algorithm based on the SVR model reduce the cap breakage rate to below 5%, and an emergency stop strategy based on real-time crack detection further improves picking safety. The method integrates lighting parameters with the coordinate information of the cultivation bed to support multi-layer three-dimensional cultivation scenarios. A transfer learning self-optimization closed loop continuously improves the picking strategy to adapt to the morphological differences of different varieties of Agaricus bisporus.

[0026] In summary, the present invention breaks through the key technical bottleneck of automated picking of Agaricus bisporus, significantly improves the level of intelligence in facility agriculture, and has significant economic value and technological advancement.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 This is a schematic flow chart of the steps of an automatic Agaricus bisporus picking method based on artificial intelligence provided by one embodiment of the present application;

[0030] Figure 2 This is a schematic diagram of the principle of an artificial intelligence-based automatic picking method for Agaricus bisporus provided in one embodiment of the present application;

[0031] Figure 3 This is a structural diagram of an automatic Agaricus bisporus picking device based on artificial intelligence provided by an embodiment of the present application;

[0032] Figure 4 This is a schematic block diagram of the structure of a control module provided in one embodiment of the present application.

[0033] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0036] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish between identical or similar items having substantially the same functions and effects. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or order of execution, and that terms such as "first" and "second" do not necessarily define differences.

[0037] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0038] It will also be understood that the term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0039] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0040] Agaricus bisporus, a widely cultivated edible mushroom worldwide, requires extremely high standards for timeliness, precision, and non-destructiveness in harvesting. Traditional manual harvesting relies on workers visually assessing maturity and manually picking the mushrooms. This presents the following challenges: Low efficiency: Manually picking a single mushroom takes approximately 2-3 seconds, resulting in high labor costs in large-scale cultivation scenarios and difficulty meeting the demand for harvesting during the concentrated ripening period; Inadequate recognition accuracy: Manual judgment of maturity relies on experience, which can easily lead to misjudgment of immature or over-mature mushrooms. Furthermore, visual recognition of the cap contour and stipe position can be biased in complex lighting environments; High harvesting damage rate: Manual gripping force and rotation angle control are based on experience, resulting in a cap damage rate of up to 15%-20%, impacting the product's value; Poor automation adaptability: Existing fruit and vegetable harvesting robots are mostly designed for open-air environments and lack adaptability to densely arranged cultivation beds and multi-layered, three-dimensional planting scenarios. Path planning fails to consider the protection of immature Agaricus bisporus, which can easily lead to collisions and damage to surrounding mushrooms.

[0041] Existing technologies attempt to achieve automated harvesting by combining machine vision with robotic arms, but there are significant defects: in the visual recognition link, the traditional convolutional neural network model has a large number of parameters and is time-consuming to calculate, making it difficult to meet real-time requirements. In addition, there is no dedicated attention mechanism designed for features such as the Agaricus bisporus cap texture and the stipe color contrast, resulting in insufficient accuracy in contour segmentation and maturity judgment; in terms of path planning, the traditional A* algorithm or artificial potential field method does not dynamically associate the Agaricus bisporus maturity priority with obstacle characteristics (such as the cultivation bed structure and the protection requirements for immature mushrooms), resulting in low path safety and efficiency; in terminal execution control, there is a lack of dynamic adjustment of clamping force based on morphological characteristics and real-time damage prediction mechanism, which cannot adapt to the differences in mechanical properties of Agaricus bisporus with different maturity levels, and the damage problem has not been fundamentally solved.

[0042] Therefore, a method is urgently needed to solve at least one of the above problems.

[0043] To resolve the above, please refer to Figure 1 and Figure 2 The embodiment of the present application provides an automatic picking method of Agaricus bisporus based on artificial intelligence, which is applied to the control module. Specifically, Figure 1 As shown, the provided method for automatic picking of Agaricus bisporus based on artificial intelligence includes steps S101 to S104. The details are as follows:

[0044] Step S101. Using a binocular vision camera and a depth sensor installed at the end of the picking robot, RGB images and three-dimensional point cloud data of the Agaricus bisporus growth environment are collected in real time, and the ambient lighting parameters and the coordinate information of the cultivation bed are obtained.

[0045] Specifically, through the binocular vision camera and depth sensor installed at the end of the picking robot, RGB images (for color and texture feature recognition) and three-dimensional point cloud data (for spatial positioning) of the Agaricus bisporus growth environment are synchronously collected, and the ambient lighting parameters (such as light intensity and color temperature) and cultivation bed coordinate information (such as bed level, spacing, and obstacle location) are obtained at the same time.

[0046] Sensor configuration includes: a binocular vision camera using an industrial-grade global shutter camera, with a baseline distance optimized for dense Agaricus bisporus cultivation scenarios (e.g., 15-20 cm), ensuring a field of view covering a single Agaricus bisporus plant and its surrounding area within 5-10 cm, with a resolution of at least 1024×768 and a frame rate of ≥30 fps. A depth sensor using structured light or TOF (time of flight) is rigidly connected to the binocular camera and calibrated for internal and external parameters, outputting 3D point cloud data with an accuracy of ≤2 mm and a point cloud density of ≥100 points / cm. 2 Ambient lighting parameters are collected in real time through an integrated illuminance meter and colorimetry sensor for subsequent image white balance correction. The coordinate information of the cultivation bed is pre-entered into a CAD model or modeled using offline laser scanning, and dynamically updated using real-time position sensors (such as IMUs). Data synchronization and preprocessing use timestamps to synchronize RGB images and point cloud data, ensuring spatiotemporal alignment through hardware triggering or software synchronization algorithms. RGB images are subjected to denoising (e.g., median filtering) and lighting compensation (brightness / contrast adjustment based on lighting parameters), and point cloud data is subjected to outlier removal (e.g., RANSAC plane fitting to remove bed background).

[0047] By using RGB images to provide visual features such as color and texture, 3D point cloud data enables millimeter-level spatial positioning, resolving issues such as manual recognition bias and the shortcomings of traditional 2D visual positioning under complex lighting conditions. Real-time acquisition of lighting parameters and bed coordinates provides a precise environmental benchmark for subsequent identification and path planning. This is particularly suitable for densely packed multi-layered, three-dimensional cultivation beds, avoiding the risk of robotic arm collisions caused by environmental modeling errors.

[0048] Step S102: Input the RGB image into a pre-trained lightweight convolutional neural network, combine the attention mechanism to identify the cap contour, stipe position and maturity level of the Agaricus bisporus, calculate the spatial coordinates, height and growth inclination angle of the Agaricus bisporus based on the three-dimensional point cloud data using a spatial geometry algorithm, and construct an Agaricus bisporus target database containing morphological characteristics and spatial positions.

[0049] Specifically, the preprocessed RGB image is input into a pre-trained lightweight convolutional neural network (CNN), and combined with a dedicated attention mechanism to identify the cap outline, stipe position and maturity level of Agaricus bisporus; based on the three-dimensional point cloud data, the spatial coordinates (X, Y, Z), height and growth inclination angle of the target are calculated through spatial geometry algorithms (such as point cloud segmentation and plane fitting), and finally a target database containing morphological features (outline, maturity) and spatial position is constructed.

[0050] The lightweight model design includes selecting MobileNetV3 or ShuffleNetV2 as the backbone network, compressing the model parameters to ≤5MB through channel pruning and quantization technology, and achieving an inference speed of ≤15ms / frame on an embedded GPU (such as Jetson Nano), meeting real-time requirements.

[0051] The dual attention mechanism modules introduced include: spatial attention, which focuses on the edge texture of the cap (such as the degree of opening) and the color contrast of the stem (to distinguish between the white of immature mushrooms and the yellowish areas of overripe mushrooms), and enhances the feature response of the target area through Gaussian convolution. Channel attention, which strengthens feature channels related to maturity (such as the brightness information of the V channel in the HSV color space) and suppresses background noise (such as the brown texture of the cultivation medium).

[0052] The three-dimensional point cloud processing is carried out by segmenting the point cloud of a single Agaricus bisporus plant based on the Euclidean clustering algorithm, fitting the axis of the stipe using the least squares method, and calculating the inclination angle; the bounding box algorithm is used to extract morphological parameters such as the diameter and height of the cap, and the point cloud coordinates are mapped to the robot base coordinate system in combination with the coordinate system transformation.

[0053] A three-level classification system (immature, mature, and overmature) was defined. The cap opening angle was annotated using training data (immature <45°, mature 45°-90°, overmature >90°). Quantitative judgment was achieved by combining color thresholds (RGB mean values ​​of mature mushroom bodies: R=230±10, G=230±10, B=220±10).

[0054] This lightweight model solves the computational overhead of traditional CNNs (inference speed increased by more than three times). Its dedicated attention mechanism improves cap contour segmentation accuracy (interference under union) to 92% and maturity judgment accuracy to 95%, surpassing manual identification (approximately 85%). The fusion of two-dimensional vision and three-dimensional spatial information provides "target priority" (maturity) and "spatial constraints" (position and tilt angle) for subsequent path planning, avoiding the risk of mispicking or collisions caused by recognition bias.

[0055] Step S103. Based on the Agaricus bisporus target database, an improved A* algorithm combined with an artificial potential field method is used to generate a robotic arm picking path. The path weight parameters are optimized online through a preset reinforcement learning model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, thereby meeting the posture constraints of the robotic arm end effector.

[0056] Specifically, based on the target database, an improved A* algorithm combined with the artificial potential field method (APF) is used to generate the robotic arm picking path. The path weight parameters are optimized online through the reinforcement learning (RL) model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, while satisfying the posture constraints of the end effector (such as approach angle ≤30° tilt).

[0057] Improved A* algorithm design: State space definition: Using the robot arm joint angles as nodes, combined with the spatial coordinates (X, Y, Z) and tilt angle of the target Agaricus bisporus, adding "end effector pose constraints" as heuristic functions (e.g., prioritizing paths with an angle less than 20° with the stem axis). Dynamic weight allocation: Introducing maturity priority (mature mushrooms with a weight of 1.5, immature mushrooms with a weight of 0.8), and adding a penalty term for path nodes close to immature mushrooms to avoid collisions.

[0058] Artificial potential field integration: Attraction field: The target mushroom stem is used as the center of gravity, with the magnitude of the force positively correlated with maturity (attraction coefficient K_att = 20 for mature mushrooms, K_att = 5 for immature mushrooms). Repulsion field: The edge of the cultivation bed and immature mushroom bodies serve as repulsion sources, with the repulsion range dynamically adjusted based on the size of the mushroom body (radius of immature mushroom body + 2cm safety distance) to prevent the end of the robotic arm from reaching the target mushroom body.

[0059] Reinforcement learning optimization: A Markov decision process (MDP) was constructed, with the state representing the current robot arm position and the distribution of surrounding obstacles. The action set consisted of joint angle increments. The reward function was designed as follows: successfully avoiding an immature mushroom body: +10 points; path length optimization: -0.1 × path distance (cm); collision penalty: -50 points. The PPO (Proximal Policy Optimization) algorithm was used for online training, with potential field parameters (such as the repulsive force attenuation coefficient) updated every 50 picking attempts to adapt to varying planting densities.

[0060] Compared to the traditional A* algorithm, path planning time is reduced by 40%, and the immature mushroom protection rate exceeds 98%. This overcomes the traditional algorithm's inability to dynamically correlate maturity and obstacle characteristics. A reinforcement learning mechanism enables the path planning strategy to autonomously adapt to changes in the cultivation bed layout (such as differences in spacing between multiple beds), eliminating manual parameter adjustments and improving the robot's safety and efficiency in dense cultivation environments.

[0061] Step S104. Dynamically adjust the gripping force and rotation angle of the end effector based on the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation, and separation when contacting the stem through a force-controlled closed-loop system. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus. The end effector integrates a pressure sensor and a visual feedback module.

[0062] Specifically, according to the maturity level and morphological characteristics of Agaricus bisporus (cap diameter, stipe thickness), the clamping force (immature mushrooms ≤ 1N, mature mushrooms 1-2N) and rotation angle (60°-90° rotation along the stipe axis) of the end effector are dynamically adjusted, and a force-controlled closed-loop system is used to achieve a smooth "light clamping-rotation-separation" action. The integrated pressure sensor and visual feedback module can predict the probability of cap damage in real time and correct the control parameters.

[0063] The end effector design utilizes a flexible gripper (e.g., silicone material with a distributed pressure sensor) with a gripper curvature tailored to the diameter of the Agaricus bisporus stem (5-30 mm), and a single-finger pressure resolution of ≤ 0.1 N. An integrated miniature visual feedback module (e.g., a USB camera) monitors the cap deformation in real time as the gripper closes (with pixel-level displacement detection accuracy).

[0064] The control strategy includes force-position hybrid control, which uses a preset initial clamping force based on maturity (e.g., 1.2 N for mature mushrooms). After contact with the stem, a closed-loop force control (PID controller with a proportional coefficient Kp = 0.8 and an integral coefficient Ki = 0.1) dynamically adjusts the force to ensure a force deviation of ≤ ±0.2 N. The breakage prediction model utilizes an LSTM neural network, taking as input pressure sensor time series data (pressure change rate over the previous 50 ms) and visual feedback (the degree of wrinkling at the edge of the cap) to output a breakage probability (with a threshold set at 20%; exceeding this threshold triggers a clamping force callback).

[0065] The separation action is optimized by adaptively adjusting the rotation angle according to the inclination angle of the stem (for example, when the inclination is 15°, the rotation axis is offset by 15°), and the separation speed is set at 5mm / s (immature mushrooms) to 10mm / s (mature mushrooms) to avoid pulling damage.

[0066] Through closed-loop force control and breakage prediction, the cap breakage rate has been reduced from 15%-20% with traditional methods to below 5%, significantly increasing the product's value. Dynamically adjusting the strategy based on the mechanical properties of Agaricus bisporus at different maturity levels (immature mushrooms are brittle and tender, while overripe mushrooms are prone to breakage) addresses the pain points of traditional manual harvesting, which relies heavily on experience, and achieves fully automated, precise control.

[0067] The above method systematically solves the problems of efficiency, accuracy, damage rate and environmental adaptability of Agaricus bisporus picking through a closed-loop architecture of multimodal perception → lightweight recognition → intelligent planning → flexible control. Compared with traditional manual and existing robotic technology, it can increase picking efficiency by more than 3 times in large-scale cultivation scenarios and reduce labor costs by 60%, providing core technical support for the intelligent upgrade of the edible fungus industry.

[0068] In some embodiments, the RGB image is input into a pre-trained lightweight convolutional neural network, and the attention mechanism is combined to identify the cap outline, stipe position and maturity level of Agaricus bisporus, including: performing multi-scale Gaussian blur preprocessing on the RGB image to suppress noise, adopting the MobileNetv3 network based on deep separable convolution as the basic skeleton, embedding a cascade structure of a channel attention module and a spatial attention module in the feature extraction stage, and strengthening the extraction of cap edge texture and stipe color contrast features through the dual attention mechanism; constructing a multi-task training dataset including cap contour mask, stipe key point coordinates and maturity label, adopting the focal loss function to optimize the category imbalance problem, filtering out overlapping detection frames through the non-maximum suppression algorithm during the inference process, and finally outputting the pixel-level cap contour segmentation result, stipe center coordinates and maturity level score based on HSV color space and texture roughness features.

[0069] In the preprocessing stage, the RGB image is subjected to multi-scale Gaussian blurring (kernel sizes of 3×3, 5×5, and 7×7, with standard deviations σ = 1.0, 1.5, and 2.0, respectively). Weighted fusion is then used to suppress noise at different scales (such as noise from particles in the cultivation substrate and artifacts caused by uneven lighting), while preserving the high-frequency texture information at the edge of the cap. The image is then converted to the HSV color space, with the luminance channel (V channel) separated and histogram equalization performed to enhance the contrast between the stipe (white) and the background (brown substrate), reducing the interference of complex lighting on color recognition.

[0070] Network Architecture: MobileNetV3-Large is used as the basic framework, replacing standard convolutions with depthwise separable convolutions. A cascade of channel-wise attention modules (CAMs) and spatial attention modules (SAMs) is embedded in feature extraction layers (e.g., layers 5 and 9). The CAM performs global average pooling and global max pooling on feature maps, generating channel weights through a shared MLP (Multi-Layer Perceptron) to emphasize color channels related to maturity (e.g., saturation information in the S channel of HSV). The SAM performs a 7×7 convolution on the feature map to generate a spatial attention mask, focusing on the edge contours of the cap (using Canny edge detection preprocessing to mark edge regions) and suppressing responses in non-target areas.

[0071] The multi-task training dataset was constructed by annotating data including cap contour masks (pixel-level binary masks with an IOU annotation accuracy of ≥95%), stipe keypoint coordinates (bottom circle center, middle, and top), and maturity labels (three levels: immature, mature, and overripe). The dataset contained ≥100,000 samples, covering scenarios with varying lighting conditions (200-2000 lux) and density (single plant vs. clustered). To address the imbalance in maturity categories (immature samples account for 60%), a focal loss function was used with a fixed α parameter (immature α = 0.3, mature α = 0.5, overripe α = 0.2) to reduce the weight of easily classified samples and improve the recognition accuracy of samples at the overripe / immature boundary.

[0072] After the inference phase processes the output detection boxes, non-maximum suppression (NMS) is used to filter out overlapping boxes (with an IOU threshold of 0.3), retaining the detections with the highest confidence. Morphological processing (dilation and erosion) is performed on the cap contour mask to fill small holes, ultimately outputting pixel-level segmentation results (contour edge error ≤ 2 pixels). The maturity rating is based on a fusion of HSV color mean (V channel mean ≥ 220 for mature mushrooms) and texture roughness features (contrast parameters calculated using the gray-level co-occurrence matrix, with contrast <15 for overripe mushrooms). A weighted average (60% color, 40% texture) is used to generate a quantitative score from 0 to 100.

[0073] Multi-scale Gaussian blurring effectively suppresses complex environmental noise, improving the IOU of contour segmentation by 18% in low-light (<300 lux) scenarios and resolving the problem of under-detection in shadowed areas caused by traditional methods. A dual attention mechanism reduces the stipe localization error from 8mm in traditional CNNs to 3mm, achieving 97% consistency between maturity scores and manual annotations. The algorithm particularly improves misjudgment in the early stages of parachute deployment (critical maturity), reducing the misjudgment rate from 12% to 4%. The lightweight network boasts only 4.2MB of parameters and achieves an inference speed of ≤25ms / frame on embedded CPUs (such as the Raspberry Pi 4B), meeting the computing power constraints of real-time robot control.

[0074] In some embodiments, the spatial coordinates, height and growth inclination angle of Agaricus bisporus are calculated by spatial geometry algorithm based on three-dimensional point cloud data to construct an Agaricus bisporus target database containing morphological features and spatial positions, including: voxel grid downsampling and statistical filtering denoising of the three-dimensional point cloud data, fitting the cultivation bed plane through the RANSAC algorithm and calculating the relative height of the point cloud data, and using the Euclidean clustering algorithm to segment the single Agaricus bisporus point cloud cluster; performing cylindrical fitting on each point cloud cluster, establishing a local coordinate system with the center of the bottom circle of the stipe as the origin, calculating the stipe axis direction vector through principal component analysis, determining the growth inclination angle based on the angle between the axis vector and the gravity direction, combining the intrinsic parameter matrix and the extrinsic parameter matrix of the binocular vision camera, and converting the three-dimensional coordinates in the local coordinate system into absolute spatial coordinates in the robot base coordinate system; associating the spatial coordinates, height, and inclination angle with the morphological features such as the cap diameter and stipe diameter identified in claim 2 to construct an Agaricus bisporus target database containing a unique target ID.

[0075] Point cloud preprocessing includes: voxel grid downsampling: setting the voxel size to 5 mm × 5 mm × 5 mm to reduce the amount of point cloud data (density dropped from 200 points / cm² to 50 points / cm²) while preserving the outline details of the stipe; statistical filtering (k = 50, standard deviation threshold 2.0) was used to remove outliers (such as splashed matrix particles).

[0076] The bed plane fitting includes: fitting the cultivation bed plane through the RANSAC algorithm (1000 iterations, distance threshold 10 mm), calculating the relative height of each point cloud (the height above the bed plane is regarded as the growth height of Agaricus bisporus), and filtering out invalid points below the bed.

[0077] Individual plant segmentation and feature calculation: Euclidean clustering: Set a search radius of 15 mm (covering a maximum cap diameter of 30 mm) to isolate clusters of densely populated Agaricus bisporus plants, ensuring that clusters contain only individual plants (cap and stipe). Cylindrical fitting and pose calculation: Perform a cylindrical fitting (least squares method) on the stipe point cloud (manually annotated with the stipe area or filtered using a contour mask) to obtain the stipe axis direction vector. Principal component analysis (PCA) is used to calculate the angle between the axis and the direction of gravity (Z axis) and determine the growth inclination angle (accuracy ±1.5°).

[0078] Coordinate Transformation and Database Construction: Coordinate system transformation: Using the binocular camera's intrinsic parameter matrix (focal length, distortion parameters) and extrinsic parameter matrix (rotation and translation matrix from the camera to the robot's base coordinate system, obtained using Zhang's calibration method), we transform the point cloud coordinates in the local coordinate system (with the center of the stem bottom circle as the origin) into absolute coordinates (X, Y, Z) in the robot's base coordinate system, with an error of ≤5mm. The cap diameter (circumscribed circle diameter of the contour mask) and stipe diameter (cylinder fitting diameter) are extracted from the 2D visual results and correlated with the height and tilt angle of the 3D point cloud. Each object is assigned a unique ID (e.g., "Bed3-Cluster12"), and a structured database containing 12 feature fields is constructed.

[0079] Through RANSAC plane fitting and Euclidean clustering, individual plant segmentation is achieved in densely planted environments (with a 99.2% success rate), addressing the problem of traditional methods misjudging spatial position in clustered mushrooms. Multimodal data fusion links 2D visual morphological features (diameter, maturity) with spatial parameters of 3D point clouds (coordinates, tilt angle), providing a composite "position + posture + priority" decision-making basis for subsequent path planning. For example, mushrooms with tilt angles greater than 30° are prioritized for sideways approach paths to avoid collisions caused by vertical downward pressure. Voxel downsampling and statistical filtering increase point cloud processing speed by three times (from 120ms / frame to 40ms / frame), while preserving key geometric features and adapting to high-dust and high-humidity cultivation environments (point cloud noise suppression rate reaches 85%).

[0080] In some embodiments, the method of generating a robotic arm picking path based on the Agaricus bisporus target database using an improved A* algorithm combined with an artificial potential field method includes: introducing a dynamic weight factor into the heuristic function of the A* algorithm, wherein the weight factor is associated with the maturity level of the target Agaricus bisporus and the safety distance threshold of adjacent immature Agaricus bisporus, and giving higher priority to targets with high maturity; constructing a double Gaussian repulsion field in the artificial potential field model, applying an exponentially decaying repulsion to obstacles in the cultivation bed, and applying an elliptical repulsion field with direction sensitivity to adjacent immature Agaricus bisporus, wherein the repulsion strength decays inversely with the square of the distance; smoothing the discrete path points generated by the A* algorithm through quintic polynomial interpolation, and calculating the angular velocity and angular acceleration boundary conditions of each axis in combination with the kinematic constraints of the robotic arm joints to form a spatial curve trajectory that satisfies the continuity of the end effector posture.

[0081] A* algorithm improvement and weight design: Heuristic function expansion: The traditional A* heuristic function h(n) = Euclidean distance, which is improved to h(n) = ω1×D(n,goal)+ω2×(1 / maturity), where ω1=0.6, ω2=0.4, and maturity is the maturity score (0-100, normalized). Goals with higher maturity are given smaller heuristic values ​​(higher priority).

[0082] Safety distance penalty: If the distance between the current node and the immature mushroom body is less than the safety threshold (radius of the immature mushroom body + 10mm), a penalty term Δh = k × (1-d / safety threshold) (k = 50) is added to force the path away from the immature target.

[0083] Artificial potential field optimization: Double Gaussian repulsion field: Apply exponential decay repulsion to obstacles in the cultivation bed (metal frame, support column): F_rep=K_rep×(1 / d 2 ) × exp(-d / d0), where d0 = 20 mm, to avoid path oscillation caused by strong repulsive forces at close range. An elliptical repulsive field is applied to immature Agaricus bisporus (taking into account the inclination of growth): the range of the repulsive force along the stem axis (major axis) is expanded by 50%, while the vertical range (minor axis) remains unchanged. By rotating the elliptical coordinate system to match the inclination angle of the mushroom, directionally sensitive obstacle avoidance is achieved.

[0084] Trajectory smoothing and kinematic constraints include: quintic polynomial interpolation is used to interpolate the discrete path points (interval 20 mm) generated by A* to generate a C2 continuous space curve, ensuring smooth changes in the end effector speed and acceleration (maximum angular acceleration ≤ 10 rad / s 2 Based on the inverse kinematics model of the robotic arm, the angular velocity limits of each joint are calculated (e.g., shoulder joint ≤ 60° / s), and the interpolated trajectory is optimized secondary to avoid mechanical shock caused by excessive joint motion.

[0085] Priority-driven planning reduces the delay in harvesting mature mushrooms from 15 seconds per plant using traditional algorithms to 5 seconds per plant. During peak ripening periods (where mature mushrooms account for >40%), efficiency increases by 60%, effectively addressing time-sensitive harvesting requirements. An elliptical repulsive field reduces the collision rate of immature mushrooms from 8% using traditional APF to 1.2%. Especially in scenarios with tilted mushrooms, the minimum safe distance between the end effector and adjacent mushrooms is increased from 8mm to 15mm, protecting the continued growth of immature mushrooms. Quintic polynomial interpolation reduces the end effector's jerk (acceleration rate) by 70%, minimizing mechanical vibration disturbances to surrounding mushrooms and extending the life of the robotic arm (reducing joint wear by 40%).

[0086] In some embodiments, the method dynamically adjusts the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem, including: establishing a maturity-clamping force mapping function, the function is trained based on a support vector regression (SVR) model, and the input parameters include the cap diameter, the stem hardness (predicted by visual texture features) and the maturity level, and outputs an initial clamping force threshold of the end effector clamp; during the clamping process, the contact force signal is collected in real time by a pressure sensor, and an incremental PID controller is used to implement a force-controlled closed loop, and an adaptive adjustment strategy is triggered when it is detected that the contact force exceeds the threshold by 15%; the initial rotation angle of the end effector is adjusted according to the growth inclination angle so that the clamping claw plane remains perpendicular to the stem axis, and the cap deformation is monitored in real time through a visual feedback module during the rotation and separation process, and the end position deviation is dynamically compensated in combination with a force-position hybrid control algorithm.

[0087] Clamping force modeling and initial parameter setting: Support Vector Regression (SVR) training: Input features include cap diameter (mm), stipe diameter (mm), maturity score (0-100), and visual texture roughness (calculated using the LBP operator). The output is the initial clamping force threshold F0 (N). Training data comes from 5000 physical experiments. The optimal separation force (maximum clamping force with a breakage rate of ≤5%) for different mushroom morphologies was recorded using a pressure sensor. The kernel function used was RBF, with penalty parameters C=100 and ε=0.1.

[0088] Application of mapping function: For example, the calculated F0 for mature mushrooms (25 mm diameter, 15 mm stem diameter, score 85) is 1.8 N, while the calculated F0 for immature mushrooms (10 mm diameter, 8 mm stem diameter, score 30) is 0.8 N.

[0089] Force Control Closed-Loop and Adaptive Regulation: Incremental PID control: The controller input is the deviation between the real-time pressure value F_t and the threshold F_0, e_t = F_t-F_0. The output is the gripper closing distance Δu = Kp(e_t-e_{t-1})+Ki e_t+Kd(e_t-2e_{t-1}+e_{t-2}), where Kp = 1.2, Ki = 0.05, and Kd = 0.8. The control cycle is 5ms, ensuring real-time convergence of force deviation. Over-limit triggering strategy: When F_t > 1.15F_0 (exceeding the threshold by 15%), adaptive regulation is triggered: the gripper is first released at a rate of 0.1N / s while simultaneously capturing images of the cap deformation (the visual feedback module has a resolution of 640×480 and a frame rate of 60fps). If the deformation pixel displacement exceeds 5px, the target force is further reduced to 0.9F_0.

[0090] Adaptive Pose Adjustment: Rotational angle compensation uses the inclination angle θ calculated by 3D point cloud computing to adjust the gripper coordinate system using the end effector's six-axis force sensor, ensuring that the angle between the gripper plane normal and the stem axis is ≤10° (traditional methods use a fixed vertical angle with an error of ±30°). Separation control is achieved by setting the rotational angular velocity to θ × 0.1° / s (the larger the inclination angle, the slower the rotation speed) to prevent cap tearing caused by angular deviation. The separation displacement is 1.2 times the stem height (to ensure complete detachment from the culture medium).

[0091] The SVR model predicts a root mean square error (RMS) of only 0.12N in gripping force. Combined with a PID closed loop, the actual gripping force fluctuates ≤±0.15N, significantly improving stability compared to traditional empirical control (fluctuation ±0.5N). The breakage rate of immature mushrooms has been reduced from 20% to 4.5%. Adaptive posture adjustment reduces the angular error between the gripper plane and the stem axis from 25° to 8° compared to traditional methods. This reduces torsional stress on the cap during rotational separation by 60%, and particularly reduces the damage rate by 75% for tilted mushrooms (which account for approximately 30%). A real-time feedback mechanism achieves dynamic force-position coupling control within a 5ms control cycle, completing gripping force calibration at the moment of contact (within 50ms), adapting to random growth posture variations (such as varying stem curvature).

[0092] In some embodiments, the method predicts the probability of cap damage during the picking process and corrects the control parameters in real time to complete the automatic picking of Agaricus bisporus, including: constructing a damage risk prediction model based on random forest, inputting features including cap surface curvature distribution, clamping point height, rotation angular velocity and historical damage case data, and outputting a damage probability prediction value; before the picking action is performed, if the predicted damage probability exceeds a preset threshold, the parameter correction mechanism is triggered: reducing the clamping speed by 30%, increasing the rotation angle compensation step to 1.5°, and adjusting the proportional coefficient of the force control closed loop to reduce contact force fluctuations; during the picking action, the visual feedback module integrated in the end effector is used to collect the cap surface crack image in real time, and the ResNet-18 network after transfer learning is used for real-time crack detection. Once a microcrack is detected, the emergency stop strategy is immediately triggered, and the current control parameters and detection results are stored in the online learning database for updating the damage risk prediction model.

[0093] The random forest model uses input features including cap surface curvature (calculated using a 3D point cloud; curvatures > 0.05 are considered prone to breakage), gripping point height (risk increases when < 10 mm from the base of the cap), rotational angular velocity (risk index increases by 20% when > 15° / s), and control parameters for historical breakage cases (such as the gripping force threshold that has previously caused breakage) to output a breakage probability (0-100%). The model is trained on 100,000 harvests, ranking features by Gini impurity reduction. Curvature and gripping point height account for 60% of the features' importance. Preset threshold strategies include triggering parameter correction when the breakage probability exceeds 30%, and skipping the mushroom body (marking it as "high risk" and requiring manual review) when the probability exceeds 50%.

[0094] Pre-picking parameter correction: If the predicted probability is 30%-50%, the following actions will be taken: the clamping speed is reduced from 20 mm / s to 14 mm / s (a 30% reduction) to reduce contact impact; the rotation angle compensation step is increased from 1° to 1.5°, and the posture is adjusted in stages to adapt to irregular stems; the force control closed-loop proportional coefficient Kp is reduced from 1.2 to 1.0 to reduce control response sensitivity and avoid clamping force fluctuations caused by excessive adjustment.

[0095] Real-time detection and emergency response during harvesting: Real-time crack detection: The visual feedback module collects images of the cap surface and detects them using a transfer-learned ResNet-18 model (pre-trained on ImageNet and fine-tuned on 500 crack samples). The model achieves 92% crack recognition accuracy and ≤20ms detection latency. Emergency stop strategy: Once a microcrack (pixel-level edge discontinuity) is detected, a stop signal (hardware interrupt has the highest priority) is immediately sent. The gripper maintains its current force for 3 seconds before slowly releasing it to avoid forced separation and complete damage. The current parameters (clamping force 1.9N, rotation angle 85°) and crack images are stored in an online database (the model is updated daily).

[0096] Through pre-harvest prediction, the breakage rate for high-risk harvesting has been reduced from 40% with traditional methods to 12%. Combined with real-time crack detection and a dynamic correction strategy, the overall breakage rate has been further reduced to 3.8%, meeting commercial-grade non-destructive harvesting standards. The online learning database adds approximately 200 valid data points daily, and the random forest model is automatically retrained every 72 hours, resulting in a 1.5% monthly improvement in prediction accuracy. This adapts to the varying mechanical properties of different Agaricus bisporus varieties (for example, the breakage thresholds of "Mingu No. 1" and "AS2796" differ by 0.3N). The emergency stop strategy has an interrupt response time of less than 1ms, ensuring timely stoppage at the microcrack stage at the hardware level. Compared to software logic (response time of 50ms), this reduces the probability of crack propagation by 90%, protecting high-value mature mushrooms. (A single damaged mushroom costs approximately 0.5 yuan, saving over 2,000 yuan per day in large-scale cultivation.)

[0097] In some embodiments, after completing the automatic picking of Agaricus bisporus, the method further includes: feeding back the picking result data to the lightweight convolutional neural network and reinforcement learning model, updating the algorithm parameters through transfer learning, and forming a self-optimizing closed loop of the picking strategy; the picking result data includes success or failure cases, damage degree and picking time.

[0098] Feedback data is collected by recording success / failure cases (failure definition: breakage rate > 5% or picking timeout > 30s), damage degree (divided into four levels: no damage / minor abrasion / cap crack / stem break), picking time (total time from identification to separation). Each record is associated with the environmental parameters during picking (light intensity, number of bed layers), target characteristics (maturity, tilt angle), and control parameters (clamping force, rotation angle).

[0099] The transfer learning update mechanism is updated through a lightweight CNN: 5,000 failure cases are collected each month, and the channel weights of the attention module are fine-tuned through transfer learning. In particular, for the missed detection scenario of "the yellow edges of over-ripe mushrooms are confused with the matrix color", the attention weight of the yellow channel in the HSV color space is increased (from 0.3 to 0.5).

[0100] The reinforcement learning strategy optimization uses the picking time as a new reward function item (reward +5 points for every 1s saved), retrains the path planning model through the PPO algorithm, and makes region-specific adjustments to the path weights for high-rise beds (height > 1.5m) (the gravity compensation coefficient is increased by 0.2), thereby improving the movement efficiency of the robotic arm at different heights.

[0101] The closed-loop verification process involves verifying the model 300 times in a simulation environment (built in Unity, containing 100 complex planting scenarios) each time the model is updated. A breakage rate of less than 5% and a running time of less than 8 seconds are considered passed. The model is then deployed to an actual robot for 50 field tests. The performance differences between the new and old strategies are compared using a two-sample t-test (significance level α=0.05).

[0102] After three months of continuous operation, the self-optimizing closed-loop system increased overall picking efficiency by 12%. The breakage rate in complex scenarios (three-layer densely planted beds) dropped from an initial 6.2% to 2.7%, resolving the problem of traditional algorithms' "one-size-fits-all" strategies being unable to adapt to changes in varieties and environments. The online database, which accumulates millions of harvesting data points, has developed a library of specialized strategies for different cultivation modes (ground planting, bed planting, and three-dimensional cultivation). Transfer learning can be used to rapidly adapt to new farm deployments (initialization time is reduced from two weeks to two days). For emerging issues (such as changes in separation force due to increased mushroom hardness in winter), the clamping force model is automatically adjusted based on feedback data, eliminating the need for manual intervention. This achieves intelligent, full-chain "identification-planning-control-optimization" and reduces operation and maintenance costs by over 40%.

[0103] See also Figure 3 As shown, Figure 3 Schematic diagram of an AI-based automatic mushroom harvesting device 200 according to an embodiment of the present application. This AI-based automatic mushroom harvesting device 200 is configured to execute the steps of the AI-based automatic mushroom harvesting method described in the aforementioned embodiments. This AI-based automatic mushroom harvesting device 200 can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0104] like Figure 3 As shown, the automatic Agaricus bisporus picking device 200 based on artificial intelligence includes:

[0105] The data acquisition unit 201 is used to collect RGB images and three-dimensional point cloud data of the Agaricus bisporus growth environment in real time through a binocular vision camera and a depth sensor installed at the end of the picking robot, and obtain environmental lighting parameters and cultivation bed coordinate information;

[0106] The image input unit 202 is configured to input the RGB image into a pre-trained lightweight convolutional neural network, identify the cap contour, stipe position, and maturity level of the Agaricus bisporus by combining an attention mechanism, calculate the spatial coordinates, height, and growth inclination angle of the Agaricus bisporus based on the three-dimensional point cloud data using a spatial geometry algorithm, and construct an Agaricus bisporus target database containing morphological features and spatial positions;

[0107] The trajectory generation unit 203 is configured to generate a picking path for the robotic arm based on the Agaricus bisporus target database using an improved A* algorithm combined with an artificial potential field method, and to optimize the path weight parameters online using a preset reinforcement learning model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, thereby satisfying the posture constraints of the robotic arm end effector.

[0108] The angle adjustment unit 204 is used to dynamically adjust the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus. The end effector is integrated with a pressure sensor and a visual feedback module.

[0109] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described artificial intelligence-based automatic Agaricus bisporus picking device and each module can refer to the corresponding processes in the embodiments of the artificial intelligence-based automatic Agaricus bisporus picking method described in the above embodiments, and will not be repeated here.

[0110] The above-mentioned method for automatically picking Agaricus bisporus based on artificial intelligence can be implemented in the form of a computer program. The computer program can be used in Figure 3 Run on the device shown.

[0111] See also Figure 4 , Figure 4 1 is a schematic block diagram of the structure of a control module provided in an embodiment of the present application. The control module includes a processor, a memory and a network interface connected via a device bus, wherein the memory may include a storage medium and an internal memory.

[0112] The storage medium can store an operating device and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the automatic picking methods of Agaricus bisporus based on artificial intelligence.

[0113] The processor is used to provide computing and control capabilities and support the operation of the entire control module.

[0114] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the automatic picking methods of Agaricus bisporus based on artificial intelligence.

[0115] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the terminal to which the solution of the present application is applied. The specific control module may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0116] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0118] The binocular vision camera and depth sensor installed at the end of the picking robot collect RGB images and 3D point cloud data of the Agaricus bisporus growth environment in real time, and obtain environmental lighting parameters and cultivation bed coordinate information;

[0119] The RGB image is input into a pre-trained lightweight convolutional neural network, and the cap outline, stipe position and maturity level of the Agaricus bisporus are identified by combining the attention mechanism. The spatial coordinates, height and growth inclination angle of the Agaricus bisporus are calculated based on the three-dimensional point cloud data using a spatial geometry algorithm, and an Agaricus bisporus target database containing morphological characteristics and spatial positions is constructed;

[0120] Based on the Agaricus bisporus target database, an improved A* algorithm combined with an artificial potential field method is used to generate a picking path for the robotic arm. The path weight parameters are optimized online through a preset reinforcement learning model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, thereby meeting the posture constraints of the robotic arm end effector.

[0121] According to the maturity level and morphological characteristics of Agaricus bisporus, the clamping force and rotation angle of the end effector are dynamically adjusted to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus; the end effector integrates a pressure sensor and a visual feedback module.

[0122] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the artificial intelligence-based automatic picking method of Agaricus bisporus provided in any embodiment of the present application.

[0123] The computer-readable storage medium may be an internal storage unit of the control module described in the aforementioned embodiment, such as a hard disk or memory of the control module. The computer-readable storage medium may also be an external storage device of the control module, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the control module.

[0124] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An automatic picking method for Agaricus bisporus based on artificial intelligence, characterized in that: include: The binocular vision camera and depth sensor installed at the end of the picking robot collect RGB images and 3D point cloud data of the Agaricus bisporus growth environment in real time, and obtain environmental lighting parameters and cultivation bed coordinate information; The RGB image is input into a pre-trained lightweight convolutional neural network, and the cap outline, stipe position and maturity level of the Agaricus bisporus are identified by combining the attention mechanism. The spatial coordinates, height and growth inclination angle of the Agaricus bisporus are calculated based on the three-dimensional point cloud data using a spatial geometry algorithm, and an Agaricus bisporus target database containing morphological characteristics and spatial positions is constructed; Based on the Agaricus bisporus target database, an improved A* algorithm combined with an artificial potential field method is used to generate a picking path for the robotic arm. The path weight parameters are optimized online through a preset reinforcement learning model to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, thereby meeting the posture constraints of the robotic arm end effector. According to the maturity level and morphological characteristics of Agaricus bisporus, the clamping force and rotation angle of the end effector are dynamically adjusted to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus; the end effector integrates a pressure sensor and a visual feedback module.

2. The method according to claim 1, characterized in that The step of inputting the RGB image into a pre-trained lightweight convolutional neural network and combining the attention mechanism to identify the cap outline, stipe position, and maturity level of Agaricus bisporus includes: The RGB image is preprocessed with multi-scale Gaussian blur to suppress noise. A MobileNetv3 network based on depthwise separable convolution is used as the basic framework. A cascade structure of a channel attention module and a spatial attention module is embedded in the feature extraction stage. This dual attention mechanism enhances the extraction of cap edge texture and stipe color contrast features. A multi-task training dataset containing cap contour masks, stipe key point coordinates and maturity labels was constructed. The focal loss function was used to optimize the class imbalance problem. During the inference process, the overlapping detection frames were filtered out through the non-maximum suppression algorithm. Finally, the pixel-level cap contour segmentation results, stipe center coordinates and maturity grade scores based on HSV color space and texture roughness features were output.

3. The method according to claim 1, characterized in that The method of calculating the spatial coordinates, height, and growth inclination angle of Agaricus bisporus by using a spatial geometry algorithm based on the three-dimensional point cloud data, and constructing an Agaricus bisporus target database including morphological characteristics and spatial positions, comprises: The three-dimensional point cloud data is subjected to voxel grid downsampling and statistical filtering denoising, the cultivation bed plane is fitted by a RANSAC algorithm and the relative height of the point cloud data is calculated, and the single Agaricus bisporus point cloud cluster is segmented using a Euclidean clustering algorithm; A cylindrical fitting is performed on each point cloud cluster, and a local coordinate system is established with the center of the stipe bottom as the origin. The stipe axis direction vector is calculated through principal component analysis. The growth inclination angle is determined based on the angle between the axis vector and the direction of gravity. The three-dimensional coordinates in the local coordinate system are converted into absolute spatial coordinates in the robot base coordinate system by combining the intrinsic and extrinsic parameter matrices of the binocular vision camera. The spatial coordinates, height, and tilt angle are associated with the morphological features such as the cap diameter and the stipe diameter identified in claim 2 to construct an Agaricus bisporus target database containing a unique target ID.

4. The method according to claim 1, wherein The method of generating a picking path for a robotic arm based on the Agaricus bisporus target database and using an improved A* algorithm combined with an artificial potential field method comprises: A dynamic weighting factor is introduced into the heuristic function of the A* algorithm. The weighting factor is associated with the maturity level of the target Agaricus bisporus and the safety distance threshold of adjacent immature Agaricus bisporus, giving higher priority to targets with higher maturity. A double Gaussian repulsive field was constructed in the artificial potential field model to apply an exponentially decaying repulsive force to obstacles in the cultivation bed, and an elliptical repulsive field with direction sensitivity was applied to adjacent immature Agaricus bisporus, with the repulsive force intensity decaying inversely with the square of the distance. The discrete path points generated by the A* algorithm are smoothed by quintic polynomial interpolation. The boundary conditions of the angular velocity and angular acceleration of each axis are calculated in combination with the kinematic constraints of the robot joints to form a spatial curve trajectory that satisfies the continuity of the end effector's posture.

5. The method according to claim 1, wherein The method dynamically adjusts the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem, including: A maturity-gripping force mapping function was established. This function was trained based on a support vector regression (SVR) model. The input parameters included cap diameter, stipe hardness (predicted by visual texture features), and maturity level. The function outputted the initial gripping force threshold of the end-effector gripper. During the clamping process, the contact force signal is collected in real time through a pressure sensor, and an incremental PID controller is used to implement a closed-loop force control. When the contact force exceeds the threshold of 15%, the adaptive adjustment strategy is triggered. The initial rotation angle of the end effector is adjusted according to the growth inclination angle to keep the gripper plane perpendicular to the axis of the stem. During the rotational separation process, the cap deformation is monitored in real time through the visual feedback module, and the end position deviation is dynamically compensated in combination with the force-position hybrid control algorithm.

6. The method according to claim 1, wherein The method of predicting the probability of cap damage during the picking process and correcting control parameters in real time to complete automatic picking of Agaricus bisporus includes: A random forest-based breakage risk prediction model was constructed. The input features included the cap surface curvature distribution, clamping point height, rotation angular velocity, and historical breakage case data, and the output was a predicted value for the breakage probability. Before the picking action is performed, if the predicted breakage probability exceeds a preset threshold, a parameter correction mechanism is triggered: the gripping speed is reduced by 30%, the rotation angle compensation step is increased to 1.5°, and the proportional coefficient of the force control closed loop is adjusted to reduce contact force fluctuations; During the picking process, the visual feedback module integrated in the end effector collects real-time images of cracks on the cap surface, and the ResNet-18 network after transfer learning is used for real-time crack detection. Once microcracks are detected, the emergency stop strategy is immediately triggered. At the same time, the current control parameters and detection results are stored in the online learning database to update the damage risk prediction model.

7. The method according to claim 1, characterized in that After the automatic picking of Agaricus bisporus is completed, the method further includes: The picking result data is fed back to the lightweight convolutional neural network and reinforcement learning model, and the algorithm parameters are updated through transfer learning to form a self-optimizing closed loop of the picking strategy; the picking result data includes success or failure cases, damage degree and picking time.

8. An automatic Agaricus bisporus picking device based on artificial intelligence, characterized in that: include: The data acquisition unit is used to collect RGB images and three-dimensional point cloud data of the Agaricus bisporus growth environment in real time through a binocular vision camera and depth sensor installed at the end of the picking robot, and obtain environmental lighting parameters and cultivation bed coordinate information; An image input unit is configured to input the RGB image into a pre-trained lightweight convolutional neural network, identify the cap contour, stipe position, and maturity level of the Agaricus bisporus by combining an attention mechanism, calculate the spatial coordinates, height, and growth inclination angle of the Agaricus bisporus based on the three-dimensional point cloud data using a spatial geometry algorithm, and construct an Agaricus bisporus target database containing morphological features and spatial positions; A trajectory generation unit is used to generate a picking path for the robotic arm based on the Agaricus bisporus target database using an improved A* algorithm combined with an artificial potential field method, and to optimize the path weight parameters online through a preset reinforcement learning model, so as to dynamically avoid obstacles in the cultivation bed and adjacent immature Agaricus bisporus, and meet the posture constraints of the end effector of the robotic arm; The angle adjustment unit is used to dynamically adjust the clamping force and rotation angle of the end effector according to the maturity level and morphological characteristics of the Agaricus bisporus, so as to achieve a smooth picking action of light clamping, rotation and separation through a force-controlled closed-loop system when contacting the stem. The probability of cap damage during the picking process is predicted and the control parameters are corrected in real time to complete the automatic picking of Agaricus bisporus. The end effector is integrated with a pressure sensor and a visual feedback module.

9. A control module, characterized in that: The control module includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 7.

Images