Intelligent greenhouse bothromyces harvesting method and system based on multi-modal perception and dynamic decision

Abstract

The present application belongs to the field of collection control, and particularly relates to a method and system for intelligent double-belly mushroom collection in a greenhouse based on multi-modal sensing and dynamic decision-making. The method first synchronously collects multi-modal data representing the maturity of double-belly mushrooms through an integrated sensor array, and calculates a first maturity index in combination with environmental parameters and a multi-modal fusion model. Then, based on the index and a maturity-preservation period curve, the method discriminates to generate information sequences of harvestable and unharvestable double-belly mushrooms. Subsequently, a coordinated reinforcement learning model is used, which includes harvesting control, movement control, and coordination sub-models, to comprehensively generate coordinated control instructions for an AGV movement platform and a six-degree-of-freedom robot end flexible gripper, with preservation period, damage rate, energy consumption, and instruction coordination consistency as reward functions. Finally, the instructions are executed to complete automated collection. The present application realizes precise sensing of double-belly mushroom maturity and dynamic optimization of collection decision-making, effectively improving collection quality and work efficiency.

Description

Technical Field

[0001] This invention belongs to the field of data acquisition and control, and particularly relates to an intelligent greenhouse mushroom harvesting method and system based on multimodal perception and dynamic decision-making. Background Technology

[0002] Currently, the automated harvesting of button mushrooms faces multiple bottlenecks, from identification and decision-making to execution. Harvesting operations are highly reliant on manual labor, resulting in low efficiency, immense manpower pressure during peak fruiting seasons, and subjective judgments of maturity by harvesters, often leading to improper harvesting timing, directly causing yield losses and shortened shelf life. Existing automated equipment and academic research solutions have failed to effectively address this core issue. Commercial equipment typically relies on single visual features for identification, ignoring key maturity indicators such as gill morphology and stem firmness, leading to high misharvesting rates. Furthermore, its rigid mechanical structure easily causes physical damage to surrounding mushrooms in densely planted environments. While academic research has attempted to introduce more sensing methods, it is often limited by interference from contact measurements on the mushrooms, complex algorithms failing to meet real-time requirements in the field, and a general lack of post-harvest quality feedback loops, hindering the development of continuously optimized harvesting strategies. Crucially, both manual methods and existing technological solutions completely ignore the dynamic and continuous impact of the greenhouse microenvironment on the growth rate and post-harvest quality of button mushrooms. This results in static and isolated harvesting decisions, unable to adaptively adjust based on the coupling relationship between the environment and growth status. Therefore, constructing an intelligent system capable of multi-dimensional precise perception, integrating environmental factor analysis, and executing low-loss flexible harvesting has become an urgent need to improve the efficiency and quality of the entire industry. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes an intelligent greenhouse mushroom harvesting method and system based on multimodal perception and dynamic decision-making. The method first uses an integrated sensor array to synchronously collect multimodal data characterizing the maturity of mushrooms, including cap color, gill moisture content, cap wax layer thickness, and stem hollowness. A first maturity index is calculated using environmental parameters and expert algorithms. Then, based on this index and the maturity-shelf-life curve, a sequence of harvestable and unharvested mushroom information is generated. Subsequently, a coordinated reinforcement learning model, comprising harvesting control, movement control, and collaborative sub-models, is used. With shelf life, damage rate, energy consumption, and command consistency as reward functions, collaborative control commands are generated for the AGV mobile platform and the six-degree-of-freedom robot end effector. Finally, these commands are executed to complete automated harvesting. This invention achieves accurate perception of mushroom maturity and dynamic optimization of harvesting decisions, effectively improving harvesting quality and operational efficiency.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A multimodal sensing and dynamic decision-making method for harvesting button mushrooms in an intelligent greenhouse includes:

[0006] The first maturity index, which characterizes the maturity of *Agaricus bisporus*, is collected by a configured sensor array in the target area. The first maturity index is obtained by combining the color characteristics of the cap, the moisture content of the gills, the thickness of the waxy layer on the cap surface, the hollowness inside the stem, and the environmental correction coefficient collected by the sensors with a preset multimodal fusion model.

[0007] Based on the first maturity index and the maturity-shelf life curve, the first discrimination is collected to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp.

[0008] Based on the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested, a coordinated reinforcement learning model is combined with a reward function constructed from shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy, and instruction coordination consistency delay to generate coordinated control instructions.

[0009] Execute coordinated control commands.

[0010] Specifically, the coordinated reinforcement learning model includes a harvesting control sub-model, a movement control sub-model, and a collaboration sub-model. The movement control sub-model is used to generate movement control instructions for the AGV (Automated Guided Vehicle) based on the location information of each harvestable twin mushroom in the harvestable twin mushroom information sequence and the harvestable prediction timestamp of the twin mushroom to be harvested, combined with a path optimization algorithm. The harvesting control sub-model is used to generate harvesting control instructions for a three-finger flexible gripper based on the first maturity index, the maturity-harvesting intensity-damage probability mapping relationship, and an impedance control algorithm optimized by reinforcement learning. The collaboration sub-model is used to generate a collaboration feature vector based on the consistency requirements of movement and harvesting, combined with a collaboration attention network, to characterize the collaboration consistency between the movement control instructions and the harvesting control instructions during execution.

[0011] Specifically, the multimodal fusion model includes an image extraction layer, a spectral extraction layer, an edge detection layer, a hardness assessment layer, and a linear assessment layer; it collects a first maturity index characterizing the maturity of *Agaricus bisporus* in the target region, including:

[0012] The configured RGB camera and short-wave near-infrared camera are synchronized for exposure time and spatially registered using FPGA hardware and Zhang's calibration method to establish a dual-sensor image coordinate system and a dual-sensor pixel transformation matrix; the dual-sensor pixel transformation matrix is ​​used to characterize the spatial transformation relationship between RGB image and short-wave near-infrared image at the same pixel point.

[0013] The RGB image sequence of *Agaricus bisporus* and the infrared image sequence of *Agaricus bisporus* cap folds were acquired in the dual-sensor image coordinate system, and the two types of images were spatially and temporally aligned.

[0014] The collected RGB image sequence of Twin Mushrooms is input into the image extraction layer of a preset multimodal fusion model to extract the color features of the cap and the first cap fold features.

[0015] Specifically, the primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes:

[0016] Simultaneously, the infrared image sequence of the umbrella surface wrinkles of the twin mushroom is input into a preset spectral extraction layer to extract the second umbrella surface spectral wrinkle features and water content features. At the same time, the first umbrella cover wrinkle features are mapped to the second umbrella surface spectral wrinkle features through a dual-sensor pixel conversion matrix to obtain the corrected second umbrella surface spectral wrinkle features and water content features.

[0017] Based on the corrected second umbrella surface spectral fold features combined with a preset edge detection layer, the gill opening characteristics of Mushroom taenia are obtained.

[0018] Consistency verification was performed based on the gill opening characteristics of *Agaricus bisporus* combined with the corresponding moisture content characteristics of *Agaricus bisporus* gills, and the verified gill opening of *Agaricus bisporus* was obtained.

[0019] Specifically, the primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes:

[0020] The thickness of the waxy layer on the cap of *Agaricus bisporus* was collected using a configured laser spectrometer, while the hollowness inside the stem was collected using millimeter-wave radar. The collected data was then input into a configured filter for filtering and time alignment.

[0021] The aligned wax layer thickness and the hollowness inside the mushroom stem are input into the hardness evaluation layer to obtain the hardness of the twin mushroom stem.

[0022] Environmental assessment parameters, including temperature, humidity, CO2 concentration, and light intensity, are obtained through configured environmental sensors.

[0023] Environmental correction coefficients are obtained by combining environmental assessment parameters with an environmental coefficient mapping table.

[0024] Specifically, the primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes:

[0025] Historical first maturity index and corresponding harvesting damage rate, as well as cap color characteristics, agaricus gill opening, agaricus stem hardness, and environmental correction coefficient were obtained. Factor analysis was conducted with harvesting damage rate as the first target factor and first maturity index as the second target factor. The contribution weights of cap color characteristics, agaricus gill opening, agaricus stem hardness, and environmental correction coefficient to the first target factor that minimize the first target factor were obtained and used as the maturity assessment weight sequence.

[0026] The cap color characteristics, gill opening of *Agaricus bisporus*, stem hardness of *Agaricus bisporus*, environmental correction coefficient, and maturity assessment weight sequence are input into the linear assessment layer to obtain the first maturity index of each *Agaricus bisporus* in the target area.

[0027] Specifically, obtaining the harvestable twin mushroom information sequence and the unharvested twin mushroom information sequence includes:

[0028] An initial identification path is preset, and the maturity of *Twinia foetida* is monitored and identified in real time in the target area along the initial identification path to obtain the real-time first maturity index of *Twinia foetida* at each location in the target area.

[0029] Based on the real-time first maturity index of the mushroom corresponding to each position and the preset maturity-shelf life curve, the real-time storage time of the mushroom corresponding to each position is obtained.

[0030] A preset maturity threshold is set. When the real-time first maturity index of the twin mushroom at a certain location in the target area is greater than the maturity threshold, the corresponding location is determined to be a harvestable twin mushroom; otherwise, it is a twin mushroom to be harvested. The harvestable twin mushroom information sequence and the twin mushroom to be harvested information sequence are obtained.

[0031] Specifically, the movement control commands include:

[0032] Based on the real-time first maturity index of the twin mushrooms at each location in the information sequence of the twin mushrooms to be harvested, combined with the growth prediction model optimized by preset environmental parameters, the harvestable prediction timestamp of the twin mushrooms at each location in the information sequence of the twin mushrooms to be harvested is obtained.

[0033] Based on the real-time storage time of the twin mushrooms at each location, a second harvesting priority sequence for twin mushrooms is constructed.

[0034] Using the first maturity index corresponding to the harvestable twin mushroom information sequence of the target area and the real-time storage time length, combined with an adaptive clustering algorithm, the region is clustered and segmented with the goal of minimizing the maturity deviation between pairs of twin mushrooms in the region and the deviation of the real-time storage time length, to obtain the harvestable twin mushroom cluster region sequence of the target area.

[0035] Specifically, the movement control commands also include:

[0036] Based on the first maturity index corresponding to the information sequence of twin mushrooms to be harvested and the harvestable prediction timestamp length of the twin mushrooms to be harvested at each location point, the region is clustered and segmented with the goal of minimizing the maturity deviation and harvestable prediction timestamp length deviation between any two pairs of twin mushrooms to be harvested in the region, and the clustering region sequence of twin mushrooms to be harvested in the target region is obtained.

[0037] Based on the clustering sequence of harvestable twin mushrooms and the clustering sequence of unharvested twin mushrooms in the target area, the first priority constraint is the maximum first harvesting priority constructed by combining the first maturity index and the comprehensive real-time storage time length. The second priority constraint is the minimum deviation between the harvestable prediction timestamp length and the waiting harvesting time length. The third priority constraint is the minimum real-time energy consumption of AGV carts and harvesting robots. Combined with the path planning algorithm of particle swarm optimization, a dynamic harvesting path parameter space is obtained.

[0038] Based on the dynamic picking path parameter space combined with fuzzy control algorithm, dynamic movement control commands are obtained.

[0039] A multimodal perception and dynamic decision-making intelligent greenhouse mushroom harvesting system includes: a data acquisition module, a discrimination module, a collaborative control module, and an execution module;

[0040] The acquisition module acquires a first maturity index characterizing the maturity of the mushroom in the target area through a configured sensor array. The first maturity index is obtained by combining the mushroom cap color characteristics, gill moisture content characteristics, cap surface wax layer thickness, stem hollowness, environmental correction coefficient, and expert evaluation algorithm collected by the sensors.

[0041] The discrimination module collects first discrimination based on the first maturity index and the maturity-shelf life curve to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp.

[0042] The collaborative control module generates collaborative control commands based on a coordinated reinforcement learning model that combines the information sequences of harvestable and unharvested twin mushrooms with a reward function constructed from factors such as shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy, and command coordination consistency delay.

[0043] The execution module is used to execute collaborative control instructions.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] This invention addresses the shortcomings of existing technologies by integrating multiple sensor arrays, including RGB, short-wave near-infrared, and laser spectrometers, with FPGA synchronization and Zhang's calibration to achieve precise data acquisition. Multimodal feature fusion and consistency verification improve the accuracy of maturity assessment, effectively reducing the false positive rate. Harvesting areas are divided based on adaptive clustering, and dynamic paths are planned using particle swarm optimization and multi-priority constraints to reduce unnecessary AGV movement and secondary harvesting, thus lowering energy consumption. A coordinated reinforcement learning model enables collaborative control of movement and harvesting; impedance control optimized by reinforcement learning reduces harvesting damage; and a collaborative sub-model ensures command synchronization, improving harvesting efficiency. The entire process achieves closed-loop intelligent management from maturity perception and harvesting decision-making to execution, adapting to the needs of large-scale greenhouse cultivation, significantly reducing post-harvest losses and operating costs, and improving harvesting quality and automation levels. Attached Figure Description

[0046] Figure 1 This is a flowchart of the intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making of the present invention.

[0047] Figure 2 This is a flowchart of the Bert model of the intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making of the present invention.

[0048] Figure 3 This is a structural diagram of the intelligent greenhouse mushroom harvesting system with multimodal perception and dynamic decision-making according to the present invention. Detailed Implementation

[0049] Example 1

[0050] Please see Figure 1 The present invention provides an embodiment of an intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making, comprising the following steps:

[0051] S1. Collect a first maturity index characterizing the maturity of the mushroom in the target area using a configured sensor array; the first maturity index is obtained by combining the mushroom cap color characteristics, gill moisture content characteristics, cap surface wax layer thickness, stem hollowness, environmental correction coefficient, and expert evaluation algorithm collected by the sensors.

[0052] S2. Based on the first maturity index and the maturity-shelf life curve, a first discrimination is performed to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp.

[0053] S3. Based on the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested, a coordinated reinforcement learning model is combined with a reward function constructed from shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy and instruction coordination consistency delay to generate coordinated control instructions.

[0054] S4. Execute the coordinated control command.

[0055] It should be further explained that the coordinated reinforcement learning model in this embodiment includes a harvesting control sub-model, a movement control sub-model, and a collaboration sub-model. The movement control sub-model is used to generate movement control instructions for the AGV vehicle based on the location information of each harvestable twin mushroom in the harvestable twin mushroom information sequence and the harvestable prediction timestamp of the twin mushroom to be harvested, combined with a path optimization algorithm. The harvesting control sub-model is used to generate harvesting control instructions for a three-finger flexible gripper based on the first maturity index, the maturity-harvesting intensity-damage probability mapping relationship, and an impedance control algorithm optimized by reinforcement learning. The collaboration sub-model is used to generate a collaboration feature vector based on the consistency requirements of movement and harvesting, combined with a collaboration attention network, to characterize the collaboration consistency between the movement control instructions and the harvesting control instructions during execution.

[0056] Further explanation is needed; please refer to [link / reference]. Figure 2 The multimodal fusion model in this embodiment includes an image extraction layer, a spectral extraction layer, an edge detection layer, a hardness assessment layer, and a linear assessment layer. It should be further noted that this embodiment collects a first maturity index characterizing the maturity of *Agaricus bisporus* in the target region, including:

[0057] The configured RGB camera and short-wave near-infrared camera are synchronized in exposure time and spatially registered using FPGA hardware and Zhang's calibration method to establish a dual-sensor image coordinate system and a dual-sensor pixel transformation matrix. The dual-sensor pixel transformation matrix is ​​used to characterize the spatial transformation relationship between the RGB image and the short-wave near-infrared image at the same pixel point. In this embodiment, the motivation for this operation is to solve the problem of asynchronous exposure time and spatial misalignment caused by the differences in hardware characteristics and installation positions of the RGB camera and the short-wave near-infrared camera when acquiring images. This ensures that the twin mushroom images acquired by the two sensors are accurately matched in time and space, providing reliable data support for subsequent multimodal feature fusion and accurate extraction of maturity-related features. The core reason is that asynchronous exposure using dual sensors creates a time difference between the RGB and infrared images of the same twin mushroom, failing to accurately reflect the mushroom's state at the same moment. Spatial misalignment leads to inconsistent pixel coordinates for the same physical point in the two images, directly affecting the correlation and matching of cross-modal features such as cap color and gill moisture content. FPGA hardware can generate high-precision synchronous trigger signals to achieve strict synchronization of exposure time, and Zhang's calibration method can solve for sensor intrinsic and extrinsic parameters, thereby establishing a unified image coordinate system and pixel transformation matrix. This enables precise spatial mapping of pixels from different sensor images, ensuring the effectiveness of multimodal data fusion and the accuracy of maturity assessment.

[0058] Furthermore, it should be noted that this embodiment is designed for high humidity scenarios (RH>95%) and employs a technical logic of sensor performance assurance, image data quality control, and dynamic optimization of harvesting decisions. The specific technical means and principles are as follows: Anti-fogging compensation is achieved by equipping the SWIR camera with a heating element. The principle is that high humidity environments easily cause fogging on the camera lens, leading to blurred infrared images. The heating element maintains the lens temperature above the environmental dew point, preventing fog formation and ensuring the clarity of the infrared image sequence of the mushroom cap folds. At the decision-making level, an adjustment strategy of reducing the CMI threshold by 5% and prioritizing harvesting the edge areas is adopted. The principle is that high humidity accelerates the maturation process of mushrooms and increases the risk of spoilage. Reducing the CMI threshold can appropriately relax the harvestable maturity standard, avoiding losses due to premature harvesting. The edge areas typically have different ventilation conditions than the central areas, making them more prone to abnormal maturation or spoilage under high humidity. Prioritizing harvesting further avoids post-harvest losses, achieving synergistic assurance of harvesting quality and efficiency in high humidity scenarios.

[0059] The dual-sensor image coordinate system acquires RGB image sequences and infrared image sequences of mushroom cap wrinkles, and performs spatial and temporal alignment processing on the acquired images. Furthermore, this embodiment dynamically switches the RGB / SWIR acquisition weights during day-night cycles. The principle is to adapt to the impact of periodic changes in light intensity on the imaging quality of the dual sensors, and the spatiotemporal alignment of the dual-sensor images ensures the stability and accuracy of multimodal feature extraction. During the day, a 7:3 weight allocation is set because, due to sufficient light, the RGB camera captures visual features such as mushroom cap color and surface texture with higher accuracy, dominating visual feature extraction. At night, a 3:7 weight adjustment is set because the reduced light intensity leads to a decrease in the signal-to-noise ratio and feature recognition of the RGB images, while the SWIR camera is less affected by natural light and can accurately extract deeper information such as gill moisture content and spectral characteristics of cap wrinkles, thus becoming the dominant feature extractor.

[0060] The collected RGB image sequence of Twin Mushrooms is input into the image extraction layer of the preset multimodal fusion model to extract the color features of the cap and the first cap fold features;

[0061] It should be further explained that the image extraction layer in this embodiment is preferably the MobileNetV3 lightweight network. The specific process of extracting the cap color features and the first cap wrinkle features from the RGB image sequence of *Agaricus bisporus* is as follows: First, the input *Agaricus bisporus* RGB image sequence is preprocessed, including image normalization, Gaussian denoising, and cropping of the region of interest in the cap area, removing background interference, and unifying the image size to adapt to the network input; the preprocessed RGB image frames are input into the MBV3 lightweight network, which utilizes its depthwise separable convolution and inverse residual structure to reduce computation and improve inference efficiency. The shallow convolutional kernels of the network capture the low-dimensional visual features of the cap color, and the SE attention mechanism is combined to enhance the color. The weighting of color channel features accurately extracts core features representing the maturity of the canopy, such as color mean, hue distribution, and color gradient. Simultaneously, leveraging the network's deep convolutional layers' ability to perceive local image texture, the network focuses on the wrinkled areas of the canopy surface, extracting structural features such as the edge contour, texture density, and wrinkle depth of the wrinkles as the first canopy wrinkle feature. During this process, the network's built-in BatchNorm layer and activation function enhance the non-linear expression of features. Finally, the features extracted from multiple frames are temporally fused and normalized, outputting a unified, highly recognizable canopy color feature vector and a first canopy wrinkle feature vector, providing high-quality visual feature support for subsequent cross-modal feature fusion.

[0062] Simultaneously, the infrared image sequence of the mushroom cap wrinkles is input into a preset spectral extraction layer to extract the second cap spectral wrinkle features and water content features. The first cap wrinkle features are mapped to the second cap spectral wrinkle features using a dual-sensor pixel transformation matrix to obtain the corrected second cap spectral wrinkle features and water content features. It should be further explained that the specific process of extracting and correcting the second cap spectral wrinkle features and water content features using one-dimensional convolution in this embodiment is as follows: Preprocessing operations are performed on the mushroom cap wrinkle infrared image sequence, including spectral dimension noise reduction, temporal frame alignment, and 1450nm water content feature peak band extraction. This unifies the sequence data format and removes environmental noise and invalid spectral information. The preprocessed infrared image sequence is expanded into a one-dimensional spectral vector based on pixel-level temporal spectral data and input into the preset spectral extraction layer. This spectral extraction layer uses a one-dimensional convolution structure, capturing the spectral response differences characterizing the cap wrinkle structure in the spectral dimension through multi-scale convolution kernels. The method generates second umbrella-surface spectral wrinkle features and extracts spectral parameters such as intensity, full width at half maximum (FWHM), and peak area of ​​the 1450nm feature peak to generate water content features. The nonlinear expression of the effective features is enhanced by pooling layers and activation functions, and dimensionality is compressed. First umbrella-surface wrinkle features extracted from the RGB image sequence of *Tetracentron sinense* are input into a dual-sensor pixel transformation matrix. Matrix operations map the spatial coordinates and feature dimensions of the first umbrella-surface wrinkle features to the pixel coordinate system of the infrared image, achieving precise spatial alignment between the first and second umbrella-surface spectral wrinkle features. Based on the aligned first umbrella-surface wrinkle features, a feature weighted fusion strategy is used to correct the second umbrella-surface spectral wrinkle features. The structural information in the first umbrella-surface wrinkle features supplements the deviation in the spectral features' wrinkle morphology representation. Simultaneously, the water content features are synchronously calibrated by combining the physiological correlation between the corrected second umbrella-surface spectral wrinkle features and the water content features. Finally, the corrected second umbrella-surface spectral wrinkle feature vector and the water content feature vector are output.

[0063] Based on the corrected second umbrella-surface spectral wrinkle features combined with a preset edge detection layer, the gill opening characteristics of *Agaricus bisporus* are obtained. It should be further noted that obtaining the gill opening characteristics of *Agaricus bisporus* in this embodiment includes:

[0064] Based on the corrected second umbrella-shaped spectral wrinkle features, a target feature map is obtained by using a feature map reconstruction algorithm and normalization enhancement processing to enhance the difference between the gill region and the background features. Based on the target feature map, a preset edge detection layer calls an edge detection algorithm to perform preliminary gill edge extraction. Noise interference edges are filtered by setting an adaptive threshold to obtain preliminary edge extraction results. Based on the preliminary edge extraction results, edge connection and isolated noise point removal are achieved through morphological dilation and erosion operations to obtain optimized gill edge contours. Based on the spatial coordinate information corresponding to the dual-sensor pixel transformation matrix, the optimized gill edge contours are used for region localization to obtain the edge contour of the target region where the gills of *Agaricus bisporus* are located. Based on the edge contour of the target region, the closed contour curve of the gill edge is obtained by using an edge fitting algorithm. Geometric parameters such as the angle between the two endpoints of the contour curve and the center of the gill, the pixel area ratio of the open region, and the unfolded length of the contour are calculated. After normalization processing of the geometric parameters, the gill opening feature, which characterizes the degree of opening of *Agaricus bisporus* gills, is obtained.

[0065] Consistency verification was performed based on the gill opening characteristics of *Agaricus bisporus* combined with the corresponding moisture content characteristics of the gills, resulting in the verified gill opening of *Agaricus bisporus*. It should be further noted that the consistency verification process in this embodiment includes:

[0066] Based on historically collected gill opening characteristics of *Agaricus bisporus*, moisture content characteristics corresponding to the 1450nm characteristic peak detected by a short-wave near-infrared camera, and maturity labels, a dynamic correlation model between gill opening and moisture content at different maturity stages is constructed using a nonlinear regression algorithm to obtain the threshold range of the standard mapping relationship between the two. Based on the current gill opening characteristics of *Agaricus bisporus* extracted and optimized using an edge detection algorithm through an edge detection layer, combined with the moisture content characteristics of the 1450nm characteristic peak extracted from the short-wave near-infrared spectrum at the same time, an absolute deviation value is obtained between the actual values ​​of both and the standard mapping relationship of the corresponding maturity stages in the dynamic correlation model through a deviation calculation model. Based on a preset consistency verification threshold, it is determined whether the absolute deviation value is within the acceptable range. Within the allowable range; if the deviation value is within the allowable range, the current gill opening feature is directly used as the verified gill opening value of *Agaricus bisporus*; if the deviation value exceeds the allowable range, the theoretical gill opening value adapted to the current moisture content feature is derived in reverse based on the dynamic correlation model, and the deviation weighted correction algorithm is used to adjust it in combination with the current actual gill opening value feature. The weighting weight is dynamically allocated according to the magnitude of the deviation value and the historical verification accuracy. At the same time, the correction result is verified a second time based on the integrity and continuity features of the gill edge contour extracted by the edge detection layer, and abnormal opening value data caused by physical damage are removed. Finally, the verified gill opening value that conforms to the physiological law of maturation of *Agaricus bisporus* (gill opening driven by the decrease of gill moisture content) is obtained.

[0067] It should be further explained that in this embodiment, the water content of the gills at the characteristic peak of 1450nm is detected using a short-wave near-infrared camera. The core principle and motivation of this method is to directly monitor the key physiological process driving the maturation of *Agaricus bisporus*: in preparation for spore release, the transpiration of the gills is significantly enhanced, leading to a specific decrease in water content. This water loss directly reduces the turgor pressure of the gill cells, thereby physically inducing the gills to open from a closed state. Therefore, the water content detection aims to accurately quantify this intrinsic physiological driving force that leads to morphological changes. Its continuous downward trend constitutes a quantifiable early maturation signal that precedes the visible morphological changes. The gill opening degree obtained by edge detection based on the same SWIR image is a direct observation and measurement of the final external morphological result caused by the above physiological process. The gill water content detected by short-wave near-infrared spectroscopy and the gill opening degree recognized by image recognition constitute a deep collaborative verification system from intrinsic physiological driving force to external morphological performance. The synergistic mechanism is manifested in the following ways: Moisture content serves as a leading causal indicator. For example, when the gill moisture content is monitored to continuously decrease from 85% to 78%, even if the gill morphology has not yet changed significantly, the system can predict that it will enter the opening stage within 2 hours based on this physiological trend, thus achieving forward-looking decision-making. Opening angle, as a morphological verification indicator, directly confirms the final expression of the physiological process. For example, when the gill opening angle is detected to be greater than 45 degrees, it corroborates the already low moisture content data, jointly confirming the maturity state. This two-way verification is particularly important: if the system finds that the gills have opened but the moisture content is still high, it can determine that the opening originates from physical damage rather than natural maturation, thus avoiding mis-collection; conversely, if the moisture content has decreased significantly but the opening angle is insufficient, it suggests that there may be environmental inhibition, requiring adjustment of microenvironment parameters. This dual-modal perception closed loop of intrinsic physiology and extrinsic morphology, through real-time data cross-validation, significantly improves the predictive foresight, judgment accuracy, and decision robustness of maturity assessment.

[0068] The thickness of the waxy layer on the cap of *Agaricus bisporus* was collected using a configured laser spectrometer, while the hollowness inside the stem was collected using millimeter-wave radar. The collected data was then input into a configured filter for filtering and time alignment.

[0069] The aligned wax layer thickness and the hollowness of the mushroom stem are input into the hardness evaluation layer to obtain the hardness of the twin mushroom stem; it should be further noted that, in this embodiment, obtaining the hardness of the twin mushroom stem includes:

[0070] Based on the filtered and time-aligned data of the waxy layer thickness on the cap of *Agaricus bisporus* and the hollowness of the stem, a standardization process is first performed using a Min-Max normalization algorithm to obtain standardized waxy layer thickness and stem hollowness features. Based on these standardized features, a weighted fusion strategy with weights determined by historical data factor analysis is used to perform feature fusion, obtaining a fused feature vector encompassing the waxy layer state and the internal structure of the stem. Based on this fused feature vector, a preset hardness assessment layer is input. This layer incorporates a gradient boosting tree model trained and optimized using historical waxy layer thickness, stem hollowness, and corresponding measured stem hardness labels. The model inference establishes a mapping relationship between the fused features and stem hardness, obtaining a preliminary stem hardness value. Based on preset hardness calibration coefficients for different maturity stages of *Agaricus bisporus*, the preliminary stem hardness value is dynamically calibrated, ultimately obtaining a precise stem hardness value that accurately characterizes the firmness of the *Agaricus bisporus* stem.

[0071] Environmental assessment parameters, including temperature, humidity, CO2 concentration, and light intensity, are obtained through configured environmental sensors.

[0072] Environmental correction coefficients are obtained based on environmental assessment parameters and an environmental coefficient mapping table.

[0073] It should be further explained that, in this embodiment, an environmental sensor array covering temperature, humidity, CO2 concentration, and light intensity monitoring is configured to collect environmental assessment parameters of the target area in real time and perform data preprocessing. The preprocessing includes outlier removal, time-series alignment, and numerical standardization. An environmental coefficient mapping table is pre-constructed based on historical environmental assessment parameter samples, corresponding deviation data of mushroom maturity assessment under the samples, and industry expert experience. This mapping table is divided into multiple continuous intervals according to the numerical range of temperature, humidity, CO2 concentration, and light intensity, with each interval corresponding to a unique basic correction coefficient. The mapping table also associates the influence weight of each environmental parameter on maturity assessment, and the weights are determined through historical data factor analysis. The preprocessed real-time environmental assessment parameters are matched to the corresponding numerical intervals in the environmental coefficient mapping table to obtain the basic correction coefficients corresponding to each environmental assessment parameter. Based on the preset influence weights of each environmental parameter, the obtained basic correction coefficients are weighted and summed to obtain the final environmental correction coefficient. This environmental correction coefficient is used to dynamically calibrate the mushroom maturity assessment results to eliminate the interference of environmental factors on the accuracy of maturity assessment.

[0074] This method acquires historical first maturity indicators and their corresponding harvesting damage rates, as well as cap color characteristics, *Agaricus bisporus* gill opening, *Agaricus bisporus* stem hardness, and environmental correction coefficients. Factor analysis is performed with harvesting damage rate as the first target factor and the first maturity indicator as the second target factor. The contribution weights of cap color characteristics, *Agaricus bisporus* gill opening, *Agaricus bisporus* stem hardness, and environmental correction coefficients that minimize the first target factor are obtained and used as the maturity assessment weight sequence. It should be further explained that the core motivation for using harvesting damage rate as the first target factor and the first maturity indicator as the second target factor in this embodiment is to adapt to the core requirements of large-scale harvesting of *Agaricus bisporus* in intelligent greenhouses, namely, prioritizing post-harvest quality, reducing losses, and ensuring the accuracy of maturity assessment. The core principle is that harvesting damage rate directly determines the post-harvest quality and economic value of *Agaricus bisporus*. Excessive damage rate leads to mushroom rot and decreased marketability, which is the primary constraint affecting harvesting efficiency. The first maturity indicator is the basis for judging the harvesting timing and needs to be accurately assessed under the premise of low damage. Factor analysis was used to explore the correlation between features such as cap color and gill opening and two target factors. The contribution weight of each feature was obtained with minimizing the damage rate as the primary constraint. This allows the subsequent maturity assessment process to allocate weights more favorably to feature dimensions that can reduce harvesting damage. This ensures that the maturity assessment results are consistent with the actual growth state and avoids high damage risks in advance from the perspective of indicator weight design. It achieves synergistic optimization of accurate maturity assessment and post-harvest quality assurance, which meets the dual requirements of efficiency and quality for large-scale harvesting.

[0075] The cap color characteristics, gill opening of *Agaricus bisporus*, stem hardness of *Agaricus bisporus*, environmental correction coefficient, and maturity assessment weight sequence are input into a linear assessment layer to obtain the first maturity index for each *Agaricus bisporus* within the target area. It should be further noted that the linear assessment layer in this embodiment is configured with a linear weighted summation algorithm for assessment calculations.

[0076] It should be further explained that, in this embodiment, obtaining the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested includes:

[0077] An initial identification path is preset, and the maturity of *Tricholoma matsutake* is monitored and identified in real time along the initial identification path in the target area. The real-time first maturity index of *Tricholoma matsutake* at each location in the target area is obtained. It should be further noted that the construction process of the initial identification path in this embodiment includes:

[0078] Based on prior planting information such as the geometric dimensions of the target area in the greenhouse, the spacing between rows of mushroom cultivation, and the planting density, combined with equipment constraint parameters such as the detection range of the configured sensor array, the field of view, and the turning radius of the mobile platform, a grid map of the planting layout of the target area is constructed using a grid map modeling method. Based on this grid map, a serpentine path planning algorithm is adopted, with the optimization objectives of no detection blind spots in the target area, the shortest total path length, and the uniform distribution of detection points, to plan and generate a continuous and coherent path trajectory. At the same time, the grid coordinates of the path trajectory are correlated and calibrated with the physical coordinates of the target area, and finally a preset initial recognition path adapted to the planting layout and equipment performance is obtained.

[0079] Based on the real-time first maturity index of the mushrooms at each location and a preset maturity-storage period curve, the real-time storage time of the mushrooms at each location is obtained. It should be further explained that the core motivation of the maturity-storage period curve in this embodiment is to establish a quantitative correlation between the maturity of mushrooms and the post-harvest storage time, providing a basis for harvesting sequence planning to reduce post-harvest over-ripening losses. The construction method is based on historically collected storage time data of mushrooms at different maturity levels under standard storage conditions, combined with corrections for environmental factors such as temperature and humidity. A nonlinear mapping model between maturity and storage time is established through polynomial fitting or regression analysis, and the curve parameters are determined after multiple verifications and optimizations. Its function is to quickly query or calculate the corresponding real-time storage time based on the real-time first maturity index of the mushrooms at each location, providing key data support for subsequent identification of harvestable and unharvested mushrooms and the construction of a second harvesting priority sequence, ensuring that harvesting decisions align with post-harvest quality assurance requirements.

[0080] A preset maturity threshold is set. When the real-time first maturity index of the twin mushroom at a certain location in the target area is greater than the maturity threshold, the corresponding location is determined to be a harvestable twin mushroom; otherwise, it is a twin mushroom to be harvested. The harvestable twin mushroom information sequence and the twin mushroom to be harvested information sequence are obtained.

[0081] It should be further explained that the movement control commands in this embodiment include:

[0082] Based on the real-time first maturity index of the twin mushrooms at each location in the information sequence of the twin mushrooms to be harvested, combined with a growth prediction model optimized by preset environmental parameters, a harvestable prediction timestamp is obtained for each location in the information sequence of the twin mushrooms to be harvested. The growth prediction model optimized by environmental parameters is used to dynamically predict the harvestable prediction time length required for each location in the information sequence of the twin mushrooms to be harvested to grow from the current first maturity index to the harvestable maturity threshold. Specifically, this includes: based on a multi-sensor network deployed in the greenhouse, real-time collection of environmental parameters such as temperature, humidity, carbon dioxide concentration, and light intensity is performed, and a time-series dataset of environmental parameters with timestamps is constructed. Based on this dataset, combined with historically accumulated continuous monitoring data of the maturity index of twin mushrooms, a dynamic growth rate prediction model is constructed and trained using a Long Short-Term Memory (LSTM) network. This model uses the aforementioned time-series environmental parameters as input. The output is the instantaneous growth rate of the maturity of the twin mushrooms at the next time step. Based on this, the trained LSTM model and the current first maturity index of each individual in the information sequence of twin mushrooms to be harvested are used to deduce the maturity evolution trajectory of each individual under the influence of the predicted environmental parameter change sequence in the future. Based on this maturity evolution trajectory, the numerical integration method is used to calculate the total time required for the maturity index to accumulate from the current value to the preset harvestable maturity threshold, thereby obtaining the harvestable prediction timestamp of each individual twin mushroom to be harvested, accurate to the minute level. Finally, based on the harvestable prediction timestamp obtained by this calculation, combined with the absolute position coordinates of the twin mushroom, the corresponding entries in the information sequence of twin mushrooms to be harvested are dynamically updated, providing accurate time window constraints and decision-making basis for the global path planning algorithm of the automated guided mobile platform and the task queuing system of the six-degree-of-freedom collaborative robot, realizing the optimal allocation of harvesting resources in the spatiotemporal dimension.

[0083] Based on the real-time storage time of the twin mushrooms corresponding to each location, a second harvesting priority sequence for twin mushrooms is constructed. It should be further explained that in this embodiment, the second harvesting priority sequence is a ranking list established for each individual twin mushroom within the target area, based on the urgency of its remaining shelf life. Specifically, based on the real-time storage time (i.e., the predicted remaining shelf life) of each twin mushroom determined by the maturity-shelf-life curve, all twin mushrooms already determined to be harvestable are sorted in ascending order of this time length to generate the second harvesting priority sequence. The twin mushroom with the shortest real-time storage time receives the highest harvesting priority because it is closest to its spoilage period. This sequence provides a crucial decision-making basis for the real-time scheduling of the harvesting system, ensuring that the system prioritizes processing individuals with the most urgent shelf life, thereby effectively reducing post-harvest losses due to over-ripening globally.

[0084] Using the first maturity index corresponding to the harvestable twin mushroom information sequence of the target area and the real-time storage time length, combined with an adaptive clustering algorithm, the region is clustered and segmented with the goal of minimizing the maturity deviation and real-time storage time length deviation between any two twin mushrooms in the region, thereby obtaining the harvestable twin mushroom cluster region sequence of the target area.

[0085] Based on the first maturity index corresponding to the information sequence of twin mushrooms to be harvested and the harvestable prediction timestamp length of the twin mushrooms to be harvested at each location point, the region is clustered and segmented with the goal of minimizing the maturity deviation and harvestable prediction timestamp length deviation between any two pairs of twin mushrooms to be harvested in the region, and the clustering region sequence of twin mushrooms to be harvested in the target region is obtained.

[0086] The clustering method in this embodiment aims to plan harvesting units with consistent operational logic and optimal execution efficiency for the mobile harvesting platform, rather than simply performing geometric spatial segmentation. For harvestable twin mushrooms, clustering is performed with the goal of minimizing the deviation in maturity and the deviation in real-time storage time. This ensures that mushroom groups divided into the same area not only have similar maturity states, facilitating the unified configuration of harvesting parameters such as gripping force for the robot, but more importantly, their freshness urgency is highly consistent, i.e., their storage time is similar. This allows the AGV to perform continuous, uninterrupted harvesting operations in that area in one go, avoiding forced reversals due to premature spoilage of individual mushrooms, thereby maximizing harvesting efficiency and minimizing post-harvest losses. Similarly, for twin mushrooms to be harvested, if the goal is to minimize the deviation between maturity deviation and the predicted harvestable timestamp length, then mushrooms expected to reach harvestable standards within the same time period can be divided into the same future work area. This allows the system to plan an efficient and predictive inspection path for the AGV, enabling it to arrive at each harvestable area sequentially within a precise time window, thereby significantly reducing the robot's idle waiting and ineffective movement, and achieving a decision-making upgrade from single-point response to regionalized batch processing.

[0087] The core motivation for performing the above two clustering methods in this embodiment stems from a deep understanding of the non-uniform distribution of *Tricholoma matsutake* growth in actual planting scenarios. The principle is based on clustering optimization using the dual principles of task type isolation and spatial continuity maintenance. Specifically, within a single physical area, harvestable and unharvested *Tricholoma matsutake* are randomly distributed in space. If mixed clustering is performed, the resulting clustered areas will be highly fragmented spatially and unclear in terms of task logic due to the fundamental difference in the decision-making objectives of the two types of mushrooms (immediate harvesting versus anticipated harvesting). This would cause the execution paths of the mobile platform and robot to be filled with ineffective round trips and waiting, severely reducing harvesting efficiency. Therefore, this embodiment adopts a virtual space segmentation method from a task perspective: when clustering for harvestable tasks, the algorithm temporarily marks and hides adjacent unharvested mushrooms in physical location as spatial noise points during calculation, allowing the clustering algorithm to focus on spatially identifying hotspot areas composed of highly mature mushrooms with continuous harvesting value; conversely, the same applies when clustering for unharvested tasks. The key advantage of this method lies in its ability to construct two independent, spatially continuous, and highly unified virtual region sequences at the task decision level without altering the actual geographical coordinates of the mushrooms. Ultimately, this allows the path planning algorithm to generate two logically clear and spatially coherent optimal operating paths for the AGV and robot: one for efficiently clearing currently mature areas, and the other for proactively inspecting areas that are about to mature, thereby maximizing harvesting efficiency in complex, unstructured environments.

[0088] Based on the clustering sequence of harvestable twin mushrooms and the clustering sequence of unharvested twin mushrooms in the target area, the first priority constraint is the maximum first harvesting priority constructed by combining the first maturity index and the comprehensive real-time storage time length. The second priority constraint is the minimum deviation between the harvestable prediction timestamp length and the waiting harvesting time length. The third priority constraint is the minimum real-time energy consumption of AGV carts and harvesting robots. Combined with the path planning algorithm of particle swarm optimization, a dynamic harvesting path parameter space is obtained.

[0089] It should be further explained that the comprehensive first maturity index and comprehensive real-time storage time in this embodiment are obtained by weighting the first maturity index and the corresponding real-time storage time of all the twin mushrooms in each harvestable twin mushroom cluster area, and are used to characterize the comprehensive harvesting priority of each cluster area; the waiting time for harvesting here is the waiting time from the current harvestable twin mushroom cluster area to any subsequent harvestable twin mushroom cluster area.

[0090] The core logic of this embodiment, which sets up the above three-level priority constraints and carries out dynamic harvesting path planning, addresses the core pain points in the large-scale harvesting scenario of twin mushrooms in large greenhouses, such as the imbalance of harvesting time, high energy consumption of secondary operations, and insufficient utilization rate of mature twin mushrooms to be harvested. Based on the growth physiological characteristics of twin mushrooms and the operation rules of intelligent equipment, a closed-loop system of priority quantitative modeling, time dynamic matching, and algorithm iterative optimization is constructed. Through the precise coupling of technical means, the optimal synergy of harvesting quality, efficiency, and cost is achieved. At the technical implementation level, firstly, for the clustered areas of harvestable and unharvested button mushrooms, a weighted summation algorithm is used to quantify the comprehensive first maturity index of each clustered area. The weight coefficients are determined through factor analysis of historical harvesting data. With the goal of minimizing the harvesting damage rate, the contribution weights of cap color characteristics, gill opening, stem hardness, and environmental correction coefficient are selected. Simultaneously, the comprehensive real-time storage time of each clustered area is combined to construct the first priority constraint. An initial harvesting sequence is generated through index normalization processing and sorting operations. The core purpose is to ensure that clustered areas with high maturity and low storage redundancy are harvested first, avoiding quality loss caused by over-maturation and decay of button mushrooms or insufficient storage period. Secondly, relying on the growth prediction model optimized by environmental parameters, the harvestable prediction timestamps of each cluster of areas to be harvested are output. The waiting time length from the current time to the planned harvest time of the area is obtained by the timestamp difference calculation. An absolute error or mean square error algorithm is used to construct a deviation evaluation model. Minimizing the deviation between the harvestable prediction timestamp and the waiting time length is taken as the second priority constraint and embedded in the dynamic control process of the initial harvest sequence. Technically, by synchronizing the output results of the growth prediction model with the waiting time parameters in the path planning in real time, the waiting period is accurately matched with the growth period of the mushrooms to be harvested. This promotes the batch harvesting of mushrooms in the area to reach the harvestable standard, fundamentally avoiding secondary harvesting operations in the same physical space and improving the batch and continuity of harvesting. Finally, the first two priority constraints are embedded as boundary conditions into the particle swarm optimization algorithm. Using the real-time energy consumption of the AGV and the harvesting robot as the fitness function, the particle swarm characterizes the dynamic harvesting path parameter space. Combined with the correlation model between the intelligent equipment's energy consumption characteristic curve and path distance and operation time, the harvesting order of clustered regions and the equipment's trajectory are optimized through iterative updates of particle position and velocity. Technically, by integrating the path planning node weight allocation mechanism with energy consumption model parameter calibration, global optimization of the equipment's operating trajectory is achieved. This maximizes the reduction of ineffective energy consumption and operating costs while meeting harvesting priority and time matching requirements. The overall process employs a progressive technical path of indicator quantification modeling, dynamic time matching, and algorithm iterative optimization. This ensures the priority harvesting of high-value harvestable twin mushrooms, improves the maturity utilization rate of unharvested twin mushrooms through time matching, and achieves precise energy consumption control through algorithm optimization, providing efficient, stable, and economical technical support for the large-scale harvesting of twin mushrooms in large greenhouses.

[0091] Based on the parameter space of each dynamic picking path combined with the fuzzy control algorithm, dynamic movement control commands are obtained.

[0092] It should be further explained that the harvesting control sub-model in this embodiment is used to generate harvesting control instructions for a 3-finger flexible gripper based on the information of each harvestable twin mushroom cluster region obtained from the dynamic harvesting path parameter space and the second harvesting priority of twin mushrooms at each location point within the region, combined with the first maturity index of each location point. The specific implementation is as follows:

[0093] The first step is to determine the harvesting order of each cluster region and the second harvesting priority of specific locations within each cluster region based on the results output by the dynamic harvesting path planning, thereby forming an overall harvesting sequence plan. Here, the cluster region refers to a set of twin mushrooms divided according to spatial location and maturity similarity, and the second harvesting priority refers to the harvesting order of each mushroom within the region after the region order is determined, in order to optimize the movement path and avoid obstacles.

[0094] The second step involves calculating the initial picking force and the corresponding damage probability threshold for each target twin mushroom in the time-series planning, based on the preset mapping relationship between maturity, picking force, and damage probability, combined with the first maturity index of the mushroom. This initial force is then set as the input parameter of the impedance control algorithm. The first maturity index refers to the maturity level obtained through comprehensive evaluation by the multimodal sensing system.

[0095] The third step is to construct a reinforcement learning reward function with the goals of reducing the probability of damage, shortening the harvesting time, and reducing the energy consumption of the gripper. The clamping force, displacement deviation, and first maturity index output in real time by the impedance control algorithm are used as state inputs. A set of suitable impedance control stiffness coefficients and damping coefficients are obtained through iterative optimization.

[0096] The fourth step, during the harvesting process, involves using the force sensor built into the three-finger flexible gripper to collect the contact force signal with the mushroom body in real time. Combined with the safety threshold of the elastic modulus of the twin mushroom body, the stiffness and damping coefficient are dynamically adjusted through an optimized impedance control algorithm. The displacement increment, upper limit of safe clamping force, and movement speed parameters of each finger of the gripper are calculated and output in real time.

[0097] Fifth, based on the displacement increment of each finger of the gripper, the upper limit of the safe clamping force, and the motion speed parameters, combined with a fuzzy control algorithm, a customized harvesting control command is generated for the target mushroom, driving the gripper to complete the harvest. After completing the operation in the current cluster area according to the time sequence plan, it automatically switches to the next area and repeats steps two through five until all targets are harvested in an orderly manner. In this embodiment, the stiffness coefficient is set at 500-800 N / m, and the clamping force is set at 2.5-5 N.

[0098] The motivation behind the collaborative sub-model design in this embodiment is to address the core issues of low harvesting efficiency, high mushroom damage rate, and increased equipment energy consumption caused by asynchronous execution of movement control commands and poor coordination between harvesting control commands and movement control commands. The core principle is based on cross-modal temporal feature fusion and an attention mechanism to accurately capture the collaborative correlation between AGV movement and gripper harvesting actions, constructing a quantified collaborative consistency representation, and forming a closed loop of command execution, collaborative evaluation, and feedback optimization. This ensures the temporal matching and state adaptation of the two actions. Specifically, the technical implementation involves: obtaining multi-dimensional temporal feature sequences of movement and harvesting based on AGV path planning information obtained from the dynamic harvesting path parameter space, operational parameters such as real-time position, speed, and estimated arrival time output by the movement control sub-model, and execution parameters such as gripper clamping force, displacement, and harvesting progress output by the harvesting control sub-model. These sequences are then synchronized using a timestamp alignment algorithm. The aligned cross-modal temporal feature sequences are input into a collaborative attention network, using movement state features as query vectors and harvesting state features as key-value vectors. Attention weights are used to calculate and focus on key parameters affecting coordination, such as the time deviation of the AGV reaching the target area. The system achieves deep cross-modal feature fusion by considering factors such as the matching degree with the gripper's ready state, the time difference between the gripper's action initiation and the AGV's positioning completion, and the adaptability of the AGV's movement speed to the gripper's picking rhythm. Based on the fused features, a collaborative feature vector is output through a fully connected layer and normalization processing. This vector encompasses quantitative dimensions such as time synchronization deviation coefficient, position matching accuracy, and action coordination efficiency, accurately representing the coordination consistency of the movement and picking command execution processes. The collaborative feature vector is fed back to the reward function of the coordinated reinforcement learning model, and the reward weight is dynamically adjusted in conjunction with indicators such as the picking damage rate and energy consumption during the shelf life. If the collaborative feature vector shows timing deviation or position misalignment, the reward value is reduced. Simultaneously, the collaborative feature vector is input into the movement control sub-model and the picking control sub-model respectively, to correct the timing of AGV movement speed adjustment, positioning accuracy compensation parameters, gripper action initiation timing, and gripping force adjustment rhythm in real time. This achieves dynamic collaborative optimization of movement and picking commands, ensuring that the gripper starts picking in a timely manner after the AGV is accurately positioned, and that the AGV quickly switches to the next area after picking, improving the overall coordination, stability, and efficiency of the harvesting process.

[0099] It should be further explained that this embodiment assigns a unique QR code to each pair of mushrooms or harvest batch. This QR code uses a dynamic generation algorithm, combined with the harvest timestamp, greenhouse area coordinates, and batch number, to ensure uniqueness. The QR code stores information including key data from the entire harvesting process, specifically covering the first maturity index of the pair of mushrooms at different growth stages, and basic data such as cap color characteristics, gill opening characteristics, stem hardness, and environmental correction coefficients that constitute this index; harvesting control command parameters during the harvesting process, including the clamping force, stiffness damping coefficient, and harvesting sequence of the three-finger flexible gripper; collaborative control execution data, including AGV movement speed, positioning accuracy, command synchronization deviation, and collaborative feature vectors; and post-harvest quality inspection data, covering multi-dimensional indicators such as mushroom integrity, gill damage, storage time, and decay status. In each stage of post-harvest sorting, storage, and sales, post-harvest quality data of corresponding nodes is collected in real time through barcode scanning devices. After the collected data is processed by edge computing nodes for format standardization, noise filtering, and time sequence alignment, it is uploaded to the model management system through an encrypted transmission protocol. It is then matched and integrated with the full-process data associated with the QR code to build a closed loop of traceable data across the entire chain from planting perception and harvest execution to post-harvest quality.

[0100] It should be further explained that this embodiment also includes a model management subsystem, which is equipped with a timed task scheduling module. The incremental update process is automatically triggered every morning at midnight. Before the update, the system first filters and cleans the newly added data for the day. Based on the post-harvest quality data traceability results, it selects accurate maturity assessment cases and cases with assessment deviations. Accurate cases must meet the requirement that the first maturity index matches the actual post-harvest quality better than a preset threshold, while cases with deviations must clearly explain the reasons for the assessment deviation and the corresponding influencing factors. From the filtered data, RGB images and short-wave near-infrared images of *Agaricus bisporus*, along with corresponding multimodal feature data such as cap color characteristics, gill opening, moisture content characteristics, and stem hardness, are extracted to construct the daily labeled samples. The number of labeled samples is strictly controlled to within one hundred to balance model update efficiency and stability, avoiding excessive update time and overfitting risks caused by excessive samples. The sample labeling process adopts a semi-automated approach combined with manual verification to ensure labeling accuracy.

[0101] It should be further explained that this embodiment uses a transfer learning algorithm for online incremental updates. Specifically, for the multimodal fusion model, the core parameters of the bottom convolutional layers and the one-dimensional convolution of the spectral extraction layer of the MobileNetV3 lightweight network are frozen, and only the parameters of the top fully connected sublayer of the image extraction layer, the feature fusion sublayer of the spectral extraction layer, the edge detection layer, and the weights of the linear evaluation layer are fine-tuned. For the coordinated reinforcement learning model, the parameters of the maturity-harvesting intensity-damage probability mapping relationship and the weights of the reinforcement learning reward function in the harvesting control sub-model, the attention allocation coefficients of the collaborative attention network in the collaborative sub-model, and the path planning-related parameters in the movement control sub-model are fine-tuned. For the growth prediction model with optimized environmental parameters, the hidden layer parameters and output layer weights of the LSTM network are fine-tuned. During the update process, the core parameters and training results of the historical model are retained, and the model adaptation parameters are dynamically adjusted only based on the newly labeled samples. After the update is completed, the updated model is validated using the model performance evaluation module with metrics such as accuracy, recall, and the reduction in damage rate. If the verification is successful, the updated model parameters will be deployed to the actual application system; if it fails, the process will return to adjust the sample selection criteria or model update parameters and re-execute the update process.

[0102] This embodiment embeds a PDCA (Plan-Do-Check-Act) cycle mechanism throughout the process: In the planning phase, daily incremental update targets are set based on quality shortcomings and model performance bottlenecks identified in the traceability data; in the execution phase, sample construction, model updates, and preliminary validation are completed; in the inspection phase, changes in core indicators such as maturity assessment accuracy, harvest damage rate, and control command coordination are compared before and after the update, and the correlation between post-harvest quality improvement and model parameter adjustments is analyzed; in the processing phase, validated model update results are solidified into standard parameter configurations, and front-end sensor acquisition strategies and feature extraction algorithms are optimized to address data acquisition vulnerabilities or insufficient feature extraction discovered during the update process. Simultaneously, typical deviation cases and solutions are incorporated into the knowledge base to support subsequent model optimization. Through this approach, continuous iterative optimization of the model and full-chain traceability of post-harvest quality are achieved, ensuring that the maturity assessment accuracy, control command coordination, and post-harvest quality assurance capabilities of the harvesting system dynamically improve with the application scenario.

[0103] This method for harvesting twin mushrooms integrates multimodal sensing technology, coordinated reinforcement learning, and dynamic path planning to achieve synergistic optimization of harvesting accuracy, efficiency, and quality. Its core benefits include a significant reduction in harvesting damage rate, effective extension of post-harvest shelf life, precise control of overall energy consumption, and continuous enhancement of system adaptability. The specific effects are derived from the precise data acquisition and fusion of a multi-sensor array. RGB and short-wave near-infrared cameras are synchronized via FPGA hardware and spatially registered using Zhang's calibration method, resolving the problem of spatiotemporal misalignment of cross-modal images. Combined with a dynamic day / night weight switching mechanism, the extraction stability of cap color features and gill moisture content features under different lighting conditions is ensured. Furthermore, visual and spectral features are extracted through the image extraction layer and spectral extraction layer of the multimodal fusion model, respectively. A dual-sensor pixel transformation matrix is ​​used to achieve spatial mapping correction between the first cap fold features and the second cap surface spectral fold features. Then, the gill opening feature is obtained through an edge detection layer and verified for consistency with the moisture content feature in terms of physiological drive and morphological performance. This effectively avoids misjudgments caused by physical damage and improves the reliability of maturity assessment from the data source. Based on this, the wax layer thickness and the hollowness inside the mushroom stem collected by the laser spectrometer and millimeter-wave radar are converted into mushroom stem hardness through the hardness assessment layer. Combined with parameters such as temperature and humidity obtained by environmental sensors, an environmental correction coefficient is generated. Finally, the maturity assessment weight sequence determined by factor analysis makes the first maturity index output by the linear assessment layer have both the ability to characterize the growth status and the ability to predict damage risk, providing accurate input for subsequent decision-making. The effects are further extended to the harvesting decision-making stage. The maturity-shelf-life curve transforms the real-time first maturity index into a shelf-life constraint. Combined with the comprehensive coverage of the initial identification path, a sequence of information on harvestable and unharvested twin mushrooms is generated. The movement control sub-model dynamically calculates the harvestable prediction timestamp of unharvested twin mushrooms through a growth prediction model optimized by environmental parameters. It then performs regional clustering with the goal of minimizing maturity and time deviations to form a logically coherent sequence of harvesting units. Based on a three-level priority constraint (prioritizing both maturity and shelf-life, minimizing time matching deviation, and minimizing energy consumption), a particle swarm optimization path planning is generated to generate a dynamic harvesting path parameter space. This ensures that high-value areas are harvested first while avoiding secondary operations, reducing overripe losses and ineffective movement from the decision-making level. The harvesting control sub-model generates the clamping force and stiffness parameters of the flexible gripper based on the maturity-harvesting force-damage probability mapping relationship and the impedance control algorithm optimized by reinforcement learning. This adapts to the physical characteristics of mushrooms at different maturity levels while ensuring low-damage harvesting. The collaborative sub-model integrates cross-modal temporal features of movement and harvesting through a collaborative attention network, outputs a collaborative feature vector to quantify the consistency of instruction execution, and feeds it back to the reward function to adjust control parameters in real time. This solves the temporal mismatch problem between AGV movement and gripper actions, thereby improving the synchronization and stability of the harvesting rhythm at the execution level.The system also uses QR code full-process traceability and daily incremental updates of the model management subsystem to optimize the parameters of the multimodal fusion model and the coordinated reinforcement learning model based on post-harvest quality data, forming a closed-loop iteration from perception and decision-making to execution feedback. This enables the maturity recognition accuracy, command coordination consistency and post-harvest quality assurance capabilities to dynamically improve with the application scenario, providing reliable technical support for large-scale intelligent harvesting.

[0104] Example 2

[0105] Please see Figure 3 Another embodiment of the present invention provides: a multimodal perception and dynamic decision-making intelligent greenhouse mushroom harvesting system, comprising: a data acquisition module, a discrimination module, a collaborative control module and an execution module;

[0106] The acquisition module collects a first maturity index characterizing the maturity of the mushroom in the target area through a configured sensor array. The first maturity index is obtained by combining the mushroom cap color characteristics, gill moisture content characteristics, cap surface wax layer thickness, stem hollowness, environmental correction coefficient, and expert evaluation algorithm collected by the sensors.

[0107] The discrimination module collects and makes a first discrimination based on the first maturity index and the maturity-shelf life curve to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp.

[0108] The collaborative control module generates collaborative control commands based on a coordinated reinforcement learning model that combines the information sequences of harvestable and unharvested twin mushrooms with a reward function constructed from factors such as shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy, and command coordination consistency delay.

[0109] The execution module is used to execute collaborative control instructions.

[0110] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

[0111] If the technical solution disclosed herein involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution disclosed herein involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of explicit consent. For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A smart greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making, characterized in that, include: The first maturity index, which characterizes the maturity of *Agaricus bisporus*, is collected by a configured sensor array in the target area. The first maturity index is obtained by combining the color characteristics of the cap, the moisture content of the gills, the thickness of the waxy layer on the cap surface, the hollowness inside the stem, and the environmental correction coefficient collected by the sensors with a preset multimodal fusion model. Based on the first maturity index and the maturity-shelf life curve, the first discrimination is collected to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp. Based on the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested, a coordinated reinforcement learning model is combined with a reward function constructed from shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy, and instruction coordination consistency delay to generate coordinated control instructions. The coordinated reinforcement learning model includes a harvesting control sub-model, a movement control sub-model, and a coordination sub-model. The movement control sub-model generates movement control instructions for the AGV based on the location information of each harvestable twin mushroom in the harvestable twin mushroom information sequence and the harvestable prediction timestamp of the twin mushroom to be harvested, combined with a path optimization algorithm. The harvesting control sub-model generates harvesting control instructions for a three-finger flexible gripper based on the first maturity index, the maturity-harvesting intensity-damage probability mapping relationship, and an impedance control algorithm optimized by reinforcement learning. The coordination sub-model generates a coordination feature vector based on the consistency requirements of movement and harvesting, combined with a coordination attention network, to characterize the coordination consistency between the movement control instructions and the harvesting control instructions during execution. The movement control commands include: Based on the real-time first maturity index of the twin mushrooms at each location in the information sequence of the twin mushrooms to be harvested, combined with the growth prediction model optimized by preset environmental parameters, the harvestable prediction timestamp of the twin mushrooms at each location in the information sequence of the twin mushrooms to be harvested is obtained. Based on the real-time storage time of the twin mushrooms at each location, a second harvesting priority sequence for twin mushrooms is constructed. Using the first maturity index corresponding to the harvestable twin mushroom information sequence of the target area and the real-time storage time length, combined with an adaptive clustering algorithm, the region is clustered and segmented with the goal of minimizing the maturity deviation between pairs of twin mushrooms in the region and the deviation of the real-time storage time length, to obtain the harvestable twin mushroom cluster region sequence of the target area.

2. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 1, characterized in that, The multimodal fusion model includes an image extraction layer, a spectral extraction layer, an edge detection layer, a hardness evaluation layer, and a linear evaluation layer; The primary maturity indicators characterizing the maturity of *Agaricus bisporus* in the target area were collected, including: The configured RGB camera and short-wave near-infrared camera are synchronized for exposure time and spatially registered using FPGA hardware and Zhang's calibration method to establish a dual-sensor image coordinate system and a dual-sensor pixel transformation matrix; the dual-sensor pixel transformation matrix is ​​used to characterize the spatial transformation relationship between RGB image and short-wave near-infrared image at the same pixel point. The RGB image sequence of *Agaricus bisporus* and the infrared image sequence of *Agaricus bisporus* cap folds were acquired in the dual-sensor image coordinate system, and the two types of images were spatially and temporally aligned. The collected RGB image sequence of Twin Mushrooms is input into the image extraction layer of a preset multimodal fusion model to extract the color features of the cap and the first cap fold features.

3. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 2, characterized in that, The primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes: Simultaneously, the infrared image sequence of the umbrella surface wrinkles of the twin mushroom is input into a preset spectral extraction layer to extract the second umbrella surface spectral wrinkle features and water content features. At the same time, the first umbrella cover wrinkle features are mapped to the second umbrella surface spectral wrinkle features through a dual-sensor pixel conversion matrix to obtain the corrected second umbrella surface spectral wrinkle features and water content features. Based on the corrected second umbrella surface spectral fold features combined with a preset edge detection layer, the gill opening characteristics of Mushroom taenia are obtained. Consistency verification was performed based on the gill opening characteristics of *Agaricus bisporus* combined with the corresponding moisture content characteristics of *Agaricus bisporus* gills, and the verified gill opening of *Agaricus bisporus* was obtained.

4. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 3, characterized in that, The primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes: The thickness of the waxy layer on the cap of *Agaricus bisporus* was collected using a configured laser spectrometer, while the hollowness inside the stem was collected using millimeter-wave radar. The collected data was then input into a configured filter for filtering and time alignment. The aligned wax layer thickness and the hollowness inside the mushroom stem are input into the hardness evaluation layer to obtain the hardness of the twin mushroom stem. Environmental assessment parameters, including temperature, humidity, CO2 concentration, and light intensity, are obtained through configured environmental sensors. Environmental correction coefficients are obtained by combining environmental assessment parameters with an environmental coefficient mapping table.

5. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 4, characterized in that, The primary maturity index for characterizing the maturity of *Agaricus bisporus* in the target area also includes: Historical first maturity index and corresponding harvesting damage rate, as well as cap color characteristics, agaricus gill opening, agaricus stem hardness, and environmental correction coefficient were obtained. Factor analysis was conducted with harvesting damage rate as the first target factor and first maturity index as the second target factor. The contribution weights of cap color characteristics, agaricus gill opening, agaricus stem hardness, and environmental correction coefficient to the first target factor that minimize the first target factor were obtained and used as the maturity assessment weight sequence. The cap color characteristics, gill opening of *Agaricus bisporus*, stem hardness of *Agaricus bisporus*, environmental correction coefficient, and maturity assessment weight sequence are input into the linear assessment layer to obtain the first maturity index of each *Agaricus bisporus* in the target area.

6. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 5, characterized in that, The process of obtaining the harvestable twin mushroom information sequence and the unharvested twin mushroom information sequence includes: An initial identification path is preset, and the maturity of the twin mushrooms is monitored and identified in real time in the target area along the initial identification path to obtain the real-time first maturity index of the twin mushrooms at each location in the target area. Based on the real-time first maturity index of the mushroom corresponding to each position and the preset maturity-shelf life curve, the real-time storage time of the mushroom corresponding to each position is obtained. A preset maturity threshold is set. When the real-time first maturity index of the twin mushroom at a certain location in the target area is greater than the maturity threshold, the corresponding location is determined to be a harvestable twin mushroom; otherwise, it is a twin mushroom to be harvested. The harvestable twin mushroom information sequence and the twin mushroom to be harvested information sequence are obtained.

7. The intelligent greenhouse mushroom harvesting method based on multimodal perception and dynamic decision-making as described in claim 6, characterized in that, The movement control command also includes: Based on the first maturity index corresponding to the information sequence of the twin mushrooms to be harvested and the harvestable prediction timestamp length of the twin mushrooms to be harvested at each location point, the region is clustered and segmented with the goal of minimizing the maturity deviation and harvestable prediction timestamp length deviation between any two pairs of twin mushrooms to be harvested in the region, and the clustering region sequence of the twin mushrooms to be harvested in the target region is obtained. Based on the clustering sequence of harvestable twin mushrooms and the clustering sequence of unharvested twin mushrooms in the target area, the first priority constraint is the maximum first harvesting priority constructed by combining the first maturity index and the comprehensive real-time storage time length. The second priority constraint is the minimum deviation between the harvestable prediction timestamp length and the waiting harvesting time length. The third priority constraint is the minimum real-time energy consumption of AGV carts and harvesting robots. Combined with the path planning algorithm of particle swarm optimization, a dynamic harvesting path parameter space is obtained. Based on the dynamic picking path parameter space combined with fuzzy control algorithm, dynamic movement control commands are obtained.

8. A multimodal sensing and dynamic decision-making intelligent greenhouse mushroom harvesting system, used to implement the multimodal sensing and dynamic decision-making intelligent greenhouse mushroom harvesting method according to any one of claims 1-7, characterized in that, include: The module comprises an acquisition module, a discrimination module, a collaborative control module, and an execution module. The acquisition module acquires a first maturity index characterizing the maturity of the mushroom in the target area through a configured sensor array. The first maturity index is obtained by combining the mushroom cap color characteristics, gill moisture content characteristics, cap surface wax layer thickness, stem hollowness, environmental correction coefficient, and expert evaluation algorithm collected by the sensors. The discrimination module collects first discrimination based on the first maturity index and the maturity-shelf life curve to obtain the information sequence of harvestable twin mushrooms and the information sequence of twin mushrooms to be harvested; the information sequence of harvestable twin mushrooms includes at least location information and the corresponding first maturity, and the information sequence of twin mushrooms to be harvested includes at least location information, the corresponding first maturity, and a harvestable prediction timestamp. The collaborative control module generates collaborative control commands based on a coordinated reinforcement learning model that combines the information sequences of harvestable and unharvested twin mushrooms with a reward function constructed from factors such as shelf life, harvesting damage rate, coordinated harvesting energy consumption, maturity recognition accuracy, and command coordination consistency delay. The execution module is used to execute collaborative control instructions.

Images