Intelligent greenhouse agaricus bisporus harvesting method and system based on multi-modal perception and dynamic decision

Abstract

The invention belongs to the field of collection control, and particularly relates to an intelligent greenhouse agaricus bisporus harvesting method and system.The method comprises the steps that firstly, multi-modal data representing the maturity of agaricus bisporus are synchronously collected through an integrated sensor array, and a first maturity index is calculated in combination with environmental parameters and a multi-modal fusion model; judging based on the index and a maturity-storage period curve, and generating a pickable and to-be-picked agaricus bisporus information sequence; afterwards, a coordinated reinforcement learning model is utilized, the model comprises a picking control sub-model, a movement control sub-model and a coordinated sub-model, the refreshing time, the damage rate, the energy consumption and the instruction coordinated consistency serve as reward functions, and a coordinated control instruction of the AGV mobile platform and the flexible gripper at the tail end of the six-degree-of-freedom robot is comprehensively generated; and finally executing the instruction to complete automatic harvesting. According to the method, precise perception of the maturity of the agaricus bisporus and dynamic optimization of the harvesting decision are realized, and the harvesting quality and the working efficiency are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of collection control, and particularly relates to an intelligent greenhouse bicolored mushroom collection method and system based on multi-modal perception and dynamic decision-making. BACKGROUND

[0002] Currently, the automatic collection of bicolored mushrooms faces multiple bottlenecks from recognition, decision-making to execution. The collection operation is highly dependent on manual labor, not only inefficient, but also facing great labor pressure during the peak of mushroom production. Moreover, the judgment of maturity by pickers is subjective, often leading to improper picking time, directly causing yield loss and shortening of commodity shelf life. The existing automated equipment and academic research solutions have not effectively solved this core problem. Commercial equipment usually relies on only a single feature in vision for recognition, ignoring key maturity indicators such as gill state and stem hardness, resulting in high mispicking rate. At the same time, its rigid mechanical structure is prone to physical damage to surrounding mushrooms in a dense planting environment. Academic research has attempted to introduce more sensing means, but is often limited by the interference of contact measurement on mushrooms, the difficulty of complex algorithms to meet real-time requirements in the field, and the lack of feedback loops for post-collection quality, making it impossible to form a continuously optimized picking strategy. Most importantly, both manual labor and existing technical solutions completely ignore the dynamic and continuous influence of the greenhouse microenvironment on the growth rate and post-harvest quality of bicolored mushrooms, resulting in static and isolated picking decisions that cannot be adjusted adaptively based on the coupling relationship between the environment and growth state. Therefore, it is an urgent need to develop an intelligent system that can achieve multi-dimensional accurate perception, integrate environmental factor analysis, and perform low-damage flexible collection, in order to improve the efficiency and quality of the entire industry. SUMMARY

[0003] To overcome the deficiencies of the prior art, the present application provides an intelligent greenhouse bicolored mushroom collection method and system based on multi-modal perception and dynamic decision-making. The method first synchronously collects multi-modal data representing the maturity of bicolored mushrooms through an integrated sensor array, including umbrella color, gill moisture content, cap wax layer thickness, and stem hollow degree, and calculates a first maturity index in combination with environmental parameters and expert algorithms. Then, based on the index and the maturity-shelf life curve, the information sequence of harvestable and non-harvestable bicolored mushrooms is generated. Subsequently, a coordination reinforcement learning model is used, which includes picking control, movement control, and coordination sub-models, to generate coordinated control instructions for the AGV movement platform and the six-degree-of-freedom robot end flexible gripper, taking shelf life, damage rate, energy consumption, and instruction coordination consistency as the reward function. Finally, the instructions are executed to complete the automated collection. The present application realizes accurate perception of bicolored mushroom maturity and dynamic optimization of picking decisions, effectively improving collection quality and operation efficiency.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0005] The intelligent double mushroom harvesting method of multi-modal perception and dynamic decision in greenhouse comprises:

[0006] A first maturity index representing the maturity of the target area of the double mushroom is collected by the configured sensor array; the first maturity index is obtained by combining the color characteristics of the double mushroom cap, the moisture content characteristics of the gill, the thickness of the wax layer on the cap surface, the internal hollow degree of the stem, the environmental correction coefficient, and the pre-set multi-modal fusion model;

[0007] Based on the first maturity index and the maturity-keeping period curve, a first discrimination is performed to obtain a harvestable double mushroom information sequence and a to-be-harvested double mushroom information sequence; the harvestable double mushroom information sequence at least includes position information and corresponding first maturity, and the to-be-harvested double mushroom information sequence at least includes position information and corresponding first maturity and a harvestable time stamp;

[0008] Based on the harvestable double mushroom information sequence and the to-be-harvested double mushroom information sequence, a reward function is constructed based on a coordinated reinforcement learning model, a preservation period, a harvesting damage rate, a coordinated harvesting energy consumption, a maturity recognition accuracy, and an instruction coordination consistency delay, and a collaborative control instruction is generated;

[0009] The collaborative control instruction is executed.

[0010] Specifically, the coordinated reinforcement learning model comprises a harvesting control sub-model, a moving control sub-model, and a coordination sub-model; the moving control sub-model is used to generate a moving control instruction of an AGV car according to the position information of each harvestable double mushroom in the harvestable double mushroom information sequence and the harvestable time stamp of the to-be-harvested double mushroom combined with a path optimization algorithm; the harvesting control sub-model is used to generate a harvesting control instruction of a 3-finger flexible gripper according to the first maturity index, a maturity-harvesting intensity-damage probability mapping relationship, and an impedance control algorithm optimized by reinforcement learning; the coordination sub-model is used to generate a coordination feature vector according to the moving and harvesting consistency requirement combined with a collaborative attention network, which is used to represent the collaborative consistency of the moving control instruction and the harvesting control instruction in the execution process.

[0011] Specifically, the multi-modal fusion model comprises an image extraction layer, a spectrum extraction layer, an edge detection layer, a hardness evaluation layer, and a linear evaluation layer; the first maturity index representing the maturity of the target area of the double mushroom is collected, comprising:

[0012] The configured RGB camera and short-wave near-infrared camera are synchronized in exposure time and registered in space by FPGA hardware and Zhang's calibration method to establish a double-sensor image coordinate system and a double-sensor pixel conversion matrix; the double-sensor pixel conversion matrix is used to represent the spatial conversion relationship of the RGB image and the short-wave near-infrared image at the same pixel point;

[0013] The double-sensor image coordinate system collects a double-gill mushroom RGB image sequence and a double-gill mushroom cap wrinkle infrared image sequence, and performs spatial and temporal alignment processing on the collected two kinds of images;

[0014] The collected double-gill mushroom RGB image sequence is input into an image extraction layer of a preset multi-modal fusion model to extract cap color features and first cap wrinkle features.

[0015] Specifically, the first maturity index representing the maturity of the target area of the double-gill mushroom also includes:

[0016] Meanwhile, the double-gill mushroom cap wrinkle infrared image sequence is input into a preset spectrum extraction layer to extract second cap spectral wrinkle features and moisture content features, and the first cap wrinkle features are mapped to the second cap spectral wrinkle features through a double-sensor pixel transformation matrix to obtain corrected second cap spectral wrinkle features and moisture content features;

[0017] Based on the corrected second cap spectral wrinkle features, an edge detection layer is combined to obtain double-gill mushroom gill opening degree features;

[0018] Based on the double-gill mushroom gill opening degree features, the moisture content features of the corresponding double-gill mushroom gill are verified for consistency to obtain verified double-gill mushroom gill opening degrees.

[0019] Specifically, the first maturity index representing the maturity of the target area of the double-gill mushroom also includes:

[0020] The wax layer thickness of the double-gill mushroom cap surface is collected by a configured laser spectrometer, and the internal hollow degree of the stem is collected by a millimeter wave radar, and the collected data is input into a configured filter for filtering and time alignment;

[0021] The aligned wax layer thickness and stem internal hollow degree are input into the hardness evaluation layer to obtain the double-gill mushroom stem hardness;

[0022] An environmental sensor is configured to obtain environmental evaluation parameters, including temperature, humidity, CO2 concentration, and light intensity;

[0023] Based on the environmental evaluation parameters and an environmental coefficient mapping table, an environmental correction coefficient is obtained.

[0024] Specifically, the first maturity index representing the maturity of the target area of the double-gill mushroom also includes:

[0025] The first maturity index, the corresponding collection damage rate, the umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, and the environmental correction coefficient are obtained. The factor analysis is performed with the collection damage rate as the first target factor and the first maturity index as the second target factor. The contribution weight of the umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, and the environmental correction coefficient to the first target factor is obtained as the maturity evaluation weight sequence.

[0026] The umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, and the environmental correction coefficient are input into the linear evaluation layer together with the maturity evaluation weight sequence to obtain the first maturity index of each double mushroom in the target area.

[0027] Specifically, the pickable double mushroom information sequence and the to-be-picked double mushroom information sequence are obtained, including:

[0028] An initial identification path is preset, and real-time double mushroom maturity monitoring and identification are performed along the initial identification path in the target area to obtain the real-time first maturity index of the double mushroom corresponding to each position in the target area.

[0029] Based on the real-time first maturity index of the double mushroom corresponding to each position, a maturity-preservation period curve is obtained.

[0030] A maturity discrimination threshold is preset. When the real-time first maturity index of the double mushroom corresponding to a position in the target area is greater than the maturity discrimination threshold, the corresponding position is determined to be a pickable double mushroom, otherwise it is a to-be-picked double mushroom. The pickable double mushroom information sequence and the to-be-picked double mushroom information sequence are obtained.

[0031] Specifically, the movement control instruction includes:

[0032] Based on the real-time first maturity index of the double mushroom at each position point in the to-be-picked double mushroom information sequence, a growth prediction model optimized based on the environmental parameters is obtained.

[0033] Based on the real-time storage time length of the double mushroom corresponding to each position, a double mushroom second picking priority sequence is constructed.

[0034] The first maturity index and the real-time storage time length of the pickable double mushroom information sequence corresponding to the target area are combined with an adaptive clustering algorithm to perform regional clustering segmentation with the minimum maturity deviation and real-time storage time length deviation of the double mushrooms in the region as the target. The pickable double mushroom clustering region sequence of the target area is obtained.

[0035] Specifically, the movement control instruction further includes:

[0036] Based on the first maturity index corresponding to the to-be-picked double-bolet information sequence and the pickable predicted time stamp length of the to-be-picked double-bolet at each position point, the maturity deviation and the pickable predicted time stamp length deviation of the to-be-picked double-bolet in the region are clustered and segmented to minimize the maturity deviation and the pickable predicted time stamp length deviation, and a to-be-picked double-bolet clustering region sequence of the target region is obtained.

[0037] Based on the to-be-picked double-bolet clustering region sequence and the pickable double-bolet clustering region sequence of the target region, a first picking priority is constructed by combining the first maturity index and the real-time storage time length, a second picking priority is constructed by minimizing the deviation of the pickable predicted time stamp length and the waiting picking time length, a third picking priority is constructed by minimizing the real-time energy consumption of the AGV and the picking robot, and a dynamic picking path parameter space is obtained by combining the path planning algorithm of the particle swarm optimization;

[0038] Based on the dynamic picking path parameter space and the fuzzy control algorithm, a dynamic movement control instruction is obtained.

[0039] The intelligent greenhouse double-bolet picking system based on multi-modal perception and dynamic decision-making comprises a collection module, a discrimination module, a cooperative control module and an execution module.

[0040] The collection module collects the first maturity index representing the maturity of double-bolet in the target region through a configured sensor array; the first maturity index is obtained by combining the double-bolet cap color feature, the laceration moisture content feature, the cap surface wax layer thickness, the stem internal hollow degree, the environmental correction coefficient collected by the sensor and the expert evaluation algorithm;

[0041] The discrimination module performs collection first discrimination based on the first maturity index and the maturity-keeping period curve to obtain a pickable double-bolet information sequence and a to-be-picked double-bolet information sequence; the pickable double-bolet information sequence at least includes position information and corresponding first maturity, and the to-be-picked double-bolet information sequence at least includes position information, corresponding first maturity and a pickable time stamp;

[0042] The cooperative control module generates a cooperative control instruction based on the pickable double-bolet information sequence and the to-be-picked double-bolet information sequence, the reward function constructed by combining the coordination reinforcement learning model, the keeping period, the picking damage rate, the coordinated picking energy consumption, the maturity recognition accuracy and the instruction cooperative consistency delay;

[0043] The execution module is configured to execute the cooperative control instruction.

[0044] Compared with the prior art, the intelligent greenhouse double-bolet picking system based on multi-modal perception and dynamic decision-making has the following beneficial effects:

[0045] The present application aims at the deficiencies of the prior art, through the fusion of RGB, short wave near infrared, laser spectrometer and other devices by a multi-sensor array, combined with FPGA synchronization and Zhang's calibration to realize accurate data acquisition, multi-modal feature fusion and consistency verification to improve the accuracy of maturity evaluation, effectively reducing the misjudgment rate. Based on adaptive clustering division of the harvesting area, combined with particle swarm optimization and multi-priority constraint planning dynamic path, the invalid movement and secondary picking of AGV are reduced, and the energy consumption is reduced. The coordination reinforcement learning model realizes the collaborative control of movement and picking, the impedance control optimized by reinforcement learning reduces the picking damage, and the collaborative sub-model ensures the synchronization of instructions, improves the harvesting efficiency. The whole process realizes the whole closed-loop intelligent management and control from maturity perception, harvesting decision to execution, adapts to the demand of greenhouse large-scale planting, significantly reduces the post-harvest loss and operation cost, and improves the harvesting quality and automation level. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 A flow chart of the intelligent greenhouse double-belly mushroom harvesting method of the present application for multi-modal perception and dynamic decision-making;

[0047] Figure 2 A Bert model flow chart of the intelligent greenhouse double-belly mushroom harvesting method of the present application for multi-modal perception and dynamic decision-making;

[0048] Figure 3 A structural diagram of the intelligent greenhouse double-belly mushroom harvesting system of the present application for multi-modal perception and dynamic decision-making. DETAILED DESCRIPTION

[0049] Embodiment 1

[0050] Please refer to Figure 1 The present application provides an embodiment: a multi-modal perception and dynamic decision-making intelligent greenhouse double-belly mushroom harvesting method, comprising the following steps:

[0051] S1, acquiring a first maturity index representing the maturity of the target area of double-belly mushrooms by a configured sensor array; the first maturity index is obtained by combining the color characteristics of the double-belly mushroom cap, the moisture content characteristics of the gill, the thickness of the wax layer on the cap surface, the internal hollow degree of the stem, the environmental correction coefficient and the expert evaluation algorithm;

[0052] S2, based on the first maturity index, combining the maturity-preservation period curve to make a first discrimination, and obtaining a pickable double-belly mushroom information sequence and a to-be-picked double-belly mushroom information sequence; the pickable double-belly mushroom information sequence at least includes position information and corresponding first maturity, and the to-be-picked double-belly mushroom information sequence at least includes position information, corresponding first maturity and pickable time stamp;

[0053] S3, based on the combination of the pickable double mushroom information sequence and the to-be-picked double mushroom information sequence, a coordination reinforcement learning model is constructed, and a reward function is constructed based on the shelf life, the picking damage rate, the coordinated picking energy consumption, the maturity recognition accuracy, and the instruction coordination consistency delay, and a cooperative control instruction is generated;

[0054] S4, execute the cooperative control instruction.

[0055] It should be further explained that the coordination reinforcement learning model of the embodiment includes a picking control sub-model, a movement control sub-model, and a coordination sub-model; the movement control sub-model is used to generate a movement control instruction of the AGV car according to the position information of each pickable double mushroom in the pickable double mushroom information sequence and the pickable time stamp of the to-be-picked double mushroom in combination with a path optimization algorithm; the picking control sub-model is used to generate a picking control instruction of the 3-finger flexible gripper according to the first maturity index in combination with the maturity-picking force-damage probability mapping relationship and the impedance control algorithm optimized by reinforcement learning; the coordination sub-model is used to generate a coordination feature vector according to the movement and picking consistency requirement in combination with a coordination attention network, which is used to represent the coordination consistency of the movement control instruction and the picking control instruction in the execution process.

[0056] It should be further explained that please refer to Figure 2 The multi-modal fusion model of the embodiment includes an image extraction layer, a spectrum extraction layer, an edge detection layer, a hardness evaluation layer, and a linear evaluation layer; it should be further explained that the first maturity index collected by the embodiment to represent the maturity of the double mushroom includes:

[0057] The configured RGB camera and short-wave near-infrared camera are synchronized in exposure time and spatially registered by the Zhang calibration method through FPGA hardware to establish a double-sensor image coordinate system and a double-sensor pixel conversion matrix; the double-sensor pixel conversion matrix is used to represent the spatial conversion relationship of the RGB image and the short-wave near-infrared image at the same pixel point; in this embodiment, the motivation of this operation is to solve the problems of exposure time asynchrony and spatial position misplacement of the RGB camera and the short-wave near-infrared camera caused by the differences in hardware characteristics and different installation positions when collecting images, to ensure that the double images of the double-belly mushroom obtained by the two sensors are accurately matched in time and space dimensions, and to provide reliable data support for subsequent multi-modal feature fusion and maturity-related feature accurate extraction. The core reason is that asynchronous exposure of double sensors will cause time difference between the RGB image and the infrared image of the same double-belly mushroom, which cannot truly reflect the state of the mushroom body at the same time; spatial position misplacement will cause the pixel coordinates of the same physical point in the two images not to correspond, which will directly affect the correlation matching of the cross-modal features such as umbrella color and gill moisture content, while the FPGA hardware can generate high-precision synchronous trigger signals to realize strict synchronization of the exposure time, and the Zhang calibration method can solve the sensor internal and external parameters, and then establish a unified image coordinate system and a pixel conversion matrix, realize the spatial accurate mapping of the images of different sensors, and ensure the effectiveness of multi-modal data fusion and the accuracy of maturity evaluation.

[0058] In addition, it needs to be further explained that, for the scene adaptation of high humidity scene (RH>95%), the embodiment adopts the technical logic of sensor performance guarantee-image data quality control-harvesting decision dynamic optimization, and the specific technical means and principles are as follows: the heating sheet is configured for the SWIR camera to realize anti-fog compensation, the principle is that high humidity environment is easy to make the camera lens condense fog, resulting in blurred infrared image, the heating sheet can maintain the lens temperature higher than the environmental dew point, avoid fog generation, and ensure the collection clarity of the double-belly mushroom umbrella fold wrinkle infrared image sequence; the adjustment strategy of reducing the CMI threshold by 5% and preferentially harvesting the edge area is adopted at the decision level, the principle is that high humidity will accelerate the maturity process of double-belly mushroom and increase the risk of corruption, reducing the CMI threshold can moderately relax the harvestable maturity standard, avoid loss caused by too fast maturity and not timely harvesting, and the edge area usually has different ventilation conditions from the center area, and is more prone to maturity abnormalities or corruption hazards under high humidity, so preferential harvesting can further avoid post-harvest loss, and realize the coordinated guarantee of harvesting quality and efficiency under high humidity.

[0059] The RGB image sequence and the gill fold infrared image sequence of the double gill mushroom are collected in the dual-sensor image coordinate system, and the collected two kinds of images are subjected to space and time alignment processing; in addition, in this embodiment, the RGB / SWIR collection weight is dynamically switched during the day and night transition period, the principle is to adapt to the influence of the periodic change of light intensity on the imaging quality of the dual-sensor, and to combine the dual-sensor image space-time alignment to ensure the stability and accuracy of multi-modal feature extraction. Among them, the weight allocation is set to 7:3 during the day, because the light is sufficient, the RGB camera has higher accuracy in capturing visual features such as color and surface texture of the double gill mushroom cap, and dominates the visual feature extraction; the weight is adjusted to 3:7 at night, because the signal-to-noise ratio of the RGB image decreases and the feature recognition degree decreases due to the weakening of the light, while the SWIR camera is less affected by natural light and can accurately extract deep information such as gill moisture content and cap fold spectral features, and then it dominates the feature extraction.

[0060] The collected double gill mushroom RGB image sequence is input into the image extraction layer of the preset multi-modal fusion model to extract the cap color feature and the first cap fold feature.

[0061] It needs to be further explained that the image extraction layer of the embodiment is preferably a MobileNetV3 lightweight network, and the specific process of extracting the cap color feature and the first cap fold feature in the double gill mushroom RGB image sequence is as follows: first, the input double gill mushroom RGB image sequence is preprocessed, including image normalization, Gaussian denoising and cap region of interest area cropping, background interference is removed and the image size is unified to adapt to the network input; the preprocessed RGB image frame is input into the MBV3 lightweight network, the depth separable convolution and the inverse residual structure are used to reduce the calculation amount and improve the inference efficiency, the low-dimensional visual features of the cap color are captured through the network shallow convolution kernel, the weight distribution of the color channel features is strengthened through the SE attention mechanism, and the core features such as color mean, hue distribution and color gradient representing the maturity of the cap are accurately extracted; at the same time, the network deep convolution layer is used to perceive the local texture of the image, focus on the cap fold area, and extract the edge profile, texture density and fold depth of the fold as the first cap fold feature; in the process, the BatchNorm layer and the activation function built-in the network are used to enhance the nonlinear expression ability of the features, and finally the features extracted from multiple images are subjected to time sequence fusion and normalization processing, the cap color feature vector and the first cap fold feature vector with uniform output dimension and high recognition degree are output, which provides high-quality visual feature support for subsequent cross-modal feature fusion.

[0062] Meanwhile, the umbrella gill wrinkle infrared image sequence is input into a preset spectrum extraction layer to extract second umbrella spectrum wrinkle features and water content features, and the first umbrella cover wrinkle features are mapped to the second umbrella spectrum wrinkle features through a double-sensor pixel conversion matrix to obtain corrected second umbrella spectrum wrinkle features and water content features. It needs to be further explained that the specific process of the spectrum extraction layer of the embodiment for extracting and correcting the second umbrella spectrum wrinkle features and the water content features is as follows: the umbrella gill wrinkle infrared image sequence is subjected to preprocessing operations of spectral dimension noise reduction, time sequence frame alignment and 1450 nm water content feature peak band interception, the sequence data format is unified and environmental noise and invalid spectral information are removed; the preprocessed infrared image sequence is expanded into a one-dimensional spectral vector according to pixel time sequence spectral data, and is input into the preset spectrum extraction layer. The spectrum extraction layer adopts a one-dimensional convolution structure, captures spectral response differences representing umbrella wrinkle structures in the spectral dimension through multi-scale convolution kernels to generate second umbrella spectrum wrinkle features, extracts spectral parameters such as intensity values, half-widths and peak areas of 1450 nm feature peaks to generate water content features, and strengthens nonlinear expression of effective features through a pooling layer and an activation function and performs dimension compression; the first umbrella cover wrinkle features extracted from the umbrella gill RGB image sequence are input into the double-sensor pixel conversion matrix, the spatial coordinates and feature dimensions of the first umbrella cover wrinkle features are mapped to the pixel coordinate system of the infrared image through matrix operation, and spatial accurate alignment of the first umbrella cover wrinkle features and the second umbrella spectrum wrinkle features is realized; the second umbrella spectrum wrinkle features are corrected based on the aligned first umbrella cover wrinkle features using a feature weighted fusion strategy, the deviation of spectral features in wrinkle morphology representation is supplemented through structure information in the first umbrella cover wrinkle features, and the water content features are simultaneously calibrated in combination with the physiological correlation of the corrected second umbrella spectrum wrinkle features and water content features, and finally the corrected second umbrella spectrum wrinkle feature vector and water content feature vector are output.

[0063] Based on the corrected second umbrella spectrum wrinkle features and a preset edge detection layer, gill opening degree features of the double-belly mushroom are obtained. It needs to be further explained that the gill opening degree features of the double-belly mushroom obtained by the embodiment include:

[0064] Based on the corrected second umbrella spectrum fold feature, through the feature map reconstruction algorithm and the normalization enhancement processing, the target feature map with enhanced difference between the gill fold region and the background feature is obtained; based on the target feature map, the edge detection algorithm is called by the preset edge detection layer to preliminarily extract the gill fold edge, the adaptive threshold is set to filter the noise interference edge, and the preliminary edge extraction result is obtained; based on the preliminary edge extraction result, the edge connection and the isolated noise point removal are realized through the morphological dilation and corrosion operation, and the optimized gill fold edge contour is obtained; based on the spatial coordinate information corresponding to the pixel conversion matrix of the double sensor, the optimized gill fold edge contour is regionally positioned, and the target region edge contour where the double gill gill fold is located is obtained; based on the target region edge contour, the closed contour curve of the gill fold edge is obtained through the edge fitting algorithm, the included angle between the two endpoints of the contour curve and the gill fold center, the pixel area proportion of the opening region, and the contour development length and other geometric parameters are calculated, and after the geometric parameters are normalized, the gill opening degree feature representing the opening degree of the double gill gill fold is obtained.

[0065] Based on the gill opening degree feature of the double gill gill fold and the water content feature corresponding to the double gill gill fold, consistency verification is performed, and the verified gill opening degree of the double gill gill fold is obtained; it needs to be further explained that the process of consistency verification in this embodiment includes:

[0066] Based on the historical collected gill opening degree feature of the double gill gill fold, the water content feature corresponding to the 1450nm characteristic peak detected by the short wave near infrared camera, and the maturity label, a dynamic correlation model of gill opening degree and water content at different maturity stages is constructed through a nonlinear regression algorithm, and the standard mapping relationship threshold interval of the two is obtained; based on the gill opening degree feature of the double gill gill fold extracted and optimized by the edge detection algorithm through the edge detection layer, the 1450nm characteristic peak water content feature extracted by the short wave near infrared spectrum at the same period, and the absolute deviation value of the actual numerical value of the two and the standard mapping relationship of the corresponding maturity stage in the dynamic correlation model is obtained through a deviation calculation model; based on the preset consistency verification threshold, it is judged whether the absolute deviation value is within the allowable range; if the deviation value is within the allowable range, the current gill opening degree feature is directly taken as the verified gill opening degree of the double gill gill fold; if the deviation value exceeds the allowable range, the theoretical gill opening degree adapted to the current water content feature is deduced based on the dynamic correlation model, and the deviation weighted correction algorithm is used to adjust the current actual gill opening degree feature, and the weighted weight is dynamically allocated according to the deviation value and the historical verification accuracy; at the same time, the correction result is secondarily checked based on the integrity and continuity features of the gill fold edge contour extracted by the edge detection layer, and the abnormal opening degree data caused by physical damage is removed, and finally the verified gill opening degree conforming to the maturity physiological law (gill water content reduction driving opening) of the double gill is obtained.

[0067] It needs to be further explained that in the present embodiment, the short-wave near-infrared camera is used to detect the water content of the gill at the 1450 nm characteristic peak, and the core principle and motivation lies in directly monitoring the key physiological process driving the maturity of the double-belly mushroom: the gill is preparing for spore release, and the transpiration will be significantly enhanced, resulting in a specific decrease in water content; this water loss will directly reduce the turgor pressure of the gill cell, thereby physically triggering the gill to change from a closed state to an open state; therefore, the water content detection aims to accurately quantify this intrinsic physiological driving force that leads to morphological changes, and its continuous downward trend constitutes an early maturity signal that precedes visible morphological changes; and the gill opening degree obtained by edge detection based on the same SWIR image is a direct observation and measurement of the final external morphological result caused by the above physiological process; the gill water content detected by short-wave near-infrared spectrum and the gill opening degree identified by image recognition constitute a deep collaborative verification system from internal physiological driving to external morphological manifestation. Its collaborative mechanism specifically manifests as: the water content as a leading causal indicator, for example, when the gill water content is continuously decreased from 85% to 78%, even if the gill morphology has not changed significantly at this time, the system can predict that it will enter the open stage within 2 hours according to the physiological trend, thereby realizing forward-looking decision-making; and the opening degree as a morphological verification indicator directly confirms the final expression of the physiological process, for example, when the gill opening angle is greater than 45 degrees, it confirms the maturity state together with the water content data which is already at a low level; this two-way verification is particularly important: if the system finds that the gill has opened but the water content is still high, it can be determined that the opening is caused by physical damage rather than natural maturity, thereby avoiding mispicking; on the contrary, if the water content has decreased significantly but the opening degree is insufficient, it indicates that there may be environmental inhibition, and the microenvironment parameters need to be adjusted. This two-modal perception closed loop of internal physiology and external morphology, through real-time data cross verification, significantly improves the prediction foresight, judgment accuracy and decision robustness of maturity assessment.

[0068] The wax layer thickness of the double-belly mushroom cap surface is collected by the configured laser spectrometer, and the hollow degree inside the stem is collected by the millimeter wave radar, and the collected data is input into the configured filter for filtering and time alignment;

[0069] The aligned wax layer thickness and hollow degree inside the stem are input into the hardness evaluation layer to obtain the hardness of the double-belly mushroom stem; it needs to be further explained that in the present embodiment, the hardness of the double-belly mushroom stem is obtained, including:

[0070] Based on the filtered and time-aligned double-bolet cap surface wax layer thickness data and the double-bolet stem internal hollow degree data, the Min-Max normalization algorithm is used for scale unification processing to obtain standardized wax layer thickness features and double-bolet stem internal hollow degree features; based on the standardized features, a weighted fusion strategy with weights determined by historical data factor analysis is used for feature fusion to obtain a fusion feature vector covering the wax layer state and double-bolet stem internal structure information; based on the fusion feature vector, a preset hardness evaluation layer is input, the evaluation layer carries a gradient boosting tree model trained and optimized by historical wax layer thickness, double-bolet stem internal hollow degree and corresponding measured double-bolet stem hardness label, and the mapping relationship between the fusion features and the double-bolet stem hardness is established by model inference to obtain a preliminary double-bolet stem hardness value; based on the preset double-bolet hardness calibration coefficients at different maturity stages, the preliminary double-bolet stem hardness value is dynamically calibrated to finally obtain the double-bolet stem hardness accurately representing the firmness of the double-bolet stem.

[0071] Through the configured environmental sensor, environmental evaluation parameters are obtained, including temperature, humidity, CO2 concentration, and light intensity;

[0072] Based on the environmental evaluation parameters and the environmental coefficient mapping table, an environmental correction coefficient is obtained.

[0073] It should be further explained that in the present embodiment, an environmental sensor array covering temperature, humidity, CO2 concentration, and light intensity monitoring is configured to collect real-time environmental evaluation parameters of the target area and perform data preprocessing, which includes outlier rejection, time alignment, and numerical standardization processing; an environmental coefficient mapping table is constructed in advance based on historical environmental evaluation parameter samples, corresponding sample double-bolet maturity evaluation deviation data, and industry expert experience; the mapping table is divided into multiple continuous intervals according to the numerical ranges of temperature, humidity, CO2 concentration, and light intensity, each interval corresponds to a unique basic correction coefficient, and the mapping table associates the influence weights of each environmental parameter on maturity evaluation, which are determined by historical data factor analysis; the preprocessed real-time environmental evaluation parameters are matched to the corresponding numerical intervals in the environmental coefficient mapping table to obtain the basic correction coefficients corresponding to each environmental evaluation parameter; based on the preset influence weights of each environmental parameter, the weighted sum operation is performed on the obtained multiple basic correction coefficients to obtain the final environmental correction coefficient, which is used to dynamically calibrate the double-bolet maturity evaluation results to eliminate the interference of environmental factors on the maturity evaluation accuracy.

[0074] The first maturity index, the corresponding collection damage rate, the umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, and the environmental correction coefficient are obtained. The factor analysis is performed with the collection damage rate as the first target factor and the first maturity index as the second target factor. The contribution weight of the umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, and the environmental correction coefficient to the first target factor is obtained, which is used as the maturity evaluation weight sequence. It needs to be further explained that the core motivation of the factor analysis of the first target factor and the second target factor in this embodiment is to adapt to the core demand of the intelligent greenhouse double mushroom large-scale collection, that is, to prioritize postharvest quality and reduce loss while ensuring the accuracy of maturity evaluation. The core principle is that the collection damage rate directly determines the postharvest quality and economic value of double mushrooms. High damage rate will lead to mushroom rot and decrease in commodity value, which is the primary constraint factor affecting the collection benefit. The first maturity index is the basis for determining the collection time and needs to be accurately evaluated under the premise of low damage. Through factor analysis, the correlation between the umbrella cover color feature, the gill opening degree, and the two target factors is obtained. The contribution weight of each feature is obtained by minimizing the damage rate as the primary constraint. This can make the weight distribution in the subsequent maturity evaluation process more biased towards the feature dimension that can reduce the collection damage, ensuring that the maturity evaluation result is consistent with the actual growth state and avoiding high damage risk from the index weight design level, achieving the coordinated optimization of accurate maturity evaluation and postharvest quality guarantee, and meeting the dual needs of efficiency and quality for large-scale collection.

[0075] The umbrella cover color feature, the double mushroom gill opening degree, the double mushroom stem hardness, the environmental correction coefficient, and the maturity evaluation weight sequence are input into the linear evaluation layer to obtain the first maturity index of each double mushroom in the target area. It needs to be further explained that the linear evaluation layer in this embodiment is configured with a linear weighted sum algorithm for evaluation operation.

[0076] It needs to be further explained that the acquisition of the pickable double mushroom information sequence and the to-be-picked double mushroom information sequence in this embodiment includes:

[0077] A preset initial recognition path is set. The double mushroom maturity real-time monitoring and recognition are performed along the initial recognition path in the target area to obtain the real-time first maturity index of the double mushroom corresponding to each position in the target area. It needs to be further explained that the construction process of the initial recognition path in this embodiment includes:

[0078] Based on the prior planting information of the geometric size of the target area of the greenhouse, the planting row spacing and the planting density of the double-gilled mushroom, combined with the detection range, the field of view angle and the turning radius of the mobile platform of the configured sensor array and other equipment constraint parameters, the planting layout grid map of the target area is constructed by the grid map modeling method, and based on the grid map, the snake path planning algorithm is used to plan and generate continuous and coherent path trajectories, and the grid coordinates of the path trajectories are associated and calibrated with the physical coordinates of the target area, so that the preset initial identification path suitable for the planting layout and the equipment performance is finally obtained.

[0079] Based on the real-time first maturity index of the double-gilled mushroom corresponding to each position and the preset maturity-keeping period curve, the real-time storage time length of the double-gilled mushroom corresponding to each position is obtained. It needs to be further explained that the core motivation of the maturity-keeping period curve in this embodiment is to establish the quantitative correlation between the maturity of the double-gilled mushroom and the storage time length after harvesting, so as to provide a basis for the harvesting timing planning to reduce the post-harvest over-ripening loss. The construction means is to establish a non-linear mapping model of maturity and storage time based on the storage time data of double-gilled mushrooms with different maturity collected in the standard storage environment, combined with the correction of temperature and humidity and other environmental factors, through polynomial fitting or regression analysis, and the curve parameters are determined through multiple verification and optimization. Its role is to quickly query or calculate the corresponding real-time storage time length according to the real-time first maturity index of the double-gilled mushroom at each position, so as to provide key data support for the subsequent discrimination of harvestable and unharvestable double-gilled mushrooms, the construction of the second harvesting priority sequence, and to ensure that the harvesting decision meets the post-harvest quality guarantee requirements.

[0080] The preset maturity discrimination threshold is used to determine whether the corresponding position is a harvestable double-gilled mushroom or an unharvestable double-gilled mushroom. When the real-time first maturity index of the double-gilled mushroom corresponding to a position in the target area is greater than the maturity discrimination threshold, the corresponding position is determined to be a harvestable double-gilled mushroom, otherwise it is an unharvestable double-gilled mushroom, and the harvestable double-gilled mushroom information sequence and the unharvestable double-gilled mushroom information sequence are obtained.

[0081] It needs to be further explained that the mobile control instruction in this embodiment includes:

[0082] Based on the real-time first maturity index of the double mushroom at each position point in the double mushroom information sequence to be picked and a preset environment parameter optimized growth prediction model, a pickable prediction time stamp of the double mushroom to be picked at each position point in the double mushroom information sequence to be picked is obtained; the environment parameter optimized growth prediction model is used for dynamically predicting a pickable prediction time length required for the double mushroom at each position point in the double mushroom information sequence to be picked to grow from the current first maturity index to a pickable maturity threshold; specifically, based on a multi-sensor network deployed in a greenhouse, temperature, humidity, carbon dioxide concentration, light intensity and other environment parameters are collected in real time, and an environment parameter time series dataset with a time stamp is constructed; based on the dataset, combined with the historically accumulated double mushroom maturity index continuous monitoring data, a dynamic growth rate prediction model is constructed and trained using a long short-term memory network (LSTM), the model takes the aforementioned time series environment parameters as input, and the output is the instantaneous growth rate of the double mushroom maturity at the next time step; based on the trained LSTM model and the current first maturity index of each individual in the double mushroom information sequence to be picked, the maturity evolution trajectory of the double mushroom under the influence of the predicted environment parameter change sequence in a future period of time is deduced; based on the maturity evolution trajectory, the time required for the maturity index to accumulate and grow from the current value to the preset pickable maturity threshold is calculated by using a numerical integration method, thereby obtaining the pickable prediction time stamp of each double mushroom to be picked accurate to the minute level; finally, based on the pickable prediction time stamp obtained by calculation, combined with the absolute position coordinates of the double mushroom, the corresponding entry in the double mushroom information sequence to be picked is dynamically updated, providing accurate time window constraints and decision basis for the global path planning algorithm of the automatic guided mobile platform and the task queuing system of the six-degree-of-freedom collaborative robot, and realizing the optimal configuration of the harvesting resources in the time and space dimensions.

[0083] Based on the real-time storage time length of the double mushroom corresponding to each position, a double mushroom second picking priority sequence is constructed; it needs to be further explained that the second picking priority sequence in the embodiment is an emergency ranking list established for each double mushroom individual in the target area based on the remaining shelf life; the specific technical means is that, based on the real-time storage time length (i.e. the predicted remaining shelf life) of each double mushroom determined by the maturity-shelf life curve, all double mushrooms that have been determined to be pickable are sorted in order of the time length from short to long, and a second picking priority sequence is generated; among them, the double mushroom with the shortest real-time storage time length has the highest picking priority because it is closest to the spoilage period. The sequence provides a key decision basis for the real-time scheduling of the harvesting system, ensuring that the system can prioritize handling individuals with the most urgent shelf life, thereby effectively reducing post-harvest losses caused by over-ripeness on a global scale.

[0084] The first maturity index corresponding to the pickable double mushroom information sequence of the target area is combined with the real-time storage time length to adaptively cluster the algorithm, and the maturity deviation and real-time storage time length deviation of the double mushroom in the region are minimized to cluster and segment the region, and the pickable double mushroom clustering region sequence of the target area is obtained.

[0085] Based on the first maturity index corresponding to the to-be-picked double mushroom information sequence and the pickable prediction timestamp length of the to-be-picked double mushroom at each position point, the maturity deviation and the pickable prediction timestamp length deviation of the to-be-picked double mushroom in the region are minimized to cluster and segment the region, and the to-be-picked double mushroom clustering region sequence of the target area is obtained.

[0086] The clustering method of the embodiment aims to plan a work logic consistent and execution efficiency optimal harvesting unit for the mobile harvesting platform, rather than simply geometric spatial segmentation. For the pickable double mushroom, the maturity deviation and the real-time storage time length deviation are minimized to cluster, which can ensure that the mushroom groups divided into the same region not only have similar maturity states, but also have uniform configuration of picking parameters such as grabbing force. More importantly, the urgency of the preservation period, i.e. the storage time length, is highly consistent, which enables the AGV to perform one-time and uninterrupted continuous harvesting work on the region, avoiding forced return due to premature decay of individual mushroom bodies in the region, thereby maximizing harvesting efficiency and minimizing post-harvest loss. Similarly, for the to-be-picked double mushroom, the maturity deviation and the pickable prediction timestamp length deviation are minimized to cluster, which can divide the mushroom bodies that are expected to reach the pickable standard successively in the same period into the same future work area. This enables the system to plan an efficient predictive inspection path for the AGV, so that it can reach each to-be-picked area in sequence within the accurate time window, thereby significantly reducing the robot's empty waiting and invalid movement, and realizing the decision upgrade from single-point response to regional batch processing.

[0087] The core motivation of the two clustering methods in the embodiment is based on the deep understanding of the non-uniform distribution of double-gilled mushroom growth in actual planting scenarios. The principle is to optimize clustering based on the dual principles of task type isolation and spatial continuity maintenance. Specifically, in a single physical area, the double-gilled mushrooms that can be picked and those that cannot be picked are randomly mixed in space. If mixed clustering is performed, the decision objectives of the two types of mushrooms (i.e., immediate harvesting and expected harvesting) will be essentially different, resulting in highly fragmented spatial clustering areas and mixed task logic, which will make the execution path of the mobile platform and the robot full of ineffective round trips and waiting, and severely reduce the harvesting efficiency. Therefore, the embodiment adopts a virtual space segmentation method from the perspective of tasks: when clustering the harvestable task, the algorithm temporarily marks and hides the adjacent double-gilled mushrooms in physical location as spatial noise points in calculation, so that the clustering algorithm can focus on identifying hotspots in space that are composed of high maturity mushrooms and have continuous harvesting value; conversely, the same applies to clustering the non-harvestable task. The key advantage of this method is that it builds two independent virtual region sequences that are spatially continuous and have highly unified internal characteristics (maturity and time urgency) at the task decision level without changing the real geographical coordinates of the mushrooms. Ultimately, this enables the path planning algorithm to generate two logically clear and spatially coherent optimal work paths for the AGV and the robot, one for efficiently cleaning the current mature area and the other for prospectively patrolling the area that will mature soon, thereby maximizing harvesting efficiency in complex unstructured environments.

[0088] Based on the harvestable double-gilled mushroom clustering region sequence and the non-harvestable double-gilled mushroom clustering region sequence in the target area, the first priority constraint is constructed by taking the maximum of the first maturity index and the comprehensive real-time storage time length, the second priority constraint is constructed by taking the minimum of the deviation between the harvestable predicted time stamp length and the waiting harvesting time length, and the third priority constraint is constructed by taking the minimum of the real-time energy consumption of the AGV and the harvesting robot. Combined with the particle swarm optimization path planning algorithm, the dynamic harvesting path parameter space is obtained.

[0089] It should be further noted that the comprehensive first maturity index and the comprehensive real-time storage time length in the embodiment are obtained by weighting the first maturity index and the corresponding real-time storage time length of all double-gilled mushrooms in each harvestable double-gilled mushroom clustering region, which represents the comprehensive harvesting priority of each clustering region. The waiting harvesting time length is the waiting time from the current harvestable double-gilled mushroom clustering region to any subsequent harvestable double-gilled mushroom clustering region.

[0090] The core logic of the above three-level priority constraints and the development of dynamic picking path planning in this embodiment is aimed at the core pain points of imbalance of picking timing, high energy consumption of secondary operation, and insufficient utilization rate of matured double-bolet to be picked in large-scale double-bolet picking scenarios in large greenhouses. Based on the physiological characteristics of double-bolet growth and the operation rules of intelligent equipment, a whole-process closed-loop system of priority quantization modeling-time dynamic matching-algorithm iterative optimization is constructed to achieve the coordinated optimization of picking quality, efficiency and cost through precise coupling of technical means. On the technical implementation level, firstly, for the cluster areas of pickable and to-be-picked double-bolet, a weighted summation algorithm is used to quantitatively calculate the comprehensive first maturity index of each cluster area. The weight coefficients are determined by factor analysis of historical picking data. The contribution weights of umbrella color features, gill opening degree, stem hardness and environmental correction coefficient are selected to minimize the damage rate. At the same time, the first priority constraint is constructed by combining the comprehensive real-time storage time length of each cluster area. The initial picking sequence is generated through index normalization processing and sorting operation. The core purpose is to ensure that the cluster areas with high maturity and low storage redundancy are picked first, and to avoid quality loss caused by over-mature and rotten double-bolet or insufficient storage period. Secondly, relying on the growth prediction model optimized by environmental parameters, the pickable prediction timestamp of each to-be-picked cluster area is output. The waiting picking time length from the current time to the planned picking time of the area is obtained through timestamp difference operation. The absolute error or mean square error algorithm is used to construct the deviation evaluation model. The deviation between the pickable prediction timestamp and the waiting picking time length is minimized as the second priority constraint, which is embedded in the dynamic regulation process of the initial picking sequence. Technically, by synchronously outputting the results of the real-time growth prediction model and the waiting time parameter in path planning, the precise matching of the waiting period and the growth period of the to-be-picked double-bolet is achieved, which promotes the to-be-picked double-bolet batch in the area to reach the pickable standard, avoids secondary picking operation in the same physical space from the root, and improves the batch and continuity of picking. Finally, the first two priority constraints are embedded as boundary conditions into the particle swarm optimization algorithm, taking the real-time energy consumption of AGV and picking robot as the fitness function, initializing the particle population to represent the dynamic picking path parameter space, combining the correlation model of intelligent equipment energy consumption characteristic curve and path distance, operation time, and updating the particle position and velocity to optimize the cluster area picking sequence and equipment driving trajectory. Technically, by integrating the path planning node weight distribution mechanism and the energy consumption model parameter calibration, the global optimization of the equipment running trajectory is achieved, which maximizes the reduction of invalid energy consumption and operation cost under the premise of meeting the requirements of picking priority and time matching. The whole process follows the progressive technical path of index quantization modeling-time dynamic matching-algorithm iterative optimization, which not only ensures the priority picking of high-value pickable double-bolet, but also improves the maturity utilization rate of to-be-picked double-bolet through time matching. With the help of algorithm optimization, precise energy consumption control is achieved, providing efficient, stable and economic technical support for large-scale double-bolet picking in large greenhouses.

[0091] Based on each dynamic picking path parameter space combined with a fuzzy control algorithm, a dynamic movement control instruction is obtained.

[0092] It should be further explained that the picking control sub-model of the embodiment is used to obtain the second picking priority of each position point of the double mushroom in each cluster area based on the dynamic picking path parameter space, and generate the picking control instruction of the 3-finger flexible gripper in combination with the first maturity index of each position point. The specific implementation is as follows:

[0093] Firstly, the picking sequence of each cluster area and the second picking priority of each position point in each cluster area are determined according to the output result of the dynamic picking path planning, so as to form the overall picking time sequence planning. The cluster area refers to a double mushroom set divided according to the spatial position and maturity similarity, and the second picking priority refers to the picking sequence of each mushroom in the area determined for optimizing the movement path and obstacle avoidance.

[0094] Secondly, for each target double mushroom in the time sequence planning, the initial picking force and the corresponding damage probability threshold are calculated according to the mapping relationship among the preset maturity, picking force and damage probability, in combination with the first maturity index of the mushroom, and the initial force is set as the input parameter of the impedance control algorithm. The first maturity index refers to the maturity level obtained by comprehensive evaluation through a multi-modal perception system.

[0095] Thirdly, a reinforcement learning reward function is constructed to reduce the damage probability, shorten the picking time and reduce the energy consumption of the gripper, the clamping force, displacement deviation and first maturity index output by the impedance control algorithm are taken as state inputs, and a set of adaptive impedance control stiffness coefficient and damping coefficient are obtained through iterative optimization.

[0096] Fourthly, in the picking execution stage, the contact force signal with the mushroom is collected in real time by using the force sensor built in the three-finger flexible gripper, in combination with the elastic modulus safety threshold of the double mushroom, the stiffness and damping coefficient are dynamically adjusted through the optimized impedance control algorithm, and the displacement increment, safe clamping force upper limit and movement speed parameters of each finger of the gripper are calculated and output in real time.

[0097] Fifthly, based on the displacement increment, safe clamping force upper limit and movement speed parameters of each finger of the gripper combined with the fuzzy control algorithm, the customized picking control instruction for the target double mushroom is generated to drive the gripper to complete picking. After completing the current cluster area operation according to the time sequence planning, the next area is automatically switched and the second step to the fifth step is repeated until all the target ordered harvesting is completed. The stiffness coefficient of the embodiment is set to 500-800 N / m, and the clamping force is set to 2.5-5 N.

[0098] The design motivation of the cooperative sub-model of the embodiment is to solve the core problems of low harvesting efficiency, high mushroom body damage rate, and increased equipment energy consumption caused by the asynchronous execution of mobile control instructions and picking control instructions and poor cooperation. The core principle is based on cross-modal temporal feature fusion and attention mechanism, accurately capturing the cooperative correlation of AGV movement and gripper picking actions, constructing a quantitative cooperative consistency representation, forming a closed loop of instruction execution-cooperation evaluation-feedback optimization, ensuring the timing matching and state adaptation of the two actions, and the specific technical implementation is as follows: based on the AGV path planning information obtained from the dynamic picking path parameter space, the real-time position and speed prediction arrival time of the mobile control sub-model output, and the execution parameters such as the gripper clamping force displacement picking progress output by the picking control sub-model, the multi-dimensional temporal feature sequence of movement and picking is obtained, and the sequence is synchronized processed through timestamp alignment algorithm; the aligned cross-modal temporal feature sequence is input into the cooperative attention network, taking the movement state feature as the query vector and the picking state feature as the key value vector, and through the attention weight calculation, the key parameters affecting the cooperation are focused, such as the time deviation of AGV arriving at the target area and the matching degree of the gripper readiness state, the time difference between the starting time of the gripper action and the positioning completion time of the AGV, the adaptability of the AGV moving speed and the picking rhythm of the gripper, etc., realizing the deep fusion of cross-modal features; based on the fused features, the cooperative feature vector is output through full connection layer and normalization processing, which covers time synchronization deviation coefficient, position matching accuracy, action cooperation efficiency, etc. quantitative dimensions, accurately representing the cooperative consistency of the movement and picking instruction execution process; the cooperative feature vector is fed back to the reward function of the coordination reinforcement learning model, and the reward weight is dynamically adjusted combined with the indicators such as preservation period picking damage rate and energy consumption, and the reward value is reduced if the cooperative feature vector shows time deviation or position misalignment; at the same time, the cooperative feature vector is input into the mobile control sub-model and the picking control sub-model respectively, the moving speed adjustment timing of the AGV, the positioning accuracy compensation parameters, and the action starting time sequence of the gripper, the clamping force adjustment rhythm are corrected in real time, the dynamic cooperative optimization of the mobile and picking instructions is realized, and the AGV accurately positions the gripper to start picking in time after the picking is completed, the AGV quickly switches to the next area, and the overall cooperation, stability and efficiency of the harvesting are improved.

[0099] It needs to be further explained that the embodiment allocates a unique identification two-dimensional code to each pair of Boletus or harvesting batch, which uses a dynamic generation algorithm combined with harvesting timestamp, greenhouse area coordinates and batch number to ensure uniqueness. The two-dimensional code stores information including key data throughout the harvesting process, specifically covering the first maturity index of Boletus growth stage, and the basic data constituting the index, such as cap color characteristics, gill opening degree characteristics, stem hardness, environmental correction coefficient, etc.; picking control instruction parameters during the harvesting execution process, including the clamping force of the 3-finger flexible gripper, stiffness damping coefficient, picking timing, etc.; cooperative control execution data, including AGV moving speed, positioning accuracy, instruction synchronization deviation, cooperative feature vector, etc.; and post-harvest quality detection data, covering multi-dimensional indicators such as mushroom integrity, gill damage degree, storage duration, and rotting condition. In the sorting, storage, sales and other links after harvesting, the post-harvest quality data of the corresponding nodes is collected in real time through the code scanning equipment. After the collected data is standardized in format, filtered for noise and time-aligned by the edge computing node, it is uploaded to the model management system through an encrypted transmission protocol, and the whole process data associated with the two-dimensional code is matched and integrated to build a full-link traceable data closed loop from planting perception, harvesting execution to post-harvest quality.

[0100] It needs to be further explained that the embodiment also configures a model management subsystem, which carries a timed task scheduling module that automatically triggers an incremental update process every morning. Before updating, the system first filters and cleans the newly added data for the day, based on the post-harvest quality data traceability results, and selects mature degree assessment accurate cases and assessment deviation cases. The accurate cases need to meet the condition that the matching degree of the first maturity index and the actual post-harvest quality is higher than the preset threshold, and the deviation cases need to clearly identify the assessment deviation reasons and corresponding impact factors. From the filtered data, Boletus RGB images, short-wave near-infrared images and corresponding cap color characteristics, gill opening degree, moisture content characteristics, stem hardness and other multi-modal feature data are extracted to build daily annotation samples, with the number of annotation samples strictly controlled within one hundred to balance the model update efficiency and stability, and to avoid the risk of excessive sample size leading to long update time and overfitting. The sample annotation process uses a semi-automatic method combined with manual verification to ensure annotation accuracy.

[0101] It needs to be further explained that the embodiment adopts a transfer learning algorithm for online incremental updating, specifically: for the multi-modal fusion model, the core parameters of the MobileNetV3 lightweight network bottom convolution layer and the one-dimensional convolution of the spectrum extraction layer are frozen, and only the top full connection sublayer of the image extraction layer, the feature fusion sublayer of the spectrum extraction layer, the edge detection layer parameters and the linear evaluation layer weight are fine-tuned; for the coordination reinforcement learning model, the focus is on fine-tuning the maturity-picking intensity-damage probability mapping relationship parameters in the picking control submodel and the reinforcement learning reward function weight, the attention allocation coefficient of the collaborative attention network in the coordination submodel, and the path planning related parameters in the mobile control submodel; for the growth prediction model optimized by environmental parameters, the hidden layer parameters and output layer weight of the LSTM network are fine-tuned. During the updating process, the core parameters and training results of the historical model are retained, and only the model adaptability parameters are dynamically adjusted based on the newly added labeled samples. After the update is completed, the model performance evaluation module is used to verify the updated model through accuracy, recall rate, damage rate reduction amplitude and other indicators. After verification, the updated model parameters are deployed to the actual application system; if not, return to adjust the sample screening conditions or model update parameters and re-execute the update process.

[0102] The whole process of the embodiment is embedded in the PDCA cycle mechanism: in the planning stage, based on the quality short board in the traceability data and the model performance bottleneck, the daily incremental update target is formulated; in the execution stage, sample construction, model updating and preliminary verification are completed; in the inspection stage, the changes of the model in maturity evaluation accuracy, picking damage rate, control instruction cooperativity and other core indicators before and after updating are compared, and the correlation between postharvest quality improvement effect and model parameter adjustment is analyzed; in the processing stage, the model update results verified are solidified to form standard parameter configuration, and the data acquisition loopholes or feature extraction deficiencies found in the updating process are optimized by optimizing the front-end sensor acquisition strategy and feature extraction algorithm, and typical deviation cases and solutions are included in the knowledge base to support subsequent model optimization; in this way, the continuous iteration and optimization of the model and the whole link traceability of the postharvest quality are realized, and the maturity evaluation accuracy, control instruction cooperativity and postharvest quality guarantee capability of the harvesting system are dynamically improved with the application scene.

[0103] The present double mushroom harvesting method realizes the coordinated optimization of harvesting precision, efficiency and quality by integrating multi-modal sensing technology, coordinating reinforcement learning and dynamic path planning. The core beneficial effects are reflected in the significant reduction of harvesting damage rate, the effective extension of post-harvest preservation period, the precise control of overall energy consumption and the continuous enhancement of system adaptive ability. The specific effect derivation starts from the precise data acquisition and fusion of multi-sensor array. The RGB and short-wave near-infrared cameras are synchronized by FPGA hardware and spatially registered by Zhang's calibration method, solving the spatio-temporal misplacement problem of cross-modal images. Combined with the dynamic switching mechanism of day and night weight, the extraction stability of umbrella color features and gill moisture content features under different light conditions is ensured. Then, the multi-modal fusion model extracts visual and spectral features through the image extraction layer and the spectrum extraction layer respectively, and realizes the spatial mapping correction of the first umbrella fold features and the second umbrella spectrum fold features by using the double-sensor pixel conversion matrix. The gill opening degree features are obtained through the edge detection layer, and the consistency verification of physiological driving and morphological performance is carried out with the moisture content features, effectively avoiding misjudgment caused by physical damage, and improving the reliability of maturity assessment from the data source. On this basis, the wax layer thickness and the internal hollow degree of the stem collected by the laser spectrometer and the millimeter wave radar are converted into the stem hardness through the hardness evaluation layer, and the environmental correction coefficient is generated by combining the temperature and humidity parameters obtained by the environmental sensor. Finally, the maturity assessment weight sequence determined by factor analysis makes the first maturity index output by the linear evaluation layer have the ability to represent the growth state and predict the damage risk, providing accurate input for subsequent decision-making. The effect is further derived to the harvesting decision-making stage. The maturity-preservation period curve converts the real-time first maturity index into the preservation period constraint, and generates the pickable and non-pickable double mushroom information sequence by combining the comprehensive coverage of the initial identification path. The mobile control sub-model dynamically calculates the pickable timestamp of the non-pickable double mushroom through the growth prediction model optimized by environmental parameters, and forms a logically coherent harvesting unit sequence by region clustering with the minimum maturity deviation and time deviation as the goal. Based on the three-level priority constraint (comprehensive maturity and preservation period priority, minimum time matching deviation, and minimum energy consumption), the particle swarm optimization path planning generates a dynamic picking path parameter space, ensuring that high-value areas are picked first while avoiding secondary operations, thereby reducing over-mature loss and ineffective movement from the decision-making level. The picking control sub-model generates the clamping force and stiffness parameters of the flexible gripper based on the maturity-picking force-damage probability mapping relationship and the impedance control algorithm optimized by reinforcement learning, which adapts to the physical properties of double mushrooms with different maturity on the premise of ensuring low-damage picking. The collaborative sub-model solves the timing mismatch problem of AGV movement and gripper action by fusing the cross-modal time sequence features of movement and picking through the collaborative attention network, outputting the collaborative feature vector quantization instruction execution consistency and feeding back to the reward function to adjust the control parameters in real time.The system also updates daily increments of the two-dimensional code full-process traceability and model management subsystem, reversely optimizes multi-modal fusion model and coordination reinforcement learning model parameters based on postharvest quality data, forms a closed-loop iteration from perception decision to execution feedback, dynamically improves maturity recognition accuracy, instruction coordination consistency and postharvest quality guarantee capability with application scenarios, and provides reliable technical support for large-scale intelligent harvesting.

[0104] Embodiment 2

[0105] Please refer to Figure 3 Another embodiment provided by the present application is an intelligent greenhouse double-belly mushroom harvesting system based on multi-modal perception and dynamic decision, which comprises a collection module, a discrimination module, a collaborative control module and an execution module.

[0106] The collection module collects a first maturity index representing the maturity of the target area of the double-belly mushroom through a configured sensor array; the first maturity index is obtained by combining the color feature of the double-belly mushroom cap, the moisture content feature of the lobe, the thickness of the wax layer on the cap surface, the internal hollow degree of the stem, the environmental correction coefficient and the expert evaluation algorithm;

[0107] The discrimination module performs collection first discrimination based on the first maturity index and a maturity-keeping period curve to obtain a harvestable double-belly mushroom information sequence and a to-be-harvested double-belly mushroom information sequence; the harvestable double-belly mushroom information sequence at least includes position information and corresponding first maturity, and the to-be-harvested double-belly mushroom information sequence at least includes position information and corresponding first maturity as well as a harvestable time stamp;

[0108] The collaborative control module generates a collaborative control instruction based on the harvestable double-belly mushroom information sequence and the to-be-harvested double-belly mushroom information sequence, a coordination reinforcement learning model, a preservation period, a harvesting damage rate, a coordinated harvesting energy consumption, a maturity recognition accuracy and a reward function constructed by a delay of instruction coordination consistency;

[0109] The execution module is configured to execute the collaborative control instruction.

[0110] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above specific embodiments, and the above specific embodiments are only illustrative and not restrictive. Those skilled in the art can make changes, modifications, replacements and variations to the above embodiments without departing from the purpose and scope of the present application, and these are all within the protection scope of the present application.

[0111] If the technical solutions of the present disclosure involve personal information, the product applying the technical solutions of the present disclosure has clearly informed the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solutions of the present disclosure involve sensitive personal information, the product applying the technical solutions of the present disclosure has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of explicit consent. For example, at the personal information collection device such as a camera, a clear and prominent sign is set to inform that the personal information collection range has been entered and the personal information will be collected. If the person voluntarily enters the collection range, it is considered to agree to collect the personal information. Or on the device for processing personal information, through the pop-up information or by asking the person to upload his / her personal information, the personal authorization is obtained under the condition that the obvious sign / information informs the personal information processing rules. The personal information processing rules can include the personal information processor, the purpose of processing personal information, the processing method, the type of processed personal information and other information.

Claims

1. A method for intelligent harvesting of bicolored mushrooms in a greenhouse using multi-modal perception and dynamic decision making, characterized in that, The method comprises the following steps: Collecting a first maturity index representing the maturity of the target area of the double-booth mushroom through a configured sensor array; the first maturity index is obtained by combining the color characteristics of the double-booth mushroom cap, the moisture content characteristics of the gill, the thickness of the wax layer on the cap surface, the internal hollow degree of the stem, the environmental correction coefficient, and a pre-set multi-modal fusion model; Based on the first maturity index and the maturity- shelf life curve, a first discrimination is performed to obtain a pickable double-booth mushroom information sequence and a to-be-picked double-booth mushroom information sequence; the pickable double-booth mushroom information sequence at least includes position information and corresponding first maturity, and the to-be-picked double-booth mushroom information sequence at least includes position information and corresponding first maturity and a pickable time stamp; Based on the pickable double-booth mushroom information sequence and the to-be-picked double-booth mushroom information sequence, a reward function is constructed based on a coordinated reinforcement learning model, a shelf life, a picking damage rate, a coordinated picking energy consumption, a maturity recognition accuracy, and an instruction coordination consistency delay, and a collaborative control instruction is generated; The collaborative control instruction is executed.

2. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 1, wherein, The coordinated reinforcement learning model includes a picking control sub-model, a movement control sub-model, and a coordination sub-model; the movement control sub-model is used to generate a movement control instruction of an AGV car according to the position information of each pickable double-booth mushroom in the pickable double-booth mushroom information sequence and the pickable time stamp of the to-be-picked double-booth mushroom combined with a path optimization algorithm; the picking control sub-model is used to generate a picking control instruction of a 3-finger flexible gripper according to the first maturity index, a maturity-picking force-damage probability mapping relationship, and an impedance control algorithm optimized by reinforcement learning; the coordination sub-model is used to generate a coordination feature vector according to the movement and picking consistency requirements combined with a collaborative attention network, which is used to represent the collaborative consistency of the movement control instruction and the picking control instruction in the execution process.

3. The intelligent dual-purpose mushroom harvesting method of a multi-modal perception and dynamic decision-making greenhouse according to claim 2, wherein, The multi-modal fusion model includes an image extraction layer, a spectrum extraction layer, an edge detection layer, a hardness evaluation layer, and a linear evaluation layer. Collecting a first maturity index representing the maturity of the target area of the double-booth mushroom, comprising: Synchronizing the exposure time and spatial registration of the configured RGB camera and short-wave near-infrared camera through FPGA hardware and Zhang's calibration method to establish a double-sensor image coordinate system and a double-sensor pixel conversion matrix; the double-sensor pixel conversion matrix is used to represent the spatial conversion relationship of the RGB image and the short-wave near-infrared image at the same pixel point; Collecting double-booth mushroom RGB image sequences and double-booth mushroom umbrella surface wrinkle infrared image sequences in the double-sensor image coordinate system, and performing spatial and temporal alignment processing on the collected two kinds of images; Inputting the collected double-booth mushroom RGB image sequence into the image extraction layer of the pre-set multi-modal fusion model to extract the cap color feature and the first cap wrinkle feature.

4. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 3, wherein, Collecting a first maturity index representing the maturity of the target area of the double-booth mushroom, further comprising: Meanwhile, the double-belly mushroom umbrella surface fold infrared image sequence is input into a preset spectrum extraction layer to extract second umbrella surface spectrum fold characteristics and water content characteristics, and the first umbrella cover fold characteristics are mapped to the second umbrella surface spectrum fold characteristics through a double-sensor pixel conversion matrix to obtain corrected second umbrella surface spectrum fold characteristics and water content characteristics; Based on the corrected second umbrella surface spectrum fold characteristics, an edge detection layer is combined to obtain double-belly mushroom gill opening degree characteristics; Based on the double-belly mushroom gill opening degree characteristics, consistency verification is performed on the water content characteristics of the corresponding double-belly mushroom gill to obtain verified double-belly mushroom gill opening degree.

5. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 4, wherein, The first maturity index representing the maturity of the double-belly mushroom in the target area also includes: The wax layer thickness of the double-belly mushroom cap surface is collected by a configured laser spectrometer, and the hollow degree inside the stem is collected by a millimeter wave radar, and the collected data is input into a configured filter for filtering and time alignment; The aligned wax layer thickness and the hollow degree inside the stem are input into the hardness evaluation layer to obtain the double-belly mushroom stem hardness; An environmental evaluation parameter is obtained by a configured environmental sensor, including temperature, humidity, CO2 concentration, and light intensity; Based on the environmental evaluation parameter, an environmental correction coefficient is obtained by combining an environmental coefficient mapping table.

6. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 5, wherein, The first maturity index representing the maturity of the double-belly mushroom in the target area also includes: The historical first maturity index, the corresponding collection damage rate, the umbrella cover color characteristics, the double-belly mushroom gill opening degree, the double-belly mushroom stem hardness, and the environmental correction coefficient are obtained, and factor analysis is performed on the collection damage rate as the first target factor and the first maturity index as the second target factor to obtain the contribution weight of the umbrella cover color characteristics, the double-belly mushroom gill opening degree, the double-belly mushroom stem hardness, and the environmental correction coefficient to the first target factor, which is used as a maturity evaluation weight sequence; The umbrella cover color characteristics, the double-belly mushroom gill opening degree, the double-belly mushroom stem hardness, the environmental correction coefficient, and the maturity evaluation weight sequence are input into a linear evaluation layer to obtain the first maturity index of each double-belly mushroom in the target area.

7. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 6, wherein, The acquisition of the pickable double-belly mushroom information sequence and the to-be-picked double-belly mushroom information sequence includes: An initial identification path is preset, and real-time maturity monitoring and identification of double-belly mushrooms are performed along the initial identification path in the target area to obtain real-time first maturity indexes of double-belly mushrooms corresponding to each position in the target area; Based on the real-time first maturity indexes of double-belly mushrooms corresponding to each position, a preset maturity-shelf life curve is combined to obtain a real-time storage time length of double-belly mushrooms corresponding to each position; A maturity discrimination threshold is preset, and when the real-time first maturity index of a double-belly mushroom corresponding to a position in the target area is greater than the maturity discrimination threshold, the corresponding position is determined to be a pickable double-belly mushroom, otherwise it is a to-be-picked double-belly mushroom, and a pickable double-belly mushroom information sequence and a to-be-picked double-belly mushroom information sequence are obtained.

8. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 7, wherein, The movement control instruction includes: Based on the real-time first maturity index of the double-bolet at each position point in the double-bolet information sequence to be picked and a preset environment parameter optimized growth prediction model, a pickable prediction time stamp of the double-bolet to be picked at each position point in the double-bolet information sequence to be picked is obtained; Based on the real-time storage time length of the double-bolet at each position, a double-bolet second picking priority sequence is constructed; The first maturity index and the real-time storage time length of the pickable double-bolet information sequence of the target area are combined with an adaptive clustering algorithm, and the maturity deviation and the real-time storage time length deviation of the double-bolets in the target area are minimized to perform regional clustering segmentation, and a pickable double-bolet clustering region sequence of the target area is obtained.

9. The intelligent dual-purpose mushroom harvesting method of a smart greenhouse with multi-modal perception and dynamic decision making as claimed in claim 8, wherein, The movement control instruction further comprises: Based on the first maturity index corresponding to the double-bolet information sequence to be picked and the pickable prediction time stamp length of the double-bolet to be picked at each position point, the maturity deviation and the pickable prediction time stamp length deviation of the double-bolets to be picked in the target area are minimized to perform regional clustering segmentation, and a double-bolet clustering region sequence to be picked in the target area is obtained; Based on the pickable double-bolet clustering region sequence of the target area and the double-bolet clustering region sequence to be picked, a first picking priority is constructed by combining the first maturity index and the real-time storage time length, a second priority constraint is obtained by minimizing the deviation of the pickable prediction time stamp length and the waiting picking time length, a third priority constraint is obtained by minimizing the real-time energy consumption of the AGV and the picking robot, and a dynamic picking path parameter space is obtained by combining a particle swarm optimization path planning algorithm; Based on the dynamic picking path parameter space and a fuzzy control algorithm, a dynamic movement control instruction is obtained.

10. An intelligent greenhouse double coelomyces harvesting system with multi-modal perception and dynamic decision making, for implementing the intelligent greenhouse double coelomyces harvesting method with multi-modal perception and dynamic decision making of any one of claims 1-9, characterized in that, It comprises: a collection module, a discrimination module, a collaborative control module, and an execution module; The collection module collects a first maturity index representing the maturity of double-bolets in the target area through a configured sensor array; the first maturity index is obtained by combining the double-bolet cap color feature, the laceration moisture content feature, the cap surface wax layer thickness, the stem internal hollow degree, the environmental correction coefficient, and the expert evaluation algorithm; The discrimination module performs collection first discrimination based on the first maturity index and a maturity-preservation period curve to obtain a pickable double-bolet information sequence and a double-bolet information sequence to be picked; the pickable double-bolet information sequence at least includes position information and corresponding first maturity, and the double-bolet information sequence to be picked at least includes position information, corresponding first maturity, and a pickable time stamp; The collaborative control module generates a collaborative control instruction based on the pickable double-bolet information sequence and the double-bolet information sequence to be picked, a coordination reinforcement learning model, a preservation period, a picking damage rate, a coordinated picking energy consumption, a maturity recognition accuracy, and a delay of instruction coordination consistency; The execution module is configured to execute the collaborative control instruction.

Images