Vision-based multi-modal data fusion intelligent monitoring culture method and system

Abstract

The invention belongs to the technical field of agricultural sensing data management and control, and particularly relates to a vision-based multi-modal data fusion intelligent monitoring culture method and system, and the method comprises the steps: obtaining the growth image data and environmental parameter data of a strain based on a multi-sensor information source; identifying the growth image data through a deep learning algorithm, and outputting an image identification result; performing fusion processing on the image recognition result and the environment parameter data through a multi-data fusion algorithm to generate a comprehensive environment evaluation index; importing the comprehensive environment evaluation index and the abnormal state information into a growth environment control model, generating an environment regulation and control instruction conforming to a preset strain growth index, and executing the environment regulation and control instruction through an environment regulation and control module; the system can be used for realizing comprehensive environment monitoring, accurately evaluating the strain growth environment condition, and avoiding errors and one-sidedness caused by a single data source, so that culture environment parameters are dynamically adjusted.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural sensor data management and control, and specifically relates to an intelligent monitoring and cultivation method and system based on vision-based multimodal data fusion. Background Art

[0002] Due to the particularity of the edible fungus growth environment, in order to improve the efficiency and refinement of the environmental control of the edible fungus culture cabin, it is necessary to carry out intelligent monitoring of the edible fungus culture cabin, apply the Internet of Things technology to the intelligent monitoring of the mushroom house, and use a large number of sensors and other equipment in series with the data management platform to provide accurate and reliable perception sensing signals for the back-end automation control.

[0003] However, the sensor network in the existing edible fungus cultivation cabin is prone to data drift, instability, and inaccuracy under harsh working conditions such as high temperature and high humidity, making it impossible to achieve optimal environmental control and making it difficult to make the environment in the edible fungus cultivation cabin most suitable for the growth of edible fungi. In addition, the current edible fungus cultivation cabin focuses on relatively single indicators during the cultivation process, rigidly setting system values, ignoring the development status of the mushroom sticks themselves, resulting in energy waste. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention provides an intelligent monitoring and cultivation method and system based on vision-based multimodal data fusion to solve the problems of data drift, instability, and inaccuracy in the existing intelligent monitoring and cultivation cabin sensor network under harsh working conditions such as high temperature and high humidity, which cannot achieve optimal environmental control, make it difficult to make the environment in the mushroom house most suitable for the growth of edible fungi, and ignore the development status of the mushroom sticks themselves, resulting in energy waste.

[0005] One solution of the present invention provides an intelligent monitoring and cultivation method and system based on multimodal data fusion based on vision, which is applied to edible fungus cultivation cabins and specifically includes the following steps:

[0006] S1. Acquire bacterial growth image data and environmental parameter data based on a multi-sensor information source, wherein the environmental parameter data at least includes ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity;

[0007] S2. Recognize the growth image data using a deep learning algorithm and output an image recognition result, wherein the image recognition result includes at least bacterial growth rate, morphological characteristics, and abnormal state information;

[0008] S3, fusing the image recognition result and the environmental parameter data through a multi-data fusion algorithm to generate a comprehensive environmental assessment index;

[0009] S4. Importing the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate environmental control instructions that meet preset bacterial strain growth indicators, wherein the growth environment control model is trained based on historical growth image data of the bacterial strain and corresponding environmental parameter data;

[0010] S5. Execute the environmental control instructions through the environmental control module to dynamically adjust the culture environment parameters.

[0011] In one of the solutions of the present invention, the multi-sensor information source includes an environmental parameter sensor node and an image sensor node;

[0012] Several environmental parameter sensor nodes are deployed at multiple points inside the edible fungus cultivation cabin so that the sensing data of the several environmental parameter sensor nodes are evenly distributed in space. The environmental parameter sensor nodes include at least a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, and a light sensor.

[0013] The control range of the plurality of image sensor nodes covers the entire planting area in the edible fungus cultivation cabin, the image sensor nodes are connected to the Internet of Things via a wireless connection module or a wired connection module, and any image sensor node is communicatively connected to at least another adjacent image sensor node;

[0014] The temperature sensor and humidity sensor are manufactured using MEMS technology and have a waterproof and dustproof structure.

[0015] In one solution of the present invention, the step S1 further includes preprocessing the environmental parameter data:

[0016] S11: Obtain raw data that needs to be preprocessed based on the time series, use a filtering algorithm to remove noise from the sensor data, and check whether there are duplicate values, abnormal values, or missing values in the raw data;

[0017] S12: Cleaning the original data containing duplicate values, abnormal values, or missing values, and outputting the data as preprocessed data, wherein the cleaning includes: deleting duplicate values in the data, or deleting, correcting, or replacing abnormal values in the data, or filling missing values in the data;

[0018] S13: Importing the pre-processed data into an exponential smoothing model to obtain a smoothed value based on a time series, thereby forming a smoothed data sequence.

[0019] In one solution of the present invention, the step S1 further includes decoupling the environmental parameter data:

[0020] S14: Decoupling the smoothed data sequence through a decoupling algorithm to obtain environmental influencing factors corresponding to ambient temperature, relative air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity;

[0021] S15: Standardize the decoupled data of each environmental impact factor to eliminate the influence of different dimensions and dimensional units.

[0022] In one embodiment of the present invention, the step S2 of identifying the growth image data using a deep learning algorithm specifically includes:

[0023] S21, extracting features from the growth image data using an edge detection algorithm or a contour extraction algorithm to obtain size changes or area changes of corresponding bacterial species at different time points, and outputting a growth rate based on the size changes or area changes;

[0024] S22. Using the trained deep learning model to perform feature extraction on the growth image data, to obtain semantic features corresponding to the growth image data;

[0025] S23, extracting current morphological features of the strain through semantic features, wherein the morphological features include color distribution and shape features;

[0026] S24. Calculate a growth anomaly value by comparing the morphological features extracted from the growth image data with the preset morphological features, and compare the calculated growth anomaly value with a growth anomaly threshold. If the calculated growth anomaly value is greater than the growth anomaly threshold, output the morphological features and the anomaly value extracted from the growth image data as abnormal state information.

[0027] In one of the solutions of the present invention, step S2 further includes: a neural network model constructed based on the YOLO neural network method, performing real-time target detection and classification of the bacterial growth stage through growth image data and environmental parameter data, and constructing a bacterial growth status database.

[0028] In one embodiment of the present invention, step S2 further includes:

[0029] S25, fusing the extracted semantic features, color distribution features, and shape features to form a comprehensive feature vector, wherein the comprehensive feature vector is used to describe the comprehensive features of the bacterial species;

[0030] S26. Importing the comprehensive feature vector into a trained classifier for classification to obtain the growth stage of the strain at the current time.

[0031] In one embodiment of the present invention, the construction and training of the deep learning algorithm model specifically includes:

[0032] An initial model is built based on a convolutional neural network. The initial model includes multiple convolutional layers and pooling layers to automatically extract features from bacterial growth image data.

[0033] Collect bacterial growth image data and environmental parameter data, and pre-process the data;

[0034] The preprocessed data is divided into training set, validation set and test set. The training set is used to initialize the model and the parameters of the model are continuously adjusted through the back propagation algorithm to minimize the loss function.

[0035] Use the validation set to evaluate the performance of the model to prevent overfitting, and adjust the model's hyperparameters based on the results of the validation set;

[0036] The test set is used to evaluate the initial model after training, and the model whose output evaluation results meet the preset requirements is the deep learning algorithm model.

[0037] In one solution of the present invention, the fusion processing of the image recognition result and the environmental parameter data in step S3 specifically includes:

[0038] S31, obtaining a smoothed and standardized data sequence;

[0039] S32, importing the environmental parameter data into an adaptive trust model, determining the trust relationship between the data nodes based on the reliability and historical performance of several data nodes in the multi-sensor information source, and establishing a trust matrix;

[0040] S33. Performing weighted optimization on the standardized data sequence using the trust matrix to increase the weight of reliable data and reduce the weight of unreliable data, and outputting the weighted optimized data sequence;

[0041] S34. The weighted optimized data sequence and the image recognition result are fused by a multi-data fusion algorithm to generate a comprehensive environmental assessment index.

[0042] In one solution of the present invention, the step S4 of importing the comprehensive environmental assessment index and the abnormal state information into the growth environment control model further includes:

[0043] S41, obtaining the current growth stage of the strain based on the image recognition result, and outputting the growth stage recognition result;

[0044] S42. Based on the growth stage identification result, match the growth environment control model corresponding to the current growth stage.

[0045] In one embodiment of the present invention, the growth stage identification result includes one of the following growth stages: mycelium growth stage, fruiting body primordium formation stage, fruiting body young ear growth stage or fruiting body adult ear color change stage.

[0046] In one embodiment of the present invention, step S42 specifically includes reading the growth stage identification result, and matching a growth environment control model based on key control points affecting yield quality within the growth stage;

[0047] If the growth stage identification result is the mycelial growth stage, the key control point affecting yield quality is the day-night temperature difference, and the first growth environment control model is matched;

[0048] Alternatively, if the growth stage identification result is the fruiting body primordium formation period, the key control point affecting yield and quality is the ambient temperature, and the second growth environment control model is matched;

[0049] Alternatively, if the growth stage identification result is the fruiting body young ear growth stage, the key control point affecting yield and quality is the air relative humidity range, and the third growth environment control model is matched;

[0050] Alternatively, if the growth stage identification result is the fruiting body ear-forming and color-changing period, the key control point affecting yield quality is light intensity, which matches the fourth growth environment control model.

[0051] In one embodiment of the present invention, the step S4 of generating the environmental control instruction that meets the preset bacterial growth index specifically includes:

[0052] Obtaining a growth abnormality value of a corresponding bacterial species through the abnormal state information;

[0053] Inputting the growth abnormality value and the comprehensive environmental assessment index into the growth environment control model to obtain a compensation value corresponding to the environmental parameter data and an environmental control instruction corresponding to the compensation value;

[0054] Among them, the growth abnormality value is used to reflect the difference between the current morphological characteristics of the strain and the preset morphological characteristics, and the environmental control instruction corresponds to at least the adjustment amount of environmental parameters such as ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity.

[0055] In one of the schemes of the present invention, the growth environment control model is constructed based on a fuzzy algorithm, and after executing the environmental adjustment instruction, the growth status of the strain and the changes in environmental parameters are monitored in real time, and the new growth anomalies and comprehensive environmental assessment indicators are fed back to the fuzzy algorithm system, and the control rules and membership functions are optimized and updated to form a closed-loop control.

[0056] In one of the solutions of the present invention, the method further includes: uploading the growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal status information to a cloud monitoring platform through an Internet of Things communication module, and realizing remote control and abnormal status warning based on a visual interface.

[0057] One of the solutions of the present invention also provides an intelligent monitoring and cultivation system based on multimodal data fusion based on vision, including:

[0058] The data acquisition module is used to obtain bacterial growth image data and environmental parameter data through multiple sensor information sources;

[0059] A data recognition module is used to recognize the growth image data through a deep learning algorithm and output an image recognition result;

[0060] A comprehensive environmental assessment module is used to fuse the image recognition results and the environmental parameter data using a multi-data fusion algorithm to generate a comprehensive environmental assessment index;

[0061] A control instruction generation module is used to import the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate an environmental control instruction that meets the preset bacterial growth index;

[0062] The environment control module is used to execute the environment control instructions and dynamically adjust the culture environment parameters.

[0063] In one of the schemes of the present invention, the system also includes a bacterial strain growth status database, which has a neural network model constructed based on the YOLO neural network method, and performs real-time target detection and classification of bacterial strain growth stages through growth image data and environmental parameter data to form standard morphological characteristics of bacterial strains at different growth stages.

[0064] In one of the solutions of the present invention, a cloud monitoring platform is also included, which is used to receive the growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal status information uploaded by the Internet of Things communication module.

[0065] The intelligent monitoring and cultivation method and system based on vision-based multimodal data fusion provided by the present invention can achieve the following technical effects:

[0066] 1. Through multiple sensor information sources, including environmental parameter sensor nodes and image sensor nodes, bacterial growth image data and environmental parameter data are obtained, and the image recognition results and environmental parameter data are fused and processed to generate comprehensive environmental assessment indicators. The multimodal data fusion of the present invention realizes comprehensive environmental monitoring and can more accurately assess the overall condition of the bacterial growth environment, avoiding the errors and one-sidedness that may be caused by a single data source.

[0067] 2. The current growth stage of the strain is obtained through image recognition results, and the growth environment control model corresponding to the current growth stage is matched. The growth abnormality value and the comprehensive environmental assessment index are input into the growth environment control model to obtain the compensation value of the environmental parameter data and the environmental control instruction corresponding to the compensation value. Therefore, the present invention can accurately control the key control points of different growth stages.

[0068] 3. Use fuzzy algorithms to construct a growth environment control model. By defining the membership functions of input and output variables, formulating control rules, performing fuzzy reasoning and defuzzification steps, generating and executing environmental adjustment instructions, the growth status of the strain and changes in environmental parameters are monitored in real time. New growth anomalies and comprehensive environmental assessment indicators are fed back to the fuzzy algorithm system to achieve dynamic adjustment of environmental parameters, optimize and update control rules and membership functions, and form a closed-loop control. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0070] Figure 1 A schematic diagram showing the process of the present invention;

[0071] Figure 2 A schematic diagram showing a flow chart of data preprocessing of environmental parameter data according to the present invention;

[0072] Figure 3 A schematic diagram showing a process of decoupling environmental parameter data according to the present invention;

[0073] Figure 4 A schematic diagram showing a process of identifying growth image data according to the present invention;

[0074] Figure 5 A schematic diagram showing a process for identifying growth stages according to the present invention;

[0075] Figure 6 A schematic diagram showing the process of fusing image recognition results and environmental parameter data according to the present invention;

[0076] Figure 7 A schematic diagram showing the flow of the present invention for importing comprehensive environmental assessment indicators and abnormal state information into a growth environment control model. DETAILED DESCRIPTION

[0077] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0078] Please refer to Figure 1 One embodiment of the present invention provides an intelligent monitoring and cultivation method and system based on multimodal data fusion based on vision, which is applied to an edible fungus cultivation cabin and specifically includes the following steps:

[0079] S1. Acquire bacterial growth image data and environmental parameter data based on a multi-sensor information source, wherein the environmental parameter data at least includes ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity;

[0080] S2. Recognize the growth image data using a deep learning algorithm and output an image recognition result, wherein the image recognition result includes at least bacterial growth rate, morphological characteristics, and abnormal state information;

[0081] S3, fusing the image recognition result and the environmental parameter data through a multi-data fusion algorithm to generate a comprehensive environmental assessment index;

[0082] S4. Importing the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate environmental control instructions that meet preset bacterial strain growth indicators, wherein the growth environment control model is trained based on historical growth image data of the bacterial strain and corresponding environmental parameter data;

[0083] S5. Execute the environmental control instructions through the environmental control module to dynamically adjust the culture environment parameters.

[0084] It can be understood that in this embodiment, multi-sensor information sources, including environmental parameter sensor nodes and image sensor nodes, are used to obtain bacterial growth image data and environmental parameter data, and the image recognition results and environmental parameter data are fused and processed to generate comprehensive environmental assessment indicators. The multimodal data fusion of the present invention realizes comprehensive environmental monitoring, which can more accurately evaluate the overall condition of the bacterial growth environment and avoid the errors and one-sidedness that may be caused by a single data source.

[0085] In one embodiment of the present invention, the multi-sensor information source includes an environmental parameter sensor node and an image sensor node;

[0086] Several environmental parameter sensor nodes are deployed at multiple points inside the edible fungus cultivation cabin so that the sensing data of the several environmental parameter sensor nodes are evenly distributed in space. The environmental parameter sensor nodes include at least a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, and a light sensor.

[0087] The control range of the plurality of image sensor nodes covers the entire planting area in the edible fungus cultivation cabin, the image sensor nodes are connected to the Internet of Things via a wireless connection module or a wired connection module, and any image sensor node is communicatively connected to at least another adjacent image sensor node;

[0088] The temperature sensor and humidity sensor are manufactured using MEMS technology and have a waterproof and dustproof structure.

[0089] It can be understood that the image sensor nodes in this embodiment can not only capture the growth status of edible fungi in real time, but also ensure the integrity and accuracy of the data through mutual communication connections.

[0090] Alternatively, when an image sensor node fails or the data is abnormal, the adjacent image sensor nodes can be immediately supplemented and verified, thereby greatly improving the reliability and stability of the monitoring system.

[0091] Please refer to Figure 2 In one embodiment of the present invention, the step S1 further includes preprocessing the environmental parameter data:

[0092] S11: Obtain raw data that needs to be preprocessed based on the time series, use a filtering algorithm to remove noise from the sensor data, and check whether there are duplicate values, abnormal values, or missing values in the raw data;

[0093] S12: Cleaning the original data containing duplicate values, abnormal values, or missing values, and outputting the data as preprocessed data, wherein the cleaning includes: deleting duplicate values in the data, or deleting, correcting, or replacing abnormal values in the data, or filling missing values in the data;

[0094] S13: Importing the pre-processed data into an exponential smoothing model to obtain a smoothed value based on a time series, thereby forming a smoothed data sequence.

[0095] It can be understood that the data preprocessing in this embodiment is used to eliminate invalid or noisy data caused by sensor failure or transmission error, so as to ensure the accuracy of subsequent analysis.

[0096] Among them, duplicate values are deleted through data deduplication strategies to ensure the uniqueness of the data; outliers are identified using statistical methods or machine learning algorithms, and are retained, corrected, or deleted based on actual conditions; missing values are processed using interpolation, regression, or other appropriate filling techniques to ensure the integrity and continuity of the data.

[0097] In addition, the smoothed data series can better reflect the changing trends of the edible fungus growth environment parameters and provide a basis for subsequent data analysis and decision-making.

[0098] Please refer to Figure 3 In one embodiment of the present invention, the step S1 further comprises decoupling the environmental parameter data:

[0099] S14: Decoupling the smoothed data sequence through a decoupling algorithm to obtain environmental influencing factors corresponding to ambient temperature, relative air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity;

[0100] S15: Standardize the decoupled data of each environmental impact factor to eliminate the influence of different dimensions and dimensional units.

[0101] It can be understood that in this embodiment, the decoupling algorithm is used to identify and separate the independent effects of various environmental parameters on the growth environment of edible fungi, making subsequent analysis more accurate and targeted, and being able to clearly understand how various factors such as ambient temperature, relative air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity act individually on the growth environment of edible fungi.

[0102] In addition, standardization is used to ensure that all environmental impact factor data are at the same level, eliminating the differences between different dimensions and dimensional units, and facilitating subsequent data analysis and model building.

[0103] Please refer to Figure 4 In one embodiment of the present invention, the step S2 of identifying the growth image data using a deep learning algorithm specifically includes:

[0104] S21, extracting features from the growth image data using an edge detection algorithm or a contour extraction algorithm to obtain size changes or area changes of corresponding bacterial species at different time points, and outputting a growth rate based on the size changes or area changes;

[0105] S22. Using the trained deep learning model to perform feature extraction on the growth image data, to obtain semantic features corresponding to the growth image data;

[0106] S23, extracting current morphological features of the strain through semantic features, wherein the morphological features include color distribution and shape features;

[0107] S24. Calculate a growth anomaly value by comparing the morphological features extracted from the growth image data with the preset morphological features, and compare the calculated growth anomaly value with a growth anomaly threshold. If the calculated growth anomaly value is greater than the growth anomaly threshold, output the morphological features and the anomaly value extracted from the growth image data as abnormal state information.

[0108] In one embodiment of the present invention, step S2 further includes: a neural network model constructed based on the YOLO neural network method, performing real-time target detection and classification of the bacterial growth stage through growth image data and environmental parameter data, and constructing a bacterial growth status database.

[0109] Understandably, the neural network model can efficiently identify bacterial strains at different growth stages from complex image data, and at the same time combine environmental parameter data such as temperature, humidity, and light intensity to further improve the accuracy and reliability of detection. Through real-time target detection and classification, the system can automatically track and record the entire process of bacterial strains from inoculation to maturity, ensuring the continuity and integrity of the data; in addition, the constructed bacterial strain growth status database provides a solid foundation for subsequent data analysis and intelligent decision-making.

[0110] Please refer to Figure 5 In one embodiment of the present invention, step S2 further includes:

[0111] S25, fusing the extracted semantic features, color distribution features, and shape features to form a comprehensive feature vector, wherein the comprehensive feature vector is used to describe the comprehensive features of the bacterial species;

[0112] S26. Importing the comprehensive feature vector into a trained classifier for classification to obtain the growth stage of the strain at the current time.

[0113] It can be understood that in this embodiment, a comprehensive feature vector integrating semantic features, color distribution features and shape features is used as input. The classifier can automatically determine the growth stage of the strain at the current time by learning and analyzing the feature vector.

[0114] In one embodiment of the present invention, the construction and training of the deep learning algorithm model specifically includes:

[0115] An initial model is built based on a convolutional neural network. The initial model includes multiple convolutional layers and pooling layers to automatically extract features from bacterial growth image data.

[0116] Collect bacterial growth image data and environmental parameter data, and pre-process the data;

[0117] The preprocessed data is divided into training set, validation set and test set. The training set is used to initialize the model and the parameters of the model are continuously adjusted through the back propagation algorithm to minimize the loss function.

[0118] Use the validation set to evaluate the performance of the model to prevent overfitting, and adjust the model's hyperparameters based on the results of the validation set;

[0119] The test set is used to evaluate the initial model after training, and the model whose output evaluation results meet the preset requirements is the deep learning algorithm model.

[0120] Please refer to Figure 6 In one embodiment of the present invention, the fusion processing of the image recognition result and the environmental parameter data in step S3 specifically includes:

[0121] S31, obtaining a smoothed and standardized data sequence;

[0122] S32, importing the environmental parameter data into an adaptive trust model, determining the trust relationship between the data nodes based on the reliability and historical performance of several data nodes in the multi-sensor information source, and establishing a trust matrix;

[0123] S33. Performing weighted optimization on the standardized data sequence using the trust matrix to increase the weight of reliable data and reduce the weight of unreliable data, and outputting the weighted optimized data sequence;

[0124] S34. The weighted optimized data sequence and the image recognition result are fused by a multi-data fusion algorithm to generate a comprehensive environmental assessment index.

[0125] It can be understood that the generation of comprehensive environmental assessment indicators in this embodiment facilitates comparative analysis with preset bacterial growth conditions, identifies the degree of impact of the current environment on bacterial growth, and determines whether environmental conditions need to be adjusted to optimize bacterial growth.

[0126] Please refer to Figure 7 In one embodiment of the present invention, the step S4 of importing the comprehensive environmental assessment index and the abnormal state information into the growth environment control model further includes:

[0127] S41, obtaining the current growth stage of the strain based on the image recognition result, and outputting the growth stage recognition result;

[0128] S42. Based on the growth stage identification result, match the growth environment control model corresponding to the current growth stage.

[0129] It can be understood that in this embodiment, the growth environment control model corresponding to the current growth stage is matched based on the growth stage identification results. According to the current growth requirements of the strain and the current environmental conditions, corresponding environmental adjustment suggestions can be analyzed and generated to adjust parameters such as temperature, humidity, and light intensity to ensure that the strain is cultured under the optimal growth conditions.

[0130] In one embodiment of the present invention, the growth stage identification result includes one of the following growth stages: mycelium growth stage, fruiting body primordium formation stage, fruiting body young ear growth stage or fruiting body adult ear color change stage.

[0131] The step S42 specifically includes reading the growth stage identification result and matching the growth environment control model based on the key control points affecting yield quality within the growth stage;

[0132] If the growth stage identification result is the mycelial growth stage, the key control point affecting yield quality is the day-night temperature difference, and the first growth environment control model is matched;

[0133] Alternatively, if the growth stage identification result is the fruiting body primordium formation period, the key control point affecting yield and quality is the ambient temperature, and the second growth environment control model is matched;

[0134] Alternatively, if the growth stage identification result is the fruiting body young ear growth stage, the key control point affecting yield and quality is the air relative humidity range, and the third growth environment control model is matched;

[0135] Alternatively, if the growth stage identification result is the fruiting body ear-forming and color-changing period, the key control point affecting yield quality is light intensity, which matches the fourth growth environment control model.

[0136] It can be understood that each growth environment control model in this embodiment includes an optimal control parameter range for the corresponding key control point, for example:

[0137] In the first growth environment control model, the day and night temperature difference is set within an optimal range to promote the healthy growth of mycelium;

[0138] Alternatively, in the second growth environment control model, the environmental temperature is precisely regulated to ensure that the fruiting body primordia can form under optimal temperature conditions;

[0139] Alternatively, in the third growth environment control model, the relative humidity range of the air is finely adjusted to meet the special humidity requirements of the fruiting body during the growth period of the young ears;

[0140] Or in the fourth growth environment control model, light intensity is precisely controlled to promote color change and ripening of fruiting bodies.

[0141] In one embodiment of the present invention, the step S4 of generating an environmental control instruction that meets the preset bacterial growth index specifically includes:

[0142] Obtaining a growth abnormality value of a corresponding bacterial species through the abnormal state information;

[0143] Inputting the growth abnormality value and the comprehensive environmental assessment index into the growth environment control model to obtain a compensation value corresponding to the environmental parameter data and an environmental control instruction corresponding to the compensation value;

[0144] Among them, the growth abnormality value is used to reflect the difference between the current morphological characteristics of the strain and the preset morphological characteristics, and the environmental control instruction corresponds to at least the adjustment amount of environmental parameters such as ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity.

[0145] It is understandable that in this embodiment, the environmental parameters in the culture chamber are automatically adjusted to ensure that the bacteria can grow in the optimal environment, for example:

[0146] If the growth anomaly value shows that the current morphological characteristics of the strain deviate from the preset morphological characteristics, and this deviation is caused by the ambient temperature being too high or too low, then the environmental control instructions will instruct the system to lower or increase the temperature in the culture chamber; similarly, if parameters such as air humidity, carbon dioxide concentration, culture medium temperature and humidity, or light intensity deviate from the optimal range, corresponding adjustments will also be made according to the environmental control instructions.

[0147] The intelligent environmental control method in this embodiment not only improves the efficiency and yield of bacterial culture, but also ensures the stability and consistency of bacterial quality.

[0148] In one embodiment of the present invention, the growth environment control model is constructed based on a fuzzy algorithm, and after executing the environmental adjustment instruction, the growth status of the strain and the changes in environmental parameters are monitored in real time, and the new growth anomalies and comprehensive environmental assessment indicators are fed back to the fuzzy algorithm system, and the control rules and membership functions are optimized and updated to form a closed-loop control.

[0149] Understandably, when this embodiment detects significant changes in bacterial growth status or environmental parameters, it can quickly respond and adjust the control strategy to adapt to the new growth conditions. This closed-loop control approach enables the growth environment control model to continuously and dynamically output environmental adjustment instructions to adjust the culture environment, ensuring that the bacterial strains are always in an optimal growth state.

[0150] In one embodiment of the present invention, the method further includes: uploading the growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal status information to a cloud monitoring platform through an Internet of Things communication module, and realizing remote control and abnormal status warning based on a visual interface.

[0151] Understandably, for growth image data that is widely identified as abnormal state information, an alarm mechanism will be further triggered, and an alarm message will be sent to the user through a preset communication method to ensure that the user can promptly discover and deal with the abnormal growth of the strain;

[0152] At the same time, the system will also record abnormal growth events, including the time of occurrence, strain type, abnormal characteristics and other information, forming a complete abnormal growth log to provide a basis for subsequent analysis and improvement.

[0153] In one embodiment of the present invention, an intelligent monitoring and cultivation system based on multimodal data fusion of vision is further provided, which can be used to implement the intelligent monitoring and cultivation method based on multimodal data fusion of vision described in any of the above embodiments, including:

[0154] The data acquisition module is used to obtain bacterial growth image data and environmental parameter data through multiple sensor information sources;

[0155] A data recognition module is used to recognize the growth image data through a deep learning algorithm and output an image recognition result;

[0156] A comprehensive environmental assessment module is used to fuse the image recognition results and the environmental parameter data using a multi-data fusion algorithm to generate a comprehensive environmental assessment index;

[0157] A control instruction generation module is used to import the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate an environmental control instruction that meets the preset bacterial growth index;

[0158] The environment control module is used to execute the environment control instructions and dynamically adjust the culture environment parameters.

[0159] In one embodiment of the present invention, the system also includes a bacterial strain growth status database, which has a neural network model constructed based on the YOLO neural network method, and performs real-time target detection and classification of bacterial strain growth stages through growth image data and environmental parameter data to form standard morphological characteristics of bacterial strains at different growth stages.

[0160] In one embodiment of the present invention, a cloud monitoring platform is further included for receiving the growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal status information uploaded by the Internet of Things communication module.

[0161] The intelligent monitoring and cultivation system based on multimodal data fusion based on vision provided by the present invention can achieve the following technical effects:

[0162] 1. Through multiple sensor information sources, including environmental parameter sensor nodes and image sensor nodes, bacterial growth image data and environmental parameter data are obtained, and the image recognition results and environmental parameter data are fused and processed to generate comprehensive environmental assessment indicators. The multimodal data fusion of the present invention realizes comprehensive environmental monitoring and can more accurately assess the overall condition of the bacterial growth environment, avoiding the errors and one-sidedness that may be caused by a single data source.

[0163] 2. The current growth stage of the strain is obtained through image recognition results, and the growth environment control model corresponding to the current growth stage is matched. The growth abnormality value and the comprehensive environmental assessment index are input into the growth environment control model to obtain the compensation value of the environmental parameter data and the environmental control instruction corresponding to the compensation value. Therefore, the present invention can accurately control the key control points of different growth stages.

[0164] 3. Use fuzzy algorithms to construct a growth environment control model. By defining the membership functions of input and output variables, formulating control rules, performing fuzzy reasoning and defuzzification steps, generating and executing environmental adjustment instructions, the growth status of the strain and changes in environmental parameters are monitored in real time. New growth anomalies and comprehensive environmental assessment indicators are fed back to the fuzzy algorithm system to achieve dynamic adjustment of environmental parameters, optimize and update control rules and membership functions, and form a closed-loop control.

[0165] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An intelligent monitoring and cultivation method based on multimodal data fusion of vision, characterized in that: The application in the edible fungus cultivation cabin includes the following steps: S1. Acquire bacterial growth image data and environmental parameter data based on a multi-sensor information source, wherein the environmental parameter data at least includes ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity; S2. Recognize the growth image data using a deep learning algorithm and output an image recognition result, wherein the image recognition result includes at least bacterial growth rate, morphological characteristics, and abnormal state information; S3, fusing the image recognition result and the environmental parameter data through a multi-data fusion algorithm to generate a comprehensive environmental assessment index; S4. Importing the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate environmental control instructions that meet preset bacterial strain growth indicators, wherein the growth environment control model is trained based on historical growth image data of the bacterial strain and corresponding environmental parameter data; S5. Execute the environmental control instructions through the environmental control module to dynamically adjust the culture environment parameters.

2. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 1, characterized in that: The multi-sensor information source includes an environmental parameter sensor node and an image sensor node; Several environmental parameter sensor nodes are deployed at multiple points inside the edible fungus cultivation cabin so that the sensing data of the several environmental parameter sensor nodes are evenly distributed in space. The environmental parameter sensor nodes include at least a temperature sensor, a humidity sensor, a carbon dioxide concentration sensor, and a light sensor. The control range of the plurality of image sensor nodes covers the entire planting area in the edible fungus cultivation cabin, the image sensor nodes are connected to the Internet of Things via a wireless connection module or a wired connection module, and any image sensor node is communicatively connected to at least another adjacent image sensor node; The temperature sensor and humidity sensor are manufactured using MEMS technology and have a waterproof and dustproof structure.

3. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 2, characterized in that: The step S1 further includes performing data preprocessing on the environmental parameter data: S11: Obtain raw data that needs to be preprocessed based on the time series, use a filtering algorithm to remove noise from the sensor data, and check whether there are duplicate values, abnormal values, or missing values in the raw data; S12: Cleaning the original data containing duplicate values, abnormal values, or missing values, and outputting the data as preprocessed data, wherein the cleaning includes: deleting duplicate values in the data, or deleting, correcting, or replacing abnormal values in the data, or filling missing values in the data; S13: Importing the pre-processed data into an exponential smoothing model to obtain a smoothed value based on a time series, thereby forming a smoothed data sequence.

4. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 3, characterized in that: The step S1 further includes decoupling the environmental parameter data: S14: Decoupling the smoothed data sequence through a decoupling algorithm to obtain environmental influencing factors corresponding to ambient temperature, relative air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity; S15: Standardize the decoupled data of each environmental impact factor to eliminate the influence of different dimensions and dimensional units.

5. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 3, characterized in that: The step S2 of identifying the growth image data by using a deep learning algorithm specifically includes: S21, extracting features from the growth image data using an edge detection algorithm or a contour extraction algorithm to obtain size changes or area changes of corresponding bacterial species at different time points, and outputting a growth rate based on the size changes or area changes; S22. Using the trained deep learning model to perform feature extraction on the growth image data to obtain semantic features corresponding to the growth image data; S23, extracting current morphological features of the strain through semantic features, wherein the morphological features include color distribution and shape features; S24. Calculate a growth anomaly value by comparing the morphological features extracted from the growth image data with the preset morphological features, and compare the calculated growth anomaly value with a growth anomaly threshold. If the calculated growth anomaly value is greater than the growth anomaly threshold, output the morphological features and the anomaly value extracted from the growth image data as abnormal state information.

6. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 5, characterized in that: The step S2 further includes: The neural network model constructed based on the YOLO neural network method performs real-time target detection and classification of bacterial growth stages through growth image data and environmental parameter data, and builds a bacterial growth status database.

7. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 6, characterized in that: The step S2 further includes: S25, fusing the extracted semantic features, color distribution features, and shape features to form a comprehensive feature vector, wherein the comprehensive feature vector is used to describe the comprehensive features of the bacterial species; S26. Importing the comprehensive feature vector into a trained classifier for classification to obtain the growth stage of the strain at the current time.

8. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 5, characterized in that: The construction and training of the deep learning algorithm model specifically includes: An initial model is built based on a convolutional neural network. The initial model includes multiple convolutional layers and pooling layers to automatically extract features from bacterial growth image data. Collect bacterial growth image data and environmental parameter data, and pre-process the data; The preprocessed data is divided into training set, validation set and test set. The training set is used to initialize the model and the parameters of the model are continuously adjusted through the back propagation algorithm to minimize the loss function. Use the validation set to evaluate the performance of the model to prevent overfitting, and adjust the model's hyperparameters based on the results of the validation set; The test set is used to evaluate the initial model after training, and the model whose output evaluation results meet the preset requirements is the deep learning algorithm model.

9. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision as claimed in claim 5, characterized in that: The fusion processing of the image recognition result and the environmental parameter data in step S3 specifically includes: S31, obtaining a smoothed and standardized data sequence; S32, importing the environmental parameter data into an adaptive trust model, determining the trust relationship between the data nodes based on the reliability and historical performance of several data nodes in the multi-sensor information source, and establishing a trust matrix; S33. Performing weighted optimization on the standardized data sequence using the trust matrix to increase the weight of reliable data and reduce the weight of unreliable data, and outputting the weighted optimized data sequence; S34. The weighted optimized data sequence and the image recognition result are fused by a multi-data fusion algorithm to generate a comprehensive environmental assessment index.

10. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 1, characterized in that: The step S4 of importing the comprehensive environmental assessment index and the abnormal state information into the growth environment control model further includes: S41, obtaining the current growth stage of the strain based on the image recognition result, and outputting the growth stage recognition result; S42. Based on the growth stage identification result, match the growth environment control model corresponding to the current growth stage.

11. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 10, characterized in that: The growth stage identification result includes one of the following growth stages: mycelium growth stage, fruiting body primordium formation stage, fruiting body young ear growth stage or fruiting body adult ear color change stage.

12. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 11, characterized in that: The step S42 specifically includes reading the growth stage identification result and matching the growth environment control model based on the key control points affecting yield quality within the growth stage; If the growth stage identification result is the mycelial growth stage, the key control point affecting yield quality is the day-night temperature difference, and the first growth environment control model is matched; Alternatively, if the growth stage identification result is the fruiting body primordium formation stage, the key control point affecting yield and quality is the ambient temperature, and the second growth environment control model is matched; Alternatively, if the growth stage identification result is the fruiting body young ear growth stage, the key control point affecting yield and quality is the air relative humidity range, and the third growth environment control model is matched; Alternatively, if the growth stage identification result is the fruiting body ear-forming and color-changing period, the key control point affecting yield quality is light intensity, which matches the fourth growth environment control model.

13. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 10, characterized in that: The step S4 in which the environmental control instructions that meet the preset bacterial growth index are generated specifically includes: Obtaining a growth abnormality value of a corresponding bacterial species through the abnormal state information; Inputting the growth abnormality value and the comprehensive environmental assessment index into the growth environment control model to obtain a compensation value corresponding to the environmental parameter data and an environmental control instruction corresponding to the compensation value; Among them, the growth abnormality value is used to reflect the difference between the current morphological characteristics of the strain and the preset morphological characteristics, and the environmental control instruction corresponds to at least the adjustment amount of environmental parameters such as ambient temperature, air humidity, carbon dioxide concentration, culture medium temperature and humidity, and light intensity.

14. The intelligent monitoring and cultivation method based on vision-based multimodal data fusion according to claim 10, characterized in that: The growth environment control model is constructed based on a fuzzy algorithm. After executing the environmental adjustment instruction, the growth status of the strain and the changes in environmental parameters are monitored in real time. New growth anomalies and comprehensive environmental assessment indicators are fed back to the fuzzy algorithm system, and the control rules and membership functions are optimized and updated to form a closed-loop control.

15. The intelligent monitoring and cultivation method based on multimodal data fusion based on vision according to claim 1, characterized in that: The method further comprises: The growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal state information are uploaded to the cloud monitoring platform through the Internet of Things communication module, and remote control and abnormal state warning are realized based on the visual interface.

16. An intelligent monitoring and cultivation system based on multimodal data fusion based on vision, characterized in that: The intelligent monitoring and cultivation method based on vision-based multimodal data fusion that can be used in any one of claims 1 to 15 comprises: The data acquisition module is used to obtain bacterial growth image data and environmental parameter data through multiple sensor information sources; A data recognition module is used to recognize the growth image data through a deep learning algorithm and output an image recognition result; A comprehensive environmental assessment module is used to fuse the image recognition results and the environmental parameter data using a multi-data fusion algorithm to generate a comprehensive environmental assessment index; A control instruction generation module is used to import the comprehensive environmental assessment index and the abnormal state information into a growth environment control model to generate an environmental control instruction that meets the preset bacterial growth index; The environment control module is used to execute the environment control instructions and dynamically adjust the culture environment parameters.

17. The intelligent monitoring and cultivation system based on vision-based multimodal data fusion according to claim 16, characterized in that: It also includes a bacterial strain growth status database, which has a neural network model built based on the YOLO neural network method. It performs real-time target detection and classification of bacterial strain growth stages through growth image data and environmental parameter data, and is used to form standard morphological characteristics of bacterial strains in different growth stages.

18. The intelligent monitoring and cultivation system based on vision-based multimodal data fusion according to claim 16, characterized in that: It also includes a cloud monitoring platform for receiving the growth image data, the environmental parameter data, the comprehensive environmental assessment index and the abnormal status information uploaded by the Internet of Things communication module.

Images