A multi-modal crop growth intelligent analysis system based on an agricultural digital smart shelter

Abstract

The embodiment of the application discloses a multi-modal crop growth intelligent analysis system based on an agricultural digital smart shelter, which comprises a multi-spectral LED array arranged in the agricultural digital smart shelter, a 3D point cloud sensor for acquiring crop point cloud data in the agricultural digital smart shelter, an image acquisition module for acquiring crop spectral data in the agricultural digital smart shelter, and a data processing module for realizing early crop disease identification, growth prediction, nutrient deficiency detection and pruning suggestion output according to the crop point cloud data and the crop spectral data. The application can automatically realize crop growth monitoring and analysis, improve the monitoring and analysis efficiency, and reduce the artificial inspection cost and the amount of medicine and fertilizer used in the shelter. In addition, the system is especially suitable for precision agricultural management in the plant factory, intelligent greenhouse and vertical agricultural scene.

Description

Technical Field

[0001] This invention relates to the field of smart agriculture technology, specifically to a multimodal intelligent crop growth analysis system based on an agricultural digital cabin. Background Technology

[0002] Current crop growth monitoring mainly relies on manual inspections or traditional image processing techniques, which have the following drawbacks:

[0003] 1. Manual inspection: Low efficiency (average daily inspection area <500m²), highly subjective, and difficult to detect early-stage diseases.

[0004] 2. Single-modal monitoring: Visible light images are easily affected by illumination, hyperspectral equipment is expensive, and it is impossible to acquire morphological and physiological information simultaneously;

[0005] 3. Cloud computing: Relies on network transmission, has poor real-time performance (average latency > 5 seconds), and is difficult to support high-frequency decision-making. Summary of the Invention

[0006] In view of the deficiencies of the prior art mentioned in the background, the purpose of this invention is to provide a multimodal intelligent crop growth analysis system based on an agricultural digital cabin.

[0007] To achieve the above objectives, embodiments of the present invention provide a multimodal intelligent crop growth analysis system based on an agricultural digital cabin, comprising:

[0008] A multispectral LED array is installed inside an agricultural smart cabin.

[0009] A 3D point cloud sensor is used to acquire crop point cloud data within the agricultural digital cabin.

[0010] The image acquisition module is used to acquire crop spectral data within the agricultural digital cabin;

[0011] The data processing module is used to realize early identification of crop diseases, growth prediction, nutrient deficiency detection and pruning suggestions based on the crop point cloud data and crop spectral data.

[0012] As one specific implementation of this application, the data processing module is specifically used for:

[0013] After histogram equalization of the crop spectral data, it is input into the YOLOv8 target detection model for early identification of pests and diseases, and the early pest and disease categories of crops are output.

[0014] After filtering and denoising the crop point cloud data, growth prediction is performed by integrating the current environmental data and the CycleGAN growth prediction network, and the pruning angle and fertilizer application amount are calculated based on the growth prediction results.

[0015] Furthermore, the data processing module is specifically used for:

[0016] Crop morphological features, including plant height, canopy structure, number and distribution of stems, are extracted from the crop point cloud data.

[0017] Crop growth characteristics were extracted based on the crop spectral data, including leaf color, chlorotic lesion areas, leaf curling degree, and leaf NIR reflectance.

[0018] Nutrient deficiency is detected based on the crop morphological and ecological characteristics, and a nutrient deficiency warning is triggered based on the detection results.

[0019] As a specific implementation of this application, the construction and training process of the CycleGAN growth prediction network is as follows:

[0020] Construct a growth prediction network architecture, including two generators and two discriminators;

[0021] Images of sampled crops, including images from the early growth stage and the middle growth stage;

[0022] The growth prediction network is trained and evaluated based on the crop sample images.

[0023] As an optional implementation of this application, the model evaluation of the growth prediction network specifically involves:

[0024] Multi-source data were collected, including cultivation substrate data for crop growth, historical planting data within the container, and environmental data affecting crop growth. The cultivation substrate data included substrate moisture content, substrate solution pH value, and trace elements. The historical planting data within the container included crop yield and pest and disease records. The environmental data included container temperature, container humidity, container air data, and container spectral data.

[0025] The multi-source data is preprocessed, including normalization, interpolation, outlier removal, and elimination.

[0026] A digital model of the cultivation substrate for crop growth is constructed, and the preprocessed data is used as the initial parameters of the digital model.

[0027] The preprocessed multi-source data is mapped onto the digital model of the cultivation substrate;

[0028] Receive crop growth parameters input by the user, including irrigation data, fertilization data, and pesticide data;

[0029] The cultivation substrate digital model is simulated and evaluated based on the crop growth parameters.

[0030] As an optional implementation of this application, the simulation evaluation of the digital model of the cultivation substrate further includes:

[0031] During the simulation evaluation, shading data inside the cabin was obtained from the spectral data inside the cabin. The daily shading data was used as a stress factor, and its impact on biomass accumulation was dynamically calculated by modifying the daily photosynthetically active radiation value. The shading data inside the cabin was caused by the shading of the cabin building.

[0032] Furthermore, as a preferred implementation of this application, the system further includes a disaster early warning module, used for:

[0033] Connect with meteorological systems to obtain historical meteorological data and weather forecast data;

[0034] The first data collected from the agricultural intelligent modular cabin includes air data and light data inside the cabin;

[0035] Disaster early warning can be achieved within the mobile shelter based on the aforementioned historical meteorological data, weather forecast data, and ground data.

[0036] Furthermore, as a preferred implementation of this application, the system further includes an irrigation early warning module, used for:

[0037] Collect the second data from the aforementioned agricultural digital cabin;

[0038] The water and fertilizer demand index was obtained by analyzing the second data.

[0039] Irrigation / fertilization equipment is controlled based on the water and fertilizer demand index. When the water and fertilizer demand index exceeds the dynamic threshold, the control signal for the irrigation / fertilization equipment is triggered.

[0040] Implementing embodiments of the present invention provides a multimodal intelligent crop growth analysis solution based on an agricultural digital intelligent container, which can automatically monitor and analyze crop growth, improve monitoring and analysis efficiency, and reduce the cost of manual inspections and the amount of pesticides and fertilizers used within the container. Furthermore, the system provided by the present invention is particularly suitable for precision agricultural management in plant factories, intelligent greenhouses, and vertical agriculture scenarios. Attached Figure Description

[0041] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below.

[0042] Figure 1 This is a structural diagram of the multimodal intelligent crop growth analysis system based on an agricultural digital cabin provided in this embodiment of the invention;

[0043] Figure 2This is a comparison chart of the monitoring results of the analysis system provided in this embodiment of the invention and traditional technology. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0046] Agricultural digital cabins are modular agricultural production and monitoring equipment that integrates cutting-edge technologies such as modern biology, the Internet of Things, and artificial intelligence. They are usually based on containers or customized cabins. By creating a controllable and sealed growth environment and combining it with an intelligent control system, they can achieve efficient and standardized planting and full-process digital management of crops. They are currently widely used in the cultivation of crops and plants.

[0047] Please refer to Figure 1 This invention provides a multimodal intelligent crop growth analysis system based on an agricultural digital cabin, comprising a multispectral LED array, a 3D point cloud sensor, an image acquisition module, a data processing module, and a disaster early warning module. Each component will be described separately below.

[0048] (a) Multispectral LED array

[0049] In this embodiment, the multispectral LED array is installed inside the agricultural smart cabin, with wavelength combinations of 450nm, 660nm, and 850nm. Specifically, it can be configured with a power ratio of 450nm:660nm:850nm = 1:2:1, and dynamic spectral combination can be achieved through PWM modulation.

[0050] (ii) 3D point cloud sensor

[0051] A 3D point cloud sensor is used to acquire crop point cloud data within the agricultural digital cabin. In this embodiment, the 3D point cloud sensor employs a Livox Avia lidar with a scanning frequency of 20Hz and an angular resolution of 0.1°.

[0052] Understandably, a 3D point cloud sensor is a device capable of acquiring three-dimensional spatial information of an object or scene and representing it in the form of a point cloud. This point cloud data contains information such as three-dimensional coordinates (X, Y, Z) and can be used for various applications such as object recognition, scene reconstruction, and 3D modeling. This embodiment employs a lidar system, which generates high-precision three-dimensional coordinate information by emitting laser pulses and calculating the flight time of the reflected signals. It is suitable for scanning large-scale scenes, supports thousands of data acquisitions per second, and can also integrate a high-frequency vibration sensor to enhance dynamic monitoring capabilities.

[0053] (III) Image Acquisition Module

[0054] The image acquisition module is used to acquire crop spectral data within the agricultural digital cabin. Understandably, the image acquisition module can be a multispectral camera, such as a 5-band or 10-band camera, but is not limited to this.

[0055] (iv) Data Processing Module

[0056] The data processing module is the core module of this invention, and it is mainly used to: realize early identification of crop diseases, growth prediction, nutrient deficiency detection, and output pruning suggestions based on the crop point cloud data and crop spectral data. The pruning suggestions include the stem cutting position with an accuracy of ±1cm.

[0057] In practical implementation, the data processing module is specifically used for:

[0058] After histogram equalization of the crop spectral data, it is input into the YOLOv8 target detection model for early identification of pests and diseases, and the early pest and disease categories of crops are output.

[0059] After filtering and denoising the crop point cloud data, growth prediction is performed by integrating the current environmental data and the CycleGAN growth prediction network, and the pruning angle and fertilizer application amount are calculated based on the growth prediction results.

[0060] The CycleGAN growth prediction network mentioned above is pre-trained, and its construction and training process is as follows:

[0061] 1. Construct a growth prediction network architecture

[0062] This growth prediction network architecture consists of two generators and two discriminators. The two generators are primarily responsible for converting crop images from the source domain (e.g., wheat seedlings) into images representing different growth stages in the target domain (e.g., wheat heading). A ResNet-based residual network architecture is typically used, but an improved U-Net architecture can be selected depending on the need to preserve crop details. The two discriminators determine whether the input image represents the actual crop growth stage or a prediction from the generators. Through adversarial interaction with the generators, they strive to make the generated crop growth images more closely resemble the actual growth stage. The two discriminators, D_Y and D_X, correspond to the two generators, both employing a CycleGAN structure.

[0063] (2) Sample crop images, including images from the early growth stage and images from the middle growth stage.

[0064] (3) The growth prediction network is trained and evaluated based on the crop sample images.

[0065] The specific process of model evaluation is as follows:

[0066] ① Collect data from multiple sources

[0067] The multi-source data includes cultivation substrate data for crop growth, historical planting data within the container, and environmental data affecting crop growth. The cultivation substrate data includes substrate moisture content, total substrate porosity, aeration porosity, water-holding porosity, particle size distribution, substrate solution pH, conductivity, cation exchange capacity, organic matter content, macroelements (N / P / K), mesoelements (Ca / Mg / S), microelements (Fe / Zn / Cu / Mn / B / Mo), nutrient release rate (N / P / K), total microorganisms (bacteria + fungi), number of beneficial microorganisms (nitrogen-fixing bacteria / phosphate-solubilizing bacteria / actinomycetes), harmful microorganisms (pathogens / nematodes), and enzyme activity. The historical planting data within the container includes crop yield and pest and disease records. The environmental data includes container temperature, container humidity, container air data, and container spectral data.

[0068] The cabin temperature and humidity are collected by temperature and humidity sensors installed inside the cabin. The process for acquiring cabin air data involves high-speed ultraviolet light sterilizing outside air after it enters the cabin, resulting in purified clean air. The cabin spectral data is provided by a multispectral LED array installed inside the cabin. During crop growth inside the cabin, the spectral data can be adjusted by monitoring crop leaves. For example, if crop stems are found to be thin and elongated or internodes are too long, the light intensity and wavelength can be adjusted to obtain a more suitable spectral formula for crop growth. Furthermore, spectral adjustments can be coordinated with temperature, humidity, CO2 concentration, and substrate data to achieve integrated light-temperature-water-fertilizer control.

[0069] ② The multi-source data is preprocessed, including normalization, interpolation, outlier removal and elimination.

[0070] By performing the above preprocessing on multi-source data, the spatiotemporal scale and dimensions of different data are unified, which solves the problem of incomparability caused by differences in format and dimension of multi-source data. This lays a standardized and accurate data foundation for subsequent analysis and can avoid large deviations caused by data disorder.

[0071] ③ Construct a digital model of the cultivation substrate for crop growth, and use the preprocessed data as the initial parameters of the digital model.

[0072] ④ Map the preprocessed multi-source data into the digital model of the cultivation substrate.

[0073] ⑤ Receive crop growth parameters input by the user, including irrigation data, fertilization data, and pesticide data.

[0074] ⑥ The cultivation substrate digital model is simulated and evaluated based on the crop growth parameters.

[0075] Furthermore, the simulation evaluation of the digital model of the cultivation substrate also includes:

[0076] During the simulation evaluation, shading data inside the cabin was obtained from the spectral data inside the cabin. The daily shading data was used as a stress factor, and its impact on biomass accumulation was dynamically calculated by modifying the daily photosynthetically active radiation value. The shading data inside the cabin was caused by the shading of the cabin building.

[0077] Understandably, this embodiment first obtains the shading data from the spectral data inside the cabin, and then inputs the daily shading data as a light stress factor into the photosynthetic production sub-model. By modifying the daily photosynthetically active radiation value, its impact on biomass accumulation is dynamically calculated. This avoids the problem of excessive deviation between the biomass accumulation curve, yield estimate and actual production caused by ignoring the impact of shading. It provides accurate data support for users to formulate agricultural production plans that are more suitable for the light conditions of the plot (such as selecting shade-tolerant crop varieties, adjusting planting density, etc.).

[0078] Furthermore, the data processing module is specifically used for:

[0079] Crop morphological features, including plant height, canopy structure, number and distribution of stems, are extracted from the crop point cloud data.

[0080] Crop growth characteristics were extracted based on the crop spectral data, including leaf color, chlorotic lesion areas, leaf curling degree, and leaf NIR reflectance.

[0081] Nutrient deficiency detection is achieved based on the crop morphological and ecological characteristics, and a nutrient deficiency warning is triggered based on the detection results. For example, it is triggered when the leaf NIR reflectance is below a threshold (<12% in the 850nm band).

[0082] (v) Disaster Early Warning Module

[0083] This disaster early warning module is mainly used for:

[0084] Connect with meteorological systems to obtain historical meteorological data and weather forecast data;

[0085] The first data collected from the agricultural intelligent modular cabin includes air data and light data inside the cabin;

[0086] Disaster early warning can be achieved within the mobile shelter based on the aforementioned historical meteorological data, weather forecast data, and ground data.

[0087] In practice, the process is as follows:

[0088] 1. Normalize historical meteorological data, weather forecast data, and surface data to ensure a consistent data format and remove outliers. Historical meteorological data includes, but is not limited to, rainfall and temperature in the area where the makeshift hospital is located over the past 7 days. Surface data includes, but is not limited to, geographical data surrounding the makeshift hospital, data about the makeshift hospital itself, and data about the surrounding environment. Data about the makeshift hospital itself includes, but is not limited to, the building materials, wind resistance rating, waterproofing capabilities, ventilation equipment parameters, and air conditioning equipment parameters.

[0089] 2. Extract multiple disaster early warning factors from the data in step 1, assign different disaster early warning factors a certain score, and allocate weights to multiple disaster early warning factors. For example, the rainstorm and waterlogging factor is worth 50 points, with a corresponding weight of 40%, and the strong wind damage risk factor is worth 60 points, with a corresponding weight of 60%.

[0090] 3. Calculated based on disaster early warning factors and weights: 50 points × 40% + 60 points × 60% = 56 points.

[0091] 4. Based on the obtained scores and the rules set by the authorities, the disaster warning level is determined to be Level 1, and relevant staff are notified to take timely preventive measures.

[0092] Furthermore, as a preferred implementation of this application, the system further includes an irrigation early warning module, used for:

[0093] The second data of the agricultural digital cabin is collected; the second data includes, but is not limited to, leaf water content, chlorophyll content, soil moisture and soil fertility, etc.

[0094] The water and fertilizer demand index was obtained by analyzing the second data.

[0095] Irrigation / fertilization equipment is controlled based on the water and fertilizer demand index. When the water and fertilizer demand index exceeds the dynamic threshold, the control signal for the irrigation / fertilization equipment is triggered.

[0096] The aforementioned water and fertilizer demand index is a comprehensive indicator that quantifies the current degree of water and fertilizer deficit in crops, and can be obtained in the following way:

[0097] Historical data on crop yield under different water and fertilizer supplies are collected. A regression model is trained based on this historical data. The second set of collected data is input, and the output is the water and fertilizer demand index.

[0098] Furthermore, the calculated water and fertilizer demand index is compared with the set threshold. When the index exceeds the upper limit of the dynamic threshold, the crop is determined to be in a state of water and fertilizer deficiency, and the controller immediately generates a control signal. If the index is lower than the lower limit of the threshold, water and fertilizer are determined to be sufficient, and no signal is triggered. If the index is within the threshold range, the current equipment state is maintained.

[0099] As can be seen from the above description, implementing the embodiments of the present invention provides a multimodal intelligent crop growth analysis scheme based on an agricultural digital intelligent container, which can automatically realize crop growth monitoring and analysis, improve monitoring and analysis efficiency, and reduce the cost of manual inspection and the amount of pesticides and fertilizers used in the container. Related comparisons include... Figure 2 As shown. Furthermore, the system provided by this invention is particularly suitable for precision agriculture management in plant factories, smart greenhouses, and vertical agriculture scenarios.

[0100] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0103] Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units can be implemented in hardware or as software functional units. When using each module, user information is collected and stored only with the user's full authorization and in compliance with relevant laws and regulations, protecting the security and privacy of user data, and strictly prohibiting unauthorized access; data processing will be conducted within the scope stipulated by law and will not exceed the purpose and scope authorized by the user; at the same time, users have the rights to access, correct, delete, restrict processing, and refuse their personal data; and must strictly comply with applicable laws and regulations and conduct compliance reviews.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multimodal intelligent crop growth analysis system based on an agricultural digital cabin, characterized in that, include: A multispectral LED array is installed inside an agricultural smart cabin. A 3D point cloud sensor is used to acquire crop point cloud data within the agricultural digital cabin. The image acquisition module is used to acquire crop spectral data within the agricultural digital cabin; The data processing module is used to realize early identification of crop diseases, growth prediction, nutrient deficiency detection and pruning suggestions based on the crop point cloud data and crop spectral data.

2. The system as described in claim 1, characterized in that, The data processing module is specifically used for: After histogram equalization of the crop spectral data, it is input into the YOLOv8 target detection model for early identification of pests and diseases, and the early pest and disease categories of crops are output. After filtering and denoising the crop point cloud data, growth prediction is performed by integrating the current environmental data and the CycleGAN growth prediction network, and the pruning angle and fertilizer application amount are calculated based on the growth prediction results.

3. The system as described in claim 2, characterized in that, The data processing module is specifically used for: Crop morphological features, including plant height, canopy structure, number and distribution of stems, are extracted from the crop point cloud data. Crop growth characteristics were extracted based on the crop spectral data, including leaf color, chlorotic lesion areas, leaf curling degree, and leaf NIR reflectance. Nutrient deficiency is detected based on the crop morphological and ecological characteristics, and a nutrient deficiency warning is triggered based on the detection results.

4. The system as described in claim 2, characterized in that, The construction and training process of the CycleGAN growth prediction network is as follows: Construct a growth prediction network architecture, including two generators and two discriminators; Images of sampled crops, including images from the early growth stage and the middle growth stage; The growth prediction network is trained and evaluated based on the crop sample images.

5. The system as described in claim 4, characterized in that, The specific steps for evaluating the growth prediction network are as follows: Multi-source data were collected, including cultivation substrate data for crop growth, historical planting data within the container, and environmental data affecting crop growth. The cultivation substrate data included substrate moisture content, substrate solution pH value, and trace elements. The historical planting data within the container included crop yield and pest and disease records. The environmental data included container temperature, container humidity, container air data, and container spectral data. The multi-source data is preprocessed, including normalization, interpolation, outlier removal, and elimination. A digital model of the cultivation substrate for crop growth is constructed, and the preprocessed data is used as the initial parameters of the digital model. The preprocessed multi-source data is mapped onto the digital model of the cultivation substrate; Receive crop growth parameters input by the user, including irrigation data, fertilization data, and pesticide data; The cultivation substrate digital model is simulated and evaluated based on the crop growth parameters.

6. The system as described in claim 5, characterized in that, The simulation evaluation of the digital model of the cultivation substrate also includes: During the simulation evaluation, shading data inside the cabin was obtained from the spectral data inside the cabin. The daily shading data was used as a stress factor, and its impact on biomass accumulation was dynamically calculated by modifying the daily photosynthetically active radiation value. The shading data inside the cabin was caused by the shading of the cabin building.

7. The system as described in claim 1, characterized in that, The system also includes a disaster early warning module, used for: Connect with meteorological systems to obtain historical meteorological data and weather forecast data; The first data collected from the agricultural intelligent modular cabin includes air data and light data inside the cabin; Disaster early warning can be achieved within the mobile shelter based on the aforementioned historical meteorological data, weather forecast data, and ground data.

8. The system as described in claim 7, characterized in that, The system also includes an irrigation early warning module for: Collect the second data from the aforementioned agricultural digital cabin; The water and fertilizer demand index was obtained by analyzing the second data. Irrigation / fertilization equipment is controlled based on the water and fertilizer demand index. When the water and fertilizer demand index exceeds the dynamic threshold, the control signal for the irrigation / fertilization equipment is triggered.

9. The system as described in claim 1, characterized in that, The wavelengths of the multispectral LED array include 450nm, 660nm and 850nm.

10. The system as claimed in claim 1, characterized in that, The pruning recommendations include the stem cutting position, with an accuracy of ±1cm.

Images