An intelligent monitoring and early warning system and method for greenhouse pests and diseases based on the Internet of Things

Abstract

The present application relates to the field of monitoring and early warning technology, and specifically discloses a greenhouse pest and disease intelligent monitoring and early warning system and method based on the Internet of Things. The system first uses a target detection model to perform a preliminary screening of visible light RGB images of greenhouse plants to obtain preliminary screening results of pests and diseases and corresponding confidence scores. At the same time, the greenhouse environmental data is analyzed in time series to determine whether it is suitable for pest and disease breeding, and the confidence threshold is dynamically adjusted accordingly. The preliminary screening results are graded, that is, high-confidence results are directly warned, low-confidence results are filtered, and suspicious pest and disease images with intermediate confidence scores are further fused with their corresponding multispectral images. By performing deep correlation fusion analysis on the two, accurate identification and early warning of pests and diseases are achieved. This method uses a coarse screening followed by fine judgment mechanism, and only introduces multispectral data for in-depth analysis in suspicious scenes. While improving the accuracy of pest and disease monitoring, it avoids unnecessary computational overhead.

Description

Technical Field

[0001] The present application relates to the field of monitoring and early warning technology, and more specifically, to an intelligent monitoring and early warning system and method for greenhouse pests and diseases based on the Internet of Things. Background Art

[0002] Greenhouse agriculture, a vital component of modern agriculture, achieves efficient, high-density, and off-season crop production by creating a relatively closed and controlled environment. This plays a vital role in ensuring a stable supply of agricultural products and improving agricultural economic returns. However, while the constant, suitable temperature and humidity within the greenhouse creates favorable conditions for crop growth, it also creates a breeding ground for pests and diseases and their rapid spread. Once a pest or disease breaks out, it spreads extremely quickly within the confined space of the greenhouse, potentially causing widespread crop yield reductions or even complete crop failures in a short period of time, resulting in severe economic losses for agricultural production.

[0003] Traditional pest and disease monitoring relies primarily on regular manual inspections and empirical judgment by agricultural technicians or farmers. This approach is not only labor-intensive and inefficient, but also highly subjective. Lack of experience or timely observation can lead to missed opportunities for optimal control, leading to the spread of pests and diseases. With the advancement of computer vision technology, some solutions have begun to incorporate image recognition technology for automated detection of crop pests and diseases. Currently, existing solutions typically use visible light (RGB) cameras to capture crop images and apply image processing algorithms to identify disease spots or pests within them. However, in complex greenhouse environments, early symptoms of pests and diseases (such as tiny spots, slight leaf curl, and tiny eggs) can be visually highly similar to background information such as leaf texture, water stains, mud spots, and shadows caused by varying lighting. This significantly reduces recognition accuracy and can easily result in a high number of false positives and missed negatives.

[0004] Therefore, an optimized greenhouse pest and disease intelligent monitoring and early warning system and method based on the Internet of Things is expected. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a greenhouse pest and disease intelligent monitoring and early warning system and method based on the Internet of Things, which first uses a target detection model to perform a preliminary screening of the visible light RGB image of the greenhouse plant to determine whether the plant has pests and diseases, and obtain the preliminary screening results of the pests and diseases and the corresponding confidence score. At the same time, the greenhouse environmental data is subjected to time series analysis to determine whether it is suitable for the breeding of pests and diseases, and the confidence threshold is dynamically adjusted accordingly, and the preliminary screening results are graded, that is, high-confidence results are directly warned, low-confidence results are filtered, and suspicious pest and disease images with medium confidence scores are further fused with their corresponding multispectral images, and accurate identification and early warning of pests and diseases are achieved by performing deep correlation and fusion analysis on the two. This method uses a coarse screening followed by fine judgment mechanism, and only introduces multispectral data for in-depth analysis in suspicious scenes, while improving the accuracy of pest and disease monitoring and avoiding unnecessary computational overhead.

[0006] According to one aspect of the present application, a greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things is provided, which includes:

[0007] Collect visible light RGB images, key band multispectral images of greenhouse plants, and time series of plant growth environment data within a preset time window, wherein the plant growth environment data includes the ambient temperature, ambient humidity, and soil moisture in the greenhouse;

[0008] Inputting the visible light RGB image into a pest and disease preliminary screening module based on the YOLOv5 model to obtain a preliminary pest and disease screening result of the visible light RGB image and a corresponding confidence score;

[0009] Dynamically regulating the lower confidence threshold of the preliminary screening results of the pests and diseases based on the time series of the plant growth environment data;

[0010] Based on the confidence lower limit threshold and the preset confidence upper limit threshold, filtering the visible light RGB image, and screening pest and disease warning or suspicious pest and disease image;

[0011] In response to the visible light RGB image being a suspicious pest image, the suspicious pest image and the corresponding key band multispectral image are input into a pest and disease precise early warning module based on multimodal image fusion to determine whether to issue a pest and disease early warning.

[0012] According to another aspect of the present application, there is provided an IoT-based greenhouse pest and disease intelligent monitoring and early warning system, which includes:

[0013] A greenhouse data acquisition module is used to collect visible light RGB images of greenhouse plants, multispectral images of key bands, and time series of plant growth environment data within a preset time window. The plant growth environment data includes the ambient temperature, ambient humidity, and soil moisture in the greenhouse;

[0014] A preliminary pest and disease screening module is used to input the visible light RGB image into a preliminary pest and disease screening module based on the YOLOv5 model to obtain a preliminary pest and disease screening result of the visible light RGB image and a corresponding confidence score;

[0015] A confidence threshold control module, configured to dynamically control the confidence lower limit threshold of the preliminary screening result of the plant growth environment data based on the time series of the plant growth environment data;

[0016] A hierarchical screening module, configured to filter the visible light RGB image, perform pest warning or suspicious pest image screening based on the confidence lower limit threshold and a preset confidence upper limit threshold;

[0017] The pest and disease early warning module is used to input the suspicious pest and disease image and the corresponding key band multispectral image into the pest and disease precise early warning module based on multimodal image fusion to determine whether to issue a pest and disease early warning in response to the visible light RGB image being a suspicious pest and disease image.

[0018] Compared with the existing technology, the greenhouse pest and disease intelligent monitoring and early warning system and method based on the Internet of Things provided by this application first uses the target detection model to perform a preliminary screening of the visible light RGB images of greenhouse plants to determine whether the plants have pests and diseases, and obtain the preliminary screening results of pests and diseases and the corresponding confidence scores. At the same time, by performing time series analysis on the greenhouse environmental data to determine whether it is suitable for the breeding of pests and diseases, and dynamically adjusting the confidence threshold accordingly, the preliminary screening results are graded, that is, high-confidence results are directly warned, low-confidence results are filtered, and suspicious pest and disease images with medium confidence scores are further fused with their corresponding multispectral images. By performing deep correlation and fusion analysis on the two, accurate identification and early warning of pests and diseases are achieved. This method adopts a coarse screening followed by fine judgment mechanism, and only introduces multispectral data for in-depth analysis in suspicious scenes. While improving the accuracy of pest and disease monitoring, it avoids unnecessary computing overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 The flowchart is a method for intelligent monitoring and early warning of greenhouse pests and diseases based on the Internet of Things according to an embodiment of the present application.

[0021] Figure 2 This is a data flow diagram of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to an embodiment of the present application.

[0022] Figure 3 This is a flowchart of sub-step S3 of the IoT-based greenhouse pest and disease intelligent monitoring and early warning method according to an embodiment of the present application.

[0023] Figure 4 This is a flowchart of sub-step S5 of the IoT-based greenhouse pest and disease intelligent monitoring and early warning method according to an embodiment of the present application.

[0024] Figure 5 This is a flowchart of sub-step S52 of the IoT-based greenhouse pest and disease intelligent monitoring and early warning method according to an embodiment of the present application.

[0025] Figure 6 This is a flowchart of sub-step S522 of the IoT-based greenhouse pest and disease intelligent monitoring and early warning method according to an embodiment of the present application.

[0026] Figure 7 This is a block diagram of an IoT-based greenhouse pest and disease intelligent monitoring and early warning system according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0028] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0029] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0031] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0032] In response to the technical problems described in the above background technology, this application proposes an intelligent monitoring and early warning method for greenhouse pests and diseases based on the Internet of Things. It first uses a target detection model to perform a preliminary screening of visible light RGB images of greenhouse plants to determine whether the plants have pests and diseases, and obtain preliminary screening results of pests and diseases and corresponding confidence scores. At the same time, it performs time series analysis on greenhouse environmental data to determine whether it is suitable for the breeding of pests and diseases, and dynamically adjusts the confidence threshold accordingly. The preliminary screening results are graded, that is, high-confidence results are directly warned, low-confidence results are filtered, and suspicious pest and disease images with medium confidence scores are further fused with their corresponding multispectral images. By performing deep correlation and fusion analysis on the two, accurate identification and early warning of pests and diseases are achieved. This method uses a coarse screening followed by fine judgment mechanism, and only introduces multispectral data for in-depth analysis in suspicious scenarios. While improving the accuracy of pest and disease monitoring, it avoids unnecessary computing overhead.

[0033] Figure 1 The flowchart is a method for intelligent monitoring and early warning of greenhouse pests and diseases based on the Internet of Things according to an embodiment of the present application. Figure 2 The data flow diagram of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to the embodiment of the present application is shown in FIG. Figure 1 and Figure 2As shown, the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things includes the following steps: S1, collecting visible light RGB images, key band multispectral images and a time series of plant growth environment data within a preset time window of greenhouse plants, wherein the plant growth environment data includes the ambient temperature, ambient humidity and soil moisture in the greenhouse; S2, inputting the visible light RGB images into a pest and disease preliminary screening module based on the YOLOv5 model to obtain a preliminary pest and disease screening result of the visible light RGB images and a corresponding confidence score; S3, dynamically adjusting the confidence lower limit threshold of the preliminary pest and disease screening result based on the time series of the plant growth environment data; S4, filtering the visible light RGB images, performing pest and disease warning or screening of suspicious pest and disease images based on the confidence lower limit threshold and a preset confidence upper limit threshold; S5, in response to the visible light RGB image being a suspicious pest and disease image, inputting the suspicious pest and disease image and the corresponding key band multispectral image into a pest and disease precise early warning module based on multimodal image fusion to determine whether to perform a pest and disease warning.

[0034] In the above-mentioned IoT-based greenhouse pest and disease intelligent monitoring and early warning method, step S1 collects visible light RGB images, key band multispectral images, and a time series of plant growth environment data within a preset time window of greenhouse plants. The plant growth environment data includes the ambient temperature, ambient humidity, and soil moisture in the greenhouse. It should be understood that traditional single visible light images are difficult to distinguish between the early and weak symptoms of pests and diseases and background noise (such as light spots and water stains) when faced with a complex greenhouse environment, resulting in low recognition accuracy. At the same time, the occurrence of pests and diseases is closely related to environmental conditions. Simply analyzing images while ignoring environmental risk factors will make the early warning model one-sided and passive. Therefore, the present application realizes all-round data capture of the monitored objects by deploying an IoT sensor network in the greenhouse. Specifically, a high-definition RGB camera regularly captures morphological images of plant leaves and fruits for visual symptom identification. A multispectral camera simultaneously captures multispectral images in specific key bands (such as near-infrared and red edge bands, which are highly sensitive to plant health) to capture early physiological stress information such as chlorophyll abnormalities and water stress, which are invisible to the naked eye. Furthermore, temperature, humidity, and soil moisture sensors deployed in the greenhouse continuously record key parameters of the plant growth microenvironment, monitor pest and disease breeding conditions, and generate a time series of plant growth environment data with temporal context. This approach enables comprehensive, multimodal monitoring of greenhouse plant pests and diseases, providing a rich data foundation for subsequent pest and disease early warning.

[0035] During implementation, high-definition RGB cameras, multispectral cameras, and temperature and humidity sensors are strategically placed throughout the greenhouse to capture information from various dimensions. For visible light RGB imagery, high-resolution cameras are used to periodically capture morphological images of plant leaves and fruits. These images clearly demonstrate changes in color, shape, and texture on the plant surface, which is key to identifying the early symptoms of pests and diseases. Choosing the appropriate shooting time and frequency is crucial. For example, shooting at specific times of the day can improve image quality by avoiding shadows and reflections caused by direct sunlight. Adjusting the shooting interval based on the plant's growth cycle helps to promptly capture any potential abnormal changes.

[0036] At the same time, using a multispectral camera to acquire multispectral images in specific wavelengths is also a crucial measure. These wavelengths, such as near-infrared and red edge, are extremely sensitive to plant health and can reveal subtle differences imperceptible to the naked eye, such as changes in chlorophyll content or water stress. By analyzing this information, plant stress signals can be detected earlier, allowing appropriate measures to prevent the disease from worsening. When setting the parameters of a multispectral camera, the characteristics of the plant species and its growth stage must be considered to ensure that the selected wavelengths best reflect subtle changes in plant health.

[0037] Furthermore, data on ambient temperature, humidity, and soil moisture are equally essential. These three types of data can be obtained using temperature and humidity sensors and soil moisture sensors installed within the greenhouse. These sensors should be evenly distributed throughout the greenhouse to provide comprehensive coverage of all growing areas. When monitoring ambient temperature and humidity, it's important to pay attention not only to regular daytime and nighttime fluctuations but also to rapid changes during extreme weather conditions. As for soil moisture, measurement depth and frequency should be adjusted based on the needs of different crops and soil types to ensure accurate and representative data.

[0038] Given that different plants have significantly different requirements for environmental conditions, these specific requirements must be fully accounted for during data collection. For example, some tropical plants may require higher humidity to maintain normal growth, while some drought-tolerant crops can thrive in relatively dry conditions. Given this, when deploying the sensor network, it is important not only to consider the uniformity of the overall greenhouse environment but also to customize the configuration based on the characteristics of different growing areas. This ensures that plants in every corner receive the most suitable growing conditions and provides a more accurate reference for subsequent data analysis.

[0039] Furthermore, with the advancement of technology, it is now possible to use drones equipped with cameras or other remote sensing equipment to collect data over large areas. This approach is particularly suitable for large greenhouses or locations that are difficult to reach manually. By programming drones to fly along pre-set routes and pause at designated locations to capture images, a more extensive and detailed data set can be obtained. This approach not only reduces labor costs but also improves efficiency, making it particularly suitable for assessing crop health over large areas.

[0040] In the above-mentioned greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things, in step S2, the visible light RGB image is input into the pest and disease primary screening module based on the YOLOv5 model to obtain the pest and disease preliminary screening result of the visible light RGB image and the corresponding confidence score. It should be understood that this application takes into account that directly performing in-depth analysis on the full amount of multispectral data will result in huge computing overhead, and the greenhouse plant pest and disease monitoring scenario needs to take into account both real-time and resource constraints. Therefore, in order to establish an efficient pest and disease primary screening mechanism, this application adopts a lightweight target detection algorithm, and constructs a pest and disease primary screening module based on the YOLOv5 model to quickly screen the visible light RGB images of greenhouse plants to preliminarily determine whether the plants have pests and diseases, and give corresponding confidence scores. Specifically, the YOLOv5 model is pre-trained on a large number of labeled greenhouse pest and disease image data sets, and can accurately identify visual features such as a variety of disease spots, pest entities or bite marks. During runtime, the captured visible light RGB image is fed into the YOLOv5 model. The model performs feature extraction and bounding box prediction on the input visible light RGB image to quickly output the location (bounding box) of any suspected pest or disease targets detected, along with a quantified confidence score. The confidence score represents the model's confidence in its initial pest and disease screening result: "Pest or disease targets are present in this area." A higher score indicates a greater likelihood of pest or disease presence. This method enables efficient preliminary screening of greenhouse plant visual data, filtering out healthy images of no concern and assigning a quantifiable suspicion index to all potentially problematic images, providing guidance for subsequent, refined pest and disease screening.

[0041] In the above-mentioned IoT-based greenhouse pest and disease intelligent monitoring and early warning method, the step S3 dynamically adjusts the confidence lower limit threshold of the preliminary screening results of pests and diseases based on the time series of the plant growth environment data. Specifically, considering that the outbreak of pests and diseases is not an isolated incident, but the result of the long-term action of environmental factors, the use of a fixed confidence threshold to judge the preliminary screening results of pests and diseases ignores the influence of environmental factors on the probability of occurrence of pests and diseases, and may miss early symptoms due to excessively high thresholds in a high-incidence environment, or introduce too much interference due to excessively low thresholds in a safe environment. Therefore, in order to adaptively adjust the warning sensitivity according to environmental risks and achieve more intelligent warning decisions, the present application further evaluates the risk of pest and disease breeding in the current environment by performing data analysis on the time series of the plant growth environment data, and dynamically adjusts the confidence lower limit threshold of the preliminary screening results of pests and diseases accordingly, so that it matches the risk level of the current greenhouse environment to avoid omissions or false alarms. Among them, Figure 3 FIG is a flowchart of sub-step S3 of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to an embodiment of the present application. Figure 3 As shown, the step S3 includes the following steps: S31, inputting the time series of the plant growth environment data into the pest and disease risk assessment model based on the LSTM model to obtain the pest and disease risk level under the current plant growth environment; S32, based on a preset pest and disease risk level-confidence threshold mapping table, determining the confidence threshold corresponding to the pest and disease risk level under the current plant growth environment as the confidence lower limit threshold of the preliminary pest and disease screening result.

[0042] Specifically, in step S31, the time series of the plant growth environment data is input into a pest and disease risk assessment model based on an LSTM model to obtain the pest and disease risk level under the current plant growth environment. Specifically, since the occurrence and development of pests and diseases are not instantaneous events, but the cumulative result of the continuous action of specific environmental conditions, for example, the outbreak of certain fungal diseases often requires a high temperature and high humidity environment for several hours or days. Therefore, in order to effectively capture the risk evolution characteristics with delayed effects and long-term dependencies, thereby revealing the potential pest and disease threats caused by environmental factors, this application is based on time series modeling technology and an LSTM network to construct a pest and disease risk assessment model for analyzing the evolution trend of environmental data, so as to achieve dynamic classification of the current comprehensive environmental risk. Specifically, first, the ambient temperature, ambient humidity, and soil moisture data continuously collected within a preset time window (such as the past 48 hours) are integrated into a multivariate time series and input into a pre-trained long short-term memory network (LSTM) model. Due to its unique gated recurrent unit structure, the LSTM network can effectively learn and memorize long-term dependencies in time series, thereby deeply understanding the complex impact of different environmental parameter combinations on the probability of disease occurrence over time. Through the fully connected neural network and Softmax function of the output layer, the learned multi-dimensional environmental parameter temporal contextual association features are mapped to specific pest and disease risk levels. Here, after being trained by associating a large amount of historical environmental data with the actual outbreaks of corresponding pests and diseases, the pest and disease risk assessment model can accurately output a comprehensive assessment result representing the current plant growth environment's friendliness to pests and diseases, namely specific pest and disease risk levels such as "low", "medium", and "high", thereby providing a reliable guide for the intelligent regulation of the sensitivity of subsequent pest and disease warnings.

[0043] Specifically, the step S32, based on the preset pest and disease risk level-confidence threshold mapping table, determines the confidence threshold corresponding to the pest and disease risk level in the current plant growth environment as the confidence lower limit threshold of the preliminary screening result of the pest and disease. It should be understood that in order to convert the pest and disease risk level in the current plant growth environment into specific operating parameters that can directly act on the image screening process, so that the strictness of the image screening can be adaptively adjusted as the environmental risk changes, this application is based on regularized mapping and expert experience, and completes the conversion from risk level to specific confidence numerical threshold by establishing a preset pest and disease risk level-confidence threshold mapping table. Specifically, the preset pest and disease risk level-confidence threshold mapping table is jointly formulated by agricultural pathology experts and algorithm engineers based on historical pest and disease outbreak cases and image screening effect feedback, and clearly defines the initial screening confidence lower limit threshold corresponding to different risk levels. For example, when the risk level is high, the confidence lower threshold is lowered to 0.45 accordingly, allowing more suspicious samples to enter the review during the high-risk period to reduce missed detections; when the risk level is medium, the threshold is set to 0.6, maintaining moderate screening stringency to balance missed detections and false positives; and when the risk level is low, the confidence lower threshold is raised to 0.75 to enhance screening stringency and reduce false positives. In this way, it ensures that in environments with high disease incidence, potential threats can be captured more sensitively, and suspected targets with low review scores but may be early symptoms; while under environmental safety conditions, noise is filtered out with higher standards, thus achieving an intelligent match between warning sensitivity and actual environmental risks, and improving the accuracy and practicality of the pest and disease early warning system.

[0044] In the aforementioned IoT-based greenhouse pest and disease intelligent monitoring and early warning method, step S4 filters the visible light RGB image, screening for pest and disease warning or suspected pest and disease images based on the lower confidence threshold and a preset upper confidence threshold. It should be understood that due to the variability in the confidence scores of preliminary pest and disease screening results, different confidence scores represent different levels of screening reliability. To address this, the present application, based on the principle of hierarchical decision-making, constructs a three-interval decision logic based on a dynamically set lower confidence threshold and a fixed upper confidence threshold to triage preliminary pest and disease screening results. Specifically, the confidence score of each preliminary pest and disease screening result is compared with a dynamically controlled lower confidence threshold and a preset upper confidence threshold (e.g., 0.9). If the confidence score of the preliminary pest and disease screening result is greater than or equal to the upper confidence threshold, it indicates that the pest and disease characteristics are extremely obvious and typical, and the case is directly determined to be a confirmed case, immediately triggering the pest and disease early warning mechanism to issue a pest and disease early warning. In response to the confidence score of the preliminary screening results of the pests and diseases being less than the lower confidence threshold, it is considered that the credibility of the test result is extremely low under the current environmental risk, and it is likely to be background noise or model misjudgment. The visible light RGB image is directly filtered and no further analysis is performed. In response to the confidence score of the preliminary screening results of the pests and diseases being greater than or equal to the lower confidence threshold and less than the upper confidence threshold, it indicates that the image is suspected of having certain pests and diseases, but the features are not clear enough and fall into the suspicious category. The visible light RGB image is then treated as a suspicious pest and disease image, and the next step of the precise diagnosis procedure is initiated. In this way, immediate response to high-risk events, effective filtering of low-risk information, and accurate verification of medium-risk doubts are ensured. While ensuring the reliability of the warning, the operating efficiency of the pest and disease monitoring and early warning system is greatly optimized.

[0045] In the above-mentioned intelligent monitoring and early warning method for greenhouse pests and diseases based on the Internet of Things, in step S5, in response to the visible light RGB image being a suspicious pest and disease image, the suspicious pest and disease image and the corresponding key band multispectral image are input into a precise pest and disease early warning module based on multimodal image fusion to determine whether to perform a pest and disease early warning. Specifically, for suspicious pest and disease images, their pest and disease symptoms are often in the early, atypical, or highly confused with the background stage. Simply relying on visible light information has reached the limit of its resolution ability, and it is difficult to make an accurate judgment. Therefore, in order to achieve the final and accurate confirmation of pests and diseases, this application is based on the idea of ​​deep complementary fusion of multimodal information. When the visible light RGB image is judged to be suspicious, the key band multispectral image collected at the same time and location is called to perform a joint deep diagnosis of multimodal information, so as to fully explore the subtle feature differences of pests and diseases under different spectra, and achieve high-precision recognition of pest and disease symptoms. Among them, Figure 4FIG is a flowchart of sub-step S5 of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to an embodiment of the present application. Figure 4 As shown, the step S5 includes the steps of: S51, respectively extracting the image features of the suspicious pest and disease image and the key band multispectral image to obtain a pest and disease visible light image feature map and a pest and disease multispectral image feature map; S52, cascading the pest and disease visible light image feature map and the pest and disease multispectral image feature map along the channel dimension into a multimodal pest and disease feature fusion coding feature map, and performing feature sparsification processing based on the attention mechanism to obtain a multimodal pest and disease feature sparse coding feature map; S53, inputting the multimodal pest and disease feature sparse coding feature map into a classifier-based pest and disease warning decision model to determine whether to perform a pest and disease warning.

[0046] Specifically, in one specific example of the present application, step S51 includes performing image feature extraction based on the FPN model on the suspected pest image and the key band multispectral image, respectively, to obtain a visible light pest image feature map and a multispectral pest image feature map. It should be understood that, given that plant pest symptoms often exhibit diverse scale characteristics, ranging from tiny early spots to larger areas of chlorosis or curling, conventional convolutional neural networks are prone to losing fine features of small targets during layer-by-layer downsampling. Therefore, in order to simultaneously capture disease characteristics from both macroscopic and microscopic levels from image data of two modalities, the present application, based on the Feature Pyramid Network (FPN) model, constructs two parallel multiscale feature extraction pathways to perform image feature extraction on the suspected pest image and the key band multispectral image, respectively, to generate a visible light pest image feature map and a multispectral pest image feature map that combine deep semantics with high-resolution detail. Specifically, the FPN model extracts semantic features at different levels of the image through a bottom-up feedforward path, and integrates the strong semantic features of the high-level with the rich spatial detail information of the low-level through a top-down path and lateral connections. It can fully capture large-scale disease symptoms at the macro scale and tiny lesions or abnormal textures at the micro scale, thereby obtaining an image feature representation containing multi-scale information, providing a solid feature foundation for subsequent accurate early warning of pests and diseases.

[0047] Specifically, in step S52, after concatenating the pest and disease visible light image feature map and the pest and disease multispectral image feature map along the channel dimension to form a multimodal pest and disease feature fusion coding feature map, feature sparsification processing based on the attention mechanism is performed to obtain a multimodal pest and disease feature sparse coding feature map. It should be understood that the present application takes into account that the pest and disease visible light image feature map and the pest and disease multispectral image feature map, after simple concatenation and merging, will inevitably contain redundant information (such as contour information captured by both) and noise interference (such as artifacts generated in visible light by changes in illumination), which may cause key and complementary diagnostic information to be submerged, affecting the accuracy of the final judgment. Therefore, in order to effectively remove the redundant and noisy information in the multimodal pest and disease feature fusion coding feature map and highlight the key diagnostic features, the present application further performs feature sparse processing based on the attention mechanism. By taking the various feature channels of the multimodal pest and disease feature fusion coding feature map as the basic unit, the gating mechanism is used to dynamically evaluate the importance of each feature channel, and based on this, feature channels with high diagnostic value are selectively retained, while redundant or unimportant feature channels are suppressed, thereby obtaining a refined and key diagnostic information-rich multimodal pest and disease feature sparse coding feature map, which helps to focus on the core essence of the disease, while reducing the subsequent computational complexity, and greatly improving the accuracy and robustness of pest and disease identification. Among them, Figure 5 FIG is a flowchart of sub-step S52 of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to an embodiment of the present application. Figure 5 As shown, the step S52 includes the steps of: S521, performing context-associated coding based on channel feature units on the multimodal pest and disease feature fusion coding feature map to obtain a sequence distribution of multimodal pest and disease image channel unit context-associated feature coding vectors; S522, performing channel unit sparse processing on the multimodal pest and disease feature fusion coding feature map based on the channel significance of each multimodal pest and disease image channel unit context-associated feature coding vector to obtain the multimodal pest and disease feature sparse coding feature map.

[0048] More specifically, in a specific example of the present application, step S521 includes: first, performing feature unitization on the multimodal pest and disease feature fusion encoding feature map along the channel dimension to obtain a sequence distribution of the multimodal pest and disease image channel unit feature vector, which is expressed as follows:

[0049]

[0050]

[0051] in, Represents the multimodal pest and disease feature fusion coding feature map, represents the set of real numbers, 、 and Respectively represent the height, width and number of channels of the multimodal pest and disease feature fusion encoding feature map, represents the sequence distribution of the channel unit feature vector of multimodal pest and disease images, 、 and They represent the first, second and third order in the sequence distribution of the channel unit feature vector of the multimodal pest and disease image. Multimodal pest and disease image channel unit feature vector.

[0052] That is, the multimodal pest and disease feature fusion encoding feature map is decomposed into independent channel feature matrices in the channel dimension, and through matrix pooling operations, the spatial features of each channel are transformed into feature units that can be processed separately, providing a basic format for subsequent feature screening and optimization based on inter-channel correlation and significance, so as to more accurately capture feature information valuable for pest and disease identification. Based on this, by converting the multimodal pest and disease feature fusion encoding feature map into a sequence distribution of multimodal pest and disease image channel unit feature vectors, the model can effectively mine channel context associations, thereby eliminating redundant information while retaining key features, and improving the effectiveness and pertinence of feature representation.

[0053] Then, the sequence distribution of the multimodal pest and disease image channel unit feature vector is input into the channel feature correlation encoder based on the Transformer architecture to obtain the sequence distribution of the multimodal pest and disease image channel unit context correlation feature encoding vector, which is expressed as follows:

[0054]

[0055]

[0056] in, represents the Transformer architecture, represents the sequence distribution of context-related feature encoding vectors of multimodal pest and disease image channel units, 、 and They represent the first, second and third sequences in the sequence distribution of the context-related feature encoding vectors of the multimodal pest and disease image channel unit. Context-related feature encoding vector of multimodal pest and disease image channel units.

[0057] That is, the self-attention mechanism of the Transformer architecture is used to capture and encode the long-distance dependencies and complex contextual associations between each channel unit in the sequence distribution of the multimodal pest and disease image channel unit feature vectors, thereby refining each multimodal pest and disease image channel unit feature vector into a multimodal pest and disease image channel unit contextual association feature encoding vector that incorporates global channel context information, providing a feature basis with semantic associations between channels for subsequent feature sparsification processing based on channel saliency, so as to more accurately screen out feature channels that play a key role in pest and disease identification.

[0058] Figure 6 FIG is a flowchart of sub-step S522 of the greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to an embodiment of the present application. Figure 6 As shown, the step S522 includes the steps of: S5221, calculating the channel significance factor of each multimodal pest and disease image channel unit context-associated feature coding vector in the sequence distribution of the multimodal pest and disease image channel unit context-associated feature coding vector; S5222, based on the channel significance factor of each multimodal pest and disease image channel unit context-associated feature coding vector, performing a sparse processing on the sequence distribution of the multimodal pest and disease image channel unit feature vector to obtain a sparse sequence distribution of the multimodal pest and disease image channel unit feature vector; S5223, performing a feature reshape on the sparse sequence distribution of the multimodal pest and disease image channel unit feature vector to obtain the multimodal pest and disease feature sparse coding feature map.

[0059] In a specific example of the present application, step S5221 includes: first, calculating the Shannon entropy weighted sum between the multimodal pest and disease image channel unit context-related feature coding vector, the multimodal pest and disease image channel unit feature vector, and the difference vector between the two as the channel significance evaluation index of the multimodal pest and disease image channel unit context-related feature coding vector; then, performing a sigmoid function-based normalization process on the channel significance evaluation index to obtain the channel significance factor, which is expressed as follows:

[0060]

[0061] in, represents the sigmoid activation function, 、 and denote the learnable balance parameters, is a vector The Shannon entropy of for The Shannon entropy of for The Shannon entropy of express The channel significance factor.

[0062] That is, by calculating the Shannon entropy weighted sum of the context-related feature encoding vector of the multimodal pest and disease image channel unit, the multimodal pest and disease image channel unit feature vector and its differential vector, the context-related changes and information differences of each channel feature are quantified from the perspective of information entropy, and then the contribution of the channel to the expression of pest and disease features is evaluated. It is then normalized and scaled to between 0 and 1 through the sigmoid function to obtain the channel significance factor that can be directly used for feature screening, providing an accurate screening basis for the subsequent saliency-based channel unit sparse processing, so that the model can adaptively retain high-contribution channel features and suppress redundant features, thereby optimizing the effectiveness and pertinence of multimodal feature representation.

[0063] Preferably, use vector , and When calculating the dimensional discrimination of the Shannon entropy, considering the possible local mutation of the sequence distribution of the multimodal pest and disease image channel unit context-related feature encoding vector obtained after the context-related encoding through the Transformer architecture relative to the sequence distribution of the multimodal pest and disease image channel unit feature vector, the uncertainty measurement of the Shannon entropy may also fall into the non-multi-stationary characteristics, which makes it difficult to use only the balance parameter 、 and It may be difficult to compensate. Based on this, in a preferred example of the present application, the step S5221 calculates the Shannon entropy weighted sum between the multimodal pest and disease image channel unit context-associated feature coding vector, the multimodal pest and disease image channel unit feature vector and the difference vector thereof as the channel significance evaluation index of the multimodal pest and disease image channel unit context-associated feature coding vector, and then further performs a multi-stable balance correction based on the entropy metric space on the channel significance evaluation index of the multimodal pest and disease image channel unit context-associated feature coding vector to obtain a corrected channel significance evaluation index.

[0064] Specifically, for the uncertainty quantification method of Shannon entropy, the uncertainty configuration space matrix is ​​first introduced to express the uncertainty configuration distribution in the steady-state configuration space:

[0065]

[0066]

[0067]

[0068] in, 、 and Represents vectors ,vector , and vector The corresponding entropy measure uncertainty configuration space matrix, Represents the transpose of a vector;

[0069] Furthermore, considering that the multiple equilibrium states should correspond to the local optimal points in the uncertain configuration distribution, the matrix 、 and The spectral minimum of , and , and for local cusp mutations, the structural morphology needs to satisfy the low-dimensional submanifold constraint. Therefore, the orbital integration algorithm of the weighted algebraic framework is applied to achieve the algebraic symmetry preservation of multiple equilibrium states, namely:

[0070]

[0071]

[0072]

[0073] in, 、 、 Respectively 、 and The corresponding corrected balance parameters, represents the F norm of the matrix, The uncertainty space matrix for entropy measurement is constructed 、 and 's index;

[0074] Therefore, the Shannon entropy is used to describe the multi-stable eigenvalues ​​based on uncertainty measurement through nonlinear algebraic transformation of uncertainty configuration, and the multi-stable convergence point control is defined by topological reorganization of configuration space. Finally, the modified equilibrium parameter 、 and Substitute into the calculation of the channel significance evaluation index to achieve multi-stable equilibrium correction of the channel significance evaluation index, and then perform normalization processing based on the sigmoid function on the corrected channel significance evaluation index to obtain the channel significance factor: , thereby improving the multi-equilibrium convergence characteristics of the channel significance measure.

[0075] In a specific example of the present application, step S5222 is expressed as follows:

[0076]

[0077] in, Represents a mask operation, Indicates the gating threshold.

[0078] Specifically, a gating mechanism is used to selectively retain and suppress the context-related feature encoding vectors of multimodal pest and disease image channel units, allowing high-significance channel features to be retained or enhanced while low-significance channel features to be suppressed or eliminated. This allows for sparse feature screening, focusing on features that are critical to pest and disease identification and reducing information redundancy. The resulting sparse sequence distribution of multimodal pest and disease image channel unit feature vectors effectively filters redundant feature channels while retaining feature channels containing key pest and disease information, making feature representation more targeted and providing streamlined and high-value feature input for subsequent feature reconstruction and pest and disease early warning decision-making, thereby improving model processing efficiency and recognition accuracy.

[0079] In a specific example of the present application, step S5223 reshapes the sparse sequence distribution of the multimodal pest and disease image channel unit feature vector to obtain the multimodal pest and disease feature sparse coding feature map. That is, the sparse sequence distribution of the multimodal pest and disease image channel unit feature vector after the sparse processing is restored to the standard multidimensional feature map format, thereby restoring the spatial structure information of the feature map and obtaining the multimodal pest and disease feature sparse coding feature map. This ensures the plug-and-play nature of the network module in the deep learning model while realizing intelligent screening and sparsification of channel information, and provides format-adapted feature input for subsequent pest and disease warning decision-making.

[0080] Specifically, in step S53, the multimodal pest and disease feature sparse coding feature map is input into a pest and disease warning decision model based on a classifier to determine whether to perform a pest and disease warning. That is, in order to convert the multimodal pest and disease feature sparse coding feature map into a clear instruction for pest and disease warning, the present application is based on a supervised learning classification algorithm, and by training the pest and disease warning decision model based on a classifier, it learns the mapping relationship between the multimodal fusion features of a large number of known samples and whether they actually exist pests and diseases, thereby achieving effective analysis and accurate classification of the multimodal pest and disease feature sparse coding feature map. Specifically, first, the multimodal pest and disease feature sparse coding feature map is subjected to a global average pooling operation to convert it into a one-dimensional vector of a fixed length, and then the one-dimensional vector is sent to a lightweight classifier, which is composed of a plurality of fully connected layers and a softmax activation function of an output layer. The probability distribution of whether to perform a pest and disease warning is output through the softmax activation function, and the category with the largest probability is selected as the final warning decision result. If the result is a pest and disease warning, the warning signal will be triggered immediately, and the warning information together with the relevant image data will be sent to the manager's smart terminal so that the manager can quickly understand the situation and take appropriate prevention and control measures.

[0081] In summary, the IoT-based greenhouse pest and disease intelligent monitoring and early warning method based on the embodiment of the present application is explained. It first uses the target detection model to perform a preliminary screening of the visible light RGB images of greenhouse plants to determine whether the plants have pests and diseases, and obtain the preliminary screening results of pests and diseases and the corresponding confidence scores. At the same time, the greenhouse environmental data is analyzed in time series to determine whether it is suitable for the breeding of pests and diseases, and the confidence threshold is dynamically adjusted accordingly. The preliminary screening results are graded, that is, high-confidence results are directly warned, low-confidence results are filtered, and suspicious pest and disease images with medium confidence scores are further fused with their corresponding multispectral images. By performing deep correlation and fusion analysis on the two, accurate identification and early warning of pests and diseases are achieved. This method uses a coarse screening followed by fine judgment mechanism, and only introduces multispectral data for in-depth analysis in suspicious scenes. While improving the accuracy of pest and disease monitoring, it avoids unnecessary computational overhead.

[0082] Furthermore, an intelligent monitoring and early warning system for greenhouse pests and diseases based on the Internet of Things is also provided.

[0083] Figure 7 FIG is a block diagram of an intelligent monitoring and early warning system for greenhouse pests and diseases based on the Internet of Things according to an embodiment of the present application. Figure 7As shown, according to the embodiment of the present application, the greenhouse pest and disease intelligent monitoring and early warning system 100 based on the Internet of Things includes: a greenhouse data acquisition module 110, which is used to collect visible light RGB images, key band multispectral images and time series of plant growth environment data within a preset time window of greenhouse plants, wherein the plant growth environment data includes the ambient temperature, ambient humidity and soil moisture in the greenhouse; a pest and disease preliminary screening module 120, which is used to input the visible light RGB image into the pest and disease preliminary screening module based on the YOLOv5 model to obtain the pest and disease preliminary screening results of the visible light RGB image and the corresponding confidence score; The confidence threshold control module 130 is used to dynamically control the lower confidence threshold of the preliminary screening result of the pests and diseases based on the time series of the plant growth environment data; the hierarchical screening module 140 is used to filter the visible light RGB image, issue pest and disease warnings, or screen suspicious pest and disease images based on the confidence lower limit threshold and a preset confidence upper limit threshold; the pest and disease warning module 150 is used to input the suspicious pest and disease image and the corresponding key band multispectral image into the pest and disease precise warning module based on multimodal image fusion to determine whether to issue a pest and disease warning in response to the visible light RGB image being a suspicious pest and disease image.

[0084] Here, those skilled in the art will understand that the specific operations of each module in the above-mentioned greenhouse pest and disease intelligent monitoring and early warning system based on the Internet of Things have been referred to above. Figures 1 to 6 The description of the intelligent monitoring and early warning method for greenhouse pests and diseases based on the Internet of Things has been introduced in detail, and therefore, its repeated description will be omitted.

[0085] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0086] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0087] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0088] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0089] Finally, it should be noted that the above description has been provided for the purpose of illustration and description. In addition, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the technical solutions may be modified or replaced with equivalents with reference to the preferred embodiments, they do not depart from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent monitoring and early warning method for greenhouse pests and diseases based on the Internet of Things, characterized in that: include: Collect visible light RGB images, key band multispectral images of greenhouse plants, and time series of plant growth environment data within a preset time window, wherein the plant growth environment data includes the ambient temperature, ambient humidity, and soil moisture in the greenhouse; Inputting the visible light RGB image into a pest and disease preliminary screening module based on the YOLOv5 model to obtain a preliminary pest and disease screening result of the visible light RGB image and a corresponding confidence score; Dynamically regulating the lower confidence threshold of the preliminary screening results of the pests and diseases based on the time series of the plant growth environment data; Based on the confidence lower limit threshold and the preset confidence upper limit threshold, filtering the visible light RGB image, and screening pest and disease warning or suspicious pest and disease image; In response to the visible light RGB image being a suspicious pest image, the suspicious pest image and the corresponding key band multispectral image are input into a pest and disease precise early warning module based on multimodal image fusion to determine whether to issue a pest and disease early warning; Inputting the suspected pest image and the corresponding key band multispectral image into a pest and disease precise early warning module based on multimodal image fusion to determine whether to issue a pest and disease early warning, including: Extracting image features of the suspected pest and disease image and the key band multispectral image respectively to obtain a pest and disease visible light image feature map and a pest and disease multispectral image feature map; After concatenating the pest and disease visible light image feature map and the pest and disease multispectral image feature map along the channel dimension into a multimodal pest and disease feature fusion coding feature map, a feature sparsification process based on an attention mechanism is performed to obtain a multimodal pest and disease feature sparse coding feature map; The multimodal pest and disease feature sparse coding feature map is input into a classifier-based pest and disease warning decision model to determine whether to issue a pest and disease warning.

2. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 1 is characterized in that: Dynamically regulating the lower confidence threshold of the preliminary screening results of the pests and diseases based on the time series of the plant growth environment data includes: Inputting the time series of the plant growth environment data into a pest and disease risk assessment model based on an LSTM model to obtain the pest and disease risk level under the current plant growth environment; Based on a preset pest and disease risk level-confidence threshold mapping table, a confidence threshold corresponding to the pest and disease risk level under the current plant growth environment is determined as the confidence lower limit threshold of the preliminary pest and disease screening result.

3. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 1 is characterized in that: Based on the confidence lower limit threshold and the preset confidence upper limit threshold, filtering the visible light RGB image, and screening pest and disease warning or suspicious pest and disease image, including: In response to the confidence score of the preliminary pest and disease screening result being greater than or equal to the upper confidence threshold, issuing a pest and disease early warning; In response to the confidence score of the preliminary pest and disease screening result being less than the confidence lower limit threshold, filtering the visible light RGB image; In response to the confidence score of the preliminary pest and disease screening result being greater than or equal to the confidence lower threshold and less than the confidence upper threshold, the visible light RGB image is used as a suspicious pest and disease image.

4. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 3 is characterized in that: Extracting image features of the suspicious pest image and the key band multispectral image respectively to obtain a pest visible light image feature map and a pest multispectral image feature map, including: Image feature extraction based on the FPN model is performed on the suspicious pest image and the key band multispectral image to obtain the pest visible light image feature map and the pest multispectral image feature map.

5. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 4 is characterized in that: After the pest and disease visible light image feature map and the pest and disease multispectral image feature map are cascaded along the channel dimension to form a multimodal pest and disease feature fusion coding feature map, feature sparsification processing based on the attention mechanism is performed to obtain a multimodal pest and disease feature sparse coding feature map, including: Performing context-associated coding based on channel feature units on the multimodal pest and disease feature fusion coding feature map to obtain a sequence distribution of multimodal pest and disease image channel unit context-associated feature coding vectors; Based on the channel significance of the context-related feature coding vector of each multimodal pest and disease image channel unit, the multimodal pest and disease feature fusion coding feature map is subjected to channel unit sparse processing to obtain the multimodal pest and disease feature sparse coding feature map.

6. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 5 is characterized in that: The multimodal pest and disease feature fusion coding feature map is subjected to context-associated coding based on channel feature units to obtain a sequence distribution of multimodal pest and disease image channel unit context-associated feature coding vectors, including: Performing feature unitization on the multimodal pest and disease feature fusion encoding feature map along the channel dimension to obtain a sequence distribution of multimodal pest and disease image channel unit feature vectors; The sequence distribution of the multimodal pest and disease image channel unit feature vector is input into the channel feature correlation encoder based on the Transformer architecture to obtain the sequence distribution of the multimodal pest and disease image channel unit context correlation feature encoding vector.

7. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 6 is characterized in that: Based on the channel saliency of the channel unit context-related feature coding vectors of each multimodal pest and disease image channel, channel unit sparse processing is performed on the multimodal pest and disease feature fusion coding feature map to obtain the multimodal pest and disease feature sparse coding feature map, including: Calculating a channel significance factor of each multimodal pest and disease image channel unit context-related feature coding vector in the sequence distribution of the multimodal pest and disease image channel unit context-related feature coding vector; Based on the channel significance factors of the context-related feature coding vectors of the multimodal pest and disease image channel units, performing a sparse processing on the sequence distribution of the multimodal pest and disease image channel unit feature vectors to obtain a sparse sequence distribution of the multimodal pest and disease image channel unit feature vectors; The sparse sequence distribution of the channel unit feature vector of the multimodal pest and disease image is reshaped to obtain the multimodal pest and disease feature sparse coding feature map.

8. The greenhouse pest and disease intelligent monitoring and early warning method based on the Internet of Things according to claim 7 is characterized in that: Calculating the channel significance factor of each multimodal pest and disease image channel unit context-related feature coding vector in the sequence distribution of the multimodal pest and disease image channel unit context-related feature coding vector includes: Calculating the Shannon entropy weighted sum of the multimodal pest and disease image channel unit context-related feature coding vector, the multimodal pest and disease image channel unit feature vector, and a difference vector thereof as a channel significance evaluation index of the multimodal pest and disease image channel unit context-related feature coding vector; Performing a multi-stable equilibrium correction based on an entropy metric space on the channel significance evaluation index of the context-related feature encoding vector of the channel unit of the multimodal pest and disease image to obtain a corrected channel significance evaluation index; The corrected channel significance evaluation index is normalized based on a sigmoid function to obtain the channel significance factor.

9. An intelligent monitoring and early warning system for greenhouse pests and diseases based on the Internet of Things, used to execute the method according to any one of claims 1 to 8, characterized in that: include: A greenhouse data acquisition module is used to collect visible light RGB images of greenhouse plants, multispectral images of key bands, and time series of plant growth environment data within a preset time window. The plant growth environment data includes the ambient temperature, ambient humidity, and soil moisture in the greenhouse; A preliminary pest and disease screening module is used to input the visible light RGB image into a preliminary pest and disease screening module based on the YOLOv5 model to obtain a preliminary pest and disease screening result of the visible light RGB image and a corresponding confidence score; A confidence threshold control module, configured to dynamically control the confidence lower limit threshold of the preliminary screening result of the plant growth environment data based on the time series of the plant growth environment data; A hierarchical screening module, configured to filter the visible light RGB image, perform pest warning or suspicious pest image screening based on the confidence lower limit threshold and a preset confidence upper limit threshold; The pest and disease early warning module is used to input the suspicious pest and disease image and the corresponding key band multispectral image into the pest and disease precise early warning module based on multimodal image fusion to determine whether to issue a pest and disease early warning in response to the visible light RGB image being a suspicious pest and disease image.

Images