Multi-modal data mining system for tobacco leaf disease intelligent identification and risk early warning
Through the intelligent identification system of tobacco leaf disease with multimodal data fusion and dynamic threshold adjustment, the problems of inaccurate disease identification and lag in early warning in complex environments are solved, and efficient disease identification and timely early warning are achieved.
Patent Information
- Application Number
- CN202510634019.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-16
- Publication Date
- 2025-08-26
AI Technical Summary
In the prior art, the identification of tobacco leaf disease relies on manual field patrols and simple sensor detection, and cannot be monitored in real time in complex environments, resulting in inaccurate disease identification and lag in early warning.
A multimodal data mining system is adopted to integrate visible light images, infrared thermal imaging, environmental parameter data and blade spectral data, combined with deep learning models and convolutional neural networks, disease identification and risk warning are carried out through the disease index Dq, and the warning threshold is dynamically adjusted.
In complex environments, the accuracy of disease identification and timeliness of early warning are significantly improved, and the risk of disease spread is reduced.
Smart Images
Figure CN120544360A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tobacco leaf disease identification, and in particular to a multimodal data mining system for intelligent tobacco leaf disease identification and risk early warning. Background Art
[0002] Tobacco, the leaves of the Nicotiana plant in the Solanaceae family, are primarily used as raw material in the tobacco industry and possess both medicinal and economic value. However, tobacco leaf diseases pose significant risks during cultivation. Severe tobacco leaf diseases can lead to reduced yields and severe quality damage, resulting in increased economic losses. Tobacco leaf diseases primarily include anthrax, brown spot, and mosaic virus, requiring targeted prevention and control measures based on the specific disease. However, before implementing preventive measures, the presence of the disease must be promptly identified and early warnings must be provided to ensure timely control measures are implemented.
[0003] However, in the existing technology, tobacco leaf disease identification mainly relies on manual field inspections combined with simple sensor detection, which has the following problems: the traditional method is susceptible to interference in visible light image acquisition in complex environments (such as cloudy weather leading to uneven lighting and temperature and humidity fluctuations inside the greenhouse), and the disease feature extraction is inaccurate. At the same time, it relies on periodic manual inspections and cannot monitor disease dynamics in real time, delaying the opportunity for prevention and control.
[0004] Therefore, to address the problem of inaccurate identification of tobacco leaf diseases in complex planting environments, the present invention proposes a multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases. Summary of the Invention
[0005] The purpose of the present invention is to provide a multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases, so as to solve the problems of inaccurate identification of tobacco leaf diseases and delayed warning in complex planting environments raised in the above background technology.
[0006] To achieve the above objectives, the present invention provides a multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases, the system comprising a multimodal data acquisition module, a data processing unit, a disease identification module, a risk warning module, and a communication module;
[0007] The multimodal data acquisition module is used to collect multimodal data of the tobacco planting area. The multimodal data includes visible light images, infrared thermal imaging data, environmental parameter data and leaf spectrum data. The environmental parameter data includes air humidity H air , light intensity L, soil moisture S soil and ambient temperature T;
[0008] The data processing unit is used to pre-process the collected data, including image denoising, data standardization and feature extraction;
[0009] The disease identification module performs fusion analysis on the pre-processed multimodal data based on the deep learning model and calculates the disease index D q , the disease index D q The calculation formula is:
[0010] D q =(W1×H air )+(W2×L)+(W3×S soil )+(W4×T)+α×T env
[0011] Among them, W1, W2, W3 and W4 are the weight coefficients of each environmental parameter data, T env is the environmental parameter correction factor, and α is the preset coefficient;
[0012] The risk warning module is used to calculate the disease index D q Compared with the preset threshold, when D q Generate risk warning signals when thresholds are exceeded;
[0013] The communication module is used to send the warning signal and multimodal data to the user interaction module.
[0014] As a further improvement of this technical solution, the environmental parameter correction factor T env The calculation formula is:
[0015]
[0016] As a further improvement of the present technical solution, the disease recognition module uses a convolutional neural network to extract disease features from visible light images and locates abnormal temperature areas in combination with infrared thermal imaging data.
[0017] As a further improvement of this technical solution, the risk warning module is based on the disease index D q The numerical range of risk is divided into the following levels:
[0018] Preset thresholds D1 and D2, and D1 < D2;
[0019] When D q When ≤D1, it is marked as low risk and a low risk warning signal is generated;
[0020] When D1<D q When ≤D2, it is marked as medium risk and a medium risk warning signal is generated;
[0021] When D q When it is greater than D2, it is marked as high risk and a high-risk warning signal is generated.
[0022] As a further improvement of this technical solution, the thresholds D1 and D2 are dynamically adjusted according to historical disease data and seasonal factors, specifically:
[0023] Get the number of disease occurrences N in the same period of history hist And the total number of monitoring times N total ;
[0024] According to the formula Calculate the adjusted threshold D k , where D base is the basic threshold D1 or D2, and β is the adjustment coefficient.
[0025] As a further improvement of the present technical solution, the system further includes a user interaction module for displaying a disease distribution heat map, risk levels and prevention and control recommendations, and providing a historical data query function.
[0026] As a further improvement of the present technical solution, the data processing unit includes a data enhancement module for performing denoising and enhancement processing on the visible light image, specifically:
[0027] Generate adversarial network models to deblur visible light images affected by low light or haze, and output enhanced clear images.
[0028] The histogram equalization algorithm is used to enhance the color contrast between the diseased area and the healthy leaves;
[0029] The enhanced image is input into the disease recognition module.
[0030] As a further improvement of the present technical solution, the data processing unit further includes a data normalization module, an anomaly detection and cleaning module, and a time series data slicing module;
[0031] The data standardization module is used to uniformly convert multimodal data into a standardized format, specifically:
[0032] Normalize the pixel values of visible light images, convert the temperature difference data of infrared thermal imaging data into standard temperature units, and perform Z-score standardization on environmental parameter data;
[0033] The anomaly detection and cleaning module is used to identify and remove abnormal data collected by the sensor, specifically:
[0034] The isolation forest algorithm detects abnormal noise in visible light images, including reflective spots and obstructions. A sliding window is used to measure the continuity of environmental parameters. If a parameter fluctuates beyond a preset threshold within 10 seconds, it is marked as abnormal and triggers data re-collection. Values in the leaf spectral data that exceed the reasonable band range are truncated or interpolated.
[0035] The time series data slicing module is used to extract periodic change features based on the time series characteristics of environmental parameters, specifically:
[0036] A sliding window is used to segment the temperature and humidity time series data. Statistical features, including mean, variance, peak value, and slope, are calculated in each window, and a time series feature vector is generated. The time series features are fused with image features and input into the disease recognition module.
[0037] Compared with the prior art, the present invention has the following beneficial effects:
[0038] 1. In the multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases, by setting up a multimodal data acquisition module and utilizing multimodal data fusion, the system can effectively overcome the interference of a single data source in complex planting environments such as uneven lighting, temperature and humidity fluctuations. At the same time, it combines environmental correction factors to quantify the impact of temperature and humidity on disease development, thereby greatly improving the accuracy of disease identification.
[0039] 2. In this multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases, based on a dynamic threshold adjustment mechanism and in conjunction with a risk warning module, the system can automatically optimize the warning threshold based on historical data and seasonal changes. For example, in the rainy season, the humidity-related threshold is lowered to provide early warning of high-humidity diseases, thereby greatly shortening the warning response time and effectively preventing the spread of diseases. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 This is the overall principle block diagram of the multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases of the present invention. DETAILED DESCRIPTION
[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0042] In a specific embodiment, Figure 1 As shown in the figure, the multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases includes a multimodal data acquisition module, a data processing unit, a disease identification module, a risk warning module, a communication module, and a user interaction module. This system architecture can comprehensively integrate and analyze multimodal data, improving the accuracy of disease identification and the timeliness of warnings.
[0043] The multimodal data acquisition module is used to collect multimodal data of the tobacco planting area. The multimodal data includes visible light images, infrared thermal imaging data, environmental parameter data and leaf spectrum data. The environmental parameter data includes air humidity H air , light intensity L, soil moisture S soil and ambient temperature T. The collection of multimodal data can provide more comprehensive disease diagnosis information and overcome the limitations of a single data source.
[0044] The multimodal data acquisition module transmits the collected multimodal data to the data processing unit, and uses the data processing unit to preprocess the collected data, including image denoising, data standardization and feature extraction.
[0045] Among them, the data processing unit includes a data enhancement module, a data standardization module, an anomaly detection and cleaning module, and a time series data slicing module.
[0046] The data enhancement module is used to denoise and enhance visible light images, specifically:
[0047] The first step is to use a generative adversarial network model to deblur visible light images affected by low light or haze, outputting an enhanced, clear image, thereby improving image quality and enhancing the visibility of disease features.
[0048] The second step is to use the histogram equalization algorithm to enhance the color contrast between the diseased area and the healthy leaves, further highlighting the disease characteristics and facilitating subsequent identification;
[0049] The third step is to input the enhanced image into the disease recognition module.
[0050] The data standardization module is used to convert multimodal data into a standardized format. Specifically:
[0051] The first step is to normalize the pixel values of the visible light image to ensure data consistency and facilitate model processing;
[0052] The second step is to convert the temperature difference data of infrared thermal imaging data into standard temperature units, unify the data measurement standards, and improve the accuracy of analysis;
[0053] The third step is to perform Z-score standardization on the environmental parameter data to eliminate data scale differences and improve the model training effect.
[0054] The anomaly detection and cleaning module is used to identify and remove abnormal data collected by sensors. Specifically:
[0055] The first step is to detect abnormal noise in visible light images, including reflective points and occlusions, based on the isolation forest algorithm, to reduce noise interference and improve data quality.
[0056] The second step is to use a sliding window to count the continuity of environmental parameters. If a parameter fluctuates beyond a preset threshold within 10 seconds, it is marked as an anomaly and triggers data re-collection. This allows for timely detection and correction of abnormal data to ensure data accuracy.
[0057] The third step is to truncate or interpolate the values in the leaf spectral data that exceed the reasonable band range to repair the abnormal data and maintain the integrity of the data.
[0058] The time series data slicing module is used to extract periodic change characteristics based on the time series characteristics of environmental parameters. Specifically:
[0059] The first step is to use a sliding window to segment the temperature and humidity time series data to capture the changing trends of environmental parameters;
[0060] The second step is to calculate statistical features in each window, including mean, variance, peak value, and slope, and generate a time series feature vector to extract key features and enhance the model's sensitivity to environmental changes.
[0061] The third step is to fuse the time series features with the image features, input them into the disease recognition module, and fuse multi-source features to improve the accuracy of disease recognition.
[0062] The disease recognition module is used to identify and receive pre-processed multimodal data, and the pre-processed multimodal data is fused and analyzed based on the deep learning model to calculate the disease index D q The fusion analysis capability of the deep learning model can deeply explore the relationship between data and accurately judge the disease situation. Disease Index D q The calculation formula is:
[0063] D q =(W1×H air )+(W2×L)+(W3×S soil )+(W4×T)+α×T env
[0064] Among them, W1, W2, W3 and W4 are the weight coefficients of each environmental parameter data, T env is the environmental parameter correction factor, and α is the preset coefficient. The disease index calculation formula takes into account the weight coefficients of various environmental parameter data and the environmental parameter correction factor, which can quantify the impact of temperature and humidity on disease development. This refines disease assessment indicators and improves assessment accuracy.
[0065] Among them, the environmental parameter correction factor T env The calculation formula is:
[0066]
[0067] The disease recognition module also uses a convolutional neural network to extract disease features from visible light images and locates abnormal temperature areas based on infrared thermal imaging data. This combination of multiple technologies enables accurate identification of disease features.
[0068] Then, the risk warning module is used to calculate the disease index D q The numerical range of the disease index D q Compared with the preset threshold, when D q When the threshold is exceeded, a risk warning signal is generated. The dynamic risk assessment and warning mechanism can respond to disease risks in a timely manner and reduce disease losses. Specifically:
[0069] Preset thresholds D1 and D2, and D1 < D2;
[0070] When D q When ≤D1, it is marked as low risk and a low risk warning signal is generated;
[0071] When D1<D q When ≤D2, it is marked as medium risk and a medium risk warning signal is generated;
[0072] When D q When it is greater than D2, it is marked as high risk and a high-risk warning signal is generated.
[0073] Among them, the thresholds D1 and D2 are dynamically adjusted according to historical disease data and seasonal factors, specifically:
[0074] Get the number of disease occurrences N in the same period of history hist And the total number of monitoring times N total ;
[0075] According to the formula Calculate the adjusted threshold D k , where D base The default threshold is D1 or D2, and β is the adjustment coefficient. The preset threshold is dynamically adjusted based on historical disease data and seasonal factors, automatically optimizing the warning threshold based on seasonal disease occurrence. Adaptive adjustment of the warning threshold improves the pertinence and accuracy of warnings.
[0076] Finally, the early warning signal and multimodal data are sent to the user interaction module through the communication module, which displays the disease distribution heat map, risk level and prevention and control suggestions, and provides historical data query function to facilitate user use.
[0077] The present invention sets up a multimodal data acquisition module and utilizes multimodal data fusion to effectively overcome the interference of a single data source in a complex planting environment and improve the accuracy of disease identification. At the same time, it combines the environmental correction factor to quantify the impact of temperature and humidity on disease development, further refines disease assessment and improves accuracy. And based on the dynamic threshold adjustment mechanism and in conjunction with the risk warning module, it realizes adaptive adjustment of the warning threshold, shortens the warning response time, and effectively avoids the spread of diseases.
[0078] The beneficial effects of the present invention are described below by means of specific embodiments:
[0079] Example 1: Real-time identification and early warning of brown spot disease in a greenhouse environment.
[0080] Due to continuous rainy weather, the greenhouse at a tobacco plantation suffered from insufficient light and elevated humidity (reaching 85%), resulting in patches of tobacco leaves suspected of brown spot disease. The system achieved rapid identification and early warning through multimodal data fusion and dynamic threshold adjustment.
[0081] The first step is multimodal data collection.
[0082] Visible light images: The camera deployed on the top of the greenhouse collects leaf images, but the images are blurry and have low contrast due to rain.
[0083] Infrared thermal imaging: The infrared sensor detects that the temperature of the diseased area is 2.5℃ higher than that of the healthy area (preset ΔT max =5℃).
[0084] Environmental parameters: air humidity H air =85%, soil moisture S soil =70%, light intensity L=200Lux, ambient temperature T=20℃.
[0085] Spectral data: The reflectivity in the near-infrared band (900nm) was abnormally reduced, indicating that the leaf tissue structure was damaged.
[0086] Step 2: Data processing and enhancement.
[0087] Image deblurring: The adversarial network model is used through the data augmentation module to denoise the blurred visible light image and output a clear image.
[0088] Data normalization: visible light pixels were normalized to [0, 1], infrared temperature differences were converted to standard units (lesion ΔT = 2.5°C), and environmental parameters were Z-score normalized.
[0089] Anomaly detection: Use the isolation forest algorithm to remove image noise points caused by reflections, thereby improving data availability.
[0090] Step 3: Calculation and identification of disease index.
[0091] The disease recognition module extracts the texture features of red star lesions from visible light images through convolutional neural networks, and combines infrared data to locate the lesion area, which accounts for 30%.
[0092] Calculate environmental parameter correction factors
[0093] The preset weight coefficients W1=0.6, W2=0.4, W3=0.3 and W4=0.2, the preset coefficient α=0.01, and the disease index D is calculated. q =(0.6×85%)+(0.4×200)+(0.3×70%)+(0.2×20)+0.01×4857=133.29.
[0094] Step 4: Dynamic threshold adjustment and risk warning.
[0095] Basic threshold D1 = 103, D2 = 123, the number of disease occurrences in the same period in history N hist =102 times, total monitoring times N total =120 times.
[0096] Seasonal correction: The rainy season is sensitive to humidity, so the adjustment coefficient β = 0.2, calculate the adjusted threshold
[0097] Warning trigger: According to the adjusted thresholds D1 and D2, the risk level is determined to be medium risk (D1 = 120.51 < D q =133.29≤D2=143.91), the system sends the warning signal and multimodal data to the user interaction module through the communication module, and pushes the warning: a medium risk of brown spot disease is detected, and it is recommended to spray 50% carbendazim wettable powder.
[0098] In Example 1, the present invention significantly improves the accuracy in rainy environments by setting up a multimodal data acquisition module and utilizing multimodal data fusion.
[0099] Example 2: Robustness verification in a foggy mountainous environment.
[0100] The visible light image quality of a tobacco field in a certain mountainous area is poor due to heavy fog. The system ensures recognition stability through data enhancement and regional adaptive modules.
[0101] Data enhancement: After dehazing using the GAN model, the image contrast is improved, and histogram equalization strengthens the edges of the lesions.
[0102] Regional adaptation: Increase the infrared sampling frequency to 1 time / second, and adjust the preset coefficient α from 0.01 to 0.02;
[0103] Enable the rain removal algorithm and reduce the lighting weight.
[0104] Early warning effect: Calculate the disease index D according to the formula q , triggering the corresponding risk warning;
[0105] Farmers receive early warning information in real time through the user interaction module and reduce the disease spread rate after spraying and prevention.
[0106] To sum up, the present invention sets a multimodal data acquisition module and utilizes multimodal data fusion, so that the system can effectively overcome the interference of a single data source in complex planting environments such as uneven lighting, temperature and humidity fluctuations, and at the same time combines environmental correction factors to quantify the impact of temperature and humidity on disease development, thereby greatly improving the accuracy of disease identification; at the same time, the present invention is based on a dynamic threshold adjustment mechanism and cooperates with a risk warning module, so that the system can automatically optimize the warning threshold according to historical data and seasonal changes. For example, in the rainy season, the humidity-related threshold is lowered to warn of high-humidity diseases in advance, thereby greatly shortening the warning response time and effectively avoiding the spread of diseases.
[0107] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely preferred examples of the present invention and are not intended to limit the present invention. Various changes and improvements may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and improvements fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.
Claims
1. A multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases, characterized by: The system includes a multimodal data acquisition module, a data processing unit, a disease identification module, a risk warning module and a communication module; The multimodal data acquisition module is used to collect multimodal data of the tobacco planting area. The multimodal data includes visible light images, infrared thermal imaging data, environmental parameter data and leaf spectrum data. The environmental parameter data includes air humidity H air , light intensity L, soil moisture S soil and ambient temperature T; The data processing unit is used to pre-process the collected data, including image denoising, data standardization and feature extraction; The disease identification module performs fusion analysis on the pre-processed multimodal data based on the deep learning model and calculates the disease index D q , the disease index D q The calculation formula is: D q =(W1×H air )+(W2×L)+(W3×S soil )+(W4×T)+α×T env Among them, W1, W2, W3 and W4 are the weight coefficients of each environmental parameter data, T env is the environmental parameter correction factor, and α is the preset coefficient; The risk warning module is used to calculate the disease index D q Compared with the preset threshold, when D q Generate risk warning signals when thresholds are exceeded; The communication module is used to send the warning signal and multimodal data to the user interaction module.
2. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1 is characterized in that: The environmental parameter correction factor T env The calculation formula is:
3. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1 is characterized in that: The disease recognition module uses a convolutional neural network to extract disease features from visible light images and locates temperature abnormality areas in combination with infrared thermal imaging data.
4. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1, characterized in that: The risk warning module is based on the disease index D q The numerical range of risk is divided into the following levels: Preset thresholds D1 and D2, and D1 < D2; When D q When ≤D1, it is marked as low risk and a low risk warning signal is generated; When D1<D q When ≤D2, it is marked as medium risk and a medium risk warning signal is generated; When D q When it is greater than D2, it is marked as high risk and a high-risk warning signal is generated.
5. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 4 is characterized in that: The thresholds D1 and D2 are dynamically adjusted based on historical disease data and seasonal factors, specifically: Get the number of disease occurrences N in the same period of history hist And the total number of monitoring times N total ; According to the formula Calculate the adjusted threshold D k , where D base is the basic threshold D1 or D2, and β is the adjustment coefficient.
6. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1, characterized in that: The system also includes a user interaction module for displaying disease distribution heat maps, risk levels and prevention and control suggestions, and providing historical data query functions.
7. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1, characterized in that: The data processing unit includes a data enhancement module for performing denoising and enhancement processing on the visible light image, specifically: Generate adversarial network models to deblur visible light images affected by low light or haze, and output enhanced clear images. The histogram equalization algorithm is used to enhance the color contrast between the diseased area and the healthy leaves; The enhanced image is input into the disease recognition module.
8. The multimodal data mining system for intelligent identification and risk warning of tobacco leaf diseases according to claim 1, characterized in that: The data processing unit also includes a data normalization module, an anomaly detection and cleaning module, and a time series data slicing module; The data standardization module is used to uniformly convert multimodal data into a standardized format, specifically: Normalize the pixel values of visible light images, convert the temperature difference data of infrared thermal imaging data into standard temperature units, and perform Z-score standardization on environmental parameter data; The anomaly detection and cleaning module is used to identify and remove abnormal data collected by the sensor, specifically: The isolation forest algorithm detects abnormal noise in visible light images, including reflective spots and obstructions. A sliding window is used to measure the continuity of environmental parameters. If a parameter fluctuates beyond a preset threshold within 10 seconds, it is marked as abnormal and triggers data re-collection. Values in the leaf spectral data that exceed the reasonable band range are truncated or interpolated. The time series data slicing module is used to extract periodic change features based on the time series characteristics of environmental parameters, specifically: A sliding window is used to segment the temperature and humidity time series data. Statistical features, including mean, variance, peak value, and slope, are calculated in each window, and a time series feature vector is generated. The time series features are fused with image features and input into the disease recognition module.