Fruit tree pest diagnosis and early warning system based on acousto-optic feature recognition
The fruit tree disease and pest diagnosis and early warning system, which uses sound and light feature recognition, solves the problems of existing technologies that cannot process diseases in a graded manner and maintain optimal status in real time. It enables early detection, early intervention and precise control of fruit tree diseases and pests, and improves the accuracy and stability of diagnosis and early warning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-17
- Publication Date
- 2026-04-03
AI Technical Summary
Existing fruit tree disease and pest diagnosis and early warning schemes cannot formulate diagnosis and early warning plans in advance based on the internal conditions of the orchard, and cannot perform graded processing, resulting in large errors in diagnosis and early warning results and an inability to maintain the optimal state in real time.
A fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition is adopted, including data acquisition, diagnosis and processing and preset training units. Through multimodal data fusion, edge computing and cloud intelligent analysis, different levels of diagnosis and early warning schemes are formulated and trained to ensure real-time accuracy.
It has improved the efficiency of pest and disease control, enabling early detection, early intervention, and precise management. Through multimodal data fusion and edge computing, it ensures the accuracy and stability of diagnosis and early warning.
Smart Images
Figure CN121786636A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of disease and pest diagnosis and early warning technology, specifically to a fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition. Background Technology
[0002] Currently, fruit tree diseases and pests are the core issues affecting orchard yield and fruit quality. They are mainly divided into two categories: diseases (caused by fungi, bacteria, viruses, etc.) and pests (caused by insects, mites, etc.). The core of fruit tree disease and pest diagnosis and early warning is to achieve dynamic monitoring and risk warning of diseases and pests through early detection, accurate judgment, and timely prevention, combined with multiple technical means, so as to provide support for precise prevention and control in orchards.
[0003] Existing diagnostic and early warning schemes, during use, cannot formulate corresponding diagnostic and early warning schemes in advance based on the internal conditions of the orchard, nor can they classify and process the diagnostic and early warning schemes in a tiered manner. This leads to errors in the final diagnostic and early warning results. In other words, the diagnostic and early warning schemes cannot be trained, resulting in the diagnostic and early warning schemes not being in an optimal state in real time. Summary of the Invention
[0004] The purpose of this invention is to provide a fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition, so as to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition, comprising a data acquisition unit, a diagnosis and processing unit, and a preset training unit;
[0006] The data acquisition unit collects various data and establishes a data storage repository after the collection is completed. The collected data is centrally collected and then classified and labeled after processing.
[0007] The diagnostic processing unit extracts features from the collected and categorized data, performs fusion processing after feature extraction, and sets dynamic thresholds for decision adaptation.
[0008] The preset training unit builds a preprocessing training scheme during use, and sets up training scenarios to work in conjunction with the training processing scheme to train and improve the implementation scheme.
[0009] Preferably, the data acquisition unit includes a data acquisition module, a feature processing module, and a multi-processing module. The data acquisition module includes acquiring acoustic data, optical data, and environmental data, and establishing an environmental early warning threshold. When the environmental data exceeds the early warning threshold, an early warning alarm will be generated. The acoustic data is acquired by deploying a MEMS microphone array or directional pickup to collect sound signal data such as fruit tree leaf vibration and pest feeding. The optical data is acquired by integrating a hyperspectral camera or multispectral imaging equipment to capture changes in the spectral reflectance of leaves. The environmental data is acquired by synchronously monitoring meteorological parameters such as temperature, humidity, light intensity, and wind speed to dynamically adjust the early warning threshold and ensure the accuracy of the early warning threshold during operation.
[0010] Preferably, the feature processing module extracts and preprocesses features, including processing acoustic data by denoising and framing the sound signal, extracting features such as MFCC, energy, and pulse number, processing optical data by eliminating soil background interference through principal component analysis or derivative transformation, extracting vegetation index and lesion texture features, and deploying a quantized deep learning model on the edge device to achieve real-time preliminary detection and reduce cloud transmission pressure.
[0011] Preferably, the multiple processing modules perform multimodal data fusion, formulate deep learning model training, and perform dynamic early warning and decision-making. The multimodal data fusion adopts an intermediate fusion strategy, concatenating acoustic and optical features and inputting them into classifiers such as Adaboost and random forest, or dynamically allocating modal weights through a cross-attention mechanism. When training the deep learning model, a large-scale labeled dataset is used to train a multimodal large model to support the prediction of pest and disease types, severity, and insect population density. When performing dynamic early warning and decision-making, a pest and disease occurrence probability prediction model is established by combining historical data and meteorological models, and dynamic thresholds are set through a rule engine to trigger graded early warnings.
[0012] Preferably, the diagnostic processing unit includes a feature fusion module, a dynamic processing module, and a collaborative optimization module. The feature fusion module first establishes an acoustic feature database based on the differences generated when different pests feed, then establishes an optical feature database by using hyperspectral imaging technology to capture early biochemical changes, and finally uses mid-term fusion technology to stitch and fuse the data within the acoustic feature database and the optical feature database to improve the accuracy during use.
[0013] Preferably, the dynamic processing module sets a detection threshold and uses the detection threshold to determine whether there is an abnormality during the detection process. When a non-compliance is detected, an alert is issued. The specific construction steps are as follows:
[0014] (1) Phenological dynamic decision-making: Adjust the detection threshold according to the growth stage of fruit trees (such as flower bud differentiation period, fruit enlargement period);
[0015] (2) Environmental Adaptation: Modal weights can be dynamically adjusted based on parameters such as real-time light intensity, temperature and humidity.
[0016] (3) Online learning mechanism: The cloud model is updated regularly to absorb new pest and disease characteristics, and the long-term detection accuracy decay is less than three percent.
[0017] Preferably, the collaborative optimization module performs lightweight model deployment and event-driven wake-up respectively. The lightweight model deployment compresses the model parameter size to 8.2M through channel pruning and knowledge distillation, achieving 42ms / frame inference on Jetson Nano with power consumption reduced to 6.3W, making it compatible with solar power. The event-driven wake-up edge device is normally in a low-power sleep state, triggering cloud-based deep analysis only when abnormal sound or spectral changes are detected, thus reducing energy consumption.
[0018] Preferably, the preset training unit includes a training scheme module and a training scenario module. The training scheme module formulates diagnostic and early warning schemes, including a first diagnostic and early warning scheme, a second diagnostic and early warning scheme, and a third diagnostic and early warning scheme. The first diagnostic and early warning scheme has the highest processing level, the second diagnostic and early warning scheme has a medium processing level, and the third diagnostic and early warning scheme has the lowest processing level. The first, second, and third diagnostic and early warning schemes undergo complex environmental adaptation adjustments, sample learning, and cost maintenance. The complex environmental adaptation adjustments address the potential interference of factors such as wind, rain, and day-night temperature differences with the acquisition of audio-visual signals. Furthermore, the preprocessing algorithm and hardware anti-interference design are optimized. In the sample learning, the scarcity of labeled data for new pests or regional pests necessitates the exploration of transfer learning and meta-learning technologies. Cost and maintenance considerations drive the development of domestic alternatives and modular design.
[0019] Preferably, the training scenario module establishes a training scheme to train the diagnostic warning scheme, and the training scheme includes a first training scheme, a second training scheme and a third training scheme, wherein the first training scheme trains the first diagnostic warning scheme, the second training scheme trains the second diagnostic warning scheme, and the third training scheme trains the third diagnostic warning scheme.
[0020] During the training process, data generated during training is centrally collected, and three databases are established. The first database collects and stores data generated during the training of the first diagnostic and early warning scheme, the second database collects and stores data generated during the training of the second diagnostic and early warning scheme, and the third database collects data generated during the training of the third diagnostic and early warning scheme. After the data collection is completed, intelligent analysis and expert analysis are conducted. Intelligent analysis uses edge computing to quickly process real-time data, and cloud models optimize classification and prediction algorithms. Expert analysis involves inviting experts to conduct online or offline analysis, and the diagnostic training scheme is optimized after the analysis.
[0021] Compared with the prior art, the beneficial effects of the present invention are:
[0022] This invention, in its diagnostic and early warning processing, formulates different levels of diagnostic and early warning schemes to classify and process diagnostic and early warning situations. By using different levels of processing, the accuracy of diagnostic and early warning is improved. Corresponding training schemes are also formulated to train the diagnostic and early warning schemes in advance, ensuring that the schemes remain stable and accurate in real time. Through multimodal data fusion, edge computing, and cloud-based intelligent analysis, it provides a solution for early detection, early intervention, and precise treatment, which not only improves the efficiency of pest and disease control. Attached Figure Description
[0023] Figure 1 A system flowchart is provided for embodiments of the present invention. Detailed Implementation
[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0025] Please see Figure 1 The present invention provides a technical solution: a fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition, including a data acquisition unit, a diagnosis and processing unit and a preset training unit;
[0026] The data acquisition unit collects various data and establishes a data storage repository after the collection is completed. The collected data is centrally collected and then classified and labeled after processing.
[0027] The diagnostic processing unit extracts features from the collected and categorized data, performs fusion processing after feature extraction, and sets dynamic thresholds for decision adaptation.
[0028] The preset training unit builds a preprocessing training scheme during use, and sets up training scenarios to work in conjunction with the training processing scheme to train and improve the implementation scheme.
[0029] The data acquisition unit includes a data acquisition module, a feature processing module, and multiple processing modules. The data acquisition module collects acoustic data, optical data, and environmental data, and establishes an environmental early warning threshold. When the environmental data exceeds the early warning threshold, an early warning alarm is generated. The acoustic data is collected by deploying a MEMS microphone array or directional pickup to collect sound signal data such as fruit tree leaf vibration and insect feeding. The optical data is captured by integrating a hyperspectral camera (such as a pushbroom camera covering the 400-1000nm band) or multispectral imaging equipment to capture changes in the spectral reflectance of leaves. The environmental data is collected by synchronously monitoring meteorological parameters such as temperature, humidity, light intensity, and wind speed to dynamically adjust the early warning threshold and ensure the accuracy of the early warning threshold during operation.
[0030] The feature processing module extracts and preprocesses features. Acoustic data is processed by denoising the sound signal (e.g., wavelet transform) and framing and windowing, and features such as MFCC (Mel frequency cepstral coefficients), energy, and pulse number are extracted. Optical data is processed by eliminating soil background interference through principal component analysis (PCA) or derivative transform, and vegetation indices (e.g., NDVI, PRI) and lesion texture features are extracted. The quantized deep learning model is then deployed on the edge device to achieve real-time preliminary detection and reduce cloud transmission pressure.
[0031] The multiple processing modules perform multimodal data fusion, train deep learning models, and conduct dynamic early warning and decision-making. The multimodal data fusion adopts an intermediate fusion strategy, concatenating acoustic and optical features and inputting them into classifiers such as Adaboost and random forest, or dynamically allocating modal weights through the Cross-Attention mechanism. When training the deep learning model, a large-scale labeled dataset (such as an orchard pest and disease image library and an insect sound database) is used to train a large multimodal model (such as ERNIE-4.5-VL) to support the prediction of pest and disease types, severity, and insect population density. When conducting dynamic early warning and decision-making, a pest and disease occurrence probability prediction model is established by combining historical data and meteorological models. Dynamic thresholds are set through a rule engine (such as adjusting detection confidence based on phenological differences) to trigger graded early warnings (such as mobile phone SMS and APP push).
[0032] By building a visual interface using big data and virtual technology, it provides orchard maps, real-time monitoring data, and heat maps of pest and disease distribution, as well as prevention and control suggestions (such as pesticide selection and optimal application time). It also develops a remote control solution that supports remote adjustment of sensor parameters and starting and stopping of equipment via IoT protocols, or linkage with drones for precise pesticide application.
[0033] The diagnostic processing unit includes a feature fusion module, a dynamic processing module, and a collaborative optimization module. The feature fusion module first establishes an acoustic feature database based on the differences generated when different pests feed. It then establishes an optical feature database by using hyperspectral imaging technology to capture early biochemical changes. Finally, it uses mid-term fusion (feature splicing) technology to splice and fuse the data within the acoustic and optical feature databases to improve the accuracy during use.
[0034] The dynamic processing module sets detection thresholds and uses these thresholds to determine whether there are any abnormalities during the detection process. When a non-compliance is detected, an alert is issued. The specific construction steps are as follows:
[0035] (1) Phenological dynamic decision-making: Adjust the detection threshold according to the growth stage of fruit trees (such as flower bud differentiation period and fruit enlargement period). For example, the detection threshold for diseases and pests during the wheat seedling stage is set to 0.45, and then reduced to 0.38 during the grain filling period to reduce false detections caused by morphological changes.
[0036] (2) Environmental Adaptation: The modal weights can be dynamically adjusted based on parameters such as real-time light intensity, temperature and humidity. For example, the thermal infrared feature weight is increased to 0.7 in low-light scenarios, which enhances the robustness of disease identification.
[0037] (3) Online learning mechanism: The cloud model is updated regularly to absorb new pest and disease characteristics (such as new pest invasions), and the long-term detection accuracy decay is less than three percent.
[0038] The collaborative optimization module performs lightweight model deployment and event-driven wake-up respectively. The lightweight model deployment compresses the model parameter size to 8.2M through channel pruning and knowledge distillation, achieving 42ms / frame inference on Jetson Nano with power consumption reduced to 6.3W, making it compatible with solar power. The event-driven wake-up edge device is normally in a low-power sleep state, triggering cloud-based deep analysis only when abnormal sound or spectral changes are detected, thus reducing energy consumption.
[0039] The preset training unit includes a training scheme module and a training scenario module. The training scheme module formulates diagnostic and early warning schemes, including a first diagnostic and early warning scheme, a second diagnostic and early warning scheme, and a third diagnostic and early warning scheme. The first diagnostic and early warning scheme has the highest processing level, the second diagnostic and early warning scheme has a medium processing level, and the third diagnostic and early warning scheme has the lowest processing level. The first, second, and third diagnostic and early warning schemes undergo complex environmental adaptation adjustments, sample learning, and cost maintenance. The complex environmental adaptation adjustments take into account factors such as wind, rain, and day-night temperature differences that may interfere with the acquisition of sound and light signals, and further optimize the preprocessing algorithm and hardware anti-interference design. Among the sample learning, the scarcity of labeled data for new pests or regional pests necessitates the exploration of transfer learning and meta-learning technologies. Cost and maintenance drive the development of domestic substitution and modular design.
[0040] Once the risk warning levels are determined, warning information can be sent via an app or SMS, along with prevention and control recommendations.
[0041] The training scenario module establishes a training scheme to train the diagnostic warning scheme, and the training scheme includes a first training scheme, a second training scheme and a third training scheme, wherein the first training scheme trains the first diagnostic warning scheme, the second training scheme trains the second diagnostic warning scheme, and the third training scheme trains the third diagnostic warning scheme.
[0042] During the training process, data generated during training is centrally collected, and three databases are established. The first database collects and stores data generated during the training of the first diagnostic and early warning scheme, the second database collects and stores data generated during the training of the second diagnostic and early warning scheme, and the third database collects data generated during the training of the third diagnostic and early warning scheme. After the data collection is completed, intelligent analysis and expert analysis are conducted. Intelligent analysis uses edge computing to quickly process real-time data, and cloud models optimize classification and prediction algorithms. Expert analysis involves inviting experts to conduct online or offline analysis, and the diagnostic training scheme is optimized after the analysis.
[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition, characterized in that: It includes a data acquisition unit, a diagnostic processing unit, and a pre-set training unit; The data acquisition unit collects various data and establishes a data storage repository after the collection is completed. The collected data is centrally collected and then classified and labeled after processing. The diagnostic processing unit extracts features from the collected and categorized data, performs fusion processing after feature extraction, and sets dynamic thresholds for decision adaptation. The preset training unit builds a preprocessing training scheme during use, and sets up training scenarios to work in conjunction with the training processing scheme to train and improve the implementation scheme.
2. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 1, characterized in that: The data acquisition unit includes a data acquisition module, a feature processing module, and a multi-processing module. The data acquisition module collects acoustic data, optical data, and environmental data, and establishes an environmental early warning threshold. When the environmental data exceeds the early warning threshold, an early warning alarm is generated. The acoustic data is collected by deploying a MEMS microphone array or directional pickup to collect sound signal data such as fruit tree leaf vibration and insect feeding. The optical data is captured by integrating a hyperspectral camera or multispectral imaging equipment to capture changes in the spectral reflectance of leaves. The environmental data is collected by synchronously monitoring meteorological parameters such as temperature, humidity, light intensity, and wind speed to dynamically adjust the early warning threshold and ensure the accuracy of the early warning threshold during operation.
3. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 2, characterized in that: The feature processing module extracts and preprocesses features, including acoustic data processing, noise reduction and frame windowing of sound signals, extraction of features such as MFCC, energy and pulse number, optical data processing, elimination of soil background interference through principal component analysis or derivative transformation, extraction of vegetation index and lesion texture features, and deployment of quantized deep learning models on edge devices to achieve real-time preliminary detection and reduce cloud transmission pressure.
4. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 3, characterized in that: The multiple processing modules perform multimodal data fusion, train deep learning models, and conduct dynamic early warning and decision-making. The multimodal data fusion adopts an intermediate fusion strategy, concatenating acoustic and optical features and inputting them into classifiers such as Adaboost and random forest, or dynamically allocating modal weights through a cross-attention mechanism. When training the deep learning model, a large-scale labeled dataset is used to train a multimodal model, supporting the prediction of pest and disease types, severity, and insect population density. When conducting dynamic early warning and decision-making, a pest and disease occurrence probability prediction model is established by combining historical data and meteorological models, and dynamic thresholds are set through a rule engine to trigger graded early warnings.
5. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 4, characterized in that: The diagnostic processing unit includes a feature fusion module, a dynamic processing module, and a collaborative optimization module. The feature fusion module first establishes an acoustic feature database based on the differences produced when different pests feed. It then establishes an optical feature database by using hyperspectral imaging technology to capture early biochemical changes. Finally, it uses mid-term fusion technology to stitch and fuse the data within the acoustic and optical feature databases to improve the accuracy during use.
6. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 5, characterized in that: The dynamic processing module sets detection thresholds and uses these thresholds to determine whether there are any abnormalities during the detection process. When a non-compliance is detected, an alert is issued. The specific construction steps are as follows: (1) Phenological dynamic decision-making: Adjust the detection threshold according to the growth stage of fruit trees (such as flower bud differentiation period, fruit enlargement period); (2) Environmental Adaptation: Modal weights can be dynamically adjusted based on parameters such as real-time light intensity, temperature and humidity. (3) Online learning mechanism: The cloud model is updated regularly to absorb new pest and disease characteristics, and the long-term detection accuracy decay is less than three percent.
7. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 6, characterized in that: The collaborative optimization module performs lightweight model deployment and event-driven wake-up respectively. The lightweight model deployment compresses the model parameter size to 8.2M through channel pruning and knowledge distillation, achieving 42ms / frame inference on Jetson Nano with power consumption reduced to 6.3W, making it compatible with solar power. The event-driven wake-up edge device is normally in a low-power sleep state, triggering cloud-based deep analysis only when abnormal sound or spectral changes are detected, thus reducing energy consumption.
8. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 7, characterized in that: The preset training unit includes a training scheme module and a training scenario module. The training scheme module formulates diagnostic and early warning schemes, including a first diagnostic and early warning scheme, a second diagnostic and early warning scheme, and a third diagnostic and early warning scheme. The first diagnostic and early warning scheme has the highest processing level, the second diagnostic and early warning scheme has a medium processing level, and the third diagnostic and early warning scheme has the lowest processing level. The first, second, and third diagnostic and early warning schemes undergo complex environmental adaptation adjustments, sample learning, and cost maintenance. The complex environmental adaptation adjustments take into account factors such as wind, rain, and day-night temperature differences that may interfere with the acquisition of audio and visual signals, and further optimize the preprocessing algorithm and hardware anti-interference design. Among the sample learning, the scarcity of labeled data for new pests or regional pests necessitates the exploration of transfer learning and meta-learning technologies. Cost and maintenance drive the development of domestic substitution and modular design.
9. The fruit tree disease and pest diagnosis and early warning system based on acoustic and optical feature recognition according to claim 8, characterized in that: The training scenario module establishes a training scheme to train the diagnostic warning scheme, and the training scheme includes a first training scheme, a second training scheme and a third training scheme, wherein the first training scheme trains the first diagnostic warning scheme, the second training scheme trains the second diagnostic warning scheme, and the third training scheme trains the third diagnostic warning scheme. During the training process, data generated during training is centrally collected, and three databases are established. The first database collects and stores data generated during the training of the first diagnostic and early warning scheme, the second database collects and stores data generated during the training of the second diagnostic and early warning scheme, and the third database collects data generated during the training of the third diagnostic and early warning scheme. After the data collection is completed, intelligent analysis and expert analysis are conducted. Intelligent analysis uses edge computing to quickly process real-time data, and cloud models optimize classification and prediction algorithms. Expert analysis involves inviting experts to conduct online or offline analysis, and the diagnostic training scheme is optimized after the analysis.