Insect situation detection method and early warning system based on visual intelligence and large model

Abstract

The application discloses a pest situation detection method and early warning system based on visual intelligence and a large model, and belongs to the technical field of intelligent agriculture. The application solves the problems of low efficiency, incomplete data collection and subjectivity in the prior art. The application can obtain multi-dimensional data of pest situation through a multispectral collection device integrating an RGB camera, a thermal imager and a laser radar, can effectively make up for the deficiency of a single visual perception method, makes pest situation detection more comprehensive and accurate, inputs the collected multispectral data into a multimodal large model architecture, can provide multi-dimensional information of pest damage, and helps precise prevention and control decision-making. The pest situation detection result based on the multimodal large model architecture quantifies the risk level of pest outbreak through a three-level early warning system, so that agricultural managers or relevant personnel can timely and reasonably allocate resources and formulate corresponding prevention and control strategies according to the risk level, thereby effectively reducing the loss that may be caused by pests.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent agricultural technology, in particular to a pest situation detection method and early warning system based on visual intelligence and large models. BACKGROUND

[0002] Traditional pest monitoring methods rely on manual field surveys, which have low efficiency, incomplete data collection, strong subjectivity and other drawbacks, and cannot meet the needs of modern large-area pest real-time monitoring and precise prevention and control. Therefore, the existing needs are not met, and for this we propose a pest situation detection method and early warning system based on visual intelligence and large models. SUMMARY

[0003] The purpose of the present application is to provide a pest situation detection method and early warning system based on visual intelligence and large models, which can effectively compensate for the shortcomings of single visual perception means through multi-spectral data acquisition methods, making pest detection more comprehensive and accurate. The collected multi-spectral data is input into a multi-modal large model architecture, which can provide multi-dimensional information on pests and assist in precise prevention and control decisions. The pest situation detection results based on the multi-modal large model architecture quantify the risk level of pest outbreaks through a three-level early warning system, providing a scientific basis for pest control and solving the problems raised in the above background technology.

[0004] To achieve the above purpose, the present application provides the following technical solution: a pest situation detection method based on visual intelligence and large models, which is a detection method based on the integration of multiple visual perception and large models for perceiving, analyzing and warning of pest multi-dimensional information;

[0005] The pest situation detection method comprises:

[0006] The range and shape of the monitoring area are determined, and a multi-spectral acquisition device integrating an RGB camera, a thermal imager and a laser radar is used to collect multi-spectral data of the pest situation and perform preprocessing, wherein the multi-spectral data includes RGB images, thermal infrared images and laser radar point cloud data;

[0007] After determining the range and shape of the monitoring area, the ambient light intensity parameter is compared with the preset threshold value for ratio processing, and the infrared fill light mode is automatically switched to;

[0008] The preprocessed multi-spectral data is input into a multi-modal large model architecture, which sequentially passes through a feature layer fusion architecture for spectral feature interaction and fusion, an LSTM-Transformer hybrid architecture of a spatio-temporal memory network to capture pest evolution rules, and an optimization processing of a knowledge distillation mechanism to obtain pest situation detection results, including the location, type, severity and development trend of the pest;

[0009] Based on the pest situation detection results, the risk level of pest outbreak is quantified through a three-level early warning system.

[0010] Further, the pre-processed multi-spectral data is input into a multi-modal large model architecture, including the following steps:

[0011] The multi-spectral data enters a feature layer fusion architecture, which interacts and fuses the features of data from different spectral sources, specifically:

[0012] The color texture features in the RGB image are correlated with the temperature distribution features in the thermal infrared image, identifying the connection between abnormal color changes and temperature rises in the pest area;

[0013] The pest shape and spatial distribution information provided by the laser radar is fused with the RGB image and the thermal infrared image to form a pest situation feature representation;

[0014] The fused multi-spectral data enters the LSTM-Transformer hybrid architecture of the spatio-temporal memory network, capturing the evolution rules of the pest in the time and space dimensions, specifically:

[0015] LSTM is used to process time series information, recording the evolution state of the pest over time through the internal forget gate, input gate and output gate structure, capturing the occurrence, development and possible outbreak trend of the pest;

[0016] At the same time, the Transformer architecture part further enhances the processing capability of spatial features, analyzing the distribution pattern of the pest from different spatial perspectives;

[0017] After the above feature interaction fusion and spatio-temporal evolution rule capturing operations, a knowledge distillation mechanism is introduced to integrate historical data in the pest situation detection field into the model, guiding the model to learn the pest situation feature representation through knowledge distillation, and finally obtaining the pest situation detection result.

[0018] Further, the knowledge distillation mechanism is implemented in the following ways:

[0019] The rules summarized from the historical data in the pest situation detection field are used as prior knowledge;

[0020] Through knowledge distillation, these prior knowledge is integrated into the learning process of the model, guiding the model to better learn the pest situation feature representation, wherein the model includes a teacher model and a student model, specifically:

[0021] The teacher model is trained based on a large amount of historical data to provide high-level representation of the pest situation features;

[0022] The student model learns the key features of pest situation detection by learning the output and intermediate features of the teacher model;

[0023] After optimization by the knowledge distillation mechanism, the final pest detection results are obtained, including the location, type, severity, and development trend of the pests.

[0024] Further, collect multispectral data on insect infestation, including the following steps:

[0025] Based on the actual needs of insect monitoring, GIS data is used in conjunction with topographic and vegetation distribution information to draw the boundary of the monitoring area on the map and determine the scope and shape of the monitoring area.

[0026] A multispectral acquisition device is set up within the defined monitoring area. The device autonomously scans and collects data from the area, ultimately gathering multispectral data on insect infestation and performing preprocessing, including:

[0027] Start the RGB camera to acquire RGB images of the monitored area;

[0028] Simultaneously activate the thermal imager to acquire thermal infrared images of the monitored area;

[0029] The lidar is activated to scan the monitoring area and acquire lidar point cloud data within the monitoring area;

[0030] After determining the scope and shape of the monitoring area, the ambient light intensity of the determined monitoring area is monitored in real time by an ambient light sensor. The ambient light intensity parameters are obtained and compared with a preset threshold. When the ambient light intensity parameters are lower than the preset threshold, the infrared supplementary lighting mode is automatically switched; otherwise, the visible light supplementary lighting mode is used.

[0031] Furthermore, it automatically switches to infrared fill light mode, including:

[0032] When the ambient light intensity parameter is lower than a preset threshold, the ambient light intensity parameter is compared with the preset threshold to obtain the light intensity ratio coefficient.

[0033] When the ambient light intensity parameter is lower than a preset threshold, a preset time period is retrieved from the database, wherein the value of the preset time period is in the range of 30min-60min;

[0034] According to the time length corresponding to the preset time period, the ambient light illuminance parameter within the preset time period before the time when the ambient light illuminance parameter is lower than the preset threshold is extracted and used as the reference ambient light illuminance parameter.

[0035] The illumination fluctuation coefficient is obtained based on the reference ambient illuminance parameters;

[0036] Compare the illuminance ratio coefficient with the illuminance fluctuation coefficient;

[0037] When the illumination ratio coefficient does not exceed the illumination fluctuation coefficient, a preset reference compensation intensity coefficient is called; wherein, the preset reference compensation intensity coefficient is in the range of 1.14-1.26;

[0038] Light is compensated according to the preset reference compensation intensity coefficient in the infrared light compensation mode; wherein, the corresponding first target light compensation amount after light compensation is K*L; wherein, K represents the preset reference compensation intensity coefficient; L represents the current ambient illumination parameter;

[0039] When the illumination ratio coefficient exceeds the illumination fluctuation coefficient, a second target light compensation amount is obtained using the illumination fluctuation coefficient and the illumination ratio coefficient;

[0040] Light is compensated according to the second target light compensation amount in the infrared light compensation mode.

[0041] Further, the collected pest situation multispectral data is preprocessed, including:

[0042] The RGB image, thermal infrared image and laser radar point cloud data are time-stamped matched to ensure that each set of multispectral data corresponds to the corresponding environmental conditions;

[0043] The RGB image is preprocessed, including removing noise interference in the RGB image using Gaussian filtering and median filtering algorithm, and improving the contrast and clarity of the RGB image through histogram equalization method;

[0044] The thermal infrared image is radiometrically corrected and temperature corrected, and at the same time, the thermal infrared image is gray scale stretched;

[0045] The laser radar point cloud data is filtered and denoised to remove noise points caused by environmental interference;

[0046] After the above preprocessing operations are completed, the multispectral data is integrated into a unified data set according to the preset data format and channel order, and then input to the multi-modal large model architecture.

[0047] Further, based on the pest situation detection result, the risk level of pest outbreak is quantified through a three-level early warning system, including the following steps:

[0048] The pest situation detection result is analyzed, specifically:

[0049] Information is extracted from the pest situation detection result, including pest type, severity and development trend prediction data;

[0050] According to the pest type, severity and development trend factors, the corresponding weight coefficients are set, i.e. the pest type weight is set to 30%, the severity weight is set to 50%, and the development trend weight is set to 20%;

[0051] a risk score is calculated, and a pest outbreak risk level is determined according to the risk score.

[0052] Further, the risk score is calculated according to the formula:

[0053] Risk score = (pest type value x 30%) + (severity level value x 50%) + (development trend prediction value x 20%).

[0054] Further, the pest outbreak risk level is divided into first-level warning, second-level warning and third-level warning, and a warning threshold is set for each pest outbreak risk level, specifically:

[0055] The first-level warning threshold is 80-100 points, when the risk score reaches the first-level warning threshold, a red warning signal is generated, and first-level warning information is immediately sent to the management department and management personnel;

[0056] The second-level warning threshold is 60-79 points, when the risk score reaches the second-level warning threshold, a yellow warning signal is generated, second-level warning information is sent and attention is reminded and corresponding prevention and control measures are taken;

[0057] The third-level warning threshold is 0-59 points, when the risk score reaches the third-level warning threshold, a green warning signal is generated, third-level warning information is sent and management personnel are prompted to continue observation.

[0058] The pest situation warning system based on visual intelligence and large models is used to realize a pest situation detection method based on visual intelligence and large models, comprising:

[0059] The data acquisition module is configured to integrate an RGB camera, a thermal imager and a laser radar to collect multispectral data including RGB images, thermal infrared images and laser radar point cloud data, and to preprocess the collected multispectral data;

[0060] The data analysis module is configured to input the preprocessed multispectral data into a multimodal large model architecture, which includes a feature layer fusion architecture, an LSTM-Transformer hybrid architecture of a spatiotemporal memory network and a knowledge distillation mechanism, wherein:

[0061] The feature layer fusion architecture is used to interactively fuse the features of data from different spectral sources;

[0062] The LSTM-Transformer hybrid architecture of the spatiotemporal memory network is used to capture the evolution law of the pest in the time and space dimensions;

[0063] The knowledge distillation mechanism is used for optimization processing;

[0064] The hierarchical early warning module is configured to quantize the pest outbreak risk level and generate a corresponding early warning signal through a three-level early warning system according to the pest detection result output by the multi-modal large model architecture.

[0065] Compared with the prior art, the present application has the following advantages:

[0066] The multi-spectral acquisition device integrating an RGB camera, a thermal imager and a laser radar can obtain multi-dimensional data of the pest situation, and the multi-spectral data acquisition method can effectively make up for the shortcomings of a single visual perception method, so that the pest detection is more comprehensive and accurate. The multi-spectral data collected is input into the multi-modal large model architecture, the feature layer fusion architecture in the multi-modal large model architecture can realize spectral feature interaction and fusion, fully excavate effective information in different spectral data, form a more comprehensive and accurate pest situation feature description, and help improve the accuracy of pest detection. The LSTM-Transformer hybrid architecture of the spatio-temporal memory network can capture the evolution law of the pest, provide a strong basis for taking preventive measures in advance, and finally obtain the pest detection result by introducing the knowledge distillation mechanism. The three-level early warning system constructed based on the pest detection result can quantize the pest outbreak risk level, make the pest risk assessment more scientific and systematic, and facilitate the agricultural managers or relevant personnel to timely and reasonably allocate resources and formulate corresponding prevention and control strategies, thereby effectively reducing the losses caused by the pest. BRIEF DESCRIPTION OF DRAWINGS

[0067] Fig. 1 The flowchart of the pest situation detection method based on visual intelligence and large model of the present application;

[0068] Fig. 2 The structure diagram of the pest situation early warning system based on visual intelligence and large model of the present application. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0070] To solve the technical problems that the existing pest monitoring methods mostly rely on manual field investigation, have low efficiency, incomplete data collection, strong subjectivity and many other disadvantages, and are difficult to meet the needs of modern large-area pest real-time monitoring and precise prevention and control, please refer to Figs. 1-2 The present embodiment provides the following technical solutions:

[0071] The insect situation detection method based on visual intelligence and a large model is a detection method based on integrated multi-dimensional visual perception and a large model for perceiving, analyzing, and warning insect situation multi-dimensional information.

[0072] The insect situation detection method comprises:

[0073] The range and shape of the monitoring area are determined, multi-spectral data of the insect situation are collected by using a multi-spectral collection device integrating an RGB camera, a thermal imager, and a laser radar, and the multi-spectral data are preprocessed, wherein the multi-spectral data comprise RGB images, thermal infrared images, and laser radar point cloud data.

[0074] After the range and shape of the monitoring area are determined, the infrared fill light mode is automatically switched by performing ratio processing on the ambient light intensity parameter and a preset threshold value.

[0075] The preprocessed multi-spectral data are input into a multi-modal large model architecture, sequentially pass through a feature layer fusion architecture for spectral feature interaction fusion, an LSTM-Transformer hybrid architecture of a space-time memory network for capturing insect evolution rules, and an optimization processing of a knowledge distillation mechanism, and finally obtain insect situation detection results, including the position, type, severity, and development trend of the insect pests.

[0076] Based on the insect situation detection results, the insect outbreak risk level is quantified through a three-level early warning system.

[0077] The technical effects of the above technical solution are as follows: the multi-spectral collection device integrating an RGB camera, a thermal imager, and a laser radar can obtain multi-spectral data including RGB images, thermal infrared images, and laser radar point cloud data. This multi-dimensional data collection method can more comprehensively reflect various types of insect situation information compared with single visual data, thereby providing a richer and more accurate data basis for subsequent insect situation analysis and helping to improve the accuracy of insect situation detection. The preprocessed multi-spectral data are input into a multi-modal large model architecture, and spectral feature interaction fusion is realized through a feature layer fusion architecture, so that the features in different modal data can be mutually complementary and correlated to mine deeper insect situation feature information. An LSTM-Transformer hybrid architecture of a space-time memory network can effectively capture the evolution rules of insect pests in the time and space dimensions, and the model is optimized through a knowledge distillation mechanism. When facing insect situation data in different regions, different crop types, and different environmental conditions, the model can better adapt and accurately detect. Finally, insect situation detection results are obtained. Based on these detection results, the insect outbreak risk level is quantified through a three-level early warning system to timely issue warning signals. This hierarchical early warning mechanism can take appropriate prevention and control measures according to the urgency of the insect pests, realize precise prevention and control, effectively reduce the losses caused by insect pests to crops, and improve the sustainability and economic benefits of agricultural production.

[0078] The pre-processed multi-spectral data is input into a multi-modal large model architecture, including the following steps:

[0079] The multi-spectral data enters a feature layer fusion architecture, which interacts and fuses the features of data from different spectral sources, specifically:

[0080] The color texture features in the RGB image are correlated with the temperature distribution features in the thermal infrared image, identifying the connection between abnormal color changes and temperature rises in the pest area;

[0081] The pest morphology and spatial distribution information provided by the laser radar are fused with the RGB image and the thermal infrared image to form a pest situation feature representation;

[0082] The fused multi-spectral data enters the LSTM-Transformer hybrid architecture of the spatio-temporal memory network, capturing the evolution rules of the pest in the time and space dimensions, specifically:

[0083] LSTM is used to process time series information, recording the evolution state of the pest over time through the internal forget gate, input gate and output gate structure, capturing the occurrence, development and possible outbreak trend of the pest;

[0084] At the same time, the Transformer architecture part further enhances the processing capability of spatial features, analyzing the distribution pattern of the pest from different spatial perspectives;

[0085] After the above feature interaction fusion and spatio-temporal evolution rule capturing operations, a knowledge distillation mechanism is introduced to integrate historical data in the pest situation detection field into the model, guiding the model to learn the pest situation feature representation through knowledge distillation, and finally obtaining the pest situation detection result;

[0086] Among them, the knowledge distillation mechanism is implemented in the following ways:

[0087] The rules summarized from the historical data in the pest situation detection field are used as prior knowledge;

[0088] Through knowledge distillation, these prior knowledge is integrated into the learning process of the model, guiding the model to better learn the pest situation feature representation, where the model includes a teacher model and a student model, specifically:

[0089] The teacher model is trained based on a large amount of historical data, providing a high-level representation of the pest situation features;

[0090] The student model learns the key features of pest detection by learning the output and intermediate features of the teacher model;

[0091] After the optimization processing of the knowledge distillation mechanism, the pest detection result is finally obtained, including the position, type, severity and development trend of the pest.

[0092] The technical effects of the above technical scheme are: through the feature layer fusion architecture, the color texture features of the RGB image and the temperature distribution features of the thermal infrared image are correlated with each other, the relationship between the abnormal color change and the temperature rise of the pest area is accurately identified, at the same time, the pest morphology and spatial distribution information provided by the laser radar are fused with the above two, forming a complete and rich pest feature representation, the interactive fusion of such multi-dimensional features can more accurately identify the pest area compared with single spectral feature analysis, effectively improving the accuracy and reliability of the pest detection, reducing the false detection rate and the missed detection rate, after the fusion of the multispectral data, the multispectral data enter the LSTM-Transformer hybrid architecture of the spatio-temporal memory network, realizing the accurate capture of the spatio-temporal evolution law of the pest, on the one hand, the LSTM can effectively process the time sequence information by means of its forget gate, input gate and output gate structure, and record the whole process evolution state of the pest from occurrence to development and possible outbreak in detail, providing a strong basis for predicting the future development trend of the pest, on the other hand, the Transformer architecture enhances the processing capability of spatial features, and analyzes the distribution mode of the pest from different spatial perspectives, which helps to more comprehensively understand the diffusion and spread of the pest in the farmland and other areas, thereby providing more accurate spatial information for formulating targeted prevention and control measures, after the above series of processing steps, the knowledge distillation mechanism is introduced, the historical data in the pest detection field are integrated into the model, so that the model can learn more representative and general pest feature representation by referring to historical experience, and finally the pest detection result is obtained, which provides comprehensive pest insight for agricultural managers, so that they can timely and accurately understand the pest situation in the farmland and other areas, and thus take corresponding prevention and control measures to realize precise prevention and control.

[0093] The multispectral data of the pest are collected, including the following steps:

[0094] According to the actual needs of pest monitoring, the boundaries of the monitoring area are drawn on the map by using GIS data combined with topography and vegetation distribution information, and the range and shape of the monitoring area are determined;

[0095] The multispectral collection device is arranged in the determined monitoring area, and the monitoring area is autonomously scanned and data is collected by the multispectral collection device, finally the multispectral data of the pest are collected and preprocessed, including:

[0096] The RGB camera is turned on to obtain the RGB image of the monitoring area;

[0097] The thermal imager is started synchronously to obtain the thermal infrared image of the monitoring area;

[0098] The laser radar is started to scan the monitoring area to obtain laser radar point cloud data in the monitoring area.

[0099] The technical effects of the above technical solution are that the GIS data is combined with the terrain and vegetation distribution information to accurately draw the boundary of the monitoring area on the map, the range and shape are determined, the selection of the monitoring area is more scientific and reasonable, closely matches the actual needs of pest monitoring, the pertinence and effectiveness of data collection are improved, the monitoring area is synchronously and autonomously scanned and data is collected by the multispectral collection device, the synchronous collection mode ensures the consistency of different spectral data in time and space, avoids the problems of data mismatch and incoherence caused by time difference, provides high-quality and reliable basic data for subsequent data fusion and comprehensive analysis, and the collected multispectral data comprehensively reflects various characteristics of the pest situation from multiple dimensions, compared with single spectral data, can provide richer information, enables the pest detection model to more comprehensively learn and understand the complex characteristics of the pest situation, and thus improves the accuracy, fineness and reliability of pest detection, and provides a more solid data foundation for subsequent pest analysis, prediction and early warning.

[0100] After the range and shape of the monitoring area are determined, the ambient illuminance of the determined monitoring area is monitored in real time by the ambient illuminance sensor, the ambient illuminance parameter is obtained and compared with the preset threshold, when the ambient illuminance parameter is lower than the preset threshold, the infrared fill light mode is automatically switched to, and otherwise the visible light fill light mode is used;

[0101] The infrared fill light mode is enabled when the illuminance is insufficient, which can ensure that the collection quality of the thermal infrared image is not affected by the low light condition, and ensure that the thermal infrared image can accurately reflect the temperature distribution characteristics of the pest area, and the visible light fill light mode is used when the light is sufficient, which can enhance the definition and color restoration of the RGB image, and make the color texture features in the RGB image more distinct and accurate.

[0102] The technical effects of the above technical solution are that the ambient illuminance is monitored in real time by the ambient illuminance sensor, and compared with the preset threshold, and the fill light mode is automatically switched according to the monitoring result, this adaptive fill light mode switching mechanism effectively avoids the problem of image quality degradation caused by changes in ambient illuminance, improves the stability and reliability of multispectral data collection under different light conditions, and provides a better and stable data foundation for subsequent pest detection analysis.

[0103] Specifically, the infrared fill light mode is automatically switched to, including:

[0104] When the ambient illuminance parameter is lower than the preset threshold, the ambient illuminance parameter is processed by ratio with the preset threshold to obtain an illuminance ratio coefficient;

[0105] retrieve a preset time period from the database when the ambient light illumination parameter is lower than the preset threshold value, wherein the preset time period ranges from 30 min to 60 min;

[0106] cut the ambient light illumination parameter in the preset time period corresponding to the time length before the time when the ambient light illumination parameter is lower than the preset threshold value according to the time length corresponding to the preset time period as a reference ambient light illumination parameter;

[0107] obtain an illumination fluctuation coefficient according to the reference ambient light illumination parameter;

[0108] wherein the illumination fluctuation coefficient is obtained by the following formula:

[0109]

[0110] wherein S represents the illumination fluctuation coefficient; n represents the number of the reference ambient light illumination parameters; L i and L i+1 represent the parameter values corresponding to the i th reference ambient light illumination parameter and the i+1 th reference ambient light illumination parameter respectively; L th represents the value corresponding to the preset threshold value; L σ represents the standard deviation of the illumination corresponding to the n reference ambient light illumination parameters; specifically, max(L i+1 ,L i ) takes the larger value of the adjacent two reference ambient light illumination parameters. In the illumination change analysis, the larger value reflects the relatively stronger state of the illumination intensity at the adjacent time.(L th -L i ) calculates the difference value of the i th reference ambient light illumination parameter and the preset threshold value. When the illumination is lower than the threshold value, this difference value measures the degree of difference between the current illumination and the threshold value, and the larger the difference value is, the weaker the current illumination is and the farther the current illumination is from the threshold value. max(L i+1 ,L i )*(L th -L i ) takes the larger value of the adjacent two reference ambient light illumination parameters. In the illumination change analysis, the larger value reflects the relatively stronger state of the illumination intensity at the adjacent time.(L i+1 -L i | takes the absolute value of the difference value of the adjacent two illumination parameters, reflecting the change amplitude of the adjacent illumination, and the larger the change amplitude is, the larger the value is.[max(L i+1 ,L i )+|L i+1 -L i |] takes the larger value of the adjacent two reference ambient light illumination parameters. In the illumination change analysis, the larger value reflects the relatively stronger state of the illumination intensity at the adjacent time.(L σ +1) in L σis a standard deviation of the illuminance, which measures the dispersion degree of the reference ambient illuminance parameter, and the greater the standard deviation, the more intense the illumination fluctuation. Adding 1 is to avoid the case where the standard deviation is 0, and also to smooth the calculation results to some extent. The first to the nth reference ambient illuminance parameters are summed according to the above formula, and then averaged by 1 / n, to obtain the comprehensive illumination fluctuation measurement value in the entire reference time period. Through the averaging operation, the excessive influence of individual data fluctuation on the result is eliminated, and a relatively stable illumination fluctuation coefficient is obtained. The formula comprehensively considers the size relationship of adjacent illuminations, the difference between the illumination and the threshold, and the change amplitude of adjacent illuminations, and can capture the change characteristics of the illumination in the time sequence in detail, which can more comprehensively reflect the illumination fluctuation than simply considering the illumination intensity value. By combining the standard deviation of the illuminance, the calculated illumination fluctuation coefficient can adapt to different illumination fluctuation levels. Through the summation and averaging calculation method, the excessive interference of individual illumination outliers on the overall fluctuation measurement is avoided, and a relatively stable and reliable illumination fluctuation coefficient is obtained, which has good stability and accuracy in measuring the illumination fluctuation performance index.

[0111] comparing the illuminance ratio coefficient with the illumination fluctuation coefficient;

[0112] when the illuminance ratio coefficient does not exceed the illumination fluctuation coefficient, a preset reference compensation intensity coefficient is retrieved; wherein the value range of the preset reference compensation intensity coefficient is 1.14-1.26;

[0113] using the infrared light compensation mode to compensate light according to the preset reference compensation intensity coefficient; wherein the corresponding first target compensation light amount after compensation is K*L; wherein K represents the preset reference compensation intensity coefficient; and L represents the current ambient illuminance parameter;

[0114] when the illuminance ratio coefficient exceeds the illumination fluctuation coefficient, a second target compensation light amount is obtained by using the illumination fluctuation coefficient and the illuminance ratio coefficient;

[0115] wherein the second target compensation light amount is obtained by the following formula:

[0116]

[0117] wherein L b represents the second target compensation light amount; B represents the illuminance ratio coefficient; specifically, B-S is the subtraction of the two, which measures the surplus of the illuminance ratio coefficient relative to the illumination fluctuation coefficient. K is a preset reference compensation intensity coefficient, representing a basic compensation adjustment factor, and 1+K scales the result of B-S, adjusts the final calculation result according to the preset compensation intensity rule, and makes the calculation more consistent with the actual light compensation demand adjustment logic. The part is based on 1 plus the previously calculated proportion value, which is to add the basic light intensity (i.e. the proportion corresponding to the current ambient light L) to the adjustment proportion calculated based on the illumination ratio coefficient and the illumination fluctuation coefficient, to obtain a comprehensive light compensation adjustment coefficient. The coefficient reflects the multiple relationship of the adjustment required relative to the current light after considering the illumination ratio and the fluctuation. The formula comprehensively considers the illumination ratio coefficient, the illumination fluctuation coefficient, and the preset reference compensation intensity coefficient and other factors affecting light compensation. It can more accurately calculate the required light compensation amount according to the proportional relationship between the current ambient light and the threshold, the light fluctuation, and the preset basic compensation rule, to avoid overcompensation or insufficient compensation. The light compensation amount can be dynamically adjusted according to different ambient light conditions. When the illumination ratio coefficient changes or the light fluctuation changes, the calculated second target light compensation amount will also change accordingly, so that the light compensation system has better environmental adaptability and can provide appropriate light compensation effect in different light scenes, improving the rationality and effectiveness of light compensation.

[0118] Compensate light according to the second target light compensation amount in the infrared light compensation mode.

[0119] The technical effect of the above technical solution is: the core goal of the light supplementing system is to make up for the lack of environmental light to make the light intensity of the monitoring area meet the working requirements of the imaging device (such as a camera). The infrared light supplementing mode illuminates by emitting infrared light (invisible to the human eye), and the amount of light supplementing needs to be inversely proportional to the environmental light: the weaker the environmental light, the stronger the amount of light supplementing needs to be. The light intensity ratio coefficient (current light / threshold value) reflects the degree of lack of current light. The light fluctuation coefficient is calculated by analyzing the historical light data (reference environmental light intensity parameter) of the past 30-60 minutes, and is used to quantify the stability of the environmental light. If the fluctuation coefficient is small, it means that the light changes smoothly; if the fluctuation coefficient is large, the light may have sudden changes (such as cloud cover, day and night alternation). Stable light scene (ratio coefficient ≤ fluctuation coefficient): use a fixed reference compensation intensity coefficient (K = 1.14-1.26), and the amount of light supplementing is K x L. At this time, the system assumes that the light change is within the expected range, and simplifies the calculation by using a fixed coefficient to reduce the real-time processing complexity. When the sudden light scene (ratio coefficient > fluctuation coefficient), the amount of light supplementing is dynamically calculated, and the compensation intensity is adjusted in combination with the fluctuation coefficient and the ratio coefficient. Traditional light supplementing amount determination usually only switches control based on whether the current light is lower than the threshold value, and the amount of light supplementing is fixed or simply linearly adjusted, which is easy to cause over-light supplementing (energy waste) or under-light supplementing (decrease of imaging quality). The present embodiment distinguishes between "stable low light" and "sudden low light" scenes through historical fluctuation analysis, and dynamically adjusts the light supplementing strategy. For example, during sunset (stable low light), the fixed coefficient ensures smooth light supplementing; in the case of sudden cloud cover (sudden low light), dynamic calculation responds quickly to avoid imaging interruption. Energy saving and device protection: the fixed reference coefficient (K = 1.14-1.26) is slightly higher than 1, indicating that the amount of light supplementing is slightly higher than the theoretical requirement (K x L), which not only compensates for the lack of environmental light, but also avoids energy waste and infrared lamp life decay caused by excessive light supplementing. Dynamic calculation is only triggered when necessary, reducing the burden on the system caused by frequent parameter adjustments. Imaging quality optimization: under the infrared light supplementing mode, the amount of light supplementing is proportional to the current light (L), ensuring sufficient compensation when the light is extremely low, while preventing the picture from being bright and dark due to light fluctuation (such as the fixed light supplementing amount of the prior art may fail when the light fluctuates). Intelligence and automation: by retrieving historical data from the database, the system can adapt to different environments without manual intervention to adjust parameters, reducing maintenance costs. The above technical solution of the present embodiment realizes fine control of the amount of light supplementing through double-layer logic of real-time monitoring + historical fluctuation analysis. Its physical essence is to optimize energy distribution by using the spatiotemporal correlation of light (current state and historical trend), which not only follows the basic physical law of light compensation, but also intelligently improves the adaptability of the system to complex environments. Compared with the prior art, the core advantage lies in balancing light supplementing effect, energy consumption and device life, especially suitable for monitoring scenes with variable light conditions.

[0120] The collected insect multi-spectral data is pre-processed, including:

[0121] Timestamp matching of RGB images, thermal infrared images and lidar point cloud data ensures that each set of multispectral data corresponds to the corresponding environmental conditions;

[0122] Preprocessing of RGB images includes removing noise interference in RGB images using Gaussian filtering and median filtering algorithms, and improving the contrast and clarity of RGB images through histogram equalization method;

[0123] Radiation correction and temperature correction are performed on the thermal infrared image, and the thermal infrared image is also subjected to gray scale stretching processing;

[0124] Filtering and denoising of lidar point cloud data removes noise points caused by environmental interference;

[0125] After the above preprocessing operations, the multispectral data is integrated into a unified data set according to the preset data format and channel order, and then input into the multi-modal large model architecture.

[0126] The technical effects of the above technical solutions are: timestamp matching of multispectral data ensures that RGB images, thermal infrared images and lidar point cloud data are accurately corresponding in the time dimension, so that each set of multispectral data can be accurately associated with the environmental conditions at the time of collection, providing time consistency guarantee for subsequent fusion analysis, using Gaussian filtering and median filtering algorithm to remove noise interference in RGB image, effectively reducing random noise in the image caused by environmental factors or sensor itself, radiation correction and temperature correction are performed on the thermal infrared image, which can eliminate measurement errors caused by sensor radiation characteristics, atmospheric transmission effect and other factors, so that the thermal infrared image more accurately reflects the actual temperature distribution, filtering and denoising of lidar point cloud data removes noise points caused by environmental interference (such as flying birds, dust, other non-related objects, etc.), and retains effective point cloud data related to vegetation and insect pests in the monitoring area, after the above preprocessing operations, the multispectral data is integrated into a unified data set according to the preset data format and channel order, so that it matches the requirements of the multi-modal large model architecture in data structure and format, which helps the multi-modal large model to better perform feature fusion and analysis, fully utilizes the advantages of multispectral data, and achieves more accurate insect situation detection results.

[0127] Based on the insect situation detection results, the risk level of insect outbreak is quantified through a three-level early warning system, including the following steps:

[0128] Analyzing the insect situation detection results, specifically:

[0129] Extracting information from the insect situation detection results, including the type, severity and development trend prediction data of insect pests;

[0130] According to the type of insect pests, severity and development trend factors, set the corresponding weight coefficient, that is, the weight of the type of insect pests is set to 30%, the weight of the severity is set to 50%, and the weight of the development trend is set to 20%;

[0131] Calculate the risk score, and determine the insect outbreak risk level according to the risk score, wherein the calculation formula is:

[0132] Risk score = (insect type value x 30%) + (severity level value x 50%) + (development trend prediction value x 20%).

[0133] The technical effects of the above technical solutions are: extracting the key information of the type of insect pests, the severity and the development trend from the insect situation detection results, and setting reasonable weight coefficients for these factors respectively, through scientific weight distribution, fully considering the contribution degree of different types of factors to the outbreak risk of insect pests, converting the qualitative insect situation information into quantitative risk score, determining the outbreak risk level of insect pests according to the calculated risk score, dividing the outbreak risk of insect pests into three clear levels, and corresponding to different degrees of insect threat, making the definition of risk level more clear and intuitive, facilitating the agricultural managers to quickly understand and convey, providing a scientific basis for the agricultural managers to take corresponding prevention and control measures, and realizing precise prevention and control.

[0134] The outbreak risk level of insect pests is divided into first-level warning, second-level warning and third-level warning, and a warning threshold is set for each outbreak risk level of insect pests, specifically:

[0135] The first-level warning threshold is 80-100 points, when the risk score reaches the first-level warning threshold, a red warning signal is generated, and first-level warning information is immediately sent to the management department and the management personnel;

[0136] The second-level warning threshold is 60-79 points, when the risk score reaches the second-level warning threshold, a yellow warning signal is generated, second-level warning information is sent and attention is reminded to take corresponding prevention and control measures;

[0137] The third-level warning threshold is 0-59 points, when the risk score reaches the third-level warning threshold, a green warning signal is generated, third-level warning information is sent and the management personnel is prompted to continue observation.

[0138] The technical effects of the above technical solutions are as follows: the risk levels of pest outbreak are divided into first-level warning, second-level warning and third-level warning, each level corresponds to a specific risk degree and corresponding measures, when the risk score reaches the corresponding warning threshold, the warning signal and information can be sent to the management department and the management personnel in time, ensuring the timeliness and accuracy of information transmission, and through the clear division of warning levels and corresponding measures, the attention and risk awareness of agricultural managers and relevant personnel to pest risks are improved, different levels of warning signals can attract different degrees of attention and attention, prompting management personnel to be more active and proactive in participating in pest monitoring and prevention and control work, thereby improving the efficiency and effectiveness of the entire pest management system.

[0139] Specifically, the embodiment also proposes a pest situation early warning system based on visual intelligence and large models, which is used to implement a pest situation detection method based on visual intelligence and large models, and includes:

[0140] A data acquisition module is configured to integrate an RGB camera, a thermal imager and a laser radar to collect multispectral data including RGB images, thermal infrared images and laser radar point cloud data, and to pre-process the collected multispectral data;

[0141] A data analysis module is configured to input the pre-processed multispectral data into a multimodal large model architecture, which includes a feature layer fusion architecture, an LSTM-Transformer hybrid architecture of a space-time memory network and a knowledge distillation mechanism, wherein:

[0142] The feature layer fusion architecture is used to interactively fuse the features of data from different spectral sources;

[0143] The LSTM-Transformer hybrid architecture of the space-time memory network is used to capture the evolution law of pests in the time and space dimensions;

[0144] The knowledge distillation mechanism is used for optimization processing;

[0145] A hierarchical warning module is configured to quantify the risk level of pest outbreak and generate corresponding warning signals through a three-level warning system according to the pest situation detection results output by the multimodal large model architecture.

[0146] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0147] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention.

Claims

1. A pest detection method based on visual intelligence and large models, characterized in that, The pest situation detection method is a detection method based on integrated multi-vision perception and large models to perceive, analyze and warn multi-dimensional information of pest situation; The pest situation detection method comprises: Determine the range and shape of the monitoring area, use the multispectral acquisition device integrating RGB camera, thermal imager and laser radar to collect multispectral data of pest situation and perform preprocessing, wherein the multispectral data includes RGB image, thermal infrared image and laser radar point cloud data; After determining the range and shape of the monitoring area, automatically switch to infrared fill light mode by ratio processing of ambient light intensity parameter and preset threshold value; Input the preprocessed multispectral data into a multi-modal large model architecture, sequentially pass through a feature layer fusion architecture for spectral feature interaction fusion, a LSTM-Transformer hybrid architecture of a spatio-temporal memory network to capture pest evolution law and an optimization processing of a knowledge distillation mechanism, and obtain a pest situation detection result including position, type, severity and development trend of the pest; Based on the pest situation detection result, quantify the risk level of pest outbreak through a three-level early warning system; The collection of multispectral data of pest situation comprises the following steps: According to the actual demand of pest situation monitoring, use GIS data combined with topography and vegetation distribution information to draw the boundary of the monitoring area on the map, determine the range and shape of the monitoring area; Set the multispectral acquisition device in the determined monitoring area, perform autonomous scanning and data acquisition on the monitoring area through the multispectral acquisition device, and finally collect multispectral data of pest situation and perform preprocessing, including: Turn on the RGB camera to obtain the RGB image of the monitoring area; Synchronously start the thermal imager to obtain the thermal infrared image of the monitoring area; Start the laser radar to scan the monitoring area and obtain the laser radar point cloud data in the monitoring area; After determining the range and shape of the monitoring area, first monitor the ambient light intensity of the determined monitoring area in real time through the ambient light intensity sensor, obtain the ambient light intensity parameter and compare it with the preset threshold value, when the ambient light intensity parameter is lower than the preset threshold value, automatically switch to the infrared fill light mode, otherwise use the visible light fill light mode; Specifically, automatically switching to the infrared fill light mode comprises: When the ambient light intensity parameter is lower than the preset threshold value, ratio process the ambient light intensity parameter and the preset threshold value to obtain the light intensity ratio coefficient; When the ambient light intensity parameter is lower than the preset threshold value, retrieve the preset time period from the database, wherein the value range of the preset time period is 30min-60min; Cut the ambient light intensity parameter within the time length corresponding to the preset time period before the time when the ambient light intensity parameter is lower than the preset threshold value as the reference ambient light intensity parameter according to the time length corresponding to the preset time period; Obtain the light fluctuation coefficient according to the reference ambient light intensity parameter; Wherein, the illumination fluctuation coefficient is obtained by the following formula: Wherein, S represents the illumination fluctuation coefficient; n represents the number of reference ambient illuminance parameters; L i And L i+1 Respectively represent the parameter value corresponding to the i th reference ambient illuminance parameter and the i+1 th reference ambient illuminance parameter; L th L σ Respectively represent the parameter value corresponding to the i th reference ambient illuminance parameter and the i+1 th reference ambient illuminance parameter; L th L σ Respectively represent the parameter value corresponding to the i th reference ambient illuminance parameter and the i+1 th reference ambient illuminance parameter; L th L σ Respectively represent the parameter value corresponding to the i th reference ambient illuminance parameter and the i+1 th reference ambient illuminance parameter; L th L σ Respectively represent the parameter value corresponding to the i th reference ambient illuminance parameter and the i+1 th reference ambient illuminance parameter; L th L <000000 Compare the light intensity ratio coefficient with the light fluctuation coefficient; When the illumination ratio coefficient does not exceed the illumination fluctuation coefficient, a preset reference compensation intensity coefficient is called; wherein, the preset reference compensation intensity coefficient is in a range of 1.14-1.26; Light is compensated according to the preset reference compensation intensity coefficient in the infrared light compensation mode; wherein, a first target light compensation amount corresponding to the light compensation is K*L; wherein, K represents the preset reference compensation intensity coefficient; L represents the current ambient illumination parameter; When the illumination ratio coefficient exceeds the illumination fluctuation coefficient, a second target light compensation amount is obtained by using the illumination fluctuation coefficient and the illumination ratio coefficient; The second target light supplement amount is obtained by the following formula: L b represents the second target light supplement amount; B represents a light intensity ratio coefficient. Light is compensated according to the second target light compensation amount in the infrared light compensation mode.

2. The pest detection method based on visual intelligence and large models according to claim 1, characterized in that: The preprocessed multispectral data is input into a multimodal large model architecture, including the following steps: The multispectral data enters a feature layer fusion architecture, which interacts and fuses the features of data from different spectral sources, specifically: The color texture features in the RGB image and the temperature distribution features in the thermal infrared image are correlated with each other to identify the relationship between abnormal color changes and temperature rises in the pest area; The pest morphology and spatial distribution information provided by the laser radar are fused with the RGB image and the thermal infrared image to form a pest feature representation; The fused multispectral data enters the LSTM-Transformer hybrid architecture of the spatio-temporal memory network to capture the evolution law of the pest in the time and space dimensions, specifically: LSTM is used to process time series information, records the evolution state of the pest over time through the internal forget gate, input gate and output gate structure, and captures the occurrence, development and possible outbreak trend of the pest; At the same time, the Transformer architecture part further enhances the processing capability of spatial features, analyzes the distribution pattern of the pest from different spatial perspectives; After the above feature interaction fusion and spatio-temporal evolution law capturing operations, a knowledge distillation mechanism is introduced to integrate historical data in the pest detection field into the model, and through knowledge distillation, the model learns the pest feature representation, and finally obtains the pest detection result.

3. The pest detection method based on visual intelligence and large models according to claim 2, characterized in that, The knowledge distillation mechanism is implemented in the following way: The rules summarized from the historical data in the pest detection field are used as prior knowledge; Through knowledge distillation, these prior knowledge is integrated into the learning process of the model to guide the model to better learn the pest feature representation, wherein the model includes a teacher model and a student model, specifically: The teacher model is trained based on a large amount of historical data to provide a high-level representation of the pest feature; The student model learns the key features of pest detection by learning the output and intermediate features of the teacher model; After optimization processing by the knowledge distillation mechanism, the final pest detection result is obtained, including the location, type, severity and development trend of the pest.

4. The pest situation detection method based on visual intelligence and large models according to claim 1, characterized in that: The collected pest multispectral data is preprocessed, including: Timestamp matching of RGB images, thermal infrared images and laser radar point cloud data to ensure that each set of multispectral data corresponds to the corresponding environmental conditions; The RGB image is preprocessed, including removing noise interference in the RGB image by using Gaussian filtering and median filtering algorithm, and improving the contrast and clarity of the RGB image by histogram equalization method; The thermal infrared image is subjected to radiation correction and temperature correction processing, and at the same time, the thermal infrared image is subjected to gray scale stretching processing; The laser radar point cloud data is subjected to filtering and denoising processing to remove noise points caused by environmental interference; After the above preprocessing operations are completed, the multispectral data is integrated into a unified data set according to the preset data format and channel order, and then input to the multi-modal large model architecture.

5. The pest situation detection method based on visual intelligence and large models according to claim 1, characterized in that: Based on the pest detection result, the risk level of pest outbreak is quantified through a three-level early warning system, including the following steps: The pest detection result is analyzed, specifically: Extract information from the pest detection result, including the type, severity and development trend prediction data of the pest; According to the type, severity and development trend factors of the pest, set the corresponding weight coefficients, that is, set the weight of the pest type to 30%, the weight of the severity to 50%, and the weight of the development trend to 20%; Calculate the risk score, and determine the risk level of pest outbreak according to the risk score.

6. The pest detection method based on visual intelligence and large models according to claim 5, characterized in that: The calculation formula is: Risk score = (pest type value x 30%) + (severity level value x 50%) + (development trend prediction value x 20%).

7. The pest situation detection method based on visual intelligence and large models according to claim 5, characterized in that: The risk level of pest outbreak is divided into first-level warning, second-level warning and third-level warning, and a warning threshold is set for each risk level of pest outbreak, specifically: The first-level warning threshold is 80-100 points, when the risk score reaches the first-level warning threshold, a red warning signal is generated, and first-level warning information is immediately sent to the management department and management personnel; The second-level warning threshold is 60-79 points, when the risk score reaches the second-level warning threshold, a yellow warning signal is generated, and second-level warning information is sent and attention is reminded to take corresponding prevention and control measures; The third-level warning threshold is 0-59 points, when the risk score reaches the third-level warning threshold, a green warning signal is generated, and third-level warning information is sent and the management personnel is prompted to continue observation.

8. The pest situation early warning system based on visual intelligence and large models, for realizing the pest situation detection method based on visual intelligence and large models according to any one of claims 1-7, characterized in that, It includes: A data acquisition module is configured to integrate RGB cameras, thermal imagers and laser radars to collect multispectral data, including RGB images, thermal infrared images and laser radar point cloud data, and to preprocess the collected multispectral data; The data analysis module is configured to input the preprocessed multispectral data into the multi-modal large model architecture, which includes a feature layer fusion architecture, an LSTM-Transformer hybrid architecture of a space-time memory network, and a knowledge distillation mechanism, wherein: The feature layer fusion architecture is used to interactively fuse the features of data from different spectral sources; The LSTM-Transformer hybrid architecture of the space-time memory network is used to capture the evolution law of the pest in the time and space dimensions; The knowledge distillation mechanism is used for optimization processing; The hierarchical warning module is configured to quantify the risk level of pest outbreak through a three-level early warning system and generate corresponding warning signals according to the pest detection result output by the multi-modal large model architecture.

Images