Multispectral imaging and deep learning intelligent detection system for pests and diseases

Abstract

The present invention relates to the field of pest and disease detection technology, and further relates to a multispectral imaging and deep learning intelligent detection system for pest and disease images. The system comprises: a multispectral radiance calibration unit, an image analysis and processing unit, and a pest and disease probability detection unit. The multispectral radiance calibration unit is used to perform multi-band imaging of vegetation leaf surfaces within a forest area, and convert the raw digital counts into radiance of the corresponding band after deducting the dark field digital counts; the image analysis and processing unit is used to convert the radiance into the apparent reflectance of the ground object; the lesion spectral gradient contrast index is then multiplied by the spectral heterogeneity index to obtain coupled pest and disease characteristics; and the pest and disease probability detection unit is used to map the convolution outputs of each layer into pest and disease probability values ​​based on a generalized error function. The present invention significantly improves the accuracy and practicality of pest and disease detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of pest and disease detection, and specifically relates to a multispectral imaging and deep learning intelligent detection system for pest and disease images. Background Art

[0002] The health of forest ecosystems is directly related to biodiversity, carbon sequestration capacity, and ecological security. Therefore, timely and accurate monitoring of forest pests and diseases has always been a crucial task in forestry research and production management. Traditional approaches rely primarily on manual ground surveys or low-altitude visual inspections. While these methods can provide intuitive information, they are limited by operational efficiency, intensity, and accessibility, making them difficult to achieve timely, large-scale, and high-resolution monitoring of large forest areas and complex terrain. With the advancement of remote sensing technology, researchers have begun exploring the use of multispectral aerial cameras, satellite payloads, and visible / near-infrared sensors onboard drones to identify pests and diseases in the forest canopy. Among existing publicly available techniques, a common approach is to infer the occurrence of pests and diseases based on changes in vegetation indices. For example, the Normalized Difference Vegetation Index (NDVI) is calculated using near-infrared and red light channels, or the Red Edge Normalized Difference Index (RED-EDG) is calculated using the red-edge band. These methods, to some extent, reflect changes in leaf chlorophyll content and canopy structure. However, the forest canopy is characterized by its rich vertical layers and significant interspecies diversity. The thresholds for the same indicator vary across different forest types, making fixed-threshold index methods prone to missed or false positives. In addition, vegetation indices often have difficulty distinguishing spectral changes caused by diseases, insect pests, and abiotic stresses (such as drought, nutrient deficiency, and wind damage), which limits their applicability in forestry management practices.

[0003] To improve the accuracy of discrimination, research teams at home and abroad have gradually introduced machine learning and deep learning frameworks to map multi-channel images to pest and disease category labels. Convolutional neural networks can automatically learn complex feature representations through end-to-end training, but most work is still limited to four wide bands of red, green, blue, and near-infrared, or three-channel RGB images. This band setting makes it difficult to capture the subtle spectral gradient information of lesions in the red edge and short-wave near-infrared narrow bands. Especially in coniferous forests or mixed broad-leaved forests with multiple tree species, the contrast of lesions in the visible light range may be very low, thus limiting the generalization ability of the model. In view of the large terrain undulations in forest areas and the rapid changes in solar incidence angle with slope aspect, some methods introduce empirical linear atmospheric corrections or simplify the radiometric calibration process. However, their assumptions about lighting conditions and sensor gain drift are relatively idealized, and they often fail under the interference of local shadows or cloud shadows, thereby transmitting the radiometric error to the subsequent feature extraction and classification stages. Summary of the Invention

[0004] The main purpose of this invention is to provide a multispectral imaging and deep learning intelligent detection system for pest and disease images. This system uses a multispectral radiance calibration unit to obtain high-precision multispectral images of forest areas. The image analysis and processing unit extracts the spectral gradient contrast index and spectral heterogeneity index of disease spots, and couples these to form pest and disease signatures. A pest and disease probability detection unit then uses multi-level upsampling and complex image enhancement methods to accurately map the coupled signatures to each layer of a feature pyramid network. The resulting responses are then integrated to generate pest and disease probability values. Finally, the severity of forest pests and diseases is calculated based on these probability values ​​and the pixel ground area. This system offers advantages such as precise spectral modeling, adequate spatial structure expression, interpretable probability output, and objective severity assessment, significantly improving the accuracy and practicality of pest and disease detection.

[0005] In order to solve the above problems, the technical solution of the present invention is achieved as follows:

[0006] A multispectral imaging and deep learning intelligent detection system for pest and disease images, the system comprising: a multispectral radiance calibration unit, an image analysis and processing unit, and a pest and disease probability detection unit; the multispectral radiance calibration unit is used to perform multi-band imaging of the vegetation leaf surface in the forest area to obtain a multispectral image of the forest area, obtain the original digital count and dark field digital count of each pixel therein, and combine the image sensor parameters to deduct the dark field digital count from the original digital count and convert it into the radiance of the corresponding band; the image analysis and processing unit is used to convert the radiance into the apparent reflectance of the ground object; the apparent reflectance of the ground object is calculated for each pixel in the disease sensitive band The first-order spectral gradient of the reflectance is obtained by multiplying the adjacent central wavelength distance by the first-order spectral gradient and calculating the ratio with the reference band reflectance to obtain the lesion spectral gradient contrast index. At the same time, the square root of the sum of the squares of the apparent reflectance differences between the eight neighboring pixels and the central pixel in the preset analysis band is calculated to obtain the spectral heterogeneity index. The lesion spectral gradient contrast index is then multiplied by the spectral heterogeneity index to obtain the coupled pest and disease characteristics. The pest and disease probability detection unit is used to map the coupled pest and disease characteristics to each layer of the feature pyramid network by upsampling to obtain the convolution output of each layer; and map the convolution output of each layer to the pest and disease probability value based on the generalized error function.

[0007] Furthermore, the system also includes: a pest and disease severity assessment unit, which is used to multiply and accumulate the pest and disease probability values ​​of the pixels in the multispectral image of each forest area with the ground area corresponding to the pixels, and divide them by the total area of ​​the forest area to obtain the pest and disease severity of the forest area.

[0008] Furthermore, the disease-sensitive band is a number of center wavelengths evenly distributed in the range of 680 nanometers to 720 nanometers; the reference band is a number of center wavelengths evenly distributed in the range of 760 nanometers to 800 nanometers; the preset analysis band is 720 nanometers; and the image sensor parameters include: image sensor radiation conversion factor and exposure time.

[0009] Further, Pixels in the band Radiance for:

[0010] ;

[0011] in, For the Pixels in the band The raw digital count of ; For the band Dark field digital counts; is the image sensor radiation conversion factor; is the exposure time.

[0012] Furthermore, the radiance Converted to apparent reflectance of ground objects The formula is:

[0013] ;

[0014] in, For the Pixels in the band The apparent reflectivity of the ground object; For the band atmospheric path radiation; is the average relative distance between the sun and the earth on that day; For the band Atmospheric transmittance; For the band Top-floor solar irradiance; is the solar zenith angle.

[0015] Furthermore, the pest and disease probability detection unit maps the coupled pest and disease features to each layer of the feature pyramid network by upsampling, including the following steps: performing mirror filling on all sides of the coupled pest and disease features, with the filling width being equal to the receptive field radius of the minimum layer of the feature pyramid network; using bilinear interpolation on a spectral channel basis to amplify the coupled pest and disease features to half of the original pixel size; calculating the horizontal first-order difference and the vertical first-order difference on the bilinear interpolation results, performing a 3×3 Laplace inverse compensation operation on the gradient peak value exceeding the set threshold to obtain a feature map; performing a fifth-order Savitzky-Golay spatial domain smoothing on the feature map, with the window length set to 9 pixels, to obtain a smoothed feature map; and performing a spectral channel-by-spectral interpolation on the smoothed feature map. A two-dimensional discrete Fourier transform is calculated to obtain a frequency domain feature map. A sub-pixel circular shift is performed on the phase spectrum in the frequency domain to keep the main phase difference between adjacent pixels in the range of −π / 8 to π / 8. An inverse transform is then performed to restore the spatial domain feature map to ensure that the amplified features have no phase jumps in the row and column directions. A four-directional one-dimensional cubic spline interpolation is introduced to supplement the sub-pixel intensity values ​​at the positions with one pixel intervals in each row and column. Then, a low-rank approximation is performed on the local 4×4 interpolation blocks using double singular value decomposition to eliminate local ringing artifacts caused by high-order interpolation, thereby obtaining the corrected feature map. Based on the ratio of the receptive field radius of the smallest layer and the second smallest layer of the feature pyramid network, the corrected feature map is aligned with the original pixel size in a 1:1 ratio and mapped to each layer of the feature pyramid network.

[0016] Furthermore, a 3×3 Laplace inverse compensation operation is performed on the gradient peaks that exceed the set threshold. The process of obtaining the feature map includes: calculating the absolute value of the gradient amplitude for the same pixel in the horizontal first-order difference and the vertical first-order difference of each spectral channel, and comparing it with the preset threshold; when the absolute value of the gradient amplitude is greater than the threshold and greater than the corresponding value of its eight neighboring pixels at the same time, marking the pixel as a gradient peak candidate point; for each gradient peak candidate point, the spectral variance of the coupled pest and disease characteristics in its 3×3 neighborhood is counted, and the ratio of the variance to the threshold is calculated. Generate an adaptive Laplace coefficient with a value ranging from 0.5 to 2; apply a 3×3 two-dimensional Laplace kernel [−1,−1,−1;−1,8,−1;−1,−1,−1][-1,-1,-1;-1,8,-1;-1,-1,-1][−1,−1,−1;−1,8,−1;−1,−1,−1] to the 3×3 neighborhood of the gradient peak candidate point, take the inverse of the output and multiply it by the adaptive Laplace coefficient, then add it pixel by pixel with the coupled pest and disease features to obtain the feature map.

[0017] Furthermore, the pest and disease probability detection unit calculates the extreme difference of the convolution output of the same pixel at each layer of the feature pyramid network. and mean , subtract the mean from each convolution output and divide it by the range to get the interval The normalized response map within the image is calculated; the 3×3 Sobel gradient magnitude is calculated on the normalized response map, and the pixel responses whose magnitude is lower than the preset noise floor threshold are set to zero; the remaining non-zero responses are Mapped to log-likelihood value through a fixed-parameter bi-segment logarithmic function ; Take the harmonic average of the log-likelihood values ​​of the same pixel in all layers to obtain the single-scale consistency confidence of the pixel ; Then calculate in 5×5 and 9×9 sliding windows respectively The spatial mean of and , and finally take As the multi-scale fusion confidence; multi-scale fusion confidence Input the generalized error function to obtain the probability value of pests and diseases.

[0018] Furthermore, the formula of the generalized error function is:

[0019] ;

[0020] in, is the probability value of pests and diseases.

[0021] The multispectral imaging and deep learning intelligent detection system for pest and disease images of the present invention has the following beneficial effects: First, during the image acquisition stage, the present invention introduces multi-band imaging and a precise multispectral radiance calibration mechanism, which can effectively eliminate dark field noise interference. Combined with the image sensor radiation conversion factor and exposure time parameters, it achieves accurate conversion of reflection intensities in different bands, so that subsequent pest and disease identification is based on quantitative and comparable physical quantities. Second, during the image analysis process, by coupling and fusing the first-order spectral gradient of the apparent reflectance of the ground object in the red-edge disease-sensitive band with the spectral heterogeneity within the spatial range, a joint feature that is more sensitive to pests and diseases is constructed. This feature comprehensively considers the spectral change trend of the lesions and the local spatial texture mutation, significantly improving the ability to distinguish between lesion areas and healthy leaves. Third, during the feature mapping phase, the system employs hierarchical upsampling and incorporates complex image enhancement mechanisms such as frequency-domain phase alignment, high-order interpolation between spectral channels, and low-rank artifact removal. This ensures that the fine-grained information of the original coupled features is preserved to the greatest extent possible during the mapping process to the feature pyramid network, effectively suppressing interference factors such as blurred lesion edges and interpolation ringing. Furthermore, during the generation of pest and disease probability values, the system constructs a spatial confidence measure based on the consistent distribution of multi-scale response maps and uses sliding window fusion to form a more stable fusion index, thus avoiding the oversensitivity of single-scale detection to local anomalies. Ultimately, a continuous pest and disease probability value is output through a fixed-form probability function mapping, which offers greater interpretability and stability. Furthermore, during the severity assessment phase, the system calculates a pest and disease severity value that closely correlates with the actual forest area based on the weighted integral of each pixel's pest and disease probability value and the actual ground area. This significantly outperforms traditional binary mask assessment methods and more accurately reflects the overall impact of pests in the forest area. Therefore, the present invention demonstrates significant advantages in accuracy, stability, interpretability and practicality, and is suitable for rapid assessment and precise prevention and control management of large-scale forest pests and diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the system structure of the multispectral imaging and deep learning intelligent detection system for pest and disease images provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a comparison curve of the apparent reflectance of a healthy leaf and a diseased area within the wavelength range of 680nm to 800nm ​​provided by an embodiment of the present invention;

[0024] Figure 3 The comparative experimental results of the present invention show the detection effect of vegetation leaf spots in different bands;

[0025] Figure 4A schematic diagram of the complete extraction process from original multispectral images to coupled pest and disease features and its effect provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram showing the mapping of coupled pest and disease features to each layer of a feature pyramid network through upsampling provided in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of Sobel gradient amplitude processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0029] refer to Figure 1 :Multispectral imaging and deep learning intelligent detection system for pest and disease images, the system includes: a multispectral radiance calibration unit, an image analysis and processing unit and a pest and disease probability detection unit; the multispectral radiance calibration unit is used to perform multi-band imaging of the vegetation leaf surface in the forest area, obtain a multispectral image of the forest area, obtain the original digital count and dark field digital count of each pixel, combine the image sensor parameters, deduct the dark field digital count from the original digital count and convert it into the radiance of the corresponding band; the image analysis and processing unit is used to convert the radiance into the apparent reflectance of the ground object; the apparent reflectance of the ground object is calculated for each pixel in the disease sensitive band The first-order spectral gradient of the emissivity is calculated, and the adjacent central wavelength distance is multiplied by the first-order spectral gradient and the ratio is calculated with the reference band reflectivity to obtain the lesion spectral gradient contrast index. At the same time, the square root of the sum of the squares of the apparent reflectivity differences between the eight neighboring pixels and the central pixel is calculated in the preset analysis band to obtain the spectral heterogeneity index. The lesion spectral gradient contrast index is then multiplied by the spectral heterogeneity index to obtain the coupled pest and disease characteristics. The pest and disease probability detection unit is used to map the coupled pest and disease characteristics to each layer of the feature pyramid network by upsampling to obtain the convolution output of each layer; and map the convolution output of each layer to the pest and disease probability value based on the generalized error function.

[0030] Plant diseases and pests can cause simultaneous changes in chlorophyll, carotenoids, anthocyanins, intercellular water content, and internal structure in leaf tissue. The multispectral radiance calibration unit uses a spectral image sensor calibrated with laboratory absolute radiometry before the imaging optical path to perform multi-band imaging of plant foliage within a forest. The radiant energy in each band is accurately recorded as radiance. The image analysis and processing unit then converts the radiance into apparent reflectance based on physical radiation transfer relationships. This conversion essentially separates the incoming and outgoing radiation, allowing subsequent calculations to completely eliminate interference from external factors such as illumination, solar altitude, and image sensor gain, while retaining spectral information directly related to light absorption and scattering within the plant. The system then compares the disease-sensitive band with a reference band, introducing the first-order spectral gradient of the apparent reflectance to characterize the absorption difference between the bands. This difference is multiplied by the center wavelength distance to obtain the spectral gradient contrast index for the lesion, highlighting the sudden reflectance changes caused by pigment loss in the lesion. At the same time, the system calculates the square root of the sum of the reflectance differences between a pixel and its eight neighboring pixels within a single preset analysis band to generate a spectral heterogeneity index, which quantifies the degree of spatial texture discontinuity caused by cellular structural damage. The spectral gradient contrast index (SGI) emphasizes spectral domain abrupt changes, while the spectral heterogeneity index emphasizes spatial domain roughness. Multiplying the two creates coupled pest and disease signatures, seamlessly integrating complementary spectral and spatial information into a single-channel scalar, avoiding model redundancy introduced by high-dimensional inputs in subsequent networks.

[0031] After receiving the coupled pest and disease probability detection unit, it first amplifies the features through deterministic geometric operations such as mirror padding and bilinear interpolation to achieve multi-scale discrimination. The spectral attenuation caused by interpolation is then locally restored using first-order row-column differences combined with three-by-three Laplace inverse compensation. Savitzky-Golay spatial smoothing and Fourier phase alignment ensure that the true phase position of the disease edge is maintained while controlling the signal power. Sub-pixel ringing artifacts are then structurally suppressed through spline interpolation and double singular value decomposition, reducing high-frequency noise to negligible levels in both spatial and frequency domains. The resulting corrected feature map is fed into each layer of the feature pyramid network as a common input. The feature pyramid network only executes a fixed filter bank in the hardware convolution array, extracting texture and speckle morphology responses of varying depths through pure forward convolution, and outputs a convolution feature map. At this point, the system does not rely on any weight de-tuning or learning mechanisms. Instead, it uses cross-layer response statistical normalization to eliminate differences in amplitude scale across layers. A three-by-three Sobel operator is then used to remove low-gradient noise points, ensuring that the convolution output retains only high-confidence regions associated with actual pests and diseases. The remaining responses are mapped to log-likelihood values ​​using a two-segment logarithmic function. This function is designed to compress high-response saturation regions in the numerical domain while expanding low-response effective regions, thereby maintaining sensitivity to mild lesions. The system uses harmonic averaging to probabilistically converge the log-likelihood values ​​of different layers for the same pixel. Spatial means of different window sizes are then used to further integrate surrounding information, ultimately yielding a multi-scale fusion confidence score.

[0032] The multi-scale fusion confidence input generalized error function, which can be regarded as a scaled and translated form of the standard normal cumulative distribution function, has a monotonically increasing property that ensures a one-to-one correspondence between the confidence and the output pest and disease probability value. At the same time, the steepness of the probability distribution is uniformly calibrated by fixing the slope and intercept to avoid threshold drift between different forest areas and different vegetation varieties. The pest and disease probability value and the pixel ground area are sent to the back-end pest and disease severity assessment unit. The back-end unit losslessly maps the pixel-level probability to the total area scale of the forest area through area-weighted accumulation, and outputs the quantified forest-level pest and disease severity. The key principle of this embodiment is to take the spectral physiological mechanism as the starting point, ensure the authenticity and reliability of the spectral data through physical radiation calibration, use the combination of gradient and heterogeneity to capture the dual characteristics of the lesions, and ensure the complete transmission of the features through multi-scale space-frequency fine alignment.

[0033] The system also includes a pest severity assessment unit, which multiplies and accumulates the pest probability value of each pixel in the forest multispectral image by the corresponding ground area, and then divides it by the total forest area to obtain the pest severity score for that forest area. To ensure real-time processing of large images, the assessment unit uses a streaming block pipeline architecture. The forest multispectral image is written to a high-speed cache as fixed-size tiles. A hardware multiplier performs a per-pixel multiplication of the pest probability value by the corresponding ground area for each tile. The resulting product is immediately accumulated into an on-chip double-buffered chord register to form a weighted sum of the partitioned areas. Simultaneously, the system constructs the total forest area in real time by individually accumulating the pixel ground areas. As the last tile is processed, the pest severity assessment unit obtains the average forest area-weighted probability without any iterations and identifies this average as the pest severity score. To ensure intuitive forest management, a five-level grading mechanism based on pest severity values ​​is pre-set: a pest severity between 0 and 0.2 indicates a healthy state; a severity between 0.2 and less than 0.4 indicates mild pests; a severity between 0.4 and less than 0.6 indicates moderate pests; a severity between 0.6 and less than 0.8 indicates severe pests; and a severity between 0.8 and less than 1 indicates extremely severe pests. The pest severity assessment unit outputs the severity value and writes the corresponding level label directly to the drone ground station, providing guidance for precise variable-rate spraying, differentiated harvesting, and intelligent forest inspections.

[0034] Furthermore, the multispectral imaging and deep learning intelligent detection system for pest and disease images uses spectral layout based on the changes in chlorophyll absorption and cellular structural decline during plant disease processes. The disease-sensitive bands are set to several central wavelengths evenly distributed between 680 and 720 nanometers to capture the red-edge shift signals caused by chlorophyll a decline and carotenoid imbalance. Simultaneously, several central wavelengths evenly distributed between 760 and 800 nanometers are selected as reference bands to characterize the stable scattering properties of the near-infrared high-reflectivity platform, providing a baseline comparison for healthy and diseased images in spectral ratio calculations. The system uses a spectral beam splitter to synchronously project these two sets of bands onto the linear array detector elements, maintaining the consistent order of the bands along the detector columns to ensure a one-to-one spatial correspondence between the disease-sensitive bands and the reference bands within the same scan line. The preset analysis band is fixed at 720 nanometers, which is located at the red edge inflection point. It overlaps with the upper limit of the disease-sensitive band and is close to the lower limit of the reference band. Therefore, it can simultaneously reflect the combined effects of color attenuation and internal scattering during spatial texture analysis. It is the optimal compromise point for calculating the square root of the sum of squared reflectance differences of eight-neighborhood pixels.

[0035] The image sensor's radiometric conversion factor (RFC) is derived from the reflectance of a laboratory radiometric calibration plate and the measured digital counts. Before the mission begins, the system automatically reads a lookup table stored in the airframe's non-volatile memory and performs segmented interpolation corrections as the detector temperature changes to ensure consistent radiometric conversion accuracy across temperature zones. The onboard controller adjusts the exposure time in real time, closed-loop, based on the current visible light irradiance, to keep the digital counts in the disease-sensitive band within the mid-dynamic range, avoiding saturation or excessive quantization noise. The exposure time, along with the frame number, is written into the metadata stream. After imaging is complete, the multispectral radiance calibration unit uses the RFC and exposure time to rapidly convert the raw digital counts into radiance. The image analysis and processing unit then uses the radiance, combined with the solar zenith angle and atmospheric transmittance, to convert the surface reflectance of the object. At this point, the disease-sensitive band and the reference band are aligned in the spectral dimension, and the texture indicators of the preset analysis band are also generated synchronously. The spectral difference term and spatial heterogeneity term in the coupled disease and insect pest characteristics thus obtain physically comparable and dimensionally consistent inputs, and enter the subsequent disease and insect pest probability detection unit to achieve quantitative, rapid and stable identification of forest disease conditions.

[0036] Furthermore, the raw digital count The digital counts of the dark field are directly quantized by the analog-to-digital converter during the integration period of the linear array detector. However, this value also includes electronic readout noise, thermal dark current, and fixed offset. Therefore, the system uses a shutter closure to record the dark field digital counts before each scanning task. , and interpolated to each frame time point in the acquisition sequence, thereby synchronously compensating for dynamic temperature drift and inter-row readout mode differences. The net count after compensation is still in the electron count domain and needs to be converted by the image sensor radiation conversion factor Normalized to the radiation power domain. This factor is used under laboratory conditions with an integrating sphere uniform light source as the standard. Multiple sets of radiant illuminance-digital count pairs are collected for each band and each gain level. The slope is obtained by least squares linear fitting and stored in a table inside the detector body. When the system is running in the forest, the corresponding values ​​are read from the lookup table based on the selected band and current gain. .because Affected by the slight temperature fluctuation of the detector, the hardware temperature sensor writes the real-time temperature into the register at a fixed time, and the calibration unit automatically corrects it according to the pre-stored segmented interpolation curve , ensuring that the sensitivity change caused by thermal drift is within one percent.

[0037] Exposure time The onboard controller makes closed-loop adjustments based on the incident irradiance of the photometric unit, so that the disease-sensitive band and the reference band are simultaneously in the middle of the dynamic range of the analog-to-digital converter, avoiding saturation and suppressing quantization noise. The radiance calculation module reads the corresponding exposure time synchronously when reading the image frame to ensure that the timing and measurement are one-to-one. Through the on-chip multiplier and After multiplication, write to pipeline register and then subtract in subtractor The result of this is to complete the conversion from the count domain to the radiation power domain; then the division unit divides by , and obtain the radiance per unit time per unit solid angle per unit wavelength. To ensure real-time processing of large-format data streams, the hardware pipeline fully parallelizes the band dimension and supports continuous scanning in a circular buffer manner. The delay of single-pixel radiance does not exceed several clock cycles. It carries spectral, spatial and radiometric triple information and is used uniformly as input for the conversion of surface reflectance in the image analysis and processing unit, laying a strict and consistent foundation for the subsequent high-precision calculation of the lesion spectral gradient contrast index, spectral heterogeneity index and coupled pest and disease characteristics. It also enables the pest and disease probability detection unit to maintain stable detection thresholds and transferable results across forest areas, species and even seasons. Pixels in the band Radiance for:

[0038] ;

[0039] in, For the Pixels in the band The raw digital count of ; For the band Dark field digital counts; is the image sensor radiation conversion factor; is the exposure time.

[0040] Furthermore, the image sensor directly measures Contains both vegetation leaves in the band The real reflection on the The upwelling atmospheric scattered radiation represented by is produced by the multiple effects of gas Rayleigh scattering, aerosol Mie scattering and water vapor absorption in the visible and near-infrared bands. It is an additive noise term that varies with wavelength, observation geometry and meteorological conditions. The net reflective contribution of the leaf surface to solar radiation can be obtained. The factor is multiplied because the system uses the Lambertian body approximation to convert the radiation in the unit solid angle direction into hemispherical integrated flux, which is consistent with the top solar irradiance. In the same geometric datum. The square term is used to correct the annual cycle irradiance amplitude fluctuation caused by the change of the Earth's position in the elliptical orbit; when the system performs forest patrol imaging on any day of the year, this term ensures that the reflectivity is absolutely comparable across seasons. The one-way energy loss from the top to the leaf surface is included in the model to avoid the misinterpretation of the difference in incident radiation intensity due to different aerosol loads or water vapor column concentrations as differences in vegetation reflectance. It is the fixed value of the internationally accepted solar radiation spectrum table, which is loaded once by the system before the task is started. The term reflects the projection relationship between the ground horizontal plane and the direction of solar incidence. Its physical meaning is that the same radiation is distributed to a larger horizontal area at a larger zenith angle, and the radiation received per unit area is reduced. If this correction is not performed, the low incident energy during the dawn and dusk periods will be misjudged as a decrease in vegetation reflectance.

[0041] The system hardware calculates in real time through ephemeris and inertial navigation measurements , and use the cache table to quickly obtain The product of all denominators in the formula constitutes a unified normalized benchmark for incident energy, geometric projection and atmospheric attenuation, which is divided by The value of becomes a dimensionless ratio determined purely by the optical properties of the vegetation surface. During the implementation process, the image analysis and processing unit first searches for the on-chip and These values ​​are obtained by interpolation from the MODTRAN scenario library that was previously run offline based on the sensor height, viewing angle, and real-time weather conditions; then the Julian day is calculated. , and then use the flight time and position to calculate All parameters are written into the vector multiplication and division pipeline in floating point format. The above formula is calculated in parallel for each pixel and each band. It is directly fed into the calculation module of the lesion spectral gradient contrast index and spectral heterogeneity index. Through this rigorous correction based on the physical radiation transfer equation, the system achieves spectral consistency under different flight batches, different aerosol conditions and different seasonal sunshine conditions, providing a stable and reliable input for the pest and disease probability detection unit, allowing the deep learning inference stage to focus on the intrinsic spectral differences caused by vegetation lesions rather than external imaging condition differences. Converted to apparent reflectance of ground objects The formula is:

[0042] ;

[0043] in, For the Pixels in the band The apparent reflectivity of the ground object; For the band atmospheric path radiation; is the average relative distance between the sun and the earth on that day; For the band Atmospheric transmittance; For the band Top-floor solar irradiance; is the solar zenith angle.

[0044] Furthermore, the coupled pest and disease features initially have the same band dimensions as the original pixels, but only have a spatial sampling rate half the original pixel size. Directly inputting them into the smallest layer of the feature pyramid network would introduce aliasing errors due to spatial resolution mismatch. Therefore, the pest and disease probability detection unit first performs four-sided mirror padding. This mirroring extends the unknown pixels outside the boundary in a mirrored manner through even-odd symmetry, allowing the convolution kernel to obtain local statistics consistent with the internal region even in areas close to the image frame, without creating unnatural high-contrast steps caused by zero padding. The mirror width is chosen to be equal to the receptive field radius of the smallest layer of the feature pyramid network because edge effects will not propagate inwards only when the maximum receptive range of the convolution kernel is completely covered by real or mirrored pixels.

[0045] After mirroring, bilinear interpolation is used on a per-spectral channel basis to amplify the coupled pest and disease features. Bilinear interpolation essentially performs a first-order local planar approximation of the continuous signal on the original two-dimensional sampling grid. Its frequency response exhibits a low-pass characteristic, capable of filling sampling holes without overshoot, but it also inevitably attenuates high-frequency detail. To restore the high-gradient energy at the lesion edge after interpolation, the system calculates first-order horizontal and vertical differences to estimate the local one-dimensional derivative. Threshold detection is used to identify high-amplitude gradient peaks, and then a three-by-three Laplace inverse compensation is performed. The Laplace operator is essentially a second-order difference kernel, which weights grayscale changes and takes the negative energy at the center. Inverse compensation involves inverting the sign of its output and proportionally superimposing it back onto the original image. This is equivalent to applying a fixed-amplitude gain to the high-frequency components suppressed by interpolation in the frequency domain, restoring sharp edges. Laplace inverse compensation will also amplify the quantization noise. To maintain the signal-to-noise ratio, the unit continues to use fifth-order Savitzky-Golay spatial domain smoothing for local polynomial regression fitting. This method uses fifth-order polynomial least squares to fit the original spectral curve within the window, retaining the zero-order and first-order derivative information while attenuating high-order oscillations, smoothing the noise and maintaining the edge position of the patch.

[0046] While edge contrast is enhanced, the sub-pixel misalignment caused by interpolation and regression can cause phase jumps in the row and column directions. To unify the phase, the system performs a two-dimensional discrete Fourier transform on each band, mapping the spatial signal to the frequency domain. The phase spectrum characterizes the image structure and position. Sub-pixel-level circular shifting compresses the principal phase difference between adjacent pixels to a range of −π divided by eight to π divided by eight, achieving sub-pixel phase alignment. An inverse transform is then performed back to the spatial domain, theoretically avoiding edges where different layers sample the same point at different phases during multi-scale convolution. Even after phase alignment, sampling holes still exist due to bilinear weight discretization. To address this, a four-directional one-dimensional cubic spline interpolation unit is used. A cubic polynomial segment function is used to generate a second-order continuous curve between every two original sampling points. This sub-pixel intensity value is added to the positions where the rows and columns are separated by one pixel, thereby improving the spatial Nyquist sampling rate.

[0047] Cubic splines are prone to ringing artifacts in highly textured areas. These artifacts manifest as localized high-order harmonics in the frequency domain. To further suppress these artifacts, the system performs a low-rank approximation using double singular value decomposition (DSVD) within each four-by-four interpolation block. This method concentrates energy on the first few singular values, corresponding to the principal directions where the texture is supported by the true structure. High-order ringing artifacts, which are generated solely by overfitting, are projected onto low-energy singular vectors and discarded during reconstruction. This is equivalent to a locally data-adaptive high-frequency filtering. The resulting corrected feature map undergoes the aforementioned multi-stage processing, with independent spectral channels and spatial consistency. It simultaneously achieves the three goals of restoring sharp lesion texture, suppressing subpixel alignment errors, and robustly filtering noise and artifacts. Finally, based on the ratio of the receptive field radius between the smallest and next-smallest layers of the feature pyramid network, the corrected feature map is mapped one-to-one to restore the original pixel resolution. The map is then directly copied into the reserved input tensor area of ​​each layer of the feature pyramid network, ensuring that all convolutional layers observe the coupled pest and disease features of the same physical point at the same coordinates. At this time, the multi-scale convolution kernel's perception of the patch texture, the transition between the edge of the lesion and the background will remain strictly synchronized in both the spatial and spectral domains, laying the foundation for high-fidelity input with consistent phase and complete spectrum for the subsequent convolution output response statistical normalization and generalized error function mapping. In principle, it ensures that the probability detection of pests and diseases can still correctly distinguish between lesions and non-lesions areas without introducing any learning weights.

[0048] Furthermore, the coupled pest and disease characteristics must undergo an irreversible low-pass smoothing after bilinear interpolation. The essence of this smoothing is to treat the original discrete sampling as a baseband pulse train and then multiply it by a set of first-order local basis functions for reconstruction; this process is equivalent to wrapping the original signal with The ideal interpolation kernel of the shape suppresses a large amount of high-frequency energy near the diffraction limit. The edges of lesions are precisely the multi-band refractive index discontinuities caused by the combination of leaf pigment mutation and epidermal microstructural collapse. These discontinuities manifest as steep first-order singularities in the spatial spectrum. If the interpolation kernel is allowed to attenuate these energies, the response amplitude of the subsequent convolution kernel to the boundary will be directly reduced, leading to an underestimation of the lesion probability. The pest and disease probability detection unit uses gradient peak-driven Laplace inverse compensation. The core mechanism is to use first-order differences to locate areas of high-frequency distortion, then apply local gain to these areas in the form of negative feedback using second-order differences, thereby reshaping the spectral components lost during interpolation in the spatial domain.

[0049] The system first performs horizontal and vertical first-order differences on the coupled pest and disease characteristics. The gradient amplitude is mathematically the Euclidean norm of the local one-dimensional derivative. Derivative operations correspond to multiplying the frequency in the frequency domain, thus enhancing high-frequency components. Threshold identification and comparison with the extreme values ​​of the eight-neighborhood ensure that compensation is triggered at the true structural turning point, which is equivalent to maximum a posteriori edge detection from a statistical decision-making perspective. A three-by-three two-dimensional Laplace kernel is then used to extract second-order differences. Second-order differences approximate curvature in the sense of a Taylor expansion, thus emphasizing corners and line segment inflections—the geometric characteristics of lesion edges. Taking the inverse of the convolution result and adding it back to the original signal is equivalent to performing a local inverse solution of the discrete Poisson equation, restaging the high-frequency phase suppressed by the interpolated low-pass filter. In order to avoid amplifying the noise during the compensation process, spectral variance is introduced as a suppression factor: when the reflectance changes in multiple bands at the same location are consistent in direction and large in amplitude, it means that it is very likely to be the edge of a lesion or an insect-eaten hole, the spectral variance is large, and the adaptive Laplace coefficient is improved; if only a single band mutates while other bands are stable, it is mostly random noise or soil interference, the spectral variance is low, and the coefficient is suppressed.

[0050] This strategy uses multi-spectral redundancy to improve signal reliability in the sense of decision theory, which is equivalent to introducing a soft gating based on the covariance matrix trace for the Laplace output. The coefficient range is limited to 0.5 to 2 to ensure that the compensation is a controllable gain rather than an unbounded amplification, which meets the gain margin stability condition. After the compensation is completed, it is added to the original signal, which is a residual learning. However, unlike the trainable network, the residual here is obtained analytically and does not involve gradient backpropagation. Therefore, the entire operator can be implemented in a single-cycle pipeline in on-chip parallel logic. Since the Laplace kernel size is fixed at three by three, its frequency response covers 2 / 3 of the Nyquist frequency, which is just used to recover the bilinear interpolation factor. The bandwidth holes caused by falling zeros will not be introduced into the area beyond Nyquist. In terms of information theory, the whole process is equivalent to first low-pass and then reconstructing a part of the distorted spectrum with band-limited compensation filtering, while the threshold and coefficient modulation ensure that the main energy of the restored spectrum is concentrated in the discriminative sideband. The feature map obtained by this process is highly consistent with the original uninterpolated map in terms of spatial gradient and spectral amplitude energy distribution. It not only meets the premise that the convolution kernel is sensitive to edges, but also ensures that the newly added sample points after interpolation contain sufficient disease texture information for the feature pyramid network to capture. This pure signal processing method abandons learnable weights, is not limited by the scale of training samples, and is not affected by the difference in lighting between different plots. It has the advantages of being interpretable, traceable, and easy to hardware. It provides the best balance between the fidelity of lesion details and noise suppression under the conditions of real-time collection in the forest.

[0051] Furthermore, after completing the convolution feature extraction, the pest and disease probability detection unit needs to convert the convolution outputs distributed in each layer of the feature pyramid network into pest and disease probability values. The convolution outputs have great differences in amplitude in space, scale, and spectral dimensions. If they are directly added or averaged, the high-response layer will suppress the low-response layer, or the low-response layer noise will drown out the details of the high-response layer. To ensure the statistical consistency of cross-layer fusion, the unit first calculates the range of the convolution outputs of the same pixel in all layers. and mean The range describes the amplitude fluctuation range of the pixel in the scale domain, and the mean gives the center position. Then, the mean is subtracted from the output of each convolution layer and divided by the range, so that the transformed value is compressed to This linear normalization is equivalent to implementing zero-mean and unit-amplitude transformation on a local scale, which not only eliminates the amplitude bias of each layer feature, but also avoids the amplitude drift caused by spike distortion when only variance normalization is used. Then calculate on the standardized response map Sobel gradient amplitude. The Sobel kernel is a combination of first-order difference and mean filtering, which can suppress high-frequency noise while highlighting the directional gradient. Since the normalized value has fallen within , the gradient amplitude theoretically falls within To further shield against pseudo-high responses caused by quantization errors, photosensitive noise, or background texture, the unit sets a noise floor threshold, directly returning pixel responses below the threshold to zero. This process is equivalent to introducing a band-stop window in the frequency domain, simultaneously attenuating low-amplitude high-frequency noise and high-amplitude low-frequency noise, while retaining the medium- and high-amplitude gradients truly caused by the lesion texture.

[0052] After threshold screening, the remaining non-zero responses Still distributed in arrive The asymmetric density range of the disease and insect pest categories is long-tailed; if a linear mapping is used to the probability domain, the moderate disease spots will be compressed into a narrow range. The unit uses a fixed parameter double-segment logarithmic function to convert Transformed into log-likelihood The double-segment logarithmic function is to A larger slope is used to amplify subtle responses and enhance the detectability of mild lesions; to The slope is slowed down between the two, preventing excessive compression of the probability after the response of severe lesions is saturated. This nonlinear mapping redistributes the dynamic range through the concave function of the logarithm, so that the severity of the lesion is spread out in logarithmic geometric intervals, which conforms to the log-linear scaling assumption of visual psychology and Bayesian likelihood.

[0053] The harmonic mean of the log-likelihood values ​​of the same pixel in all layers is taken instead of the arithmetic mean. The reason is that the harmonic mean tends to be dominated by the minimum value, which can quickly reduce the total confidence when any layer lacks a response, prevent the high response of a single layer from masking the uncertainty caused by the low response of other layers, and improve the false detection penalty from the perspective of decision theory. The harmonic mean output is defined as the single-scale consistency confidence Confidence It is still a point estimate and may be affected by isolated noise or isolated omissions. and Simultaneous calculation within the sliding window The spatial mean of and Smaller windows emphasize local coherence and capture consistent responses within spots; larger windows provide a smoother regional background to filter out isolated point false positives. Perform weighted linear combination , where weight 2 is given Higher influence, reflecting the fact that lesions are usually distributed in clusters within a limited neighborhood, while retaining The low-frequency information of is used as a global constraint to suppress the error propagation within a single window.

[0054] Multi-scale fusion confidence It is still a dimensionless confidence. In order to project it into the real probability space, the unit introduces a generalized error function. The generalized error function is the standard error function The monotone S-shaped curve obtained by linear scaling and translation has the same mathematical properties as the cumulative distribution function. Data is mapped to The fixed slope and intercept are used because the system is physically scaled to ensure that Falls in a finite interval; the slope value controls the steepness of the curve, corresponding to mapping the confidence uncertainty band to the width of the middle of the probability; the intercept determines the probability zero point shift, so that When the mapping probability is equal to the empirical threshold, it is approximately This fixed mapping avoids re-estimating topological shapes under varying vegetation and lighting conditions. It is also unaffected by data imbalance, as is the case with Sigmoid training parameters, maintaining consistency across forest areas. The output pest and disease probability values ​​are then sent to the Pest and Disease Severity Assessment Unit, where they are weighted by pixel ground area to achieve forest-level pest and disease quantification.

[0055] Furthermore, the formula of the generalized error function is:

[0056] ;

[0057] in, is the probability value of pests and diseases.

[0058] The error function itself is defined as the cumulative distribution function of a zero-mean, unit-variance Gaussian distribution that is scaled and then translated as a whole. Therefore, it has four key characteristics: First, the output interval is strictly limited to and The slope parameter 1.5 and the bias parameter minus 0.3 are derived from the principle of statistical discrimination rather than arbitrary empirical settings. The system has been tested on large-scale forest data collected from more than ten common vegetation species (including wheat, rice, corn, soybeans, cotton, etc.). The central limit theorem analysis was performed on the mapping relationship between the proportion of manually marked lesions.

[0059] The results show that when the actual proportion of diseased pixels in a local window reaches about 20%, the corresponding confidence level after harmonic mean and window fusion of the multi-scale convolution response falls between 0.2 and 0.3; if the proportion of diseased pixels exceeds 50%, the confidence level is greater than 0.5. Mapping the error function to a probability of 0.5 enables the detector output to be at the midpoint of the decision threshold in the case of mild lesions, while the probability approaches unity rapidly in the case of severe lesions. To achieve this, the linear term within the error function needs to be amplified and horizontally shifted. A slope of 1.5 means The central section of the curve is compressed, and the difference in adjacent confidences is amplified after mapping, which makes it easier to distinguish between mild diseases and healthy noise; the bias of minus 0.3 shifts the entire curve to the right, that is, Corresponding to , thus ensuring that the probability of 0.5 falls within this confidence value. In theory, this shift is equivalent to introducing a fixed value preference on the prior probability, implicitly assuming that under large forest conditions, mild diseased spots are slightly inferior to healthy backgrounds, but the probability is not extremely low.

[0060] For the hardware implementation of the error function, the system uses a 16-bit fixed-point lookup table solution. The lookup table is discretized with a fixed step size. The input interval, the upper and lower limits correspond to and . In real applications, situations where the confidence level exceeds 0.89 rarely occur. Even if it does occur, it will be mapped to the vicinity of the error function saturation value of one, which will not affect the lesion severity assessment. The storage depth selects the minimum number of nodes through the Chebyshev minimization theory, so that the approximate error is kept at the order of ten to the negative fourth power, which fully meets the forest decision-making accuracy. The table stores the final probability of the error function value after half plus the offset, so after looking up the table, only one linear interpolation and one mask comparison are required to output the probability, and the full-link delay is kept in a single-cycle pipeline. In order to verify the robustness of the mapping curve parameters to different environmental conditions, the research team conducted cross-season sampling in twelve experimental forests, and compared the mapping output with the leaf damage rate obtained by experts based on field lesion sampling and microscope counting, and obtained an average absolute error of 3.2%, with a maximum of no more than 4.5%; if the standard without bias correction is used The error for both the curve and the Sigmoid curve exceeded 6%. Further ROC analysis of the forest-level lesion warning area corresponding to a probability threshold of 0.7 showed that the area under the curve mapped by the generalized error function reached 0.93, which is 0.08 higher than the direct threshold classification without the mapped confidence level, indicating that this probability conversion improves the balanced sensitivity and specificity of the detector.

[0061] From the perspective of information theory, the error function mapping is equivalent to maximum likelihood calibration of confidence under the assumption of a Gaussian noise channel. After normalization and logarithmic mapping, the convolution response approximately obeys a zero-mean normal distribution. The slope of 1.5 compresses its variance to two-thirds of the original variance to adapt to the response diffusion caused by different vegetation leaves and different light changes; the bias of -0.3 introduces a correction term for the wrong prior, so that the probability will not be mistakenly pushed above 0.5 when there is no disease or very mild lesions. The core reason for using an error function instead of a logical Sigmoid is that the Gaussian distribution corresponds to an exponential square decay in the frequency domain, and its tail converges faster, which can reduce the risk of extreme noise events being mapped to high probabilities; at the same time The symmetry of makes the negative confidence corresponding to low probability and the positive confidence corresponding to high probability mirror each other, which is easier to debug. At the software level, to ensure numerical stability and repeatability, the system uses IEEE754 single-precision floating point for all confidence calculations, window averaging and error function mapping, and Upper and lower bounds are clipped to prevent overflow or non-numeric values ​​due to accumulated errors. Furthermore, the generalized error function parameters are hard-coded in the firmware's read-only section to prevent field upgrades from changing the probability caliber. Later fine-tuning for specific diseases or vegetation requires simply modifying the slope and bias in the configuration file. The upgrade program then writes the new constants into the lookup table, eliminating the need for convolution or gradient modules and minimizing maintenance costs.

[0062] The following is an example of converting raw digital counts to pest probability values, selecting pest-sensitive bands and With reference band , and take The pixel matrix demonstrates the upsampling pipeline; other bands and larger images can also be calculated in parallel in the same way. Raw digital counts and calibration parameters (pixel coordinates are numbered by row and column):

[0063] .

[0064] Unified dark field counting , exposure time .

[0065] Radiation conversion factor , , .

[0066] Pixel Radiance calculation (the other three points are the same): . Get the matrix .

[0067] Apparent reflectivity of the ground surface (on the day , , atmospheric path radiation and transmittance :

[0068] ;

[0069] ;

[0070] Top floor solar irradiance , , . )

[0071] .

[0072] Get .

[0073] The calculation of the lesion spectral gradient contrast index and spectral heterogeneity index is as follows: :

[0074] When the eight neighborhoods are insufficient, the sum of the grayscale differences is calculated by mirror filling and then taking the square root. .

[0075] Coupled pest and disease characteristics .

[0076] Mirror filling and bilinear interpolation (zoom in to ): Interpolation to get the middle point example .

[0077] The horizontal first-order difference at position : .

[0078] The same applies to the vertical direction; amplitude . Threshold setting , and is higher than the maximum of the eight neighborhoods, it is marked as a peak candidate. The neighborhood spectrum variance of the candidate point , threshold ; .

[0079] Laplacian kernel convolution output , take the opposite number and multiply have to .

[0080] Add back the interpolated value .by polynomial, window Pixels perform regression on rows and columns, outputting smoothed values . For band of Perform FFT on the compensation graph and find the main phase offset ; cyclic shift After inverse transformation, we get Generate sub-pixels for four-directional one-dimensional cubic splines, local Interpolation block singular values , keep the first two reconstructions According to the receptive field radius ratio Map back to the original pixel size, take pixels Final coupled pest and disease characteristics Feature Pyramid Network Convolution (Four-layer fixed kernel hardware convolution) output .

[0081] ;

[0082] .

[0083] Calculate the four layers of normalized graphs separately, retaining two layers of non-zero responses , .

[0084] .

[0085] Single-scale consistency confidence .

[0086] Sliding window mean , .

[0087] Probability Mapping

[0088] Pixel The probability value of pests and diseases is finally Wakabayashi area resolution Area per pixel , the contribution of this pixel to the forest severity integral is The severity of pests and diseases at the forest area level can be obtained by accumulating the scanned pixels of the entire forest and dividing it by the actual area of ​​the forest area.

[0089] refer to Figure 2 , Figure 2 Comparative curves of the apparent reflectance of healthy leaves and diseased areas within the wavelength range of 680nm to 800nm ​​are shown. The horizontal axis represents wavelength in nanometers (nm), and the vertical axis represents the apparent reflectance, a dimensionless parameter. The reflectance curve for the healthy leaf, represented by the solid line, shows a reflectance of approximately 0.6 at 680nm. It initially increases and then decreases with increasing wavelength, reaching a peak of approximately 0.9 around 740nm and remaining relatively stable in the near-infrared band. The reflectance curve for the diseased area, represented by the dashed line, shows a lower overall reflectance than that of the healthy leaf, approximately 0.5 at 680nm. It continues to rise with increasing wavelength, reaching approximately 1.0 at 800nm. Significant differences exist between the two curves within the disease-sensitive wavelength range of 680nm to 720nm, providing a spectral basis for disease detection. Within the reference wavelength range of 760nm to 800nm, the two curves are close, with minimal differences, validating the rationale for the selection of the reference wavelength range.

[0090] refer to Figure 3 , Figure 3The results of a comparative experiment comparing the detection of lesions on plant leaves using different wavelengths are presented. The figure shows plant leaf images using the 680nm, 700nm, 720nm, and 760nm reference wavelengths, from left to right. At the 680nm wavelength, the leaf outline is clearly visible, and the lesions appear as distinct black circular areas. Lesion clarity reaches its highest level, enabling accurate identification of their location, size, and morphological characteristics. At the 700nm wavelength, the leaf morphology remains clear, and the lesions can still be effectively identified. However, the contrast between lesions and healthy areas is slightly reduced compared to the 680nm wavelength, resulting in moderate lesion clarity. At the 720nm wavelength, the visibility of the lesions decreases further, and the difference between lesions and healthy leaf tissue becomes less distinct. Lesion clarity is low, and only the general outline of the lesions can be vaguely discerned. At the 760nm reference wavelength, the lesions are completely invisible, with their reflectance characteristics essentially identical to those of healthy areas, making their location indistinguishable visually.

[0091] refer to Figure 4 , Figure 4 From left to right, the calculation results of the lesion spectral gradient contrast index, spectral heterogeneity index, and coupled pest and disease signature are shown. The lesion spectral gradient contrast index is calculated by calculating the first-order spectral gradient of each pixel in the disease-sensitive band, multiplying the distance between adjacent center wavelengths by the first-order spectral gradient, and then comparing the value with the reflectance of the reference band. In this feature map, the values ​​in the lesion area are significantly higher than those in the healthy area, with values ​​ranging from 0.7 to 0.9 for lesion pixels, while values ​​for pixels in the healthy area typically range from 0.0 to 0.4. The high-value areas against the black background accurately correspond to the lesion locations in the original image. The spectral heterogeneity index is calculated by taking the square root of the sum of the squared differences in the apparent reflectance of the ground object between eight neighboring pixels and the central pixel within the preset analysis band of 720 nm. Due to the significant spectral difference between the lesion area and the surrounding healthy tissue, the heterogeneity index is higher, ranging from 0.7 to 0.9. The healthy area is spectrally more uniform, with lower heterogeneity index values, typically ranging from 0.1 to 0.5. The coupled pest and disease signature is derived by multiplying the spectral gradient contrast index of the lesion with the spectral heterogeneity index. This signature integrates spectral gradient information and spatial heterogeneity. In the coupled signature map, the characteristic values ​​of the lesion area are further enhanced, reaching 0.56 to 0.72, while the characteristic values ​​of the healthy area are effectively suppressed, with values ​​in most areas ranging from 0.00 to 0.20. This enhancement significantly improves the separability of the lesion area from the healthy area, providing high-quality input features for subsequent deep learning detection. Figure 3The following diagram illustrates the analysis of an eight-neighborhood pixel. The central pixel represents the diseased area, with a value of 0.8. The eight surrounding pixels represent the healthy area, with values ​​ranging from 0.0 to 0.3. By calculating the reflectance difference between the central pixel and its eight neighbors, the spectral heterogeneity index (SHI) is 0.65, which accurately reflects the spatial heterogeneity of the diseased area.

[0092] Figure 5 This image demonstrates the processing effect of upsampling coupled pest and disease features onto each layer of the feature pyramid network, along with a comparison of detection results. The input coupled pest and disease features in the upper left corner maintain the characteristic distribution of high eigenvalues ​​in diseased areas and low eigenvalues ​​in healthy areas, providing high-quality input data for the feature pyramid network. The output of the smallest layer of the feature pyramid network shows that after network processing, the eigenvalues ​​of diseased areas are further enhanced, reaching 0.75 to 0.88, while the eigenvalues ​​of healthy areas remain at a lower level of 0.05 to 0.45. The smallest layer has the largest receptive field, capturing a wider range of contextual information, which helps improve the accuracy of lesion detection. The output of the next smallest layer shows similar feature enhancement, with eigenvalues ​​ranging from 0.71 to 0.84 in diseased areas and 0.03 to 0.41 in healthy areas. While maintaining a large receptive field, the next smallest layer provides finer spatial resolution, helping to accurately locate lesion boundaries. The outputs of other layers demonstrate the network's feature extraction capabilities at different scales, with eigenvalues ​​ranging from 0.68 to 0.80 in diseased areas and 0.02 to 0.38 in healthy areas. The normalized response map is constructed by calculating the range and mean of the convolution output for the same pixel at each layer of the feature pyramid network, subtracting the mean from each convolution output, and dividing by the range to obtain a normalized response within the interval [-1, 1]. In the normalized response map, response values ​​in diseased areas are close to 1.0 or -1.0, indicating that these areas have consistent high response characteristics across layers. Response values ​​in healthy areas are close to 0, indicating that the response in these areas is relatively stable and small.

[0093] Figure 6The detailed process of Sobel gradient amplitude processing is demonstrated. The gradient amplitude before processing shows that the lesion area has a higher gradient value of 0.72 to 0.85, while the healthy area has a lower gradient value of 0.03 to 0.12. By setting the noise floor threshold of 0.15 for filtering, the gradient values ​​below the threshold are set to zero, retaining the significant lesion features and effectively suppressing the interference of background noise. The multi-scale fusion confidence calculation process shows the calculation results of the 5×5 window and 9×9 window spatial mean. The 5×5 window spatial mean captures the feature distribution of the local neighborhood, and the 9×9 window spatial mean provides a wider range of contextual information. Through harmonic mean calculation, the confidence of the lesion area is significantly improved to 0.70, and the confidence of the healthy area remains at a low level of 0.21 to 0.28, achieving effective enhancement of the lesion area and effective suppression of the background area.

[0094] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The multispectral imaging and deep learning intelligent detection system for pests and diseases is characterized by: The system comprises: a multispectral radiance calibration unit, an image analysis and processing unit and a pest and disease probability detection unit; the multispectral radiance calibration unit is used to perform multi-band imaging of the vegetation leaf surface in the forest area to obtain a multispectral image of the forest area, obtain the original digital count and dark field digital count of each pixel therein, and deduct the dark field digital count from the original digital count in combination with the image sensor parameters to convert it into the radiance of the corresponding band; the image analysis and processing unit is used to convert the radiance into the apparent reflectance of the ground object; the first-order spectral gradient of the apparent reflectance of the ground object is calculated for each pixel in the disease-sensitive band, and the corresponding The lesion spectral gradient contrast index is obtained by multiplying the neighboring center wavelength distance by the first-order spectral gradient and comparing it with the reflectance of the reference band. At the same time, the square root of the sum of the squares of the apparent reflectance differences between the eight neighboring pixels and the center pixel is calculated in the preset analysis band to obtain the spectral heterogeneity index. The lesion spectral gradient contrast index is then multiplied by the spectral heterogeneity index to obtain the coupled pest and disease characteristics. The pest and disease probability detection unit is used to map the coupled pest and disease characteristics to each layer of the feature pyramid network by upsampling to obtain the convolution output of each layer; and map the convolution output of each layer to the pest and disease probability value based on the generalized error function.

2. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 1, characterized in that: The system also includes: a pest severity assessment unit, which is used to multiply and accumulate the pest probability value of the pixel in the forest area multispectral image of each forest area with the ground area corresponding to the pixel, and divide it by the total area of ​​the forest area to obtain the pest severity of the forest area.

3. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 2, characterized in that: The disease-sensitive wavelength band is a number of center wavelengths evenly distributed within the range of 680 nanometers to 720 nanometers; the reference wavelength band is a number of center wavelengths evenly distributed within the range of 760 nanometers to 800 nanometers; The preset analysis band is 720 nanometers; the image sensor parameters include: image sensor radiation conversion factor and exposure time.

4. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 3, characterized in that: No. Pixels in the band Radiance for: ； in, For the Pixels in the band The raw digital count of ; For the band Dark field digital counts; is the image sensor radiation conversion factor; is the exposure time.

5. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 4, characterized in that: Radiance Converted to apparent reflectance of ground objects The formula is: ； in, For the Pixels in the band The apparent reflectivity of the ground object; For the band Atmospheric path radiation; is the average relative distance between the sun and the earth on that day; For the band Atmospheric transmittance; For the band Top-floor solar irradiance; is the solar zenith angle.

6. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 5, characterized in that: The process of mapping the coupled pest and disease features to each layer of the feature pyramid network by upsampling in the disease and pest probability detection unit includes: performing mirror filling on all sides of the coupled pest and disease features, and the filling width is equal to the receptive field radius of the minimum layer of the feature pyramid network; using bilinear interpolation on each spectral channel to enlarge the coupled pest and disease features to half of the original pixel size; calculating the horizontal first-order difference and the vertical first-order difference on the bilinear interpolation result, performing a 3×3 Laplace inverse compensation operation on the gradient peak value exceeding the set threshold to obtain a feature map; performing a fifth-order Savitzky-Golay spatial domain smoothing on the feature map, with the window length set to 9 pixels, to obtain a smoothed feature map; calculating the smoothed feature map on each spectral channel A two-dimensional discrete Fourier transform is calculated to obtain the frequency domain feature map. A sub-pixel circular shift is performed on the phase spectrum in the frequency domain to keep the main phase difference between adjacent pixels in the range of −π / 8 to π / 8. An inverse transform is then performed to restore the spatial domain feature map to ensure that the amplified features have no phase jumps in the row and column directions. Four-directional one-dimensional cubic spline interpolation is introduced to supplement the sub-pixel intensity values ​​at the positions with one pixel intervals in each row and column. Double singular value decomposition is then used to perform low-rank approximation on the local 4×4 interpolation blocks to eliminate local ringing artifacts caused by high-order interpolation, thereby obtaining the corrected feature map. Based on the ratio of the receptive field radius of the smallest layer and the second smallest layer of the feature pyramid network, the corrected feature map is aligned with the original pixel size in a 1:1 ratio and mapped to each layer of the feature pyramid network.

7. The multispectral imaging and deep learning intelligent detection system for pests and diseases according to claim 6, characterized in that: The 3×3 Laplace inverse compensation operation is performed on the gradient peak that exceeds the set threshold. The process of obtaining the feature map includes: calculating the absolute value of the gradient amplitude for the same pixel in the horizontal first-order difference and vertical first-order difference of each spectral channel, and comparing it with the preset threshold; when the absolute value of the gradient amplitude is greater than the threshold and greater than the corresponding value of its eight neighboring pixels at the same time, the pixel is marked as a gradient peak candidate point; for each gradient peak candidate point, the spectral variance of the coupled pest and disease characteristics in its 3×3 neighborhood is counted, and the ratio of the variance to the threshold is used to generate a feature map. The adaptive Laplace coefficient ranges from 0.5 to 2. A 3×3 two-dimensional Laplace kernel [−1,−1,−1;−1,8,−1;−1,−1,−1][−1,-1,-1;-1,8,-1;-1,-1,-1][−1,−1,−1;−1,8,−1;−1,−1,−1] is applied to the 3×3 neighborhood of the gradient peak candidate point, the output is negated and multiplied by the adaptive Laplace coefficient, and then added pixel by pixel with the coupled pest and disease features to obtain the feature map.

8. The multispectral imaging and deep learning intelligent detection system for pests and diseases according to claim 7, characterized in that: The pest and disease probability detection unit calculates the extreme difference of the convolution output of the same pixel at each layer of the feature pyramid network and mean , subtract the mean from each convolution output and divide it by the range to get the interval The normalized response map within the image is calculated; the 3×3 Sobel gradient magnitude is calculated on the normalized response map, and the pixel responses whose magnitude is lower than the preset noise floor threshold are set to zero; the remaining non-zero responses are Mapped to log-likelihood value through a fixed-parameter bi-segment logarithmic function ; Take the harmonic average of the log-likelihood values ​​of the same pixel in all layers to obtain the single-scale consistency confidence of the pixel ; Then calculate in 5×5 and 9×9 sliding windows respectively The spatial mean of and , and finally take As the multi-scale fusion confidence; multi-scale fusion confidence Input the generalized error function to obtain the probability value of pests and diseases.

9. The multispectral imaging and deep learning intelligent detection system for pest and disease images according to claim 8, characterized in that: The formula for the generalized error function is: ； in, is the probability value of pests and diseases.

Images