Method based on unmanned aerial vehicle-mounted snapshot type hyperspectral imaging system

Abstract

The invention discloses a method based on an unmanned aerial vehicle-mounted snapshot-type hyperspectral imaging system, and the method comprises the steps: carrying out the hardware innovation of synchronously obtaining a space image and spectral information through single-frame exposure, combining with the synergistic effect of a metasurface light splitting structure and a CCD array, breaking through the dependence of a conventional push-broom system on a flight attitude, and on this basis, carrying out the recognition of a hyperspectral image. Innovatively fusing an NDVI vegetation index and a gray level co-occurrence matrix texture parameter, and constructing a disease feature library of a multi-dimensional feature vector; compared with the prior art, the multi-dimensional analysis system enables the detection rate of wheat scab and other tiny disease spots to be improved, advanced prediction of a disease diffusion path is achieved through a Markov chain and logistic regression dual-mode prediction model and coupling analysis of a disease density transfer matrix and meteorological factors, and a variable spraying system is matched, so that the disease diffusion path is predicted in advance. And through mapping of a disease probability thermodynamic diagram, precise pesticide application with flow dynamic adjustment is realized.

Description

Technical Field

[0001] The present invention relates to a spectral imaging system, and in particular to a method based on an unmanned aerial vehicle-mounted snapshot hyperspectral imaging system. Background Art

[0002] In recent years, drones, owing to their flexibility, ease of operation, and low cost, have been widely used in a variety of fields, including agricultural monitoring, geological exploration, forest surveys, urban planning, and environmental monitoring. They have become a crucial component of the next-generation aerial remote sensing observation platform. Hyperspectral imaging technology, which can acquire reflectance or transmission spectral information of target objects across multiple continuous wavelengths, offers powerful capabilities for material identification and composition analysis. Integrating hyperspectral imaging systems onto drone platforms enables low-altitude, close-range, and high-frequency data acquisition while maintaining high resolution, addressing the limitations of traditional remote sensing methods in terms of mobility and real-time performance. This "flexible aerial platform + precise spectral imaging" technology offers a novel solution for complex tasks such as agricultural growth analysis, water pollution monitoring, and disaster site assessments. Currently, drone-based hyperspectral systems have been applied in a variety of real-world scenarios, demonstrating promising development prospects and practical value.

[0003] However, in practical applications, the hyperspectral imaging systems used by drones are mostly push-broom systems. These systems rely on stable platform flight and line-by-line scanning by linear array detectors to complete imaging, requiring high flight attitude and trajectory accuracy. Although some drone platforms are equipped with attitude-stabilized gimbals, which can mitigate the effects of camera shake to a certain extent, this can significantly impact image quality and recognition accuracy. Existing drones are also being used to identify farmland diseases, but the recognition rate is still insufficient due to the specific conditions of the fields. Summary of the Invention

[0004] The present invention overcomes the deficiencies of the prior art and provides a method based on an unmanned aerial vehicle (UAV)-mounted snapshot hyperspectral imaging system.

[0005] To achieve the above-mentioned object, the technical solution adopted by the present invention is: a method based on an unmanned aerial vehicle-mounted snapshot hyperspectral imaging system, comprising: S1, using a snapshot hyperspectral camera to simultaneously collect a two-dimensional spatial image and corresponding spectral information of a ground target area through single-frame exposure, and preprocessing;

[0006] S2, extracting spectral and spatial features from the information preprocessed in S1;

[0007] S3, construct a disease identification and classification module based on the features of S2;

[0008] S4. Generate a disease distribution map through the disease identification and classification module and predict the transmission path;

[0009] S5. Plan the drone path based on the disease distribution map and infection path.

[0010] In a preferred embodiment of the present invention, in step S1, the snapshot hyperspectral camera model is PX-VIS-1, which is installed under the drone platform through an adjustable gimbal to ensure that the lens is vertically facing downward. It is integrated with a 5G / LoRa dual-mode wireless transmission module to support real-time data backhaul.

[0011] In a preferred embodiment of the present invention, the snapshot hyperspectral camera adopts a metasurface spectroscopic structure, which is composed of a periodic nano-unit array on a subwavelength scale to achieve efficient spectral separation.

[0012] In a preferred embodiment of the present invention, in step S2, the specific steps are: extracting multi-dimensional features from the pre-processed data and constructing a disease recognition feature library:

[0013] Extract spectral features: select disease-sensitive bands and calculate the vegetation index (NDVI). This quantifies the photosynthetic activity of vegetation. The NDVI value of healthy vegetation is usually greater than 0.7, while the NDVI value of diseased or withered vegetation is lower.

[0014] Extract spatial features: Use gray-level co-occurrence matrix to calculate texture contrast, and extract morphological features to extract the geometric shape features of the diseased area to assist in distinguishing lesions from normal tissues:

[0015] Among them, Circularity is the circularity, close to 1 is a circular lesion, deviating from 1 is an irregular shape, Area is the area, that is, the number of pixels in the lesion;

[0016] The extracted spatial features and spectral features are fused to obtain a multi-dimensional feature vector.

[0017] In a preferred embodiment of the present invention, in step S3, the specific steps of constructing the disease identification and classification module are: using the CNN model architecture and SVM classification model and performing model optimization; wherein the model optimization is carried out by training the model through the cross entropy loss function and the Adam optimizer.

[0018] In a preferred embodiment of the present invention, in step S4, the specific steps of generating a disease heat map based on the classification results are as follows: inputting the fused feature vector, calculating the probability value of each pixel being healthy, mildly diseased, or severely diseased;

[0019] Generate a binary mask by setting the disease probability threshold:

[0020] Smoothing of the calculated binary mask: M clean=Close(Open(M)) to obtain a cleaner segmentation result of the diseased area.

[0021] In a preferred embodiment of the present invention, in step S4, the specific steps of predicting the propagation path are:

[0022] Among them, P i,j is the probability of disease transmission from region i to region j, and the disease density of adjacent region j is the proportion of diseased pixels in region j.

[0023] In a preferred embodiment of the present invention, in step S5, the specific steps for planning the drone path using the disease distribution map and the infection path are as follows: the binary mask and the disease transmission probability map generated in step S4 are input to generate a comprehensive decision map, and the drone prioritizes covering areas with high transmission risks while reducing repeated flight paths.

[0024] In a preferred embodiment of the present invention, the spraying process of the drone can be controlled, and the spraying amount can be dynamically adjusted according to the probability of the disease: Q j =Q base ×(1+k·C j ), so the diseased area (C j =1) Spraying amount can reach Q base ×1.3, healthy area (C j =0) Only the base flow rate is sprayed.

[0025] A snapshot hyperspectral imaging system based on an unmanned aerial vehicle, and a method based on the snapshot hyperspectral imaging system based on an unmanned aerial vehicle:

[0026] A snapshot hyperspectral imaging system based on drones:

[0027] Data acquisition module: used to synchronously collect two-dimensional images and spectral data of ground targets;

[0028] Wireless transmission device, including 5G / LoRa dual-mode wireless transmission unit, used to transmit data back to the ground terminal in real time;

[0029] Data processing modules, including edge computing units, are used to deploy radiation correction, disease identification, and path planning algorithms;

[0030] The pesticide application execution module includes a variable spray system for adjusting the spray volume.

[0031] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0032] (1) Through hardware innovation of synchronously acquiring spatial images and spectral information through single-frame exposure, combined with the synergistic effect of metasurface spectroscopic structure and CCD array, the traditional push-broom system's dependence on flight posture is broken through. On this basis, the NDVI vegetation index (healthy vegetation > 0.7 vs diseased vegetation < 0.3) and grayscale co-occurrence matrix texture parameters are innovatively integrated to construct a disease feature library of multi-dimensional feature vectors. Compared with existing technologies, this multi-dimensional analysis system improves the detection rate of tiny lesions such as wheat fusarium wilt and reduces the misjudgment rate compared to single spectral analysis, providing a new technical path for intelligent diagnosis of diseases in complex farmland scenarios.

[0033] (2) Through the dual-mode prediction model of Markov chain and logistic regression, and through the coupling analysis of disease density transfer matrix and meteorological factors, 24-hour advance prediction of disease diffusion path is achieved, and the variable spray system is paired, and through disease probability heat map mapping, precise spraying with dynamic flow regulation is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments described in the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive efforts.

[0035] Figure 1 is a flow chart of a preferred embodiment of the present invention;

[0036] Figure 2 It is a module working diagram of a preferred embodiment of the present invention; DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0039] Application Overview: Wheat fusarium head blight is a worldwide fungal disease caused by Fusarium graminearum. The pathogen can be spread by wind and rain, infecting the wheat ear during the flowering period, causing the grains to shrivel and mold. This not only results in yield reduction (generally 10%-30% loss, and up to 50% or more in severe cases), but also produces vomitoxin (DON), which endangers the health of humans and animals. When routinely used with push-broom hyperspectral systems, drones equipped with wind disturbances or undulating terrain can easily cause spectral distortion. Traditional methods rely on RGB images or a single spectral band (such as the red edge of visible light), resulting in insufficient disease recognition rates and a lack of ability to predict disease spread. They rely on manual experience to determine the timing of pesticide application, resulting in missed prevention and control windows or excessive use of pesticides.

[0040] like Figure 1 As shown, a method based on an unmanned aerial vehicle-mounted snapshot hyperspectral imaging system includes:

[0041] S1. Using a snapshot hyperspectral camera, a two-dimensional spatial image of the ground target area and the corresponding spectral information are collected and preprocessed through single-frame exposure.

[0042] S2, extracting spectral and spatial features from the information preprocessed in S1;

[0043] S3, construct a disease identification and classification module based on the features of S2;

[0044] S4. Generate a disease distribution map through the disease identification and classification module and predict the transmission path;

[0045] S5. Plan the drone path based on the disease distribution map and infection path.

[0046] In step S1, the snapshot hyperspectral camera model is PX-VIS-1, which is installed under the UAV platform through an adjustable gimbal to ensure that the lens is vertically facing downward. It is integrated with a 5G / LoRa dual-mode wireless transmission module to support real-time data backhaul;

[0047] More specifically, for single-frame exposure imaging: Data cube = metasurface spectrometer × CCD array (spectral resolution ≤ 5nm, field of view ≥ 55°)

[0048] The 400-1000nm continuous spectrum and RGB image are acquired through a single exposure of the metasurface structure.

[0049] In step S1, the specific steps of preprocessing are: radiometric correction of the two-dimensional spatial image and spectral data: Among them, L λ is the corrected radiance, DN λ is the original digital value, DN dark is the dark current calibration value, DN whiteis the white plate calibration value, ρ std is the reflectivity of the standard reference plate;

[0050] After radiometric correction, the spectral curve is smoothed using a Savitzky-Golay filter: Among them, ω i is the polynomial fitting weight coefficient, and M is the window half-width.

[0051] In step S2, the specific steps are: extracting multi-dimensional features from the preprocessed data and building a disease recognition feature library:

[0052] Extract spectral features: Select the disease-sensitive band (the abnormal reflectance area of ​​wheat scab in 750–850 nm) and calculate the vegetation index NDVI: Among them, ρ NIR is the spectral reflectance in the near-infrared band (750–1000 nm), reflecting the health of vegetation, ρ RED The NDVI is the spectral reflectance in the red light band (600–700 nm) and is significantly affected by chlorophyll absorption. It quantifies the photosynthetic activity of vegetation by calculation. The NDVI value of healthy vegetation is usually greater than 0.7, while the NDVI value of diseased or withered vegetation is lower.

[0053] Extract spatial features: Use gray-level co-occurrence matrix (GLCM) to calculate texture contrast: Through morphological feature extraction, the geometric shape features of the diseased area are extracted to assist in distinguishing lesions from normal tissues: Among them, Circularity is the circularity, close to 1 is a circular lesion, deviating from 1 is an irregular shape, Area is the area, that is, the number of pixels in the lesion;

[0054] The extracted spatial features and spectral features are fused to obtain a multi-dimensional feature vector:

[0055] Fused multidimensional vector = [NDVI, Contrast, Circularity].

[0056] In step S3, the specific steps of building the disease identification and classification module are as follows: through the CNN model architecture and

[0057] SVM classification model and model optimization;

[0058] CNN model:

[0059]

[0060] Among them, σ is the ReLU activation function, W is the convolution kernel weight, b is the bias term, and GAP is the global average pooling;

[0061] SVM classification model:

[0062] Among them, K(x i ,x) is the Gaussian kernel function, α i is the Lagrange multiplier, and b is the bias term.

[0063] Model optimization uses the cross entropy loss function and Adam optimizer to train the model:

[0064]

[0065] Among them, η is the learning rate and β2 is the momentum parameter.

[0066] In step S4, a disease heat map is generated based on the classification results: the fused feature vector is input and the probability value of each pixel being healthy, mildly diseased, or severely diseased is calculated:

[0067] P class =Softmax(W fc GAP(X)+b fc ); where W fc is the weight of the fully connected layer, b fc is the bias term, GAP(X) is the global average pooling operation, which compresses the spatial features into a global feature vector, and X is the input feature map;

[0068] By setting the disease probability threshold (P class =0.7), generate a binary mask:

[0069] Smoothing of the calculated binary mask: M clean = Close(Open(M)), where M is the initial binary mask (the diseased area is marked as 1 and the healthy area is 0), Open(M) is an opening operation (first erosion and then expansion), which is used to smooth the boundary and remove isolated noise, and Close(M) is a closing operation (first expansion and then erosion), which is used to fill the internal voids of the diseased area. clean is the optimized binary mask (cleaner disease area segmentation result).

[0070] In step S4, the specific steps of predicting the transmission path are to use Markov chain to calculate the probability of disease transmission between regions: Among them, P i,j is the probability of disease transmission from region i to region j, and the disease density of adjacent neighboring region j is the proportion of diseased pixels in region j (disease area / total area);

[0071] Risk prediction via logistic regression:

[0072] Among them, P(y=1) is the probability that area y is high risk (high probability of disease spread), X i are the input environmental characteristics (such as wind speed, humidity, soil pH value), μ0, μ1, ..., μn are the weight coefficients of the logistic regression model, which are obtained by fitting the training data.

[0073] In step S5, the specific steps for planning the drone path based on the disease distribution map and the infection path are as follows:

[0074] The binary mask generated in step S4 (with diseased areas marked as 1 and healthy areas as 0) and the disease transmission probability map are input to generate a comprehensive decision map. The drones will prioritize covering areas with high transmission risk while reducing duplicate flight paths.

[0075] f(n)=g(n)+λ·h(n),

[0076] Where f(n) is the total cost function of node n, which determines the search priority; g(n) is the actual moving cost (path distance) from the starting point to node n; h(n) is the estimated cost (Euclidean distance) from node n to the target node; λ is the weight coefficient (λ = 0.7, balancing coverage and efficiency);

[0077] Here, the starting point of the drone is set as the starting point, and the high-risk area of ​​the disease is set as the target node set. The node with the smallest total cost f(n) is preferentially selected for expansion. After the shortest path is generated, the path order is adjusted according to the disease distribution map to ensure that the high-risk area is covered first. By planning the drone to pass through the diseased area multiple times, the drone can identify and enhance data collection multiple times, and the sprayer can be preset to spray the diseased area.

[0078] Furthermore, the spray volume during the drone spraying process can be controlled, and the spray volume can be dynamically adjusted according to the probability of disease: Q j =Q base ×(1+k·C j ),

[0079] Among them, Q j is the spraying flow rate of the j-th diseased area, Q base is the basic spray volume (default flow when there is no disease), k is the adjustment coefficient (k = 0.3, controlling the flow rate increase), C j is the disease coverage rate of the jth region (0≤C j ≤1). This will make the diseased area (C j =1) Spraying amount can reach Q base ×1.3, healthy area (C j =0) Only the base flow rate is sprayed.

[0080] like Figure 2 As shown in the figure, a snapshot hyperspectral imaging system based on drone:

[0081] Data acquisition module: used to synchronously collect two-dimensional images and spectral data of ground targets;

[0082] Wireless transmission device, including 5G / LoRa dual-mode wireless transmission unit, used to transmit data back to the ground terminal in real time;

[0083] Data processing module, including edge computing unit (powered by NVIDIA Jetson Xavier NX), used to deploy radiation correction, disease identification and path planning algorithms;

[0084] The pesticide application execution module includes a variable spray system for adjusting the spray volume.

[0085] Specifically, the snapshot hyperspectral imaging camera uses a single-frame exposure method to simultaneously obtain the two-dimensional image and spectral information of the ground target in one imaging process.

[0086] The camera adopts a metasurface spectroscopic structure, which is composed of a subwavelength-scale periodic nanometer unit array to achieve efficient spectrum separation.

[0087] More specifically, the hyperspectral imaging camera integrates a spectral reconstruction algorithm chip that parses and processes the original encoded image in real time, rapidly reconstructing a three-dimensional hyperspectral data cube containing both spatial and spectral information. Each pixel corresponds to a complete spectral curve, encompassing reflectance information across the entire wavelength range, with high spectral resolution and high spatial consistency.

[0088] Furthermore, the system integrates a high-speed wireless transmission module and a ground-based receiving terminal, supporting real-time downlink and remote analysis of hyperspectral data. The ground-based receiving module includes an image preprocessing unit, a spectral reconstruction module, and an intelligent target recognition and analysis unit. These units enable rapid identification and spatial positioning of targets in diverse application scenarios, making them suitable for efficient response in a variety of tasks, including large-scale target inspections, emergency monitoring, and resource surveys.

[0089] The above description is based on the ideal embodiment of the present invention. Based on the above description, relevant personnel can make various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the content of the specification and must be determined according to the scope of the claims.

Claims

1. A method based on an unmanned aerial vehicle snapshot hyperspectral imaging system, characterized in that: include: S1. Using a snapshot hyperspectral camera, a two-dimensional spatial image of the ground target area and the corresponding spectral information are collected and preprocessed through single-frame exposure. S2, extracting spectral and spatial features from the information preprocessed in S1; S3, construct a disease identification and classification module based on the features of S2; S4. Generate a disease distribution map through the disease identification and classification module and predict the transmission path; S5. Plan the drone path based on the disease distribution map and infection path.

2. The method according to claim 1, wherein: In step S1, the snapshot hyperspectral camera model is PX-VIS-1, which is installed under the drone platform through an adjustable gimbal to ensure that the lens is facing vertically downward. It is integrated with a 5G / LoRa dual-mode wireless transmission module to support real-time data backhaul.

3. The method according to claim 1, wherein: The snapshot hyperspectral camera adopts a metasurface spectroscopic structure, which is composed of a sub-wavelength-scale periodic nanometer unit array to achieve efficient spectrum separation.

4. The method according to claim 1, wherein: In step S2, the specific steps are: extracting multi-dimensional features from the preprocessed data and building a disease recognition feature library: Extract spectral features: Select disease-sensitive bands and calculate the vegetation index (NDVI). This quantifies the photosynthetic activity of vegetation. The NDVI value of healthy vegetation is usually greater than 0.7, while the NDVI value of diseased or withered vegetation is lower. Extract spatial features: Use gray-level co-occurrence matrix to calculate texture contrast, and extract morphological features to extract the geometric shape features of the diseased area to assist in distinguishing lesions from normal tissues: Among them, Circularity is the circularity, close to 1 is a circular lesion, deviating from 1 is an irregular shape, Area is the area, that is, the number of pixels in the lesion; The extracted spatial features and spectral features are fused to obtain a multi-dimensional feature vector.

5. The method according to claim 1, wherein: In step S3, the specific steps of constructing the disease identification and classification module are as follows: using the CNN model architecture and SVM classification model and performing model optimization; wherein the model optimization is carried out by training the model through the cross entropy loss function and the Adam optimizer.

6. The method according to claim 1, wherein: In step S4, the disease heat map is generated based on the classification results. The specific steps are as follows: inputting the fused feature vector and calculating the probability value of each pixel being healthy, mildly diseased, or severely diseased; Generate a binary mask by setting the disease probability threshold: Smoothing of the calculated binary mask: M clean =Close(Open(M)) to obtain a cleaner segmentation result of the diseased area.

7. The method according to claim 6, wherein: In step S4, the specific steps of predicting the propagation path are: Among them, P i,j is the probability of disease transmission from region i to region j, and the disease density of adjacent region j is the proportion of diseased pixels in region j.

8. The method according to claim 1, wherein: In step S5, the specific steps for planning the drone path using the disease distribution map and the infection path are as follows: input the binary mask and disease transmission probability map generated in step S4 to generate a comprehensive decision map. The drone will give priority to covering areas with high transmission risks while reducing repeated flight paths.

9. The method according to claim 8, wherein: The spray volume of the drone spraying process can be controlled and the spray volume can be dynamically adjusted according to the probability of disease: Q j =Q base ×(1+k·C j ), so the diseased area (C j =1) Spraying amount can reach Q base ×1.3, healthy area (C j =0) Only the base flow rate is sprayed.

10. A snapshot hyperspectral imaging system based on an unmanned aerial vehicle, and a method based on a snapshot hyperspectral imaging system based on an unmanned aerial vehicle according to any one of claims 1 to 9, characterized in that: A snapshot hyperspectral imaging system based on drones: Data acquisition module: used to synchronously collect two-dimensional images and spectral data of ground targets; Wireless transmission device, including 5G / LoRa dual-mode wireless transmission unit, used to transmit data back to the ground terminal in real time; Data processing modules, including edge computing units, are used to deploy radiation correction, disease identification, and path planning algorithms; The pesticide application execution module includes a variable spray system for adjusting the spray volume.

Images