A water and grass coverage detection method based on airborne hyperspectrum
By using airborne hyperspectral imaging equipment and neural network models, the problems of time-consuming, labor-intensive, and inaccurate traditional aquatic plant detection methods have been solved, enabling rapid and accurate detection of aquatic plant coverage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGNAN UNIV
- Filing Date
- 2024-05-30
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional methods for detecting aquatic plant growth rely on manual observation, which is time-consuming, labor-intensive, and inaccurate. Remote sensing methods based on RGB or multispectral images are also difficult to accurately identify aquatic plants, resulting in low detection accuracy.
Aquatic plant coverage data were collected using an airborne hyperspectral imaging device. Feature bands were selected by step-by-step band selection, and fused features were constructed by combining spectral continuum removal transformation. A neural network model was used to train an aquatic plant recognition model to achieve pixel-level classification and detection.
It achieves rapid and accurate detection of aquatic plant coverage, and can accurately distinguish aquatic plants from other aquatic plants in complex backgrounds, thus improving the automation and accuracy of the detection.
Smart Images
Figure CN118691989B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of hyperspectral technology, and in particular to a method for detecting aquatic plant coverage based on airborne hyperspectral imaging. Background Technology
[0002] Aquatic plants play an important role in crab farming. They not only serve as an important food source and shelter for crabs, but also help increase dissolved oxygen levels and purify water quality. This effectively prevents soil erosion and reduces water pollution, providing a solid foundation for the healthy development of the crab pond ecosystem. Therefore, monitoring the growth of aquatic plants is beneficial for assessing the development of the crab pond ecosystem.
[0003] Traditional methods for detecting aquatic plant growth mainly rely on manual observation, which is time-consuming, labor-intensive, and highly susceptible to subjective influence. With technological advancements, remote sensing methods based on spectral analysis have been gradually applied to aquatic plant growth detection. However, the chlorophyll in aquatic plants makes the water appear green overall, and the colors of different aquatic plants in RGB or multispectral images obtained through remote sensing are often very similar, making it difficult to accurately analyze and identify aquatic plants from the images. This results in low detection accuracy and hinders widespread adoption. Summary of the Invention
[0004] This application addresses the aforementioned problems and technical requirements by proposing a method for detecting aquatic plant coverage based on airborne hyperspectral imaging. The technical solution of this application is as follows:
[0005] A method for detecting aquatic plant cover based on airborne hyperspectral imaging, the method comprising:
[0006] The hyperspectral images of samples from water bodies with aquatic plants were captured using a hyperspectral imaging device mounted on a drone, and the classification results of each pixel in the hyperspectral images of the samples were labeled.
[0007] The radiance of each pixel in the hyperspectral image of the sample is converted into the surface reflectance, and the corresponding sample reflectance spectral curve is obtained.
[0008] Feature bands are selected based on the characteristics of the sample reflectance spectrum curve;
[0009] The spectral index is calculated after performing a continuum removal transformation on the sample reflectance spectral curve.
[0010] The sample hyperspectral image is divided into several training sample images. The spectral features of each training sample image in the feature band and the spectral index of each training sample image are extracted respectively. The spectral features and spectral index of each training sample image are concatenated in the channel dimension to obtain the fusion feature of the training sample image.
[0011] Using the fusion features of each training sample image as input and the classification results of pixels in the training sample image as output, a water plant recognition model is trained based on a neural network model.
[0012] The classification results of each pixel in the hyperspectral image of the water area to be detected are determined by using an aquatic plant recognition model, and the proportion of pixels belonging to the aquatic plant category in the total number of pixels is calculated to obtain the aquatic plant coverage of the water area to be detected.
[0013] The beneficial technical effects of this application are:
[0014] This application discloses a method for detecting aquatic plant coverage based on airborne hyperspectral imaging. This method utilizes a drone equipped with a hyperspectral imaging device to acquire sample hyperspectral images. The hyperspectral images are then processed by band selection to identify characteristic bands for data dimensionality reduction. A fusion feature combining spectral continuum removal transformation and spectral indices is constructed. This fusion feature is used to train an aquatic plant recognition model, which can perform pixel-level classification detection on hyperspectral images, thereby quickly obtaining aquatic plant coverage detection results. Hyperspectral images provide rich spectral information. This detection method based on hyperspectral images can accurately distinguish aquatic plants from other aquatic plants of similar color. Furthermore, the application of fusion features enables accurate and rapid pixel-level classification even against complex backgrounds. This method has a high degree of automation and high detection accuracy, making it highly valuable for practical applications.
[0015] This method uses a step-by-step band selection method to screen feature bands, and based on the idea that the class separability is inversely proportional to the feature correlation, it introduces an improved integrated JM distance calculation method to evaluate the degree of distinguishability between different classes. This can enhance the difference and separability between aquatic plants and other classes, thereby improving classification accuracy and the precision of aquatic plant coverage detection. Attached Figure Description
[0016] Figure 1 This is a flowchart of a method for detecting aquatic plant coverage based on airborne hyperspectral imaging, according to an embodiment of this application.
[0017] Figure 2 This is a flowchart of a method for screening characteristic bands using a step-by-step band selection method in one embodiment.
[0018] Figure 3 This is a diagram of the neural network model used to train the aquatic plant recognition model. Detailed Implementation
[0019] The specific embodiments of this application will be further described below with reference to the accompanying drawings.
[0020] This application discloses a method for detecting aquatic plant cover based on airborne hyperspectral imaging. Please refer to [link / reference]. Figure 1 The flowchart shown illustrates the aquatic plant coverage detection method, which includes:
[0021] Step 1: Take hyperspectral images of water samples with aquatic plants using a hyperspectral imaging device mounted on a drone, and label the classification results of each pixel in the hyperspectral images. The categories of the pixels in the hyperspectral images include aquatic plants and several other categories, such as various other aquatic plants, aquatic animals, and underwater obstacles.
[0022] Generally, water areas are vast, making it difficult for hyperspectral imaging devices to directly acquire the required full-area hyperspectral images of the sample. Therefore, in one embodiment, the drone is controlled to fly sequentially along multiple parallel flight paths, and during the flight along each path, it vertically captures single-path remote sensing images of the water area using its onboard hyperspectral imaging device. The single-path remote sensing images captured during the flight along multiple flight paths are then sequentially stitched together to obtain the sample hyperspectral image. The specific image stitching method uses common image processing techniques, which will not be elaborated upon in this step.
[0023] Step 2: Convert the radiance of each pixel in the hyperspectral image of the sample into surface reflectance to obtain the corresponding sample reflectance spectral curve. The method of converting radiance into surface reflectance is a common method in this field, and this step will not be described in detail here.
[0024] Step 3: Select characteristic bands based on the curve characteristics of the sample reflectance spectrum curve.
[0025] Because the hyperspectral images of the samples contain many bands, the data processing volume is large. Therefore, the first step is to screen characteristic bands to achieve dimensionality reduction of the hyperspectral images. In one embodiment, a step-by-step band selection method is used to screen characteristic bands, including the following steps (please refer to [reference]). Figure 2 Flowchart:
[0026] (1) Based on the curve characteristics of the sample reflectance spectrum curve, select several representative bands containing a large amount of spectral information, including:
[0027] (a) Based on the curve characteristics of the sample reflectance spectral curve, the sample reflectance spectral curve is divided into several band subsets, including:
[0028] First, based on the peaks and troughs of the sample reflectance spectral curve, several original bands are roughly divided to obtain several original bands. Each original band includes the band range between adjacent peaks and troughs.
[0029] Then, based on the slope of the sample reflectance spectral curve within each original band, the original band is further subdivided into several band subsets. This includes dividing the original band into several band subsets at the points where the slope of the sample reflectance spectral curve within the original band reaches a slope threshold. Each band subset includes continuous bands within the original band where the slope of the sample reflectance spectral curve does not exceed the slope threshold. The slope threshold is a user-defined preset value.
[0030] A smaller slope in the sample reflectance spectral curve indicates a higher correlation between adjacent bands, in which case they can be merged into the same band subset. Conversely, a larger slope in the sample reflectance spectral curve indicates a lower correlation between bands, requiring them to be divided into different band subsets. In this way, each original band can be further divided into multiple band subsets according to the degree of change in the sample reflectance spectral curve.
[0031] (b) Calculate the standard deviation of reflectance of the sample reflectance spectral curve in each band subset. The standard deviation of reflectance reflects the distribution of the sample reflectance spectral curve in the band subset. The larger the standard deviation of reflectance, the greater the amount of information and the wider the range of pixel value distribution. Therefore, select several band subsets with the largest standard deviation of reflectance as candidate bands. The number of candidate bands retained can be set by customization.
[0032] (c) Calculate the correlation coefficient between different candidate bands, select several candidate bands with smaller correlation coefficients as representative bands, filter out multiple candidate bands with larger correlation coefficients, and the number of representative bands retained can be set by the user.
[0033] (2) Multiple band combinations are obtained by performing full permutation and combination of multiple representative bands, and each band combination includes several representative bands.
[0034] (3) Extract the multispectral images of the sample hyperspectral images under each band combination, and calculate the OIF value of the multispectral images and the comprehensive JM distance J. new .
[0035] The OIF (Optimal Index Factor) value of a multispectral image can be calculated using a standard formula, which will not be elaborated further in this embodiment. The comprehensive JM distance J of the multispectral image... new Used to characterize the distinguishability of aquatic plant categories from other categories, the larger the overall JM distance, the better. new The higher the degree of distinguishability between aquatic plant categories and other categories.
[0036] The combined JM distance of multispectral images newThe distance between the aquatic plant category and any other category j is calculated based on the JM distance. Therefore, the JM distance between the aquatic plant category and any other category j is first calculated based on the multispectral image. j The specific calculation formula adopts the standard calculation formula, which will not be repeated in this embodiment.
[0037] When dealing with multi-class problems, traditional methods typically calculate the average JM distance between different classes to characterize the distinguishability between them. However, this method considers the JM distances of each class individually, only assessing the overall distinguishability and masking the fact that classes with smaller JM distances have lower separation. To address this issue, this embodiment calculates the comprehensive JM distance J based on the idea that class distinguishability is inversely proportional to feature relevance. new :
[0038]
[0039] Where n is the total number of categories contained in the sample hyperspectral image. The i-th category represents the aquatic plant category, therefore j ≠ i. R j It is the Pearson correlation coefficient between the aquatic plant category determined based on multispectral images and the j-th category, and:
[0040]
[0041] Among them, X ik X is the sum of pixel values of pixels belonging to the aquatic plant category in the k-th band of the multispectral image. jk It is the sum of pixel values of pixels belonging to the j-th category in the k-th band of the multispectral image; It is the X-ray of each band in the multispectral image. ik The average value, It is the X-ray of each band in the multispectral image. jk The average value is K, where K is the total number of bands contained in the multispectral image.
[0042] (4) Determine the comprehensive JM distance J new The representative band in the band combination corresponding to the multispectral image that reaches the distance threshold and has the largest OIF value is the characteristic band.
[0043] Step 4: After performing a continuum removal transformation on the sample reflectance spectral curve, the spectral index is calculated. The continuum removal transformation of the spectrum can enhance the difference in reflectance spectrum between aquatic plants and other categories. This application analyzes the characteristics of the aquatic plant growth environment and selects vegetation indices that contribute significantly to the calculation of spectral indices, including: calculating the enhanced vegetation index, photochemical reaction index, normalized water index, and green light normalized vegetation index separately, and then merging them to obtain a high-dimensional matrix as the spectral index.
[0044] Step 5: Divide the sample hyperspectral image into several training sample images, and extract the spectral features and spectral indices of each training sample image in the feature bands. Then, concatenate the spectral features and spectral indices of each training sample image along the channel dimension to obtain the fused features of the training sample image.
[0045] In one embodiment, a sliding cutting window method is used to divide the training sample image, including defining the side length of the cutting window and the sliding step size as x. Then, the hyperspectral image of the sample containing H rows of pixels is added. Add background pixels to the sample hyperspectral image containing W columns of pixels. Background pixels are added to obtain the augmented hyperspectral image of the sample. The surface reflectance of the added background pixels is 0. Finally, a sliding cut is performed on the augmented hyperspectral image of the sample using a cutting window, and the image within the cutting window is extracted as the training sample image. Indicates to Round up. Indicates to Round up.
[0046] Step 6: Using the fusion features of each training sample image as input and the classification results of pixels in the training sample image as output, a water plant recognition model is trained based on the neural network model.
[0047] In one embodiment, the model structure of the neural network model used to train the aquatic plant recognition model is as follows: Figure 3 As shown, the neural network model includes a downsampling module, a feature extraction module, an upsampling feature fusion module, and a segmentation prediction module:
[0048] The downsampling module consists of three cascaded convolutional layers. After extracting features from the input image, it produces three sets of feature maps at different levels, each with dimensions equal to or greater than the size of the input image. The first two feature maps and the original input image are used as inputs to the three branches of the upsampling feature fusion module, and the last feature map with the smallest size is input to the feature extraction module.
[0049] The feature extraction module comprises multiple cascaded transformer modules. At its core, each transformer module is a multi-head self-attention module that fuses channel attention. The channel attention module primarily extracts pixel relationships across different bands, while the multi-head self-attention module enhances the model's ability to represent global pixel relationships. Subsequently, the output feature maps from the channel attention module and the multi-head self-attention module are fused with the input feature maps to generate an intermediate feature map. This intermediate feature map is then passed through the MLP module to output the transformer module's feature map.
[0050] The upsampling feature fusion module integrates multi-scale feature map information to improve the resolution of high-level feature maps while reconstructing detailed image features. This module is mainly divided into local and global branches, which learn the local and global pixel relationships of the feature maps, respectively. For the local branch, a linear transformation is performed in the width dimension, adjusting the dimension to ensure that identical feature points are densely distributed within the local region. For the global branch, a linear transformation is performed in the height dimension, adjusting the dimension to ensure that feature points are evenly distributed globally. The feature maps from the two branches are then fused, and the channel dimensions are adjusted to output the feature map from this upsampling feature fusion module.
[0051] The segmentation prediction module adjusts the feature map channel dimensions to the predicted category through convolutional layers and outputs the pixel classification results.
[0052] Step 7: Use the aquatic plant recognition model to determine the classification result of each pixel in the hyperspectral image of the water area to be detected, and calculate the proportion of the number of pixels belonging to the aquatic plant category in the total number of pixels to obtain the aquatic plant coverage of the water area to be detected.
[0053] The method for acquiring the hyperspectral image of the water area to be detected is similar to the method for acquiring the sample hyperspectral image in step 1, and will not be described again in this embodiment. After acquiring the hyperspectral image of the water area to be detected, the fusion features of the hyperspectral image are extracted. The method for extracting the fusion features is similar to the method for obtaining the fusion features of each training sample image during the training phase. The fusion features of the hyperspectral image of the water area to be detected are input into the aquatic plant recognition model to obtain the prediction results of the category of each pixel in the hyperspectral image of the water area to be detected.
[0054] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.
Claims
1. A method for detecting aquatic plant cover based on airborne hyperspectral imaging, characterized in that, The method for detecting aquatic plant coverage includes: The hyperspectral images of aquatic plants growing in the water were captured by a hyperspectral imaging device mounted on a drone, and the classification results of each pixel in the hyperspectral images of the samples were labeled. The radiance of each pixel in the hyperspectral image of the sample is converted into the surface reflectance to obtain the corresponding sample reflectance spectral curve. Feature bands are selected based on the curve characteristics of the sample reflectance spectrum curve; The spectral index is calculated after performing a continuum removal transformation on the reflectance spectral curve of the sample. The sample hyperspectral image is divided into several training sample images, and the spectral features and spectral indices of each training sample image under the feature band are extracted. The spectral features and spectral indices of each training sample image are concatenated in the channel dimension to obtain the fusion feature of the training sample image. Using the fusion features of each training sample image as input and the classification results of the pixels in the training sample image as output, a water plant recognition model is trained based on a neural network model. The classification results of each pixel in the hyperspectral image of the water area to be detected are determined by the aquatic plant recognition model, and the proportion of the number of pixels belonging to the aquatic plant category in the total number of pixels is calculated to obtain the aquatic plant coverage of the water area to be detected. The step of selecting characteristic bands based on the curve characteristics of the sample reflectance spectral curve includes: selecting several representative bands with a large amount of spectral information based on the curve characteristics of the sample reflectance spectral curve; performing full permutation combinations on the multiple representative bands to obtain multiple band combinations, each band combination including several representative bands; extracting multispectral images of the sample hyperspectral image under each band combination; calculating the OIF value of the multispectral image; and calculating the aquatic plant category and other arbitrary values based on the multispectral image. JM distance between categories And calculate the integrated JM distance of the multispectral image as follows: , It is the total number of categories contained in the hyperspectral image of the sample, the first... The first category is aquatic plants. The aquatic plant category determined based on the multispectral image is compared with other arbitrary... Pearson correlation coefficients for each category and ,in, It is the first of the multispectral images The sum of pixel values of pixels belonging to the aquatic plant category in each band. It is the first of the multispectral images Among the bands belonging to the first The sum of pixel values of pixels in each category; It is the various bands in the multispectral image. The average value, It is the various bands in the multispectral image. The average value, It is the total number of bands contained in the multispectral image; combined with JM distance Used to characterize the distinguishability of aquatic plant categories from other categories, combined with JM distance. The larger the size, the more distinguishable the aquatic plant category is from other categories; determine the overall JM distance. The representative band in the band combination corresponding to the multispectral image that reaches the distance threshold and has the largest OIF value is the characteristic band.
2. The water weed coverage detection method according to claim 1, characterized by, The step of selecting several representative bands with a large amount of spectral information based on the curve characteristics of the sample reflectance spectrum curve includes: Based on the curve characteristics of the sample reflectance spectral curve, the sample reflectance spectral curve is divided into several band subsets. Calculate the standard deviation of reflectance in each band subset of the sample reflectance spectral curve, and select several band subsets with the largest standard deviation of reflectance as candidate bands; Calculate the correlation coefficient between different candidate bands, and select several candidate bands with smaller correlation coefficients as representative bands.
3. The method of claim 2, wherein The reflectance spectrum curve of the sample is divided into several band subsets, including: Based on the peaks and troughs of the sample reflectance spectral curve, several original bands are obtained by coarsely dividing the bands. Each original band includes the band range between adjacent peaks and troughs. Based on the slope of the sample reflectance spectral curve in each original band, the original band is subdivided into several band subsets.
4. The method of claim 3, wherein, The subdivision of each original band includes: The original band is divided into several band subsets, with the point where the slope of the sample reflectance spectral curve in the original band reaches the slope threshold as the dividing position. Each band subset includes continuous bands in the original band where the slope of the sample reflectance spectral curve does not exceed the slope threshold.
5. The method of claim 1, wherein, The calculation of spectral indices includes: After calculating the enhanced vegetation index, photochemical reaction index, normalized water index, and green light normalized vegetation index separately, they were combined to obtain a high-dimensional matrix as the spectral index.
6. The method of claim 1, wherein, The process of capturing hyperspectral images of water samples with aquatic plants using a hyperspectral imaging device mounted on a drone includes: The unmanned aerial vehicle is controlled to fly along a plurality of parallel routes in sequence, and a single-route remote sensing image of the water area is vertically photographed by the carried hyperspectral imaging device during flight along each route.
7. The method of claim 1, wherein, The sample hyperspectral image is divided into a plurality of training sample images, including: The length of the cutting window and the sliding step are both defined as ; For containing Adding hyperspectral images of row pixels Row background pixels, for those containing Adding hyperspectral images of column pixels By listing the background pixels, the augmented sample hyperspectral image is obtained. The surface reflectance of the added background pixels is 0. Indicates to Round up. Indicates to Round up; The expanded sample hyperspectral image is cut by sliding a cutting window, and the image in the cutting window is extracted as a training sample image.