A crop hyperspectral image visual detection method based on a decolorization method

Abstract

The application provides a kind of crop hyperspectral image visualization detection method based on decolorization method, it is related to agricultural product image processing technical field, including the following detection steps, acquisition from spectral camera hyperspectral image, the multidimensional hyperspectral image obtained is divided into adjacent hyperspectral subset of band, in each subset, average method is used to fuse, obtain hyperspectral band, the hyperspectral band obtained is divided into subset of equal size, the image after S3 processing is decolorized, obtain single-band image, form visualization image, the visualization image obtained is input into classification model, realize the detection of crop, the image content information and structure information are extracted using depth method, and the appropriate gray level is automatically selected for decolorization, the saliency contrast information in local and global of original image can be considered when visualizing, the real color of image is restored as much as possible and the spatial information and spectral characteristics of image are preserved as much as possible.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural product image processing, and particularly to a crop hyperspectral image visualization detection method based on a decolorization method. BACKGROUND

[0002] Hyperspectral imaging combines traditional imaging and spectral technology to simultaneously acquire spatial and spectral information of an object. Each pixel of the image covers the entire spectral range of wavelengths. The information of these images can accurately reflect the attribute differences between different ground object categories, achieving accurate extraction and identification of ground objects. However, how to display hyperspectral images containing hundreds of spectral bands is a challenging problem, because existing computer displays are designed to display images containing one or three bands. To solve this problem, the common practice is to map the original hyperspectral image into the red, green, and blue (RGB) channels for visualization, which is commonly referred to as hyperspectral image visualization. In the past few years, many visualization methods have been developed, which can be mainly divided into two categories, namely, transformation-based methods and band selection-based methods.

[0003] The main idea of the spectral dimension reduction-based method is to reduce the spectral dimension of the hyperspectral image, and finally obtain three new bands for visualizing the hyperspectral image. The spectral dimension reduction-based method can be mainly divided into four categories: transformation-based methods, weighting-based methods, optimization-based methods, and deep learning-based methods. The transformation-based method usually selects three important features as the RGB channels for visualizing the hyperspectral image, such as principal component analysis (PCA), independent principal component analysis, etc. The weighting-based method is used to generate a fused image by linear combination of input image bands, such as the color matching function and the bilateral filter-based method, etc. The optimization-based method is to establish an objective function according to a specific optimization criterion. The deep learning-based method is to design a corresponding network structure to achieve effective spectral data reduction.

[0004] The main idea of the band selection-based method is to select three optimal bands from the original hyperspectral image as the RGB channels for visualizing the hyperspectral image. However, the band selection-based method has two main limitations. The main difficulty is how to select three least correlated bands containing a large amount of complementary information. In addition, since band selection involves an optimization process, this kind of method often has the problem of high computational complexity.

[0005] Image desaturation is another challenging problem in computer vision, which is color image greying, aiming to map the three-channel color values of a color image to a single-channel grayscale value. The general goal of image desaturation is to preserve as much color information as possible using the limited range of grayscale to maintain the original features and prominent characteristics of the reproduced colors, so that the resulting grayscale image is more consistent with human visual perception. Traditional desaturation algorithms usually use direct mapping in color space, extracting the luminance channel value of the image as the grayscale value to obtain the grayscale image, but this will cause the contrast information and structure information of the pixel color in the color image to be severely lost. Therefore, in recent years, many desaturation methods have been studied to solve this problem. According to the range of the calculation of the pixel neighborhood, it can be roughly divided into two categories: local greying method and global greying method. The local greying method mainly considers the independent pixels or adjacent pixels in the color image, and usually relies on local chroma edges for enhancement; the global greying method maintains the contrast information of local color and global color at the same time.

[0006] Problem one, for the hyperspectral visualization method: the method based on spectral dimension reduction can better preserve the spatial information and structural features of the hyperspectral image, but it is difficult to solve the color distortion problem of the visualization image. The method based on band selection can obtain a natural color visualization image, but it is usually insufficient to present rich information.

[0007] Problem two, for the image desaturation method: the local greying method can effectively maintain the local contrast of the original color image, but it often leads to boundary distortion and even produces boundaries that do not exist in the original image. The local greying method uses local color boundaries to enhance the contrast of the grayscale image. Different pixels in the original color image will be mapped to different grayscale values according to the image boundary or gradient information. Compared with the traditional constant mapping desaturation method, it can distinguish pixels with different local equal brightness colors and weaken the information loss problem of local color in the desaturation process. Global greying is to maintain the color contrast between each pixel and all other pixels, which can fully maintain the overall contrast of the original image. However, the preservation of contrast between all pixels may cause the contrast between adjacent pixels in the grayscale image to be compressed, so that the adjacent regions in the grayscale result cannot be distinguished, that is, the local features of the original image may not be preserved after greying. SUMMARY

[0008] (I) Technical problems solved

[0009] In view of the deficiencies of the prior art, the present application provides a crop hyperspectral image visualization detection method based on a decolorization method, which can maintain the visualization image close to nature, i.e., restore the true color, while preserving the spatial information and spectral characteristics of the image. In addition, for the generated visualization image, a corresponding classification model is combined to realize rapid, non-destructive and accurate detection of crops.

[0010] 1. The color distortion problem of the visualization image based on the spectral dimension reduction method is solved, and the problem of insufficient information presentation based on the waveband selection method.

[0011] 2. The problems of boundary distortion caused by the local gray-scale method, the generation of boundaries that do not exist in the original image, and the preservation of the contrast between adjacent pixels in the gray-scale image, which may cause the contrast between adjacent regions in the gray-scale result to be compressed, so that the local features of the original image cannot be distinguished after gray-scale processing, are solved.

[0012] (II) Technical solutions

[0013] To achieve the above object, the present application is implemented by the following technical solutions: a crop hyperspectral image visualization detection method based on a decolorization method, comprising the following detection steps:

[0014] S1, collecting a hyperspectral image from a spectral camera, and dividing the obtained multi-dimensional hyperspectral image into a hyperspectral subset of adjacent hyperspectral wavebands;

[0015] S2, fusing each subset by using an average method to obtain a hyperspectral waveband;

[0016] S3, dividing the obtained hyperspectral waveband into subsets of equal size;

[0017] S4, decolorizing the image processed in S3 to obtain a single waveband image and form a visualization image;

[0018] S5, inputting the obtained visualization image into a classification model to realize detection of crops.

[0019] Preferably, each subset has adjacent and non-repeating wavebands.

[0020] Preferably, in step S4, the following steps are included:

[0021] S41, optimizing the noise image of a single channel to minimize the representation distance in the network level with the color image of three channels;

[0022] S42, define a perceptual loss function to measure the low-level and high-level perceptual differences between color and gray images, obtain perceptual features;

[0023] S43, the perceptual features are obtained by a VGG-19 network comprising 16 convolutional layers and 5 pooling layers;

[0024] S44, each layer in the VGG-19 network in S43 is provided with a linear filter and a nonlinear activation function, corresponding to dense perception and sparse perception respectively.

[0025] Preferably, the parameters in the VGG-19 network are obtained by training.

[0026] Preferably, the network is a neural network, and the algorithm weight range of the high-level perceptual layer and the low-level perceptual layer of the neural network is 0-1, and the weights of the high-level perceptual layer and the low-level perceptual layer are independent of each other.

[0027] Preferably, the image decolorization processing algorithm of step S4 further comprises any one of DHV-GcsDecolor or DHV-SPDecolor.

[0028] (I) beneficial effects

[0029] The application provides a crop hyperspectral image visualization detection method based on a decolorization method, which has the following beneficial effects:

[0030] 1. The essence of the decolorization method based on the VGG19 network model for hyperspectral image visualization is based on human visual perception, which extracts image content information and structure information using deep method, and automatically selects appropriate gray scale for decolorization. In visualization, the saliency contrast information in the local and global of the original image can be considered, the real color of the image is restored as much as possible, and the spatial information and spectral characteristics of the image are preserved as much as possible, solving the problems of local gray processing boundary features and unclear local features in global gray processing.

[0031] 2. The scheme combines a classification model to identify the category of crops, and the obtained visualization image can be directly input, compared with the original hyperspectral image input, not only the information is preserved as much as possible, but also the dimension of the data is greatly reduced, the speed of classification and identification is greatly improved, and the problems of color distortion and information richness are solved. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a basic flowchart of the embodiment method of the application.

[0033] Figure 2 It is a decolorization method flowchart based on the VGG19 convolutional neural network of the application.

[0034] Figure 3 The schematic diagram of hyperspectral image data set used for experimental verification in the embodiment of the present application.

[0035] Figure 4 The comparison of visualization results of the method of the embodiment of the present application and the existing method.

[0036] Figure 5 The comparison of classification and recognition results of the visualization results of the method of the embodiment of the present application and the existing method combined with SVM;

[0037] Figure 6 The objective performance indicators of three typical hyperspectral image visualization methods of the present application on the Salinas data set;

[0038] Figure 7 The recognition accuracy of each crop of the visualization images generated by the three typical hyperspectral image visualization methods of the present application on the Salinas data set combined with SVM. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0040] Embodiment one:

[0041] As shown in Figure 1 , the purpose of the present application is to provide a crop hyperspectral visualization detection method based on a decolorization method, and the obtained visualization image is closer to the real color, more image spatial information and spectral details are preserved, and based on the visualization image, rapid, non-destructive and accurate detection of crops is realized.

[0042] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: a crop hyperspectral visualization detection method based on a decolorization method, comprising the following steps:

[0043] S1, for hyperspectral images from different hyperspectral cameras, the obtained N-dimensional hyperspectral image I is divided into 9 hyperspectral subsets of adjacent hyperspectral bands; wherein the function expression of the obtained k∈(1, 2,..9)th spectral subset H k is:

[0044]

[0045] S2, due to the strong correlation between adjacent bands, the average-based method is used in each subset for fusion, and 9 hyperspectral bands are obtained, and the kth hyperspectral band P k The function expression is:

[0046]

[0047] represents the nth band of the kth subset, and N k represents the total number of bands in the nth subset,

[0048] S3, the 9 hyperspectral bands P k obtained by dimension reduction are divided into three subsets C i of equal size, where i∈(1, 2, 3), each subset has adjacent and non-repeating bands, and is regarded as a pseudo-color image,

[0049] S4, three pseudo-color images are processed by the decolorization algorithm to obtain three single-band images, which are regarded as R, G and B channels respectively, and form the final visualization image D,

[0050] D i = DECO(C i )

[0051] D i = (P 3i-2 , P 3i-1 , P 3i ) represents the ith pseudo-color image composed of three adjacent bands, and DECO represents the decolorization algorithm,

[0052] S5, the obtained visualization image is input into the classification model to realize crop detection.

[0053] Example two:

[0054] As shown in Figure 2 , step S4 in the embodiment is implemented by using a VGG-19 convolutional neural network model, including the following steps:

[0055] S41, optimize the single-channel noise image to minimize the representation distance in the network level with the three-channel color image;

[0056] S42, define a perceptual loss function to measure the low-level and high-level perceptual differences between color and gray images;

[0057] ​S43, the perception features are provided by a VGG-19 network comprising 16 convolutional layers and 5 pooling layers; S44, each layer in the network is provided with a linear filter (Conv) and a nonlinear activation function (ReLU), corresponding to dense perception and sparse perception, respectively, wherein the dense perception describes the convolutional output of high-order perception, and the sparse perception describes the convolutional input of feature selection,

[0058] For image decolorization processing, the feature reconstruction loss of the low-level perception layer (1st layer) and the high-level perception layer (2nd, 4th, 8th, and 16th layers) of the VGG19 neural network is calculated, and the high-level perception layer can be set to different levels according to requirements. In addition, the low-level perception and the high-level perception can be respectively set with a weight coefficient, so that the network focuses on different levels of features. For example, in the embodiment, the weight of the bottom perception layer is 1, and the weight of the high-level perception layer decreases from 1 to 0.25 as the network depth increases.

[0059] The purpose of the perception loss is to make the color image and the grayscale image have similar perception representation. The purpose of the decolorization method is to minimize the perception loss between the grayscale image and the color image by using the deep features of the image. The decolorization method takes into account the saliency contrast information in the local and global of the original image during processing.

[0060] Most of the parameters in the VGG19 convolutional neural network in the embodiment are obtained through network training, so when visualizing and detecting hyperspectral images obtained by different hyperspectral instruments in different scenes, the structure of the network does not need to be changed, only a few parameters need to be changed, and the network has strong universality and robustness.

[0061] Embodiment three:

[0062] As Figures 3-4As shown, in order to verify the decoloration-based crop hyperspectral image visualization detection method of the present embodiment, the hyperspectral image obtained by the AVIRIS sensor in the Salinas Valley of California, USA, is used for verification experiment, the image of the data set contains 204 bands, the spatial size is 512*217, and a total of 111104 pixels are included, of which 56975 pixels are background pixels, 54129 pixels can be used for classification, and these pixels have a total of 16 categories, including fallow land (Fallow), celery (Celery), corn (Corn) and the like. In the actual process, 10% of the data of each category in the Salinas data set is used as a training set, and 90% of the data is used as a test set. Four typical hyperspectral image visualization methods are compared, and the evaluation index of the visualization image has four kinds, which are information entropy (Entropy) and fractal dimension (CFD), wherein the information entropy represents the degree of image information preservation, that is, the average amount of information contained in the image, the greater the value, the more information contained, and the fractal dimension is used to evaluate the complexity of the visualization image, the greater the value, the better, and the comparison of the visualization results of the three typical hyperspectral visualization methods PCA, DHV-GcsDecolor, DHV-SPDecolor and the method proposed in the present embodiment on the Salinas data set is shown, Figure 4 (A) is the result image of the first three principal components selected by the PCA method, Figure 4 (B) is the visualization result image based on the GcsDecolor decolorization method, Figure 4 (C) is the visualization result image based on the SPDecolor method, Figure 4 (D) is the visualization result image of the method (DHV-VGG19) proposed in the present embodiment, and it can be seen from the image that the method proposed in the present embodiment is closer to human visual perception and more consistent with real color.

[0063] Embodiment Four:

[0064] Figure 6 The objective evaluation index of the imaging experiment of the three typical imaging methods PCA, DHV-GcsDecolor (DHV-1), DHV-SPDecolor (DHV-2) and the method proposed in the present embodiment (DHV-VGG19, DHV-3) on the Salinas data set is shown, and the best numerical result is marked black.

[0065] From Figure 6As can be seen from the above table, all objective evaluation indexes of the method (DHV-3) of the embodiment are better than those of other methods, because the method (DHV-3) of the embodiment perceives edge information of an image by relying on the first convolutional layer in the visualization process, perceives saliency information of the image by relying on four high-level convolutional layers of 2, 4, 8 and 16, and gives consideration to saving local information and global information of the image in the visualization process, and saves spatial and spectral details of the image as much as possible, so that the generated image is closer to human vision.

[0066] Figure 7 The recognition results of crops obtained by combining the visualization images obtained by three typical hyperspectral visualization methods and the method proposed in the embodiment on the Salinas data set with a support vector machine (SVM) machine learning model are shown, in which there are four recognition accuracy evaluation indexes, namely, overall classification accuracy OA, classification accuracy CA of each class, average classification accuracy AA and Kappa coefficient, and the greater the value, the higher the classification recognition accuracy.

[0067] From Figure 6 As can be seen from the above table, the classification accuracy CA of each class of crops in the SVM model of the method (DHV-3) of the embodiment is mostly better than or almost the same as that of other methods, the overall classification accuracy OA, the average classification accuracy AA and the Kappa coefficient all exceed 90, which are better than those of the remaining methods, and the overall classification effect is relatively ideal, because compared with the remaining methods, the method of the embodiment retains more feature distinguishability and spatial details of the hyperspectral image, so that the method has the best overall performance in the visualization of the hyperspectral image and the classification accuracy of crops, and it should be noted that the classification model here is only a simple SVM model, and a more optimal classification model can be designed or introduced to achieve more accurate classification and recognition.

[0068] Figure 5 The crop classification and recognition images obtained by combining the visualization images obtained by three typical hyperspectral visualization methods PCA, DHV-1, DHV-2 and the method proposed in the embodiment on the Salinas data set with a SVM model are shown, Figure 5 (A) is a crop classification and recognition image based on a PCA visualization image, Figure 5 (B) is a crop classification and recognition image based on a visualization image of a GcsDecolor decolorization method, Figure 5 (C) is a crop classification and recognition image based on a visualization image of a SPDecolor method, Figure 5 (D) is a crop classification and recognition image based on a visualization image of the method proposed in the embodiment, and as can be seen from the image, the method proposed in the embodiment has the best overall crop classification accuracy.

[0069] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A method for visual detection of crop hyperspectral images based on a desaturation method, characterized in that: The testing steps include the following: S1. Acquire hyperspectral images from the spectral camera and divide the obtained multidimensional hyperspectral images into hyperspectral subsets of adjacent hyperspectral bands; S2. The hyperspectral bands are obtained by fusion using the averaging method in each subset. S3. Divide the obtained hyperspectral bands into subsets of equal size; S4. Desaturate the image processed in S3 to obtain a single-band image, thus forming a visual image. S5. Input the obtained visualization image into the classification model to detect crops; Step S4 includes the following steps: S41. Optimize the noisy image in a single channel to minimize the distance between its representation and that of the three-channel color image in the network hierarchy; S42. Define a perceptual loss function to measure the low-level and high-level perceptual differences between color and grayscale images and obtain perceptual features; S43. The perceptual features are obtained by a VGG-19 network containing 16 convolutional layers and 5 pooling layers. S44. Each layer of the VGG-19 network in step S43 has a linear filter and a nonlinear activation function, corresponding to dense sensing and sparse sensing, respectively.

2. The crop hyperspectral image visualization detection method based on decolorization method according to claim 1, characterized in that: Each subset has adjacent and non-repeating bands.

3. The crop hyperspectral image visualization detection method based on decolorization method according to claim 1, characterized in that: The parameters in the VGG-19 network are obtained through training.

4. The crop hyperspectral image visualization detection method based on decolorization method according to claim 1, characterized in that: The network is a neural network, and the algorithm weights of the high-level and low-level perceptual layers of the neural network are in the range of 0-1, and the weights of the high-level and low-level perceptual layers are independent of each other.

Images