A hyperspectral image segmentation method based on correlation difference of spectral reflectance curve

Abstract

The present application relates to a hyperspectral image segmentation method based on spectral reflectance curve correlation difference, belonging to the field of hyperspectral image processing, comprising the following steps: black and white calibration of original crop hyperspectral data; using the method of logarithmic transformation combined with first derivative to pretreat the data; screening out the feature band with representative information based on information gain method; based on the correlation characteristics of pixel spectral reflectance curve, the image is segmented to obtain a binary image; the binary image after full-band spectral image segmentation is superimposed with the binary image after feature band spectral image segmentation; based on the closed operation processing, the segmentation burrs and gaps in the binary image are eliminated; the final segmentation image is obtained by mask processing with the result after closed processing as a mask. The present application improves the background segmentation accuracy of crop canopy hyperspectral image containing complex light interference, and can more accurately and quickly obtain hyperspectral target information.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of hyperspectral image processing, and particularly relates to a crop canopy hyperspectral image background segmentation method collected under complex lighting conditions. BACKGROUND

[0002] As an indirect analysis method, hyperspectral analysis technology has the advantages of non-destructive and rapid, and has been widely used in crop information detection. When collecting hyperspectral images, the target region of the crop canopy will inevitably contain a background region, which will affect the accuracy of crop information detection. Therefore, it is important to quickly and accurately segment the target region and the background region in the hyperspectral image in the research of hyperspectral image analysis technology. When the hyperspectral image is collected in outdoor or other complex lighting interference conditions, the interference noise in the image will seriously affect the background segmentation. For complex lighting interference images, traditional segmentation methods such as threshold method, edge detection method and K-means clustering method generally have problems such as missegmentation of canopy shadow region and fuzzy segmentation of canopy edge. Deep learning segmentation methods have the disadvantages of long model training time and complicated learning process. Therefore, the current methods for crop canopy hyperspectral images with complex lighting interference cannot meet the requirements of fast, accurate and simple background segmentation. The present application uses the difference in the correlation of the spectral reflectance curves of each component in the spectral image, and proposes a crop canopy hyperspectral image segmentation method based on the difference in the correlation coefficient of the spectral reflectance curves between each pixel point in the hyperspectral image, to solve the problems of inaccurate and complicated background segmentation of crop canopy hyperspectral images under complex lighting interference conditions.

[0003] After searching, there is no related patent application. SUMMARY

[0004] The present application provides a crop canopy hyperspectral image background segmentation method under complex lighting conditions, which aims to solve the problem of crop canopy hyperspectral image background segmentation under non-ideal lighting conditions such as outdoors and fields. It provides technical support for fast and accurate detection of crop information in hyperspectral images.

[0005] The present application achieves the above-mentioned purpose by the following technical means:

[0006] A hyperspectral image segmentation method based on the difference in the correlation of spectral reflectance curves, comprising the following steps:

[0007] The original hyperspectral image black and white calibration step uses standard black and white board data collected at the same time and same place as the sample to calibrate the original hyperspectral picture; the image preprocessing step uses a method combining logarithmic transformation (LT) and first derivative (FD) as the image preprocessing method; the characteristic wavelength screening step selects the characteristic waveband with representative information from the full waveband spectrum through the information gain method IG; the image is segmented based on the correlation characteristics of the pixel point spectral reflectivity curve; the binary image segmented after the full waveband spectral image and the binary image segmented after the characteristic waveband spectral image are superimposed; the superimposed binary image is closed processed; and the final segmented image is obtained through mask processing using the closed processing result as a mask.

[0008] Further, the hyperspectral image data black and white calibration formula is as follows:

[0009]

[0010] In the formula, R is the corrected image, I is the original spectral image, B is the standard blackboard image, and W is the standard whiteboard image.

[0011] Further, the method combining logarithmic transformation (LT) and first derivative (FD) is used as the image preprocessing method:

[0012] The first derivative FD calculation formula is as follows:

[0013]

[0014] In the formula, x i represents the spectral value of the i-th waveband in the sample; and g represents the window width. In this study, the window width of the first derivative is 1.

[0015] The logarithmic transformation LT calculation formula is as follows:

[0016] X LT = log(X) = [log(x1), log(x2), …, log(x i ), …, log(x n )]

[0017] In the formula, X represents the original spectrum of the sample; x i represents the spectral value of the i-th waveband of the original spectrum of the sample; and n represents the number of wavebands of the spectrum.

[0018] Further, the information gain method IG is used to select the characteristic waveband of the original hyperspectral image to expand the correlation difference between the pixel point spectral reflectivity curves in different parts of the image.

[0019] Further, the correlation coefficient between the spectral reflectance curve of each pixel point of the hyperspectral image and the spectral reflectance curve of the reference pixel point of the crop leaf is calculated to obtain a correlation coefficient matrix of the same type as the single-band pixel point matrix.

[0020] Further, in the binarization processing, the threshold of the correlation coefficient matrix is 0.9.

[0021] Further, the obtained two binarized images are multiplied to obtain a superimposed binarized image.

[0022] Further, the superimposed binarized image is subjected to a closing operation processing to eliminate burrs and gaps generated in the binarization, and the closing operation expression formula is as follows:

[0023]

[0024] In the formula, A is a binarized image after segmentation, represents expansion, represents corrosion, and S is a 3*3 structure element in the present application.

[0025] Compared with the prior art, the present application can at least achieve the following beneficial effects:

[0026] 1. The present application can accurately segment the background of an outdoor crop hyperspectral image containing complex light interference and various background elements.

[0027] 2. Under the condition of ensuring the segmentation accuracy, the present application avoids constructing a deep learning model and simplifies the process of hyperspectral image background segmentation.

[0028] 3. The present application combines logarithmic transformation (LT) and first derivative (FD) to pre-process the spectral data, thereby specifically eliminating the influence of outdoor complex light on image segmentation and improving the image segmentation accuracy.

[0029] 4. The present application selects characteristic bands from the original hyperspectral image to reduce the data calculation amount. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings are included to provide a further understanding of the present application, and are incorporated herein and constitute a part of the detailed description. The same reference numbers in the drawings indicate the same elements.

[0031] Figure 1 The present application is a flowchart of a method based on the correlation difference of spectral reflectance curves in the embodiments of the present application.

[0032] Figure 2The figure is a high spectrum reflectivity curve diagram of each part in the example of the present application, in which the solid curve is the spectrum reflectivity curve of the background part, the dotted curve is the spectrum reflectivity curve of the shadow canopy, and the dash curve is the spectrum reflectivity curve of the light canopy.

[0033] Figure 3 The figure is a reflectivity curve extraction in the example of the present application, in which IC is the high spectrum reflectivity curve of any pixel point in the crop canopy region in the high spectrum image, which is used as the reference reflectivity curve.

[0034] Figure 4 The figure is a correlation matrix diagram after calculation, in which the correlation matrix after calculation, each element in the matrix corresponds to the pixel point at the corresponding position in the high spectrum image.

[0035] Figure 5 The figure is a pretreatment effect diagram, (a) is the original reflectivity curve of each part, and (b) is the reflectivity curve of each part after pretreatment.

[0036] Figure 6 The figure is a diagram of screening 33 characteristic wave bands from 176 full wave bands by the information gain (IG) method.

[0037] Figure 7 The figure is a superimposed binary image obtained by correlating the full wave band high spectrum image and the IG characteristic wave band high spectrum image.

[0038] Figure 8 The figure is a closed operation on the obtained superimposed binary image, (a) is the original binary image after pretreatment, and (b) is the binary image after the closed operation.

[0039] Figure 9 The figure is a segmented lettuce image in the example of the present application, (a) is a lettuce diagram before segmentation, and (b) is a lettuce diagram after segmentation. DETAILED DESCRIPTION

[0040] The present application is further described below in combination with the drawings and specific embodiments.

[0041] The application is a hyperspectral image segmentation method based on spectral reflectance curve correlation difference, as shown in Figure 1, which is a flowchart of the application, including the following steps: black and white calibration is performed on original spectral data to eliminate the influence of light intensity and baseline drift on the image; the image data after black and white calibration is processed by first derivative and logarithmic transformation to reduce the influence of stray light in the data on subsequent image segmentation; the overlapping wavebands that blur the difference between the spectral data of different substances are screened out by the information gain method (IG), and the correlation difference between the spectral data of different substances is highlighted; after the feature wavelength screening, the spectral reflectance curve of any canopy pixel point is taken as a reference to perform correlation analysis on the spectral reflectance curves of the remaining pixel points in the image, and a correlation matrix is calculated; a suitable threshold is set to perform binary processing on the correlation matrix, and a closed operation method is applied to fill in the segmentation burrs and gaps in the matrix after binary processing; the binary matrix after the closed operation is taken as a mask to obtain a segmentation image result after masking the original spectral image.

[0042] S1: The core of the application is a hyperspectral image segmentation method based on spectral reflectance curve correlation difference, and the principle is as follows: Figure 2 As shown in the figure, the full-waveband spectral reflectance curves of the illuminated leaves, shadow leaves, ground, flowerpot and soil in the crop canopy are shown. As can be seen from the figure, although there is a large difference in reflectivity between the illuminated leaves and the shadow leaves in the wavelength range of 800nm-1000nm, the reflectivity curves of the two are basically coincident before 800nm, and the wavelength reflectivity curves of the two show similar trends, and have high correlation. Compared with the background reflectance curves of the ground, flowerpot and soil, the leaf spectral curve has some coincident wavelength regions with the spectral curves of other parts, but the overall spectral change trend has a large difference, and the correlation between the leaf spectral curve and the spectral curves of other background parts is not high. Based on the characteristics that there is a correlation difference between the spectral curves of the pixel points in the leaf area of the crop canopy and the spectral curves of the pixel points in the background area, the crop canopy hyperspectral image is segmented.

[0043] The specific steps are as follows: Figure 3 In the spectral matrix X m×n×λ , the full-waveband reflectivity I c of any pixel point in the canopy leaf area is taken as a reference to perform correlation analysis on the full-waveband reflectivity I y of the remaining pixel points. After calculation, the correlation coefficient matrix X r is obtained (as shown in Figure 4 ), and the correlation coefficient r=0.9 is taken as a threshold to perform binary processing on the matrix X rThe binary image is obtained by binaryzation, and the hyperspectral image of the lettuce canopy is masked by using the binary image as a mask to eliminate the background. The threshold value needs to be set in an appropriate range. Too high threshold value will cause over-segmentation, and too low threshold value will cause under-segmentation.

[0044] X m×n×λ Original three-dimensional spectral matrix of m rows, n columns and λ bands

[0045] X r Two-dimensional correlation coefficient matrix of m rows and n columns

[0046] I c Reflectivity curve of λ band

[0047] Before the S1 step, other steps are needed to process the data.

[0048] S2: The original hyperspectral image is calibrated to black and white, and the calibration formula is:

[0049]

[0050] In the formula, I represents the original spectral data of the lettuce canopy

[0051] R represents the corrected spectral data of the lettuce canopy

[0052] W represents the standard white plate spectral data

[0053] B represents the standard black plate spectral data

[0054] S3: The data calibrated in the S2 step is preprocessed. The present application selects to combine logarithmic transformation (LT) and first derivative (FD) to weaken the influence of various non-target factors on the target spectrum and improve the spectral resolution.

[0055] The calculation formula of the first derivative (FD) is as follows:

[0056]

[0057] In the formula, x i represents the spectral reflectivity of the i-th band in the spectrum; g represents the differential width. The window width of the first derivative in the present research is 1.

[0058] The calculation formula of the logarithmic transformation (LT) is as follows:

[0059] X LT=log(X)=[log(x1),log(x2),…,log(x i ),…,log(x n )]

[0060] In the formula, X represents the original information of the sample; x i represents the original spectral value of the i-th band; n represents the number of bands contained in the original spectrum.

[0061] like Figure 5 (a) shows the original spectral data for each part. Figure 5 (b) shows the spectral data of each part after preprocessing.

[0062] S4: After preprocessing in step S3, feature band screening is performed on the spectral data: Since correlation analysis of the preprocessed full-spectrum band curves is affected by overlapping bands, reducing segmentation accuracy, redundant bands in the full spectrum are eliminated using the information gain (IG) method. The specific calculation process is as follows:

[0063] The hyperspectral dataset consisting of N pixels and L bands is represented as X∈R. N×L If these N pixels contain m categories, then the IG calculation formula for any band B is:

[0064] IG(X,B)=E(X)-E(X|B)

[0065] In the formula, E(X) represents the information entropy of the hyperspectral dataset X; E(X|B) represents the conditional entropy of the dataset X under the condition of band B.

[0066] Let n i This represents the number of pixels in the i-th category, i.e. The formula for calculating information entropy is:

[0067]

[0068] In the formula, p i This represents the proportion of pixels in the i-th category to the total number of samples, i.e. Based on the range of spectral values ​​in band B, the dataset X is divided into k distinct sets X. j (j=1,2,…k), and n ij In set X j If the number of pixels belonging to the i-th category is given, then the formula for calculating the conditional entropy is:

[0069]

[0070] In the formula, p j ′ represents set X jThe proportion of the included pixel points in the total pixel points, that is The proportion of the pixel points of the i-th type in the set X j The proportion of the total pixel points in the set X j

[0071] As shown in FIG. (6), the IG curve obtained after calculation, the greater the IG value corresponding to the wave band, the more information the corresponding wave band contains. In order to avoid information overlap between adjacent wave bands, the peak point on the IG curve is selected as the characteristic wavelength, and 33 characteristic wave bands are selected. The spectral curve composed of the 33 characteristic wave bands is used for subsequent image segmentation.

[0072] S5: In order to further improve the segmentation accuracy, the present application selects the hyperspectral image of the full spectral wave band of the same sample and the hyperspectral image of the IG characteristic wave band to be processed by S1 step, as shown in FIG. Figure 7 The two processed matrixes are multiplied to obtain a superimposed binary image.

[0073] S6: The binary image segmented by step S5 is prone to misjudgment at the edge of the leaf and the shadow leaf, forming a gap and reducing the image segmentation accuracy. The closing operation has the effect of making the profile of the segmented image smooth, filling the narrow discontinuity and filling the small area cavity. The closing operation expression formula is as follows:

[0074]

[0075] In the formula, A is the binary image after segmentation, represents expansion, represents corrosion, and S is 3*3 structure element in the present application. Figure 8 (a) is the original binary image after processing, Figure 8 (b) is the binary image after closing operation.

[0076] S7: As shown in FIG. Figure 9 The binary image obtained by step S6 is used as a mask to process the original crop hyperspectral image to obtain the final segmentation result.

[0077] Compared with the prior art, the hyperspectral image segmentation method based on the correlation difference of spectral reflectivity curve provided by the present application improves the segmentation accuracy of the crop hyperspectral image collected under complex lighting conditions, simplifies the segmentation process, reduces the calculation amount in image segmentation, and has good self-adaptability to the size and morphology of the crop canopy. The hyperspectral image of different leaf crops can be effectively segmented. The present application solves the problem that the prior art has certain difficulty in accurately and quickly segmenting the crop canopy hyperspectral image containing complex lighting interference factors.​

Claims

1. A hyperspectral image segmentation method based on spectral reflectance curve correlation difference, characterized in that, It comprises the following steps: The original hyperspectral image black and white calibration step uses the standard black and white board data collected at the same time and the same place as the sample to calibrate the original hyperspectral image; the image preprocessing step uses the method of combining logarithmic transformation LT and first-order derivative FD as the image preprocessing method; The characteristic wavelength screening step selects the characteristic waveband with representative information from the full waveband spectrum through the information gain method IG; The image is segmented based on the correlation characteristics of the pixel point spectral reflectance curve; the binary image segmented after the full waveband spectral image is superimposed on the binary image segmented after the characteristic waveband spectral image; the closed processing is performed on the superimposed binary image; the final segmentation image is obtained by performing mask processing on the closed processing result as a mask; The image is segmented based on the correlation characteristics of the pixel point spectral reflectance curve, specifically: the correlation coefficient between the spectral reflectance curve of each pixel point of the hyperspectral image and the spectral reflectance curve of the reference pixel point of the crop leaf is calculated, and a correlation coefficient matrix of the same type as the single waveband pixel point matrix is obtained; the correlation coefficient matrix is binarized based on the correlation coefficient threshold.

2. The method of claim 1, wherein the method is based on the correlation difference of spectral reflectance curves. The black and white calibration formula of hyperspectral image data is as follows: ​ In the formula, R is the corrected image, I is the original spectral image, B is the standard blackboard image, and W is the standard whiteboard image.

3. The method of claim 1, wherein the method is based on a difference in correlation of spectral reflectance curves. The method of combining the log transformation (LT) with the first derivative (FD) is used as the image preprocessing method, and the first derivative calculation formula is shown as follows: ​ where x i denotes the spectral value at the i-th wavelength band in the sample; g denotes the window width, which is 1 in this study for the first derivative. The logarithmic transformation LT calculation formula is as follows: ； In the formula, X represents the original spectrum of the sample; x i represents the spectral value of the original spectrum of the sample at the i th waveband; and n represents the number of wavebands of the spectrum.

4. The method of claim 1, wherein the method is based on a difference in correlation of spectral reflectance curves. The characteristic waveband is selected from the original hyperspectral image by the information gain method IG to expand the correlation difference between the spectral reflectance curves of different parts of the pixel points in the image.

5. The method of claim 1, wherein the method is based on the correlation difference of spectral reflectance curves. In the binarization process, the correlation coefficient matrix threshold is 0.

9.

6. The method of claim 1, wherein the method is based on a difference in correlation of spectral reflectance curves. The two binary images obtained by segmenting the image based on the correlation characteristics of the pixel point spectral reflectance curve are multiplied to obtain the superimposed binary image.

7. The method of claim 1, wherein the method is based on the correlation difference of spectral reflectance curves. The closed operation is performed on the superimposed binary image to eliminate burrs and segmentation gaps generated in the binaryzation, and the expression formula of the closed operation is as follows: ​ where A is a divided binary image, represents dilation, represents erosion, and S is a 3 x 3 structuring element in the present invention.

Images