Spectral Region Joint Recognition Method for Deep Learning

Abstract

The present invention discloses a method for jointly identifying spectral regions for deep learning, belonging to the technical fields of hyperspectral imaging and deep learning. By using the frequency-domain features of hyperspectral images, spectral data is divided into multiple regions in the spatial dimension to achieve high-precision image classification. During the identification process, the central region is preferentially identified and diffused to the surrounding regions. In the partitions where the low frequency is the main component, identification is carried out first, and the identification results are diffused within the joint region to obtain a confidence diffusion map. For the regions with a relatively low diffusion concentration, secondary identification is performed to improve the identification speed and the spatial region consistency of the identification results. The present invention combines hyperspectral technology and spectral analysis technology, and utilizes the characteristic that the object to be identified occupies multiple continuous pixel regions, thereby improving the classification and identification speed of spectral images.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral imaging and deep learning, and particularly relates to a method for jointly identifying spectral regions for deep learning. Background Art

[0002] In the process of hyperspectral identification, there are often many continuous regions of the same substance. Repeatedly identifying these parts often leads to a waste of a large amount of computing performance. In order to avoid this part of the performance waste, the image partition can be identified by combining the frequency domain characteristics of the image. In the spectral identification algorithm constructed by deep learning, it is often necessary to identify the spectral lines of each individual spectrum, and the identification process often takes a lot of time, including convolutional operations and the operation process of the fully connected layer of the neural network. However, it can be seen from the spatial dimension of the image that adjacent pixel points in the image often have the same classification result. In principle, this part of the calculation can be omitted, but it is difficult to achieve under the current identification technology. In traditional spectral classification technology, after building a deep learning model, it is necessary to identify all pixel points in the entire spectral image in sequence. However, most regions in the spectral image are the same substance, which can be observed in the frequency domain. Therefore, a technology that can discover and identify large low-frequency regions is of great significance for improving the identification efficiency of deep learning algorithms. Summary of the Invention

[0003] The technical problem solved by the present invention: Provide a method for jointly identifying spectral regions for deep learning that combines hyperspectral technology and spectral analysis technology, and utilizes the characteristic that the object to be identified occupies multiple continuous pixel regions to improve the classification and identification speed of spectral images.

[0004] Technical Solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0005] A method for jointly identifying spectral regions for deep learning, characterized in that: using the frequency domain characteristics of hyperspectral images, dividing spectral data into multiple regions from the spatial dimension to achieve high-precision image classification; preferentially identifying the central region and spreading to the surrounding regions during the identification process; first identifying the partition where the low frequency accounts for the main component, and spreading the identification result within the joint region to obtain a confidence diffusion map, and then performing secondary identification on the regions with a lower diffusion concentration to achieve improving the identification speed and the spatial region consistency of the identification result. Specifically, it includes the following steps:

[0006] S1: Collect some hyperspectral images mixed with different substances, and correct and denoise the data;

[0007] S2: Label and classify the spectral data of the collected hyperspectral images;

[0008] S3: During the recognition process, relying on the frequency-domain features and the spectral image annotation results pre-labeled, the object to be recognized is segmented into multiple blocks to be recognized in the spatial dimension; the image is divided in the frequency domain to obtain the frequency-domain feature map of the image, and then the high-frequency region is divided in the frequency-domain feature map, and the low-frequency region is identified and divided in the frequency-domain feature map;

[0009] S4: During the recognition process, a deep neural network is used for classification. During the classification process, the centroids of each region are first classified, and the results of the centroids are spread to the surroundings.

[0010] Further, in step S3, the segmentation of the recognition blocks requires a high-frequency edge image as the basis. First, the multi-channel hyperspectral image is processed to obtain the wavelength channel with the highest contribution degree. On this basis, high-pass filtering is performed to obtain the high-frequency region edge map. The specific steps are as follows:

[0011] (1) Combining the discrimination degree between each category of spectral data, select the spatial feature map, calculate and select the wavelength channel with the highest contribution degree;

[0012] (2) Perform frequency-domain conversion on the selected wavelength channel with the highest contribution degree:

[0013] (3) Perform frequency-domain segmentation on the converted image, perform high-pass filtering, multiply the frequency-domain image with the Gaussian function point by point to obtain the high-frequency partition line of the image;

[0014] (4) Perform inverse Fourier transform on the obtained high-frequency region to obtain the high-frequency region edge map.

[0015] Further, the recognition blocks are segmented based on the high-frequency edge map to realize the identification and division of the low-frequency region of the frequency-domain features. The specific process is as follows: (1) Initialize the index variables x and y to 0, 0, initialize the block area label layer c = 1, initialize the high-frequency edge threshold h, initialize the all-0 partition matrix K with the same size as the high-frequency edge map, and initialize the row connection list r to be empty;

[0016] (2) Access the pixel point L(x, y) in the high-frequency region edge map. If L(x, y) < h, then assign and mark the corresponding partition matrix K(x, y) with the block label c, let x = x + 1, and add the current x value of the row to the row connection list r. Repeat step (2). If L(x, y) ≥ h, then increase y, y = y + 1;

[0017] (3) Assign the values in the row connection list r to x in turn until L(x, y) < h, then stop the sequential assignment and empty the row connection list r. If all values satisfy L(x, y) ≥ h, then make c = c + 1;

[0018] (4) Start traversing from x, y = 0, 0 until a point is found that satisfies K(x, y) = 0 and K(x, y) < h. If no point that meets the conditions is found, proceed to step (5).

[0019] (5) Start traversing from x, y = 0, 0. If a point where K(x, y + 1)·K(x, y)·(K(x, y + 1) - K(x, y)) ≠ 0 appears, then change the value equal to max(K(x, y + 1), K(x, y)) in K to min(K(x, y + 1), K(x, y)).

[0020] Furthermore, the spectral contribution degree of the wavelength is calculated according to the following formula:

[0021]

[0022] In the formula, q i represents the spectral contribution degree of the i-th wavelength, A represents all A different spectral categories, i and j represent any two different spectral categories, and α represents the currently calculated spectral category; for each collected spectral wavelength, the contribution degree q i is calculated. After calculating all the wavelengths, select the wavelength channel with the highest contribution degree.

[0023] Furthermore, the frequency domain conversion of the image is implemented in the following way:

[0024]

[0025] In the formula, ω1 and ω2 represent two spectral dimensions, and m and n respectively represent the two axes of the image.

[0026] Furthermore, in step S4, during the diffusion process, the Gaussian function is used for diffusion. The confidence level in the central region is 1, and the confidence level in the edge region gradually decreases. Regions with a confidence level higher than 0.5 are not subject to secondary recognition. Subsequently, the regions to be recognized are recognized, and the variance of the Gaussian function is selected according to the degree of continuity of the substance.

[0027] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0028] (1) The present invention applies the spectral analysis technology to the field of hyperspectral image recognition. For the same substance regions that originally required multiple repeated runs of calculations, using the frequency domain analysis technology, the number of spectral points to be classified is reduced from the spatial characteristics, thereby improving the recognition speed for spectral image recognition based on deep learning.

[0029] (2) The present invention can adapt to different impurity sizes by setting the diffusion range of the recognition results. For large-area impurity blocks, it can automatically perform a large-range diffusion of the recognition results, thereby improving the overall recognition speed of the system and can be optimized according to different impurity classification conditions.

[0030] (3) Through the low-frequency region determination and partitioning step, the present invention can quickly divide the low-frequency regions with similar substances, thus avoiding the problem of slow partitioning speed in the traditional connected domain partitioning. Description of the Drawings

[0031] Figure 1 is the characteristic channel extracted from the collected spectral image;

[0032] Figure 2 is the high-frequency partitioning map obtained after filtering;

[0033] Figure 3 is the image segmentation result obtained according to the high-frequency partitioning map;

[0034] Figure 4 is the recognition result after segmentation;

[0035] Figure 5 is the diffusion schematic diagram with the diffusion degree set to 5;

[0036] Figure 6 is the diffusion schematic diagram with the diffusion degree set to 15. Detailed Embodiments

[0037] The following further clarifies the present invention in conjunction with specific embodiments. The embodiments are implemented on the premise of the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0038] The spectral region joint recognition method for deep learning of the present invention includes the following steps:

[0039] S1: Collect some hyperspectral images mixed with different substances and correct and denoise the data;

[0040] Use the FX10E series hyperspectral imager of SPECIM Company in Finland to obtain the reflection spectral images of tobacco and impurities in the range of 400 to 1000 nm;

[0041] S2: Label and classify the spectral data of the collected hyperspectral images to ensure that the number of each category to be recognized is sufficient;

[0042] S3: During the recognition process, relying on the frequency-domain features and the spectral image annotation results pre-annotated, the object to be recognized is segmented into multiple blocks to be recognized in the spatial dimension; the image is divided in the frequency domain to obtain the frequency-domain feature map of the image, and then the high-frequency region is divided in the frequency-domain feature map, and the low-frequency region is identified and divided in the frequency-domain feature map.

[0043] S31: The segmentation of the recognition block requires the high-frequency edge image as the basis. First, process the multi-channel hyperspectral image to obtain the wavelength channel with the highest contribution degree (step (1)), and perform high-pass filtering on this basis (steps (2)-(4)) to obtain the high-frequency region edge map. The obtaining steps are as follows:

[0044] (1) Combining the discrimination between each category of spectral data, select the spatial feature map and calculate according to the following formula:

[0045]

[0046] In the formula, q i represents the spectral contribution degree of the i-th wavelength, A represents all A different spectral categories, i and j represent any two different spectral categories, and α represents the currently calculated spectral category. For each collected spectral wavelength, calculate the contribution degree g i , and after calculating all the wavelengths, select the wavelength channel with the highest contribution degree.

[0047] The characteristic channel among the obtained spectral data. As Figure 1 shown. It can be found through the extracted characteristic channel that this channel can effectively reflect the spatial relationship between various substances and provides an effective data basis for subsequent spectrum analysis.

[0048] (2) For the wavelength channel with the highest contribution degree selected, perform the frequency-domain conversion of the image using the following formula:

[0049]

[0050] In the formula, ω1 and ω2 represent two spectral dimensions, and m and n represent the two axes of the image respectively.

[0051] (3) Perform frequency-domain segmentation on the converted image, perform high-pass filtering, multiply the frequency-domain picture F(ω1, ω2) by the Gaussian function to obtain the high-frequency partition line of the image. The filtering function used is:

[0052]

[0053] In the formula, d represents the distance attenuation coefficient. The larger d is, the faster the attenuation is, and the stricter the high-frequency screening of the image is. d takes 10. After multiplying this function with the frequency-domain picture, the obtained image is the high-frequency region.

[0054] (4) Perform inverse Fourier transform on the obtained high-frequency region to obtain the high-frequency region edge map L.

[0055] The result of the high-frequency region edge map is as Figure 2 shown. It can be seen from the figure that the highlighted region is the high-frequency edge pixel, which enables subsequent recognition block segmentation based on frequency-domain features, thereby reducing the recognition burden.

[0056] S32: Perform recognition block segmentation based on the high-frequency edge map to realize the determination and division of the low-frequency region of the frequency-domain feature map. The specific process is as follows:

[0057] (1) Initialize the index variables x and y to 0, 0, initialize the block area label layer c = 1, initialize the high-frequency edge threshold h, initialize the all-0 partition matrix K with the same size as the high-frequency edge map, and initialize the row connection list r to be empty;

[0058] (2) Access the pixel point L(x, y) in the high-frequency region edge map. If L(x, y) < h, then assign and mark the corresponding partition matrix K(x, y) with the block label c (the block with label c represents the c-th low-frequency region), let x = x + 1, and add the current x value of the row to the row connection list r, and repeat step (2). If L(x, y) ≥ h, then increase y, y = y + 1;

[0059] (3) Assign the values in the row connection list r to x in turn until L(x, y) < h, then stop the sequential assignment and empty the row connection list r. If all values satisfy L(x, y) ≥ h, then make c = c + 1;

[0060] (4) Start traversing from x, y = 0, 0 until a point is found that satisfies K(x, y) = 0 and K(x, y) < h. If no point that meets the conditions is found, then enter step (5).

[0061] (5) Start traversing from x, y = 0, 0. If a point where K(x, y + 1)·K(x, y)·(K(x, y + 1) - K(x, y)) ≠ 0 appears, then change the value equal to max(K(x, y + 1), K(x, y)) in K to min(K(x, y + 1), K(x, y)).

[0062] For the determination and division of the low-frequency region of the frequency-domain feature, the result after division is colored as Figure 3 shown. Different gray values in the figure represent different blocks. Subsequent recognition will be based on the centroids of these regions (x z, y z ) Expand, thus greatly reducing the number of spectral points required for recognition, and further reducing the time for the deep learning algorithm to recognize the entire hyperspectral image.

[0063] S4: Use a deep neural network for classification during the recognition process. During the classification process, first classify the centroids (x z , y z ) of each region, and spread the results of the centroids to the surrounding areas. During the spreading process, use a simplified circular Gaussian function for spreading. The confidence level in the central region is 1, and the confidence level in the edge region gradually decreases. Regions with a confidence level higher than 0.5 are not recognized again. Subsequently, recognize the regions that need to be recognized, and select the spreading parameter s according to the degree of continuity of the substance. Figure 5 and Figure 6 show different spreading parameter s. In the figure, it is assumed that the centroid is 0, 0, and the spreading parameters of 5 and 15 are taken for the upper and lower figures respectively. The steps are as follows:

[0064] (1) Use the following simplified circular Gaussian function for confidence spreading.

[0065]

[0066] In the formula, z represents the confidence value, s represents the spreading parameter, x, y represent the image indices, x z , y z represent the centroid points of each region.

[0067] (2) For regions where the confidence value z > 0.5 is satisfied, directly determine them as the same category as the centroid point.

[0068] (3) For regions that do not meet the conditions in step (2), perform secondary recognition.

[0069] According to the recognition effect, reasonably adjust the spreading parameter s. When the target objects are generally large, select a larger spreading parameter s; when the spreading objects are small, select a smaller parameter.

[0070] After the image is recognized, the final recognition result is obtained as Figure 4 shown. In the figure, different image grayscales represent different recognition categories. The light grayish-white region in the figure represents the region recognized as the background, and other different gray regions represent the regions recognized as various different objects. It can be seen from this figure that using this method can effectively distinguish each category.

[0071] The present invention utilizes the frequency domain characteristics of hyperspectral images, divides spectral data into multiple regions from the spatial dimension, and realizes high-precision image classification; during the recognition process, the central region is preferentially recognized and spread to the surrounding regions, thereby reducing the computational loss during the recognition process.

[0072] Recognition is first performed on the partition where the low frequency accounts for the main component, and the recognition result is diffused within the joint region to obtain a confidence diffusion map. For the region with a relatively low diffusion concentration, secondary recognition is performed to improve the recognition speed and the spatial region consistency of the recognition result. The present invention combines the hyperspectral technology and the spectrum analysis technology, and utilizes the characteristic that the object to be recognized occupies multiple continuous pixel regions, thereby improving the classification and recognition speed of the spectral image.

[0073] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for jointly identifying spectral regions for deep learning, characterized in that: Utilize the frequency-domain features of hyperspectral images to divide spectral data into multiple regions in the spatial dimension, achieving high-precision image classification; during the recognition process, give priority to recognizing the central region and spreading to the surrounding regions; first recognize the partition where the low frequency accounts for the main component, and spread the recognition results within the joint region to obtain a confidence diffusion map, and then perform secondary recognition on the regions with a lower diffusion concentration to improve the recognition speed and the spatial region consistency of the recognition results.

2. The method for jointly identifying spectral regions for deep learning according to claim 1, characterized in that, Specifically, it includes the following steps: S1: Collect hyperspectral images with different substances mixed, and correct and denoise the data. S2: Label and classify the spectral data of the collected hyperspectral images. S3: During the recognition process, relying on the frequency-domain features and the spectral image annotation results annotated in the early stage, divide the object to be recognized into multiple blocks to be recognized from the spatial dimension; perform frequency-domain division on the image to obtain the frequency-domain feature map of the image, then perform high-frequency region division in the frequency-domain feature map, and perform low-frequency region determination and division in the frequency-domain feature map. S4: Use a deep neural network for classification during the recognition process. During the classification process, first classify the centroids of each region and spread the results of the centroids to the surrounding areas.

3. The method for jointly identifying spectral regions for deep learning according to claim 2, characterized in that: In step S3, the segmentation of the recognition block requires a high-frequency edge image as the basis. First, process the multi-channel hyperspectral image to obtain the wavelength channel with the highest contribution degree. On this basis, perform high-pass filtering to obtain the high-frequency region edge map. The specific steps are as follows: (1) Combine the discrimination degree between different categories of spectral data, select the spatial feature map, calculate and select the wavelength channel with the highest contribution degree. (2) Perform frequency-domain conversion on the selected wavelength channel with the highest contribution degree. (3) Perform frequency-domain segmentation on the converted image, perform high-pass filtering, multiply the frequency-domain image with the Gaussian function to obtain the high-frequency partition line of the image. (4) Perform inverse Fourier transform on the obtained high-frequency region to obtain the high-frequency region edge map.

4. The method for jointly identifying spectral regions for deep learning according to claim 2, characterized in that: Perform recognition block segmentation based on the high-frequency edge map to achieve low-frequency region determination and division of the frequency-domain features. The specific process is as follows: (1) Initialize the index variables x and y to 0, 0, initialize the block area label layer c = 1, initialize the high-frequency edge threshold h, initialize the all-0 partition matrix K with the same size as the high-frequency edge map, and initialize the row connection list r to be empty. (2) Access the pixel point L(x, y) in the high-frequency region edge map. If L(x, y) < h, then assign and mark the corresponding partition matrix K(x, y) with the block label c, let x = x + 1, and add the current x value of the row to the row connection list r. Repeat step (2). If L(x, y) ≥ h, then increase y, y = y + 1. (3) Assign the values in the row connection list r to x in turn until L(x, y) < h, then stop the sequential assignment and empty the row connection list r; if all values satisfy L(x, y) ≥ h, then make c = c + 1. (4) Start traversing from x, y = 0, 0 until a point is found that satisfies K(x, y) = 0 and K(x, y) < h. If no point that meets the conditions is found, then enter step (5). (5) Starting from x, y = 0, 0, traverse. If a point where K(x, y + 1)·K(x, y)·(K(x, y + 1) - K(x, y)) ≠ 0 appears, then change the value equal to max(K(x, y + 1), K(x, y)) in K to min(K(x, y + 1), K(x, y)).

5. The method for jointly identifying spectral regions for deep learning according to claim 3, characterized in that: The spectral contribution degree of the wavelength is calculated according to the following formula: where q i represents the spectral contribution degree of the i-th wavelength, A represents all A different spectral categories, i and j represent any two different spectral categories, and α represents the currently calculated spectral category; for each collected spectral wavelength, the contribution degree q i is calculated. After calculating all the wavelengths, the wavelength channel with the highest contribution degree is selected.

6. The method for jointly identifying spectral regions for deep learning according to claim 3, characterized in that: The frequency domain conversion of the image is implemented in the following way: In the formula, ω1 and ω2 represent two spectral dimensions, and m and n represent two axial directions of the image respectively.

7. The method for jointly identifying spectral regions for deep learning according to claim 2, characterized in that: In step S4, during the diffusion process, the Gaussian function is used for diffusion. The confidence level in the central region is 1, and the confidence level in the edge region gradually decreases. Regions with a confidence level higher than 0.5 are not subjected to secondary identification; subsequently, the regions to be identified are identified, and the variance of the Gaussian function is selected according to the degree of continuity of the substance.

Images