Microscopic hyperspectral data initialization method based on clustering

Abstract

The invention relates to the technical field of deep learning, in particular to a clustering-based microscopic hyperspectral data initialization method, which comprises the following steps of: firstly, clustering a color image, and recording an edge feature position and a non-edge feature position; same category clustering is carried out on the low-resolution hyperspectral image, and similar substances are merged through the Euclidean distance; for non-edge feature positions, spectral weighted average values of substances of the same category are adopted for filling, and edge feature positions are replaced by spectral average values of adjacent pixels, so that the accuracy of microscopic hyperspectral data initialization is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and in particular relates to a clustering-based microscopic hyperspectral data initialization method. Background Art

[0002] Microscopic hyperspectral images are widely used in fine structure and composition analysis of samples due to their high spectral resolution, multi-band coverage and unified spectrum, and show great potential in fields such as medicine and materials science. However, limited by the spatial and spectral resolution of the imaging system, microscopic hyperspectral images usually show low spatial resolution, which limits their application scope. Hyperspectral image fusion technology fuses multiple images collected by different sensors through specific algorithms to achieve the unity of high spatial and high spectral resolution. It is a key method to solve the resolution limitations of microscopic hyperspectral imaging systems.

[0003] In recent years, deep learning technology has been successfully applied to image fusion tasks. Existing hyperspectral image fusion methods based on convolutional neural networks usually require upsampling low-resolution hyperspectral images to achieve high spatial resolution hyperspectral data initialization. Currently, the commonly used methods for upsampling low-resolution hyperspectral images are bicubic interpolation and big data learning. However, the bicubic interpolation method only considers the surrounding data and ignores the wider pixel information, which may affect the interpolation accuracy. The effect of big data training is highly dependent on the amount of data and the quality of the learning process. Summary of the invention

[0004] In view of this, the invention aims to provide a clustering-based microscopic hyperspectral data initialization method, which obtains the global pixel information of high spatial resolution hyperspectral images through clustering; using the global pixel information, the edge position and non-edge position are considered respectively to complete the initialization of the high spatial resolution hyperspectral image, and high-accuracy high spatial resolution microscopic hyperspectral data initialization can be achieved without a large amount of data training To achieve the above object, the technical solution created by the present invention is implemented as follows: A clustering-based microscopic hyperspectral data initialization method, comprising: S1: construct the corresponding high spatial resolution hyperspectral image to be initialized according to the color image; S2: clustering the color image in step S1, and determining edge feature positions and non-edge feature positions of the high spatial resolution hyperspectral image according to the color image; S3: converting the low-resolution hyperspectral image corresponding to the color image into a low-resolution color image, and clustering the low-resolution color image; S4: class matching between color images and low-resolution color images; S5: According to the edge feature positions and non-edge feature positions determined in step S2, the high spatial resolution hyperspectral image is filled using the result of the category matching in step S4 to complete the initialization of the high spatial resolution hyperspectral image.

[0005] Furthermore, in step S1, a high spatial resolution hyperspectral image in which all pixel values ​​are 0 is constructed based on the color image, and the size of the high spatial resolution hyperspectral image is consistent with that of the color image.

[0006] Furthermore, in step S2, the color image is clustered using the K-means method to obtain multiple fixed categories; the edge feature map of each channel image in the color image is extracted respectively, and then the edge feature map is integrated; each edge feature position in the integrated edge feature map is recorded, and the edge feature position corresponds to the edge feature position of the high spatial resolution hyperspectral image.

[0007] Furthermore, in step S3, the low-resolution color image is clustered using the K-means method, and the number of categories in the color image is consistent with the number of categories in the low-resolution color image.

[0008] Further, step S4 includes: S41: respectively calculating the first centroid of each category in the color image and the second centroid of each category in the low-resolution color image; S42: Using the first centroid and the second centroid obtained in step S41, match the categories in the color image and the low-resolution color image.

[0009] Further, in step S41, the first centroid is obtained by the following formula: ; in, represents the first centroid of the i-th category in the color image, , represents the total number of categories in the color image, , and Respectively represent the average pixel value of the i-th category in the R channel, G channel, and B channel of the color image; The second centroid is obtained by the following formula: ; in, represents the second centroid of the jth category in the low-resolution color image, , represents the total number of categories in the low-resolution color image, , and Respectively represent the average pixel value of the jth category in the R channel, G channel, and B channel of the low-resolution color image.

[0010] Further, in step S42, the Euclidean distance between the first centroid and the second centroid is calculated by the following formula: ; in, represents the Euclidean distance; The categories corresponding to the first centroid and the second centroid with the smallest Euclidean distance are the same category.

[0011] Further, step S5 includes: S51: For the non-edge feature position, the category to which the corresponding pixel of the high spatial resolution hyperspectral image belongs is determined through the color image, and the pixels of the same category in the low resolution hyperspectral image are weighted and filled into the non-edge feature position; S52: For the edge feature position, the mean of the pixel values ​​of the adjacent positions of the edge feature position is calculated, and the mean is filled into the edge feature position to complete the initialization of the high spatial resolution hyperspectral image.

[0012] Further, in step S51: Through color images Each pixel , determine the high spatial resolution hyperspectral image Corresponding pixels The category to which it belongs; Low-resolution hyperspectral images Medium and high spatial resolution hyperspectral images Pixels As the reference pixel, the high spatial resolution hyperspectral image is calculated by the following formula The weight of the corresponding category pixel in: ; in, Represents high spatial resolution hyperspectral images The pixels of the corresponding category in The weight of Represents a low-resolution hyperspectral image Medium Pixels Hyperspectral images with high spatial resolution Pixels of the same category The Euclidean distance between Each pixel in the non-edge feature position is obtained by the following formula: ; in, Indicates category.

[0013] Furthermore, low-resolution hyperspectral images Pixels in Hyperspectral images with high spatial resolution The corresponding pixel satisfy: ; in, Represents a round-towards-zero operation.

[0014] Compared with the prior art, the invention can achieve the following beneficial effects: (1) The clustering-based microscopic hyperspectral data initialization method created by the present invention first clusters the color image data and records the spatial position of the edge features; then, the low-resolution hyperspectral images are clustered into the same category and the same type of substances are merged by Euclidean distance; for non-edge feature positions, the spectral weighted average value of the same type of substances is used to fill, and the edge feature positions are replaced by the average value of the spectral curves of adjacent pixels, thereby improving the accuracy of the microscopic hyperspectral data initialization; (2) Compared with the bicubic interpolation method, the clustering-based microscopic hyperspectral data initialization method created by the present invention not only considers the surrounding data but also the global pixel information, and the initialized microscopic hyperspectral data has a higher accuracy rate; compared with the big data training method, the present invention does not require a large amount of training data to complete the high-accuracy initialization of microscopic hyperspectral data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of a process flow of a clustering-based microscopic hyperspectral data initialization method according to an embodiment of the present invention; Figure 2 A flowchart of a clustering-based microscopic hyperspectral data initialization method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0017] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0018] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0019] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0020] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0021] like Figure 1 to Figure 2 As shown, the clustering-based microscopic hyperspectral data initialization method described in the embodiment of the present invention includes: S1: Construct a corresponding high spatial resolution hyperspectral image to be initialized based on the color image.

[0022] In some embodiments, a high spatial resolution hyperspectral image with all pixel values ​​being 0 is constructed based on the color image, and the size of the high spatial resolution hyperspectral image is consistent with the color image. It can be understood that the color image is represented as , and Respectively represent the length and width of the color image, d represents the number of spectral channels, and in the color image the number of spectral channels is d=3, including R channel, G channel and B channel. Since the size of the high spatial resolution hyperspectral image to be initialized is consistent with that of the color image, the high spatial resolution hyperspectral image can be expressed as , D represents the number of spectral channels of the high spatial resolution hyperspectral image.

[0023] S2: Clustering the color image in step S1, and determining edge feature positions and non-edge feature positions of the high spatial resolution hyperspectral image based on the color image.

[0024] In some embodiments, the color image is clustered using the K-means method to obtain multiple fixed categories. It is understandable that the number of categories is related to the data type. The edge feature map of each channel image in the color image is extracted respectively; the edge feature map is integrated; and each edge feature position in the integrated edge feature map is recorded, and the edge feature position corresponds to the edge feature position of the high spatial resolution hyperspectral image.

[0025] In one embodiment, the Sobel or Canny edge detection algorithms are used to extract the color images. The edge feature maps of the three channel images are obtained respectively, and the edge feature maps of the R channel are obtained , edge feature map of G channel And the edge feature map of the B channel Then, the edge feature maps of the three channel images are integrated by taking the maximum value to obtain the integrated edge feature map , as follows: ; in, Represents the pixels at each point in the integrated edge feature map, , and Respectively represent the pixels at each point in the edge feature map of the R channel, G channel, and B channel, Indicates taking the maximum value.

[0026] The position of each edge feature in the integrated edge feature map is recorded. The edge feature position corresponds to the edge feature position of the high spatial resolution hyperspectral image. It can be understood that the non-edge feature position in the color image also corresponds to the non-edge feature position of the high spatial resolution hyperspectral image.

[0027] S3: Convert the low-resolution hyperspectral image corresponding to the color image into a low-resolution color image, and cluster the low-resolution color image.

[0028] It can be understood that the number of channels of the low-resolution hyperspectral image should be consistent with the number of channels of the high spatial resolution hyperspectral image, so the low-resolution hyperspectral image can be expressed as , M and N represent the length and width of the low-resolution hyperspectral image, respectively. In addition, the size of the low-resolution color image should be consistent with the size of the low-resolution hyperspectral image, and the number of channels of the low-resolution color image should be consistent with the number of channels of the color image, so the low-resolution color image can be expressed as .

[0029] In one embodiment, a spectral response function is used to complete a low-resolution high-spectral image. To low resolution color image The conversion is as follows: ; in, Represents the spectral response function of the capture device.

[0030] In some embodiments, the low-resolution color image is clustered using a K-means method, and the number of categories in the color image is consistent with the number of categories in the low-resolution color image.

[0031] S4: Class matching between color images and low-resolution color images.

[0032] In some embodiments, multiple categories of low-resolution hyperspectral images and low-resolution color images are compared by Euclidean distance, and two categories with the smallest distance are selected as the same category to complete category mapping. Specifically, step S4 includes: S41: Calculate the first centroid of each category in the color image and the second centroid of each category in the low-resolution color image respectively.

[0033] In one embodiment, the first centroid is obtained by the following formula: ; in, Represents a color image The first centroid of the i-th category in , , Represents a color image The total number of categories in , and Represents color images The average pixel value of the i-th category in the R channel, G channel and B channel of ; The second centroid is obtained by the following formula: ; in, Represents a low-resolution color image The second centroid of the jth category in , , Represents a low-resolution color image The total number of categories in , and Represent low-resolution color images The average pixel value of the jth category in the R channel, G channel, and B channel of .

[0034] S42: Using the first centroid and the second centroid obtained in step S41, match the categories in the color image and the low-resolution color image.

[0035] In one embodiment, the Euclidean distance between the first centroid and the second centroid is calculated by the following formula: ; in, represents the Euclidean distance; The categories corresponding to the first centroid and the second centroid with the smallest Euclidean distance are the same category.

[0036] S5: According to the edge feature positions and non-edge feature positions determined in step S2, the high spatial resolution hyperspectral image is filled using the result of the category matching in step S4 to complete the initialization of the high spatial resolution hyperspectral image.

[0037] In some embodiments, for non-edge feature positions, the weighted average of the full-image spectral data of the same type of material is used for filling; for edge feature positions, the average of the spectral curves of the four adjacent positions in the vertical and horizontal directions is used for replacement, and finally the initialization of the high spatial resolution hyperspectral image data is completed. Specifically, step S5 includes: S51: For non-edge feature positions, the category to which the corresponding pixel of the high spatial resolution hyperspectral image belongs is determined through the color image, and the pixels of the same category in the low resolution hyperspectral image are weighted and filled into the non-edge feature position.

[0038] In one embodiment, a color image Each pixel , determine the high spatial resolution hyperspectral image Corresponding pixels The category to which it belongs.

[0039] Low-resolution hyperspectral images Medium and high spatial resolution hyperspectral images Pixels As the reference pixel, the high spatial resolution hyperspectral image is calculated by the following formula The weight of the corresponding category pixel in: ; in, Represents high spatial resolution hyperspectral images The pixels of the corresponding category in The weight of Represents a low-resolution hyperspectral image Medium Pixels Hyperspectral images with high spatial resolution Pixels of the same category The Euclidean distance between them is: ; is a very small positive number, which ensures that the denominator never actually becomes zero, thus avoiding division by zero errors. In practical applications, it needs to be set based on experience and can be set to a predefined minimum positive floating point number.

[0040] Each pixel in the non-edge feature position is obtained by the following formula: ; in, Indicates category.

[0041] High spatial resolution hyperspectral images Pixels in Compared with low-resolution hyperspectral images Pixels Satisfy between: ; Represents a round-towards-zero operation.

[0042] S52: For the edge feature position, calculate the average of the pixel values ​​of the adjacent positions of the edge feature position, and fill the edge feature position with the average.

[0043] In one embodiment, the pixels of the four neighborhoods of the edge feature position are calculated. and The mean of , and fill the mean into the edge feature position, specifically: ; At this point, high spatial resolution hyperspectral images are completed Initialization.

[0044] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0045] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A clustering-based microscopic hyperspectral data initialization method, characterized in that: include: S1: construct the corresponding high spatial resolution hyperspectral image to be initialized according to the color image; S2: clustering the color image in step S1, and determining edge feature positions and non-edge feature positions of the high spatial resolution hyperspectral image according to the color image; S3: converting the low-resolution hyperspectral image corresponding to the color image into a low-resolution color image, and clustering the low-resolution color image; S4: performing category matching on the color image and the low-resolution color image; S5: According to the edge feature positions and non-edge feature positions determined in step S2, the high spatial resolution hyperspectral image is filled using the result of the category matching in step S4 to complete the initialization of the high spatial resolution hyperspectral image.

2. The clustering-based microscopic hyperspectral data initialization method according to claim 1, characterized in that: In step S1: The high spatial resolution hyperspectral image in which all pixel values ​​are 0 is constructed according to the color image, and the size of the high spatial resolution hyperspectral image is consistent with that of the color image.

3. The clustering-based microscopic hyperspectral data initialization method according to claim 1, characterized in that: In step S2: Clustering the color image using a K-means method to obtain a plurality of fixed categories; Extracting edge feature maps of each channel image in the color image respectively, and then integrating the edge feature maps; Each edge feature position in the integrated edge feature map is recorded, and the edge feature position corresponds to the edge feature position of the high spatial resolution hyperspectral image.

4. The clustering-based microscopic hyperspectral data initialization method according to claim 1, characterized in that: In step S3: The low-resolution color image is clustered using a K-means method, and the number of categories in the color image is consistent with the number of categories in the low-resolution color image.

5. The clustering-based micro-hyperspectral data initialization method according to claim 1, characterized in that: Step S4 includes: S41: respectively calculating a first centroid of each category in the color image and a second centroid of each category in the low-resolution color image; S42: Using the first centroid and the second centroid obtained in step S41, matching each category in the color image and the low-resolution color image.

6. The clustering-based microscopic hyperspectral data initialization method according to claim 5, characterized in that: In step S41, the first centroid is obtained by the following formula: ； in, represents the first centroid of the ith category in the color image, , represents the total number of categories in the color image, , and Respectively represent the average pixel value of the i-th category in the R channel, G channel and B channel of the color image; The second centroid is obtained by the following formula: ； in, represents the second centroid of the j-th category in the low-resolution color image, , represents the total number of categories in the low-resolution color image, , and Respectively represent the average pixel value of the jth category in the R channel, G channel and B channel of the low-resolution color image.

7. The clustering-based microscopic hyperspectral data initialization method according to claim 6, characterized in that: In step S42, the Euclidean distance between the first centroid and the second centroid is calculated by the following formula: ； in, represents the Euclidean distance; The categories corresponding to the first centroid and the second centroid with the smallest Euclidean distance are the same category.

8. The clustering-based micro-hyperspectral data initialization method according to claim 1, characterized in that: Step S5 includes: S51: For the non-edge feature position, determine the category to which the corresponding pixel of the high spatial resolution hyperspectral image belongs through the color image, and weightedly fill the pixels of the same category in the low resolution hyperspectral image into the non-edge feature position; S52: For the edge feature position, calculate the average of the pixel values ​​of the adjacent positions of the edge feature position, and fill the edge feature position with the average, thereby completing the initialization of the high spatial resolution hyperspectral image.

9. The clustering-based microscopic hyperspectral data initialization method according to claim 8, characterized in that: In step S51: Through color images Each pixel , determine the high spatial resolution hyperspectral image Corresponding pixels The category to which it belongs; Low-resolution hyperspectral images Medium and high spatial resolution hyperspectral images Pixels As the reference pixel, the high spatial resolution hyperspectral image is calculated by the following formula The weight of the corresponding category pixel in: ； in, Represents high spatial resolution hyperspectral images The pixels of the corresponding category in The weight of Represents a low-resolution hyperspectral image Medium Pixels Hyperspectral images with high spatial resolution Pixels of the same category The Euclidean distance between Each pixel in the non-edge feature position is obtained by the following formula: ； in, Indicates category.

10. The clustering-based microscopic hyperspectral data initialization method according to claim 9, characterized in that: The low-resolution hyperspectral image Pixels in With the high spatial resolution hyperspectral image The corresponding pixel satisfy: ； in, Represents a round-towards-zero operation.

Images