Pixel point classification method and device, processor and electronic device

Abstract

The application discloses a pixel point classification method and device, a processor and electronic equipment. It is related to the field of artificial intelligence, and the method comprises the following steps: acquiring M spectral features corresponding to M pixel points in a spectral image, wherein the feature dimension number of each of the M spectral features matches the waveband number used when the spectral image is shot; determining M spatial features corresponding to the M pixel points according to the M spectral features corresponding to the M pixel points, wherein any one of the M spatial features comprises the spectral features of the adjacent pixel points of the corresponding pixel point; fusing the spectral features and the spatial features of the M pixel points to obtain M fusion features; and classifying the pixel points in the spectral image according to the M fusion features to obtain a pixel point classification result. The application solves the technical problem of low classification result accuracy when classifying the pixel points in a hyperspectral image.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a pixel classification method, apparatus, processor, and electronic device. Background Technology

[0002] Hyperspectral images contain rich spectral and spatial features, and their near-continuous spectrum and unified image-spectral structure make them effective for distinguishing land cover types. Therefore, hyperspectral image-based classification tasks are widely used in geological surveying, precision agriculture, ecological environment, and land feature mapping. Traditional hyperspectral image-based classification methods are mainly divided into two categories: classification methods based on spectral information and classification methods based on combined spectral and spatial information. Methods based on the first category only consider the spectral information of individual pixels, resulting in low classification accuracy and poor robustness. Methods based on the second category, while considering both spatial and spectral information of individual pixels, neglect the correlation between pixels, leading to persistent problems of low accuracy and poor robustness in the classification results.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a pixel classification method, apparatus, processor, and electronic device to at least solve the technical problem of low classification accuracy when classifying pixels in hyperspectral images.

[0005] According to one aspect of the present invention, a pixel classification method is provided, comprising: acquiring M spectral features corresponding to M pixels in a spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; determining M spatial features corresponding to the M pixels based on the M spectral features, wherein any one of the M spatial features includes the spectral features of adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent region of the pixel corresponding to the arbitrary spatial feature; fusing the spectral features and spatial features of the M pixels to obtain M fused features corresponding to the M pixels; and classifying the pixels in the spectral image based on the M fused features to obtain a pixel classification result for the pixels in the spectral image.

[0006] Optionally, fusing the spectral and spatial features corresponding to each of the M pixels to obtain M fused features corresponding to each of the M pixels includes: concatenating the spectral and spatial features of each of the M pixels into vectors to construct M concatenated feature vectors corresponding to each of the M pixels; constructing an initial feature matrix based on the M concatenated feature vectors; and processing the initial feature matrix using principal component analysis to obtain a feature fusion matrix, wherein the feature fusion matrix includes the M fused features corresponding to each of the M pixels.

[0007] Optionally, the step of processing the initial feature matrix using principal component analysis to obtain the feature fusion matrix includes: normalizing the initial feature matrix to obtain a normalized feature matrix; determining the covariance of the normalized feature matrix to obtain a covariance matrix; determining the eigenvalues ​​and eigenvectors of the covariance matrix, wherein the eigenvalues ​​and eigenvectors of the covariance matrix correspond one-to-one; selecting p principal component eigenvalues ​​from the eigenvalues ​​of the covariance matrix based on the eigenvalues ​​and a preset contribution rate threshold; and constructing the feature fusion matrix based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix.

[0008] Optionally, constructing the feature fusion matrix based on the p feature vectors corresponding to the p principal component feature values ​​and the normalized feature matrix includes: transposing the p feature vectors corresponding to each of the p principal component feature values ​​to obtain p transposed feature vectors; multiplying the p transposed feature vectors by the normalized feature matrix to obtain p fused feature column vectors; and constructing the feature fusion matrix based on the p fused feature column vectors, wherein the feature fusion matrix includes M rows and p columns, the M rows representing the M fused features corresponding to the M pixels, and the p columns representing that each of the M fused features has a feature dimension of p.

[0009] Optionally, classifying the pixels in the spectral image based on the M fusion features to obtain the pixel classification result of the pixels in the spectral image includes: constructing a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationship between the M nodes in the node graph; assigning weight values ​​to the edges in the node graph according to the M fusion features; determining the graph adjacency matrix of the spectral image according to the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationship between the M pixels in the spectral image; and inputting the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain the pixel classification result of the pixels in the spectral image.

[0010] Optionally, assigning weight values ​​to the edges in the node graph according to the M fusion features includes: determining the length of each edge in the node graph, wherein the length of an edge in the node graph is the Euclidean distance between the two nodes connected by the edge; assigning a weight value to the first edge in the node graph according to the M fusion features when the length of the first edge included in the edge of the node graph is less than or equal to the maximum value of the adjacent region; and setting the weight value of the second edge in the node graph to zero when the length of the second edge included in the edge of the node graph is greater than the maximum value of the adjacent region.

[0011] Optionally, assigning a weight value to the first edge in the node graph based on the M fusion features includes: determining the first fusion feature and the second fusion feature corresponding to the nodes connected by the first edge from the M fusion features; and determining the weight value of the first edge based on the norm of the first fusion feature, the norm of the second fusion feature, and the inner product between the first fusion feature and the second fusion feature.

[0012] According to another aspect of the present invention, a pixel classification device is also provided, comprising: an acquisition module, configured to acquire M spectral features corresponding to M pixels in a spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; a determination module, configured to determine M spatial features corresponding to the M pixels based on the M spectral features corresponding to the M pixels, wherein any one of the M spatial features includes the spectral features of adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent region of the pixel corresponding to the arbitrary spatial feature; a fusion module, configured to fuse the spectral features and spatial features of the M pixels to obtain M fused features corresponding to the M pixels; and a classification module, configured to classify the pixels in the spectral image based on the M fused features to obtain a pixel classification result of the pixels in the spectral image.

[0013] According to another aspect of the present invention, a processor is also provided, the processor being configured to run a program, wherein the program, when running, executes the pixel classification method described in any of the preceding embodiments.

[0014] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the pixel classification method described in any of the preceding embodiments.

[0015] In this embodiment of the invention, a method that simultaneously considers the spectral and spatial features of a hyperspectral image is adopted. By fusing the spectral and spatial features of each of the M pixels in the spectral image into M fused features, and then classifying the pixels in the spectral image based on the M fused features, the purpose of considering both spectral and spatial features of the hyperspectral image is achieved when classifying pixels in the hyperspectral image. This improves the accuracy of pixel classification in the hyperspectral image and solves the technical problem of low classification accuracy when classifying pixels in the hyperspectral image. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart of a pixel classification method provided according to an embodiment of this application;

[0018] Figure 2 This is a schematic flowchart of a pixel classification method provided by an optional embodiment of the present invention;

[0019] Figure 3 This is a structural block diagram of a pixel classification device provided according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0023] Hyperspectral imaging utilizes hyperspectral sensors (imaging spectrometers) mounted on various space platforms to simultaneously image a target region across the ultraviolet, visible, near-infrared, and mid-infrared regions of the electromagnetic spectrum, employing dozens to hundreds of continuous and subdivided spectral bands. This achieves a combination of spectral and image information, acquiring both the landmark image and its spectral data. However, related technologies suffer from low accuracy and poor robustness in classifying pixels within hyperspectral images. To address these issues, this invention provides a pixel classification method for hyperspectral images based on the fusion of multiple features.

[0024] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a pixel classification method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0025] Step S101: Obtain M spectral features corresponding to M pixels in the spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image.

[0026] Optionally, the spectral image in this embodiment can be a hyperspectral image obtained by acquiring the target area. The acquired spectral image can include multi-band spectral features, each spectral feature corresponding to a pixel in the hyperspectral image, and the feature dimension of each spectral feature can be N-dimensional. The feature value of each dimension in the N-dimensional spectral feature corresponds to a band used when acquiring the spectral image. The total number of bands used when capturing the spectral image in step S101 can be N or slightly greater than N, and is not limited here. When the total number of bands used in the spectral image is slightly greater than N, step S101 can use the features corresponding to the N bands with higher data reliability to construct the N dimensions of the spectral feature in this step. Optionally, the spectral feature can be represented as... Where N represents the number of spectral bands. Therefore, the spectral feature matrix composed of all pixels in a hyperspectral image can be represented as: Where M represents the number of pixels in the hyperspectral image.

[0027] Step S102: Based on the M spectral features corresponding to the M pixels, determine the M spatial features corresponding to the M pixels. Each of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent region of the pixel corresponding to any spatial feature.

[0028] Step S102 can determine the spatial features of each pixel in the spectral image. The spatial features of the pixels can be used to characterize the relationship between the pixels in the spectral image and other pixels around them, and thus can be used to improve the accuracy of classification and recognition of each pixel.

[0029] Optionally, the adjacent regions and adjacent pixels of each pixel can be defined as follows: For any pixel in the spectral image, define a window of size s×s, and take this arbitrary pixel as the center of the window. Then, the area covered by the s×s window can be considered as the adjacent region of this arbitrary pixel, and all pixels within this adjacent region can be considered as the adjacent pixels of this arbitrary pixel. Those skilled in the art will understand that since neighboring pixels in an image usually have a certain correlation, this correlation can be manifested in the correlation of spectral features between neighboring pixels across various dimensions. Therefore, constructing the spatial features of each pixel in the spectral image using the method of this application can effectively consider the classification hints given by these correlated features in pixel classification tasks. Optionally, the spatial features can be represented as... Where l represents the vector dimension of the spatial features, l = s × s × N. For M pixels in a hyperspectral image, we represent the spatial feature matrix as follows:

[0030] Step S103 involves fusing the spectral and spatial features of each of the M pixels to obtain M fused features corresponding to each of the M pixels. In this step, since each of the M pixels corresponds to one spectral feature and one spatial feature, the spectral and spatial features corresponding to a single pixel can be fused to obtain the fused feature for that pixel. Repeating this process yields the M fused features corresponding to each of the M pixels. Those skilled in the art will understand that the fused features constructed in this step have better expressive power than individual spectral or spatial features. The fused features obtained through feature fusion contain richer and more effective pixel sample information. Therefore, using fused features that include richer pixel information for pixel classification can improve the accuracy and robustness of pixel classification tasks.

[0031] As an optional embodiment, fusing the spectral and spatial features corresponding to each of the M pixels to obtain M fused features corresponding to each of the M pixels can include the following steps: concatenating the spectral and spatial features of each of the M pixels into vectors to construct M concatenated feature vectors corresponding to each of the M pixels; constructing an initial feature matrix based on the M concatenated feature vectors; and processing the initial feature matrix using principal component analysis to obtain a feature fusion matrix, wherein the feature fusion matrix includes the M fused features corresponding to each of the M pixels.

[0032] Alternatively, vector concatenation can be performed as follows: for the spectral features of each pixel... and spatial features First, the spectral features and spatial features are concatenated into vectors (also known as vector flattening) to obtain M concatenated feature vectors. Among them, E i =[f i ,R i ], d = N + l. Therefore, after vector concatenation, the concatenated feature vectors of M pixels can form an initial feature matrix, which can be expressed as:

[0033] Principal Component Analysis (PCA) is a statistical method that can transform a set of potentially correlated variables into a set of linearly uncorrelated variables through orthogonal transformation. This application uses PCA to reduce the dimensionality of eigenvectors, making the feature dimension of the fused features in the feature fusion matrix lower than that in the initial feature matrix. At the same time, it ensures that the fused features in the feature fusion matrix also have strong expressive power and can well express the features of the pixels, which is convenient for subsequent pixel classification.

[0034] As an optional embodiment, the initial feature matrix is ​​processed using principal component analysis to obtain a feature fusion matrix, which may include the following steps: normalizing the initial feature matrix to obtain a normalized feature matrix; determining the covariance of the normalized feature matrix to obtain a covariance matrix; determining the eigenvalues ​​and eigenvectors of the covariance matrix, wherein the eigenvalues ​​and eigenvectors of the covariance matrix correspond one-to-one; selecting p principal component eigenvalues ​​from the eigenvalues ​​of the covariance matrix based on the eigenvalues ​​of the covariance matrix and a preset contribution rate threshold; and constructing the feature fusion matrix based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix.

[0035] Normalization, also known as standardization, can optionally be performed on the feature matrix F using the zero-mean (z-score) method. flatten In an embodiment of normalization, firstly, for ease of description, the initial feature matrix F can be... flatten It can be represented in the following form:

[0036]

[0037] Among them, f i,j This represents the value of the j-th feature in the i-th pixel of the spectral image. This represents the eigenvector corresponding to the j-th feature dimension of the spectral image.

[0038] During normalization, the mean of the feature vectors corresponding to each feature dimension is first calculated column-wise. and standard deviation S j Then, the normalized eigenvalue matrix F is calculated. f ′ latten The calculation method is as follows:

[0039]

[0040]

[0041]

[0042]

[0043] F i ′ ,j F represents the normalized value of the feature value of the j-th feature dimension in the i-th pixel; j ′ represents the normalized feature vector corresponding to the j-th feature dimension.

[0044] Optionally, determining the covariance matrix corresponding to the normalized feature matrix, and determining the p principal component eigenvalues ​​based on the covariance matrix, can be achieved through the following steps:

[0045] The normalized feature matrix F f ′ latten The corresponding covariance matrix is ​​expressed as The calculation method is as follows:

[0046]

[0047]

[0048] In the above formula, k represents the count. Equivalent to F′ k,i F′ k,j Start from 1 and accumulate up to M. For example,

[0049] Calculate the eigenvalues ​​and eigenvectors of the covariance matrix A.

[0050] Aβ=λβ

[0051] |λΕ-A|=0

[0052] Where E represents the identity matrix, and λ = [λ1, λ2, ..., λ3] 2, …,λ d ] represents the eigenvalues ​​of the covariance matrix A, and the following relationship holds: λ1≥λ2≥…≥λ d >0; β=[β1,β 2, …,β d ] represents the eigenvectors of the covariance matrix A. Represents the eigenvalue λ i The eigenvectors of the corresponding covariance matrix.

[0053] Furthermore, a contribution rate threshold of α can be set. If the cumulative contribution rate of the first p eigenvalues ​​is greater than or equal to α, then these p principal components can be considered to represent the original d features. The calculation method for the cumulative contribution rate of eigenvalues ​​is as follows:

[0054]

[0055] After obtaining p principal components, the first p eigenvalues ​​(λ) of the covariance matrix can be used. i The eigenvectors (β) of the corresponding covariance matrix i The spectral features and spatial features are fused to obtain the feature fusion matrix.

[0056] As an optional embodiment, a feature fusion matrix is ​​constructed based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix. This includes: transposing the p eigenvectors corresponding to each of the p principal component eigenvalues ​​to obtain p transposed feature vectors; multiplying the p transposed feature vectors by the normalized feature matrix to obtain p fused feature column vectors; and constructing a feature fusion matrix based on the p fused feature column vectors. The feature fusion matrix comprises M rows and p columns, where the M rows represent the M fused features corresponding to the M pixels, and the p columns indicate that each of the M fused features has a feature dimension of p.

[0057] It should be noted that p is an integer less than d. By converting the initial feature matrix of d columns into a fused feature matrix of p columns through principal component analysis, the d-dimensional cascaded feature vectors are reduced to p-dimensional fused features. This reduces the complexity of the pixel classification task, enabling graph convolutional neural networks to quickly classify pixels in hyperspectral images and avoid graph convolutional neural networks not responding to output results or failing to complete calculations for a long time.

[0058] Alternatively, the feature fusion matrix can be represented as The calculation method is as follows:

[0059]

[0060] Among them, Q1 to Q p any Q in i It can be represented as

[0061]

[0062] Where F'1 represents the normalized characteristic matrix F' flatten The first column, F'2, represents the normalized characteristic matrix F' flatten The second column, and so on, F' d F' represents the normalized characteristic matrix flatten The d-th column. β 1i The eigenvector β of the covariance matrix i The first eigenvalue in , and so on, β di The eigenvector β of the covariance matrix i The d-th eigenvalue in the matrix.

[0063] The feature fusion matrix Q can be constructed using the above method. combine .

[0064] Step S104: Classify the pixels in the spectral image according to the M fusion features to obtain the pixel classification results of the pixels in the spectral image.

[0065] As an optional embodiment, classifying pixels in a spectral image based on M fusion features to obtain pixel classification results can include: constructing a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationships between the M nodes; assigning weight values ​​to the edges in the node graph based on the M fusion features; determining the graph adjacency matrix of the spectral image based on the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationships between the M pixels in the spectral image; and inputting the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain the pixel classification results of the pixels in the spectral image.

[0066] As an optional embodiment, weight values ​​are assigned to the edges in the node graph according to M fusion features, including: determining the length of each edge in the node graph, wherein the length of an edge in the node graph is the Euclidean distance between the two nodes connected by the edge; assigning a weight value to the first edge in the node graph according to the M fusion features when the length of the first edge included by the edge in the node graph is less than or equal to the maximum value of the adjacent region; and setting the weight value of the second edge in the node graph to zero when the length of the second edge included by the edge in the node graph is greater than the maximum value of the adjacent region.

[0067] As an optional embodiment, assigning a weight value to the first edge in the node graph based on M fusion features includes: determining the first fusion feature and the second fusion feature corresponding to the nodes connected by the first edge from the M fusion features; and determining the weight value of the first edge based on the norm of the first fusion feature, the norm of the second fusion feature, and the inner product between the first fusion feature and the second fusion feature.

[0068] For each pixel in a spectral image, it can be viewed as a node in a graph structure, constructing a node graph, such as an undirected graph G(v,e), where v represents the set of M nodes in the node graph, i.e., v = [v1, v2, ..., v3]. M ], where e represents the set of edges in the graph structure, with each edge connecting two nodes. Optionally, the graph adjacency matrix can be defined as follows: Let C represent the structure of the node graph G(v,e). i,j Represents node v in a node graph structurei With node v j The connection relationship. Optionally, the connection relationship of the node v can be set. j At node v i Within the adjacent region, it represents node v i With node v j They are connected, connecting node v i With node v j The edge formed is denoted as e i,j , and for e i,j Assign weight value w i,j , used to represent node v i With node v j The correlation between them, i.e., C i,j =w i,j If node v j Not in node v i Within the adjacent region, it represents node v i With node v j If they are not connected, assign a weight of 0, i.e., C. i,j =0.

[0069] Optionally, the weight value w i,j The calculation method can be as follows:

[0070]

[0071]

[0072]

[0073] Where, μ i Represents node v i The corresponding pixel fusion feature, μ j Represents node v j The corresponding pixel fusion features; δ represents a statistical constant; <μ i ,μ j > represents the fusion feature μ i and fusion features μ j The inner product; ||μ i || represents the fusion feature μ i The norm of μ j || represents the fusion feature μ j The norm of the node; S represents the size of the adjacent region, D represents the Euclidean distance between nodes, (x i ,y i ) and (x j ,y j ) represent nodes v respectively i and node v jThe coordinates of S. It should be noted that the value of S can be determined based on the size of the adjacent region. By choosing a reasonable value for S, D(v) can be optimized. i ,v j Pixels with values ​​≤ S are all pixels within their adjacent regions, while D(v) i ,v j All pixels in S are pixels outside the adjacent region.

[0074] Based on the above description, the feature fusion matrix corresponding to each node (i.e., pixel) can be obtained through calculation. and node-based graph adjacency matrix Alternatively, the multi-feature fusion matrix Q can be... combine The graph adjacency matrix C is used as input to the graph convolutional neural network (GNN), which performs the pixel classification task. For the GNN, the convolution method between layers is defined as follows:

[0075] H (l+1) =RELU(BH (l) W (l) )

[0076]

[0077]

[0078] Among them, H (l) H represents the feature matrix of the l-th layer of the graph convolutional neural network. (0) =Q combine ; RELU(·) represents the linear rectified function; W (l) C represents the parameter matrix of the l-th layer of the graph convolutional neural network; C represents the graph adjacency matrix. yes The degree matrix is ​​denoted by I, where I represents the identity matrix. The degree matrix is ​​a diagonal matrix, meaning that all elements except those on the diagonal are 0. The diagonal elements represent the degree of each node. The degree is calculated as follows: for node i, the degree is calculated by summing the counts of all non-zero elements in the i-th row of the adjacency matrix.

[0079] In the training of graph convolutional neural networks (CNNs), the cross-entropy function can be used as the loss function. The training parameters in the CNN are optimized by minimizing the cross-entropy loss function. The expression for minimizing the cross-entropy loss function is shown below:

[0080]

[0081] in, Let ε represent the cross-entropy loss function. i Ω represents the true category label of the i-th node;i represents the label category result of the i-th node output by the graph convolutional neural network; L represents the number of training samples.

[0082] Assuming the graph convolutional neural network has k layers, the last layer can be a softmax layer to perform node classification.

[0083] Z (k) =softmax(BH (k-1) W (k-1) )

[0084] Z (k) This represents the node label category result output by the last layer of the graph convolutional neural network.

[0085] Through the above steps, by simultaneously considering the spectral and spatial features of the hyperspectral image, the spectral and spatial features of each of the M pixels in the spectral image are fused into M fused features. Then, the pixels in the spectral image are classified based on the M fused features. This achieves the goal of considering both spectral and spatial features of the hyperspectral image when classifying pixels in the hyperspectral image, thereby improving the accuracy of pixel classification in the hyperspectral image and solving the technical problem of low classification accuracy when classifying pixels in the hyperspectral image.

[0086] Figure 2 This is a flowchart illustrating a pixel classification method provided by an optional embodiment of the present invention, as shown below. Figure 2 As shown, spectral and spatial features can be extracted from each pixel in the hyperspectral image. Then, principal component analysis is used to fuse these features to obtain the fused features of each pixel. Simultaneously, a graph adjacency matrix can be constructed based on the fused features and the hyperspectral image. The fused features and graph adjacency matrix obtained in the above manner can then be input into a graph convolutional neural network (PCNN) to perform the pixel classification task and obtain the classification label for each pixel.

[0087] The pixel classification method provided in this application extracts the spectral and spatial features of pixels in a hyperspectral image. It then uses principal component analysis (PCA) to fuse these spectral and spatial features, resulting in a more expressive fused feature. Next, an adjacency matrix is ​​constructed based on the fused features to represent the correlation between pixels. Finally, a graph convolutional neural network (GNN) is built, using the fused feature matrix and adjacency matrix of all pixels as input to obtain the classification label for each pixel, thus achieving the hyperspectral image classification task.

[0088] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0090] According to embodiments of the present invention, an apparatus for implementing the above-described pixel classification method is also provided. Figure 3 This is a structural block diagram of a pixel classification device provided according to an embodiment of this application, such as... Figure 3 As shown, the device includes: an acquisition module 31, a determination module 32, a fusion module 33, and a classification module 34. The device will be described in detail below.

[0091] The acquisition module 31 is used to acquire M spectral features corresponding to M pixels in the spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image;

[0092] The determination module 32 is connected to the acquisition module 31 and is used to determine the M spatial features corresponding to the M pixels based on the M spectral features corresponding to the M pixels. Each of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent area of ​​the pixel corresponding to any one of the spatial features.

[0093] The fusion module 33 is connected to the determination module 32 and is used to fuse the spectral features and spatial features of M pixels to obtain M fused features corresponding to M pixels.

[0094] The classification module 34, connected to the fusion module 33, is used to classify the pixels in the spectral image according to M fusion features to obtain the pixel classification results of the pixels in the spectral image.

[0095] It should be noted that the above-mentioned acquisition module 31, determination module 32, fusion module 33 and classification module 34 correspond to steps S101 to S104 in the implementation of the pixel classification method. The multiple modules and the corresponding steps are the same in terms of implementation instances and application scenarios, but are not limited to the content disclosed in the above embodiments.

[0096] The pixel classification device provided in this application adopts a method that simultaneously considers the spectral and spatial features of hyperspectral images. By fusing the spectral and spatial features of each of the M pixels in the spectral image into M fused features, and then classifying the pixels in the spectral image based on the M fused features, it achieves the goal of considering both spectral and spatial features of the hyperspectral image when classifying pixels in the hyperspectral image. This solves the problem of low classification accuracy when classifying pixels in hyperspectral images in related technologies, thereby improving the accuracy of pixel classification in hyperspectral images.

[0097] The pixel classification device includes a processor and a memory. The acquisition module 31, determination module 32, fusion module 33 and classification module 34 mentioned above are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.

[0098] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured; adjusting kernel parameters improves the accuracy of pixel classification in hyperspectral images.

[0099] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0100] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements a pixel classification method.

[0101] This invention provides a processor for running a program, wherein the program executes a pixel classification method during runtime.

[0102] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention, such as... Figure 4As shown, this embodiment of the invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: acquiring M spectral features corresponding to M pixels in a spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; determining M spatial features corresponding to the M pixels based on the M spectral features, wherein any one of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located within the adjacent region of the pixel corresponding to any one of the spatial features; fusing the spectral features and spatial features of the M pixels to obtain M fused features corresponding to the M pixels; and classifying the pixels in the spectral image based on the M fused features to obtain the pixel classification result of the pixels in the spectral image.

[0103] Optionally, the processor can also execute a program to perform the following steps: fusing the spectral and spatial features corresponding to each of the M pixels to obtain M fused features corresponding to each of the M pixels, including: concatenating the spectral and spatial features of each of the M pixels into vectors to construct M concatenated feature vectors corresponding to each of the M pixels; constructing an initial feature matrix based on the M concatenated feature vectors; and processing the initial feature matrix using principal component analysis to obtain a feature fusion matrix, wherein the feature fusion matrix includes the M fused features corresponding to each of the M pixels.

[0104] Optionally, the processor can also execute a program to perform the following steps: processing the initial feature matrix using principal component analysis to obtain a feature fusion matrix, including: normalizing the initial feature matrix to obtain a normalized feature matrix; determining the covariance of the normalized feature matrix to obtain a covariance matrix; determining the eigenvalues ​​and eigenvectors of the covariance matrix, wherein the eigenvalues ​​and eigenvectors of the covariance matrix correspond one-to-one; selecting p principal component eigenvalues ​​from the eigenvalues ​​of the covariance matrix based on the eigenvalues ​​of the covariance matrix and a preset contribution rate threshold; and constructing a feature fusion matrix based on the p eigenvectors corresponding one-to-one with the p principal component eigenvalues ​​and the normalized feature matrix.

[0105] Optionally, the processor can also execute a program to implement the following steps: constructing a feature fusion matrix based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix, including: transposing the p eigenvectors corresponding to the p principal component eigenvalues ​​to obtain p transposed feature vectors; multiplying the p transposed feature vectors by the normalized feature matrix to obtain p fused feature column vectors; and constructing a feature fusion matrix based on the p fused feature column vectors, wherein the feature fusion matrix includes M rows and p columns, where the M rows represent the M fused features corresponding to the M pixels, and the p columns represent the feature dimension of each of the M fused features being p-dimensional.

[0106] Optionally, the processor can also execute a program to perform the following steps: classifying pixels in the spectral image according to M fusion features to obtain pixel classification results for the pixels in the spectral image, including: constructing a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationships between the M nodes in the node graph; assigning weight values ​​to the edges in the node graph according to the M fusion features; determining the graph adjacency matrix of the spectral image according to the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationships between the M pixels in the spectral image; and inputting the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain pixel classification results for the pixels in the spectral image.

[0107] Optionally, the processor can also execute a program to perform the following steps: assigning weight values ​​to the edges in the node graph according to the M fusion features, including: determining the length of each edge in the node graph, wherein the length of an edge in the node graph is the Euclidean distance between the two nodes connected by the edge; assigning a weight value to the first edge in the node graph according to the M fusion features when the length of the first edge included by the edge in the node graph is less than or equal to the maximum value of the adjacent region; and setting the weight value of the second edge in the node graph to zero when the length of the second edge included by the edge in the node graph is greater than the maximum value of the adjacent region.

[0108] Optionally, the processor may also execute a program to perform the following steps: assigning weight values ​​to the first edge in the node graph based on M fusion features, including: determining the first fusion feature and the second fusion feature corresponding to the nodes connected by the first edge from the M fusion features; and determining the weight value of the first edge based on the norm of the first fusion feature, the norm of the second fusion feature, and the inner product between the first fusion feature and the second fusion feature.

[0109] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0110] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: acquiring M spectral features corresponding to M pixels in a spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; determining M spatial features corresponding to the M pixels based on the M spectral features corresponding to the M pixels, wherein any one of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located within the adjacent region of the pixel corresponding to any spatial feature; fusing the spectral features and spatial features of each of the M pixels to obtain M fused features corresponding to the M pixels; and classifying the pixels in the spectral image based on the M fused features to obtain the pixel classification result of the pixels in the spectral image.

[0111] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: fusing the spectral and spatial features corresponding to each of M pixels to obtain M fused features corresponding to each of the M pixels, including: concatenating the spectral and spatial features of each of the M pixels into vectors to construct M concatenated feature vectors corresponding to each of the M pixels; constructing an initial feature matrix based on the M concatenated feature vectors; and processing the initial feature matrix using principal component analysis to obtain a feature fusion matrix, wherein the feature fusion matrix includes the M fused features corresponding to each of the M pixels.

[0112] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: processing an initial feature matrix using principal component analysis to obtain a feature fusion matrix, including: normalizing the initial feature matrix to obtain a normalized feature matrix; determining the covariance of the normalized feature matrix to obtain a covariance matrix; determining the eigenvalues ​​and eigenvectors of the covariance matrix, wherein the eigenvalues ​​and eigenvectors of the covariance matrix correspond one-to-one; selecting p principal component eigenvalues ​​from the eigenvalues ​​of the covariance matrix based on the eigenvalues ​​of the covariance matrix and a preset contribution rate threshold; and constructing a feature fusion matrix based on the p eigenvectors corresponding one-to-one with the p principal component eigenvalues ​​and the normalized feature matrix.

[0113] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: constructing a feature fusion matrix based on p eigenvectors corresponding to p principal component eigenvalues ​​and a normalized feature matrix, including: transposing the p eigenvectors corresponding to each of the p principal component eigenvalues ​​to obtain p transposed feature vectors; multiplying the p transposed feature vectors by the normalized feature matrix to obtain p fused feature column vectors; and constructing a feature fusion matrix based on the p fused feature column vectors, wherein the feature fusion matrix includes M rows and p columns, the M rows representing the M fused features corresponding to M pixels, and the p columns representing the feature dimension of each of the M fused features being p-dimensional.

[0114] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: classifying pixels in a spectral image based on M fusion features to obtain pixel classification results for the pixels in the spectral image, including: constructing a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationships between the M nodes in the node graph; assigning weight values ​​to the edges in the node graph according to the M fusion features; determining the graph adjacency matrix of the spectral image based on the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationships between the M pixels in the spectral image; and inputting the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain the pixel classification results for the pixels in the spectral image.

[0115] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: assigning weight values ​​to edges in a node graph according to M fusion features, including: determining the length of each edge in the node graph, wherein the length of an edge in the node graph is the Euclidean distance between the two nodes connected by the edge; assigning a weight value to the first edge in the node graph according to the M fusion features when the length of the first edge included by the edge in the node graph is less than or equal to the maximum value of the adjacent region's area; and setting the weight value of the second edge in the node graph to zero when the length of the second edge included by the edge in the node graph is greater than the maximum value of the adjacent region's area.

[0116] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: assigning weight values ​​to a first edge in a node graph based on M fusion features, including: determining, from the M fusion features, the first fusion feature and the second fusion feature corresponding to the nodes connected by the first edge, respectively; and determining the weight value of the first edge based on the norm of the first fusion feature, the norm of the second fusion feature, and the inner product between the first fusion feature and the second fusion feature.

[0117] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0123] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0125] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A pixel classification method, characterized in that, include: Obtain M spectral features corresponding to M pixels in a spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; Based on the M spectral features corresponding to the M pixels, M spatial features corresponding to the M pixels are determined. Each of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent region of the pixel corresponding to the spatial feature. The spectral and spatial features of the M pixels are fused to obtain M fused features corresponding to the M pixels. The pixels in the spectral image are classified according to the M fusion features to obtain the pixel classification results of the pixels in the spectral image; The step of classifying pixels in the spectral image based on the M fusion features to obtain pixel classification results includes: constructing a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationships between the M nodes; assigning weight values ​​to the edges in the node graph according to the M fusion features; determining the graph adjacency matrix of the spectral image based on the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationships between the M pixels in the spectral image; and inputting the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain pixel classification results for the pixels in the spectral image.

2. The method according to claim 1, characterized in that, The step of fusing the spectral and spatial features corresponding to each of the M pixels to obtain M fused features corresponding to the M pixels includes: The spectral and spatial features of the M pixels are concatenated to construct M concatenated feature vectors that correspond one-to-one with the M pixels. Construct an initial feature matrix based on the M cascaded feature vectors; The initial feature matrix is ​​processed by principal component analysis to obtain a feature fusion matrix, wherein the feature fusion matrix includes M fusion features corresponding to the M pixels.

3. The method according to claim 2, characterized in that, The initial feature matrix is ​​processed using principal component analysis to obtain a feature fusion matrix, including: The initial feature matrix is ​​normalized to obtain a normalized feature matrix; Determine the covariance of the normalized feature matrix to obtain the covariance matrix; Determine the eigenvalues ​​and eigenvectors of the covariance matrix, wherein the eigenvalues ​​and eigenvectors of the covariance matrix correspond one-to-one; Based on the eigenvalues ​​of the covariance matrix and a preset contribution rate threshold, p principal component eigenvalues ​​are selected from the eigenvalues ​​of the covariance matrix. The feature fusion matrix is ​​constructed based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix.

4. The method according to claim 3, characterized in that, The step of constructing the feature fusion matrix based on the p eigenvectors corresponding to the p principal component eigenvalues ​​and the normalized feature matrix includes: Transpose the p eigenvectors corresponding to the p principal component eigenvalues ​​to obtain p transposed eigenvectors; Multiply the p transposed feature vectors by the normalized feature matrix to obtain p fused feature column vectors; The feature fusion matrix is ​​constructed based on the p fusion feature column vectors, wherein the feature fusion matrix includes M rows and p columns, the M rows represent the M fusion features corresponding to the M pixels, and the p columns indicate that the feature dimension of each of the M fusion features is p-dimensional.

5. The method according to claim 1, characterized in that, The step of assigning weight values ​​to the edges in the node graph based on the M fusion features includes: Determine the length of each edge in the node graph, wherein the length of an edge in the node graph is the Euclidean distance between the two nodes connected by the edge; If the length of the first edge included in the edge of the node graph is less than or equal to the maximum value of the region range of the adjacent region, a weight value is assigned to the first edge in the node graph according to the M fusion features; If the length of the second edge included in the node graph is greater than the maximum value of the adjacent region, the weight value of the second edge in the node graph is set to zero.

6. The method according to claim 5, characterized in that, Assigning weight values ​​to the first edge of the node graph based on the M fusion features includes: From the M fusion features, determine the first fusion feature and the second fusion feature corresponding to the nodes connected by the first edge, respectively; The weight value of the first edge is determined based on the norm of the first fusion feature, the norm of the second fusion feature, and the inner product between the first fusion feature and the second fusion feature.

7. A pixel classification device, characterized in that, include: The acquisition module is used to acquire M spectral features corresponding to M pixels in the spectral image, wherein the feature dimension of each of the M spectral features matches the number of bands used when capturing the spectral image; The determining module is used to determine M spatial features corresponding to the M pixels based on the M spectral features corresponding to the M pixels, wherein any one of the M spatial features includes the spectral features of the adjacent pixels of the corresponding pixel, and the adjacent pixels are located in the adjacent region of the pixel corresponding to the arbitrary spatial feature. The fusion module is used to fuse the spectral features and spatial features of the M pixels to obtain M fused features corresponding to the M pixels. The classification module is used to classify the pixels in the spectral image according to the M fusion features, and obtain the pixel classification result of the pixels in the spectral image; The classification module is further configured to construct a node graph corresponding to the spectral image, wherein the M nodes in the node graph correspond one-to-one with the M pixels in the spectral image, and the edges in the node graph represent the connection relationships between the M nodes; assign weight values ​​to the edges in the node graph according to the M fusion features; determine the graph adjacency matrix of the spectral image according to the respective weight values ​​of the edges in the node graph, wherein the graph adjacency matrix represents the adjacency relationships between the M pixels in the spectral image; and input the M fusion features and the graph adjacency matrix into a graph convolutional neural network model to obtain the pixel classification results of the pixels in the spectral image.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the pixel classification method according to any one of claims 1 to 6 when it runs.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the pixel classification method according to any one of claims 1 to 6.

Images