A method for spatial target material recognition under complex lighting conditions

Abstract

A method for identifying the material of a space target under complex illumination conditions comprises the following steps: (1) inputting a hyperspectral image of a space target under complex illumination conditions; (2) performing superpixel segmentation on the hyperspectral image input in step (1), and generating a first-layer and a second-layer global structure topology map by a method based on superpixel segmentation; (3) merging the two-layer global structure topology map generated in step (2) to form a multi-scale joint global structure topology map, generating a material recognition topology map of the hyperspectral image input in step (1), and forming a training set and a test set; (4) constructing a heterogeneous neural network material recognition model that integrates a graph convolutional neural network and a three-dimensional convolutional neural network; (5) inputting the test set in step (3) into the material recognition model in step (4) to obtain a material category recognition result. The method improves the material recognition effect of a space target under complex illumination conditions, and has a high accuracy and strong robustness.

Description

Technical Field

[0001] The invention relates to a method for identifying space target materials under complex illumination conditions, belongs to the field of optical remote sensing data processing and space target identification, and is suitable for research on space target material identification in hyperspectral images. Background Art

[0002] Space target detection is of great significance in protecting the space environment and developing and utilizing space resources safely and continuously. The large amount of information contained in hyperspectral detection in the wavelength dimension can provide a reliable basis for the identification of the target surface material, thereby meeting the needs of space security and being an effective means to accurately identify the behavior and intention of space targets. Therefore, conducting research on space target material recognition based on hyperspectral detection is an important topic in space target detection.

[0003] Existing research has been carried out from two aspects: the influencing factors of hyperspectral detection and the material identification method based on hyperspectral detection. It should be noted that when the target is relatively stationary, the existing hyperspectral detection means can obtain high-quality hyperspectral image data of space targets. Therefore, the existing research on space targets based on imaging hyperspectra is mainly aimed at three-axis stabilized controlled targets, while the research objects based on non-imaging hyperspectra include space targets in various motion states.

[0004] In terms of influencing factors, the researchers first established a hyperspectral imaging model for space targets under complex lighting conditions, and summarized the main influencing factors into lighting conditions, target posture, target material, observation angle, etc. Specifically, the lighting conditions of space targets are composed of sunlight and earth reflection light, of which the earth reflection light is affected by the target orbit altitude and the surface type of the subsatellite point, and its intensity and spectrum are uncertain. The target posture will change the incident direction of the two light rays on the target surface. Combined with the spectral bidirectional reflectance distribution function (sBRDF) of different materials, the reflection spectrum of the space target in all directions is determined, and then according to the observation angle, the reflection spectrum of the target on the hyperspectral image is finally obtained. In this process, multiple influencing factors are coupled with each other, which will have a very complex impact on the target spectral characteristics. In addition to the theoretical model, the researchers also measured the sBRDF of common materials of space targets, and measured a large amount of hyperspectral data of space targets in geosynchronous orbit, and found that the reflection spectral characteristics of space targets are indeed easy to change, and different lighting conditions and observation angles have a significant impact on the spectral characteristics.

[0005] At the same time, in terms of identification methods, researchers have proposed a variety of material identification methods based on hyperspectral, including statistical learning methods such as partial least squares, Tucker decomposition, constrained non-negative matrix decomposition, and deep learning methods such as extreme learning machine, convolutional neural network, back propagation neural network, etc. These methods use the spectral characteristics of different materials as the main basis for material identification, and have achieved good results on the hyperspectral data of space targets obtained under ideal conditions such as computer simulation and laboratory measurement. However, a large number of theoretical models and measured data of space targets have proved that the spectral characteristics of target materials under complex lighting conditions have strong uncertainty and inconsistency. The existing space target material identification methods do not fully consider the impact of complex lighting conditions on spectral characteristics. When the lighting conditions change greatly, the existing methods may fail.

[0006] In order to solve this problem, we observed the hyperspectral images taken for controlled spatial targets under different lighting conditions and found that although the spectral characteristics of the same material are inconsistent, the overall structure of the target in the image space dimension is always highly recognizable, and the structural composition of the target is closely related to the material distribution. The structural characteristics of the target can be introduced as the basis for material recognition. The currently rapidly developing graph representation and graph learning methods are very suitable for extracting spatial target structural information from hyperspectral images, and are a powerful way to solve the problem of material recognition under complex lighting conditions. Therefore, a spatial target material recognition method based on graph convolutional neural network is proposed, which achieves high accuracy and strong robustness in spatial target material recognition under complex lighting conditions. Summary of the invention

[0007] The purpose of the present invention is to realize accurate material recognition of space targets under complex lighting conditions, and provides a method for material recognition of space targets under complex lighting conditions.

[0008] The technical solution of the present invention is: according to the strong correlation between materials and functional components on the space target, a multi-scale joint global structure topology map generation method based on superpixel segmentation is designed in the hyperspectral image, and the topology map contains the connection relationship, subordination relationship and average spectrum in the primary and secondary structures and large and small components of the space target. Based on the multi-scale joint global structure topology map, a material recognition topology map is generated for each pixel, characterizing the global spatial features such as the position and connection relationship of the pixel in the global structure of the space target and its spectral features, and extracting the neighborhood spectral cube blocks that can characterize the local spatial features and spectral features of the pixel, together forming a material recognition data set. A material recognition model that integrates a graph convolutional neural network and a three-dimensional convolutional neural network is constructed. The global spatial features and spectral features of the pixel are learned by the graph convolution part, and the local spatial features and spectral features of the pixel are learned by the three-dimensional convolution part. After training with the data set, the topology map and neighborhood spectral cube blocks of the pixel are input into the material recognition model, and the material recognition result of the pixel can be output.

[0009] The present invention is a method for identifying space target material under complex lighting conditions, and the steps are as follows:

[0010] (1) Input hyperspectral images of space targets under complex lighting conditions;

[0011] (2) performing superpixel segmentation on the hyperspectral image input in step (1), and generating first-layer and second-layer global structure topology maps by a superpixel segmentation method;

[0012] (3) merging the two layers of global structure topology maps generated in step (2) to form a multi-scale joint global structure topology map, generating a material recognition topology map of the hyperspectral image input in step (1), and forming a training set and a test set;

[0013] (4) Construct a heterogeneous neural network material recognition model that integrates graph convolutional neural network and three-dimensional convolutional neural network;

[0014] (5) Input the test set in step (3) into the material recognition model in step (4) to obtain the material category recognition result.

[0015] Wherein, step (1) inputs a hyperspectral image of a space target under complex lighting conditions: inputs a hyperspectral image taken of a controlled space target under complex lighting conditions to be processed Where M×N is the pixel size of the image, C is the number of spectral bands, and the complex lighting conditions are mainly composed of sunlight and earth reflection.

[0016] Among them, step (2) performs superpixel segmentation on the hyperspectral image input in step (1), and generates the first and second layer global structure topology maps based on the superpixel segmentation method: the first step is to set an initial number of cluster centers K0, and evenly distribute the cluster centers on the hyperspectral image I. The step length of adjacent cluster centers is approximately

[0017] The second step is to calculate the l of each point on the image in turn. i With each cluster center l j The distance D ij , the distance measurement formula is

[0018]

[0019]

[0020]

[0021] Among them, d l is the spectral distance between two points, l 0 ,l 1 ,…l C-1 is the gray value of each spectral band, C is the number of spectral bands, d s is the spatial distance between two points, x i ,y i It's point l i The position coordinates in the image, ε l ,ε s is a constant value used to adjust the weights of spectral distance and spatial distance, combined with d l and d s Get the distance D ij ;

[0022] The third step is to use the minimum distance criterion as the segmentation criterion for point l i For classification, if all cluster centers have l k With l i The distance is the shortest, that is, D ik ≤D ij , then the point is divided into category k;

[0023] Step 4: Calculate the average position coordinates of all points in the same category The location that serves as the new cluster center for that category;

[0024] Repeat steps 2 to 4 until the set number of iterations is reached, and simple linear iterative clustering is completed. The set of all similar points is regarded as a superpixel, and the image is divided into q0 superpixels. Each superpixel S 0,i By p 0,i Pixels composition,

[0025] Step 5: According to S 0,i Construct a global structure topology map and transform each superpixel S 0,i As a node, connect two adjacent superpixels S in the image 0,m and S 0,n Nodes, get the node adjacency matrix The calculation method is

[0026] a m,n =1,a m,n ∈A0

[0027] a n,m =1,a n,m ∈A0

[0028] Then S 0,i The average spectrum As the characteristics of the node, it constitutes the node feature matrix X0, which is calculated as follows:

[0029]

[0030]

[0031] The first-level global structure topology graph G0 = {A0, X0} is constructed to characterize the spectral information of large components of the space target and the connection relationship between them;

[0032] Repeat the first to fifth steps to perform superpixel segmentation and global structure topology map construction on the hyperspectral image I again, where the number of initial cluster centers K1 is set, K1>K0, and the remaining steps are the same as above to obtain q1 superpixels S 1,j , 0≤j<q1, and according to S 1,j The second-layer global structure topology graph G1 = {A1, X1} is constructed, and the topology graph G1 represents the spectral information of small components and the connection relationship between them.

[0033] Among them, step (3) merges the two layers of global structure topology maps generated in step (2) to form a multi-scale joint global structure topology map, generates a material recognition topology map of the hyperspectral image input in step (1), and constitutes a training set and a test set: the first step is to merge the two layers of global structure topology maps G0 = {A0, X0} and G1 = {A1, X1} in step (2). The merging method is to merge the superpixels S that have overlapping relationships between the two groups of superpixels segmented in step (2) 0,m With S 1,n The corresponding nodes are connected, specifically, if the superpixel S 1,n The cluster center Pixels at Belongs to Super Pixel S0,m ,Right now It is considered that the superpixel S 0,m With S 1,n Overlap and connect the corresponding nodes in G1 and G0. The calculation method is

[0034]

[0035]

[0036]

[0037]

[0038] Generate a multi-scale joint global structural topology map G2 = {A2, X2}, which contains the spectral information of large and small components, and reflects the connection relationship between large components, the connection relationship between small components, and the subordinate relationship between large components and small components on the space target;

[0039] In the second step, based on the multi-scale joint global structure topology G2 = {A2, X2}, in the hyperspectral image I input in step (1), a single pixel l is independently converted into a node S l , if S l and any superpixel S i The line connecting the cluster centers of (0≤i<q0+q1) belongs to S i If the pixel ratio exceeds the threshold, then connect S l and S i The corresponding node is calculated as

[0040]

[0041]

[0042]

[0043]

[0044] Generate material recognition topology map G l ={A l ,X l}, contains the spectral information of the pixel, and reflects its global spatial characteristics such as its connection relationship with components of different scales in the global structure of the space target and its position on the space target;

[0045] The third step is to extract the neighborhood spectral cube Ne of pixel l. l , d1×d2 is the size of the region centered on l, C is the number of spectral bands, Ne is lcontains the local spatial information and spectral information of pixel l, and the {G l ,Ne l As a sample, all pixel samples are aggregated to form a material recognition dataset Then Ds is divided into training set and test set. The division method is to randomly divide the samples of each material into training set and test set in a ratio of 1:4.

[0046] Among them, step (4) constructs a heterogeneous neural network material recognition model that integrates graph convolutional neural network and three-dimensional convolutional neural network: the first step is to input the topological map G of the pixels in the training set of step (3) l ={A l ,X l} and the neighborhood spectral cube Ne l ;

[0047] The second step is to transform the node feature matrix Input one-dimensional convolution layer f C1 (·) and a one-dimensional max pooling layer f MP1 (·), to extract spectral features and reduce data dimension, f C1 (·) Using a one-dimensional convolution kernel and step length f MP-1 (·) The length is The window and step size The calculation formula is

[0048]

[0049] in l m ∈X l , σ(·) is the activation function ReLU, in

[0050]

[0051] Output node feature matrix

[0052] The third step is to and the node adjacency matrix A l Input the graph convolution layer f together G (·), learn G l The global space characteristics and The spectral features in are used to extract the fusion features of the two. The calculation formula is:

[0053] d mm =∑ n a mn , d mm∈D,a mn ∈A l

[0054]

[0055]

[0056] where f G The number of channels of (·) is Ch, D is the node degree matrix, is a symmetric normalized adjacency matrix, calculated To propagate node features according to node connection relationships and normalized weights, and For weighted fusion, the trainable weight matrix is and Plus the trainable bias matrix Finally, the new node feature matrix is ​​output

[0057] The fourth step is to transform the neighborhood spectral cube Ne l Input 3D convolutional layer f C3 (·) and the 3D max pooling layer f MP3 (·),make Learn Ne l The local spatial features and spectral features in the image are used to extract the fusion features of the two. C3 (·) Using N C3 Convolution kernel and step length f MP3 (·) Using size The window and step size The calculation formula is

[0058] Ne CNN =f MP3 (σ(f C3 (Ne l )))

[0059] get in

[0060]

[0061]

[0062]

[0063] Output convolution feature map Ne CNN ;

[0064] Step 5: Use the additive fusion strategy to GCN and Ne CNNTo fuse, specifically, GCN Perform global average pooling and Ne CNN Input flattened layer f F (·) and the fully connected layer f D (·), and then add the outputs of the two point by point to get The calculation formula is

[0065] Y A =f GAP (X GCN )+σ[f F (Ne CNN )W D ]

[0066] in It is a trainable weight matrix used to adjust the contribution ratio of graph convolution features and convolution features to material recognition;

[0067] Step 6: Y A The input is the fully connected layer f S (·), the number of nodes is Cl, the total number of spatial target material categories is Cl-1, nodes 0, 1 to Cl-1 correspond to the background, the first material to the Cl-1th material respectively, the activation function is Softmax(·), and the calculation formula is

[0068] Y Class =f S (Y A )=Softmax(Y A W S )

[0069] in is a trainable weight matrix that outputs the material category confidence vector Among them exists but

[0070]

[0071] If k=0, the material recognition result is background. If 1≤k<Cl is output, the material recognition result is the kth material. The material recognition result is output to complete the construction of the material recognition model.

[0072] In the seventh step, the material recognition model is trained using the training set in step (3). The training method is the gradient descent method. The loss function Loss is composed of the cross entropy loss function and the regularization loss function. The calculation formula is:

[0073]

[0074]

[0075] where g n (0≤g n <Cl) is the true material of the nth sample in the training set, is the confidence vector obtained after the nth sample is input into the material recognition model, w j ∈W, W is the set of trainable weight parameters in the material recognition model, and Num is the total number of samples in the training set.

[0076] In step (5), the test set in step (3) is input into the material recognition model in step (4) to obtain the material category recognition result: the topological map G of the pixels to be recognized in the test set in step (4) is sequentially input into the material recognition model in step (4) l ={A l ,X l} and the spectral cube Ne l Input the material recognition model and output k, that is, the material recognition result is the background or the kth material.

[0077] The advantages of the present invention compared with the prior art are: a multi-scale joint global structure topology map generation method based on superpixel segmentation is designed, the topology map includes the connection relationship, subordination and average spectrum in the primary and secondary structures and large and small components of the space target, based on the topology map, material recognition data consisting of the topology map and the neighborhood spectral cube is established for each pixel, which includes three features: global spatial features, local spatial features and spectral features, and a material recognition model fused with a graph convolutional neural network and a three-dimensional convolutional neural network is constructed. The global spatial features and spectral features of the pixels are learned by the graph convolution part, and the local spatial features and spectral features of the pixels are learned by the three-dimensional convolution part. The three features are used together as the basis for material recognition, which can effectively improve the material recognition effect of the space target under complex lighting conditions, has high accuracy and strong robustness, and can improve the recognition effect of the material on small components. This method has the following advantages: (1) The multi-scale joint global structure topology map includes the connection relationship, subordination relationship and average spectrum of the large and small components of the spatial target. It is an efficient and accurate form of global spatial structure representation, which helps to improve the recognition effect of materials on small components; (2) Material recognition data containing multiple features is established for each pixel, which is composed of a topology map and a neighborhood spectral cube. The topology map represents the global spatial features such as the position and connection relationship of the pixel in the global structure of the spatial target and its spectral features, and the neighborhood spectral cube represents the neighborhood spectral cube of the local spatial features and spectral features of the pixel; (3) A material recognition model that integrates a graph convolutional neural network and a three-dimensional convolutional neural network is constructed. The graph convolution part learns the global spatial features and spectral features of the pixel, and the three-dimensional convolution part learns the local spatial features and spectral features of the pixel. Multiple features are used together as the basis for material recognition, which improves the accuracy and robustness of recognition. DETAILED DESCRIPTION

[0078] In order to better illustrate the space target material recognition method of the present invention, the material recognition is performed using the space target spectral image measured in the laboratory. The present invention is a space target material recognition method under complex lighting conditions, and the specific implementation steps are as follows:

[0079] (1) Input hyperspectral images of space targets under complex lighting conditions: Input the visible light band staring hyperspectral images taken of controlled space targets under complex lighting conditions to be processed. Where M×N is the pixel size of the image, M=100, N=220, C is the number of spectral bands, C=90, the band range of the hyperspectral image is 440nm-780nm, the spectral resolution is 10nm, and the materials on the space target include antennas, sailboards, gold coatings, and heat dissipation surfaces.

[0080] (2) Perform superpixel segmentation on the hyperspectral image input in step (1). Generate the first and second layer global structure topology maps based on the superpixel segmentation method: In the first step, set the initial number of cluster centers K0, K0 = 27, and evenly distribute the cluster centers on the hyperspectral image I. The step length of adjacent cluster centers is approximately

[0081] The second step is to calculate the l of each point on the image in turn. i With each cluster center l j The distance D ij , the distance measurement formula is

[0082]

[0083]

[0084]

[0085] Among them, d l is the spectral distance between two points, l 0 ,l 1 ,…l C-1 is the gray value of each spectral band, C is the number of spectral bands, d s is the spatial distance between two points, x i ,y i It's point l i The position coordinates in the image, ε l ,ε s is a constant value used to adjust the weights of spectral distance and spatial distance, combined with d l and d s Get the distance D ij ;

[0086] The third step is to use the minimum distance criterion as the segmentation criterion for point l i For classification, if all cluster centers have l k With l i The distance is the shortest, that is, D ik ≤D ij , then the point is divided into category k;

[0087] Step 4: Calculate the average position coordinates of all points in the same category The location that serves as the new cluster center for that category;

[0088] Repeat steps 2 to 4 until the set number of iterations reaches 10, completing simple linear iterative clustering. The set of all similar points is regarded as a superpixel, and the image is divided into q0 superpixels. Each superpixel S 0,i By p 0,i Pixels composition,

[0089] Step 5: According to S 0,i Construct a global structure topology map and transform each superpixel S 0,i As a node, if two superpixels S 0,m and S 0,n Cluster Center and If the percentage of pixels on the line belonging to these two superpixels exceeds 50%, the nodes corresponding to the two superpixels are connected to obtain the node adjacency matrix The calculation method is

[0090] a m,n =1,a m,n ∈A0

[0091] a n,m =1, a n,m ∈A0

[0092] Then S 0,i The average spectrum As the characteristics of the node, it constitutes the node feature matrix X0, which is calculated as follows:

[0093]

[0094]

[0095] The first-level global structure topology graph G0 = {A0, X0} is constructed to characterize the spectral information of large components of the space target and the connection relationship between them;

[0096] Repeat the first to fifth steps to perform superpixel segmentation and global structure topology map construction on the hyperspectral image I again, where the number of initial cluster centers K1 is set, K1 = 55, K1 > K0, and the other steps are the same as above, obtaining q1 superpixels S 1,j , 0≤j<q1, and according to S 1,j The second-layer global structure topology graph G1 = {A1, X1} is constructed, and the topology graph G1 represents the spectral information of small components and the connection relationship between them.

[0097] (3) The two layers of global structure topology maps generated in step (2) are merged to form a multi-scale joint global structure topology map, and a material recognition topology map of the hyperspectral image input in step (1) is generated to form a training set and a test set: The first step is to merge the two layers of global structure topology maps G0 = {A0, X0} and G1 = {A1, X1} in step (2). The merging method is to merge the superpixels S that have overlapping relationships between the two groups of superpixels segmented in step (2) 0,m With S 1,nThe corresponding nodes are connected, specifically, if the superpixel S 1,n The cluster center Pixels at Belongs to Super Pixel S 0,m ,Right now It is considered that the superpixel S 0,m With S 1,n Overlap and connect the corresponding nodes in G1 and G0. The calculation method is

[0098]

[0099]

[0100]

[0101]

[0102] Generate a multi-scale joint global structural topology map G2 = {A2, X2}, which contains the spectral information of large and small components, and reflects the connection relationship between large components, the connection relationship between small components, and the subordinate relationship between large components and small components on the space target;

[0103] In the second step, based on the multi-scale joint global structure topology G2 = {A2, X2}, in the hyperspectral image I input in step (1), a single pixel l is independently converted into a node S l , if S l and any superpixel S i The line connecting the cluster centers of (0≤i<q0+q1) belongs to S i If the pixel ratio exceeds the threshold, then connect S l and S i The corresponding node is calculated as

[0104]

[0105]

[0106]

[0107]

[0108] Generate material recognition topology map G l ={A l ,X l}, contains the spectral information of the pixel, and reflects its global spatial characteristics such as its connection relationship with components of different scales in the global structure of the space target and its position on the space target;

[0109] The third step is to extract the neighborhood spectral cube Ne of pixel l.l , d1×d2 is the size of the region centered on l, C is the number of spectral bands, d1=d2=5, C=90, Ne l contains the local spatial information and spectral information of pixel l, and the {G l ,Ne l As a sample, all pixel samples are aggregated to form a material recognition dataset Then Ds is divided into training set and test set. The division method is to randomly divide the samples of each material into training set and test set in a ratio of 1:4.

[0110] (4) Construct a heterogeneous neural network material recognition model that integrates graph convolutional neural network and 3D convolutional neural network: In the first step, input the topological map G of the pixels in the training set in step (3) l ={A l ,X l} and the neighborhood spectral cube Ne l ;

[0111] The second step is to transform the node feature matrix Input one-dimensional convolution layer f C1 (·) and a one-dimensional max pooling layer f MP1 (·), to extract spectral features and reduce data dimension, f C1 (·) Using a one-dimensional convolution kernel and step length f MP-1 (·) The length is The window and step size The calculation formula is

[0112]

[0113] in l m ∈X l , σ(·) is the activation function ReLU, in

[0114]

[0115] Output node feature matrix

[0116] The third step is to and the node adjacency matrix A l Input the graph convolution layer f together G (·), learn G l The global space characteristics and The spectral features in are used to extract the fusion features of the two. The calculation formula is:

[0117] d mm =∑ n a mn , d mm ∈D,a mn ∈A l

[0118]

[0119]

[0120] where f G The number of channels of (·) is Ch=32, D is the node degree matrix, is a symmetric normalized adjacency matrix, calculated To propagate node features according to node connection relationships and normalized weights, and For weighted fusion, the trainable weight matrix is and Plus the trainable bias matrix Output new node feature matrix

[0121] The fourth step is to transform the neighborhood spectral cube Ne l Input 3D convolutional layer f C3 (·) and the 3D max pooling layer f MP3 (·),make Learn Ne l The local spatial features and spectral features in the image are used to extract the fusion features of the two. C3 (·) Using N C3 Convolution kernel and step length Where N C3 =16, f MP3 (·) Using size The window and step size in The calculation formula is

[0122] Ne CNN =f MP3 (σ(f C3 (Ne l )))

[0123] get in

[0124]

[0125]

[0126]

[0127] Output convolution feature map Ne CNN ;

[0128] Step 5: Use the additive fusion strategy to GCN and Ne CNN To fuse, specifically, GCN Perform global average pooling and Ne CNN Input flattened layer f F (·) and the fully connected layer f D (·), and then add the outputs of the two point by point to get The calculation formula is

[0129] Y A =f GAP (X GCN )+σ[f F (Ne CNN )W D ]

[0130] in It is a trainable weight matrix used to adjust the contribution ratio of graph convolution features and convolution features to material recognition;

[0131] Step 6: Y A The input is the fully connected layer f S (·), the number of nodes is Cl = 5, the total number of spatial target material categories is Cl-1 = 4, nodes 0, 1 to 4 correspond to background, antenna, sailboard, golden coating, and heat dissipation surface respectively, the activation function is Softmax(·), and the calculation formula is

[0132] Y Class =f S (Y A )=Softmax(Y A W S )

[0133] in is a trainable weight matrix that outputs the material category confidence vector Among them exists but

[0134]

[0135] If k=0, the material recognition result is background. If 1≤k<Cl is output, the material recognition result is the kth material. The material recognition result is output to complete the construction of the material recognition model.

[0136] In the seventh step, the material recognition model is trained using the training set in step (3). The training method is the gradient descent method. The loss function Loss is composed of the cross entropy loss function and the regularization loss function. The calculation formula is:

[0137]

[0138]

[0139] where g n (0≤g n <Cl) is the true material of the nth sample in the training set, is the confidence vector obtained after the nth sample is input into the material recognition model, w j ∈W, W is the set of trainable weight parameters in the material recognition model, and Num is the total number of samples in the training set.

[0140] (5) Input the test set in step (3) into the material recognition model in step (4) to obtain the material category recognition result: sequentially input the topological map G of the pixels to be recognized in the test set in step (4) into the material recognition model in step (4) l ={A l ,X l} and the spectral cube Ne l Input the material recognition model and output k, that is, the material recognition result is the background or the kth material, k = 0, 1, 2, 3, 4 correspond to the background, antenna, sailboard, gold coating, and heat dissipation surface respectively.

[0141] The embodiments of the present invention are described in detail above, but the contents are only preferred embodiments of the present invention and cannot be considered to limit the scope of implementation of the present invention. The implementation methods of each step can be changed, and all equal changes and improvements made within the scope of the present invention should still fall within the scope of this patent.

Claims

1. A method for identifying spatial target material under complex lighting conditions, characterized by: It contains the following steps: (1) Input hyperspectral images of space targets under complex lighting conditions; (2) The first step is to set an initial number of cluster centers , in pixel size Hyperspectral images The cluster centers are evenly distributed on the surface, and the step length between adjacent cluster centers is approximately ; The second step is to calculate each point on the image in turn With each cluster center Distance , the distance measurement formula is ； in, is the spectral distance between two points, is the gray value of each band of the spectrum, is the number of spectral bands, is the spatial distance between two points, Yes The position coordinates in the image, is a constant value used to adjust the weights of spectral distance and spatial distance, combined with and Get distance ; The third step is to use the minimum distance criterion as the segmentation criterion for the point For classification, if all cluster centers and The distance is the shortest, that is ； Then the point is divided into categories ; Step 4: Calculate the average position coordinates of all points in the same category , as the location of the new cluster center of this category; repeat steps 2 to 4 until the set number of iterations is reached, and the set of all similar points is regarded as a superpixel, then the image is segmented into superpixels, each superpixel Depend on Pixels Composition, that is ； in ; Step 5: According to Construct a global structural topology map and map each superpixel As a node, connect two adjacent superpixels in the image and Nodes, get the node adjacency matrix , calculated as ， ； ， ； Then The average spectrum As the characteristics of the node, it constitutes the node feature matrix , calculated as ； ； The first-layer global structure topology diagram , characterize the spectral information of large components of space targets and the connection relationship between them; repeat the first step to the fifth step to the hyperspectral image Perform superpixel segmentation and global structure topology map construction again, setting the initial number of cluster centers , , the rest of the steps are the same as above, and we get Superpixels , , and according to The second-layer global structure topology diagram , topology diagram Characterize the spectral information of small components and the connections between them; (3) The two layers of global structure topology maps generated in step (2) are merged to form a multi-scale joint global structure topology map, and the material recognition topology map of the hyperspectral image input in step (1) is generated to form a training set and a test set; (4) Construct a heterogeneous neural network material recognition model that integrates graph convolutional neural network and 3D convolutional neural network; (5) Input the test set in step (3) into the material recognition model in step (4) to obtain the material category recognition result.

2. According to the method for identifying the material of a space target under complex lighting conditions as described in claim 1, wherein step (1) inputs a hyperspectral image of the space target under complex lighting conditions: inputs a hyperspectral image taken of the controlled space target under complex lighting conditions to be processed ,in is the pixel size of the image, is the number of spectral bands. The complex lighting conditions are mainly composed of sunlight and earth reflected light.

3. According to the method for identifying the material of a space target under complex lighting conditions as described in claim 1, wherein step (3) merges the two layers of global structure topology maps generated in step (2) to form a multi-scale joint global structure topology map, and generates a material recognition topology map of the hyperspectral image input in step (1) to form a training set and a test set: the first step is to merge the two layers of global structure topology maps generated in step (2) and Merge: The merging method is to merge the superpixels that overlap between the two groups of superpixels segmented in step (2) and The corresponding nodes are connected, specifically, if the superpixel The cluster center Pixels at Superpixel ,Right now , then the superpixel and Overlap and and The corresponding nodes in are connected, and the calculation method is ； ； ； ； Generate a multi-scale joint global structural topology map , which contains the spectral information of large and small components, and reflects the connection relationship between large components, the connection relationship between small components, and the subordinate relationship between large components and small components on the space target; The second step is to build a global structural topology map based on multi-scale joint , the hyperspectral image input in step (1) In the example above, a single pixel Independent as a node ,if and any superpixel ( ) belongs to If the pixel ratio exceeds the threshold, the connection and The corresponding node is calculated as ； ； ； ； Generate material recognition topology map , contains the spectral information of the pixel, and reflects its global spatial characteristics such as its connection relationship with components of different scales in the global structure of the space target, its position on the space target, etc.; The third step is to extract pixels Neighborhood spectral cube , , So The size of the area centered, is the number of spectral bands, Contains pixels The local spatial information and spectral information of a single pixel As a sample, all pixel samples are aggregated to form a material recognition dataset , and then Divide into training set and test set. The division method is to randomly divide the samples of each material into training set and test set in a ratio of 1:

4.

4. According to the method for identifying spatial target materials under complex lighting conditions as described in claim 1, wherein step (4) constructs a heterogeneous neural network material recognition model that integrates a graph convolutional neural network and a three-dimensional convolutional neural network: the first step is to input the topological map of the pixels in the training set of step (3) and neighborhood spectral cube ; The second step is to transform the node feature matrix Input 1D convolutional layer and a one-dimensional max pooling layer , perform spectral feature extraction and data dimension reduction, Using one-dimensional convolution kernel and step length , The length is The window and step size , the calculation formula is ； in , , is the activation function ReLU, ,in ； , output node feature matrix ; The third step is to and the node adjacency matrix Input the graph convolution layer together ,study The global space characteristics and The spectral features in are used to extract the fusion features of the two. The calculation formula is: ； in The number of channels is , is the node degree matrix, is a symmetric normalized adjacency matrix, calculated To propagate node features according to node connection relationships and normalized weights, and For weighted fusion, the trainable weight matrix is and , plus a trainable bias matrix , output the new node feature matrix ; The fourth step is to block the neighborhood spectrum Input 3D convolutional layer and a 3D max pooling layer ,make ,study The local spatial features and spectral features in the image are combined to extract the fusion features of the two. use Convolution kernel ( ) and step length , , , The size is The window and step size , the calculation formula is ； get ,in ； Output convolution feature map ; The fifth step is to use the additive fusion strategy to and To integrate, specifically, Perform global average pooling and Input Flatten Layer and fully connected layers , and then add the outputs of the two point by point to get , the calculation formula is ； in It is a trainable weight matrix used to adjust the contribution ratio of graph convolution features and convolution features to material recognition; Step 6: Input to the fully connected layer , the number of nodes is , the total number of spatial target material categories is , node 0, node 1 to node Corresponding to the background, the first material to the materials, the activation function is , the calculation formula is ； in is a trainable weight matrix that outputs the material category confidence vector , , , where there is ( ),but ； if , the material recognition result is the background, if the output , the material recognition result is materials, output material recognition results, and complete the construction of material recognition model; Step 7: Use the training set from step (3) to train the material recognition model. The training method is gradient descent and the loss function is It consists of the cross entropy loss function and the regularization loss function, and the calculation formula is: ； ； in ( ) is the true material of the nth sample in the training set, , is the confidence vector obtained after the nth sample is input into the material recognition model. , is a collection of trainable weight parameters in the material recognition model. is the total number of samples in the training set.

5. According to the method for identifying the material of a space target under complex lighting conditions as described in claim 1, wherein step (5) inputs the test set in step (3) into the material recognition model in step (4) to obtain the material category recognition result: sequentially input the topological map of the pixels to be identified in the test set in step (4) and spectral cube Input material recognition model, output , that is, the material recognition result is the background or the Kind of material.

Images