Method and device for filling missing nighttime light data in time and space based on graph neural network

Abstract

The present invention discloses a method and device for filling missing nighttime light data in time and space based on a graph neural network, which relates to the technical field of remote sensing data processing, and the main purpose is to solve the problem of low accuracy of missing existing nighttime light data. It mainly includes extracting the time and space topology map of missing pixels in the nighttime light data to be processed; using a feature embedding network to embed node features and edge features in the time and space topology map to obtain high-dimensional representations of node features and high-dimensional representations of edge features; based on the high-dimensional representations of node features, high-dimensional representations of edge features and the adjacency matrix of the time and space topology map, using the graph attention layer in the topology processing network to perform prediction processing to obtain the time and space topology features of the missing pixels; using a feature regression network to process the time and space topology features to obtain the prediction results of the missing pixels, and using the prediction results as the pixel data of the missing pixels to perform time and space filling to obtain the time and space filling results of the nighttime light data to be processed.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and remote sensing data processing, and in particular to a method and device for filling in missing nighttime light data in a spatiotemporal manner based on a graph neural network. Background Art

[0002] As a commonly used remote sensing data source, night light data provides a unique perspective for studying urbanization and human activities, showing great application potential. However, due to factors such as cloud cover, there are often a large number of missing values ​​in the data, making it difficult to obtain relatively complete night light data. Therefore, how to fill in the missing night light data plays an important role in the analysis of night light data.

[0003] The existing methods for filling missing remote sensing data are mainly to fill missing values ​​through models such as convolutional neural networks, long short-term memory networks, or recurrent gating networks. However, when the amount of missing data is large, the above methods will cause gradient explosion and decreased generalization ability. This greatly reduces the accuracy of filling missing night light data. Summary of the invention

[0004] In view of this, the present invention provides a method and device for filling in the missing night light data in time and space based on graph neural network, the main purpose of which is to solve the problem of low accuracy of the existing night light data missing filling.

[0005] According to one aspect of the present invention, a method for spatiotemporally filling missing nighttime light data based on a graph neural network is provided, wherein spatiotemporally filling is performed based on a pre-trained spatiotemporal filling model, wherein the pre-trained spatiotemporal filling model includes a feature embedding network, a topology processing network, and a feature regression network, and the method includes:

[0006] Extract the spatiotemporal topological map of missing pixels in the night light data to be processed;

[0007] Using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph, to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features;

[0008] According to the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topological graph, a graph attention layer in the topological processing network is used to perform prediction processing to obtain the spatiotemporal topological features of the missing pixel points;

[0009] The feature regression network is used to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and the prediction results are used as the pixel data of the missing pixels for spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

[0010] Furthermore, the feature embedding network includes three layers of graph attention layers, and the graph attention layers in the topology processing network are used to perform prediction processing based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features, and the adjacency matrix of the spatiotemporal topology graph to obtain the spatiotemporal topological features of the missing pixels, including:

[0011] Using the first graph attention layer, according to the high-dimensional representation of the node features and the high-dimensional representation of the edge features, the attention scores between the nodes are calculated, and according to the adjacency matrix and the attention scores, the node information propagation path and propagation strength are determined;

[0012] Performing node feature propagation and node feature aggregation according to the propagation path and the propagation strength to update the node features in the high-dimensional representation of the node features, obtain a first spatiotemporal topological graph, and add a self-loop to the central node of the first spatiotemporal topological graph;

[0013] Using the second graph attention layer to perform a second update on the node features in the first spatiotemporal topological graph to obtain a second spatiotemporal topological graph, and using the third graph attention layer to perform a third update on the node features in the second spatiotemporal topological graph to obtain a third spatiotemporal topological graph;

[0014] Node features corresponding to the missing pixel points in the third spatiotemporal topological map are extracted, and the node features are used as spatiotemporal topological features of the missing pixel points.

[0015] Furthermore, the topology processing network further includes a first update linear layer, and the node features in the high-dimensional representation of the node features are updated once to obtain a first spatiotemporal topology graph, including:

[0016] Using the first update linear layer to update the edge features in the edge feature high-dimensional representation, and generating a first spatiotemporal topological graph based on the updated edge features and the updated node features;

[0017] The topology processing network further includes a second updated linear layer, and the second graph attention layer is used to perform a second update on the node features in the first spatiotemporal topology graph to obtain a second spatiotemporal topology graph, including:

[0018] The second updated linear layer is used to perform a second update on the edge features in the high-dimensional representation of the edge features, and a third spatiotemporal topological graph is generated based on the second updated edge features and the second updated node features.

[0019] Furthermore, the feature embedding network includes a node feature embedding network and an edge feature embedding network, and the node feature embedding and edge feature embedding of the spatiotemporal topology graph using the feature embedding network to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features include:

[0020] Using the first node linear layer and the second node linear layer in the node feature embedding network to perform feature dimension upgrade on each node attribute of the spatiotemporal topology graph in turn, and using the first standardization layer to perform standardization on the node features after dimension upgrade to obtain a high-dimensional representation of the node features;

[0021] Using the first edge linear layer and the second edge linear layer in the edge feature embedding network to sequentially perform feature dimension upgrade on each edge attribute of the spatiotemporal topological graph, and performing standardization processing on the dimension-enhanced edge features based on the second standardization layer to obtain a high-dimensional representation of the edge features;

[0022] Among them, the dimension of the high-dimensional representation of the edge feature is aligned with the dimension of the high-dimensional representation of the node feature; the activation functions of the first node linear layer, the second node linear layer and the second edge linear layer are Gaussian error linear units; the activation function of the first edge linear layer is a hyperbolic tangent function based on the origin symmetry.

[0023] Furthermore, the step of extracting the spatiotemporal topological map of missing pixels in the nighttime light data to be processed includes:

[0024] Determining missing pixels in the nighttime light data to be processed;

[0025] Taking the missing pixel point as the central node, extracting neighbor nodes that meet the preset number of nodes from the neighborhood of the space-time cube where the missing pixel point is located, and configuring the pixel data corresponding to the neighbor nodes as the node attributes of the neighbor nodes;

[0026] Complete edge construction according to the spatiotemporal topological relationship between the neighbor nodes and the central node, and between the neighbor nodes, and configure edge attributes of the edge according to the spatiotemporal deviation value between the nodes connected by the edge;

[0027] A spatiotemporal topological map of the missing pixels is generated based on the central node, the neighboring nodes and the edges.

[0028] Furthermore, the using the feature regression network to process the spatiotemporal topological features to obtain the prediction result of the missing pixel point includes:

[0029] Performing a feature dimensionality reduction on the spatiotemporal topological feature using a first regression linear layer of a feature regression network to obtain a spatiotemporal topological feature after dimensionality reduction, wherein the first regression linear layer is not configured with an activation function;

[0030] The second regression linear layer of the feature regression network is used to reduce the spatiotemporal topological features after the first dimensionality reduction to a single numerical output, and is processed by an activation function layer to obtain the prediction result of the missing pixel point, wherein the second regression linear layer includes a linear activation function and a nonlinear activation function.

[0031] Furthermore, before using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features, the method further includes:

[0032] Constructing a training sample set and an initial space-time filling model, wherein the initial space-time filling model includes a feature embedding network, a topology processing network and a feature regression network;

[0033] The feature embedding network includes a node feature embedding network and an edge feature embedding network, wherein the node feature embedding network includes a first node linear layer and a second node linear layer, and the edge feature embedding network includes a first edge linear layer and a second edge linear layer;

[0034] The topology processing network includes a first graph attention layer, a second graph attention layer, and a third graph attention layer, as well as a first update linear layer and a second update linear layer; the output of the first graph attention layer is used as the input of the first update linear layer, and the output of the second graph attention layer is used as the input of the second update linear layer;

[0035] The feature regression network includes a first regression linear layer and a second regression linear layer, wherein the first regression linear layer is not configured with an activation function and has Dropout introduced, and the second regression linear layer includes a linear activation function and a nonlinear activation function;

[0036] A loss function is constructed based on the predicted output of the initial space-time filling model trained on the training sample set and the logarithmic hyperbolic cosine function, and an adaptive moment estimation with a Nesterov momentum term is used as an optimizer to optimize the parameters of the initial space-time filling model to obtain a pre-trained space-time filling model.

[0037] According to another aspect of the present invention, a device for filling in missing nighttime light data based on a graph neural network is provided, wherein the device performs space-time filling based on a pre-trained space-time filling model, wherein the pre-trained space-time filling model includes a feature embedding network, a topology processing network, and a feature regression network, and the device includes:

[0038] An extraction module is used to extract the spatiotemporal topological map of missing pixels in the night light data to be processed;

[0039] A feature embedding module, used to embed node features and edge features in the spatiotemporal topology graph using the feature embedding network to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features;

[0040] A prediction processing module, used to perform prediction processing using the graph attention layer in the topology processing network based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topology graph, to obtain the spatiotemporal topological features of the missing pixel points;

[0041] The spatiotemporal filling module is used to use the feature regression network to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and use the prediction results as the pixel data of the missing pixels to perform spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

[0042] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned method of spatiotemporally filling in missing night light data based on graph neural network.

[0043] According to another aspect of the present invention, there is provided a terminal, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0044] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned method of filling in the missing night light data in time and space based on graph neural network.

[0045] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0046] The present invention provides a method and device for filling missing nighttime light data in time and space based on a graph neural network. The embodiment of the present invention extracts a spatiotemporal topological map of missing pixels in the nighttime light data to be processed; uses a feature embedding network to embed node features and edge features in the spatiotemporal topological map to obtain high-dimensional representations of node features and high-dimensional representations of edge features; uses a graph attention layer in a topological processing network to perform prediction processing based on the high-dimensional representations of node features, the high-dimensional representations of edge features and the adjacency matrix of the spatiotemporal topological map to obtain spatiotemporal topological features of the missing pixels; uses a feature regression network to process the spatiotemporal topological features to obtain prediction results of the missing pixels, and uses the prediction results as pixel data of the missing pixels to fill in time and space to obtain the spatiotemporal filling results of the nighttime light data to be processed, fully extracts the spatiotemporal correlation between the missing pixels and other pixels in the nighttime light data to be processed, and in the process of prediction based on the graph attention layer, can more accurately capture the complex spatiotemporal correlation between the missing pixels and other known pixels, thereby greatly improving the accuracy of feature prediction of the missing pixels.

[0047] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0049] Figure 1 A flow chart of a method for filling in missing nighttime light data in a spatiotemporal manner based on a graph neural network provided by an embodiment of the present invention is shown;

[0050] Figure 2 A flowchart of another method for filling missing nighttime light data based on a graph neural network in time and space provided by an embodiment of the present invention is shown;

[0051] Figure 3 A schematic diagram of the model structure of a time-space filling model provided by an embodiment of the present invention is shown;

[0052] Figure 4 A block diagram of a device for filling in missing nighttime light data based on a graph neural network in a spatiotemporal manner provided by an embodiment of the present invention is shown;

[0053] Figure 5A schematic diagram of the structure of a terminal provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0055] In view of the low accuracy of the existing nighttime light data missing filling, the embodiment of the present invention provides a method for filling the nighttime light data missing in time and space based on a graph neural network. The method performs time and space filling based on a pre-trained time and space filling model, wherein the pre-trained time and space filling model includes a feature embedding network, a topology processing network and a feature regression network. Figure 1 As shown, the method includes:

[0056] 101. Extract the spatiotemporal topological map of missing pixels in the night light data to be processed.

[0057] In the embodiment of the present invention, the night light data to be processed is remote sensing data collected in a night environment that needs to be filled in time and space. Missing pixels are pixels whose pixel data is missing in the night light data to be processed, including partial pixel data missing and complete pixel data missing, which is not specifically limited in the embodiment of the present invention. In order to ensure the accuracy of time and space filling when there are a large number of missing values ​​in the night light data, data missing filling is performed based on the time and space topological relationship in the night light data to be processed. Therefore, after obtaining the night light data to be processed, corresponding time and space topological maps are constructed for each missing pixel in the night light data to be processed, so as to extract the complex correlation between the current missing pixel and its neighborhood based on the time and space topological map, thereby realizing accurate prediction of the remote sensing data of the missing pixel. Among them, the time and space topological map is a graph structure constructed with missing pixels and their neighborhood pixels as nodes and the time and space relationship between the missing pixel and its neighborhood pixels as edges.

[0058] In one embodiment of the present invention, for further explanation and limitation, the step of extracting the spatiotemporal topological map of missing pixels in the nighttime light data to be processed includes:

[0059] Determining missing pixels in the nighttime light data to be processed;

[0060] Taking the missing pixel point as the central node, extracting neighbor nodes that meet the preset number of nodes from the neighborhood of the space-time cube where the missing pixel point is located, and configuring the pixel data corresponding to the neighbor nodes as the node attributes of the neighbor nodes;

[0061] Complete edge construction according to the spatiotemporal topological relationship between the neighbor nodes and the central node, and between the neighbor nodes, and configure edge attributes of the edge according to the spatiotemporal deviation value between the nodes connected by the edge;

[0062] A spatiotemporal topological map of the missing pixels is generated based on the central node, the neighboring nodes and the edges.

[0063] In an embodiment of the present invention, the nighttime light data to be processed may contain a large number of missing pixels, and the pixel data of the missing pixels need to be predicted one by one. Therefore, it is necessary to first determine the pixels with missing data, i.e., the missing pixels, based on the pixel data. Then, a corresponding spatiotemporal topological map is constructed for each missing pixel. In the process of constructing the spatiotemporal topological map, the missing pixel is taken as the central node, and high-quality pixels are extracted from the neighborhood of the missing pixel as neighbor nodes until the number of neighbor nodes reaches the preset number of nodes. After completing the node selection, the nodes are connected based on the spatiotemporal topological relationship between the central node and the neighbor nodes to complete the edge construction. Among them, the two nodes connected by the edge include the central node and the neighbor node, and also include the neighbor node and the neighbor node. The central spatiotemporal topological relationship can be determined based on the distribution of the time dimension and the space dimension of the nighttime light data in the spatiotemporal cube.

[0064] It should be noted that after completing the node selection, the pixel data of the neighboring nodes are embedded in their node attributes. After completing the edge construction, the spatiotemporal deviation value between the two nodes connected by the edge is embedded in the edge attribute of the corresponding edge. By embedding the spatiotemporal deviation value between the two nodes connected by the edge in the edge attribute, the spatiotemporal correlation between the nodes can be more fully captured in the subsequent feature embedding and feature prediction process, and the node features can be predicted based on the richer spatiotemporal correlation, thereby improving the accuracy of feature prediction.

[0065] 102. Use the feature embedding network to embed node features and edge features in the spatiotemporal topology graph to obtain high-dimensional representations of node features and high-dimensional representations of edge features.

[0066] In an embodiment of the present invention, after obtaining the spatiotemporal topological map, the pre-trained spatiotemporal filling model is used to predict the nodes where the missing pixels are located. In order to capture the association information between different nodes in the spatiotemporal topological map, a feature embedding network is configured in the pre-trained spatiotemporal filling model. The feature embedding network is used to embed features of nodes and edges in the spatiotemporal topological map respectively. That is, each node in the spatiotemporal topological map is mapped to a high-dimensional vector space to obtain the corresponding high-dimensional space node features, and the node features of all nodes constitute the high-dimensional representation of the node features. Each edge in the graph is mapped to a high-dimensional vector space to obtain the corresponding high-dimensional space edge features, and the edge features of all edges constitute the high-dimensional representation of the edge features.

[0067] It should be noted that the attributes of nodes include the pixel data of the corresponding pixels, and the attributes of edges include the spatiotemporal deviation between the two connected pixels. By embedding the features of nodes and edges, local and global information of each node can be extracted, such as the number of neighbors of the node, the connection strength between nodes, etc., and the spatiotemporal correlation and weight between the nodes connected by the edge can also be extracted, thereby providing rich feature information for the subsequent prediction of missing pixels.

[0068] In one embodiment of the present invention, for further explanation and limitation, the step of using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features includes:

[0069] Using the first node linear layer and the second node linear layer in the node feature embedding network to perform feature dimension upgrade on each node attribute of the spatiotemporal topology graph in turn, and using the first standardization layer to perform standardization on the node features after dimension upgrade to obtain a high-dimensional representation of the node features;

[0070] The first edge linear layer and the second edge linear layer in the edge feature embedding network are used to sequentially perform dimension upgrade on the edge attributes of the spatiotemporal topological graph, and the dimension upgraded edge features are standardized based on the second standardization layer to obtain a high-dimensional representation of the edge features.

[0071] In an embodiment of the present invention, the feature embedding network includes a node feature embedding network and an edge feature embedding network. The node feature embedding network uses two serially connected linear layers (a first node linear layer and a second node linear layer) to increase the dimension of the node attributes, and the edge feature embedding network also uses two serially connected linear layers (a first edge linear layer and a second edge linear layer) to increase the dimension of the edge attributes. It should be noted that the dimensions of the node dimension increase and the edge dimension increase are the same, that is, the dimension of the high-dimensional representation of the edge feature is aligned with the dimension of the high-dimensional representation of the node feature. For the stability of the features, the feature embedding network is also configured with a normalization layer, that is, a first normalization layer; the edge feature embedding network is also configured with a normalization layer, that is, a second normalization layer. Among them, the activation functions of the first node linear layer, the second node linear layer and the second edge linear layer are Gaussian error linear units, which can improve the nonlinear fitting ability. The activation function of the first edge linear layer is a hyperbolic tangent function based on the origin symmetry. Since the edge attribute is the spatiotemporal deviation between the positions of the elements in the spatiotemporal cube represented by the two connected nodes, it is approximately standard normal distribution in three dimensions, and has a considerable part of negative input. Therefore, the first edge linear layer is the first linear layer to process edge attributes. Its activation function uses the hyperbolic tangent function based on the origin symmetry, which can minimize the possibility of gradient problems in the spatiotemporal filling model. Set the node attribute set to , the edge attribute set is , then after the feature embedding processing of the above feature embedding network, the node feature high-dimensional representation , which can be expressed as:

[0072] (1);

[0073] in, is the Gaussian error linear unit, (BatchNormalization) is the standardization layer, is the first node linear layer, is the second node linear layer, where , The functions of (2); among which, is the input variable, is the linear slope, is the intercept.

[0074] Edge feature set , which can be expressed as:

[0075] ;

[0076] in, is the hyperbolic tangent function symmetric about the origin, and The functions of are also the same as formula (2).

[0077] 103. Based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topological graph, a graph attention layer in the topological processing network is used to perform prediction processing to obtain the spatiotemporal topological features of the missing pixel points.

[0078] In an embodiment of the present invention, the pre-trained spatiotemporal filling model also includes a topological processing network, and the topological processing network includes a graph attention layer constructed using a graph attention network. Among them, the graph attention network is an important branch in the graph neural network, and this network can consider the different influences of different nodes on the central node. The graph attention layer calculates the attention coefficient between nodes based on the features in the high-dimensional representation of node features and the high-dimensional representation of edge features, and uses the attention coefficient as a weight to perform weighted summation on the neighbor features of each node to update the node features. Through multiple iterations of the graph attention layer, the transmission of node feature information between different nodes, and the aggregation of neighbor features, the feature prediction of the node corresponding to the missing pixel point is realized, thereby obtaining the spatiotemporal topological features of the missing pixel point. Preferably, the second version of the graph attention network is selected, which solves the static attention problem in the original graph attention network design and has a stronger ability to capture dynamic data, that is, different weights can be selected according to the different query nodes as needed. Since the graph attention network can simultaneously propagate and aggregate information on node features and edge attributes, and has a stronger ability to capture dynamic data, it can more accurately capture the complex spatiotemporal correlation between missing pixels and other known pixels, thereby greatly improving the accuracy of feature prediction of missing pixels.

[0079] In an embodiment of the present invention, for further explanation and limitation, as Figure 2 As shown, the step is based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topological graph, and uses the graph attention layer in the topological processing network to perform prediction processing to obtain the spatiotemporal topological features of the missing pixel points, including:

[0080] 201. Utilize the first graph attention layer to calculate the attention scores between the nodes based on the high-dimensional representation of the node features and the high-dimensional representation of the edge features, and determine the node information propagation path and propagation strength based on the adjacency matrix and the attention scores.

[0081] 202. Perform node feature propagation and node feature aggregation according to the propagation path and the propagation strength to update the node features in the high-dimensional representation of the node features, obtain a first spatiotemporal topological graph, and add a self-loop to the central node of the first spatiotemporal topological graph.

[0082] 203. Use the second graph attention layer to perform a second update on the node features in the first spatiotemporal topological map to obtain a second spatiotemporal topological map, and use the third graph attention layer to perform a third update on the node features in the second spatiotemporal topological map to obtain a third spatiotemporal topological map.

[0083] 204. Extract node features corresponding to the missing pixel points in the third spatiotemporal topological map, and use the node features as spatiotemporal topological features of the missing pixel points.

[0084] In the embodiment of the present invention, the feature embedding network includes three layers of graph attention layers. That is, the nodes in the spatiotemporal topology graph undergo three layers of self-attention learning, and the node features are updated three times. Each node undergoes the same process of each node feature update. Taking the graph attention network with the second version of the graph attention layer as an example, the process of each node updating the node features in the graph attention layer can be expressed as:

[0085]

[0086] in, for The node features after the node passes through a graph attention layer, for node characteristics of the node, Belongs to Node Neighbor nodes or The node itself, represents the linear transformation weight, is the attention score. It can be expressed as:

[0087]

[0088] in, express The node to which the node corresponds in the previous update cycle, k Also belongs to node Neighbor nodes or The node itself, but the neighbor nodes are in different computation stages, and k and The loops in the calculation phases are of equal length, Representation Node and nodes The raw attention score between is a learnable attention score vector used to calculate the compatibility between nodes, LeakyReLU is a nonlinear activation function, () is an exponential function.

[0089] It should be noted that in the process of updating node features, since the central node does not have self-loops and does not point to other nodes, its own information will be shielded or ignored when passing through the first graph attention layer. Therefore, after updating the node features based on the first graph attention layer, a self-loop is added to the central node of the first spatiotemporal topological graph to ensure that the information of the central node itself can also be retained during subsequent information propagation, thereby improving the estimation accuracy of the spatiotemporal filling model for the central node value, and at the same time, minimizing the ambiguity and other problems that may be caused by the interpolation of estimated spatial features.

[0090] In one embodiment of the present invention, for further explanation and limitation, the updating of the node features in the high-dimensional representation of the node features to obtain the first spatiotemporal topological graph includes:

[0091] Using the first update linear layer to update the edge features in the edge feature high-dimensional representation, and generating a first spatiotemporal topological graph based on the updated edge features and the updated node features;

[0092] The second graph attention layer is used to perform a second update on the node features in the first spatiotemporal topological graph to obtain a second spatiotemporal topological graph, including:

[0093] The second updated linear layer is used to perform a second update on the edge features in the high-dimensional representation of the edge features, and a third spatiotemporal topological graph is generated based on the second updated edge features and the second updated node features.

[0094] In the embodiment of the present invention, in order to ensure that the edge features are not fixed in the state before being input into the spatiotemporal filling model, a first update linear layer is added after the first graph attention layer, and a second update linear layer is added after the second graph attention layer. By configuring the update linear layer after both the first graph attention layer and the second graph attention layer, the edge features can be updated synchronously, so that the edge features can be further enhanced before the next feature information propagation, and the subsequent node features can more comprehensively reflect the relationship between the nodes, thereby enhancing the feature expression ability of the spatiotemporal filling model.

[0095] 104. Use the feature regression network to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and use the prediction results as the pixel data of the missing pixels to perform spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

[0096] In an embodiment of the present invention, the pre-trained spatiotemporal filling model also includes a feature regression network. The feature regression network is mainly used to reduce the dimension of the spatiotemporal topological features of the missing pixels, expand the spatiotemporal topological features into one-dimensional features, and then obtain the prediction results of the missing pixels through the activation function output. After obtaining the prediction results, the prediction results are used as the pixel data of the missing pixels to fill the missing pixels, and the spatiotemporal filling results of the night light data to be processed can be obtained. The night light data to be processed includes multiple missing pixels, and the spatiotemporal topological maps of different missing pixels can be predicted and processed based on the pre-trained spatiotemporal filling model to obtain the prediction results of different missing pixels. After obtaining the prediction results of the missing pixels, the prediction results of each missing pixel are filled into the corresponding positions of the night light data to be processed to obtain the spatiotemporal filling results. You can also select a pixel from the missing pixels as the target pixel, and perform prediction processing on the spatiotemporal topology map of the target pixel based on the pre-trained spatiotemporal filling model, and then obtain the prediction result of the target pixel and fill the night light data to be processed, and then select the next missing pixel from the filled night light data to be processed as the target pixel, and repeat the steps of predicting the spatiotemporal topology map of the target pixel based on the pre-trained spatiotemporal filling model, and then obtain the prediction result of the target pixel and fill the night light data to be processed, until all the missing pixels are traversed to obtain the spatiotemporal filling result of the night light data to be processed.

[0097] In one embodiment of the present invention, for further explanation and limitation, the step of processing the spatiotemporal topological features using the feature regression network to obtain the prediction result of the missing pixel point includes:

[0098] Using the first regression linear layer of the feature regression network to perform a feature dimensionality reduction on the spatiotemporal topological feature, to obtain the spatiotemporal topological feature after the dimensionality reduction;

[0099] The second regression linear layer of the feature regression network is used to reduce the spatiotemporal topological features after the first dimensionality reduction to a single numerical output, and is processed by an activation function layer to obtain the prediction result of the missing pixel point.

[0100] In an embodiment of the present invention, the feature regression network performs dimensionality reduction processing on the spatiotemporal topological features through two linear layers to achieve regression output. Taking into account the mapping relationship between the definition domain and the value range of the subsequent activation function, the output of the first regression linear layer does not pass through the activation function, that is, the first regression linear layer is not configured with an activation function, so as to reduce the gradient change of the last linear layer (the second regression linear layer) while increasing the accuracy of the output value as much as possible. The second regression linear layer reduces the spatiotemporal topological features after dimensionality reduction to one dimension. The second regression linear layer includes a linear activation function and a nonlinear activation function, that is, the output is processed jointly by the linear activation function and the nonlinear activation function. Specifically, the linear activation function uses the Softplus activation function, and the nonlinear activation function uses the Sigmoid function. Set the prediction result to , the processing process of the feature regression network can be expressed as:

[0101]

[0102] in, is the node feature, Represents the base of natural logarithms.

[0103] In one embodiment of the present invention, for further explanation and limitation, before the step of using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features, the method further includes:

[0104] Construct training sample set and initial space-time filling model;

[0105] A loss function is constructed based on the predicted output of the initial space-time filling model trained on the training sample set and the logarithmic hyperbolic cosine function, and an adaptive moment estimation with a Nesterov momentum term is used as an optimizer to optimize the parameters of the initial space-time filling model to obtain a pre-trained space-time filling model.

[0106] In the embodiment of the present invention, the initial spatiotemporal filling model includes a feature embedding network, a topology processing network and a feature regression network. Figure 3As shown, the first node linear layer and the second node linear layer in the node embedding network successively upgrade the dimension of the node attributes, and the first edge linear layer and the second edge linear layer in the edge feature embedding network successively upgrade the dimension of the edge attributes. The activation function T represents the hyperbolic tangent function symmetric to the origin, and the activation function G represents the Gaussian error linear unit. The edge features and node features obtained based on feature embedding are input into the first graph attention layer of the topology processing network to update the node features once, and at the same time, the edge features are updated once based on the first update linear layer. The updated edge features and the updated node features are used as the input of the second graph attention layer to update the node features twice, and at the same time, the edge features are updated twice based on the second update linear layer. Finally, the updated edge features and the updated node features are input into the third graph attention layer to complete the feature prediction. The spatiotemporal topological features obtained through the topology processing network are regressed through the first regression linear layer and the second regression linear layer in the feature regression network in turn. Among them, the first regression linear layer is not configured with an activation function, and the second regression linear layer includes a linear activation function using the Softplus activation function (represented in the figure as ), the nonlinear activation function uses the Sigmoid function (represented in the figure as ). Among them, the first regression linear layer reduces the dimension of node features and introduces Dropout (represented as Drop in the figure), which can effectively prevent overfitting of training.

[0107] Each training sample in the training sample set is constructed by taking the non-missing pixel point as the central node to construct the node attribute set and the edge attribute set of the spatiotemporal topological graph as the training sample data, and the actual pixel data of the non-missing pixel point is used as the training sample label.

[0108] Taking into account the model's computational efficiency, result accuracy and other aspects, the training of the initial space-time filling model uses the adaptive moment estimation with the Nesterov momentum term as the optimizer. The configuration of the training parameters includes: setting the initial learning rate to 0.0001. Setting the callback function, when the loss function value of the validation set does not decrease for 8 consecutive iterations, the learning rate is reduced to the current 0.1, and the learning rate is reduced and cooled for 4 iterations. The construction of the training loss function adopts the logarithmic hyperbolic cosine function, which has the advantages of both mean absolute error and mean square error. It is smooth and differentiable at the closed (analytical) solution, and it is also differentiable twice. Therefore, during the optimization process, the function can provide a stable gradient, which is conducive to the smooth training of the model and has a certain robustness to outliers. Among them, the calculation formula of the logarithmic hyperbolic cosine function is:

[0109]

[0110] in, is the true value, is the predicted value, For the nodes, is the total number of nodes in the node set, () represents the hyperbolic cosine function.

[0111] The present invention provides a method for filling missing nighttime light data in time and space based on a graph neural network. An embodiment of the present invention extracts a spatiotemporal topological map of missing pixels in the nighttime light data to be processed; uses a feature embedding network to embed node features and edge features in the spatiotemporal topological map to obtain high-dimensional representations of node features and high-dimensional representations of edge features; uses a graph attention layer in a topological processing network to perform prediction processing based on the high-dimensional representations of node features, the high-dimensional representations of edge features and the adjacency matrix of the spatiotemporal topological map to obtain spatiotemporal topological features of the missing pixels; uses a feature regression network to process the spatiotemporal topological features to obtain prediction results of the missing pixels, and uses the prediction results as pixel data of the missing pixels to fill in time and space to obtain the spatiotemporal filling results of the nighttime light data to be processed, fully extracts the spatiotemporal correlation between the missing pixels and other pixels in the nighttime light data to be processed, and in the process of prediction based on the graph attention layer, can more accurately capture the complex spatiotemporal correlation between the missing pixels and other known pixels, thereby greatly improving the accuracy of feature prediction of the missing pixels.

[0112] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the figure, the embodiment of the present invention provides a device for filling the missing night light data based on graph neural network time and space, such as Figure 4 As shown, the device performs space-time filling based on a pre-trained space-time filling model, wherein the pre-trained space-time filling model includes a feature embedding network, a topology processing network and a feature regression network, and the device includes:

[0113] An extraction module 31 is used to extract a spatiotemporal topological map of missing pixels in the nighttime light data to be processed;

[0114] A feature embedding module 32, configured to use the feature embedding network to embed node features and edge features in the spatiotemporal topology graph, and obtain a high-dimensional representation of node features and a high-dimensional representation of edge features;

[0115] A prediction processing module 33 is used to perform prediction processing using the graph attention layer in the topology processing network based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topology graph, so as to obtain the spatiotemporal topological features of the missing pixel points;

[0116] The spatiotemporal filling module 34 is used to use the feature regression network to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and use the prediction results as the pixel data of the missing pixels to perform spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

[0117] Furthermore, the prediction processing module 33 includes:

[0118] A computing unit, configured to use the first graph attention layer to calculate the attention scores between the nodes according to the high-dimensional representation of the node features and the high-dimensional representation of the edge features, and determine the node information propagation path and propagation strength according to the adjacency matrix and the attention scores;

[0119] A first updating unit, configured to perform node feature propagation and node feature aggregation according to the propagation path and the propagation strength, so as to update the node features in the high-dimensional representation of the node features, obtain a first spatiotemporal topological graph, and add a self-loop to the central node of the first spatiotemporal topological graph;

[0120] A second updating unit is used to use the second graph attention layer to perform a second update on the node features in the first spatiotemporal topological graph to obtain a second spatiotemporal topological graph, and to use the third graph attention layer to perform a third update on the node features in the second spatiotemporal topological graph to obtain a third spatiotemporal topological graph;

[0121] The third updating unit is used to extract node features corresponding to the missing pixel points in the third spatiotemporal topological map, and use the node features as the spatiotemporal topological features of the missing pixel points.

[0122] Further, in a specific application scenario, the first updating unit is specifically used to use the first updated linear layer to update the edge features in the edge feature high-dimensional representation, and generate a first spatiotemporal topological graph based on the updated edge features and the updated node features;

[0123] The second updating unit is specifically used to perform a second update on the edge features in the high-dimensional representation of the edge features using the second updated linear layer, and generate a third spatiotemporal topological graph based on the second updated edge features and the second updated node features.

[0124] Furthermore, the feature embedding module 32 includes:

[0125] A first dimension-upgrading unit is used to perform feature dimension-upgrading on each node attribute of the spatiotemporal topological graph in turn by using the first node linear layer and the second node linear layer in the node feature embedding network, and to perform standardization processing on the node features after dimension-upgrading by using the first standardization layer to obtain a high-dimensional representation of the node features;

[0126] A second dimension-upgrading unit is used to perform feature dimension-upgrading on each edge attribute of the spatiotemporal topological graph in turn by using the first edge linear layer and the second edge linear layer in the edge feature embedding network, and to perform standardization processing on the dimension-upgraded edge features based on the second standardization layer to obtain a high-dimensional representation of the edge features;

[0127] Among them, the dimension of the high-dimensional representation of the edge feature is aligned with the dimension of the high-dimensional representation of the node feature; the activation functions of the first node linear layer, the second node linear layer and the second edge linear layer are Gaussian error linear units; the activation function of the first edge linear layer is a hyperbolic tangent function based on the origin symmetry.

[0128] Furthermore, the extraction module 31 includes:

[0129] A determination unit, used to determine missing pixels in the nighttime light data to be processed;

[0130] An extraction unit, configured to extract neighbor nodes satisfying a preset number of nodes from a neighborhood of a space-time cube where the missing pixel point is located, taking the missing pixel point as a central node, and configuring pixel data corresponding to the neighbor nodes as node attributes of the neighbor nodes;

[0131] A construction unit, configured to complete edge construction according to the spatiotemporal topological relationship between the neighbor nodes and the central node, and between the neighbor nodes, and configure edge attributes of the edge according to the spatiotemporal deviation value between the nodes connected by the edge;

[0132] A generating unit is used to generate a spatiotemporal topological map of the missing pixel points based on the central node, the neighboring nodes and the edges.

[0133] Furthermore, the space-time filling module 34 includes:

[0134] A first dimensionality reduction unit is used to perform a feature dimensionality reduction on the spatiotemporal topological feature using a first regression linear layer of a feature regression network to obtain a spatiotemporal topological feature after the first dimensionality reduction, wherein the first regression linear layer is not configured with an activation function;

[0135] The second dimensionality reduction unit is used to reduce the spatiotemporal topological features after the first dimensionality reduction to a single numerical output by using the second regression linear layer of the feature regression network, and obtain the prediction result of the missing pixel point through processing by the activation function layer, wherein the second regression linear layer includes a linear activation function and a nonlinear activation function.

[0136] Furthermore, the device also includes:

[0137] A construction module, used to construct a training sample set and an initial spatiotemporal filling model, wherein the initial spatiotemporal filling model includes a feature embedding network, a topology processing network and a feature regression network; the feature embedding network includes a node feature embedding network and an edge feature embedding network, the node feature embedding network includes a first node linear layer and a second node linear layer, and the edge feature embedding network includes a first edge linear layer and a second edge linear layer; the topology processing network includes a first graph attention layer, a second graph attention layer and a third graph attention layer, as well as a first update linear layer and a second update linear layer; the output of the first graph attention layer is used as the input of the first update linear layer, and the output of the second graph attention layer is used as the input of the second update linear layer; the feature regression network includes a first regression linear layer and a second regression linear layer, wherein the first regression linear layer is not configured with an activation function and has Dropout introduced, and the second regression linear layer includes a linear activation function and a nonlinear activation function;

[0138] A training unit is used to construct a loss function based on the predicted output of the initial space-time filling model for the training sample set and the logarithmic hyperbolic cosine function, and use the adaptive moment estimation with the Nesterov momentum term as an optimizer to optimize the parameters of the initial space-time filling model to obtain a pre-trained space-time filling model.

[0139] The present invention provides a device for spatiotemporally filling missing nighttime light data based on a graph neural network. An embodiment of the present invention extracts a spatiotemporal topological map of missing pixels in the nighttime light data to be processed; uses a feature embedding network to embed node features and edge features in the spatiotemporal topological map to obtain high-dimensional representations of node features and high-dimensional representations of edge features; uses a graph attention layer in a topological processing network to perform prediction processing based on the high-dimensional representations of node features, the high-dimensional representations of edge features and the adjacency matrix of the spatiotemporal topological map to obtain spatiotemporal topological features of the missing pixels; uses a feature regression network to process the spatiotemporal topological features to obtain prediction results of the missing pixels, and uses the prediction results as pixel data of the missing pixels to perform spatiotemporal filling to obtain the spatiotemporal filling results of the nighttime light data to be processed, fully extracts the spatiotemporal correlation between the missing pixels and other pixels in the nighttime light data to be processed, and in the process of prediction based on the graph attention layer, can more accurately capture the complex spatiotemporal correlation between the missing pixels and other known pixels, thereby greatly improving the accuracy of feature prediction of the missing pixels.

[0140] According to one embodiment of the present invention, a storage medium is provided, which stores at least one executable instruction. The computer executable instruction can execute the method of filling in the missing night light data based on graph neural network spatiotemporally in any of the above method embodiments.

[0141] Figure 5A schematic diagram of the structure of a terminal provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the terminal.

[0142] like Figure 5 As shown, the terminal may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0143] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0144] The communication interface 404 is used for network communication with other devices such as a client or other servers.

[0145] Processor 402 is used to execute program 410, and specifically can execute the relevant steps in the above-mentioned method embodiment of the method for filling in the missing night light data based on graph neural network spatiotemporally.

[0146] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0147] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the terminal may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0148] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0149] The program 410 may be specifically configured to enable the processor 402 to perform the following operations:

[0150] Extract the spatiotemporal topological map of missing pixels in the night light data to be processed;

[0151] Using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph, to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features;

[0152] According to the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topological graph, a graph attention layer in the topological processing network is used to perform prediction processing to obtain the spatiotemporal topological features of the missing pixel points;

[0153] The feature regression network is used to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and the prediction results are used as the pixel data of the missing pixels for spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

[0154] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for filling missing nighttime light data in time and space based on graph neural network, characterized in that: Performing space-time filling based on a pre-trained space-time filling model, wherein the pre-trained space-time filling model includes a feature embedding network, a topology processing network and a feature regression network, the topology processing network includes a three-layer graph attention layer, and the method includes: Extract the spatiotemporal topological map of missing pixels in the night light data to be processed; Using the feature embedding network to embed node features and edge features in the spatiotemporal topology graph, to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features; According to the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topology graph, the graph attention layer in the topology processing network is used to perform prediction processing to obtain the spatiotemporal topological features of the missing pixel points, including: using a first graph attention layer, according to the high-dimensional representation of the node features and the high-dimensional representation of the edge features, calculating the attention scores between each node, and determining the node information propagation path and propagation strength according to the adjacency matrix and the attention score; performing node feature propagation and node feature aggregation according to the propagation path and the propagation strength to update the node features in the high-dimensional representation of the node features once to obtain a first spatiotemporal topology graph, and adding a self-loop to the central node of the first spatiotemporal topology graph; using a second graph attention layer to perform a second update on the node features in the first spatiotemporal topology graph to obtain a second spatiotemporal topology graph, and using a third graph attention layer to perform a third update on the node features in the second spatiotemporal topology graph to obtain a third spatiotemporal topology graph; extracting the node features corresponding to the missing pixel points in the third spatiotemporal topology graph, and using the node features as the spatiotemporal topological features of the missing pixel points; The feature regression network is used to process the spatiotemporal topological features to obtain the prediction results of the missing pixels, and the prediction results are used as the pixel data of the missing pixels for spatiotemporal filling to obtain the spatiotemporal filling results of the night light data to be processed.

2. The method according to claim 1, characterized in that The topology processing network further includes a first update linear layer, wherein the node features in the high-dimensional representation of the node features are updated once to obtain a first spatiotemporal topology graph, including: Using the first update linear layer to update the edge features in the edge feature high-dimensional representation, and generating a first spatiotemporal topological graph based on the updated edge features and the updated node features; The topology processing network further includes a second updated linear layer, and the second graph attention layer is used to perform a second update on the node features in the first spatiotemporal topology graph to obtain a second spatiotemporal topology graph, including: The second updated linear layer is used to perform a second update on the edge features after the first update, and a third spatiotemporal topological graph is generated according to the second updated edge features and the second updated node features.

3. The method according to claim 1, characterized in that The feature embedding network includes a node feature embedding network and an edge feature embedding network, and the node feature embedding and edge feature embedding of the spatiotemporal topology graph using the feature embedding network to obtain a high-dimensional representation of the node feature and a high-dimensional representation of the edge feature include: Using the first node linear layer and the second node linear layer in the node feature embedding network to perform feature dimension upgrade on each node attribute of the spatiotemporal topology graph in turn, and using the first standardization layer to perform standardization on the node features after dimension upgrade to obtain a high-dimensional representation of the node features; Using the first edge linear layer and the second edge linear layer in the edge feature embedding network to sequentially perform feature dimension upgrade on each edge attribute of the spatiotemporal topological graph, and performing standardization processing on the dimension-enhanced edge features based on the second standardization layer to obtain a high-dimensional representation of the edge features; Among them, the dimension of the high-dimensional representation of the edge feature is aligned with the dimension of the high-dimensional representation of the node feature; the activation functions of the first node linear layer, the second node linear layer and the second edge linear layer are Gaussian error linear units; the activation function of the first edge linear layer is a hyperbolic tangent function based on the origin symmetry.

4. The method according to claim 1, characterized in that: The step of extracting a spatiotemporal topological map of missing pixels in the nighttime light data to be processed includes: Determining missing pixels in the nighttime light data to be processed; Taking the missing pixel point as the central node, extracting neighbor nodes that meet the preset number of nodes from the neighborhood of the space-time cube where the missing pixel point is located, and configuring the pixel data corresponding to the neighbor nodes as the node attributes of the neighbor nodes; Complete edge construction according to the spatiotemporal topological relationship between the neighbor nodes and the central node, and between the neighbor nodes, and configure edge attributes of the edge according to the spatiotemporal deviation value between the nodes connected by the edge; A spatiotemporal topological map of the missing pixels is generated based on the central node, the neighboring nodes and the edges.

5. The method according to claim 1, characterized in that The step of processing the spatiotemporal topological features using the feature regression network to obtain a prediction result of the missing pixel point includes: Performing a feature dimensionality reduction on the spatiotemporal topological feature using a first regression linear layer of a feature regression network to obtain a spatiotemporal topological feature after dimensionality reduction, wherein the first regression linear layer is not configured with an activation function; The second regression linear layer of the feature regression network is used to reduce the spatiotemporal topological features after the first dimensionality reduction to a single numerical output, and is processed by an activation function layer to obtain the prediction result of the missing pixel point, wherein the second regression linear layer includes a linear activation function and a nonlinear activation function.

6. The method according to claim 1, characterized in that Before embedding node features and edge features of the spatiotemporal topology graph using the feature embedding network to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features, the method further includes: Constructing a training sample set and an initial space-time filling model, wherein the initial space-time filling model includes a feature embedding network, a topology processing network and a feature regression network; The feature embedding network includes a node feature embedding network and an edge feature embedding network, wherein the node feature embedding network includes a first node linear layer and a second node linear layer, and the edge feature embedding network includes a first edge linear layer and a second edge linear layer; The topology processing network includes a first graph attention layer, a second graph attention layer, and a third graph attention layer, as well as a first update linear layer and a second update linear layer; the output of the first graph attention layer is used as the input of the first update linear layer, and the output of the second graph attention layer is used as the input of the second update linear layer; The feature regression network includes a first regression linear layer and a second regression linear layer, wherein the first regression linear layer is not configured with an activation function and has Dropout introduced, and the second regression linear layer includes a linear activation function and a nonlinear activation function; A loss function is constructed based on the predicted output of the initial space-time filling model trained on the training sample set and the logarithmic hyperbolic cosine function, and an adaptive moment estimation with a Nesterov momentum term is used as an optimizer to optimize the parameters of the initial space-time filling model to obtain a pre-trained space-time filling model.

7. A device for filling missing nighttime light data in time and space based on graph neural network, characterized in that: The device performs space-time filling based on a pre-trained space-time filling model, wherein the pre-trained space-time filling model includes a feature embedding network, a topology processing network and a feature regression network, and the device includes: An extraction module is used to extract the spatiotemporal topological map of missing pixels in the night light data to be processed; A feature embedding module, used to embed node features and edge features in the spatiotemporal topology graph using the feature embedding network to obtain a high-dimensional representation of node features and a high-dimensional representation of edge features; A prediction processing module, used to perform prediction processing using the graph attention layer in the topology processing network based on the high-dimensional representation of the node features, the high-dimensional representation of the edge features and the adjacency matrix of the spatiotemporal topology graph, to obtain the spatiotemporal topological features of the missing pixel points; A spatiotemporal filling module is used to process the spatiotemporal topological features using the feature regression network to obtain the prediction results of the missing pixels, and use the prediction results as the pixel data of the missing pixels to perform spatiotemporal filling to obtain the spatiotemporal filling results of the nighttime light data to be processed; Wherein, the prediction processing module includes: A computing unit, configured to use the first graph attention layer to calculate the attention scores between the nodes according to the high-dimensional representation of the node features and the high-dimensional representation of the edge features, and determine the node information propagation path and propagation strength according to the adjacency matrix and the attention scores; A first updating unit, configured to perform node feature propagation and node feature aggregation according to the propagation path and the propagation strength, so as to update the node features in the high-dimensional representation of the node features, obtain a first spatiotemporal topological graph, and add a self-loop to the central node of the first spatiotemporal topological graph; A second updating unit is used to use the second graph attention layer to perform a second update on the node features in the first spatiotemporal topological graph to obtain a second spatiotemporal topological graph, and to use the third graph attention layer to perform a third update on the node features in the second spatiotemporal topological graph to obtain a third spatiotemporal topological graph; The third updating unit is used to extract node features corresponding to the missing pixel points in the third spatiotemporal topological map, and use the node features as the spatiotemporal topological features of the missing pixel points.

8. A storage medium storing at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the method for spatiotemporally filling missing nighttime light data based on a graph neural network as described in any one of claims 1 to 6.

9. A terminal, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method for spatiotemporally filling missing night light data based on a graph neural network as described in any one of claims 1-6.

Images