Graph representation learning method and device considering graph node attribute missing and graph structure incompleteness, and storage medium

Abstract

The invention provides a graph representation learning method and device considering graph node attribute missing and graph structure incompleteness, and a storage medium, and belongs to the technical field of graph neural networks. The problem that an existing method takes a single node as a basic unit, the mode of deducing the full view of information in a graph does not conform to the urban regional association theory, and a regional interaction back mechanism cannot be effectively and reasonably described is solved. According to the method, a node pair is taken as a basic unit, a node attribute-structure cyclic consistency constraint is introduced, joint distribution of node attributes and edges in the graph is modeled, and missing information in the graph is complemented by using the distribution, so that the method is more matched with the complementation of the missing information of an urban area; the application background of the method is data-driven city system dynamics modeling and is used for predicting the city development situation.

Description

Technical Field

[0001] The present application relates to the technical field of graph processing and machine learning of graph representation, and in particular to a graph representation learning method, device, and storage medium that consider missing graph node attributes and incomplete graph structure. Background Art

[0002] System dynamics modeling reveals the evolution of a system by characterizing the interactions and influences between its components. Cities are composed of various interconnected spatial regions, and urban system dynamics modeling is crucial for predicting urban development trends. However, the complex dynamics of urban systems make traditional mechanistic analysis and modeling challenging. Physics-informed machine learning, with its successful use in modeling complex system dynamics in other fields, offers a data-driven modeling approach. Within this data-driven modeling approach, graph neural networks (GNNs), as the de facto standard for relational data modeling, are crucial for urban system dynamics modeling.

[0003] However, in real-world big data scenarios for urban planning, data completeness often falls short of expectations. For example, information on some urban areas is missing, and the graph structures constructed based on the available information are often incomplete. This poses a challenge to the application of GNNs to real-world problems. Representation learning on incomplete graphs holds important research value for solving practical urban problems. Based on the extent of missing information in the graph, incomplete graphs can be divided into two categories: 1) node-attribute incomplete graphs, where some node attributes are partially (or completely) missing; 2) structural incomplete graphs, where some node edges are partially (or completely) missing; and 3) incomplete graphs, where some node attributes and / or edges are partially (or completely) missing.

[0004] In learning graph representations with incomplete node attributes, some methods utilize imputation-oriented machine learning techniques, such as matrix factorization and Gaussian mixture models, to first fill in the attributes and then learn node representations using GNNs. However, the performance of these methods degrades dramatically when node attributes are completely missing. To address this issue, several targeted methods have been proposed. Initially, SAT introduced a shared latent space assumption to explicitly model the joint distribution of node attributes and structure, attempting to impute missing attribute values. Specifically, SAT first samples nodes with complete attributes from the graph, extracts their attributes and structural information (edge ​​connections) to form a sample set, then learns latent representations of node attributes and structure in a decoupled manner. Finally, by aligning these two distributions, it models the joint distribution of node attributes and structure, and finally uses this distribution to recover the missing attributes of nodes in the graph. Subsequently, Jin et al. extracted useful node information from both the node attribute and graph structure perspectives. They then used GANs to enforce attribute-structure relationships through adversarial learning, ultimately achieving an implicit joint modeling of node attributes and structure in the graph. However, existing methods use individual nodes as the basic unit and model the joint distribution of individual node attributes and all edges connecting to them. This does not align with the mechanisms underlying urban-region spatial interactions. According to urban-region association theory, the association between two regions depends on the degree of matching between their attributes (e.g., complementarity or competition). For a single region, it is difficult to infer its association with all other regions based solely on the attributes of that one region; information about the attributes of other regions is also necessary. Therefore, in urban-region spatial interaction graph models, the edges between two nodes are jointly influenced by the attributes of both nodes. Given incomplete attributes, it is necessary to explore a method for modeling information in the graph that is more consistent with urban-region association theory.

[0005] In learning representations for incomplete graphs, researchers attempt to search for an implicit graph structure tailored to downstream tasks and jointly optimize task-related graph representations. A general graph structure learning framework consists of three modules: initial graph construction, graph structure modeling, and message propagation. The initial graph is constructed based on limited prior knowledge and carries only partial information about the system, resulting in an incomplete graph structure. Graph structure modeling improves the initial graph by modeling edge connectivity. Message propagation is a general GNN information aggregation. Existing methods for representing incomplete graphs can be categorized into three types based on their graph structure modeling approach: metric-based methods, neural network-based methods, and direct methods. According to urban region association theory, the interaction between different regions is primarily reflected in the matching of various attributes, such as economic, cultural, and environmental. If two regions are complementary (competitive) in certain attributes, they will promote (constrain) each other. Complex associations exist between regions, but without an explicitly defined relationship. Urban region spatial interaction graphs are incompletely structured. Therefore, considering the interpretability of regional attribute matching, it is necessary to study graph structure modeling for downstream tasks.

[0006] In urban-region spatial interaction graphs, some regions lack information and lack explicit relationships between regions. This means both node attributes and the graph structure are incomplete. Literature research reveals that existing incomplete graph representation methods do not align with the regional interaction mechanisms within the context of urban-region association theory. This hinders the completion of missing information in urban-region spatial interaction graphs and the execution of downstream tasks. Therefore, it is necessary to design incomplete graph representation methods that are consistent with regional interaction mechanisms, taking into account urban-region association theory. Summary of the Invention

[0007] In order to solve the problem that the existing methods use a single node as the basic unit and the way of inferring the full picture of information in the graph does not conform to the urban area association theory and cannot effectively and reasonably characterize the mechanism behind regional interaction, this application proposes a graph representation learning method, device, and storage medium that considers the missing graph node attributes and incomplete graph structure. It proposes to use "node pairs" as the basic unit, introduce the circular consistency constraint of node attribute-structure, model the joint distribution of node attributes and edges in the graph, and use this distribution to complete the missing information in the graph, which is more consistent with the completion of missing information in urban areas.

[0008] The technical solution adopted in this application is: a graph representation learning method that considers missing graph node attributes and incomplete graph structure, including the following steps:

[0009] 1. Initial graph construction, including:

[0010] S1: Construct an undirected graph and determine the node set of the graph , edge set And the node attribute matrix , and according to the missing of node attributes, the node set is divided into attribute complete set , attribute part missing set and completely missing attributes , and determine the corresponding node attribute matrix 、 、 ;

[0011] 2. Complete the image information, including:

[0012] S2: Node attribute-structure association modeling, including the following steps:

[0013] S21: Set the encoder and decoder;

[0014] S22: Select attribute complete node and its edge information as training set , update the parameters according to the objective function of the overall loss function of attribute-structure association modeling;

[0015] S23: Add attributes to some missing nodes For training samples, the training model recovers Missing attributes in ;

[0016] S24: Use the trained model to complete the nodes with completely missing attributes Attributes;

[0017] S25: Output attribute-completed node attribute matrix , graph structure parameters , and attribute reconstruction parameters;

[0018] 3. Adjustment of the graph structure, including:

[0019] S3: Build a GCN model to learn node-wise vector representations for downstream tasks.

[0020] S4: Fixed node attribute reconstruction parameters and adjusted graph structure parameters ;

[0021] S5: Fixed graph structure parameters , adjust the node attribute reconstruction parameters and update the node attribute matrix ;

[0022] Steps S3 and S4 are performed alternately with steps S5 and S2 until convergence, ultimately achieving graph representation learning for downstream tasks.

[0023] Furthermore, the overall loss function of attribute-structure association modeling is expressed as:

[0024] ;

[0025] in, is the constraint loss of attributes on the graph structure, is the joint reconstruction loss, is the likelihood loss of the graph structure for the associated node attribute constraints, is the prior regularization loss, Reconstruct parameters for attributes, representing the parameters of decoder and encoder;

[0026] in and The node attribute-structure cycle consistency constraint is constructed, and the overall loss function realizes the modeling of the joint distribution of node attributes and edges in the graph.

[0027] Furthermore, the constraint loss of attributes on the graph structure The expression is:

[0028] ;

[0029] in, Represents a sample with associated relationships sampled from the graph, Characterizes the joint probability distribution of two regional attributes, is an edge set Each edge in Corresponding to a real-valued weight, representing the node and The connection strength, are learnable parameters in the graph structure.

[0030] Furthermore, the joint reconstruction loss The expression is: ;

[0031] in, represents a pair of node attributes with an associated relationship sampled from the graph, It is a hyperparameter control. 、 Both represent mapping graph domain node attributes to latent space variables. 、 Both represent the reconstruction of node attributes by the same regional latent variables. 、 Both represent the reconstruction of node attributes by associated regional latent variables.

[0032] Furthermore, the likelihood loss of the graph structure for the associated node attribute constraints The expression is: ;

[0033] in, represents the indicator function, yes A set of sampling nodes, is a hyperparameter, For the node , Its relationship with More closely related to it, if and only if the structural information .

[0034] Furthermore, the input of GCN in step S3 is the completed node attribute matrix and graph structure parameters , the node representation is learned through GNN forward propagation for downstream tasks, and then the graph structure parameters are calculated by back propagation of downstream task losses. Through adjustments, the graph structure and node representation for downstream tasks are finally learned.

[0035] Furthermore, the undirected graph constructed in step S1 is an initial graph of urban regional spatial interaction. The initial graph of urban regional spatial interaction is an incomplete graph with regions as nodes, pedestrian flow representing regional development associations, and basic information of regional spatial planning as node features.

[0036] A computer device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0037] A computer-readable storage medium stores a computer program / instruction thereon, which implements the steps of the method when executed by a processor.

[0038] A computer program product comprises a computer program / instructions which, when executed by a processor, implement the steps of the method.

[0039] The beneficial effects of this application compared to the prior art are:

[0040] (1) In the GNN-based urban system dynamics modeling, this application proposes a new incomplete graph representation learning method to complete the missing information in the graph, solve the challenges brought by incomplete data in real big data scenarios, and realize the association modeling between urban areas.

[0041] (2) This application takes the “node pair” in the graph as the basic research unit, fully explores the relationship between the topological structure and node attributes in the graph, and provides a more accurate modeling method for recovering missing information in the graph, which is more consistent with the completion of missing information in urban areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present application will be further described below with reference to the accompanying drawings:

[0043] Figure 1 The incomplete graph representation learning framework diagram provided in the embodiment of the present application;

[0044] Figure 2 This is a diagram of the attribute-structure association modeling framework provided in the embodiments of the present application. DETAILED DESCRIPTION

[0045] like Figure 1 and 2 As shown, the present application provides a graph representation learning method that takes into account missing graph node attributes and incomplete graph structure, which can be used for modeling the development association between different regions in urban system dynamics and for predicting urban development trends. The key technical problem solved by the present invention is incomplete graph representation learning. The association between different regions in urban system dynamics can be modeled by GNN, but the incompleteness of data in real big data scenarios poses a challenge to the standard GNN algorithm. Considering the simultaneous existence of missing node attributes and incomplete graph structure in the graph, studying the method of characterizing the urban area association mechanism, completing the missing information in the graph, and realizing representation learning on the incomplete graph is an important scientific problem to be solved by this application.

[0046] The following is a problem definition and symbolic description of the incomplete graph to be solved by this application.

[0047] make represents an undirected graph, where , , Represent the node set, edge set and node attribute matrix respectively. More specifically, the node set Include nodes; edge set Each edge in Corresponding to a real-valued weight , representing a node and The connection strength of node attribute matrix Represents the attribute information of the nodes in the graph, is the node feature dimension, The field of real numbers.

[0048] According to the missing of node attributes, the node collection is further divided into the complete set of attributes , attribute part missing set and completely missing attributes , the corresponding node attribute matrix is , , .make , , Respectively represent the number of three types of nodes in the graph, then there is , , , The incomplete graph structure means that the weights of some edges Information error, such as being mistakenly set to 0 or an inappropriate real value.

[0049] Definition 1 (Incomplete Graph Representation Learning): Incomplete graph representation learning aims to use the available information to learn the graph structure and dense vector representation of nodes for downstream tasks on a graph with missing node attributes and incomplete structure, that is, to use the attributes of the complete nodes in the graph. , attributes of the missing nodes And the initial structural information in the figure , learn a mapping , making ,in It is a graph structure for downstream tasks. Representation node and The connection strength, It is a node for downstream tasks dimensional dense vector representation, is the number of nodes in the graph.

[0050] Based on the above definition of undirected graph, the incomplete graph representation learning framework proposed in this application is as follows Figure 1 As shown in the figure, the framework consists of three modules: initial graph construction, graph information completion, and graph structure adjustment. The specific implementation steps are as follows:

[0051] S1: Initial graph construction:

[0052] This module constructs an initial graph of urban area spatial interactions based on mobile phone signaling data and urban operation big data. The steps include urban area spatial division and area attribute mapping. The former obtains nodes in the graph, and the latter obtains node attributes.

[0053] In the urban area spatial division stage, based on previous research, human mobility information is extracted from mobile phone signaling data, the characteristics of human mobility are analyzed, and the city is divided into grid spaces that conform to the laws of human activities; edges between grids are established based on human mobility information, and the weight of the edge is set as the flow of people between grids.

[0054] In the regional attribute mapping stage, basic land space indicator data such as demographic information, POI (Point of Interest) data, and enterprise information are mapped into the grid.

[0055] At this point, the initial graph of urban regional spatial interaction is an incomplete graph with regions as nodes, pedestrian flow representing regional development relationships (i.e., edges), and basic information on regional spatial planning as node features.

[0056] S2: Node attribute-structure association modeling:

[0057] This module introduces the node attribute-structure cycle consistency constraint, models the joint distribution of node attributes and edges in the graph, and uses this distribution to complete the missing information in the graph. , , Represents nodes respectively and The attribute vector of and the weight of the edge between these two nodes, then is a sample describing regional association. The joint log-likelihood of regional interaction on the urban region spatial interaction graph consists of the sum of the likelihoods of individual samples, i.e. ,in is the joint probability density of a pair of region attributes and edges. Further,

[0058] (1);

[0059] The first term in formula (1) This describes the influence of two region attributes on the relationship between them, which corresponds to the constraints imposed by the attributes on the graph structure. Considering the interpretability of region attribute matching, a metric-based method is chosen to describe this conditional probability:

[0060] (2);

[0061] (3);

[0062] Where, Indicates the mean , the variance is Gaussian kernel with learnable parameters is a symmetric semi-positive matrix used to model the matching relationship between regional attributes. From equations (2) and (3), we know that given the attributes of a region, Determines the structure of the graph. At this time, according to the maximum likelihood estimation method, the constraint loss of the attribute on the graph structure for:

[0063] (4);

[0064] in, Represents a sample with associated relationships sampled from the graph, is the real node pair distribution in the graph, For expectation.

[0065] The second term in formula (1) The joint probability distribution of two regional attributes is characterized. With the idea of ​​maximizing marginal likelihood by variational autoencoder (VAE), the latent space of regional attributes is introduced. , through latent variables Refactoring . Specifically, Can be processed as:

[0066] (5);

[0067] The first term in (5) is the approximate posterior True posteriori The KL divergence of , the second term in formula (5) is the log-likelihood Since the KL divergence is non-negative, formula (5) can be further transformed into:

[0068] (6);

[0069] The first term on the right side of the equal sign in formula (6) represents the joint reconstruction loss, which is determined by the node attributes. Encoding in latent space The second term on the right side of the equal sign in formula (6) represents the prior regularization loss, which measures the approximate posterior With prior degree of matching.

[0070] When dealing with the right side of the equal sign in formula (6), according to the probability chain rule:

[0071] (7);

[0072] (8).

[0073] Introducing the independence assumption, assuming that given In case, are conditionally independent, and given In the case of is conditionally independent, then formula (7) is further transformed into:

[0074] (9);

[0075] in Through the encoder Will Encoded as Implementation, which means mapping the graph domain node attributes to latent space variables, Same thing.

[0076] On the basis of the conditional independence assumption, considering the coupling of the node attributes in the associated region, formula (8) is further processed and converted into:

[0077] (10);

[0078] in Through the decoder Will Decoded as Implementation, indicating that node attributes are reconstructed by latent variables in the same region, Through the decoder Will Decoded as Implementation, which means reconstructing node attributes by associated regional latent variables, and Similarly, formula (10) reconstructs attributes by matching attributes of associated regions, where Used to reconstruct its own properties and associated region attributes , Used to reconstruct its own properties and associated region attributes .

[0079] So far, considering Equations (9) and (10), the joint reconstruction loss of the first term on the right side of Equation (6) is: (11);

[0080] in Represents a pair of node attributes with associated relationships sampled from the graph, hyperparameters Controls the cross-reconstruction loss weight.

[0081] In addition, the latent space The latent variables are underconstrained. To make the encoding process more reasonable, the latent variables should maintain the consistency of the attribute-structure relationship, that is, the latent variables in closely connected areas should be closer. To this end, the constraints of the graph structure on the attributes of the associated nodes are introduced to effectively regularize the distribution of the latent variables.

[0082] Specifically, for a node triple , agreed For the node , Its relationship with More closely related to it, if and only if the structural information At this time, the encoder The latent variables obtained should satisfy ,Right now and More similar. Indicates that the node and 、 In the relationship between the two, and The probability of a closer relationship, where is a Sigmoid function, then the likelihood loss of the graph structure for the associated node attribute constraints is:

[0083] (12);

[0084] In the formula represents the indicator function, yes A set of sampling nodes, hyperparameters Affects the complexity of the algorithm.

[0085] For the prior regularization loss of the second term on the right side of the equal sign in formula (6), based on formula (9), it is processed according to the generative adversarial method, and the result is recorded as .

[0086] In summary, the overall loss function for attribute-structure association modeling is:

[0087] (13);

[0088] in, and The node attribute-structure cycle consistency constraint is constructed, and the loss function as a whole models the joint distribution of node attributes and edges in the graph.

[0089] Based on the above theoretical derivation, the attribute-structure association modeling is as follows Figure 2 The specific implementation steps are as follows:

[0090] 1) Set up the encoder , decoder , They are 2-layer MLPs respectively.

[0091] 2) Select the attribute complete node and its edge information as training set , update the parameters according to the objective function of formula (13).

[0092] 3) Add attributes to partially missing nodes For training samples, the training model recovers Specifically, according to the idea of ​​sample denoising, the attributes of the sample are partially missing. Add random Gaussian noise G to form a new training set , at this time, the objective function of formula (13) involves The node part only calculates the loss when reconstructing visible attributes. After the training is completed, the decoder , The exponential moving average of the output is Completed attributes.

[0093] 4) Use the trained model to complete the nodes with completely missing attributes According to the prior distribution of the latent variable Sampling to obtain samples , then Via decoder , The processed exponential sliding average result is Completed attributes.

[0094] 5) Output the node attribute matrix of attribute completion , graph structure parameters , and attribute reconstruction parameters .

[0095] S3: Graph structure adjustment:

[0096] This module receives the completed node attribute matrix and graph structure parameters And as input, the node representation is learned through GNN forward propagation to perform downstream tasks, and then the graph structure parameters are calculated using the back propagation of the downstream task loss. The GNN module adopts the classic 2-layer GCN. The layer node representation is calculated as follows:

[0097] (14);

[0098] in , By graph structure parameters Sure, is the identity matrix, represents the degree matrix, Indicates the The parameters of the layer, represents the ReLU activation function, For the The node representation of the layer, and In the downstream task label Under the supervision of Update parameters , thereby achieving graph structure adjustment.

[0099] In summary, if Figure 1 As shown in FIG, the overall process of incomplete graph representation learning applied in this application to urban system dynamics modeling is as follows:

[0100] 1) Execute step ① in the diagram and use human mobility information and urban operation big data to build an initial urban spatial area interaction map.

[0101] 2) Execute step ② in the diagram to model the node attribute-structure relationship and learn the graph structure parameters And node attribute reconstruction parameters , use the learned model to complete the missing attributes in the graph and obtain the node attribute matrix .

[0102] 3) Fixation , respectively execute ③④ in the figure, use downstream tasks to adjust the graph structure, and update the graph structure parameters .

[0103] 4) Fixation , respectively execute ⑤② in the figure, adjust the attribute reconstruction parameters , update the node attribute matrix .

[0104] 5) Alternately execute ③④ and ⑤② until convergence, ultimately achieving graph representation learning for downstream tasks.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A graph representation learning method that considers missing graph node attributes and incomplete graph structure, characterized by: The following steps are involved:

1. Initial graph construction, including: S1: Construct an undirected graph and determine the node set of the graph , edge set And the node attribute matrix , and according to the missing of node attributes, the node set is divided into attribute complete set , attribute part missing set and completely missing attributes , and determine the corresponding node attribute matrix 、 、 ; 2. Complete the image information, including: S2: Node attribute-structure association modeling, including the following steps: S21: Set the encoder and decoder; S22: Select attribute complete node and its edge information as training set , update the parameters according to the objective function of the overall loss function of attribute-structure association modeling; S23: Add attributes to some missing nodes For training samples, the training model recovers Missing attributes in ; S24: Use the trained model to complete the nodes with completely missing attributes Attributes; S25: Output attribute-completed node attribute matrix , graph structure parameters , and attribute reconstruction parameters; 3. Adjustment of the graph structure, including: S3: Build a GCN model to learn node-wise vector representations for downstream tasks. S4: Fixed node attribute reconstruction parameters and adjusted graph structure parameters ; S5: Fixed graph structure parameters , adjust the node attribute reconstruction parameters and update the node attribute matrix ; Steps S3 and S4 are performed alternately with steps S5 and S2 until convergence, ultimately achieving graph representation learning for downstream tasks.

2. A graph representation learning method according to claim 1 that takes into account missing graph node attributes and incomplete graph structure, characterized in that: The overall loss function for attribute-structure association modeling is expressed as: ； in, is the constraint loss of attributes on the graph structure, is the joint reconstruction loss, is the likelihood loss of the graph structure for the attribute constraints of associated nodes, is the prior regularization loss, Reconstruct parameters for attributes, representing the parameters of decoder and encoder; in and The node attribute-structure cycle consistency constraint is constructed, and the overall loss function realizes the modeling of the joint distribution of node attributes and edges in the graph.

3. A graph representation learning method according to claim 2 that takes into account missing graph node attributes and incomplete graph structure, characterized in that: Attribute Constraint Loss on Graph Structure The expression is: ； in, Represents a sample with associated relationships sampled from the graph, Characterizes the joint probability distribution of two regional attributes, is an edge set Each edge in Corresponding to a real-valued weight, representing the node and The connection strength, are learnable parameters in the graph structure.

4. A graph representation learning method according to claim 2 that takes into account missing graph node attributes and incomplete graph structure, characterized in that: Joint reconstruction loss The expression is: ； in, represents a pair of node attributes with an associated relationship sampled from the graph, It is a hyperparameter control. 、 Both represent mapping graph domain node attributes to latent space variables. 、 Both represent the reconstruction of node attributes by the same regional latent variables. 、 Both represent the reconstruction of node attributes by associated regional latent variables.

5. The graph representation learning method according to claim 2, wherein: Likelihood loss of graph structure for associated node attribute constraints The expression is: ； in, represents the indicator function, yes A set of sampling nodes, is a hyperparameter, For the node , Its relationship with More closely related to it, if and only if the structural information .

6. The graph representation learning method according to claim 1, wherein: The input of GCN in step S3 is the completed node attribute matrix and graph structure parameters , the node representation is learned through GNN forward propagation for downstream tasks, and then the graph structure parameters are calculated using the back propagation of the downstream task loss. Through adjustments, the graph structure and node representation for downstream tasks are finally learned.

7. A graph representation learning method according to any one of claims 1 to 6, characterized in that: The undirected graph constructed in step S1 is the initial graph of urban regional spatial interaction. The initial graph of urban regional spatial interaction is an incomplete graph with regions as nodes, pedestrian flow representing regional development associations, and basic information of regional spatial planning as node features.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images