Spatial knowledge graph guided diffusion model training method and spatio-temporal data generation method, device and equipment, and medium

Abstract

The application discloses a diffusion model training method and a spatio-temporal data generation method and device based on a spatial knowledge graph, an equipment and a medium, and relates to the technical field of data processing. The diffusion model training method comprises the following steps: constructing a spatial knowledge graph according to node information in a target region; the target region comprises adjacent first and second sub-regions, and the spatial knowledge graph comprises a plurality of triples composed of a head entity, an inter-entity relationship and a tail entity, and the head entity belongs to a node in the target region; for each first head entity belonging to the first sub-region in the spatial knowledge graph, the neighborhood information of the first head entity is determined; the neighborhood information of each first head entity is aggregated respectively to generate a first embedding guide signal; and the diffusion model is trained according to the first spatio-temporal data of the first sub-region and the first embedding guide signal to obtain a target diffusion model, and the target diffusion model is used for generating second spatio-temporal data of the second sub-region. The target diffusion model can be used to realize a spatio-temporal data generation task for unknown geographical space.

Description

Spatial knowledge graph-guided diffusion model training method and spatiotemporal data generation method, device, equipment and medium Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method for training a diffusion model guided by a spatial knowledge graph and a method, apparatus, device and medium for generating spatiotemporal data. Background Technology

[0002] Spatiotemporal data refers to spatial data that changes over time (such as traffic flow data). It has temporal and spatial attributes and can be used to describe and analyze the changes and relationships of events, phenomena, or objects within a specific time and space range.

[0003] Generative models are developing rapidly in fields such as natural language processing and computer vision. This trend has also attracted widespread attention in the field of spatiotemporal data research, and the application of generative models in spatiotemporal data generation tasks is being explored step by step.

[0004] Since both spatiotemporal and time-series data are related to time factors, current spatiotemporal data generation tasks based on generative models largely refer to the field of time-series data generation. The field of time-series data generation aims to generate data with a consistent distribution for each variable. This paradigm requires the generative model to be trained by inputting and fitting known real samples (i.e., historical data) for each variable, and finally evaluating the distribution difference between the generated samples and the real samples to determine the effectiveness of time-series data generation. Following this paradigm, predicting the spatiotemporal data of a region over a future period based on its historical spatiotemporal data is essentially achieving spatiotemporal data prediction. However, since historical data for distribution comparison is unavailable for unknown geographic spaces (i.e., geographic spaces with unknown spatiotemporal data), the above paradigm cannot generalize to unknown geographic spaces, meaning it cannot generate spatiotemporal data for unknown geographic spaces. Therefore, the ability of generative models to generate spatiotemporal data still needs further exploration. Summary of the Invention

[0005] This application provides a method for training a diffusion model guided by a spatial knowledge graph, as well as a method, apparatus, device, and medium for generating spatiotemporal data, to solve or at least partially solve the defects or deficiencies in related technologies.

[0006] Firstly, this application provides a diffusion model training method guided by spatial knowledge graphs, the diffusion model training method guided by spatial knowledge graphs comprising:

[0007] Node information is collected for nodes in the target region, and a spatial knowledge graph of the target region is constructed based on the node information; the target region includes adjacent first sub-regions and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities, wherein the head entities belong to nodes in the target region;

[0008] For each first head entity belonging to the first sub-region in the spatial knowledge graph, determine the neighborhood information of the first head entity;

[0009] The neighborhood information of each first head entity is aggregated to generate a first embedding guidance signal that includes the node representation of each first head entity.

[0010] First spatiotemporal data is obtained for the first sub-region in the target region, and the diffusion model to be trained is trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

[0011] Optionally, in some embodiments of this application, the diffusion model training method guided by spatial knowledge graphs further includes:

[0012] A first inherent trend feature is obtained by mapping the first embedded guiding signal; the first inherent trend feature represents the functional mode of all the first head entities;

[0013] The step of training the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model includes:

[0014] The target diffusion model is obtained by training the diffusion model to be trained based on the first spatiotemporal data, the first embedding guidance signal, and the first inherent trend feature. The training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process. In the diffusion noise addition process, noise is sampled based on the target noise distribution in time steps, and the sampled noise is added to the first spatiotemporal data in time steps to obtain a noise addition result. In the first diffusion denoising process, noise is predicted based on the time step embedding and the first embedding guidance signal in time steps, and the predicted noise is removed from the noise addition result in time steps. The target noise distribution is a noise distribution that introduces the first inherent trend feature into the noise mean. The time step embedding is obtained by position encoding the time step.

[0015] Optionally, in some embodiments of this application, the time series length of the first spatiotemporal data is T, the number of features of the first spatiotemporal data is D, and the step of obtaining the first inherent trend feature based on the first embedding guidance signal includes:

[0016] When D=1, for each node representation of the first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the mapping module to be trained, and the first inherent trend feature is obtained.

[0017] When D > 1, for each node representation of the first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the training mapping module to obtain the initial inherent trend feature. The same D initial inherent trend features are concatenated to obtain the first inherent trend feature.

[0018] The target mapping module is jointly trained with the diffusion model to obtain the target mapping module.

[0019] Optionally, in some embodiments of this application, the mapping module to be trained includes a multilayer perceptron and / or a convolutional layer.

[0020] Optionally, in some embodiments of this application, any time step is represented as time step i. For the diffusion noise addition process, the noise distribution of the sampling noise added to the first spatiotemporal data at time step i is the target noise distribution. for:

[0021] ;

[0022] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after adding sampling noise to the first spatiotemporal data at time step i. This represents the first spatiotemporal data. This indicates the first embedded guidance signal. This indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the noise mean of the noise added to the first spatiotemporal data at time step i. This represents the noise variance of the noise added to the first spatiotemporal data at time step i.

[0023] Optionally, in some embodiments of this application, any time step is denoted as time step i, and for the first diffusion denoising process, the noise distribution of the predicted noise removed from the denoising result at time step i is... for:

[0024] ;

[0025] in:

[0026] ;

[0027] ;

[0028] ;

[0029] ;

[0030] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after removing the prediction noise from the noise-adding result at time step i-1. This represents the spatiotemporal data obtained at time step i after removing the prediction noise from the noise-adding result. This represents the first spatiotemporal data. This indicates the first embedded guidance signal. This indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the decay parameter corresponding to time step i-1. This represents the noise mean value removed from the noise addition result at time step i. This represents the noise variance of the noise removed from the noise addition result at time step i.

[0031] Optionally, in some embodiments of this application, the diffusion model to be trained is an equivariant diffusion model, which includes multiple noise prediction units corresponding one-to-one with time steps. The multiple noise prediction units are connected in sequence. The noise prediction unit is used to predict the noise corresponding to the time step based on the time step embedding of the corresponding time step and the first embedding guidance signal, and remove the predicted noise corresponding to the time step from the denoising result output by the previous noise prediction unit. The noise prediction unit includes multiple U-Net residual blocks connected in sequence. The U-Net residual block includes a normalization layer, an affine transformation layer, an activation function layer, a feature fusion layer, and an involution layer connected in sequence.

[0032] The affine transformation layer is used to: perform a first linear transformation on the time step embedding of the corresponding time step of the noise prediction unit to obtain a first affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the first affine transformation parameter to obtain a first intermediate feature; and perform the first linear transformation on the first embedding guidance signal to obtain a second affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the second affine transformation parameter to obtain a second intermediate feature.

[0033] The activation function layer is used to: perform a nonlinear transformation on the first intermediate feature using a smooth nonlinear activation function to obtain a third intermediate feature, and perform a nonlinear transformation on the second intermediate feature using the smooth nonlinear activation function to obtain a fourth intermediate feature;

[0034] The feature fusion layer is used to: add the third intermediate feature and the fourth intermediate feature to obtain a fused feature, and input the fused feature into the inner convolutional layer.

[0035] Optionally, in some embodiments of this application, determining the neighborhood information of each first head entity belonging to the first sub-region in the spatial knowledge graph includes:

[0036] Embedding mapping is performed on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity.

[0037] The multiple embedding mapping results corresponding to each neighboring entity of the first head entity are fused and subjected to a second linear transformation to obtain the neighborhood information of the first head entity; the second linear transformation is implemented by a first linear transformation matrix to be trained, and the first linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain a first target linear transformation matrix.

[0038] Optionally, in some embodiments of this application, the step of performing embedding mapping on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity includes:

[0039] Embedding mappings are performed on each neighboring entity of the first head entity using Euclidean space, hyperbolic space, and hyperspherical space, respectively, to obtain the Euclidean space embedding mapping results, hyperbolic space embedding mapping results, and hyperspherical space embedding mapping results corresponding to each neighboring entity of the first head entity.

[0040] Optionally, in some embodiments of this application, the step of aggregating the neighborhood information of each first head entity to generate a first embedding guidance signal including a node representation of each first head entity includes:

[0041] The neighborhood information of each first head entity is aggregated by the graph neural network to be trained to obtain the node representation of each first head entity; the graph neural network to be trained is jointly trained with the diffusion model to be trained to obtain the target graph neural network.

[0042] The node representations of each of the first head entities are fused to obtain the first embedded guidance signal.

[0043] Optionally, in some embodiments of this application, the step of training the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model includes:

[0044] Based on a sliding window and matrix zero-padding, the first spatiotemporal data is converted into a first delayed embedding matrix aligned with the input dimension of the diffusion model to be trained;

[0045] The target diffusion model is obtained by training the diffusion model to be trained based on the first delayed embedding matrix and the first embedding guidance signal.

[0046] Optionally, in some embodiments of this application, the step of collecting node information for nodes in the target area includes: collecting node information for nodes in the target area at multiple spatial granularities;

[0047] The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region according to the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0048] The first head entity includes nodes in the first sub-region corresponding to the target spatial granularity; the first spatiotemporal data includes spatiotemporal data of the target data type obtained for the target object in the first sub-region.

[0049] Secondly, this application provides a spatiotemporal data generation method guided by spatial knowledge graphs. The spatiotemporal data generation method generates spatiotemporal data based on a target diffusion model trained as described in the first aspect. The spatiotemporal data generation method guided by spatial knowledge graphs includes:

[0050] For each second head entity belonging to the second sub-region in the spatial knowledge graph, determine the neighborhood information of the second head entity;

[0051] The neighborhood information of each second head entity is aggregated to generate a second embedding guidance signal that includes the node representation of each second head entity;

[0052] The target diffusion model is controlled to generate second spatiotemporal data for the second sub-region based on preset noise data and the second embedding guidance signal. The generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction based on time step embedding and the second embedding guidance signal in time steps, and gradually removes the predicted noise from the preset noise data in time steps to obtain the second spatiotemporal data of the second sub-region. The time step embedding is obtained by position encoding the time step.

[0053] Optionally, in some embodiments of this application, the step of collecting node information for nodes in the target area includes: collecting node information for nodes in the target area at multiple spatial granularities;

[0054] The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region according to the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0055] The second head entity includes nodes in the second sub-region corresponding to the target spatial granularity; the second spatiotemporal data includes spatiotemporal data of the target data type generated for the target object in the second sub-region.

[0056] Thirdly, this application provides a diffusion model training device guided by spatial knowledge graphs, the diffusion model training device guided by spatial knowledge graphs comprising:

[0057] The data collection and construction module is configured to collect node information for nodes in a target region and construct a spatial knowledge graph of the target region based on the node information. The target region includes adjacent first and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities. The head entities belong to nodes in the target region.

[0058] The first information determination module is configured to determine the neighborhood information of each first head entity belonging to the first sub-region in the spatial knowledge graph.

[0059] The first signal generation module is configured to aggregate the neighborhood information of each of the first head entities to generate a first embedding guidance signal including the node representation of each of the first head entities.

[0060] The model training module is configured to acquire first spatiotemporal data for a first sub-region in the target region, and train the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

[0061] Fourthly, this application provides a spatiotemporal data generation device guided by a spatial knowledge graph. The device generates spatiotemporal data based on a target diffusion model trained using the method described in the first aspect. The spatiotemporal data generation device includes:

[0062] The second information determination module is configured to determine the neighborhood information of each second head entity belonging to the second sub-region in the spatial knowledge graph.

[0063] The second signal generation module is configured to aggregate the neighborhood information of each of the second head entities to generate a second embedded guiding signal including the node representation of each of the second head entities.

[0064] The spatiotemporal data generation module is configured to control the target diffusion model to generate second spatiotemporal data for the second sub-region based on preset noise data and the second embedding guidance signal. The generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction based on time step embedding and the second embedding guidance signal in time steps, and gradually removes the predicted noise from the preset noise data in time steps to obtain the second spatiotemporal data of the second sub-region. The time step embedding is obtained by position encoding the time step.

[0065] Fifthly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the methods described in the first and / or second aspects above.

[0066] Sixthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods described in the first and / or second aspects above.

[0067] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0068] This application provides a spatial knowledge graph-guided diffusion model training method and spatiotemporal data generation method, apparatus, equipment, and medium. Based on node information in a target region (including adjacent first and second sub-regions), a spatial knowledge graph of the target region is constructed. Neighborhood information is aggregated for head entities belonging to the first sub-region, thereby obtaining a first embedding guidance signal characterizing the node features in the first sub-region. The first spatiotemporal data of the first sub-region is used as input, and the first embedding guidance signal is used as guidance conditions to train the diffusion model. This enables the diffusion model to learn the ability to generate spatiotemporal data for a geographic space under the constraints of node features. Since adjacent geographic spaces can jointly construct a spatial knowledge graph due to geographical proximity, economic correlation, etc., the trained target diffusion model can generate spatiotemporal data of adjacent unknown geographic spaces (such as the second sub-region) under the guidance of node features in the spatial knowledge graph, achieving good generation results and thus realizing the task of generating spatiotemporal data for unknown geographic spaces. Attached Figure Description

[0069] Figure 1 is a flowchart illustrating a diffusion model training method based on spatial knowledge graphs according to an embodiment of this application.

[0070] Figure 2 is a flowchart illustrating a spatiotemporal data generation method based on spatial knowledge graphs according to an embodiment of this application.

[0071] Figure 3 is a t-SNE diagram showing the generation effect of the target diffusion model trained using a diffusion model training method based on spatial knowledge graphs provided in an embodiment of this application.

[0072] Figure 4 is a kernel density estimation diagram of the generation effect of the target diffusion model trained by a diffusion model training method based on spatial knowledge graph provided in an embodiment of this application.

[0073] Figure 5 is a schematic diagram of the functional modules of a diffusion model training device based on spatial knowledge graphs provided in an embodiment of this application;

[0074] Figure 6 is a schematic diagram of the functional modules of a spatiotemporal data generation device based on spatial knowledge graph guidance provided in an embodiment of this application;

[0075] Figure 7 is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0076] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0077] In implementing this application, the inventors discovered that the formation process of spatiotemporal data is essentially influenced by multiple spatial semantics, particularly the functional characteristics of each node and its topological information with neighboring nodes. This discovery prompted the inventors to model the spatiotemporal data generation task as a conditional generation problem based on node topological relationships when implementing this application.

[0078] In addition, the inventors also discovered during the implementation of this application that the following problems need to be solved in order to achieve accurate generation of spatiotemporal data:

[0079] (1) High-fidelity spatial conditions: The spatiotemporal data generation process requires the establishment of a spatial condition mechanism that can comprehensively encode the network topology characteristics. Existing methods mostly obtain spatial embeddings directly from the adjacency matrix of Euclidean space. Such representations are difficult to capture the inherent high-level structure and circular dependencies of real geospatial systems, which restricts the generation quality of spatiotemporal data, because the output fidelity of the encoded space is fundamentally limited by the expressive power of the spatial conditions.

[0080] (2) Heterogeneity perception generation: The spatiotemporal data of a geographic space (such as a city, a region, etc.) will show significant heterogeneity among different nodes. This is due to the specific functional patterns of the geographic location, that is, the stable change pattern of spatiotemporal data over time caused by the function of the node itself. For example, residential areas usually show a bi-peak traffic flow pattern in the morning and evening, while hospitals show a tri-peak traffic flow pattern that includes activities in the morning, noon and evening. These differentiated functional patterns lead to significant differences in the spatiotemporal data of each node in the geographic space. In other words, the spatiotemporal data of each node is affected not only by periodic fluctuations, but also by its own functional patterns. This requires the spatiotemporal data generation framework to explicitly model these differentiated functional patterns.

[0081] The following section introduces some relevant concepts in this application.

[0082] 1. Spatiotemporal data:

[0083] Spatiotemporal data refers to spatial data that has a time factor and changes over time, including flow data (such as pedestrian flow and / or vehicle flow) and speed data (such as vehicle speed). Flow data can include data in two dimensions, namely inflow data and outflow data. For example, pedestrian flow data can include pedestrian inflow data and pedestrian outflow data, and vehicle flow data can include vehicle inflow data and vehicle outflow data.

[0084] In one exemplary use case of this application, a trained target diffusion model can be used to generate zero-sample spatiotemporal data in the absence of spatiotemporal data in a geographic space (such as a region or city), thereby solving the problem of spatiotemporal data sparsity caused by limitations such as collection equipment and data privacy.

[0085] 2. Knowledge Graph:

[0086] A knowledge graph is a semantic network that describes the objective world using a graph structure (composed of nodes and edges). Essentially, it's a knowledge base describing entities and the relationships between them. Its core is building a semantic network through "entity-relation-entity" triples, revealing the connections between things. The first entity in a triple is called the head entity, and the second entity is called the tail entity. Both the head and tail entities in a triple come from the entity set defined by the knowledge graph, while the "relation" in a triple comes from the relation set defined by the knowledge graph.

[0087] In this application, a spatial knowledge graph is used to describe the network topology of geospatial space, that is, static spatial information without time factors. In the spatial knowledge graph, an entity is a node in geospatial space or a node category of a node in geospatial space. Entities in the spatial knowledge graph can have relationships such as adjacency (i.e., adjacent), proximity (i.e., close in distance), inclusion, and belonging.

[0088] 3. Diffusion Model:

[0089] The diffusion model is a generative model based on the diffusion concept of non-equilibrium thermodynamics. The learning process of the diffusion model is divided into a forward process (i.e., diffusion noise addition process) and a reverse process (i.e., diffusion noise removal process). In the forward process, the diffusion model gradually adds noise to the data. In the reverse process, the diffusion model gradually recovers the data from the noise. The reverse process gradually removes noise to restore the data through learning. The diffusion model has the ability to generate data from noise through the learning of the reverse process.

[0090] The embodiments of this application are described in detail below.

[0091] In an exemplary embodiment, as shown in Figure 1, a diffusion model training method guided by spatial knowledge graphs is provided. This method is executed by a computer device, specifically a terminal or server, or both. The terminal can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server can be a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0092] In this embodiment of the application, as shown in Figure 1, the diffusion model training method guided by spatial knowledge graph includes the following steps 101 to 104. Wherein:

[0093] Step 101: Collect node information for nodes in the target region and construct a spatial knowledge graph for the target region based on the node information; the target region includes adjacent first and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities, where the head entities belong to nodes in the target region.

[0094] In this embodiment, the target region is a geographic space, such as a city (for a city, its corresponding spatial knowledge graph can be specifically a city knowledge graph), a region, etc. The target region includes an adjacent first sub-region and a second sub-region. The spatiotemporal data of the first sub-region can be used to train the diffusion model to be trained, so that the diffusion model to be trained learns the ability to generate spatiotemporal data. The trained target diffusion model can be used to generate the spatiotemporal data of the second sub-region.

[0095] Nodes represent the spatial granularity of spatiotemporal data collection. For example, for road vehicle speed data (a type of spatiotemporal data), the node is "road"; for regional taxi traffic data (a type of spatiotemporal data), the node is "region". Optionally, nodes can include nodes with different spatial granularities such as regions, points of interest (POIs), and roads.

[0096] Spatial knowledge graphs are defined as multi-relation graphs:

[0097] ;

[0098] ;

[0099] in, , and These represent sets of entities, relations, and facts (facts being the basic units of knowledge expressed in a structured form, such as "entity-relationship-entity" or "entity-attribute-value" triples), and are located in spatiotemporal data. The nodes are included Set. Including head entity ,relation Tail-end entity The triplet.

[0100] For nodes in the target area, node information can be collected, including node location, containment relationships between nodes, adjacency relationships between nodes, and node category. In an optional implementation, this step can collect node information for nodes at various spatial granularities in the target area (e.g., nodes at different spatial granularities such as regions, points of interest, and roads).

[0101] In one example, the aforementioned target diffusion model needs to generate second spatiotemporal data for the second sub-region according to the target generation task during the inference process. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0102] For example, the target generation task can be a task for generating road vehicle speed data (a type of spatiotemporal data), where the target spatial granularity is the road, the target object is the vehicle, and the target data type of the spatiotemporal data to be generated is speed data.

[0103] For example, the target generation task can also be a generation task for interest point traffic data (a type of spatiotemporal data), where the target spatial granularity is interest point, the target object is people, and the target data type of the spatiotemporal data to be generated is traffic data.

[0104] For example, the target generation task can also be a generation task for regional traffic flow data (a type of spatiotemporal data), wherein the target spatial granularity is a region, the target object is a vehicle, and the target data type of the spatiotemporal data to be generated is traffic flow data.

[0105] Different diffusion models need to be trained for different node spatial granularities, different objects, and different spatiotemporal data types. This is because spatiotemporal data with different node spatial granularities, different objects, and different spatiotemporal data types have different characteristics and patterns. For example, the characteristics and patterns of road traffic flow and point-of-interest traffic flow are different, the characteristics and patterns of point-of-interest traffic flow and point-of-interest pedestrian flow are different, and the characteristics and patterns of road traffic flow and road speed are different.

[0106] By collecting node information from nodes at various spatial granularities (e.g., regions, points of interest, roads, etc.) within a target region, and constructing a spatial knowledge graph based on this information, a comprehensive representation of the semantics of diverse spatial environments can be achieved. Considering multiple generation tasks during the node information collection phase allows for the collection of node information at various spatial granularities, thus enabling the construction of a multi-granularity spatial knowledge graph. Using this spatial knowledge graph, multiple diffusion models can be trained to implement different generation tasks, eliminating the need to construct separate spatial knowledge graphs for each task. This allows the constructed spatial knowledge graph to be adapted for the training and inference of diffusion models for various generation tasks.

[0107] For example, by using regions within the target region as nodes (i.e., region granularity), collecting node information for these regions, and constructing a spatial knowledge graph based on this node information, this spatial knowledge graph can be used to generate spatiotemporal data (such as regional pedestrian flow data, regional vehicle flow data, regional vehicle speed data, etc.) based on region granularity in a second sub-region. Optionally, the entity set defined in the spatial knowledge graph constructed based on the region node information can include regional entities (i.e., regions), and the relationship set defined can include adjacency relationships between regions, relationships that a region contains a point of interest, and relationships that a region contains a road. The head and tail entities in the triples of this spatial knowledge graph both come from this entity set, and the relationships between entities in the triples (referred to as relationships) all come from the relationship set.

[0108] For example, by using points of interest (POIs) in the target region as nodes (i.e., POI granularity), collecting POI node information in the target region, and constructing a spatial knowledge graph based on the POI node information, this spatial knowledge graph can be used to generate spatiotemporal data (e.g., POI pedestrian flow data, POI vehicle flow data, etc.) in a second sub-region based on POI granularity. Optionally, the entity set defined in the spatial knowledge graph constructed based on the POI node information may include POI entity (i.e., POI) and POI category entity (i.e., POI category), and the defined relation set may include the relationship that (a) POI is adjacent to (a) road, and the relationship that (a) POI belongs to (a) POI category. The head entity and tail entity in the triples of this spatial knowledge graph both come from this entity set, and the relationships between entities in the triples all come from the relation set.

[0109] For example, by treating roads in the target region as nodes (i.e., road granularity), collecting node information of roads in the target region, and constructing a spatial knowledge graph based on the road node information, this spatial knowledge graph can be used to generate spatiotemporal data (such as road traffic flow data, road speed data, etc.) based on road granularity in a second sub-region. Optionally, the entity set defined in the spatial knowledge graph constructed based on the road node information may include road entities (i.e., roads) and road category entities (i.e., road categories), and the defined relation set may include adjacency relationships between roads and relationships that (a) road belongs to (a) road category. The head entity and tail entity in the triples of this spatial knowledge graph both come from this entity set, and the relationships between entities in the triples all come from the relation set.

[0110] For example, regions (i.e., region granularity), points of interest (i.e., point of interest granularity), and roads (i.e. road granularity) in the target region are used as nodes. Node information of regions, points of interest, and roads in the target region is collected. A spatial knowledge graph is constructed based on the node information of regions, points of interest, and roads. This spatial knowledge graph can be used not only to generate spatiotemporal data based on region granularity (e.g., regional pedestrian flow data, regional vehicle flow data, regional vehicle speed data, etc.) in the second sub-region, but also to generate spatiotemporal data based on point of interest granularity (e.g., point of interest pedestrian flow data, point of interest vehicle flow data, etc.) in the second sub-region, and also to generate spatiotemporal data based on road granularity (e.g., road vehicle flow data, road vehicle speed data, etc.) in the second sub-region. Optionally, the spatial knowledge graph constructed based on the node information of regions, the node information of points of interest, and the node information of roads can include the following entity sets: region entities (i.e., regions), points of interest entities (i.e., points of interest), points of interest category entities (i.e., points of interest categories), road entities (i.e., roads), and road category entities (i.e., road categories). The following relation sets can include the following relationships: adjacency between regions, adjacency between roads, (a) region contains (a) point of interest, (a) region contains (a) road, (a) point of interest is adjacent to (a) road, (a) point of interest belongs to (a) point of interest category, and (a) road belongs to (a) road category. The head entity and tail entity in the triples of this spatial knowledge graph both come from this entity set, and the relationships between entities in the triples all come from the relation set.

[0111] The more spatial granularity levels of nodes covered by a spatial knowledge graph, the more complete the description of the network topology of the target region, and the more accurate the generation results of the target diffusion model trained based on the spatial knowledge graph.

[0112] Step 102: For each first-head entity belonging to the first sub-region in the spatial knowledge graph, determine the neighborhood information of the first-head entity.

[0113] As previously mentioned, in one example, the target diffusion model is specifically used to implement the target generation task, which includes generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity. Optionally, the first head entity specifically includes the node corresponding to the target spatial granularity in the first sub-region, and the first spatiotemporal data specifically includes the spatiotemporal data of the target data type obtained for the target object in the first sub-region.

[0114] For example, if the target diffusion model is specifically used for the task of generating regional traffic flow data, where the target spatial granularity is a region, the target object is a vehicle, and the target data type of the spatiotemporal data to be generated is traffic flow data, then the first head entity specifically includes the regional nodes in the first sub-region, and the first spatiotemporal data specifically includes the traffic flow data obtained for vehicles in the first sub-region.

[0115] For example, if the target diffusion model is specifically used for the task of generating traffic data for points of interest, where the target spatial granularity is points of interest, the target object is people, and the target data type of the spatiotemporal data to be generated is traffic data, then the first head entity specifically includes the points of interest nodes in the first sub-region, and the first spatiotemporal data specifically includes the traffic data obtained for people in the first sub-region.

[0116] In this step, for each first head entity belonging to the first sub-region in the spatial knowledge graph, the neighborhood information of the first head entity can be determined, that is, the embedding mapping information of each neighboring entity of the first head entity. The neighboring entities of the first head entity are the tail entities that have inter-entity relationships with the first head entity.

[0117] In an optional implementation, this step can be achieved in the following ways:

[0118] Step 1021: Embedding mapping is performed on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity.

[0119] Traditional graph neural networks update entity representations by aggregating neighborhood information, but such operations are typically performed in Euclidean space, resulting in insufficient neighborhood information aggregation and difficulty in representing rich structural information such as high-order hierarchical structures and high-order cycles in geographic space. Therefore, this application integrates multiple geometric spaces (such as Euclidean space). Hyperbolic Space and hyperspherical space (etc.) are used to update the header entity representation.

[0120] In this step, embedding mapping can be performed on each neighboring entity of the first head entity through multiple different geometric spaces. That is, the embedding representation of each neighboring entity of the first head entity can be mapped through multiple different geometric spaces. The embedding representations of entities and relations in the spatial knowledge graph can be determined using embedding models (such as the TuckER model). Taking the TuckER model as an example, the input of the TuckER model is a triple, and the output is the embedding representation of the entity and relation in that triple. They are head entities ,relation Tail-end entity The embedding representation of the first head entity and the score of the triple are then mapped to each neighbor entity through multiple different geometric spaces. Different geometric spaces can map the same embedding representation to different embedding mapping results.

[0121] In an optional implementation, step 1021 can be implemented in the following ways:

[0122] Embedding mappings are performed on each neighboring entity of the first head entity using Euclidean space, hyperbolic space, and hyperspherical space respectively, to obtain the Euclidean space embedding mapping results, hyperbolic space embedding mapping results, and hyperspherical space embedding mapping results for each neighboring entity of the first head entity.

[0123] Suppose there exists a first-head entity The first entity neighborhood set The neighborhood set Represents the first entity All neighboring entities and neighboring entities Relationship Based on this first entity and its neighborhood set This section introduces expert methods (i.e., mapping methods) in Euclidean, hyperbolic, and hyperspherical spaces:

[0124] In Euclidean space, the expert (i.e., mapping mode) can be defined as: .

[0125] Experts (i.e., mapping methods) in hyperbolic space can be defined as: .

[0126] The expert (i.e., mapping method) in hyperspherical space can be defined as: .

[0127] in, and They are relationships and neighboring entities In Euclidean space Embedded representation in This represents Möbius scalar multiplication. ( )and ( ( ) represent hyperbolic spaces and hyperspherical space Trainable curvature parameters, This indicates that the curvature is The mapping from the tangent plane at the origin in space to the surface. The result is an embedding mapping in Euclidean space. The result of embedding the hyperbolic space is the mapping result. The result is an embedding map of the hyperspherical space.

[0128] Step 1022: Fuse and perform a second linear transformation on the multiple embedding mapping results corresponding to each neighboring entity of the first head entity to obtain the neighborhood information of the first head entity; the second linear transformation is implemented through the first linear transformation matrix to be trained, and the first linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain the (trained) first target linear transformation matrix.

[0129] In subsequent inference, the first target linear transformation matrix can be used to perform a second linear transformation on the multiple embedding mapping results corresponding to each neighbor entity of the second head entity in the second sub-region after fusion. The neighbor entities of the second head entity are the tail entities that have inter-entity relationships with the second head entity.

[0130] For example, for the first entity Euclidean space embedding mapping result for each neighboring entity Hyperbolic space embedding mapping results And hyperspherical space embedding mapping results By performing fusion (i.e., splicing) and a second linear transformation, the first head entity is obtained. Each neighbor entity's neighbor entity representation The first entity The neighborhood information includes the first entity The representation of the neighboring entities of all neighboring entities. Let be the first linear transformation matrix to be trained or the first target linear transformation matrix. Indicates embedded splicing, This indicates that the curvature is The mapping from the surface at the origin in space to the tangent plane, where, , and They can be spliced ​​in any order. After fusion and a second linear transformation, it becomes... It remains in Euclidean space to ensure compatibility with diffusion models, thereby enabling spatial semantics to effectively guide the generation of spatiotemporal data.

[0131] In this embodiment of the application, in order to address the problem of insufficient output fidelity of the coding space in the prior art, by using multiple different geometric spaces to embed and map the neighboring entities of the first entity, the expressive power of multiple different geometric spaces can be combined to capture the high-dimensional features and relationships of the real geospatial system, thereby improving the output fidelity of the coding space and thus enhancing the generation quality of spatiotemporal data.

[0132] Step 103: Aggregate the neighborhood information of each first head entity to generate a first embedding guidance signal that includes the node representation of each first head entity.

[0133] In this step, a first embedding guide signal, including the node representation of each first head entity, can be generated by aggregating neighborhood information of each first head entity.

[0134] In an optional implementation, this step can be achieved in the following ways:

[0135] The neighborhood information of each first-head entity is aggregated by the graph convolutional network (GCN) to obtain the node representation of each first-head entity; the graph convolutional network to be trained is jointly trained with the diffusion model to obtain the target graph convolutional network.

[0136] The node representations of each first-head entity are merged to obtain the first embedding guidance signal.

[0137] In the subsequent inference process, the target graph neural network can aggregate the neighborhood information of each first-head entity in the second sub-region to obtain the node representation of each second-head entity.

[0138] In this implementation, the GCN to be trained can aggregate the neighborhood information of each first head entity using the following formula:

[0139] ;

[0140] in, For the first entity The node representation, For activation function, For the first entity The number of all neighboring nodes.

[0141] Each first entity Node representation The first embedded guiding signal is obtained by fusion. for{ ; ; …; }, where N is the number of the first entity.

[0142] Step 104: Obtain first spatiotemporal data for the first sub-region in the target region, and train the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

[0143] In this step, the corresponding first spatiotemporal sub-data can be obtained for each node in the first sub-region (corresponding to each first head entity). The first spatiotemporal sub-data corresponding to each node in the first sub-region together constitute the first spatiotemporal data of the first sub-region. The first spatiotemporal data is used as input and the first embedding guidance signal is used as guidance condition to train the diffusion model to be trained.

[0144] The first spatiotemporal data is a three-dimensional array of size N×T×D, where N is the number of first-head entities in the first sub-region; T is the time series length of the first spatiotemporal data, which is the number of time points contained within a certain time period (e.g., one year, half a year, three months, etc.), and T is a natural number; D is the number of features of the first spatiotemporal data, and D is a natural number. For example, if the data type of the first spatiotemporal data is flow data, specifically including inflow and outflow data, then the number of features D = 2. As another example, if the data type of the first spatiotemporal data is speed data, then the number of features D = 1.

[0145] The first spatiotemporal data can be represented as: , ,in, This represents the spatiotemporal data corresponding to time point t1 in the first spatiotemporal sub-data of each first-head entity. , This represents the spatiotemporal data corresponding to time point t2 in the first spatiotemporal sub-data of each first-head entity. And so on.

[0146] In an optional implementation, to align the dimension of the first spatiotemporal data with the input dimension of the diffusion model to be trained, delayed embedding can be used to convert the first spatiotemporal data into data aligned with the input dimension of the diffusion model to be trained, and then input it along with the first embedding guidance signal into the diffusion model to be trained for training. Specifically:

[0147] Based on sliding window and matrix zero-padding, the first spatiotemporal data is transformed into a first delayed embedding matrix aligned with the input dimension of the diffusion model to be trained;

[0148] The target diffusion model is obtained by training the diffusion model to be trained based on the first delayed embedding matrix and the first embedding guidance signal.

[0149] In this implementation, delayed embedding can be used to convert the first spatiotemporal data into a matrix composed of data segments within a sliding window. Assuming the sliding window step size is U and the window size is V, the first spatiotemporal data can be processed through W sliding window iterations. Data interception can extract the first spatiotemporal data. Convert to matrix :

[0150] ;

[0151] in, To ensure the matrix Consistency with the input dimension of the diffusion model to be trained, i.e., ensuring The matrix will be modified when necessary. Perform zero-padding, that is, in the matrix Fill with zero-value elements.

[0152] In this step, the diffusion model to be trained takes the first spatiotemporal data as input and the first embedding guidance signal as the guiding condition for training. The diffusion model gradually adds noise to the first spatiotemporal data through a diffusion noise addition process. Then, guided by the first embedding guidance signal, it predicts each added noise through a diffusion denoising process, and gradually removes the predicted noise from the noise addition results, thereby restoring the first spatiotemporal data. By minimizing the difference between the predicted noise and the actual added noise, the diffusion model learns how to correctly predict noise, thus removing the predicted noise from the noisy data to generate the data.

[0153] In an optional embodiment of this application, the diffusion model training method guided by spatial knowledge graph may further include the following steps:

[0154] Step 105: Obtain the first inherent trend feature by mapping the first embedded guiding signal; the first inherent trend feature characterizes the functional mode of each first head entity.

[0155] As mentioned earlier, the spatiotemporal data of a geographic space (such as a city or a region) exhibits significant heterogeneity across different nodes. This is due to the specific functional patterns of the geographic location. These differentiated functional patterns lead to significant differences in the spatiotemporal data of each node in the geographic space. In other words, in addition to being affected by periodic fluctuations, the spatiotemporal data of each node is also affected by its own functional patterns. This requires the spatiotemporal data generation framework to explicitly model these differentiated functional patterns.

[0156] Therefore, in this step, the first inherent trend feature can be obtained by mapping the first embedded guiding signal. The first inherent trend feature can characterize the functional mode of each first head entity. Then, the first inherent trend feature can be introduced into the diffusion model, so that the diffusion model can consider the influence of the functional mode of each first head entity during the learning process, thereby improving the accuracy of spatiotemporal data generation.

[0157] In an optional implementation, the time series length of the first spatiotemporal data is T, and the number of features of the first spatiotemporal data is D. Step 105 can be implemented through the following steps:

[0158] When D=1, for the node representation of each first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the mapping module to be trained, and the first inherent trend feature is obtained.

[0159] When D > 1, for the node representation of each first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the mapping module to be trained to obtain the initial inherent trend feature. The same D initial inherent trend features are concatenated to obtain the first inherent trend feature.

[0160] The target mapping module is obtained by jointly training the mapping module to be trained and the diffusion model to be trained.

[0161] In subsequent inference, the target mapping module can map the node representation of each second head entity in the second embedding guidance signal (which will be introduced later) into a feature vector of length T.

[0162] In the above implementation, firstly, the first embedded guiding signal Each first entity Node representation Each feature is mapped to a feature vector of length T through the training mapping module. For N first-head entities, N feature vectors of length T can be obtained, thus forming an initial inherent trend feature of size N×T (or represented as N×T×1). The initial inherent trend feature can characterize the functional mode of each first-head entity. In this embodiment, it is necessary to map the functional mode of each first-head entity to each feature in the first spatiotemporal data. This is because each feature in the first spatiotemporal data is affected by the functional mode of the first-head entity, thus showing the data change pattern caused by the function of the first-head entity. For example, flow data can include inflow data and outflow data, so D=2. Both inflow data and outflow data are affected by the functional mode of the first-head entity, showing the inflow data change pattern and outflow data change pattern caused by the functional mode of the first-head entity. Therefore, when D>1, the same D initial inherent trend features (of size N×T) can be concatenated to obtain a first inherent trend feature of size N×T×D. (D>1), to realize the functional mode of individually mapping each feature of the first spatiotemporal data (size N×T×D) to each first head entity.

[0163] When D=1, the first spatiotemporal data (of size N×T, or represented as N×T×1) contains only one type of spatiotemporal data (e.g., only inflow data, or only outflow data). Therefore, the initial inherent trend feature is also of size N×T (or represented as N×T×1). Thus, mapping this single feature of the first spatiotemporal data (of size N×T×1) to the functional mode of each first-head entity is sufficient; there is no need to concatenate multiple initial inherent trend features, as the initial inherent trend feature is the first inherent trend feature. (D=1).

[0164] In the first embedding guide signal, node features are represented as high-dimensional vectors. These vectors (i.e., the node representations mentioned above) are designed to capture the intrinsic characteristics of the nodes. When these node features are mapped back to the time series length of the spatiotemporal data, it essentially aligns the static node representations with the time dimension, thereby inferring the change pattern of the nodes over time. This change pattern refers to the overall upward or downward direction of node features over a long time span. It reflects the fundamental direction of change in the spatiotemporal data caused by the node functional patterns, and is independent of short-term periodic and random fluctuations.

[0165] Optionally, the mapping module to be trained may include a multilayer perceptron (MLP) and / or convolutional layers. The mapping module to be trained can be any functional module capable of changing the vector dimension (e.g., converting a 128-dimensional vector to a 256-dimensional vector), and this embodiment is not intended to limit it in this way.

[0166] Taking a multilayer perceptron as an example, the multilayer perceptron may include:

[0167] The first fully connected layer is used to map the node representation from a 128-dimensional (i.e., embedding dimension) space to a 64-dimensional hidden space;

[0168] The activation layer uses the ReLU activation function to perform nonlinear feature transformation on the output of the first fully connected layer.

[0169] The second fully connected layer further maps the intermediate representation output by the activation layer to the output dimension, which is consistent with the time series length T of the first spatiotemporal data.

[0170] Accordingly, step 104 can be implemented in the following ways, including:

[0171] The diffusion model to be trained is trained based on the first spatiotemporal data, the first embedding guidance signal, and the first inherent trend feature to obtain the target diffusion model. The training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process. In the diffusion noise addition process, noise is sampled based on the target noise distribution in time steps, and the sampled noise is added to the first spatiotemporal data in time steps to obtain the noise addition result. In the first diffusion denoising process, noise is predicted based on the time step embedding and the first embedding guidance signal in time steps, and the predicted noise is removed from the noise addition result in time steps. The target noise distribution is a noise distribution that introduces the first inherent trend feature to affect the noise mean. The time step embedding is obtained by encoding the position of the time step.

[0172] If the first spatiotemporal data is a three-dimensional array of size N×T×D, then the noise addition result here is also a three-dimensional array of size N×T×D.

[0173] The training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process. In the above implementation, during the training process of the diffusion model to be trained, the influence of a first inherent trend feature can be introduced during the diffusion noise addition process, thereby affecting the noise distribution of the noise added to the first spatiotemporal data. In this way, the diffusion model to be trained can learn how to predict the noise introduced by the first inherent trend feature during the first diffusion denoising process, and then restore the first spatiotemporal data containing the first inherent trend feature from the noise addition result of the diffusion noise addition process.

[0174] In existing technologies, the noise added / removed in the diffusion model all follow a certain order. That is, a standard Gaussian distribution, where the noise mean of the added / removed noise is . The noise variance of the added / removed noise is Since different nodes have different inherent trend characteristics, the noise distribution of the noise added / removed for different nodes should also be different, rather than all being the same standard Gaussian distribution.

[0175] Therefore, in response to the problem that the existing spatiotemporal data generation framework cannot explicitly model the differentiated functional modes between nodes, the embodiments of this application introduce the influence of a first inherent trend feature into the noise distribution of the noise added during the diffusion noise process. This enables the first diffusion denoising process to learn how to predict noise on this basis, thereby enabling the generated spatiotemporal data to reflect the influence of the functional modes of each node and conform to the true characteristics of the node spatiotemporal data.

[0176] Optionally, any time step is denoted as time step i. For the diffusion noise addition process, the noise distribution of the sampling noise added to the first spatiotemporal data at time step i is the target noise distribution. for:

[0177] ;

[0178] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after adding sampling noise to the first spatiotemporal data at time step i. This represents the first spatiotemporal data. Indicates the first embedded guide signal. Indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the noise mean of the noise added to the first spatiotemporal data at time step i. This represents the noise variance of the noise added to the first spatiotemporal data at time step i.

[0179] Compared to the standard Gaussian distribution The aforementioned target noise distribution is a non-standard Gaussian distribution; the noise mean is no longer 0, but... The first inherent trend feature is considered in the noise mean. The influence of this. In the diffusion noise process... During the noise addition process, the first spatiotemporal data Scaled to At the same time, an offset term was introduced. The noise variance is Noise variance varies It increases with the increase of, when and At that time, the noise mean approaches That is, the noise added at each time step in the diffusion noise process is... It is centered on a distribution, rather than centered on 0.

[0180] Accordingly, for the first diffusion denoising process, the noise distribution of the predicted noise removed from the denoising result at time step i is... for:

[0181] ;

[0182] in:

[0183] ;

[0184] ;

[0185] ;

[0186] ;

[0187] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after removing prediction noise from the noisy result at time step i-1. This represents the spatiotemporal data obtained at time step i after removing the prediction noise from the noisy result. This represents the first spatiotemporal data. Indicates the first embedded guide signal. Indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the decay parameter corresponding to time step i-1. This represents the noise mean that is removed from the noise-adding result at time step i. This represents the noise variance of the noise removed from the noise-adding result at time step i.

[0188] The noise mean of the removed noise middle, coefficient It reflects the degree to which real data (i.e., first-time spatiotemporal data) is preserved during the diffusion and noise-adding process. coefficient It reflects the degree to which the current noise state influences the previous noise estimate; coefficient Then, to compensate for the bias introduced cumulatively during the diffusion and noise addition process, the coefficient... middle The coefficient represents the cumulative deviation generated by the diffusion noise addition process as time step i changes. middle The part reflects the reverse effect of the current time step and the historical cumulative offset on the expectation.

[0189] The noise distribution of the predicted noise can be derived by back-calculating the noise distribution added each time step using Bayes' theorem, based on the first spatiotemporal data and the noise addition results at each time step. For details, refer to existing diffusion model formula derivation techniques; this application will not elaborate further. The noise distribution of the noise removed in each step of the first diffusion denoising process can be derived using Bayes' theorem. However, the noise removed in each step itself needs to be predicted using the diffusion model. Through training, the diffusion model can make the predicted noise at each step approximate the actual noise added at each step.

[0190] The first diffusion denoising process of the diffusion model to be trained is based on the noise addition result. Time step i and the first embedded guidance signal Output the prediction noise corresponding to time step i. :

[0191]

[0192] in, and Having the same size These are the learnable parameters in the diffusion model to be trained.

[0193] In each step of the first diffusion denoising process, the time step embedding obtained by encoding time step i is incorporated through a series of operations. First embedded guiding signal It is also integrated through the same series of operations. (See the processing operations of the affine transformation layer, activation function layer and feature fusion layer below for details). The two conditional signals, time step embedding and first embedding guide signal, are injected in parallel to maintain their respective information independence.

[0194] Optionally, the diffusion model to be trained can be an Equivariant Diffusion Model (EDM). In an exemplary embodiment, the Equivariant Diffusion Model provided in the article "T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022." can be used as the basic architecture, and each U-Net residual block in this architecture can be improved to incorporate the embedded guiding signal. The remaining processing procedures remain unchanged except for the improvements. The improvements to the U-Net residual blocks include: adding affine transformation and nonlinear processing for the embedded guiding signal in the U-Net residual blocks, and fusing the features obtained by the time-step embedding after affine transformation and nonlinear processing with the features obtained by the embedded guiding signal after affine transformation and nonlinear processing, as detailed below.

[0195] The equivariant diffusion model includes multiple noise prediction units corresponding to a time step, connected sequentially. Each noise prediction unit predicts the noise corresponding to a given time step based on the time step embedding and a first embedding guide signal (a second embedding guide signal is used in subsequent inference). It then removes the predicted noise from either the noise-adding result output from the diffusion noise-adding process (using preset noise data in subsequent inference) or the denoising result output from the previous noise prediction unit. In other words, different noise prediction units use different time steps. For each noise prediction unit, the noise-adding result output from the diffusion noise-adding process (using preset noise data in subsequent inference), the time step embedding of the first time step, and the first embedding guide signal (using the second embedding guide signal in subsequent inference) are used as inputs. The output of the first noise prediction unit (i.e., its denoising result), the time step embedding of the second time step, and the first embedding guide signal (using the second embedding guide signal in subsequent inference) are used as inputs to the second noise prediction unit, and so on.

[0196] The noise prediction unit comprises a first outer convolutional layer, multiple U-Net residual blocks (connected sequentially), and a second outer convolutional layer connected in sequence. The noise addition result from the diffusion noise addition process or the denoising result from the previous noise prediction unit is used by the first outer convolutional layer to generate an intermediate feature. This intermediate feature, the time step embedding corresponding to the time step of the noise prediction unit, and the first embedding guide signal together serve as the input to the first U-Net residual block in the noise prediction unit. The output of the second U-Net residual block in the noise prediction unit, the time step embedding corresponding to the time step of the noise prediction unit, and the first embedding guide signal together serve as the input to the third U-Net residual block in the noise prediction unit, and so on. Each U-Net residual block in the same noise prediction unit uses the same time step. The output of the last U-Net residual block in the noise prediction unit serves as the input to the second outer convolutional layer, and the output of the second outer convolutional layer is the denoising result of the noise prediction unit.

[0197] The improved U-Net residual block in this embodiment includes a normalization layer, an affine transformation layer, an activation function layer, a feature fusion layer, and an inner convolution layer connected in sequence.

[0198] The affine transformation layer is used to: perform a first linear transformation on the time step embedding corresponding to time step i of the noise prediction unit to obtain a first affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the first affine transformation parameter to obtain a first intermediate feature; and perform a first linear transformation on the first embedding guidance signal to obtain a second affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the second affine transformation parameter to obtain a second intermediate feature.

[0199] The activation function layer is used to: perform a nonlinear transformation on the first intermediate feature using a smooth nonlinear activation function to obtain a third intermediate feature, and perform a nonlinear transformation on the second intermediate feature using the smooth nonlinear activation function to obtain a fourth intermediate feature.

[0200] The feature fusion layer is used to embed the third and fourth intermediate features to obtain fused features, which are then input into the inner convolutional layer. The output features of the inner convolutional layer are then input into the next U-Net residual block.

[0201] In the affine transformation layer, the time step embedding of the U-Net residual block corresponding to the time step is subjected to a first linear transformation by the second linear transformation matrix to be trained, and the first embedding guidance signal is subjected to a first linear transformation by the third linear transformation matrix to be trained. The second linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain the second target linear transformation matrix, and the third linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain the third target linear transformation matrix.

[0202] In subsequent inference, the second target linear transformation matrix can be used to perform a first linear transformation on the time step embedding corresponding to the time step of the U-Net residual block. Similarly, in subsequent inference, the third target linear transformation matrix can be used to perform a first linear transformation on the second embedding guidance signal to obtain the second affine transformation parameters. Optionally, the second and third target linear transformation matrices can be the same matrix or two identical matrices; this embodiment does not limit this.

[0203] For example, in the affine transformation layer of each U-Net residual block, both the time-step embedding (assuming the size of the time-step embedding is N×T×D=128-dimensional) and the first embedding guidance signal (assuming the size of the first embedding guidance signal is N×T×D=128-dimensional) can be mapped to a 2×128-dimensional vector through a linear transformation. In the 2×128-dimensional vector obtained by mapping the time-step embedding, the first 128-dimensional vector is used as the scaling parameter scale1, and the last 128-dimensional vector is used as the offset parameter shift1. Then, an affine transformation is performed on the output feature h of the normalized layer to obtain feature h1=scale1. Then, in the activation function layer, a nonlinear transformation is performed on feature h1 using the smoothing nonlinear activation function silu, outputting feature silu(h1). Similarly, in the 2×128-dimensional vector obtained by mapping the first embedding guide signal, the first 128-dimensional vector is used as the scaling parameter scale2, and the last 128-dimensional vector is used as the offset parameter shift2. Then, an affine transformation is performed on the output feature h of the normalization layer to obtain feature h2=scale2. Then, in the activation function layer, the feature h2 is nonlinearly transformed by the smooth nonlinear activation function silu, and the output feature silu(h2) is generated. Then, in the feature fusion layer, the features silu(h1) and silu(h2) are fused by addition, and the fused feature silu(h1) + silu(h2) is input into the inner convolution layer to continue the subsequent processing calculation of the isovariant diffusion model.

[0204] The structure and processing before the normalization layer, and the structure and processing after the convolutional layer, can refer to the structure and processing in the existing equivariant diffusion model. That is, in this embodiment, the affine transformation layer and activation function layer in the U-Net residual block of EDM are modified, and a feature fusion layer is added.

[0205] In this embodiment, linear and affine transformations of the first embedding guide signal can be added to the affine transformation layer in the U-Net residual block of the equal diffusion model, and nonlinear transformations of the first embedding guide signal can be added to the activation function layer in the U-Net residual block of the equal diffusion model. The linear, affine, and nonlinear transformations of the first embedding guide signal are performed independently of the linear, affine, and nonlinear transformations of the time step embedding, so as to realize the parallel injection of the time step embedding and the first embedding guide signal into the U-Net residual block, thereby maintaining their respective information independence.

[0206] The data processing method for a diffusion model based on a spatial knowledge graph provided in this application constructs a spatial knowledge graph of the target region (including adjacent first and second sub-regions) based on node information. Neighborhood information is aggregated for head entities belonging to the first sub-region to obtain a first embedding guidance signal characterizing the node features in the first sub-region. The first spatiotemporal data of the first sub-region is used as input, and the first embedding guidance signal is used as guidance conditions to train the diffusion model. This enables the diffusion model to learn the ability to generate spatiotemporal data of a geographic space under the constraint of node features. Since adjacent geographic spaces can jointly construct a spatial knowledge graph due to geographical proximity, economic correlation, etc., the trained target diffusion model can generate spatiotemporal data of adjacent unknown geographic spaces (such as the second sub-region) under the guidance of node features in the spatial knowledge graph, achieving good generation results and thus realizing the task of generating spatiotemporal data for unknown geographic spaces.

[0207] In an exemplary embodiment, as shown in Figure 2, a spatiotemporal data generation method guided by a spatial knowledge graph is provided. This method is executed by a computer device, specifically a terminal or server, or both. The terminal can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server can be a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0208] In this embodiment of the application, the spatiotemporal data generation method guided by spatial knowledge graph generates spatiotemporal data based on the target diffusion model trained by the aforementioned spatiotemporal diffusion model training method guided by spatial knowledge graph, as shown in Figure 2. The spatiotemporal data generation method guided by spatial knowledge graph includes steps 201 to 203. Wherein:

[0209] Step 201: For each second-head entity belonging to the second sub-region in the spatial knowledge graph, determine the neighborhood information of the second-head entity.

[0210] The method for determining the neighborhood information of the second head entity can refer to the method for determining the neighborhood information of the first head entity in step 102 above.

[0211] Step 202: Aggregate the neighborhood information of each second head entity to generate the second embedding guidance signal corresponding to the second sub-region.

[0212] The method of aggregating the neighborhood information of each second head entity to generate the second embedding guidance signal corresponding to the second sub-region can refer to the method of aggregating the neighborhood information of each first head entity to generate the first embedding guidance signal corresponding to the first sub-region in step 103 above.

[0213] Step 203: The control target diffusion model generates second spatiotemporal data for the second sub-region based on preset noise data and second embedding guidance signal. The generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction based on time step embedding and second embedding guidance signal in time steps, and gradually removes the predicted noise from the preset noise data in time steps to obtain the second spatiotemporal data of the second sub-region. The time step embedding is obtained by position encoding the time step.

[0214] In this step, the trained target diffusion model does not undergo a diffusion noise addition process during inference; instead, it only undergoes a diffusion denoising process starting from preset noise data (i.e., the second diffusion denoising process). The preset noise data can be, for example, Gaussian noise data. If the second sub-region contains M second-head entities, the preset noise data is a three-dimensional array of size M×T×D, and the generated second spatiotemporal data is also a three-dimensional array of size M×T×D. During the second diffusion denoising process, the trained model parameters can be utilized. Noise prediction is performed step by step. Each step of the noise prediction is guided by the time step embedding and the second embedding guidance signal, and the predicted noise is gradually removed from the preset noise data. For details, please refer to the first diffusion denoising process in the previous text.

[0215] The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region based on the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0216] Accordingly, the second head entity specifically includes nodes in the second sub-region that correspond to the target spatial granularity, and the second spatiotemporal data specifically includes spatiotemporal data of the target data type generated for the target object in the second sub-region.

[0217] For example, if the target spatial granularity is a region, the target object is a vehicle, and the target data type of the spatiotemporal data to be generated is traffic data, then the target diffusion model can be used to generate traffic data for each region in the second sub-region.

[0218] For example, if the target spatial granularity is points of interest, the target object is people, and the target data type of the spatiotemporal data to be generated is traffic data, then the target diffusion model can be specifically used to generate traffic data for each point of interest in the second sub-region.

[0219] In an optional embodiment, the Beijing Xicheng District taxi traffic dataset (hereinafter referred to as the Xicheng Traffic Dataset) and the New York City bicycle traffic dataset (hereinafter referred to as the New York Traffic Dataset) are used to train and test the diffusion model.

[0220] Specifically, for the New York City bicycle traffic dataset, bicycle traffic statistics spanning one year were extracted, dividing New York City into 99 regions. For the Beijing Xicheng District taxi traffic dataset, taxi traffic statistics spanning one year were extracted, dividing Beijing Xicheng District into 235 regions. For both datasets, the interval between each data point is 30 minutes. For each weekday, 48 timestamps are used as a sample. The training and test sets are divided according to the latitude of each region's center (either arranged from low to high or high to low), with a 6:4 ratio of training to test set nodes. A total of 200 time steps are set, and the spatial knowledge graph embedding (including entity embeddings and relation embeddings) has a length of 128.

[0221] This embodiment generates a sample set of nodes with the same amount of data as the real spatiotemporal data. The Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are calculated by averaging all samples at each test node. Furthermore, Maximum Mean Discrepancy (MMD) and Jensen-Shannon Divergence (JSD) are used to measure the degree of difference between the generated data distribution and the real data distribution. The smaller the values ​​of these four indicators, the higher the accuracy of the diffusion model and the closer it is to the real data distribution.

[0222] The generation results of the above embodiments are compared with those of existing technologies. Regional traffic data are generated on the same dataset, and the comparison results are shown in Tables 1 and 2 below. This embodiment compares twelve existing spatiotemporal data generation methods.

[0223] The first type is TimeGAN, which is a time series generation method based on generative adversarial networks;

[0224] The second type is TimeVAE, which is a time series generation method based on variational autoencoders;

[0225] The third type is DiffWave, which is a neural vocoder and waveform synthesizer based on a diffusion model.

[0226] The fourth type is DiffWave-con, which is a conditional generation variant of DiffWave;

[0227] The fifth type is DiffTime, which is a time series generation method based on constructing a diffusion model using a constrained optimization framework;

[0228] The sixth type is DiffTime-con, which is a conditional generation variant of DiffTime;

[0229] The seventh type is DiffusionTS, which is a diffusion model built by decoupling the encoder-decoder transformer of time representation;

[0230] The eighth type is DiffusionTS-con, which is a conditional generation variant of DiffusionTS;

[0231] The ninth type is ImagenTime, which uses a diffusion model to convert sequences into images and is suitable for generating time series of different lengths.

[0232] The tenth is ImagenTime-con, which is a conditional generation variant of ImagenTime;

[0233] The eleventh type is SDformer, which is a discrete Transformer based on the similarity vector method to learn high-quality discrete label representations of time series for time series generation.

[0234] The twelfth type, KSTDiff, is a knowledge-guided conditional diffusion model used for the controlled generation of mobile traffic data.

[0235] KGDiff (Knowledge Graph Diffusion model): The target diffusion model trained in the embodiments of this application.

[0236] Table 1. Comparison of Taxi Traffic Dataset Generation Results in Xicheng District, Beijing

[0237]

[0238] Table 2. Comparison of New York City bicycle traffic dataset generation results

[0239]

[0240] Furthermore, a comparative visualization analysis of the distribution of generated data and real data was conducted on the taxi traffic dataset of Xicheng District, Beijing, and the bicycle traffic dataset of New York City. As shown in Figures 3 and 4, the high-dimensional data was first reduced to a two-dimensional space using the t-SNE (t-Distributed Stochastic Neighbor Embedding) method, and then the probability distribution was visualized through kernel density estimation. Comparative analysis with DiffWave and DiffusionTS shows that the spatiotemporal data generated by the diffusion model trained in this embodiment has a significantly higher degree of consistency with the distribution of real data. Existing technologies struggle to generate spatiotemporal data that conforms to node characteristics under spatial constraints, while the diffusion model trained in this embodiment demonstrates excellent spatial semantic understanding capabilities, and the distribution of its generated data maintains a high degree of consistency with spatial semantic features.

[0241] The test results above show that the diffusion model trained by the spatial knowledge graph-guided diffusion model training method provided in this application outperforms the diffusion models in the prior art.

[0242] Compared with the prior art, the embodiments provided in this application have the following technical effects:

[0243] 1) Enhanced spatial semantics encoding: By embedding multiple spatial knowledge graphs, such as Euclidean space, hyperbolic space and hyperspherical space, the limitations of traditional spatial encoding (Euclidean spatial encoding) are overcome, and the multi-scale relationships of geospatial space can be captured more accurately.

[0244] 2) Adaptive modeling of node heterogeneity: By extracting the inherent trend features of nodes and dynamically injecting them into the acoustic distribution of the diffusion model, the diffusion model can adapt to different scenarios (such as residential areas and commercial areas).

[0245] 3) Zero-sample generation capability: By leveraging the semantic correlation between spatial regions, spatiotemporal data generation is achieved under the condition of no historical data in the target region, solving the problem of spatiotemporal data generation in sparse data regions.

[0246] 4) Superior model performance: Quantitatively demonstrated through experimental data (such as MAE, RMSE, MMD, JSD), the target diffusion model provided in this application embodiment is significantly superior to a variety of advanced existing models in multiple indicators.

[0247] The target diffusion model KGDiff trained in this embodiment overcomes the limitations of traditional spatial coding (Euclidean spatial coding) by embedding multi-spatial knowledge graphs. It combines multi-spatial expressive power to more accurately capture multi-scale relationships in geographic space, improving the output fidelity of the encoded space. By extracting the inherent trend features of nodes and dynamically injecting them into the acoustic distribution of the target diffusion model, the model gains adaptability to different scenarios (such as residential areas and commercial areas), thereby generating spatiotemporal data that conforms to the functional patterns of nodes. Thus, the target diffusion model KGDiff significantly outperforms many advanced existing models on multiple metrics (MAE, RMSE, MMD, JSD) for the Beijing Xicheng District taxi traffic dataset and the New York City bicycle traffic dataset.

[0248] The spatiotemporal data generation method based on spatial knowledge graph guidance provided in this application can generate spatiotemporal data through a target diffusion model. During the training process of this target diffusion model, a spatial knowledge graph of the target region is constructed based on node information in the target region (including adjacent first and second sub-regions). Neighborhood information is aggregated for head entities belonging to the first sub-region, thereby obtaining a first embedding guidance signal representing the node characteristics in the first sub-region. The first spatiotemporal data of the first sub-region is used as input, and the first embedding guidance signal is used as guidance conditions to train the diffusion model, enabling it to learn the ability to generate spatiotemporal data for a geographic space under the constraint of node characteristics. Since adjacent geographic spaces can jointly construct a spatial knowledge graph due to geographical proximity, economic correlation, etc., the trained target diffusion model can generate spatiotemporal data of adjacent unknown geographic spaces (such as the second sub-region) under the guidance of node characteristics in the spatial knowledge graph, achieving good generation results and thus realizing the task of generating spatiotemporal data for unknown geographic spaces.

[0249] Based on the same inventive concept, this application also provides a spatial knowledge graph-guided diffusion model training device for implementing the above-mentioned spatial knowledge graph-guided diffusion model training method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the spatial knowledge graph-guided diffusion model training device provided below can be found in the limitations of the spatial knowledge graph-guided diffusion model training method described above, and will not be repeated here.

[0250] In an exemplary embodiment, as shown in FIG5, a diffusion model training device guided by spatial knowledge graph is provided. The diffusion model training device guided by spatial knowledge graph includes:

[0251] The data collection and construction module is configured to collect node information for nodes in the target region and construct a spatial knowledge graph for the target region based on the node information. The target region includes adjacent first and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities. The head entities belong to nodes in the target region.

[0252] The first information determination module is configured to determine the neighborhood information of each first head entity belonging to the first sub-region in the spatial knowledge graph.

[0253] The first signal generation module is configured to aggregate the neighborhood information of each first head entity and generate a first embedding guidance signal including the node representation of each first head entity.

[0254] The model training module is configured to acquire first spatiotemporal data for the first sub-region in the target region, and train the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

[0255] Optionally, in some embodiments of this application, the diffusion model training device guided by spatial knowledge graph is further used for:

[0256] The first inherent trend feature is obtained by mapping the first embedded guiding signal; the first inherent trend feature represents the functional mode of all first head entities;

[0257] The target diffusion model is obtained by training the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal, including:

[0258] The diffusion model to be trained is trained based on the first spatiotemporal data, the first embedding guidance signal, and the first inherent trend feature to obtain the target diffusion model. The training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process. In the diffusion noise addition process, noise is sampled based on the target noise distribution in time steps, and the sampled noise is added to the first spatiotemporal data in time steps to obtain the noise addition result. In the first diffusion denoising process, noise is predicted based on the time step embedding and the first embedding guidance signal in time steps, and the predicted noise is removed from the noise addition result in time steps. The target noise distribution is a noise distribution that introduces the first inherent trend feature to affect the noise mean. The time step embedding is obtained by encoding the position of the time step.

[0259] Optionally, in some embodiments of this application, the time series length of the first spatiotemporal data is T, the number of features of the first spatiotemporal data is D, and the first inherent trend feature is obtained by mapping according to the first embedding guidance signal, including:

[0260] When D=1, for the node representation of each first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the mapping module to be trained, and the first inherent trend feature is obtained.

[0261] When D > 1, for the node representation of each first head entity in the first embedded guidance signal, the node representation is mapped to a feature vector of length T by the mapping module to be trained to obtain the initial inherent trend feature. The same D initial inherent trend features are concatenated to obtain the first inherent trend feature.

[0262] The target mapping module is obtained by jointly training the mapping module to be trained and the diffusion model to be trained.

[0263] Optionally, in some embodiments of this application, the mapping module to be trained includes a multilayer perceptron and / or a convolutional layer.

[0264] Optionally, in some embodiments of this application, any time step is denoted as time step i. For the diffusion noise addition process, the noise distribution of the sampling noise added to the first spatiotemporal data at time step i is the target noise distribution. for:

[0265] ;

[0266] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after adding sampling noise to the first spatiotemporal data at time step i. This represents the first spatiotemporal data. Indicates the first embedded guide signal. Indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the noise mean of the noise added to the first spatiotemporal data at time step i. This represents the noise variance of the noise added to the first spatiotemporal data at time step i.

[0267] Optionally, in some embodiments of this application, any time step is denoted as time step i, and for the first diffusion denoising process, the noise distribution of the predicted noise removed from the denoising result at time step i is... for:

[0268] ;

[0269] in:

[0270] ;

[0271] ;

[0272] ;

[0273] ;

[0274] in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after removing prediction noise from the noisy result at time step i-1. This represents the spatiotemporal data obtained at time step i after removing the prediction noise from the noisy result. This represents the first spatiotemporal data. Indicates the first embedded guide signal. Indicates the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the decay parameter corresponding to time step i-1. This represents the noise mean that is removed from the noise-adding result at time step i. This represents the noise variance of the noise removed from the noise-adding result at time step i.

[0275] Optionally, in some embodiments of this application, the diffusion model to be trained is an equivariant diffusion model. The equivariant diffusion model includes multiple noise prediction units that correspond one-to-one with time steps. The multiple noise prediction units are connected in sequence. The noise prediction unit is used to predict the noise corresponding to the time step based on the time step embedding and the first embedding guidance signal of the corresponding time step, and remove the predicted noise corresponding to the time step from the denoising result output by the previous noise prediction unit. The noise prediction unit includes multiple U-Net residual blocks connected in sequence. The U-Net residual block includes a normalization layer, an affine transformation layer, an activation function layer, a feature fusion layer and an inner convolution layer connected in sequence.

[0276] The affine transformation layer is used to: perform a first linear transformation on the time step embedding of the corresponding time step of the noise prediction unit to obtain a first affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the first affine transformation parameter to obtain a first intermediate feature; and perform a first linear transformation on the first embedding guidance signal to obtain a second affine transformation parameter, and perform an affine transformation on the output features of the normalization layer based on the second affine transformation parameter to obtain a second intermediate feature.

[0277] The activation function layer is used to: perform a nonlinear transformation on the first intermediate feature using a smooth nonlinear activation function to obtain the third intermediate feature, and perform a nonlinear transformation on the second intermediate feature using a smooth nonlinear activation function to obtain the fourth intermediate feature;

[0278] The feature fusion layer is used to add the third and fourth intermediate features to obtain the fused feature, and then input the fused feature into the inner convolutional layer.

[0279] Optionally, in some embodiments of this application, for each first head entity belonging to the first sub-region in the spatial knowledge graph, determining the neighborhood information of the first head entity includes:

[0280] Embedding mapping is performed on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity.

[0281] The multiple embedding mapping results corresponding to each neighboring entity of the first head entity are fused and transformed by a second linear transformation to obtain the neighborhood information of the first head entity. The second linear transformation is implemented by a first linear transformation matrix to be trained. The first linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain the first target linear transformation matrix.

[0282] Optionally, in some embodiments of this application, embedding mapping is performed on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity, including:

[0283] Embedding mappings are performed on each neighboring entity of the first head entity using Euclidean space, hyperbolic space, and hyperspherical space respectively, to obtain the Euclidean space embedding mapping results, hyperbolic space embedding mapping results, and hyperspherical space embedding mapping results for each neighboring entity of the first head entity.

[0284] Optionally, in some embodiments of this application, the neighborhood information of each first head entity is aggregated to generate a first embedding guidance signal including a node representation of each first head entity, including:

[0285] The neighborhood information of each first-head entity is aggregated by the graph neural network to be trained to obtain the node representation of each first-head entity; the graph neural network to be trained is jointly trained with the diffusion model to be trained to obtain the target graph neural network.

[0286] The node representations of each first-head entity are merged to obtain the first embedding guidance signal.

[0287] Optionally, in some embodiments of this application, training the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model includes:

[0288] Based on sliding window and matrix zero-padding, the first spatiotemporal data is transformed into a first delayed embedding matrix aligned with the input dimension of the diffusion model to be trained;

[0289] The target diffusion model is obtained by training the diffusion model to be trained based on the first delayed embedding matrix and the first embedding guidance signal.

[0290] Optionally, in some embodiments of this application, collecting node information for nodes in the target region includes: collecting node information for nodes in the target region at multiple spatial granularities;

[0291] The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region based on the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0292] The first entity includes nodes in the first sub-region that correspond to the target spatial granularity; the first spatiotemporal data includes spatiotemporal data of the target data type obtained for the target object in the first sub-region.

[0293] Based on the same inventive concept, this application also provides a spatial knowledge graph-guided spatiotemporal data generation device for implementing the above-mentioned spatial knowledge graph-guided spatiotemporal data generation method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more spatial knowledge graph-guided spatiotemporal data generation device embodiments provided below can be found in the limitations of the spatial knowledge graph-guided spatiotemporal data generation method described above, and will not be repeated here.

[0294] In an exemplary embodiment, as shown in FIG6, a spatiotemporal data generation device guided by spatial knowledge graph is provided. This device generates spatiotemporal data based on a target diffusion model trained using the aforementioned spatiotemporal knowledge graph-guided diffusion model training method. The spatiotemporal data generation device includes:

[0295] The second information determination module is configured to determine the neighborhood information of each second head entity belonging to the second sub-region in the spatial knowledge graph.

[0296] The second signal generation module is configured to aggregate the neighborhood information of each second head entity and generate a second embedded guiding signal including the node representation of each second head entity.

[0297] The spatiotemporal data generation module is configured to control the target diffusion model to generate second spatiotemporal data for the second sub-region based on preset noise data and a second embedding guidance signal. The generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction based on time step embedding and the second embedding guidance signal in time steps, and gradually removes the predicted noise from the preset noise data in time steps to obtain the second spatiotemporal data of the second sub-region. The time step embedding is obtained by position encoding the time step.

[0298] Optionally, in some embodiments of this application, collecting node information for nodes in the target region includes: collecting node information for nodes in the target region at multiple spatial granularities;

[0299] The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region based on the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity.

[0300] The second entity includes nodes in the second sub-region that correspond to the target spatial granularity; the second spatiotemporal data includes spatiotemporal data of the target data type generated for the target object in the second sub-region.

[0301] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0302] The aforementioned computer device can be exemplified as a server or terminal, and its internal structure can be shown in Figure 7. This computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores video tag processing data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a spatial knowledge graph-guided diffusion model training method and / or a spatial knowledge graph-guided spatiotemporal data generation method.

[0303] Those skilled in the art will understand that the structure shown in Figure 7 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0304] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0305] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0306] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0307] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0308] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0309] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0310] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A diffusion model training method guided by spatial knowledge graph, characterized in that, The spatial knowledge graph-guided diffusion model training method includes: collecting node information for nodes at multiple spatial granularities in a target region, and constructing a spatial knowledge graph for the target region based on the node information; the target region includes adjacent first and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities, wherein the head entities belong to nodes in the target region; the node information includes the node's position, the inclusion relationship between nodes, the adjacency relationship between nodes, and the node category to which the node belongs; for each first head entity belonging to the first sub-region in the spatial knowledge graph, determining the neighborhood information of the first head entity; aggregating the neighborhood information of each first head entity to generate a first embedding guidance signal including a node representation of each first head entity; mapping a first inherent trend feature based on the first embedding guidance signal; the first inherent trend feature characterizes all the... The first entity's functional mode; acquiring first spatiotemporal data for a first sub-region in the target region, and training the diffusion model to be trained based on the first spatiotemporal data, the first embedding guidance signal, and the first inherent trend feature to obtain a target diffusion model; the training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process, wherein the diffusion noise addition process samples noise based on the target noise distribution at time steps, and adds the sampled noise to the first spatiotemporal data at time steps to obtain a noise addition result; the first diffusion denoising process predicts noise based on the time step embedding and the first embedding guidance signal at time steps, and removes the predicted noise from the noise addition result at time steps; the target noise distribution is a noise distribution that introduces the first inherent trend feature to influence the noise mean; the time step embedding is obtained by position encoding the time step; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

2. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, The time series length of the first spatiotemporal data is T, and the number of features of the first spatiotemporal data is D. The step of obtaining the first inherent trend feature based on the first embedded guiding signal includes: when D=1, for each node representation of the first head entity in the first embedded guiding signal, mapping the node representation to a feature vector of length T through the training mapping module to obtain the first inherent trend feature; when D>1, for each node representation of the first head entity in the first embedded guiding signal, mapping the node representation to a feature vector of length T through the training mapping module to obtain the initial inherent trend feature, and concatenating the D identical initial inherent trend features to obtain the first inherent trend feature; wherein, the training mapping module is jointly trained with the training diffusion model to obtain the target mapping module.

3. The diffusion model training method based on spatial knowledge graph guided by claim 2, characterized in that, The mapping module to be trained includes a multilayer perceptron and / or a convolutional layer.

4. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, Any time step is denoted as time step i. For the diffusion noise addition process, the noise distribution of the sampling noise added to the first spatiotemporal data at time step i is the target noise distribution. for: ;in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after adding sampling noise to the first spatiotemporal data at time step i. This represents the first spatiotemporal data. This indicates the first embedded guidance signal. This represents the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the noise mean of the noise added to the first spatiotemporal data at time step i. This represents the noise variance of the noise added to the first spatiotemporal data at time step i.

5. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, Any time step is denoted as time step i. For the first diffusion denoising process, the noise distribution of the predicted noise removed from the denoising result at time step i is... for: ;in: ; ; ; ;in, Indicates a Gaussian distribution. This represents the spatiotemporal data obtained after removing the prediction noise from the noise-adding result at time step i-1. This represents the spatiotemporal data obtained at time step i after removing the prediction noise from the noise-adding result. This represents the first spatiotemporal data. This indicates the first embedded guidance signal. This represents the first inherent trend characteristic. Represents the identity matrix. This represents the decay parameter corresponding to time step i. This represents the decay parameter corresponding to time step i-1. This represents the noise mean value removed from the noise addition result at time step i. This represents the noise variance of the noise removed from the noise addition result at time step i.

6. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, The diffusion model to be trained is an equivariant diffusion model, which includes multiple noise prediction units corresponding one-to-one with each time step. These noise prediction units are connected sequentially. Each noise prediction unit is used to predict the noise corresponding to the time step based on the time step embedding and the first embedding guidance signal, and to remove the predicted noise from the denoising result output by the previous noise prediction unit. Each noise prediction unit includes multiple U-Net residual blocks connected sequentially. Each U-Net residual block includes a normalization layer, an affine transformation layer, an activation function layer, a feature fusion layer, and an involution layer connected sequentially. The affine transformation layer is used to perform a first linear transformation on the time step embedding of the corresponding time step of its noise prediction unit to obtain a first affine transformation. The parameters are used to perform an affine transformation on the output features of the normalized layer based on the first affine transformation parameters to obtain a first intermediate feature; and to perform a first linear transformation on the first embedding guidance signal to obtain a second affine transformation parameter, and to perform an affine transformation on the output features of the normalized layer based on the second affine transformation parameter to obtain a second intermediate feature; the activation function layer is used to perform a nonlinear transformation on the first intermediate feature using a smooth nonlinear activation function to obtain a third intermediate feature, and to perform a nonlinear transformation on the second intermediate feature using the smooth nonlinear activation function to obtain a fourth intermediate feature; the feature fusion layer is used to add the third intermediate feature and the fourth intermediate feature to obtain a fused feature, and to input the fused feature into the inner convolutional layer.

7. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, For each first head entity belonging to the first sub-region in the spatial knowledge graph, determining the neighborhood information of the first head entity includes: performing embedding mapping on each neighbor entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighbor entity of the first head entity; fusing the multiple embedding mapping results corresponding to each neighbor entity of the first head entity and performing a second linear transformation to obtain the neighborhood information of the first head entity; the second linear transformation is implemented through a first linear transformation matrix to be trained, and the first linear transformation matrix to be trained is jointly trained with the diffusion model to be trained to obtain a first target linear transformation matrix.

8. The diffusion model training method based on spatial knowledge graph guided by claim 7, characterized in that, The step of performing embedding mapping on each neighboring entity of the first head entity through multiple different geometric spaces to obtain multiple embedding mapping results corresponding to each neighboring entity of the first head entity includes: performing embedding mapping on each neighboring entity of the first head entity through Euclidean space, hyperbolic space and hyperspherical space respectively to obtain Euclidean space embedding mapping result, hyperbolic space embedding mapping result and hyperspherical space embedding mapping result corresponding to each neighboring entity of the first head entity.

9. The diffusion model training method based on spatial knowledge graph guided by claim 7 or 8, characterized in that, The step of aggregating the neighborhood information of each first head entity to generate a first embedding guidance signal including the node representation of each first head entity includes: aggregating the neighborhood information of each first head entity through a graph neural network to be trained to obtain the node representation of each first head entity; jointly training the graph neural network to be trained with the diffusion model to be trained to obtain a target graph neural network; and fusing the node representations of each first head entity to obtain the first embedding guidance signal.

10. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, The step of training the diffusion model to be trained based on the first spatiotemporal data and the first embedding guidance signal to obtain the target diffusion model includes: converting the first spatiotemporal data into a first delayed embedding matrix aligned with the input dimension of the diffusion model to be trained based on a sliding window and matrix zero-filling; and training the diffusion model to be trained based on the first delayed embedding matrix and the first embedding guidance signal to obtain the target diffusion model.

11. The diffusion model training method based on spatial knowledge graph guided by claim 1, characterized in that, The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region according to the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity; the first head entity includes the node in the first sub-region corresponding to the target spatial granularity; the first spatiotemporal data includes the spatiotemporal data of the target data type obtained for the target object in the first sub-region.

12. A spatiotemporal data generation method guided by spatial knowledge graphs, characterized in that, The spatiotemporal data generation method based on spatial knowledge graph guidance generates spatiotemporal data based on a target diffusion model trained by any one of claims 1-11. The method includes: for each second head entity belonging to the second sub-region in the spatial knowledge graph, determining the neighborhood information of the second head entity; aggregating the neighborhood information of each second head entity to generate a second embedding guidance signal including a node representation of each second head entity; controlling the target diffusion model to generate second spatiotemporal data for the second sub-region based on preset noise data and the second embedding guidance signal; the generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction step-by-step based on time step embedding and the second embedding guidance signal, and gradually removes the predicted noise from the preset noise data step-by-step to obtain the second spatiotemporal data of the second sub-region; the time step embedding is obtained by position encoding the time step.

13. The spatiotemporal data generation method based on spatial knowledge graphs according to claim 12, characterized in that, The target diffusion model is specifically used to generate second spatiotemporal data for the second sub-region according to the target generation task. The target generation task includes: generating spatiotemporal data of the target data type of the target object at the node of the target spatial granularity; the second head entity includes the node in the second sub-region corresponding to the target spatial granularity; the second spatiotemporal data includes the spatiotemporal data of the target data type generated for the target object in the second sub-region.

14. A diffusion model training device guided by spatial knowledge graph, characterized in that, The diffusion model training device based on spatial knowledge graph guidance includes: a data acquisition and construction module, configured to acquire node information for nodes at multiple spatial granularities in a target region, and construct a spatial knowledge graph for the target region based on the node information; the target region includes adjacent first and second sub-regions, and the spatial knowledge graph includes multiple triples consisting of head entities, inter-entity relationships, and tail entities, wherein the head entities belong to nodes in the target region; the node information includes the node's position, the inclusion relationship between nodes, the adjacency relationship between nodes, and the node category to which the node belongs; a first information determination module, configured to determine the neighborhood information of each first head entity belonging to the first sub-region in the spatial knowledge graph; a first signal generation module, configured to aggregate the neighborhood information of each first head entity to generate a first embedding guidance signal including a node representation of each first head entity; and to map a first inherent trend feature based on the first embedding guidance signal; The first inherent trend feature characterizes the functional patterns of all the first head entities; the model training module is configured to acquire first spatiotemporal data for a first sub-region in the target region, and train the diffusion model to be trained based on the first spatiotemporal data, the first embedding guidance signal, and the first inherent trend feature to obtain a target diffusion model; the training process of the diffusion model to be trained includes a diffusion noise addition process and a first diffusion denoising process, wherein the diffusion noise addition process samples noise based on the target noise distribution in time steps, and adds the sampled noise to the first spatiotemporal data in time steps to obtain a noise addition result; the first diffusion denoising process predicts noise based on the time step embedding and the first embedding guidance signal in time steps, and removes the predicted noise from the noise addition result in time steps; the target noise distribution is a noise distribution that introduces the influence of the first inherent trend feature on the noise mean; the time step embedding is obtained by position encoding the time step; the target diffusion model is used to generate second spatiotemporal data for the second sub-region.

15. A spatiotemporal data generation device guided by a spatial knowledge graph, characterized in that, The spatiotemporal data generation device guided by the spatial knowledge graph generates spatiotemporal data based on a target diffusion model trained by the method described in any one of claims 1-11. The device comprises: a second information determination module configured to determine the neighborhood information of each second head entity belonging to the second sub-region in the spatial knowledge graph; a second signal generation module configured to aggregate the neighborhood information of each second head entity to generate a second embedding guidance signal including a node representation of each second head entity; and a spatiotemporal data generation module configured to control the target diffusion model to generate second spatiotemporal data for the second sub-region based on preset noise data and the second embedding guidance signal. The generation process of the second spatiotemporal data includes a second diffusion denoising process, wherein the second diffusion denoising process performs noise prediction step-by-step based on time step embedding and the second embedding guidance signal, and gradually removes the predicted noise from the preset noise data step-by-step to obtain the second spatiotemporal data for the second sub-region; the time step embedding is obtained by position encoding the time step.

16. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1-13.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-13.

Images