Multi-source Heterogeneous Data Semantic Alignment Method, Device and Computer Equipment

Abstract

The present invention relates to the technical field of data processing, and discloses a multi-source heterogeneous data semantic alignment method, apparatus and computer device. Among them, the method includes: collecting multi-source heterogeneous spatio-temporal data through a central data node and multiple edge nodes, and performing spatial attribute feature extraction and inter-unit spatial correlation calculation to obtain an urban data semantic association degree feature matrix; inputting the urban data semantic association degree feature matrix into a multi-layer graph attention network for feature extraction to obtain a node feature vector matrix and a task association degree adjacency matrix; generating a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix; allocating data processing tasks to computing nodes according to the distributed scheduling scheme, and performing alternating iterative optimization based on the urban data semantic association degree feature matrix to obtain a semantic mapping relationship set. This method realizes the intelligent scheduling of data processing tasks and efficient semantic alignment in a distributed environment.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method, device, and computer device for semantic alignment of multi-source heterogeneous data. Background Art

[0002] With the rapid development of smart city construction, the collection, processing, and storage of spatio-temporal big data have become key technical challenges. The traditional centralized data processing architecture is difficult to meet the real-time processing requirements of massive heterogeneous data, and the emergence of distributed edge computing technology provides a new idea for solving this problem. However, there are differences and uncertainties in the semantic level of multi-source heterogeneous data, which pose great challenges to data fusion and semantic alignment.

[0003] Currently, although there are already various smart city semantic correlation coefficient methods for processing the semantic association of spatio-temporal big data, these methods often produce biases when processing specific types of data and do not fully utilize the characteristics of spatio-temporal data. In addition, the existing semantic alignment methods may not meet the semantic consistency conditions, resulting in a reduction in the reliability of data fusion and the inability to effectively eliminate semantic ambiguities. In terms of data processing scheduling, due to the limitations of heuristic algorithms and the adaptability challenges of heterogeneous data processing, the existing scheduling algorithms are difficult to meet the requirements of the smart city AI platform. Especially when processing heterogeneous data with complex spatio-temporal characteristics, the traditional scheduling methods cannot effectively balance the allocation of computing resources and data processing efficiency, resulting in a decline in the overall performance of the system. Summary of the Invention

[0004] The present invention provides a method, device, and computer device for semantic alignment of multi-source heterogeneous data, and the present invention realizes the intelligent scheduling of data processing tasks and efficient semantic alignment in a distributed environment.

[0005] In a first aspect, the present invention provides a method for semantic alignment of multi-source heterogeneous data, and the method for semantic alignment of multi-source heterogeneous data includes:

[0006] Collect multi-source heterogeneous spatio-temporal data through a central data node and multiple edge nodes, and construct a set of standardized data processing units and a corresponding distributed data index directory;

[0007] Extract spatial attribute features and calculate the spatial correlation between units for each data processing unit in the set of standardized data processing units to obtain a matrix of urban data semantic association degree features;

[0008] Input the matrix of urban data semantic association degree features into a multi-layer graph attention network for feature extraction to obtain a matrix of node feature vectors and a task association degree adjacency matrix of data processing tasks;

[0009] Generate a distributed scheduling plan for the data processing task based on the node feature vector matrix and the task correlation adjacency matrix;

[0010] Allocate the data processing task to the computing nodes according to the distributed scheduling plan, and perform alternating iterative optimization based on the urban data semantic correlation degree feature matrix to obtain the semantic mapping relationship set of the multi-source heterogeneous spatio-temporal data.

[0011] In a second aspect, the present invention provides a multi-source heterogeneous data semantic alignment device, and the multi-source heterogeneous data semantic alignment device includes:

[0012] An acquisition module, configured to acquire multi-source heterogeneous spatio-temporal data through a central data node and a plurality of edge nodes, and construct a set of standardized data processing units and a corresponding distributed data index directory;

[0013] A calculation module, configured to perform spatial attribute feature extraction and inter-unit spatial correlation calculation on each data processing unit in the set of standardized data processing units to obtain an urban data semantic correlation degree feature matrix;

[0014] A feature extraction module, configured to input the urban data semantic correlation degree feature matrix into a multi-layer graph attention network for feature extraction to obtain a node feature vector matrix and a task correlation adjacency matrix of the data processing task;

[0015] A generation module, configured to generate a distributed scheduling plan for the data processing task based on the node feature vector matrix and the task correlation adjacency matrix;

[0016] An optimization module, configured to allocate the data processing task to the computing nodes according to the distributed scheduling plan, and perform alternating iterative optimization based on the urban data semantic correlation degree feature matrix to obtain the semantic mapping relationship set of the multi-source heterogeneous spatio-temporal data.

[0017] In a third aspect of the present invention, there is provided a multi-source heterogeneous data semantic alignment device, including: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory so that the multi-source heterogeneous data semantic alignment device executes the above-mentioned multi-source heterogeneous data semantic alignment method.

[0018] In a fourth aspect of the present invention, there is provided a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is made to execute the above-mentioned multi-source heterogeneous data semantic alignment method.

[0019] In the technical solution provided by the present invention, by constructing a distributed edge computing network and a data acquisition topology structure, and combining multi-level data preprocessing and standardization mechanisms, the system can efficiently process multi-source heterogeneous spatio-temporal data, significantly improving the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of urban data semantic association degree feature matrix is constructed, effectively solving the ambiguity problem in the semantic alignment process of traditional methods and improving the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and the local and global dependencies between data units are captured through the multi-head attention mechanism, enhancing the system's expression ability for complex data structures; innovatively, deep reinforcement learning is combined with the k-dimensional tree nearest neighbor algorithm to achieve intelligent scheduling of data processing tasks and optimize the allocation efficiency of computing resources; through the alternating iteration optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, improving the overall processing performance and scalability of the system.

[0020] Other features and advantages of the present invention will be described in the following specification, and part of them will be obvious from the specification or understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims, and drawings.

[0021] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. Brief Description of the Drawings

[0022] Figure 1 It is a schematic diagram of an embodiment of the multi-source heterogeneous data semantic alignment method in an embodiment of the present invention;

[0023] Figure 2 It is a schematic diagram of an embodiment of the multi-source heterogeneous data semantic alignment device in an embodiment of the present invention;

[0024] Figure 3 It is a schematic diagram of an embodiment of the multi-source heterogeneous data semantic alignment device in an embodiment of the present invention. Detailed Embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0026] As used in the embodiments of the present invention, the terms "comprising", "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include other steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0027] For ease of understanding of this embodiment, a multi-source heterogeneous data semantic alignment method disclosed in the embodiments of the present invention will be introduced in detail first. As Figure 1 shown, the method includes the following steps:

[0028] 101. Collect multi-source heterogeneous spatio-temporal data through a central data node and multiple edge nodes, and construct a set of standardized data processing units and a corresponding distributed data index directory;

[0029] It can be understood that the execution subject of the present invention can be a multi-source heterogeneous data semantic alignment device, or a terminal or a server. Specifically, it is not limited here. The embodiments of the present invention will be described by taking the server as the execution subject as an example.

[0030] Specifically, the central data node and multiple edge nodes are deployed at different data collection points in the city to cover a wide spatial range and make full use of the distributed characteristics of edge computing to construct an efficient data collection network topology. In the topology, each node undertakes certain collection tasks and communicates with each other through low-latency network connections, forming a distributed architecture that can efficiently collect, transmit, and process multi-source heterogeneous data. Through this topology, the central data node and multiple edge nodes collect heterogeneous spatio-temporal data from multiple sources (such as sensors, traffic devices, weather stations, etc.). Preprocess these raw data, including cleaning, deduplication, format conversion, and normalization operations, to generate a cleaned and standardized dataset. Based on the data records in the cleaned and standardized dataset, establish a time-series index tree structure to effectively manage and organize these data. The time-series index tree quickly locates and retrieves relevant data records based on time, and combines with the time window sliding mechanism for analysis to mine the time-series correlation of the data. The time window sliding mechanism captures the change trend of the data in different time periods by dynamically adjusting the time span, revealing the time-series patterns hidden in the data. According to the spatial dimension characteristics, perform geographical grid division and spatial clustering on the dataset with time-series correlation. Geographical grid division divides the entire urban area into multiple grid units, and each unit represents a specific geographical range. Through spatial clustering, similar data points are aggregated according to the spatial distribution characteristics of the data to generate spatio-temporal correlation data blocks with clear spatial boundaries and attribute characteristics. Extract metadata and calculate feature vectors for the spatio-temporal correlation data blocks to obtain the target calculation results. The process of metadata extraction includes summarizing and recording the key attributes (such as time range, spatial range, data type, etc.) of each data block, while the feature vector calculation generates a multi-dimensional numerical representation based on the content of the data block. Store the target calculation results in the distributed data directory and build a distributed data index directory based on this. Perform adaptive segmentation and boundary optimization on the spatio-temporal correlation data blocks according to the preset data processing unit size. Adaptive segmentation dynamically adjusts the division method of the data blocks to make them adapt to different data processing requirements; while boundary optimization improves data utilization and processing efficiency by reducing the overlap and redundancy between data blocks. Finally, form a set of standardized data processing units.

[0031] 102. Extract the spatial attribute characteristics and calculate the inter-unit spatial correlation for each data processing unit in the set of standardized data processing units to obtain the urban data semantic association degree feature matrix;

[0032] Specifically, extract the geographical location information from each data processing unit, including latitude and longitude coordinates, spatial area range, and its distribution pattern in the overall space. Through the analysis of geographical attributes, generate the spatial attribute feature vectors of each unit. These feature vectors numerically describe the spatial position, size, and geometric shape of each data unit. Based on the spatial attribute feature vectors, calculate the spatial straight-line distance and network topology distance between data processing units. The spatial straight-line distance can reflect the physical position relationship of data units in the geographical coordinate system, while the network topology distance takes into account the path length in the actual transportation or communication network and has higher practical reference value. Through distance calculation, generate the spatial distance relationship matrix. Based on the original spatial distance relationship matrix, construct the spatial connectivity graph between data processing units. Regard the data units as nodes in the graph, and establish edge connections according to their distance relationships to form a graph model that can reflect the spatial relationships of data units. On this basis, conduct a quantitative analysis of the spatial direction relationships of each node in the graph, including the direction angles between nodes, the distribution patterns of adjacent nodes, and the directional distribution of connection weights, etc. Integrate the results of the quantitative analysis into a spatial correlation metric matrix to describe the degree of spatial correlation between nodes. By mapping the information in the spatial correlation metric matrix to a high-dimensional feature space, generate the spatial semantic feature mapping matrix of data units. The mapping process combines dimensionality reduction techniques and embedding methods to extract high-dimensional semantic features while preserving the original spatial relationship information. For example, use a graph embedding method based on deep learning to embed complex spatial relationships into feature vectors and generate a high-dimensional mapping matrix with semantic expression ability. Based on the spatial semantic feature mapping matrix, design a comprehensive evaluation function that fuses spatial proximity and functional similarity. Spatial proximity reflects the geographical proximity of data units, while functional similarity focuses on the consistency of units in semantic attributes, such as whether adjacent regions have similar functional uses or performance characteristics. The evaluation function scores the degree of semantic correlation between data units by combining these two dimensions and generates a comprehensive semantic correlation metric. Use the above evaluation function to score the correlation of all data unit pairs and organize the results into the urban data semantic association degree feature matrix.

[0033] Set a threshold for the distance values in the original spatial distance relationship matrix to filter out node pairs with significant spatial connectivity. The selection of the threshold is based on the spatial distance characteristics in the actual application scenario to ensure that only those node pairs with spatial distances within a reasonable range are retained, obtaining a filtered spatial connectivity threshold matrix. Construct a weighted undirected graph based on the spatial connectivity threshold matrix. The nodes of this graph represent the standardized data processing units, and the edges in the graph represent the spatial connection relationships between these units. To reflect the importance of different connection relationships, assign weights to each edge. The weight value is calculated inversely according to the distance value of the corresponding node pair in the spatial connectivity threshold matrix. The smaller the distance between node pairs, the higher the weight, highlighting the closer spatial relationship. After constructing the spatial connectivity graph, calculate the relative azimuth angle and direction cosine value between each node and its adjacent nodes in the graph to generate a node direction feature matrix. The relative azimuth angle represents the angular difference between two nodes in space, and the direction cosine value quantifies the numerical characteristics of the spatial direction relationship. Based on the node direction feature matrix, calculate the spatial direction similarity between node pairs. Through the cosine similarity formula, quantify the consistency of the direction relationship between node pairs to generate a direction correlation matrix. Normalize the direction correlation matrix to limit all values within a standard range for effective fusion with the weight matrix of the spatial connectivity graph. By combining the normalized direction correlation matrix with the weight matrix of the spatial connectivity graph, obtain a comprehensive correlation degree matrix. Calculate the degree centrality and betweenness centrality of the nodes in the graph based on the comprehensive correlation degree matrix to quantify the key degree of the nodes in the graph structure. Degree centrality describes the connection degree of a node, reflecting the number of nodes directly adjacent to a node, while betweenness centrality measures the intermediary role of a node in the information transmission process of the entire graph. Through these centrality indicators, identify the key nodes that have a greater impact on spatial connectivity. According to the importance indicators of the nodes, adjust the weights in the comprehensive correlation degree matrix, assigning higher weight values to more important nodes to generate a weighted spatial correlation matrix. Symmetrize the weighted spatial correlation matrix, that is, transform the matrix into a symmetric matrix to eliminate the imbalance caused by directionality. At the same time, standardize the matrix so that its numerical distribution is within a unified range for subsequent calculation and optimization, and finally generate a spatial correlation metric matrix.

[0034] 103. Input the semantic association degree feature matrix of urban data into a multi-layer graph attention network for feature extraction to obtain the node feature vector matrix and task association degree adjacency matrix of the data processing task;

[0035] Specifically, an initial graph structure is constructed based on the semantic association degree feature matrix of urban data. Each row or column of the semantic association degree feature matrix of urban data represents a data processing unit, and these units serve as the nodes of the initial graph structure. The semantic association degree values in the matrix serve as the edge weights between the nodes, forming a weighted graph model. In this initial graph structure, the nodes represent data processing tasks, and the edge weights represent the semantic association strength between the tasks. The initial graph structure is input into the first graph attention layer of the multi-layer graph attention network for feature projection transformation. The first graph attention layer performs weighted processing on the node features through the multi-head attention mechanism to capture the local dependence features between the nodes. In this layer, 8 independent attention heads are designed. Each attention head calculates the attention score through the features of the adjacent nodes, dynamically adjusts the weight distribution of each node pair, and generates the local dependence feature representation of each node. The local dependence features generated in the first layer are input into the second graph attention layer of the multi-layer graph attention network. The second graph attention layer enhances the feature extraction ability by expanding the number of attention heads to 16. In this layer, each attention head independently calculates the attention score between the nodes, and quantifies the association weight coefficient between them by comparing the feature relationships of different nodes. The weight coefficient reflects the semantic correlation strength of the node pair. Based on the association weight coefficient, a weighted aggregation operation is performed on the local dependence features generated in the first layer, so as to integrate the adjacent features of the nodes and obtain the global dependence feature representation of the second layer. On the basis of the second layer features, the global dependence features are input into the third graph attention layer of the multi-layer graph attention network. The third graph attention layer improves the expression ability of the network by introducing 32 attention heads to achieve higher-dimensional feature aggregation. Each attention head enhances the nodes in a position-encoding manner, so as to explicitly add the position information of the nodes to the node features. The incorporation of the position information helps to capture the spatial distribution and topological relationship of the nodes in the graph structure, enabling the third layer features to more accurately express the semantic status of the nodes in the entire graph. After the calculation of this layer, a node feature vector matrix of the data processing tasks is obtained. Each row of this matrix represents the high-dimensional feature vector of a node, which completely encodes the semantic information, neighborhood information, and spatial position features of the node. Calculate the cosine similarity between the node pairs based on the node feature vector matrix to quantify the semantic similarity between them. The cosine similarity effectively measures the semantic association strength of the node pair by comparing the direction consistency of the two node feature vectors, and obtains the task association degree adjacency matrix.

[0036] 104. Generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix;

[0037] Specifically, perform feature embedding operations on the node feature vector matrix and the task correlation adjacency matrix, map their high-dimensional structural features into a low-dimensional continuous vector space, and generate the state vector of the scheduling environment. Extract state information features based on the state vector of the scheduling environment. Through a deep learning model, automatically capture the implicit features and complex relationships in the state vector to generate a deep state representation. Input the deep state representation into the policy generator for state space mapping transformation. The core structure of the policy generator consists of three fully connected layers, and the input deep state representation is transformed layer by layer through a non-linear activation function to output a continuous action vector. This continuous action vector defines the potential action range of the scheduling policy in the continuous space. Based on this continuous action vector, a k-dimensional tree index structure is established to discretize the continuous space into several finite action options, generating a discrete action space, enabling the scheduling policy to be optimized and selected within a discrete but still highly expressive space. To evaluate the pros and cons of candidate actions in the discrete action space, input the candidate actions into the value evaluation network for scoring the resource allocation effect. The value evaluation network consists of two convolutional layers and three fully connected layers. The local patterns of action features are extracted through the convolutional layer, and then the information of each dimension is integrated through the fully connected layer to generate a value scoring sequence of the action. The value score of each action reflects its potential effect in resource allocation and task completion. The higher the score of an action, the better it can optimize the scheduling goal in the current state. Based on the action value sequence, calculate the reward function value of the task scheduling. The reward function measures key indicators such as task execution efficiency, node resource utilization rate, and task completion time to obtain the evaluation score of the scheduling scheme. Through this evaluation score, optimize the current policy. To achieve efficient policy optimization, adopt the proximal policy optimization algorithm, balance between stability and exploration by restricting the range of policy updates, gradually optimize the scheduling policy, and generate the target scheduling policy. Apply the target scheduling policy to the task allocation mapping. The target scheduling policy generates corresponding allocation decisions for each data processing task, allocates the tasks to the computing nodes with matching computing power, location, and resource conditions, and generates a task processing time sequence schedule, finally obtaining a distributed scheduling scheme for the data processing tasks.

[0038] 105. Allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on the urban data semantic correlation degree feature matrix to obtain a semantic mapping relationship set of multi-source heterogeneous spatio-temporal data.

[0039] Specifically, according to the distributed scheduling scheme, the task sequence is grouped and optimized to generate a node-level task allocation scheme. Considering comprehensively the semantic correlation degree between tasks, the resource constraints of computing nodes, and the priority of task execution, it is ensured that the allocation of tasks on nodes has the characteristics of high efficiency and load balancing. After completing the task grouping and optimization, the urban data semantic correlation degree feature matrix is divided into matrix blocks according to a preset block size. Each sub-block contains the semantic features of adjacent regions, which can to a certain extent retain the spatial semantic continuity between regions. The process of block division aims to reduce the complexity of global calculation, decompose the large-scale matrix into small-scale sub-blocks that are easy to process, and achieve parallel computing. Parallel semantic mapping calculations are performed on the set of feature sub-blocks. The block coordinate descent method is used to update the semantic mapping parameters. The block coordinate descent method gradually approaches the global optimal solution step by step by fixing some parameters and optimizing the remaining parameters. In parallel computing, each computing node independently processes one or more feature sub-blocks and obtains the preliminary semantic mapping results through local calculation. The local semantic mapping results contain the mapping parameters of each feature sub-block, and these parameters reflect the semantic relationship between data within a specific region. A global consistency constraint is constructed based on the local semantic mapping results. The role of the global consistency constraint is to coordinate the semantic mapping parameters between different feature sub-blocks, so that the overall semantic mapping model can take into account both the global perspective and local details. In this process, the alternating direction method of multipliers (ADMM) is used to jointly optimize the semantic mapping parameters. The ADMM method decomposes the global optimization problem into multiple sub-problems that are easy to solve, and introduces Lagrange multipliers for constraint adjustment to achieve efficient optimization of global consistency and obtain the globally optimized semantic mapping parameters. Semantic relationship calculations are performed on multi-source heterogeneous spatio-temporal data based on the globally optimized semantic mapping parameters. The optimized mapping parameters are applied to the actual data to mine the semantic association between data, and a semantic mapping relationship set is generated, which contains the mapping rules and semantic links between data.

[0040] In the embodiments of the present invention, by constructing a distributed edge computing network and a data acquisition topology structure, and combining multi-level data preprocessing and standardization mechanisms, the system can efficiently process multi-source heterogeneous spatio-temporal data, significantly improving the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of urban data semantic association degree feature matrix is constructed, effectively solving the ambiguity problem in the semantic alignment process of traditional methods and improving the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and the local and global dependencies between data units are captured through the multi-head attention mechanism, enhancing the system's expression ability for complex data structures; innovatively, deep reinforcement learning is combined with the k-dimensional tree nearest neighbor algorithm to achieve intelligent scheduling of data processing tasks and optimize the allocation efficiency of computing resources; through the alternating iteration optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, improving the overall processing performance and scalability of the system.

[0041] In a specific embodiment, the process of executing step 101 may specifically include the following steps:

[0042] Deploy the central data node and multiple edge nodes at the urban data collection points, and construct a data collection network topology diagram through the distributed edge computing network;

[0043] Perform data preprocessing on the multi-source heterogeneous spatio-temporal data collected by the central data node and multiple edge nodes based on the data collection network topology diagram to obtain a cleaned standardized data set;

[0044] Establish a time series index tree structure based on the data records in the cleaned standardized data set, and perform time window sliding mechanism analysis to obtain a data set with time series association;

[0045] According to the spatial dimension features, perform geographical grid division and spatial clustering on the data set with time series association to obtain spatio-temporal association data blocks, and perform metadata extraction and feature vector calculation on the spatio-temporal association data blocks to obtain the target calculation result;

[0046] Store the target calculation result in the distributed data directory to obtain a distributed data index directory, and perform adaptive segmentation and boundary optimization on the spatio-temporal association data blocks according to the preset data processing unit size to obtain a set of standardized data processing units.

[0047] Specifically, deploy the central data node and multiple edge nodes at different data collection points in the city to construct a distributed edge computing network topology diagram. Set the central node in a data center with high-performance computing capabilities, while deploy the edge nodes in data-intensive areas. The edge nodes collect multi-source heterogeneous spatio-temporal data locally, such as traffic flow data, meteorological data, public facility sensor data, etc. In the construction of the topology diagram, nodes and edges represent data collection points and the communication links between them respectively. Let be the topology diagram, where is the node set, is the edge set, is the weight set of the edges. The weight of the edge is defined according to the communication delay or data transmission rate. For example, the weight is expressed by the formula:

[0048] ;

[0049] where, represents the bandwidth between node and node , represents the communication delay between them. This formula can effectively quantify the quality of the communication link, facilitating subsequent optimization of data transmission. Based on the data collection network topology diagram, perform data preprocessing on the multi-source heterogeneous spatio-temporal data collected by the edge nodes, including data cleaning, missing value filling, format conversion, and unified timestamp processing. For example, for traffic flow data, fill in the missing time periods through interpolation; for meteorological data, standardize the data formats generated by different devices and unify them into the quadruple form, where is the timestamp, is the geographical coordinate, is the observed value. After processing, a cleaned and standardized data set is generated. Establish a time series index tree structure based on the data records in the standardized data set to support efficient query. Let the standardized data set contain records, and the timestamp of each record is . The time series index tree organizes these records into a balanced tree structure in chronological order, and each node stores a time range and the corresponding data entries. The query complexity is controlled to be by optimizing the depth of the tree. At the same time, introduce a time window sliding mechanism to analyze the temporal correlation of the data. The size of the sliding window is , and the data set within the window is:

[0050] ;

[0051] where, is the data with timestamp Data records. The sliding mechanism can dynamically capture temporal patterns within a short period, such as the changing trend of traffic flow during peak hours. After completing the temporal processing, geographical grid division and spatial clustering are performed on the dataset with temporal correlations according to the spatial dimension characteristics. Assume the geographical scope of the entire urban area is , which is divided into grid cells, and the size of each grid is:

[0052] ;

[0053] After allocating each data record to the corresponding grid, density clustering (such as the DBSCAN algorithm) is performed on the data points in each grid to generate spatio-temporal correlation data blocks. Each data block contains data records that are spatially close and temporally adjacent, reflecting the dynamic behavior characteristics in that area. Metadata extraction and feature vector calculation are performed on the spatio-temporal correlation data blocks. Metadata includes the time range, geographical range, average value, standard deviation, etc. of the data block, and the feature vector adopts a multi-dimensional representation form, such as:

[0054] ;

[0055] Among them, and are the geographical coordinates of the center of the data block, and are the standard deviations of the spatial distribution, and are the eigenvalues of the covariance matrix, reflecting the main direction and dispersion degree of the data distribution. The target calculation results are stored in a distributed data directory to support efficient retrieval and distributed computing. Adaptive segmentation and boundary optimization are performed on the spatio-temporal correlation data blocks to ensure that the size of the data blocks adapts to the computing capabilities of different nodes. The segmentation size is dynamically adjusted according to the following formula:

[0056] ;

[0057] Among them, is the segmentation size, and are the maximum and minimum block sizes respectively, is the computing capability of the target node, is the average computing capability of the entire network, is the average block size of the entire network. Through the above process, a set of standardized data processing units is finally generated.

[0058] In a specific embodiment, the process of executing step 102 may specifically include the following steps:

[0059] Extract the longitude and latitude coordinates, define the spatial range, and analyze the spatial distribution pattern of the geographical location information of each data processing unit in the standardized data processing unit set to obtain the spatial attribute feature vector;

[0060] Based on the spatial attribute feature vector, calculate the spatial straight-line distance and network topology distance between data processing units to obtain the original spatial distance relationship matrix;

[0061] Based on the original spatial distance relationship matrix, establish a spatial connectivity graph between data processing units, and quantitatively analyze the spatial direction relationship of the nodes in the spatial connectivity graph to obtain the spatial correlation metric matrix;

[0062] Based on the spatial correlation metric matrix, map the spatial relationship of data processing units to a high-dimensional feature space to obtain the spatial semantic feature mapping matrix;

[0063] Based on the spatial semantic feature mapping matrix, create an evaluation function that combines spatial proximity and functional similarity, and score the semantic correlation degree between data processing units according to the evaluation function to obtain the urban data semantic association degree feature matrix.

[0064] Specifically, extract the geographical location information of each data processing unit in the standardized data processing unit set. Each data unit records longitude and latitude coordinates. Let the unit set be , where each unit contains attributes , representing its longitude and latitude coordinates respectively. By extracting these coordinates, the spatial distribution of all units is obtained. To define the spatial range of each unit, use its center point and coverage radius to represent its influence range. The spatial range is expressed as:

[0065] ;

[0066] where is the spatial region of unit , is the coverage radius, which is used to define the local influence range of the unit. After extracting the spatial region, analyze the spatial distribution pattern of each unit. By calculating the position statistical features of all units, such as the average center position and dispersion degree, summarize its distribution characteristics. The average position and dispersion degree are calculated as follows:

[0067] ;

[0068] ;

[0069] These statistical features form part of the spatial attribute feature vector, while also expanding to include other information such as the number and density of neighboring units, forming a complete spatial attribute feature vector for each unit. . Based on the spatial attribute feature vectors of each data unit, calculate the spatial straight-line distance and network topological distance between data units, and construct the original spatial distance relationship matrix. The spatial straight-line distance is calculated by the Euclidean distance between two points:

[0070] ;

[0071] The network topological distance depends on the shortest path connecting nodes and is calculated by the Dijkstra algorithm or the Floyd-Warshall algorithm. These distances are stored in the matrices and respectively, representing the straight-line distance relationship matrix and the network topological distance relationship matrix. Based on these distance relationships, construct a spatial connectivity graph between data processing units. The node set of the graph is , and the edge set is constructed by threshold screening. An edge is established between node pairs that satisfy the condition , where is the connection threshold. For each node in the connectivity graph, quantify its directional relationship with other nodes by calculating the direction angle and the direction cosine value for representation:

[0072] ;

[0073] After integrating these directional relationships, obtain the spatial correlation metric matrix , where each element represents the comprehensive spatial correlation between nodes and . To enhance the expression ability of spatial relationships, map the spatial relationships of data units into a high-dimensional feature space based on the spatial correlation metric matrix to generate the spatial semantic feature mapping matrix . Through multidimensional scaling or graph embedding methods, such as using the spectral embedding algorithm, the goal is to minimize the distance distortion between nodes:

[0074] ;

[0075] where and are the embedding vectors of nodes and in the high-dimensional feature space. Based on the spatial semantic feature mapping matrix , create an evaluation function that combines spatial proximity and functional similarity , to quantify the semantic correlation between data units:

[0076] ;

[0077] Among them, and are weight parameters that control the importance of spatial proximity and functional similarity. represents functional similarity, for example, calculated by cosine similarity:

[0078] ;

[0079] Through the evaluation function score all data unit pairs to generate a semantic association degree feature matrix of urban data , where represents the data unit and the semantic association degree between them.

[0080] In this embodiment, before extracting the longitude and latitude coordinates, defining the spatial region range, and analyzing the spatial distribution pattern of the geographical location information of each data processing unit in the standardized data processing unit set to obtain the spatial attribute feature vector, the following steps are also included: constructing a parameter-free semantic aggregator for the standardized data processing unit set, where each data unit is feature-grouped according to geographical location information, time information, and business type information, and the grouped features are input into the parameter-free semantic aggregator for feature recombination and semantic alignment to obtain an initial semantic feature matrix including spatial dimension, time dimension, and business dimension; constructing a task-aware prompt template based on the initial semantic feature matrix, setting corresponding prompt identifiers for spatial, time, and business type features respectively, and establishing independent semantic constraint conditions for each type of feature to obtain a multi-dimensional semantic category prompt set; inputting the multi-dimensional semantic category prompt set into a category-constrained semantic separator, and using a hierarchical semantic boundary division strategy to independently divide and boundary-constrain the semantic spaces of the spatial dimension, time dimension, and business dimension to obtain multi-dimensional category semantic constraint boundaries; performing feature separation on the initial semantic feature matrix based on the multi-dimensional category semantic constraint boundaries, projecting features of different dimensions onto corresponding semantic subspaces, and maintaining the semantic independence between dimensions to obtain a set of category-independent feature subspaces including spatial semantic subspaces, time semantic subspaces, and business semantic subspaces; performing deep feature learning on each subspace in the set of category-independent feature subspaces respectively, using an attention mechanism to capture the semantic associations within the dimension, and extracting the semantic features of spatial categories, time categories, and business categories to obtain a multi-dimensional category semantic feature matrix; inputting the multi-dimensional category semantic feature matrix into a feature fusion network, calculating the importance weights of features of different dimensions through a self-attention mechanism, and performing feature adaptive fusion based on the weights to obtain a multi-dimensional fused semantic feature matrix; performing dimension normalization and semantic alignment on the multi-dimensional fused semantic feature matrix, and using regularization constraints to ensure the scale consistency of features of different dimensions to obtain a standardized multi-dimensional semantic feature vector; performing channel-level feature fusion on the standardized multi-dimensional semantic feature vector and the original data processing unit features, constructing a semantic enhancement layer, and realizing the semantic enhancement of the data processing unit in the spatial dimension, time dimension, and business dimension to obtain a set of multi-dimensional semantic-enhanced data processing unit features.

[0081] In a specific embodiment, the process of performing the step of establishing a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix and quantitatively analyzing the spatial direction relationship of the nodes in the spatial connectivity graph to obtain a spatial correlation metric matrix may specifically include the following steps:

[0082] Setting a threshold for screening the distance values in the original spatial distance relationship matrix to obtain a spatial connectivity threshold matrix;

[0083] Construct a weighted undirected graph based on the spatial connectivity threshold matrix, assign weights to the connection relationships between nodes in the graph to obtain a spatial connectivity graph, and calculate the relative azimuth and direction cosine values of the nodes in the spatial connectivity graph to obtain a node direction feature matrix;

[0084] Calculate the spatial direction similarity between node pairs based on the node direction feature matrix to obtain a direction correlation matrix, perform normalization processing on the direction correlation matrix, and fuse it with the weight matrix of the spatial connectivity graph to obtain a comprehensive correlation degree matrix;

[0085] Calculate the degree centrality and betweenness centrality of the nodes in the graph based on the comprehensive correlation degree matrix to obtain node importance indicators;

[0086] Adjust the weights of the comprehensive correlation degree matrix according to the node importance indicators to obtain a weighted spatial correlation matrix, and perform symmetrization and standardization processing on the weighted spatial correlation matrix to obtain a spatial correlation metric matrix.

[0087] Specifically, assume that the original spatial distance relationship matrix is , where represents the spatial straight-line distance between node and node . By setting a threshold , retain all distance values, and set the part exceeding the threshold to no connection (such as represented by ). The elements of the spatial connectivity threshold matrix are defined as:

[0088] ;

[0089] Filter out those node pairs with too far distances and only retain the potentially associated nodes. Based on , construct a weighted undirected graph , where is the node set, is the edge set, is the edge weight set. Each node corresponds to a spatial unit, and the weight of the edge is the filtered distance value . For each edge, the assignment of the weight is expressed as:

[0090] ;

[0091] where is inversely proportional to the distance, indicating that node pairs with closer distances have stronger connectivity. Based on the constructed spatial connectivity graph, calculate the relative azimuth and direction cosine values of the nodes in the graph to generate a node direction feature matrix. Assume node and node The coordinates are respectively and , and the relative azimuth between them is calculated by the following formula:

[0092] ;

[0093] At the same time, the direction cosine value and the direction sine value are respectively:

[0094] ;

[0095] These values form the node direction feature matrix . Based on the node direction feature matrix , the spatial direction similarity between node pairs is calculated to generate the direction correlation matrix . The measurement of direction similarity is achieved through the cosine similarity formula, which is defined as:

[0096] ;

[0097] where and represent the relative azimuths of two node pairs. To unify the value range of the direction correlation matrix, is normalized, and all are mapped to the range [0, 1]. The normalization formula is:

[0098] ;

[0099] The normalized direction correlation matrix is fused with the weight matrix of the spatial connectivity graph to generate the comprehensive correlation degree matrix . The fusion method is achieved through the weighted sum formula:

[0100] ;

[0101] where and are weight parameters for adjusting the importance of connectivity and direction similarity. Based on the comprehensive correlation degree matrix , the degree centrality and betweenness centrality of the nodes in the graph are calculated to evaluate the importance of the nodes. The degree centrality represents the connection degree of node , and is defined as:

[0102] ;

[0103] The betweenness centrality Then measure the node 's ability to act as a bridging point in the entire graph, defined as:

[0104] ;

[0105] where represents the number of shortest paths from node to node , and represents the number of shortest paths passing through node . According to the node importance index, the weights of the comprehensive correlation matrix are adjusted to generate a weighted spatial correlation matrix . The adjusted weights are adjusted by the node importance index as:

[0106] ;

[0107] where and are adjustment parameters used to control the influence of node importance on the correlation degree. The weighted spatial correlation matrix is symmetrized and normalized to generate the final spatial correlation metric matrix . Symmetrization is achieved by taking the symmetric part of the matrix:

[0108] ;

[0109] Normalization is then completed by mapping the value range of the matrix to [0,1]. The spatial correlation metric matrix can represent the comprehensive correlation between nodes and takes into account the directional relationship and importance of nodes.

[0110] In a specific embodiment, the process of executing step 103 may specifically include the following steps:

[0111] Construct an initial graph structure based on the semantic correlation degree feature matrix of urban data, where each data processing unit serves as a node of the initial graph structure, and the semantic correlation degree between nodes serves as the edge weight of the initial graph structure;

[0112] Input the initial graph structure into the first graph attention layer in the multi-layer graph attention network for node feature projection transformation. The first graph attention layer contains 8 attention heads to obtain the first-layer local dependence features;

[0113] Input the first-layer local dependence features into the second graph attention layer in the multi-layer graph attention network. The second graph attention layer contains 16 attention heads, and each attention head independently calculates the attention score to obtain the correlation weight coefficient between nodes;

[0114] Perform a weighted aggregation operation on the first-layer local dependence features based on the association weight coefficients between nodes to obtain the second-layer global dependence features;

[0115] Input the second-layer global dependence features into the third-layer graph attention layer in the multi-layer graph attention network. The third-layer graph attention layer contains 32 attention heads, and each attention head uses positional encoding to enhance the position information of the nodes, obtaining a node feature vector matrix for the data processing task;

[0116] Calculate the cosine similarity between node pairs based on the node feature vector matrix to obtain the task association degree adjacency matrix.

[0117] Specifically, based on the urban data semantic association degree feature matrix , construct an initial graph structure , where is the node set, representing the data processing unit; is the edge set, representing the connection relationship between nodes; is the edge weight set, and the weight value is defined by the elements of the matrix . Assume the feature matrix , where represents the semantic association degree between node and node , then the edge weight of is directly assigned as . In this initial graph structure, the initial feature vector of each node is composed of the basic attributes of the input data, such as the geographical location and time characteristics of the node. Take the constructed initial graph structure as the input and pass it into the first-layer graph attention layer of the multi-layer graph attention network. The first-layer graph attention layer contains 8 independent attention heads, and each attention head independently calculates the attention scores between nodes. Let the features of node and node be and respectively, and the weight coefficient of attention head is calculated by the following formula:

[0118] ;

[0119] where, is the weight matrix of attention head , used to perform a linear transformation on the input features, is the attention vector, representing the specific directional parameter of head ; || represents the vector concatenation operation; LeakyReLU is the activation function, used to introduce non-linearity. For all adjacent nodes' Perform softmax normalization to obtain the attention weight coefficients:

[0120] ;

[0121] The attention weight coefficients are used to describe the relationship strength between node and node . Based on each attention head, the aggregated representation of the features of node is expressed as:

[0122] ;

[0123] where is the activation function, such as ReLU. The outputs of 8 attention heads are concatenated to obtain the local dependency features of the first layer:

[0124] ;

[0125] Input the local dependency features of the first layer into the second-layer graph attention network, which contains 16 attention heads. Each attention head independently calculates the attention scores , and performs a weighted aggregation operation on the first-layer features based on these scores to generate the global dependency features . Similar to the first layer, the operations of the second layer include linear transformation, attention score calculation, softmax normalization, and feature weighted aggregation. Due to the increase in the number of attention heads, more complex inter-node relationships can be captured. Input the global dependency features of the second layer into the third-layer graph attention network, which contains 32 attention heads, and enhance the position information of the nodes through position encoding. The position encoding is introduced into the feature representation in the form of addition or concatenation of the geographical coordinates or topological positions of the nodes. For example, the enhanced feature representation of node is expressed as:

[0126] ;

[0127] where represents the position encoding information of node . The outputs of 32 attention heads finally undergo concatenation or averaging operations to generate the output of the third layer, that is, the node feature vector matrix for the data processing task:

[0128] ;

[0129] Based on the node feature vector matrix , calculate the cosine similarity between node pairs to generate the task association adjacency matrix. Node and node Similarity is defined as:

[0130] ;

[0131] Arrange the similarities of all node pairs in matrix form to obtain the task correlation adjacency matrix .

[0132] In a specific embodiment, the process of executing step 104 may specifically include the following steps:

[0133] Perform feature embedding operations on the node feature vector matrix and the task correlation adjacency matrix to obtain the state vector of the scheduling environment, and perform state information feature extraction based on the state vector of the scheduling environment to obtain the deep state representation;

[0134] Input the deep state representation into the policy generator for state space mapping transformation. The policy generator contains 3 fully connected layers to obtain the continuous action vector, and establish a k-dimensional tree index structure based on the continuous action vector to discretize and partition the continuous space to obtain the discrete action space;

[0135] Input the candidate actions in the discrete action space into the value evaluation network to score the resource allocation effect of each action. The value evaluation network contains 2 convolutional layers and 3 fully connected layers to obtain the action value sequence;

[0136] Calculate the reward function value of task scheduling based on the action value sequence to obtain the scheduling evaluation score, and adopt the proximal policy optimization algorithm to optimize the policy according to the scheduling evaluation score to obtain the target scheduling policy;

[0137] Perform task assignment mapping on the target scheduling policy, assign the data processing tasks to be processed to the corresponding computing nodes, and generate the time sequence schedule for task processing to obtain the distributed scheduling scheme for data processing tasks.

[0138] Specifically, perform feature embedding operations on the node feature vector matrix and the task correlation adjacency matrix to obtain the state vector of the scheduling environment. Let the node feature vector matrix be , where represents the feature vector of the th node; the task correlation adjacency matrix is , where represents the correlation between task and task . By fusing these two matrices, construct the state vector of the scheduling environment. The following method is used for fusion:

[0139] ;

[0140] where, denotes the fused state vector matrix, denotes the matrix concatenation operation. In this way, the correlation degree information between the feature information of the node and the task is combined to form a scheduling environment representation. For the fused state vector matrix perform state information feature extraction to obtain a deep state representation. Use a deep neural network (DNN) to process . Let the parameters of the DNN be , then the deep state representation is expressed as:

[0141] ;

[0142] where denotes the DNN function with parameters . By training the DNN, useful feature representations are extracted from the fused state vectors to form a deep state representation. Input the deep state representation into the policy generator for state space mapping transformation. The policy generator consists of 3 fully connected layers. Let the weight matrix and bias of each layer be and ([[]] ), then the calculation process of the continuous action vector is as follows:

[0143] ;

[0144] ;

[0145] ;

[0146] where denotes the activation function, such as ReLU. Through the mapping of the policy generator, the deep state representation is converted into a continuous action vector , which is used for subsequent action selection. Based on the continuous action vector , build a k-dimensional tree (k-d tree) index structure to discretize the continuous space and obtain a discrete action space. The k-d tree is a data structure for storing and retrieving points in a multi-dimensional space, suitable for efficient discretization of the continuous action space. By constructing the k-d tree, the continuous action vector space is divided into several discrete regions, and each region corresponds to a discrete action. Input the candidate actions in the discrete action space into the value evaluation network to score the resource allocation effect of each action. The value evaluation network contains 2 convolutional layers and 3 fully connected layers. Let the feature representation of the th candidate action be , then the calculation process of its value score is as follows:

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] ;

[0152] Among them, represents the operation of the th convolutional layer, and ( ) represent the weight matrix and bias of the fully connected layer. Through the value evaluation network, each candidate action is scored to obtain an action value sequence. Based on the action value sequence, the reward function value of task scheduling is calculated to obtain a scheduling evaluation score. Let the value of the th action be , and its corresponding reward function value is expressed as:

[0153] ;

[0154] Among them, represents the reward function, which is used to map the value score to the reward value. The reward function includes linear functions, non-linear functions, etc. By calculating the reward value of each action, its effect in task scheduling is evaluated. The Proximal Policy Optimization algorithm is adopted to optimize the policy according to the scheduling evaluation score to obtain the target scheduling policy. Proximal Policy Optimization is a reinforcement learning algorithm that maintains the stability of training by restricting the amplitude of policy updates. Proximal Policy Optimization is optimized through the following objective function:

[0155] ;

[0156] Among them, represents the probability ratio of the old and new policies, represents the estimated value of the advantage function, is a hyperparameter used to limit the range of policy updates. By optimizing the above objective function, the updated policy parameters are obtained to form the target scheduling policy. Task assignment mapping is performed on the target scheduling policy. Assume that the scheduling policy outputs a task assignment decision matrix , where represents whether task is assigned to node (1 means assigned, 0 means not assigned), then the task assignment is expressed as:

[0157] ;

[0158] According to the task assignment matrix , combining the processing capabilities of computing nodes and the priorities of tasks, generate a timing schedule for task processing. Assume that the computing capacity of each node is (indicating the number of tasks that can be processed per unit time), and the processing requirement of task is , then the completion time of task on node is expressed as:

[0159] ;

[0160] By sorting the completion times of all tasks, generate a global task processing timing table . Adopt a greedy algorithm or dynamic programming method to minimize the total task completion time or maximize the system throughput while ensuring task dependencies. Based on the target scheduling strategy and the timing schedule, generate a complete distributed scheduling scheme for data processing tasks.

[0161] In a specific embodiment, the process of executing step 105 may specifically include the following steps:

[0162] Group and optimize the task sequence in the distributed scheduling scheme to obtain a node-level task assignment scheme, and divide the urban data semantic association degree feature matrix into matrix blocks according to a preset block size. Each sub-block contains the semantic features of adjacent regions to obtain a set of feature sub-blocks;

[0163] Perform parallel semantic mapping calculations on the set of feature sub-blocks, and at the same time use the block coordinate descent method to update the semantic mapping parameters to obtain local semantic mapping results;

[0164] Based on the local semantic mapping results, construct global consistency constraints, and use the alternating direction multiplier method to jointly optimize the semantic mapping parameters to obtain globally optimized semantic mapping parameters;

[0165] Based on the globally optimized semantic mapping parameters, perform semantic relationship calculations on multi-source heterogeneous spatio-temporal data to obtain a corresponding set of semantic mapping relationships.

[0166] Specifically, assume that the task sequence in the distributed scheduling scheme is , where represents the A data processing task. By analyzing the characteristics of the task, such as computing resource requirements, task priorities, and dependencies between tasks, the tasks are divided into multiple groups, and each group of tasks is assigned to a specific computing node. Let the set of computing nodes be , and each node has a resource capacity of . The goal of task allocation is to optimize node load balancing while satisfying resource constraints. Assume is a decision variable. When task is assigned to node , , otherwise . The task allocation problem is modeled as the following optimization problem:

[0167] ;

[0168] where, is the resource requirement of task . By solving the above optimization problem, a node-level task allocation scheme is generated. After completing the task grouping optimization, the urban data semantic association degree feature matrix is partitioned according to a preset block size. Assume that the size of each sub-block is , then the matrix is divided into sub-blocks, and each sub-block contains the semantic features of adjacent regions. The set of sub-blocks is represented as . Parallel semantic mapping calculations are performed on the set of feature sub-blocks. The semantic mapping calculation of each sub-block is regarded as an independent sub-problem, and the semantic mapping parameters are updated by minimizing the mapping error. Let the mapping parameter of sub-block be , and the objective function of the mapping error is:

[0169] ;

[0170] where, and are the feature vectors of nodes and , and represents the Euclidean norm. The block coordinate descent method is used to optimize , that is, fix other parameters and only optimize the mapping parameters of the current sub-block, and gradually reduce the mapping error through iterative updates. After obtaining the local semantic mapping results of each sub-block, in order to ensure the consistency of the mapping parameters between different sub-blocks, a global consistency constraint is constructed. The global consistency constraint is to match and constrain the boundary features shared by adjacent sub-blocks. Let and be the mapping parameters of two adjacent sub-blocks, and the constraint objective is:

[0171] ;

[0172] Among them, represents the feature vector of the boundary node. To optimize this global constraint, the Alternating Direction Method of Multipliers (ADMM) is used for joint optimization. The goal of ADMM is to decompose a complex global problem into several sub-problems that are easy to solve, and to achieve consistency optimization by introducing Lagrange multipliers. Let the global objective function be:

[0173] ;

[0174] Among them, is the constraint strength parameter, and the globally optimized semantic mapping parameter is obtained by optimizing . Based on the globally optimized semantic mapping parameter, semantic relationship calculation is performed on multi-source heterogeneous spatio-temporal data. A semantic mapping relationship set is generated based on the optimized mapping parameter. Let the semantic mapping relationship between nodes and be , which is defined as:

[0175] ;

[0176] Among them, is the globally optimized mapping parameter, and · represents the inner product of vectors. By calculating for all node pairs, a semantic mapping relationship set is generated.

[0177] The multi-source heterogeneous data semantic alignment method in the embodiments of the present invention is described above. Next, the multi-source heterogeneous data semantic alignment device in the embodiments of the present invention will be described. Please refer to Figure 2 . An embodiment of the multi-source heterogeneous data semantic alignment device in the embodiments of the present invention includes:

[0178] The acquisition module 201 is configured to acquire multi-source heterogeneous spatio-temporal data through a central data node and a plurality of edge nodes, and construct a set of standardized data processing units and a corresponding distributed data index directory;

[0179] The calculation module 202 is configured to perform spatial attribute feature extraction and inter-unit spatial correlation calculation on each data processing unit in the set of standardized data processing units, and obtain a city data semantic association degree feature matrix;

[0180] The feature extraction module 203 is configured to input the city data semantic association degree feature matrix into a multi-layer graph attention network for feature extraction, and obtain a node feature vector matrix and a task association degree adjacency matrix of the data processing task;

[0181] A generation module 204, configured to generate a distributed scheduling scheme for data processing tasks according to a node feature vector matrix and a task correlation adjacency matrix;

[0182] An optimization module 205, configured to allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on a city data semantic correlation degree feature matrix to obtain a set of semantic mapping relationships for multi-source heterogeneous spatio-temporal data.

[0183] Through the collaborative cooperation of the above-mentioned various components, by constructing a distributed edge computing network and a data acquisition topology structure, and combining multi-level data preprocessing and standardization mechanisms, the system can efficiently process multi-source heterogeneous spatio-temporal data, significantly improving the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of city data semantic correlation degree feature matrix is constructed, effectively solving the ambiguity problem in the semantic alignment process of traditional methods and improving the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and local and global dependencies between data units are captured through a multi-head attention mechanism, enhancing the system's expression ability for complex data structures; innovatively combining deep reinforcement learning with the k-dimensional tree nearest neighbor algorithm realizes the intelligent scheduling of data processing tasks and optimizes the allocation efficiency of computing resources; through the alternating iterative optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, improving the overall processing performance and scalability of the system.

[0184] Above Figure 2 The multi-source heterogeneous data semantic alignment device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the multi-source heterogeneous data semantic alignment device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0185] Figure 3FIG. 0 is a schematic structural diagram of a multi-source heterogeneous data semantic alignment device provided by an embodiment of the present invention. The multi-source heterogeneous data semantic alignment device 300 may vary greatly due to configuration or performance differences, and may include one or more processors (central processing units, CPUs) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 for storing application programs 333 or data 332 (for example, one or more mass storage device terminals). Among them, the memory 320 and the storage media 330 may be transient storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the multi-source heterogeneous data semantic alignment device 300. Further, the processor 310 may be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the multi-source heterogeneous data semantic alignment device 300 to implement the steps of the above multi-source heterogeneous data semantic alignment method.

[0186] The multi-source heterogeneous data semantic alignment device 300 may further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, and so on. Those skilled in the art can understand that Figure 3 the shown structural diagram of the multi-source heterogeneous data semantic alignment device does not limit the multi-source heterogeneous data semantic alignment device provided by the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0187] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is caused to execute the steps of the multi-source heterogeneous data semantic alignment method.

[0188] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described system, system, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0189] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0190] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of various embodiments of the present invention.

Claims

1. A semantic alignment method for multi-source heterogeneous data, characterized in that: The method comprises: Collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; Extract spatial attribute features and calculate spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a semantic correlation feature matrix of urban data; specifically, extract longitude and latitude coordinates, define spatial area scope and analyze spatial distribution pattern of the geographical location information of each data processing unit in the standardized data processing unit set to obtain a spatial attribute feature vector; based on the spatial attribute feature vector, calculate the spatial straight-line distance and network topological distance between data processing units to obtain an original spatial distance relationship matrix; establish a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix, and quantitatively analyze the spatial direction relationship of nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix; based on the spatial correlation measurement matrix, map the spatial relationship of data processing units to a high-dimensional feature space to obtain a spatial semantic feature mapping matrix; based on the spatial semantic feature mapping matrix, create an evaluation function that integrates spatial proximity and functional similarity, and score the semantic correlation between data processing units according to the evaluation function to obtain a semantic correlation feature matrix of urban data; Inputting the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtaining a node feature vector matrix and a task relevance adjacency matrix of the data processing task; Generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association adjacency matrix; The data processing tasks are allocated to the computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

2. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The method collects multi-source heterogeneous spatiotemporal data through a central data node and multiple edge nodes, and constructs a standardized data processing unit set and a corresponding distributed data index directory, including: Deploy the central data node and multiple edge nodes at urban data collection points, and build a data collection network topology map through a distributed edge computing network; Based on the data collection network topology diagram, data preprocessing is performed on the multi-source heterogeneous spatiotemporal data collected by the central data node and multiple edge nodes to obtain a cleaned standard data set; Establishing a time series index tree structure based on the data records in the cleaned standard data set, and performing a time window sliding mechanism analysis to obtain a data set with time series association; According to the spatial dimension characteristics, the data set with time series association is subjected to geographic grid division and spatial clustering to obtain time-space association data blocks, and metadata extraction and feature vector calculation are performed on the time-space association data blocks to obtain target calculation results; The target calculation results are stored in a distributed data directory to obtain a distributed data index directory, and the spatiotemporal correlation data blocks are adaptively segmented and boundary optimized according to a preset data processing unit size to obtain a standardized data processing unit set.

3. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The step of establishing a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix, and performing quantitative analysis on the spatial direction relationship of nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix includes: Setting a threshold for the distance values ​​in the original spatial distance relationship matrix to perform screening, and obtaining a spatial connectivity threshold matrix; A weighted undirected graph is constructed based on the spatial connectivity threshold matrix, and weights are assigned to the connection relationships between the nodes in the graph to obtain a spatial connectivity graph, and relative azimuth angles and direction cosine values ​​of the nodes in the spatial connectivity graph are calculated to obtain a node direction feature matrix; Calculating the spatial directional similarity between node pairs based on the node directional feature matrix to obtain a directional correlation matrix, normalizing the directional correlation matrix, and fusing it with the weight matrix of the spatial connectivity graph to obtain a comprehensive correlation matrix; Calculate the degree centrality and betweenness centrality of nodes in the graph based on the comprehensive association matrix to obtain node importance index; The comprehensive correlation matrix is ​​weighted according to the node importance index to obtain a weighted spatial correlation matrix, and the weighted spatial correlation matrix is ​​symmetric and standardized to obtain a spatial correlation measurement matrix.

4. The multi-source heterogeneous data semantic alignment method according to claim 1, characterized in that: The step of inputting the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction to obtain a node feature vector matrix and a task relevance adjacency matrix of the data processing task includes: Constructing an initial graph structure based on the semantic relevance feature matrix of the city data, wherein each data processing unit serves as a node of the initial graph structure, and the semantic relevance between nodes serves as an edge weight of the initial graph structure; Performing node feature projection transformation on the first graph attention layer in the multi-layer graph attention network inputted with the initial graph structure, wherein the first graph attention layer includes 8 attention heads, and obtaining the first layer of local dependency features; Inputting the first layer of local dependency features into the second layer of graph attention layer in the multi-layer graph attention network, the second layer of graph attention layer includes 16 attention heads, each attention head independently calculates the attention score, and obtains the correlation weight coefficient between nodes; Performing a weighted aggregation operation on the first layer of local dependency features based on the association weight coefficients between the nodes to obtain a second layer of global dependency features; Input the second-layer global dependency features into the third-layer graph attention layer in the multi-layer graph attention network, where the third-layer graph attention layer includes 32 attention heads, and each attention head uses position encoding to enhance the position information of the node, so as to obtain a node feature vector matrix of the data processing task; The cosine similarity between node pairs is calculated based on the node feature vector matrix to obtain a task association adjacency matrix.

5. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The distributed scheduling scheme for generating data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix includes: Performing feature embedding operations on the node feature vector matrix and the task association degree adjacency matrix to obtain a state vector of the scheduling environment, and performing state information feature extraction based on the state vector of the scheduling environment to obtain a deep state representation; The deep state representation is input into a strategy generator for state space mapping conversion. The strategy generator includes three fully connected layers to obtain a continuous action vector, and a k-dimensional tree index structure is established based on the continuous action vector to discretize the continuous space and obtain a discrete action space; Inputting the candidate actions in the discrete action space into a value evaluation network, scoring the resource allocation effect of each action, wherein the value evaluation network comprises two convolutional layers and three fully connected layers, and obtaining an action value sequence; Calculating the reward function value of the task scheduling based on the action value sequence to obtain a scheduling evaluation score, and using a proximal strategy optimization algorithm to perform strategy optimization according to the scheduling evaluation score to obtain a target scheduling strategy; The target scheduling strategy is mapped to task allocation, the data processing tasks to be processed are allocated to the corresponding computing nodes, and a time sequence schedule for task processing is generated to obtain a distributed scheduling solution for the data processing tasks.

6. The multi-source heterogeneous data semantic alignment method according to claim 1, characterized in that: The data processing tasks are allocated to computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic association feature matrix of the city data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data, including: The task sequence in the distributed scheduling scheme is grouped and optimized to obtain a node-level task allocation scheme, and the city data semantic relevance feature matrix is ​​divided into blocks according to a preset block size, each sub-block contains semantic features of adjacent areas, and a feature sub-block set is obtained; Performing parallel semantic mapping calculation on the feature sub-block set, and updating semantic mapping parameters by using block coordinate descent method to obtain local semantic mapping results; Building a global consistency constraint based on the local semantic mapping result, and jointly optimizing the semantic mapping parameters using an alternating direction multiplier method to obtain globally optimized semantic mapping parameters; Based on the globally optimized semantic mapping parameters, semantic relationship calculation is performed on the multi-source heterogeneous spatiotemporal data to obtain a corresponding semantic mapping relationship set.

7. A multi-source heterogeneous data semantic alignment device, characterized in that: The device for executing the semantic alignment method of multi-source heterogeneous data according to any one of claims 1 to 6 comprises: The acquisition module is used to collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; A calculation module is used to extract spatial attribute features and calculate spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a semantic correlation feature matrix of urban data; specifically comprising: extracting longitude and latitude coordinates, defining spatial area scope and analyzing spatial distribution patterns of the geographical location information of each data processing unit in the standardized data processing unit set to obtain a spatial attribute feature vector; based on the spatial attribute feature vector, calculating the spatial straight-line distance and network topological distance between the data processing units to obtain an original spatial distance relationship matrix; based on the original spatial distance relationship matrix, establishing a spatial connectivity graph between the data processing units, and quantitatively analyzing the spatial direction relationship of the nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix; based on the spatial correlation measurement matrix, mapping the spatial relationship of the data processing units to a high-dimensional feature space to obtain a spatial semantic feature mapping matrix; based on the spatial semantic feature mapping matrix, creating an evaluation function that integrates spatial proximity and functional similarity, and scoring the semantic correlation between the data processing units according to the evaluation function to obtain a semantic correlation feature matrix of urban data; A feature extraction module is used to input the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtain a node feature vector matrix and a task relevance adjacency matrix of the data processing task; A generation module, used for generating a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix; The optimization module is used to allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

8. A multi-source heterogeneous data semantic alignment device, characterized in that: The multi-source heterogeneous data semantic alignment device comprises: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instruction in the memory so that the multi-source heterogeneous data semantic alignment device executes the multi-source heterogeneous data semantic alignment method as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the multi-source heterogeneous data semantic alignment method as described in any one of claims 1-6 is implemented.

Images