Multi-source heterogeneous data semantic alignment method and device and computer equipment

Abstract

The invention relates to the technical field of data processing, and discloses a multi-source heterogeneous data semantic alignment method and device and computer equipment.The method comprises the steps that multi-source heterogeneous spatio-temporal data are collected through a center data node and a plurality of edge nodes, spatial attribute feature extraction and inter-unit spatial correlation calculation are carried out, and multi-source heterogeneous spatial data are obtained; obtaining a city data semantic association degree feature matrix; inputting the urban data semantic association degree feature matrix into a multilayer graph attention network for feature extraction to obtain a node feature vector matrix and a task association degree adjacency matrix; generating a distributed scheduling scheme of the data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix; according to the method, data processing tasks are distributed to computing nodes according to a distributed scheduling scheme, alternate iterative optimization is carried out based on a city data semantic association degree feature matrix, a semantic mapping relation set is obtained, and intelligent scheduling of the data processing tasks and efficient semantic alignment in a distributed environment are achieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and computer equipment for semantic alignment of multi-source heterogeneous data. Background Art

[0002] With the rapid development of smart city construction, the collection, processing and storage of spatiotemporal big data have become key technical challenges. The traditional centralized data processing architecture is difficult to meet the real-time processing needs of massive heterogeneous data, and the emergence of distributed edge computing technology provides a new approach to solving this problem. However, multi-source heterogeneous data has differences and uncertainties at the semantic level, which brings huge challenges to data fusion and semantic alignment.

[0003] At present, although there are a variety of smart city semantic correlation coefficient methods for processing the semantic association of spatiotemporal big data, these methods often produce deviations when processing specific types of data and fail to fully utilize the characteristics of spatiotemporal data. In addition, existing semantic alignment methods may not meet the semantic consistency conditions, resulting in reduced data fusion reliability and inability to effectively eliminate semantic ambiguity. In terms of data processing scheduling, due to the limitations of heuristic algorithms and the adaptability challenges of heterogeneous data processing, existing scheduling algorithms are difficult to meet the needs of smart city AI platforms. Especially when processing heterogeneous data with complex spatiotemporal characteristics, traditional scheduling methods cannot effectively balance the allocation of computing resources and data processing efficiency, resulting in a decrease in the overall performance of the system. Summary of the invention

[0004] The present invention provides a method, device and computer equipment for semantic alignment of multi-source heterogeneous data, which realizes intelligent scheduling of data processing tasks and efficient semantic alignment in a distributed environment.

[0005] In a first aspect, the present invention provides a method for semantic alignment of multi-source heterogeneous data, the method comprising: Collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; Extracting spatial attribute features and calculating spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a semantic correlation feature matrix of urban data; Inputting the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtaining a node feature vector matrix and a task relevance adjacency matrix of the data processing task; Generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association adjacency matrix; The data processing tasks are allocated to the computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

[0006] In a second aspect, the present invention provides a multi-source heterogeneous data semantic alignment device, the multi-source heterogeneous data semantic alignment device comprising: The acquisition module is used to collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; A calculation module, used for extracting spatial attribute features and calculating spatial correlation between units for each data processing unit in the standardized data processing unit set, to obtain a semantic correlation feature matrix of urban data; A feature extraction module is used to input the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtain a node feature vector matrix and a task relevance adjacency matrix of the data processing task; A generation module, used for generating a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix; The optimization module is used to allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

[0007] The third aspect of the present invention provides a multi-source heterogeneous data semantic alignment device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the multi-source heterogeneous data semantic alignment device executes the above-mentioned multi-source heterogeneous data semantic alignment method.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned multi-source heterogeneous data semantic alignment method.

[0009] In the technical solution provided by the present invention, by constructing a distributed edge computing network and a data acquisition topology structure, combined with a multi-level data preprocessing and standardization mechanism, the system can efficiently process multi-source heterogeneous spatiotemporal data, and significantly improve the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of urban data semantic correlation feature matrix is ​​constructed, which effectively solves the ambiguity problem of traditional methods in the semantic alignment process and improves the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and the local and global dependencies between data units are captured through a multi-head attention mechanism, thereby enhancing the system's ability to express complex data structures; deep reinforcement learning is innovatively combined with the k-dimensional tree nearest neighbor algorithm to realize intelligent scheduling of data processing tasks and optimize the allocation efficiency of computing resources; through the alternating iterative optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, and the overall processing performance and scalability of the system are improved.

[0010] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0011] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of an embodiment of a method for semantic alignment of multi-source heterogeneous data in an embodiment of the present invention; Figure 2 A schematic diagram of an embodiment of a multi-source heterogeneous data semantic alignment device in an embodiment of the present invention; Figure 3 It is a schematic diagram of an embodiment of a multi-source heterogeneous data semantic alignment device in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0015] To facilitate understanding of this embodiment, a multi-source heterogeneous data semantic alignment method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Collect multi-source heterogeneous spatiotemporal data through a central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; It is understandable that the execution subject of the present invention may be a multi-source heterogeneous data semantic alignment device, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0016] Specifically, the central data node and multiple edge nodes are deployed at different data collection points in the city to cover a wide range of space, and the distributed characteristics of edge computing are fully utilized to build an efficient data collection network topology. In the topology, each node undertakes a certain collection task and communicates with each other through a low-latency network connection, forming a distributed architecture that can efficiently collect, transmit and process multi-source heterogeneous data. Through this topology, the central data node and multiple edge nodes collect heterogeneous spatiotemporal data from multiple sources (such as sensors, traffic equipment, weather stations, etc.). These raw data are preprocessed, including cleaning, deduplication, format conversion and normalization operations, to generate a cleaned standard data set. A time series index tree structure is established based on the data records in the cleaned standard data set to effectively manage and organize these data. The time series index tree uses time as a clue to quickly locate and retrieve relevant data records, and combines the time window sliding mechanism for analysis to mine the time series correlation of the data. The time window sliding mechanism captures the changing trend of data in different time periods by dynamically adjusting the time span, revealing the time series pattern hidden in the data. According to the spatial dimension characteristics, the data set with time series correlation is geographically gridded and spatially clustered. Geographic grid division divides the entire urban area into multiple grid cells, each of which represents a specific geographic range. Through spatial clustering, similar data points are aggregated according to the spatial distribution characteristics of the data to generate spatiotemporal correlation data blocks with clear spatial boundaries and attribute characteristics. Metadata extraction and feature vector calculation are performed on the spatiotemporal correlation data blocks to obtain the target calculation results. The metadata extraction process includes summarizing and recording the key attributes of each data block (such as time range, spatial range, data type, etc.), while the feature vector calculation generates a multi-dimensional numerical representation based on the content of the data block. The target calculation results are stored in a distributed data directory, and a distributed data index directory is built based on this. Adaptive segmentation and boundary optimization are performed on the spatiotemporal correlation data blocks according to the preset data processing unit size. Adaptive segmentation dynamically adjusts the division method of the data block to adapt it to different data processing requirements; while boundary optimization improves data utilization and processing efficiency by reducing overlap and redundancy between data blocks. Finally, a standardized set of data processing units is formed.

[0017] 102. Extract spatial attribute features and calculate spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a semantic correlation feature matrix of urban data; Specifically, the geographic location information of each data processing unit is extracted, including the latitude and longitude coordinates, the spatial area range, and its distribution pattern in the overall space. By analyzing the geographic attributes, the spatial attribute feature vectors of each unit are generated. These feature vectors describe the spatial position, size, and geometric form of each data unit in a numerical form. Based on the spatial attribute feature vectors, the spatial straight-line distance and network topological distance between the data processing units are calculated. The spatial straight-line distance can reflect the physical position relationship of the data unit in the geographic coordinate system, while the network topological distance considers the path length in the actual transportation or communication network and has a higher practical reference value. Through distance calculation, a spatial distance relationship matrix is ​​generated. The spatial connectivity graph between the data processing units is constructed based on the original spatial distance relationship matrix. The data units are regarded as nodes in the graph, and edge connections are established according to their distance relationships to form a graph model that can reflect the spatial relationship of the data units. On this basis, the spatial directional relationship of each node in the graph is quantitatively analyzed, including the directional angle between nodes, the distribution pattern of adjacent nodes, and the directional distribution of connection weights. The quantitative analysis results are integrated into a spatial correlation measurement matrix to describe the degree of spatial correlation between nodes. The spatial semantic feature mapping matrix of the data unit is generated by mapping the information in the spatial correlation measurement matrix to the high-dimensional feature space. The mapping process uses a combination of dimensionality reduction technology and embedding methods to extract high-dimensional semantic features while retaining the original spatial relationship information. For example, a graph embedding method based on deep learning is used to embed complex spatial relationships into feature vectors to generate a high-dimensional mapping matrix with semantic expression capabilities. Based on the spatial semantic feature mapping matrix, a comprehensive evaluation function that integrates spatial proximity and functional similarity is designed. Spatial proximity reflects the geographical proximity of data units, while functional similarity focuses on the consistency of units in semantic attributes, such as whether adjacent areas have similar functional uses or performance characteristics. The evaluation function scores the semantic correlation between data units by combining these two dimensions to generate a comprehensive semantic correlation measure. The above evaluation function is used to score the correlation of all data unit pairs, and the results are organized into a semantic correlation feature matrix for urban data.

[0018] A threshold is set for the distance value in the original spatial distance relationship matrix to screen out node pairs with significant spatial connectivity. The selection of the threshold is based on the spatial distance characteristics in the actual application scenario, ensuring that only those node pairs with spatial distances within a reasonable range are retained, and a screened spatial connectivity threshold matrix is ​​obtained. A weighted undirected graph is constructed based on the spatial connectivity threshold matrix. The nodes of the graph represent standardized data processing units, and the edges in the graph represent the spatial connection relationships between these units. In order to reflect the importance of different connection relationships, each edge is assigned a weight, and the weight value is calculated inversely according to the distance value of the corresponding node pair in the spatial connectivity threshold matrix. The smaller the distance, the higher the weight of the node pair, highlighting a closer spatial relationship. After constructing the spatial connectivity graph, the relative azimuth and direction cosine values ​​between each node in the graph and its adjacent nodes are calculated to generate a node direction feature matrix. The relative azimuth represents the angular difference between two nodes in space, and the direction cosine value quantifies the numerical characteristics of the spatial direction relationship. Based on the node direction feature matrix, the spatial direction similarity between node pairs is calculated. The consistency of the direction relationship between node pairs is quantified by the cosine similarity formula to generate a direction correlation matrix. The directional correlation matrix is ​​normalized to limit all values ​​to a standard range so that it can be effectively integrated with the weight matrix of the spatial connectivity graph. The comprehensive correlation matrix is ​​obtained by combining the normalized directional correlation matrix with the weight matrix of the spatial connectivity graph. The degree centrality and betweenness centrality of the nodes in the graph are calculated based on the comprehensive correlation matrix to quantify the criticality of the nodes in the graph structure. Degree centrality describes the degree of connection of the node and reflects the number of nodes directly adjacent to a node, while betweenness centrality measures the intermediary role of the node in the information transmission process of the whole graph. Through these centrality indicators, the key nodes with greater influence on spatial connectivity are identified. According to the importance index of the node, the weights in the comprehensive correlation matrix are adjusted to give more important nodes higher weight values ​​to generate a weighted spatial correlation matrix. The weighted spatial correlation matrix is ​​symmetric, that is, the matrix is ​​converted into a symmetric matrix to eliminate the imbalance caused by directionality. At the same time, the matrix is ​​standardized so that its values ​​are distributed in a uniform range for subsequent calculation and optimization, and finally a spatial correlation measurement matrix is ​​generated.

[0019] 103. Input the city data semantic relevance feature matrix into the multi-layer graph attention network for feature extraction, and obtain the node feature vector matrix and task relevance adjacency matrix of the data processing task; Specifically, an initial graph structure is constructed based on the semantic association feature matrix of urban data. Each row or column of the semantic association feature matrix of urban data represents a data processing unit. These units serve as nodes of the initial graph structure, and the semantic association values ​​in the matrix serve as edge weights between nodes to form a weighted graph model. In this initial graph structure, nodes represent data processing tasks, and edge weights represent the strength of semantic association between tasks. The initial graph structure is input into the first graph attention layer of the multi-layer graph attention network for feature projection transformation. The first graph attention layer performs weighted processing on node features through a multi-head attention mechanism to capture local dependency features between nodes. In this layer, 8 independent attention heads are designed. Each attention head calculates the attention score for the features of adjacent nodes, dynamically adjusts the weight distribution of each node pair, and generates a local dependency feature representation for each node. The local dependency features generated in the first layer are input into the second graph attention layer of the multi-layer graph attention network. The second graph attention layer enhances the feature extraction capability by expanding the number of attention heads to 16. In this layer, each attention head independently calculates the attention score between nodes, and quantifies the association weight coefficient between them by comparing the feature relationships of different nodes. The weight coefficient reflects the strength of the semantic relevance of the node pair. Based on the association weight coefficient, the local dependency features generated in the first layer are weighted and aggregated to integrate the node's adjacent features and obtain the global dependency feature representation of the second layer. Based on the second layer features, the global dependency features are input into the third layer of the multi-layer graph attention network. The third layer of the graph attention layer improves the network's expressiveness and achieves higher-dimensional feature aggregation by introducing 32 attention heads. Each attention head enhances the node by position encoding, so that the node's position information is explicitly added to the node feature. The integration of position information helps capture the spatial distribution and topological relationship of the node in the graph structure, so that the third layer of features can more accurately express the semantic status of the node in the whole graph. After this layer of calculation, the node feature vector matrix of the data processing task is obtained. Each row of the matrix represents a high-dimensional feature vector of a node, which fully encodes the semantic information, neighborhood information and spatial position characteristics of the node. The cosine similarity between node pairs is calculated based on the node feature vector matrix to quantify the degree of semantic similarity between them. Cosine similarity effectively measures the semantic relevance strength of node pairs by comparing the directional consistency of the feature vectors of two nodes, and obtains the task relevance adjacency matrix.

[0020] 104. Generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association adjacency matrix; Specifically, feature embedding operations are performed on the node feature vector matrix and the task association adjacency matrix to map their high-dimensional structural features into a low-dimensional continuous vector space to generate the state vector of the scheduling environment. State information features are extracted based on the state vector of the scheduling environment. Through the deep learning model, the implicit features and complex relationships in the state vector are automatically captured to generate a deep state representation. The deep state representation is input into the policy generator for state space mapping conversion. The core structure of the policy generator contains three fully connected layers, which transform the input deep state representation layer by layer through nonlinear activation functions and output a continuous action vector. The continuous action vector defines the potential action range of the scheduling strategy in the continuous space. Based on the continuous action vector, a k-dimensional tree index structure is established to discretize the continuous space into several limited action options to generate a discrete action space, so that the scheduling strategy can be optimized and selected in a discrete but still highly expressive space. In order to evaluate the pros and cons of candidate actions in the discrete action space, the candidate actions are input into the value evaluation network to score the resource allocation effect. The value evaluation network consists of two convolutional layers and three fully connected layers. The local pattern of action features is extracted through the convolutional layer, and the information of each dimension is integrated through the fully connected layer to generate the value score sequence of the action. The value score of each action reflects its potential effect in resource allocation and task completion. The action with a higher score can better optimize the scheduling target in the current state. Based on the action value sequence, the reward function value of task scheduling is calculated. The reward function measures the key indicators such as task execution efficiency, node resource utilization and task completion time to obtain the evaluation score of the scheduling scheme. The current strategy is optimized based on this evaluation score. In order to achieve efficient strategy optimization, the proximal strategy optimization algorithm is adopted to balance stability and exploration by limiting the scope of strategy update, gradually optimize the scheduling strategy, and generate the target scheduling strategy. The target scheduling strategy is applied to the task allocation mapping. The target scheduling strategy generates corresponding allocation decisions for each data processing task, allocates tasks to computing nodes with matching computing power, location and resource conditions, and generates a task processing time schedule, and finally obtains a distributed scheduling scheme for data processing tasks.

[0021] 105. The data processing tasks are assigned to the computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic correlation feature matrix of urban data to obtain the semantic mapping relationship set of multi-source heterogeneous spatiotemporal data.

[0022] Specifically, according to the distributed scheduling scheme, the task sequence is grouped and optimized to generate a node-level task allocation scheme. The semantic relevance between tasks, the resource constraints of computing nodes, and the priority of task execution are comprehensively considered to ensure that the allocation of tasks on nodes has high efficiency and load balancing characteristics. After completing the task grouping optimization, the city data semantic relevance feature matrix is ​​divided into blocks according to the preset block size. Each sub-block contains the semantic features of adjacent regions, which can retain the spatial semantic continuity between regions to a certain extent. The block division process aims to reduce the complexity of global calculation, decompose the large-scale matrix into small-scale sub-blocks that are easy to handle, and realize parallel calculation. Parallel semantic mapping calculation is performed on the feature sub-block set. The block coordinate descent method is used to update the semantic mapping parameters. The block coordinate descent method gradually approaches the global optimal solution in a step-by-step manner by fixing some parameters and optimizing the remaining parameters. In parallel calculation, each computing node independently processes one or more feature sub-blocks and obtains preliminary semantic mapping results through local calculation. The local semantic mapping results contain the mapping parameters of each feature sub-block, which reflect the semantic relationship between data in a specific area. Global consistency constraints are constructed based on the local semantic mapping results. The role of global consistency constraints is to coordinate the semantic mapping parameters between different feature sub-blocks so that the overall semantic mapping model can take into account both the global perspective and local details. In this process, the alternating direction multiplier method (ADMM) is used to jointly optimize the semantic mapping parameters. The ADMM method decomposes the global optimization problem into multiple easy-to-solve sub-problems and introduces Lagrange multipliers for constraint adjustment to achieve efficient optimization of global consistency and obtain globally optimized semantic mapping parameters. Semantic relationships are calculated for multi-source heterogeneous spatiotemporal data based on the globally optimized semantic mapping parameters. The optimized mapping parameters are applied to actual data to mine the semantic associations between data and generate a semantic mapping relationship set, which contains mapping rules and semantic links between data.

[0023] In the embodiment of the present invention, by constructing a distributed edge computing network and a data acquisition topology structure, combined with a multi-level data preprocessing and standardization mechanism, the system can efficiently process multi-source heterogeneous spatiotemporal data, significantly improving the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of urban data semantic correlation feature matrix is ​​constructed, which effectively solves the ambiguity problem of traditional methods in the semantic alignment process and improves the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and the local and global dependencies between data units are captured through a multi-head attention mechanism, thereby enhancing the system's ability to express complex data structures; innovatively combining deep reinforcement learning with the k-dimensional tree nearest neighbor algorithm, realizing intelligent scheduling of data processing tasks and optimizing the allocation efficiency of computing resources; through the alternating iterative optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, improving the overall processing performance and scalability of the system.

[0024] In a specific embodiment, the process of executing step 101 may specifically include the following steps: Deploy the central data node and multiple edge nodes at urban data collection points, and build a data collection network topology map through a distributed edge computing network; Based on the data collection network topology diagram, data preprocessing is performed on the multi-source heterogeneous spatiotemporal data collected by the central data node and multiple edge nodes to obtain a cleaned and standardized data set; A time series index tree structure is established based on the data records in the cleaned standard data set, and a time window sliding mechanism analysis is performed to obtain a data set with time series association; According to the spatial dimension characteristics, the dataset with time series correlation is divided into geographic grids and spatial clusters to obtain spatiotemporal correlation data blocks, and metadata extraction and feature vector calculation are performed on the spatiotemporal correlation data blocks to obtain the target calculation results; The target calculation results are stored in a distributed data directory to obtain a distributed data index directory, and the spatiotemporal correlation data blocks are adaptively segmented and boundary optimized according to the preset data processing unit size to obtain a standardized data processing unit set.

[0025] Specifically, the central data node and multiple edge nodes are deployed at different data collection points in the city to build a distributed edge computing network topology. The central node is set in a data center with high-performance computing capabilities, while the edge nodes are deployed in data-intensive areas. The edge nodes collect local multi-source heterogeneous spatiotemporal data, such as traffic flow data, meteorological data, public facility sensor data, etc. In the topology construction, nodes and edges represent data collection points and the communication links between them, respectively. is a topological graph, where is a collection of nodes, is the edge set, is the set of edge weights. The edge weights are defined based on communication delay or data transmission rate, for example, The formula is: ; in, Representation Node and nodes The bandwidth between represents the communication delay between them. This formula can effectively quantify the quality of the communication link and facilitate the subsequent optimization of data transmission. Based on the data acquisition network topology diagram, the multi-source heterogeneous spatiotemporal data collected by the edge nodes are preprocessed, including data cleaning, missing value filling, format conversion, and unified timestamp processing. For example, for traffic flow data, the missing time period is filled by interpolation; for meteorological data, the data formats generated by different devices are standardized and unified to The four-tuple form of is the timestamp, is the geographic coordinate, is the observed value. After processing, a cleaned standard data set is generated. A time series index tree structure is established based on the data records in the standard data set to support efficient query. Assume that the standard data set contains records, each with a timestamp of The time series index tree organizes these records into a balanced tree structure in chronological order, with each node storing a time range. and the corresponding data entries. The query complexity is controlled by optimizing the depth of the tree: At the same time, a time window sliding mechanism is introduced to analyze the temporal correlation of data. The size of the sliding window is , the data set in the window is: ; in, The timestamp is The sliding mechanism can dynamically capture the time series pattern in a short period of time, such as the traffic flow change trend during peak hours. After completing the time series processing, the dataset with time series association is divided into geographic grids and spatial clusters according to the spatial dimension characteristics. Suppose the geographic scope of the entire urban area is , which is divided into grid cells, each with a size of: ; After assigning each data record to the corresponding grid, the data points in each grid are density clustered (e.g., DBSCAN algorithm) to generate spatiotemporal correlation data blocks. Each data block contains data records that are close in space and adjacent in time, reflecting the dynamic behavior characteristics in the area. Metadata extraction and feature vector calculation are performed on the spatiotemporal correlation data blocks. The metadata includes the time range, geographic range, mean value, standard deviation, etc. of the data block, and the feature vector is expressed in a multidimensional form, for example: ; in, and are the geographic coordinates of the center of the data block, and is the standard deviation of the spatial distribution, and It is the eigenvalue of the covariance matrix, reflecting the main direction and discreteness of the data distribution. The target calculation results are stored in the distributed data directory to support efficient retrieval and distributed computing. Adaptive segmentation and boundary optimization are performed on the spatiotemporal correlation data blocks to ensure that the size of the data blocks is adapted to the computing power of different nodes. The segment size is dynamically adjusted according to the following formula: ; in, is the segment size, and are the maximum and minimum block sizes, respectively, is the computing power of the target node, is the average computing power of the entire network, is the average block size of the entire network. Through the above process, a standardized set of data processing units is finally generated.

[0026] In a specific embodiment, the process of executing step 102 may specifically include the following steps: Extracting longitude and latitude coordinates, defining spatial area ranges and analyzing spatial distribution patterns of the geographical location information of each data processing unit in the standardized data processing unit set, and obtaining a spatial attribute feature vector; Based on the spatial attribute feature vector, the spatial straight-line distance and network topological distance between data processing units are calculated to obtain the original spatial distance relationship matrix; Based on the original spatial distance relationship matrix, a spatial connectivity graph between data processing units is established, and the spatial direction relationship of nodes in the spatial connectivity graph is quantitatively analyzed to obtain a spatial correlation measurement matrix; Based on the spatial correlation measurement matrix, the spatial relationship of the data processing unit is mapped to the high-dimensional feature space to obtain the spatial semantic feature mapping matrix; Based on the spatial semantic feature mapping matrix, an evaluation function integrating spatial proximity and functional similarity is created. The semantic relevance between data processing units is scored according to the evaluation function to obtain the semantic association feature matrix of urban data.

[0027] Specifically, the geographical location information of each data processing unit in the standardized data processing unit set is extracted. Each data unit records the longitude and latitude coordinates. Let the unit set be , where each unit Contains attributes , respectively represent their longitude and latitude coordinates. By extracting these coordinates, the spatial distribution of all cells is obtained. In order to define the spatial area range of each cell, its center point is used and coverage radius Indicates its scope of influence, and the spatial range is expressed as: ; in For unit space region, is the coverage radius, which is used to define the local influence range of the unit. After extracting the spatial region, analyze the spatial distribution pattern of each unit. By calculating the position statistics of all units, such as the average center position and the degree of dispersion, the distribution characteristics are summarized. The average position and the degree of dispersion The calculation formula is as follows: ; ; These statistical features constitute part of the spatial attribute feature vector, and are also extended to include other information, such as the number and density of neighboring units, to form a complete spatial attribute feature vector for each unit. Based on the spatial attribute feature vector of each data unit, the spatial straight-line distance and network topological distance between data units are calculated to construct the original spatial distance relationship matrix. Calculated by the Euclidean distance between two points: ; Network topology distance It depends on the shortest path connecting the nodes, calculated by Dijkstra's algorithm or Floyd-Warshall's algorithm. These distances are stored in the matrix and In the graph, they represent the linear distance relationship matrix and the network topology distance relationship matrix respectively. Based on these distance relationships, a spatial connectivity graph between data processing units is constructed. The node set of the graph is , edge set Then, the threshold is filtered and constructed to meet the conditions Node pair Build edges between is the connection threshold. For each node in the connectivity graph, quantify its directional relationship with other nodes by calculating the direction angle and direction cosines To express: ; After integrating these directional relationships, we get the spatial correlation measure matrix , where each element Representation Node and In order to improve the expression ability of spatial relationships, the spatial relationship of data units is mapped to a high-dimensional feature space based on the spatial correlation measurement matrix to generate a spatial semantic feature mapping matrix. Through multidimensional scaling or graph embedding methods, such as using the spectral embedding algorithm, the goal is to minimize the distance distortion between nodes: ; in and Is a node and Embedding vector in high-dimensional feature space. Based on spatial semantic feature mapping matrix , create an evaluation function that combines spatial proximity and functional similarity , to quantify the semantic relevance between data units: ; in, and is a weight parameter that controls the importance of spatial proximity and functional similarity, Represents functional similarity, for example, calculated by cosine similarity: ; By evaluating the function Score all data unit pairs to generate a semantic correlation feature matrix for urban data ,in Represents data unit and The semantic relevance between them.

[0028] In this embodiment, before extracting longitude and latitude coordinates, defining the spatial area range, and analyzing the spatial distribution pattern of the geographic location information of each data processing unit in the standardized data processing unit set to obtain the spatial attribute feature vector, it also includes: constructing a parameter-free word meaning aggregator for the standardized data processing unit set, wherein each data unit is feature grouped according to the geographic location information, time information, and business type information, and the grouped features are input into the parameter-free word meaning aggregator for feature reorganization and semantic alignment to obtain an initial word meaning feature matrix containing spatial dimension, time dimension, and business dimension; constructing a task-aware prompt template based on the initial word meaning feature matrix, setting corresponding prompt identifiers for spatial, temporal, and business class features, respectively, and establishing independent semantic constraints for each type of feature to obtain a multi-dimensional semantic category prompt set; inputting the multi-dimensional semantic category prompt set into the category constraint semantic separator, adopting a hierarchical semantic boundary division strategy to independently divide and boundary constrain the semantic space of the spatial dimension, the temporal dimension, and the business dimension to obtain a multi-dimensional category semantic constraint boundary; feature dividing the initial word meaning feature matrix based on the multi-dimensional category semantic constraint boundary The features of different dimensions are projected to the corresponding semantic subspaces, and the semantic independence between dimensions is maintained, so as to obtain a set of category-independent feature subspaces including spatial semantic subspaces, temporal semantic subspaces and business semantic subspaces; deep feature learning is performed on each subspace in the set of category-independent feature subspaces, and the attention mechanism is used to capture the semantic association within the dimension, and the semantic features of spatial categories, temporal categories and business categories are extracted to obtain a multi-dimensional category semantic feature matrix; the multi-dimensional category semantic feature matrix is ​​input into the feature fusion network, and the importance weights of features of different dimensions are calculated through the self-attention mechanism, and the features are adaptively fused based on the weights to obtain a multi-dimensional fusion semantic feature matrix; the multi-dimensional fusion semantic feature matrix is ​​dimensionally normalized and semantically aligned, and regularization constraints are used to ensure the scale consistency of features of different dimensions to obtain a standardized multi-dimensional semantic feature vector; channel-level feature fusion is performed on the standardized multi-dimensional semantic feature vector and the original data processing unit features to construct a semantic enhancement layer, realize the semantic enhancement of the data processing unit in the spatial dimension, temporal dimension and business dimension, and obtain a multi-dimensional semantically enhanced data processing unit feature set.

[0029] In a specific embodiment, the execution step establishes a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix, and performs quantitative analysis on the spatial direction relationship of nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix, which may specifically include the following steps: The distance values ​​in the original spatial distance relationship matrix are screened by setting a threshold value to obtain a spatial connectivity threshold matrix; A weighted undirected graph is constructed based on the spatial connectivity threshold matrix, and the connection relationship between nodes in the graph is weighted to obtain a spatial connectivity graph. The relative azimuth and direction cosine values ​​of the nodes in the spatial connectivity graph are calculated to obtain a node direction feature matrix. The spatial direction similarity between node pairs is calculated based on the node direction feature matrix to obtain the direction correlation matrix, which is then normalized and fused with the weight matrix of the spatial connectivity graph to obtain the comprehensive correlation matrix. Based on the comprehensive correlation matrix, the degree centrality and betweenness centrality of the nodes in the graph are calculated to obtain the node importance index; The weight of the comprehensive correlation matrix is ​​adjusted according to the node importance index to obtain the weighted spatial correlation matrix, and the weighted spatial correlation matrix is ​​symmetric and standardized to obtain the spatial correlation measurement matrix.

[0030] Specifically, assume that the original spatial distance relationship matrix is ,in Representation Node and nodes By setting the threshold , keep all The distance value beyond the threshold is set as no connection (such as using denoted by ). Spatial connectivity threshold matrix The elements of are defined as: ; Filter out those node pairs that are too far apart and only keep nodes with potential connections. , construct a weighted undirected graph ,in is a collection of nodes, is the edge set, is a set of edge weights. Each node corresponds to a spatial unit, and the edge weight is the filtered distance value. For each edge, the weight The assignment is expressed as: ; in Inversely proportional to the distance, it means that the closer the distance is, the stronger the connectivity is. Based on the constructed spatial connectivity graph, the relative azimuth and direction cosine values ​​of the nodes in the graph are calculated to generate the node direction feature matrix. Assume that the node and nodes The coordinates of and , the relative azimuth of the two Calculated by the following formula: ; At the same time, the direction cosine value Sine of direction They are: ; These values ​​constitute the node direction feature matrix Based on the node direction feature matrix , calculate the spatial direction similarity between node pairs and generate the direction correlation matrix The measurement of directional similarity is achieved through the cosine similarity formula, which is defined as: ; in and In order to make the value range of the directional correlation matrix uniform, Normalize all Mapped to the range of [0, 1], the normalization formula is: ; The normalized directional correlation matrix and the weight matrix of the spatial connectivity graph Fusion to generate a comprehensive correlation matrix The fusion is achieved through the weighted sum formula: ; in and is a weight parameter that adjusts the importance of connectivity and directional similarity. Based on the comprehensive correlation matrix , calculate the degree centrality and betweenness centrality of the nodes in the graph to evaluate the importance of the nodes. Degree centrality Representation Node The degree of connectivity is defined as: ; Betweenness Centrality Then measure the node The ability to serve as a bridge point in the entire graph is defined as: ; in Representation Node To Node The number of shortest paths, Indicates passing through node According to the importance index of the node, the weight of the comprehensive association matrix is ​​adjusted to generate a weighted spatial association matrix The adjusted weights The node importance index is adjusted to: ; in and is an adjustment parameter used to control the impact of node importance on the association degree. Symmetrical and standardized processing is performed to generate the final spatial correlation measurement matrix Symmetrization is achieved by taking the symmetric part of the matrix: ; Normalization is done by mapping the value range of the matrix to [0,1]. Spatial correlation measure matrix It can represent the comprehensive relationship between nodes and take into account the directional relationship and importance of the nodes.

[0031] In a specific embodiment, the process of executing step 103 may specifically include the following steps: The initial graph structure is constructed based on the semantic relevance feature matrix of urban data, where each data processing unit is used as a node of the initial graph structure, and the semantic relevance between nodes is used as the edge weight of the initial graph structure; The node feature projection transformation is performed on the first layer of the graph attention layer in the multi-layer graph attention network inputted into the initial graph structure. The first layer of the graph attention layer contains 8 attention heads, and the first layer of local dependency features is obtained; The first-layer local dependency features are input into the second-layer graph attention layer in the multi-layer graph attention network. The second-layer graph attention layer contains 16 attention heads. Each attention head independently calculates the attention score to obtain the correlation weight coefficient between nodes. Based on the association weight coefficients between nodes, the first-layer local dependency features are weighted and aggregated to obtain the second-layer global dependency features; The second-layer global dependency features are input into the third-layer graph attention layer in the multi-layer graph attention network. The third-layer graph attention layer contains 32 attention heads. Each attention head uses position encoding to enhance the position information of the node, and obtains the node feature vector matrix of the data processing task. The cosine similarity between node pairs is calculated based on the node feature vector matrix to obtain the task association adjacency matrix.

[0032] Specifically, based on the semantic relevance feature matrix of urban data , construct the initial graph structure ,in is a collection of nodes, representing data processing units; is a set of edges, representing the connection relationship between nodes; is a set of edge weights, and the weight values ​​are given by the matrix The element definition of . Assume that the feature matrix ,in Representation Node and nodes The semantic relevance of The edge weight is directly assigned to In this initial graph structure, the initial feature vector of each node is It is composed of the basic attributes of the input data, such as the geographic location of the node, time characteristics, etc. The constructed initial graph structure is used as input and passed into the first layer of the multi-layer graph attention network. The first layer of the graph attention layer contains 8 independent attention heads, each of which independently calculates the attention score between nodes. and nodes The characteristics are and , attention head The weight coefficient Calculated by the following formula: ; in, It is the attention head The weight matrix is ​​used to linearly transform the input features. is the attention vector, representing the head The specific directional parameter of || represents the vector concatenation operation; LeakyReLU is the activation function used to introduce nonlinearity. Perform softmax normalization to obtain the attention weight coefficient: ; Attention weight coefficient Used to describe nodes and nodes The strength of the relationship. Based on each attention head, the node The feature aggregation is expressed as: ; in, is an activation function, such as ReLU. The outputs of the 8 attention heads are concatenated to obtain the local dependency features of the first layer: ; The local dependency features of the first layer Input to the second layer of the graph attention network, which contains 16 attention heads. Each attention head calculates the attention score independently , and based on these scores, perform weighted aggregation operations on the first layer features to generate global dependency features Similar to the first layer, the operations of the second layer include linear transformation, attention score calculation, softmax normalization, and feature weighted aggregation. As the number of attention heads increases, more complex relationships between nodes can be captured. The input is sent to the third layer of the graph attention network, which contains 32 attention heads and enhances the location information of the nodes through position encoding. Position encoding uses addition or concatenation to introduce the geographic coordinates or topological position of the node into the feature representation. For example, the node The enhanced feature is expressed as: ; in, Representation Node The outputs of the 32 attention heads are finally concatenated or averaged to generate the output of the third layer, which is the node feature vector matrix of the data processing task: ; Based on the node feature vector matrix , calculate the cosine similarity between node pairs and generate the task association adjacency matrix. and nodes Similarity Defined as: ; The similarities of all node pairs are sorted into a matrix form to obtain the task association adjacency matrix .

[0033] In a specific embodiment, the process of executing step 104 may specifically include the following steps: Perform feature embedding operations on the node feature vector matrix and the task association adjacency matrix to obtain the state vector of the scheduling environment, and extract state information features based on the state vector of the scheduling environment to obtain a deep state representation; The deep state representation is input into the policy generator for state space mapping conversion. The policy generator contains three fully connected layers to obtain a continuous action vector. A k-dimensional tree index structure is established based on the continuous action vector to discretize the continuous space and obtain a discrete action space. The candidate actions in the discrete action space are input into the value evaluation network to score the resource allocation effect of each action. The value evaluation network contains 2 convolutional layers and 3 fully connected layers to obtain the action value sequence. The reward function value of task scheduling is calculated based on the action value sequence to obtain the scheduling evaluation score. The proximal policy optimization algorithm is used to optimize the strategy according to the scheduling evaluation score to obtain the target scheduling strategy. The target scheduling strategy is mapped to task allocation, the data processing tasks to be processed are allocated to the corresponding computing nodes, and a time schedule for task processing is generated to obtain a distributed scheduling solution for data processing tasks.

[0034] Specifically, feature embedding operations are performed on the node feature vector matrix and the task association adjacency matrix to obtain the state vector of the scheduling environment. Suppose the node feature vector matrix is ,in Indicates The feature vector of nodes; the task association adjacency matrix is ,in Representation Task and tasks The correlation between them. By fusing these two matrices, the state vector of the scheduling environment is constructed. The fusion is performed in the following way: ; in, represents the fused state vector matrix, In this way, the feature information of the node is combined with the correlation information between tasks to form a scheduling environment representation. Extract state information features to obtain deep state representation. Use deep neural network (DNN) to Suppose the parameters of DNN are , then the deep state representation It is expressed as: ; in, Indicates that the parameter is By training the DNN, useful feature representations are extracted from the fused state vector to form a deep state representation. The input strategy generator is used for state space mapping conversion. The strategy generator consists of three fully connected layers, and the weight matrix and bias of each layer are and ( ), then the continuous action vector The calculation process is as follows: ; ; ; in, Represents an activation function, such as ReLU. Through the mapping of the policy generator, the deep state representation is converted into a continuous action vector , used for subsequent action selection. Based on continuous action vector , establish a k-dimensional tree (kd tree) index structure, discretize the continuous space, and obtain a discrete action space. The k-dimensional tree is a data structure used to store and retrieve points in a multidimensional space, and is suitable for efficient discretization of continuous action spaces. By constructing a k-dimensional tree, the continuous action vector space is divided into several discrete regions, each corresponding to a discrete action. The candidate actions in the discrete action space are input into the value evaluation network, and the resource allocation effect of each action is scored. The value evaluation network contains 2 convolutional layers and 3 fully connected layers. Assume that The feature representation of a candidate action is , then its value score The calculation process is as follows: ; ; ; ; ; in, Indicates The operation of the convolutional layer, and ( ) represents the weight matrix and bias of the fully connected layer. Through the value evaluation network, each candidate action is scored to obtain the action value sequence. Based on the action value sequence, the reward function value of the task scheduling is calculated to obtain the scheduling evaluation score. The value of an action is , and its corresponding reward function value It is expressed as: ; in, Represents the reward function, which is used to map the value score to the reward value. Reward functions include linear functions, nonlinear functions, etc. By calculating the reward value of each action, its effect in task scheduling is evaluated. The proximal policy optimization algorithm is used to optimize the strategy according to the scheduling evaluation score to obtain the target scheduling strategy. Proximal policy optimization is a reinforcement learning algorithm that maintains the stability of training by limiting the amplitude of policy updates. Proximal policy optimization is optimized through the following objective function: ; in, represents the probability ratio of the new and old strategies, represents the estimated value of the advantage function, is a hyperparameter used to limit the scope of policy updates. By optimizing the above objective function, the updated policy parameters are obtained , forming the target scheduling strategy. Map the task allocation to the target scheduling strategy. Assume that the scheduling strategy outputs the task allocation decision matrix ,in Indicates the task Is it assigned to a node? (1 means assigned, 0 means unassigned), then the assignment of tasks is expressed as: ; According to the task allocation matrix , combined with the processing power of the computing nodes and the priority of the tasks, a time schedule for task processing is generated. Assume that the computing power of each node is (indicates the number of tasks that can be processed per unit time), task The processing requirements are , then the task At the node Completion time on It is expressed as: ; By sorting the completion time of all tasks, a global task processing schedule is generated Using greedy algorithms or dynamic programming methods, the total task completion time is minimized or the system throughput is maximized while ensuring task dependencies. Based on the target scheduling strategy and time schedule, a complete distributed scheduling solution for data processing tasks is generated.

[0035] In a specific embodiment, the process of executing step 105 may specifically include the following steps: The task sequences in the distributed scheduling scheme are grouped and optimized to obtain the node-level task allocation scheme, and the city data semantic relevance feature matrix is ​​divided into blocks according to the preset block size. Each sub-block contains the semantic features of the adjacent areas, and a feature sub-block set is obtained. Perform parallel semantic mapping calculations on the feature sub-block set, and use the block coordinate descent method to update the semantic mapping parameters to obtain local semantic mapping results; Based on the local semantic mapping results, a global consistency constraint is constructed, and the semantic mapping parameters are jointly optimized using the alternating direction multiplier method to obtain the globally optimized semantic mapping parameters. Based on the globally optimized semantic mapping parameters, semantic relationship calculation is performed on multi-source heterogeneous spatiotemporal data to obtain the corresponding semantic mapping relationship set.

[0036] Specifically, assume that the task sequence in the distributed scheduling scheme is ,in Indicates By analyzing the characteristics of tasks, such as computing resource requirements, task priorities, and dependencies between tasks, tasks are divided into multiple groups, and each group of tasks is assigned to a specific computing node. The set of computing nodes is designed as , each node The resource capacity is The goal of task allocation is to optimize node load balancing while satisfying resource constraints. is a decision variable, when the task Assign to Node hour, ,otherwise The task allocation problem is modeled as the following optimization problem: ; in, It's a task By solving the above optimization problem, a node-level task allocation scheme is generated. After completing the task grouping optimization, the city data semantic association feature matrix Divide the matrix into blocks according to the preset block size. Assume that the size of each sub-block is , then the matrix Divided into sub-blocks, each of which contains the semantic features of the adjacent regions. The sub-block set is represented as . Perform parallel semantic mapping calculations on the feature sub-block set. The semantic mapping calculation of each sub-block is regarded as an independent sub-problem, and the semantic mapping parameters are updated by minimizing the mapping error. The mapping parameters are , the objective function of the mapping error is: ; in, and Is a node and The characteristic vector of represents the Euclidean norm. The block coordinate descent method is used to optimize , that is, fix other parameters and only optimize the mapping parameters of the current sub-block, and gradually reduce the mapping error through iterative updates. After obtaining the local semantic mapping results of each sub-block, in order to ensure the consistency of mapping parameters between different sub-blocks, a global consistency constraint is constructed. The global consistency constraint is set by matching the boundary features shared by adjacent sub-blocks. and is the mapping parameter of two adjacent sub-blocks, and the constraint target is: ; in, Represents the eigenvector of the boundary node. In order to optimize this global constraint, the alternating direction multiplier method (ADMM) is used for joint optimization. The goal of ADMM is to decompose the complex global problem into several easy-to-solve sub-problems and achieve consistent optimization by introducing Lagrange multipliers. Let the global objective function be: ; in, is the constraint strength parameter, which is optimized by Get the globally optimized semantic mapping parameters. Based on the globally optimized semantic mapping parameters, perform semantic relationship calculation on multi-source heterogeneous spatiotemporal data. Generate a semantic mapping relationship set based on the optimized mapping parameters. Set the node and The semantic mapping relationship is , which is defined as: ; in, is the global optimization mapping parameter, and · represents the vector inner product. , generate a set of semantic mapping relationships.

[0037] The above describes the semantic alignment method of multi-source heterogeneous data in the embodiment of the present invention. The following describes the semantic alignment device of multi-source heterogeneous data in the embodiment of the present invention. Figure 2 , an embodiment of a multi-source heterogeneous data semantic alignment device in an embodiment of the present invention includes: The acquisition module 201 is used to collect multi-source heterogeneous spatiotemporal data through a central data node and multiple edge nodes, and to construct a standardized data processing unit set and a corresponding distributed data index directory; The calculation module 202 is used to extract spatial attribute features and calculate spatial correlation between units for each data processing unit in the standardized data processing unit set, so as to obtain a semantic correlation feature matrix of urban data; Feature extraction module 203, used to input the city data semantic relevance feature matrix into the multi-layer graph attention network for feature extraction, and obtain the node feature vector matrix and task relevance adjacency matrix of the data processing task; A generation module 204, used to generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association adjacency matrix; The optimization module 205 is used to allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on the semantic association feature matrix of urban data to obtain a semantic mapping relationship set of multi-source heterogeneous spatiotemporal data.

[0038] Through the collaborative cooperation of the above components, by building a distributed edge computing network and data acquisition topology, combined with multi-level data preprocessing and standardization mechanisms, the system can efficiently process multi-source heterogeneous spatiotemporal data, significantly improving the reliability and accuracy of data processing; based on spatial attribute feature extraction and spatial correlation calculation, a new type of urban data semantic correlation feature matrix is ​​constructed, which effectively solves the ambiguity problem of traditional methods in the semantic alignment process and improves the accuracy of semantic mapping; a multi-layer graph attention network is used for feature extraction, and the local and global dependencies between data units are captured through a multi-head attention mechanism, enhancing the system's ability to express complex data structures; innovatively combining deep reinforcement learning with the k-dimensional tree nearest neighbor algorithm, realizing intelligent scheduling of data processing tasks and optimizing the allocation efficiency of computing resources; through the alternating iterative optimization strategy of the block descent method, efficient semantic alignment in a distributed environment is achieved, improving the overall processing performance and scalability of the system.

[0039] above Figure 2 The multi-source heterogeneous data semantic alignment device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The multi-source heterogeneous data semantic alignment device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0040] Figure 3 3 is a schematic diagram of the structure of a multi-source heterogeneous data semantic alignment device provided by an embodiment of the present invention. The multi-source heterogeneous data semantic alignment device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 can be short-term storage or persistent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the multi-source heterogeneous data semantic alignment device 300. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the multi-source heterogeneous data semantic alignment device 300 to implement the steps of the multi-source heterogeneous data semantic alignment method described above.

[0041] The multi-source heterogeneous data semantic alignment device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that Figure 3 The structure of the multi-source heterogeneous data semantic alignment device shown does not constitute a limitation on the multi-source heterogeneous data semantic alignment device provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0042] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-source heterogeneous data semantic alignment method.

[0043] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0044] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0045] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic alignment method for multi-source heterogeneous data, characterized in that: The method comprises: Collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; Extracting spatial attribute features and calculating spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a semantic correlation feature matrix of urban data; Inputting the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtaining a node feature vector matrix and a task relevance adjacency matrix of the data processing task; Generate a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association adjacency matrix; The data processing tasks are allocated to the computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

2. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The method collects multi-source heterogeneous spatiotemporal data through a central data node and multiple edge nodes, and constructs a standardized data processing unit set and a corresponding distributed data index directory, including: Deploy the central data node and multiple edge nodes at urban data collection points, and build a data collection network topology map through a distributed edge computing network; Based on the data collection network topology diagram, data preprocessing is performed on the multi-source heterogeneous spatiotemporal data collected by the central data node and multiple edge nodes to obtain a cleaned standard data set; Establishing a time series index tree structure based on the data records in the cleaned standard data set, and performing a time window sliding mechanism analysis to obtain a data set with time series association; According to the spatial dimension characteristics, the data set with time series association is subjected to geographic grid division and spatial clustering to obtain time-space association data blocks, and metadata extraction and feature vector calculation are performed on the time-space association data blocks to obtain target calculation results; The target calculation results are stored in a distributed data directory to obtain a distributed data index directory, and the spatiotemporal correlation data blocks are adaptively segmented and boundary optimized according to a preset data processing unit size to obtain a standardized data processing unit set.

3. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The step of extracting spatial attribute features and calculating spatial correlation between units for each data processing unit in the standardized data processing unit set to obtain a city data semantic correlation feature matrix includes: Extracting longitude and latitude coordinates, defining spatial area ranges, and analyzing spatial distribution patterns of the geographical location information of each data processing unit in the standardized data processing unit set to obtain a spatial attribute feature vector; Based on the spatial attribute feature vector, the spatial straight-line distance and the network topological distance between the data processing units are calculated to obtain an original spatial distance relationship matrix; Establishing a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix, and performing quantitative analysis on the spatial direction relationship of nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix; Based on the spatial correlation measurement matrix, the spatial relationship of the data processing unit is mapped to a high-dimensional feature space to obtain a spatial semantic feature mapping matrix; An evaluation function integrating spatial proximity and functional similarity is created based on the spatial semantic feature mapping matrix, and the semantic relevance between data processing units is scored according to the evaluation function to obtain a semantic relevance feature matrix for urban data.

4. The multi-source heterogeneous data semantic alignment method according to claim 3 is characterized in that: The step of establishing a spatial connectivity graph between data processing units based on the original spatial distance relationship matrix, and performing quantitative analysis on the spatial direction relationship of nodes in the spatial connectivity graph to obtain a spatial correlation measurement matrix includes: Setting a threshold for the distance values ​​in the original spatial distance relationship matrix to perform screening, and obtaining a spatial connectivity threshold matrix; A weighted undirected graph is constructed based on the spatial connectivity threshold matrix, and weights are assigned to the connection relationships between the nodes in the graph to obtain a spatial connectivity graph, and relative azimuth angles and direction cosine values ​​of the nodes in the spatial connectivity graph are calculated to obtain a node direction feature matrix; Calculating the spatial directional similarity between node pairs based on the node directional feature matrix to obtain a directional correlation matrix, normalizing the directional correlation matrix, and fusing it with the weight matrix of the spatial connectivity graph to obtain a comprehensive correlation matrix; Calculate the degree centrality and betweenness centrality of nodes in the graph based on the comprehensive association matrix to obtain node importance index; The comprehensive correlation matrix is ​​weighted according to the node importance index to obtain a weighted spatial correlation matrix, and the weighted spatial correlation matrix is ​​symmetric and standardized to obtain a spatial correlation measurement matrix.

5. The multi-source heterogeneous data semantic alignment method according to claim 1 is characterized in that: The step of inputting the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction to obtain a node feature vector matrix and a task relevance adjacency matrix of the data processing task includes: Constructing an initial graph structure based on the semantic relevance feature matrix of the city data, wherein each data processing unit serves as a node of the initial graph structure, and the semantic relevance between nodes serves as an edge weight of the initial graph structure; Performing node feature projection transformation on the first graph attention layer in the multi-layer graph attention network inputted with the initial graph structure, wherein the first graph attention layer includes 8 attention heads, and obtaining the first layer of local dependency features; Inputting the first layer of local dependency features into the second layer of graph attention layer in the multi-layer graph attention network, the second layer of graph attention layer includes 16 attention heads, each attention head independently calculates the attention score, and obtains the correlation weight coefficient between nodes; Performing a weighted aggregation operation on the first layer of local dependency features based on the association weight coefficients between the nodes to obtain a second layer of global dependency features; Inputting the second-layer global dependency features into the third-layer graph attention layer in the multi-layer graph attention network, the third-layer graph attention layer includes 32 attention heads, each of which uses position encoding to enhance the position information of the node, and obtaining a node feature vector matrix of the data processing task; The cosine similarity between node pairs is calculated based on the node feature vector matrix to obtain a task association adjacency matrix.

6. The semantic alignment method of multi-source heterogeneous data according to claim 1 is characterized in that: The distributed scheduling scheme for generating data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix includes: Performing feature embedding operations on the node feature vector matrix and the task association degree adjacency matrix to obtain a state vector of the scheduling environment, and performing state information feature extraction based on the state vector of the scheduling environment to obtain a deep state representation; The deep state representation is input into a strategy generator for state space mapping conversion. The strategy generator includes three fully connected layers to obtain a continuous action vector, and a k-dimensional tree index structure is established based on the continuous action vector to discretize the continuous space and obtain a discrete action space; Inputting the candidate actions in the discrete action space into a value evaluation network, scoring the resource allocation effect of each action, wherein the value evaluation network comprises two convolutional layers and three fully connected layers, and obtaining an action value sequence; Calculating the reward function value of task scheduling based on the action value sequence to obtain a scheduling evaluation score, and using a proximal strategy optimization algorithm to perform strategy optimization according to the scheduling evaluation score to obtain a target scheduling strategy; The target scheduling strategy is mapped to task allocation, the data processing tasks to be processed are allocated to the corresponding computing nodes, and a time sequence schedule for task processing is generated to obtain a distributed scheduling solution for the data processing tasks.

7. The multi-source heterogeneous data semantic alignment method according to claim 1, characterized in that: The data processing tasks are allocated to computing nodes according to the distributed scheduling scheme, and alternating iterative optimization is performed based on the semantic association feature matrix of the city data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data, including: The task sequence in the distributed scheduling scheme is grouped and optimized to obtain a node-level task allocation scheme, and the city data semantic relevance feature matrix is ​​divided into blocks according to a preset block size, each sub-block contains semantic features of adjacent areas, and a feature sub-block set is obtained; Performing parallel semantic mapping calculation on the feature sub-block set, and updating semantic mapping parameters by using block coordinate descent method to obtain local semantic mapping results; Building a global consistency constraint based on the local semantic mapping result, and jointly optimizing the semantic mapping parameters using an alternating direction multiplier method to obtain globally optimized semantic mapping parameters; Based on the globally optimized semantic mapping parameters, semantic relationship calculation is performed on the multi-source heterogeneous spatiotemporal data to obtain a corresponding semantic mapping relationship set.

8. A multi-source heterogeneous data semantic alignment device, characterized in that: The device for executing the semantic alignment method of multi-source heterogeneous data according to any one of claims 1 to 7 comprises: The acquisition module is used to collect multi-source heterogeneous spatiotemporal data through the central data node and multiple edge nodes, and build a standardized data processing unit set and a corresponding distributed data index directory; A calculation module, used for extracting spatial attribute features and calculating spatial correlation between units for each data processing unit in the standardized data processing unit set, to obtain a semantic correlation feature matrix of urban data; A feature extraction module is used to input the city data semantic relevance feature matrix into a multi-layer graph attention network for feature extraction, and obtain a node feature vector matrix and a task relevance adjacency matrix of the data processing task; A generation module, used for generating a distributed scheduling scheme for data processing tasks according to the node feature vector matrix and the task association degree adjacency matrix; The optimization module is used to allocate data processing tasks to computing nodes according to the distributed scheduling scheme, and perform alternating iterative optimization based on the semantic association feature matrix of the urban data to obtain a semantic mapping relationship set of the multi-source heterogeneous spatiotemporal data.

9. A multi-source heterogeneous data semantic alignment device, characterized in that: The multi-source heterogeneous data semantic alignment device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the multi-source heterogeneous data semantic alignment device executes the multi-source heterogeneous data semantic alignment method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the multi-source heterogeneous data semantic alignment method as described in any one of claims 1-7 is implemented.

Images