Multi-source work order oriented cross-platform heterogeneous data integration method

Abstract

The application relates to the field of artificial intelligence and specifically discloses a cross-platform heterogeneous data integration method for multi-source work orders. The method extracts metadata and field instances of the multi-source work orders, constructs a cross-platform work order semantic topology graph with fields as nodes and inter-field dependency and co-occurrence relations as edges, and introduces a time dimension to form a dynamic graph structure. A dynamic graph neural network is used to aggregate and update the node features of the dynamic graph structure, so that the drift features of the field semantics in data flow are obtained. Based on the updated node feature vectors, a semantic similarity matrix between heterogeneous fields is calculated, a dynamic threshold is set to cut off low-confidence mapping, and a dynamic alignment mapping rule of the cross-platform heterogeneous work orders is generated to complete data integration. The application overcomes the defects that static mapping cannot adapt to semantic drift and structure change, realizes adaptive tracking of heterogeneous work order data, improves alignment accuracy and system robustness, and reduces the consumption of computing resources.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and specifically discloses a cross-platform heterogeneous data integration method for multi-source work orders. Background Technology

[0002] In the field of cross-platform data integration, different business platforms generate a large amount of heterogeneous work order data. Conventional technical solutions mainly adopt mapping methods based on predefined pattern matching or static ontology to unify heterogeneous data. The specific implementation process of this conventional solution is as follows: pre-configure the field identifiers of work orders for each platform, determine the correspondence between fields between different platforms through manual comparison, and solidify this correspondence into a static mapping rule table. During the data integration execution phase, the system reads the work order data stream input from the source platform and, according to the fixed mapping rule table, directly converts or assigns the source-side field values ​​to the corresponding fields on the target platform. This process relies on the structural consistency assumption at the schema level, that is, during the integration cycle, the header structure, field identifiers, and dependencies between fields of the work order are considered constant.

[0003] In the implementation of static ontology mapping schemes, the system typically constructs a global data dictionary as a central hub, and the work order fields of each platform need to be aligned with this global dictionary. During implementation, metadata of the source and target work orders is extracted, and the similarity values ​​between fields are calculated using string similarity or dictionary matching. If the similarity value exceeds a preset fixed threshold, a static mapping relationship is established. For co-occurrence relationships or foreign key constraints existing in the work orders, existing technologies also use manual annotation to convert them into constraint rules in the global dictionary. During the data writing phase, the integration engine performs data extraction, transformation, and loading operations according to these established rules. Since the mapping rules and constraint rules are locked before the system runs and do not change during operation, the system cannot perceive the semantic changes of fields caused by business adjustments on the source platform, nor can it capture constraint changes caused by structural upgrades on the target platform.

[0004] The existing technologies described above suffer from a core technical problem: static mapping rules cannot adapt to the semantic shifts of fields during data flow and the dynamic changes in work order structures, leading to the failure of heterogeneous data integration and alignment. In real-world business processes, work order structures on different platforms are added to or deleted as business iterates, and the business meanings of fields also experience semantic shifts due to changes in the context. Existing technologies use fixed mapping rules and fixed thresholds for field alignment. When the semantics of source-side fields evolve or the topology changes, the original mapping relationship becomes invalid, and the fixed thresholds cannot filter out low-confidence mappings caused by semantic shifts, resulting in source-side data failing to map to the target-side and causing data integration and alignment failures. Summary of the Invention

[0005] The purpose of this invention is to provide a cross-platform heterogeneous data integration method for multi-source work orders, which can solve the problems in the background art mentioned above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A cross-platform heterogeneous data integration method for multi-source work orders includes: extracting metadata and field instances of multi-source work orders, constructing a cross-platform work order semantic topology graph with fields as nodes and field dependencies and co-occurrence relationships as edges, and introducing a time dimension to form a dynamic graph structure; A dynamic graph neural network from the field of artificial intelligence is used to aggregate and update the node features of the dynamic graph structure to obtain the drift features of field semantics in data flow. Based on the updated node feature vectors, the semantic similarity matrix between heterogeneous fields is calculated. By setting a dynamic threshold to cut off low-confidence mappings, dynamic alignment mapping rules for cross-platform heterogeneous work orders are generated, thus completing data integration.

[0007] Preferably, the extraction of metadata and field instances from multi-source work orders, using fields as nodes and dependencies and co-occurrence relationships between fields as edges, to construct a cross-platform work order semantic topology graph includes: parsing the header structure of the multi-source work orders to obtain field identifiers, mapping the field identifiers to the nodes, and traversing the data records of the multi-source work orders to extract field values ​​as field instances; Calculate the joint occurrence frequency of field instances corresponding to different field identifiers in the same data record, and determine the co-occurrence relationship of field identifier pairs whose joint occurrence frequency exceeds a preset frequency threshold; Extract the foreign key constraints and primary key constraints corresponding to the field identifier as the dependency relationship, and connect the corresponding nodes as the edge relationship to generate the cross-platform work order semantic topology graph.

[0008] Preferably, the introduction of the time dimension to form a dynamic graph structure includes: obtaining change log data of the multi-source work order within a historical time window, wherein the change log data includes field addition / deletion records and field type modification records; The historical time window is divided into continuous time slices according to a preset time step, and the node existence status and edge existence status of the cross-platform work order semantic topology graph are extracted in each time slice. The nodes and edges that undergo state changes between the cross-platform work order semantic topology graphs corresponding to adjacent time slices are connected across time steps to generate temporal edges that represent the topology evolution process. The dynamic graph structure is formed by the temporal edges and the cross-platform work order semantic topology graphs in each time slice.

[0009] Preferably, the step of using a dynamic graph neural network in the field of artificial intelligence to aggregate and update the node features of the dynamic graph structure and obtain the drift features of field semantics in data flow includes: at each propagation layer of the dynamic graph neural network, for the target node in the dynamic graph structure, aggregating the first feature vectors of the neighboring nodes of the target node in the current time slice to obtain spatial aggregation features; Extract the second feature vector of the target node within the historical time slice, and concatenate the second feature vector with the spatial aggregation feature using temporal features; The concatenated feature vector is input into a gated recurrent network to filter out redundant features in the historical time slices, and the updated feature vector of the target node in the current time slice is output as the representation result of the drift feature.

[0010] Preferably, the step of calculating the semantic similarity matrix between heterogeneous fields based on the updated node feature vector, and generating dynamic alignment mapping rules for cross-platform heterogeneous work orders by setting a dynamic threshold to cut off low-confidence mappings, includes: calculating the cosine distance between the updated feature vector of the source node belonging to different platforms and the updated feature vector of the target node, and converting the cosine distance into a similarity value in the semantic similarity matrix. Based on the edge density and number of nodes of the dynamic graph structure in the current time slice, the adaptive adjustment coefficient is calculated, and the dynamic threshold is obtained by multiplying the preset basic threshold by the adaptive adjustment coefficient. The mapping relationships corresponding to similarity values ​​less than the dynamic threshold in the semantic similarity matrix are filtered out, and the mapping relationships greater than or equal to the dynamic threshold are retained to construct the dynamic alignment mapping rule.

[0011] Preferably, after traversing the data records of the multi-source work orders to extract field values ​​as field instances, the method further includes: performing lexical analysis and part-of-speech tagging on the field instances to identify named entities and business keywords in the field instances; The named entities and the business keywords are combined into a feature sequence, which is then input into a pre-trained language model to extract the contextual semantic vector of the feature sequence in the contextual semantic environment. Max pooling is performed on all context semantic vectors corresponding to the same field identifier. The vector output by the max pooling operation is determined as the target semantic vector. The target semantic vector is used as the initial feature representation of the node in the cross-platform work order semantic topology graph, thereby unifying the feature dimensions of heterogeneous naming of the same business concept on different platforms.

[0012] Preferably, dividing the historical time window into continuous time slices according to a preset time step includes: counting the change frequency of the change log data in a unit time interval within the historical time window, and generating a change frequency sequence; Calculate the fluctuation variance of the change frequency sequence. When the fluctuation variance exceeds a preset fluctuation threshold, reduce the unit time interval to a first target step size. When the fluctuation variance does not exceed the preset fluctuation threshold, expand the unit time interval to a second target step size, wherein the first target step size is smaller than the second target step size. The historical time window is divided according to the determined first or second target step size to generate continuous time slices that adaptively match the fluctuation state of the change frequency sequence.

[0013] Preferably, aggregating the first feature vectors of the target node's neighboring nodes within the current time slice to obtain spatial aggregation features includes: calculating the association weight between the target node and each neighboring node within the current time slice, wherein the association weight is determined based on the dot product operation between the initial feature representation of the target node and the initial feature representation of the neighboring nodes; The association weights are normalized to obtain the attention coefficients corresponding to each neighbor node; The attention coefficient is used as a weight parameter and weighted and summed with the first feature vector of the corresponding neighbor node to generate the spatial aggregation feature of the target node in the current time slice.

[0014] Preferably, after generating the dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, the method further includes: extracting business constraint rules from the target platform data dictionary to which the target field mapped in the dynamic alignment mapping rules belongs, wherein the business constraint rules include field format constraints and logical dependency constraints between fields; The integrated data is substituted into the business constraint rules for rule verification. If the integrated data violates the field format constraints or the logical dependency constraints between fields, it is determined that there is a logical conflict in the current dynamic alignment mapping rule. Extract the target node and source node corresponding to the mapping relationship with logical conflict, delete the mapping edge between the target node and the source node, and trigger the dynamic graph neural network to re-aggregate and update features and calculate similarity for the remaining nodes.

[0015] Preferably, after generating the dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, when a new work order platform is added, the method further includes: parsing the new metadata and new field instances of the new work order platform, and constructing new nodes and new edges; The newly added node and the newly added edge are embedded into the cross-platform work order semantic topology graph of the current time slice, and the historical feature vectors of existing nodes are frozen; The aggregation update operation of the dynamic graph neural network is performed only on the newly added node to obtain the newly updated feature vector of the newly added node; Calculate the new similarity value between the newly added updated feature vector and the updated feature vector of the existing platform node. Based on the comparison result of the new similarity value and the dynamic threshold, generate an incremental alignment mapping rule and integrate the data of the newly added work order platform.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a cross-platform work order semantic topology graph by extracting metadata and field instances from multi-source work orders, and introduces a time dimension to form a dynamic graph structure. A dynamic graph neural network is used to aggregate and update the node features of the dynamic graph structure, obtaining the drift features of field semantics in data flow. A semantic similarity matrix is ​​calculated based on the updated node feature vectors, and low-confidence mappings are cut off by setting a dynamic threshold to generate dynamic alignment mapping rules. Introducing the time dimension allows the topology graph to record historical changes in field additions, deletions, and type modifications, breaking the limitations of static ontology. The dynamic graph neural network aggregates the features of neighboring nodes within the current time slice at the propagation layer and concatenates the features of historical time slices into a gated recurrent network to filter out redundant historical features, thereby extracting the drift features of field meaning evolving with context. The adaptive adjustment coefficient is calculated based on the current edge density and number of nodes in the dynamic graph structure. This coefficient is then multiplied by the base threshold to obtain the dynamic threshold. This allows for adaptive adjustment of the mapping filtering criteria based on the complexity of the topology. When semantic drift leads to a decrease in similarity, the dynamic threshold can filter out low-confidence erroneous mappings, preventing mapping failures caused by business iterations or structural changes. This ensures the alignment accuracy and system robustness of heterogeneous work order data integration.

[0017] 2. This invention extracts named entities and business keywords from field instances to form a feature sequence, inputs it into a pre-trained language model to extract contextual semantic vectors, and performs max pooling on the contextual semantic vectors corresponding to the same field identifier as the initial feature representation of the node. This unifies the feature dimensions of heterogeneous naming of the same business concepts across different platforms, eliminating differences in expression at the lexical level. The time step is adaptively adjusted based on the variance of the change frequency sequence in the change log data. When the variance is large, the time step is reduced; when the variance is small, the time step is increased, ensuring that the time slice division matches the intensity of actual business changes, balancing computational resource consumption and feature update accuracy. After generating alignment mapping rules, the integrated data is substituted into the business constraint rules in the target platform's data dictionary for verification. If the constraints are violated, a logical conflict is identified, the conflicting mapping edge is deleted, and a re-aggregation update is triggered, ensuring the logical consistency of the integrated data on the target platform. When a new platform is added, the new node and edge are embedded into the current topology graph, and existing node features are frozen. Aggregation updates are only performed on the new node to generate incremental alignment mapping rules, avoiding the resource overhead of recalculating the entire graph and reducing the processing latency of incremental integration. Attached Figure Description

[0018] Figure 1 Flowchart for parsing and extracting multi-source work unit data and constructing the initial semantic topology graph of cross-platform work orders; Figure 2 Flowcharts are generated for work order change log collection, adaptive time step calculation, and dynamic graph timing structure. Figure 3 A flowchart for obtaining drift features through spatial aggregation of nodes and temporal feature updates in a single-layer dynamic graph neural network. Figure 4 A flowchart for generating alignment rules for cross-platform field feature similarity calculation, dynamic threshold solution, and low-confidence mapping removal; Figure 5 Flowchart for integrating work order data business constraint verification, conflict mapping elimination, and secondary feature recalculation of existing nodes; Figure 6 A flowchart is generated for the new work order platform access, incremental node topology embedding, and incremental alignment mapping rules. Detailed Implementation

[0019] refer to Figure 1In one embodiment, a cross-platform heterogeneous data integration method for multi-source work orders is applied to an enterprise-level work order management system. This system interfaces with three heterogeneous work order platforms: an IT operations and maintenance work order platform, a customer service work order platform, and an equipment maintenance work order platform. The three platforms employ different technical architectures and data storage methods. The IT operations and maintenance work order platform uses a relational database to store structured work order data, the customer service work order platform uses JSON format to store semi-structured work order data, and the equipment maintenance work order platform uses XML format to store semi-structured work order data. The work order field identifiers, field types, and business meanings differ significantly among the three platforms and undergo periodic structural changes and semantic adjustments as business iterations occur.

[0020] Specifically, the first step is to extract metadata and field instances from multi-source work orders. For IT maintenance work order data stored in a relational database, structured queries are executed to obtain the database table structure information, including table name, field name, field type, primary key constraints, and foreign key constraints. The field name is used as the field identifier. For customer service work order data in JSON format, a JSON parser is used to traverse the key-value pairs of the JSON object, extracting the top-level key and nested keys as field identifiers. For fields with more than three levels of nesting, the parent key and child keys are concatenated to generate unique field identifiers. For equipment repair work order data in XML format, an XML parser is used to traverse the element nodes and attribute nodes of the XML document, extracting element tags and attribute names as field identifiers. For duplicate elements with the same tag, a serial number suffix is ​​added to generate unique field identifiers. All work order data records from the three platforms are traversed, and the field value corresponding to each field identifier is extracted as the field instance for that field. For null fields, an empty string is recorded and marked as null.

[0021] A cross-platform work order semantic topology graph is constructed, using fields as nodes and dependencies and co-occurrence relationships between fields as edges. The joint occurrence frequency of field instances corresponding to different field identifiers in the same data record is calculated. For any two field identifiers A and B, the number of records in all work order data records where both field A and field B have non-null values ​​is counted, and this number is taken as the joint occurrence frequency of field A and field B. A preset frequency threshold of 50 is used; when the joint occurrence frequency of two field identifiers exceeds 50, a co-occurrence relationship is determined between the two field identifiers. Foreign key constraints and primary key constraints corresponding to the field identifiers are extracted as dependencies. For example, in the IT operations work order platform, the work order ID field is the primary key, the associated device ID field is a foreign key, and it is associated with the device ID primary key field in the device information table. Therefore, there is a dependency relationship between the work order ID and the associated device ID, and there is also a dependency relationship between the associated device ID and the device ID. Co-occurrence relationships and dependencies are used as edges to connect the corresponding nodes, generating a cross-platform work order semantic topology graph.

[0022] A time dimension is introduced to form a dynamic graph structure. Change log data for multi-source work orders within a historical time window is obtained, with the historical time window set to the past 90 days. Change log data includes records of field additions / deletions and field type modifications. For example, the IT operations work order platform added a fault level field on day 30 and changed the processing time field type from integer to floating-point on day 60; the customer service work order platform deleted a customer level field on day 45 and added a complaint type field on day 75. The historical time window is divided into consecutive time slices according to a preset time step of 7 days, resulting in 13 consecutive time slices for the 90-day historical time window, with the 13th time slice containing the remaining 6 days. The node existence status and edge existence status of the cross-platform work order semantic topology graph within each time slice are extracted. Node existence status indicates whether the field exists in the work order structure within the corresponding time slice, and edge existence status indicates whether the relationship between the fields is valid within the corresponding time slice. Nodes and edges that undergo state changes between adjacent time slices in the cross-platform work order semantic topology graph are connected across time steps to generate temporal edges representing the topology evolution process. For example, if the fault level field is absent in time slices 1 to 4 but exists in time slices 5 to 13, a temporal edge is generated between the fault level nodes in time slices 4 and 5. The temporal edges and the cross-platform work order semantic topology graph within each time slice together constitute a dynamic graph structure.

[0023] A dynamic graph neural network (DNN) from the field of artificial intelligence is used to aggregate and update the node features of a dynamic graph structure, obtaining the drift features of field semantics in data flow. The DNN consists of three propagation layers, each corresponding to the feature processing of a time slice. In each propagation layer of the DNN, for the target node in the dynamic graph structure, the first feature vectors of the target node's neighboring nodes within the current time slice are aggregated to obtain spatial aggregated features. The second feature vector of the target node within historical time slices is extracted and concatenated with the spatial aggregated features as temporal features. The concatenated feature vector is input into a gated recurrent network (GRN) to filter out redundant features in historical time slices, outputting the updated feature vector of the target node in the current time slice as the representation of drift features. The GRN includes an update gate and a reset gate. The update gate controls the degree of retention of historical features, and the reset gate controls the degree of forgetting of historical features. The closer the output value of the update gate is to 1, the more historical features are retained; the closer the output value of the reset gate is to 0, the more historical features are forgotten.

[0024] Based on the updated node feature vectors, a semantic similarity matrix between heterogeneous fields is calculated. Low-confidence mappings are cut off by setting a dynamic threshold, generating dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration. The cosine distance between the updated feature vectors of source nodes and target nodes belonging to different platforms is calculated and converted into similarity values ​​in the semantic similarity matrix. An adaptive adjustment coefficient is calculated based on the edge density and number of nodes in the current time slice of the dynamic graph structure. A dynamic threshold is obtained by multiplying the preset base threshold by the adaptive adjustment coefficient. Mapping relationships with similarity values ​​less than the dynamic threshold in the semantic similarity matrix are filtered out, and mapping relationships greater than or equal to the dynamic threshold are retained to construct dynamic alignment mapping rules. For example, if the similarity score between the ticket ID node of the IT operations and maintenance ticket platform and the service order number node of the customer service ticket platform is 0.92, which is greater than the dynamic threshold of 0.75, then a mapping relationship from ticket ID to service order number is established; if the similarity score between the handler node of the IT operations and maintenance ticket platform and the customer service name node of the customer service ticket platform is 0.87, which is greater than the dynamic threshold of 0.75, then a mapping relationship from handler to customer service name is established; if the similarity score between the fault description node of the IT operations and maintenance ticket platform and the problem details node of the customer service ticket platform is 0.81, which is greater than the dynamic threshold of 0.75, then a mapping relationship from fault description to problem details is established.

[0025] In this embodiment, a dynamic graph structure incorporating a time dimension is constructed to record the historical changes in work order fields and relationships, breaking the assumption of a constant structure in static ontology mapping schemes. By aggregating spatial and temporal features through a dynamic graph neural network, the drift characteristics of field semantics over time are extracted, enabling the perception of semantic changes brought about by business iterations. Dynamic threshold filtering of low-confidence mappings adaptively adjusts the filtering criteria based on the complexity of the topology, avoiding mapping failures caused by semantic drift.

[0026] In a preferred embodiment, after traversing the data records of multi-source work orders to extract field values ​​as field instances, lexical analysis and part-of-speech tagging are performed on the field instances to identify named entities and business keywords within them. The named entities and business keywords are combined into a feature sequence, which is then input into a pre-trained language model to extract the contextual semantic vector of the feature sequence within the contextual semantic environment. Max pooling is performed on all contextual semantic vectors corresponding to the same field identifier, and the vector output by the max pooling operation is determined as the target semantic vector. This target semantic vector serves as the initial feature representation of nodes in the cross-platform work order semantic topology graph, unifying the feature dimensions for heterogeneous naming of the same business concepts across different platforms.

[0027] Specifically, a lexical analyzer based on conditional random fields is used to segment and tag field instances, identifying named entities such as names of people, places, organizations, time, date, numbers, IP addresses, and MAC addresses, as well as business keywords such as verbs, nouns, and adjectives related to the work order business. For example, for the instance server [192.168.1.100] (192.168.1.100) in the fault description field of the IT operations and maintenance work order platform, which had a problem with high CPU usage on October 15, 2023, the named entities identified after lexical analysis include [192.168.1.100] (192.168.1.100) (IP address), October 15, 2023 (date), and CPU (hardware device). The business keywords include server, occurrence, usage rate, high, and problem. The identified named entities and business keywords are combined into a feature sequence according to their order of appearance in the original field instances, i.e., [server, [192.168.1.100](192.168.1.100), October 15, 2023, occurrence, CPU, utilization, too high, problem]. For feature sequences longer than 512, they are truncated, retaining only the first 512 tokens; for feature sequences shorter than 512, they are padded with special padding tokens.

[0028] The feature sequence is input into a pre-trained language model, which uses a base model with 12 Transformer encoder layers, a hidden layer dimension of 768, and 12 attention heads. The pre-trained language model encodes the input feature sequence and outputs a context semantic vector corresponding to each token, with each context semantic vector having a dimension of 768. For all field instances corresponding to the same field identifier, their corresponding context semantic vector sets are extracted. Max pooling is performed on this context semantic vector set, i.e., for each dimension of the vector, the maximum value of all vectors in that dimension is taken to generate a target semantic vector with a dimension of 768. For example, the fault description field has 1000 field instances, corresponding to 1000 context semantic vector sets. After taking the maximum value for each dimension, the initial feature vector of the fault description node is obtained, with a dimension of 768. For null value field instances, they do not participate in the max pooling operation. If all instances of a field are null, an all-zero vector is used as its initial feature representation. Examples of heterogeneous work order fields on different platforms are shown in Table 1.

[0029] Table 1 Examples of heterogeneous work order fields on different platforms

[0030] In this embodiment, the contextual semantic vector of field instances is extracted through a pre-trained language model, which can capture the deep semantic information of the fields, rather than relying solely on string matching of field identifiers. The initial feature representation of nodes is generated through max pooling, unifying the feature dimensions of fields across different platforms, eliminating lexical differences caused by different naming conventions for the same business concepts across different platforms, and improving the accuracy of subsequent semantic similarity calculations.

[0031] refer to Figure 2 In another preferred embodiment, the historical time window is divided into continuous time slices according to a preset time step. This includes counting the frequency of changes in change log data within a unit time interval within the historical time window to generate a change frequency sequence. The variance of the change frequency sequence is calculated. When the variance exceeds a preset variance threshold, the unit time interval is reduced to a first target step size. When the variance does not exceed the preset variance threshold, the unit time interval is expanded to a second target step size, where the first target step size is smaller than the second target step size. The historical time window is then segmented according to the determined first or second target step size to generate continuous time slices that adaptively match the fluctuation state of the change frequency sequence.

[0032] Specifically, the historical time window is divided into multiple initial unit time intervals, with each initial unit time interval set to 1 day. The number of changes to the change log data within each initial unit time interval is counted, generating a change frequency sequence. The change count includes the sum of field additions / deletions and field type modifications. For example, a historical time window of the past 90 days generates a change frequency sequence of length 90, where each element represents the number of changes for the corresponding date.

[0033] Calculate the variance of the change frequency series. The formula for calculating the variance of the variance is: in, This represents the variance of the change frequency sequence. Indicates the length of the change frequency sequence. Indicates the first Frequency of change per unit time interval This represents the average value of the change frequency sequence.

[0034] The preset fluctuation threshold is 10. When the calculated fluctuation variance exceeds 10, it indicates that business changes are frequent and the fluctuations are large. In this case, the time step needs to be reduced to capture topology changes more precisely, and the unit time interval is reduced to the first target step of 1 day. When the fluctuation variance does not exceed 10, it indicates that business changes are relatively stable and the fluctuations are small. In this case, the time step can be increased to reduce computational resource consumption, and the unit time interval is increased to the second target step of 7 days.

[0035] The historical time window is divided into consecutive time slices according to the determined time step. For example, when the variance is 15, exceeding the preset variance threshold of 10, a 1-day time step is used to divide the 90-day historical time window into 90 consecutive time slices. When the variance is 5, not exceeding the preset variance threshold of 10, a 7-day time step is used to divide the 90-day historical time window into 13 consecutive time slices. For the last time slice, if its length is less than the determined time step, it is merged with the previous time slice to ensure that the length of all time slices is not less than half of the determined time step. Examples of time slice division under different change frequencies are shown in Table 2.

[0036] Table 2 Examples of time slice division under different change frequencies

[0037] In this embodiment, the time step is adaptively adjusted by statistically analyzing the variance of the change frequency sequence, so that the division of time slices matches the intensity of actual business changes. When business changes are frequent, a smaller time step can more accurately record changes in the topology and improve the accuracy of feature updates; when business changes are stable, a larger time step can reduce the number of time slices, reduce computational resource consumption, and balance computational efficiency and feature accuracy.

[0038] refer to Figure 3 In another preferred embodiment, the spatial aggregation features are obtained by aggregating the first feature vectors of the target node's neighboring nodes within the current time slice. This includes calculating the association weights between the target node and each neighboring node within the current time slice. The association weights are determined based on the dot product of the initial feature representations of the target node and the initial feature representations of the neighboring nodes. The association weights are normalized to obtain the attention coefficients corresponding to each neighboring node. The attention coefficients are then used as weight parameters and weighted summed with the first feature vectors of the corresponding neighboring nodes to generate the spatial aggregation features of the target node in the current time slice.

[0039] Specifically, for the target node within the current time slice Its initial characteristics are represented as Target node The set of neighbor nodes within the current time slice is Neighbor nodes The initial features are represented as Calculate the target node. with neighboring nodes Association weight between The formula for calculating the association weight is: in, This represents the transpose of the initial feature vector of the target node. This represents the initial feature vector of the neighboring nodes.

[0040] For the target node The association weights of all neighboring nodes are normalized to obtain the attention coefficient. The formula for calculating the attention coefficient is: in, This represents an exponential function, with the denominator being the target node. The exponential sum of the association weights of all neighboring nodes.

[0041] The attention coefficient is used as a weight parameter, and a weighted sum is calculated with the first feature vector of the corresponding neighbor node to generate the target node. Spatial aggregation features of the current time slice The formula for calculating spatial aggregation features is: in, Representing neighboring nodes The first eigenvector.

[0042] For isolated nodes without any neighbors, their spatial aggregation features are set as their initial feature vectors. For target nodes with more than 20 neighbors, the top 20 neighbors are selected to participate in the spatial aggregation calculation, sorted by association weight from largest to smallest, to reduce computational complexity.

[0043] In this embodiment, the association weights of neighboring nodes are calculated using a dot product-based attention mechanism. This allows for automatic adjustment of aggregation weights based on the semantic relevance between nodes, resulting in neighboring nodes with higher semantic relevance contributing more to the features of the target node. By generating spatial aggregation features through weighted summation, the contextual semantic information of the target node within the current topology can be captured more accurately, improving the expressive power of node features.

[0044] refer to Figure 4In another preferred embodiment, based on the updated node feature vectors, a semantic similarity matrix between heterogeneous fields is calculated. Low-confidence mappings are cut off by setting a dynamic threshold, generating dynamic alignment mapping rules for cross-platform heterogeneous work orders. This includes calculating the cosine distance between the updated feature vectors of source nodes belonging to different platforms and the updated feature vectors of target nodes, and converting the cosine distance into similarity values ​​in the semantic similarity matrix. An adaptive adjustment coefficient is calculated based on the edge density and number of nodes in the current time slice of the dynamic graph structure. A dynamic threshold is obtained by multiplying a preset base threshold by the adaptive adjustment coefficient. Mapping relationships corresponding to similarity values ​​less than the dynamic threshold in the semantic similarity matrix are filtered out, and mapping relationships greater than or equal to the dynamic threshold are retained to construct dynamic alignment mapping rules.

[0045] Specifically, let the set of nodes of the source platform be... The target platform's node set is Source node The updated feature vector is Target node The updated feature vector is Calculate the source node With the target node Cosine distance between The formula for calculating the cosine distance is: in, Represents the dot product of two vectors. and Let L2 norms of the two vectors be represented respectively.

[0046] Convert cosine distance into semantic similarity value The conversion method is to directly use the cosine distance as the similarity value, that is... The semantic similarity score ranges from -1 to 1, with a higher score indicating a higher semantic similarity between the two nodes.

[0047] Calculate the adaptive adjustment coefficient based on the edge density and number of nodes in the current time slice of the dynamic graph structure. Edge density The calculation formula is: in, This indicates the number of edges in the dynamic graph structure within the current time slice. This indicates the number of nodes in the dynamic graph structure within the current time slice.

[0048] Adaptive adjustment coefficient The calculation formula is: in, This indicates the maximum number of nodes in the dynamic graph structure within the historical time window.

[0049] The preset base threshold is 0.7. The dynamic threshold is obtained by multiplying the preset base threshold by the adaptive adjustment coefficient. The formula for calculating the dynamic threshold is: Those with semantic similarity values ​​less than the dynamic threshold The mapping relationship corresponding to the similarity value is filtered out, and those greater than or equal to the dynamic threshold are retained. The mapping relationships are used to construct dynamic alignment mapping rules. For example, the number of nodes in the dynamic graph structure within the current time slice. Number of edges The largest number of nodes in history Then edge density Adaptive adjustment coefficient Dynamic threshold All mappings with similarity values ​​greater than or equal to 0.23 are retained, while those less than 0.23 are filtered out. For cases where one source node corresponds to multiple target nodes, the target node with the highest similarity value is selected to establish the mapping relationship; similarly, for cases where one target node corresponds to multiple source nodes, the source node with the highest similarity value is selected to establish the mapping relationship. An example of a cross-platform field semantic similarity matrix is ​​shown in Table 3.

[0050] Table 3 Example of cross-platform field semantic similarity matrix

[0051] In this embodiment, semantic similarity between nodes is calculated using cosine distance, which accurately measures the semantic correlation between two high-dimensional feature vectors. An adaptive adjustment coefficient is calculated by combining the edge density and number of nodes in the current time slice to generate a dynamic threshold, which adaptively adjusts the mapping filtering criteria according to the complexity of the topology. When the topology is more complex and the edge density is higher, the dynamic threshold is increased accordingly, filtering out more low-confidence mappings; when the topology is simpler and the edge density is lower, the dynamic threshold is decreased accordingly, retaining more potentially valid mappings and improving the adaptability and accuracy of the mapping rules.

[0052] refer to Figure 5In another preferred embodiment, after generating dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, business constraint rules from the target platform's data dictionary to which the mapped target field belongs are extracted. These business constraint rules include field format constraints and logical dependency constraints between fields. The integrated data is substituted into the business constraint rules for rule verification. If the integrated data violates field format constraints or logical dependency constraints between fields, it is determined that the current dynamic alignment mapping rule has a logical conflict. The target node and source node corresponding to the logically conflicting mapping relationship are extracted, the mapping edge between the target node and the source node is deleted, and the dynamic graph neural network is triggered to re-aggregate and update features and calculate similarity for the remaining nodes.

[0053] Specifically, the target platform's data dictionary stores the business constraint rules for all target-side fields. Field format constraints include field type constraints, length constraints, value range constraints, and regular expression constraints. For example, the work order ID field is constrained to be a numeric string of length 10, the processing time field is constrained to be a floating-point number greater than or equal to 0, and the fault level field is constrained to be an integer with a value between {1, 2, 3, 4}. Logical dependency constraints between fields include equality constraints, inequality constraints, inclusion constraints, and causal constraints. For example, the processing completion time field must be greater than or equal to the processing start time field, the processing priority corresponding to a work order with a fault level of 1 must be urgent, and the value of the associated device ID field must exist in the device ID field of the device information table.

[0054] The integrated data is validated by being substituted into the business constraint rules one by one. For field format constraints, the integrated data is checked to ensure it conforms to the field type, length, value range, and regular expression requirements. For logical dependency constraints between fields, the integrated data is checked to ensure it satisfies the logical relationships between fields. If any piece of integrated data violates any business constraint rule, the mapping relationship corresponding to that piece of data is determined to have a logical conflict. The number of logical conflicts for each mapping relationship is counted. When the number of logical conflicts for a mapping relationship exceeds a preset conflict threshold of 10, the mapping relationship is determined to be an incorrect mapping relationship.

[0055] Extract the target node and source node corresponding to the logically conflicting mapping relationships, and delete the mapping edge between these two nodes. Trigger the dynamic graph neural network to re-aggregate and update features and calculate similarity for the remaining nodes, generating new dynamic alignment mapping rules. Repeat the above business constraint verification process until all integrated data meets the target platform's business constraint rules, or all possible mapping relationships have been traversed. See Table 4 for an example of the business constraint rule verification results.

[0056] Table 4 Examples of Business Constraint Rule Validation Results

[0057] In this embodiment, by introducing the target platform's business constraint rules to validate the integrated data, logical conflict mapping relationships caused by semantic drift or topological changes can be detected. By deleting conflicting mapping edges and triggering re-aggregation updates, erroneous mapping rules can be automatically corrected, ensuring that the integrated data meets the target platform's business logic requirements and improving the quality and reliability of data integration.

[0058] refer to Figure 6 In another preferred embodiment, after generating dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, when a new work order platform is added, the new metadata and new field instances of the new work order platform are parsed to construct new nodes and new edges. The new nodes and new edges are embedded into the cross-platform work order semantic topology graph of the current time slice, and the historical feature vectors of existing nodes are frozen. Dynamic graph neural network aggregation update operations are performed only on the new nodes to obtain the new updated feature vectors of the new nodes. The new similarity value between the new updated feature vector and the updated feature vectors of existing platform nodes is calculated. Based on the comparison result of the new similarity value and a dynamic threshold, incremental alignment mapping rules are generated and the data from the new work order platform is integrated.

[0059] Specifically, when a new work order platform is added, the same metadata extraction method as the existing platform is used to parse the new metadata of the new work order platform, including field identifiers, field types, primary key constraints, and foreign key constraints. All work order data records of the new work order platform are traversed, and the field value corresponding to each field identifier is extracted as a new field instance. Using the new field identifier as a new node, the co-occurrence relationship and dependency relationship between new nodes are calculated, and new edges are constructed. The calculation method for the co-occurrence relationship is the same as described above, and the preset frequency threshold remains 50; the dependency relationship is extracted from the metadata of the new platform.

[0060] The newly added node and edges are embedded into the cross-platform work order semantic topology graph of the current time slice to form the updated topology graph. The historical feature vectors of existing nodes are frozen, meaning their feature vectors remain unchanged across all historical time slices and do not participate in subsequent feature aggregation and update processes. Aggregation and update operations using a dynamic graph neural network are performed only on the newly added node, including spatial aggregation and temporal concatenation. Since the newly added node has no historical feature vectors, temporal concatenation uses only the spatial aggregation features as input to a gated recurrent network to obtain the newly updated feature vector of the new node. During spatial aggregation, the neighboring nodes of the new node include neighbors between new nodes and neighbors among existing nodes that are connected to the new node by edges.

[0061] Calculate the new similarity value between the newly added updated feature vector and the updated feature vectors of all nodes in the existing platform, using the same method as the semantic similarity calculation described above. Compare the new similarity value with the dynamic threshold of the current time slice, retaining new mapping relationships greater than or equal to the dynamic threshold, and generate incremental alignment mapping rules. Based on the incremental alignment mapping rules, integrate the data from the new work order platform into the target platform. For fields in the new platform that do not match existing platform nodes, create corresponding new fields in the target platform and write the data directly into the new fields.

[0062] In this embodiment, when a new work order platform is connected, by embedding the new node and edges into the current topology graph and freezing the historical features of existing nodes, the aggregation update operation is performed only on the new node, avoiding the recalculation of the entire dynamic graph structure, thus reducing computational resource consumption and processing latency. By generating incremental alignment mapping rules, data integration with new platforms can be quickly achieved, improving the system's scalability and flexibility.

Claims

1. A method for cross-platform heterogeneous data integration for multi-source work orders, characterized in that, include: Extract metadata and field instances from multi-source work orders, construct a cross-platform work order semantic topology graph with fields as nodes and dependencies and co-occurrence relationships between fields as edges, and introduce a time dimension to form a dynamic graph structure; A dynamic graph neural network from the field of artificial intelligence is used to aggregate and update the node features of the dynamic graph structure to obtain the drift features of field semantics in data flow. Based on the updated node feature vectors, the semantic similarity matrix between heterogeneous fields is calculated. By setting a dynamic threshold to cut off low-confidence mappings, dynamic alignment mapping rules for cross-platform heterogeneous work orders are generated, thus completing data integration.

2. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, The extraction of metadata and field instances from multi-source work orders involves constructing a cross-platform work order semantic topology graph with fields as nodes and dependencies and co-occurrence relationships between fields as edges. This includes: parsing the header structure of the multi-source work orders to obtain field identifiers, mapping the field identifiers to the nodes, and traversing the data records of the multi-source work orders to extract field values ​​as field instances. Calculate the joint occurrence frequency of field instances corresponding to different field identifiers in the same data record, and determine the co-occurrence relationship of field identifier pairs whose joint occurrence frequency exceeds a preset frequency threshold; Extract the foreign key constraints and primary key constraints corresponding to the field identifier as the dependency relationship, connect the co-occurrence relationship and the dependency relationship as edges to the corresponding nodes, and generate the cross-platform work order semantic topology graph.

3. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, The introduction of a time dimension to form a dynamic graph structure includes: obtaining change log data of the multi-source work orders within a historical time window, wherein the change log data includes field addition / deletion records and field type modification records; The historical time window is divided into continuous time slices according to a preset time step, and the node existence status and edge existence status of the cross-platform work order semantic topology graph are extracted in each time slice. The nodes and edges that undergo state changes between the cross-platform work order semantic topology graphs corresponding to adjacent time slices are connected across time steps to generate temporal edges that represent the topology evolution process. The dynamic graph structure is formed by the temporal edges and the cross-platform work order semantic topology graphs in each time slice.

4. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, The method of using a dynamic graph neural network in the field of artificial intelligence to aggregate and update the node features of the dynamic graph structure and obtain the drift features of field semantics in data flow includes: in each propagation layer of the dynamic graph neural network, for the target node in the dynamic graph structure, aggregating the first feature vectors of the neighboring nodes of the target node in the current time slice to obtain spatial aggregation features; Extract the second feature vector of the target node within the historical time slice, and concatenate the second feature vector with the spatial aggregation feature using temporal features; The concatenated feature vector is input into a gated recurrent network to filter out redundant features in the historical time slices, and the updated feature vector of the target node in the current time slice is output as the representation result of the drift feature.

5. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, The step of calculating the semantic similarity matrix between heterogeneous fields based on the updated node feature vector, cutting off low-confidence mapping by setting a dynamic threshold, and generating dynamic alignment mapping rules for cross-platform heterogeneous work orders includes: calculating the cosine distance between the updated feature vector of the source node belonging to different platforms and the updated feature vector of the target node, and converting the cosine distance into a similarity value in the semantic similarity matrix. Based on the edge density and number of nodes of the dynamic graph structure in the current time slice, the adaptive adjustment coefficient is calculated, and the dynamic threshold is obtained by multiplying the preset basic threshold by the adaptive adjustment coefficient. The mapping relationships corresponding to similarity values ​​less than the dynamic threshold in the semantic similarity matrix are filtered out, and the mapping relationships greater than or equal to the dynamic threshold are retained to construct the dynamic alignment mapping rule.

6. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 2, characterized in that, After extracting field values ​​from the data records of the multi-source work orders as the field instances, the method further includes: performing lexical analysis and part-of-speech tagging on the field instances to identify named entities and business keywords in the field instances; The named entities and the business keywords are combined into a feature sequence, which is then input into a pre-trained language model to extract the contextual semantic vector of the feature sequence in the contextual semantic environment. Max pooling is performed on all context semantic vectors corresponding to the same field identifier. The vector output by the max pooling operation is determined as the target semantic vector. The target semantic vector is used as the initial feature representation of the node in the cross-platform work order semantic topology graph, thereby unifying the feature dimensions of heterogeneous naming of the same business concept on different platforms.

7. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 3, characterized in that, The step of dividing the historical time window into continuous time slices according to a preset time step includes: counting the change frequency of the change log data in a unit time interval within the historical time window, and generating a change frequency sequence; Calculate the fluctuation variance of the change frequency sequence. When the fluctuation variance exceeds a preset fluctuation threshold, reduce the unit time interval to a first target step size. When the fluctuation variance does not exceed the preset fluctuation threshold, expand the unit time interval to a second target step size, wherein the first target step size is smaller than the second target step size. The historical time window is divided according to the determined first or second target step size to generate continuous time slices that adaptively match the fluctuation state of the change frequency sequence.

8. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 4, characterized in that, The step of aggregating the first feature vectors of the target node's neighboring nodes in the current time slice to obtain spatial aggregation features includes: calculating the association weight between the target node and each neighboring node in the current time slice, wherein the association weight is determined based on the dot product operation between the initial feature representation of the target node and the initial feature representation of the neighboring nodes; The association weights are normalized to obtain the attention coefficients corresponding to each neighbor node; The attention coefficient is used as a weight parameter and weighted and summed with the first feature vector of the corresponding neighbor node to generate the spatial aggregation feature of the target node in the current time slice.

9. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, After generating the dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, the method further includes: extracting the business constraint rules from the target platform data dictionary to which the target field mapped in the dynamic alignment mapping rules belongs, wherein the business constraint rules include field format constraints and logical dependency constraints between fields; The integrated data is substituted into the business constraint rules for rule verification. If the integrated data violates the field format constraints or the logical dependency constraints between fields, it is determined that there is a logical conflict in the current dynamic alignment mapping rule. Extract the target node and source node corresponding to the mapping relationship with logical conflict, delete the mapping edge between the target node and the source node, and trigger the dynamic graph neural network to re-aggregate and update features and calculate similarity for the remaining nodes.

10. The cross-platform heterogeneous data integration method for multi-source work orders according to claim 1, characterized in that, After generating the dynamic alignment mapping rules for cross-platform heterogeneous work orders and completing data integration, when a new work order platform is added, the process further includes: parsing the new metadata and new field instances of the new work order platform, and constructing new nodes and new edges. The newly added node and the newly added edge are embedded into the cross-platform work order semantic topology graph of the current time slice, and the historical feature vectors of existing nodes are frozen; The aggregation update operation of the dynamic graph neural network is performed only on the newly added node to obtain the newly updated feature vector of the newly added node; Calculate the new similarity value between the newly added updated feature vector and the updated feature vector of the existing platform node. Based on the comparison result of the new similarity value and the dynamic threshold, generate an incremental alignment mapping rule and integrate the data of the newly added work order platform.

Images