Dynamic knowledge graph updating method based on cross attention, model training method and equipment

Abstract

The invention provides a dynamic knowledge graph updating method based on cross attention and a model training method and device.The updating method comprises the steps that tetrad data of a current time slice and a historical knowledge graph are input into a model, and structured time sequence data are generated by influencing modeling, and the local association information and the random initialization information of the new entity are fused to generate an initial feature. Then, the model constructs propagation information according to the new entity features and historical entity features, and extracts static period information in combination with historical accumulated time sequence signals; and finally, interacting the to-be-updated entity features with the propagation information through a cross attention mechanism, and fusing the to-be-updated entity features with the periodic information to generate updated entity features. Semantic alignment of new knowledge of the knowledge graph and the historical period can be achieved, disastrous forgetting is effectively avoided, meanwhile, the knowledge graph updating efficiency and reliability can be effectively improved, and the resource occupancy rate of computing equipment is reduced.

Description

Technical Field

[0001] This application relates to the field of knowledge graph processing technology, and in particular to a dynamic knowledge graph update method, model training method and device based on cross-attention. Background Technology

[0002] Knowledge graphs (KGs) are a core technological support in the industrial field, widely used in search engines, recommendation systems, enterprise data management, Industrial Internet of Things (IIoT), smart manufacturing, supply chain optimization, and other scenarios. Their real-time dynamic update capability directly determines the effectiveness of system decisions and business adaptability. Traditional knowledge graphs represent entities and relationships using static triples (e, r, o), failing to capture the temporal dependencies of data; while temporal knowledge graphs (TKGs) introduce a time dimension through quadruples (e, r, o, t). To effectively improve the real-time requirements of continuous flow applications such as Industrial Internet of Things (IIoT), smart manufacturing, and supply chain management, timely dynamic updates of temporal knowledge graphs are necessary.

[0003] However, existing methods for dynamically updating time-series knowledge graphs have significant limitations. New entities (such as newly added equipment or unknown sensors) frequently appear, and existing methods struggle to initialize their features using existing entity association information, resulting in a severe cold-start problem (a technical challenge in dynamic knowledge graphs where newly added entities lack historical interaction information, making it difficult to generate effective feature representations). Data often exhibits periodic patterns (such as equipment maintenance cycles, seasonal supply chain demand fluctuations, and production process cycles), and existing methods cannot effectively capture these long-term evolutionary trends, leading to outdated feature representations for inactive entities. Furthermore, existing graph neural networks (such as GCN and RGCN) are primarily designed for static graphs; updating new data requires retraining the entire graph, resulting in extremely high computational costs and failing to meet the real-time requirements of many application scenarios.

[0004] Therefore, designing a dynamic knowledge graph update method that supports continuous flow scenarios, solves the cold start problem, captures periodic patterns, and is efficient has become a key requirement for the implementation of intelligent systems. Summary of the Invention

[0005] In view of this, embodiments of this application provide a dynamic knowledge graph update method, model training method, and device based on cross-attention, in order to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides a dynamic knowledge graph update method based on cross-attention, including: The quadruple data of the current time slice corresponding to the continuous temporal knowledge graph stream and the historical knowledge graph corresponding to the current time slice are input into the dynamic knowledge graph update model, so that the dynamic knowledge graph update model performs the following: influence modeling on the quadruple data to generate structured temporal data containing new entity information; based on the structured temporal data, generating the initial features of the new entity by fusing the local association information and random initialization information of the new entity; constructing entity propagation information based on the initial features of the new entity and its associated historical entity features; extracting static periodic information representing the periodic evolution pattern based on the historically accumulated temporal signals generated by the dynamic knowledge graph update model in previous time slices; obtaining the entity features to be updated from the historical knowledge graph, interacting the entity features to be updated with the entity propagation information through a cross-attention mechanism, and fusing the interaction result with the static periodic information to generate updated entity features; The dynamic knowledge graph is updated based on the updated entity features, and the updated dynamic knowledge graph is output as the historical knowledge graph corresponding to the next time slice.

[0007] In some embodiments of this application, the dynamic knowledge graph update model includes: The data processing module is configured to perform impact modeling on the quadruple data of the current time slice and generate structured time-series data containing information about new entities. The new entity initialization module is configured to generate the initial features of the new entity based on the structured time-series data by fusing the local association information and random initialization information of the new entity. The propagation information preprocessing module is configured to construct entity propagation information based on the initial characteristics of the new entity and the characteristics of its associated historical entities; The historical information update module is configured to extract static periodic information representing the periodic evolution pattern based on the time-series signals generated by the dynamic knowledge graph update model in previous time slices, which are accumulated historically. A cross-attention update layer is configured to obtain entity features to be updated from the historical knowledge graph, and through a cross-attention mechanism, interact the entity features to be updated with the entity propagation information, and fuse the interaction result with the static periodic information to generate updated entity features.

[0008] In some embodiments of this application, the data processing module is specifically configured to: Real-time impact modeling is performed on the quadruple data of the current time slice to extract the direct association quadruple features of new entities; Furthermore, the continuous impact modeling of the quadruple data in the current time slice includes: based on the historical knowledge graph, extracting the indirect association information of the new entity to downstream entities through its associated entities within the historical window using the relational graph convolutional network RGCN, and aggregating the indirect association information to obtain indirect association features; The structured time-series data is generated by fusing the directly associated quadruple features with the indirectly associated features.

[0009] In some embodiments of this application, the new entity initialization module is specifically configured to: Based on the new entity and the object entities associated with it in the current time slice, extract all historical entities connected to the object entity through historical relationships from the historical knowledge graph. Construct a local subgraph with the new entity, the object entity, and the historical entities as nodes and the relationships between the new entity, the object entity, and the historical entities as nodes as edges. The local subgraph is aggregated with multi-hop relation features based on the relation graph convolutional network RGCN to generate entity features based on historical context. The entity features based on historical context are mixed with a random initialization vector as random initialization information to enhance the representation and generate the initial features of the new entity.

[0010] In some embodiments of this application, the propagation information preprocessing module is specifically configured to: Based on the initial characteristics of the new entity, the characteristics of related historical entities obtained from the historical knowledge graph, and their relationship characteristics, direct propagation messages and indirect propagation messages are constructed; wherein, the direct propagation message is used to characterize the influence of the new entity on its directly related entities; the indirect propagation message is used to characterize the indirect influence of the new entity on its downstream entities through multi-hop relationships; The directly propagated messages and the indirectly propagated messages are integrated to generate entity propagation information.

[0011] In some embodiments of this application, the historical information update module is specifically configured to: Centered on the relationships associated with inactive entities, a dynamic local subgraph is sampled, and a time-series signal unit for the current time slice is generated based on the dynamic local subgraph. The timing signal unit of the current time step is stored in the periodic pattern cache, and together with the historical signal, it constitutes the historical timing signal sequence as the timing signal. The historical time-series signal sequence is read from the periodic pattern buffer, the historical time-series signal sequence is transformed in the frequency domain, a target periodic pattern is selected based on the energy intensity of the frequency components, and the signals in the historical time-series signal sequence are rearranged according to the length corresponding to the target periodic pattern to obtain the rearranged two-dimensional signal. A convolution operation is performed on the rearranged two-dimensional signal structure to output static periodic information representing the periodic evolution pattern.

[0012] In some embodiments of this application, the cross-attention update layer is specifically configured to: The entity features to be updated are obtained from the historical knowledge graph, and the entity features to be updated are concatenated with the entity propagation information to form an input matrix. Perform self-attention calculation on the input matrix to obtain the self-attention calculation result; Extract the first part of the data associated with the entity feature to be updated from the self-attention calculation result, generate a secondary key vector based on the first part of the data, and use the remaining data in the self-attention calculation result excluding the first part of the data as a query to perform cross-attention calculation with the secondary key vector to obtain the cross-attention calculation result. The cross-attention calculation result is weighted and fused with the static periodic information, and then subjected to layer normalization to generate the updated entity features.

[0013] Another aspect of this application provides a method for training a dynamic knowledge graph update model, including: Obtain a training dataset, which contains quadruple data of multiple time slices of a continuous temporal knowledge graph stream and their corresponding historical knowledge graphs. The quadruple data of each time slice in the training dataset and the corresponding historical knowledge graph are sequentially input into the dynamic knowledge graph update model so that the dynamic knowledge graph update model generates updated entity features for each time slice. Based on the updated entity features and preset labels corresponding to each time slice, the contrastive learning loss is calculated; Based on the contrastive learning loss, the parameters of the dynamic knowledge graph update model are optimized end-to-end using the backpropagation algorithm to obtain a trained dynamic knowledge graph update model for executing the cross-attention-based dynamic knowledge graph update method described in the first aspect.

[0014] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the dynamic knowledge graph update method based on cross-attention described in the first aspect, and / or implements the dynamic knowledge graph update model training method described in the second aspect.

[0015] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dynamic knowledge graph update method based on cross-attention as described in the first aspect above, and / or implements the dynamic knowledge graph update model training method as described in the second aspect above.

[0016] The fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, implements the dynamic knowledge graph update method based on cross-attention as described in the first aspect, and / or implements the dynamic knowledge graph update model training method as described in the second aspect.

[0017] The dynamic knowledge graph update method based on cross-attention provided in this application inputs the quadruple data of the current time slice corresponding to the continuous temporal knowledge graph stream and the historical knowledge graph corresponding to the current time slice into a dynamic knowledge graph update model, so that the dynamic knowledge graph update model performs: influence modeling on the quadruple data to generate structured temporal data containing new entity information; based on the structured temporal data, generating the initial features of the new entity by fusing the local association information and random initialization information of the new entity; constructing entity propagation information based on the initial features of the new entity and its associated historical entity features; and based on the historically accumulated information accumulated by the dynamic knowledge graph update model in previous time slices... The time-series signal generated by the slices is used to extract static periodic information representing the periodic evolution pattern. Entity features to be updated are obtained from the historical knowledge graph. Through a cross-attention mechanism, these entity features interact with the entity propagation information, and the interaction result is fused with the static periodic information to generate updated entity features. The dynamic knowledge graph is updated based on these updated entity features, and the updated dynamic knowledge graph is output as the historical knowledge graph corresponding to the next time slice. The cross-attention mechanism achieves coordinated updates of real-time propagation and periodic evolution, achieving semantic alignment between new knowledge and historical periods, effectively avoiding catastrophic forgetting and improving the reliability of model applications. The local update strategy reduces computational complexity to linear levels, thereby effectively improving knowledge graph update efficiency and significantly reducing the resource consumption of computing devices. Through hybrid initialization and periodic modeling, the challenges of cold start and feature obsolescence are systematically solved, ensuring the completeness and accuracy of the knowledge graph in dynamic environments. This meets the millisecond or minute-level real-time requirements for knowledge graph updates in scenarios such as real-time fault diagnosis and dynamic supply chain adjustments.

[0018] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0019] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings: Figure 1 This is a flowchart illustrating a dynamic knowledge graph update method based on cross-attention in one embodiment of this application.

[0021] Figure 2 This is a schematic diagram of the first architecture of the dynamic knowledge graph update model in one embodiment of this application.

[0022] Figure 3 This is a schematic diagram of the second architecture of the dynamic knowledge graph update model in one embodiment of this application.

[0023] Figure 4 This is a flowchart illustrating the dynamic knowledge graph update model training method in one embodiment of this application.

[0024] Figure 5 This is a schematic diagram showing the comparison of experimental results in an application example of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0026] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0027] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0028] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0029] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0030] It should be noted that in one existing approach, traditional temporal knowledge graph (TKG) methods, represented by TTransE (Temporal Extended TransE) and DE-SimplE, are based on discrete snapshot modeling. They first divide the continuous time axis into several fixed time windows, each window corresponding to a static subgraph (i.e., a snapshot). Then, feature representations are trained independently for each snapshot. For example, TTransE uses timestamps to generate translation vectors to model time information, while DE-SimplE uses a static model to capture the knowledge structure within the snapshot. When new data arrives, these methods generate new snapshots and retrain or fine-tune the model, thus achieving a "semi-dynamic" update mechanism.

[0031] However, this type of modeling based on discrete snapshots has several limitations in practical applications: First, discrete snapshots do not match continuous data streams, and fixed time windows are difficult to adapt to the real-time requirements of application scenarios. For example, sudden equipment failure data often needs to be updated immediately, otherwise it will lead to significant update delays. Second, this type of method performs poorly in integrating new entities, relying only on information within the current snapshot for initialization, and cannot fully utilize the relationships between historical entities, thus leading to prominent cold start problems. In addition, since snapshots are independent of each other, the model has difficulty capturing long-term periodic patterns across time windows (such as annual supply chain fluctuations). Finally, as time goes by and data accumulates, each new snapshot requires retraining or fine-tuning of the model, which dramatically increases computational overhead and results in low overall efficiency.

[0032] In another existing approach, semi-dynamic temporal knowledge graph (TKG) methods, represented by CyGNet and StreamE, are based on the core idea of ​​"static graph feature extraction + incremental fine-tuning". These methods typically extract features from static graphs using RGCN (Relational Graph Convolutional Network, a graph neural network that supports heterogeneous relation modeling and can aggregate node features of different relation types) or attention mechanisms to capture the complex heterogeneous relation structure in the graph. When new data arrives, fine-tuning is performed only on local subgraphs directly related to the new entities, thus avoiding retraining the entire graph. In addition, they also save feature representations of some entities through a history caching mechanism to reduce redundant computation and improve update efficiency.

[0033] These semi-dynamic methods still have many limitations in complex time-series scenarios. They lack the ability to model periodic patterns, focusing only on short-term local updates. Therefore, they cannot extract potential periodic features in the data (such as equipment operating cycles), causing the features of inactive entities to gradually become outdated. Furthermore, the update granularity of local fine-tuning is relatively limited, only covering nodes directly related to the new entity, making it difficult to achieve efficient knowledge propagation globally. For example, changes in one link of a supply chain may have a cascading effect on the entire chain. Although these methods utilize local association information to some extent to alleviate the cold-start problem, the accuracy of new entity features remains insufficient due to the lack of hybrid feature enhancement mechanisms. Frequent incremental fine-tuning may also disrupt semantic consistency, causing misalignment in the embedding spaces of new and old entities, resulting in the model losing its ability to retain historical knowledge while learning new knowledge.

[0034] Based on this, in order to design a dynamic knowledge graph update method that supports continuous flow scenarios, solves the cold start problem, captures periodic patterns, and is efficient, this application provides a dynamic knowledge graph update method based on cross-attention, a dynamic knowledge graph update model training method, an electronic device, a computer-readable storage medium, and a computer program product, respectively, aiming to solve the problem of dynamic update of existing time-series knowledge graphs in continuous flow scenarios.

[0035] The following examples will provide a detailed description.

[0036] Based on this, embodiments of this application provide a dynamic knowledge graph update method based on cross-attention, which can be implemented by a dynamic knowledge graph update device based on cross-attention. See [link to relevant documentation]. Figure 1 The dynamic knowledge graph update method based on cross-attention specifically includes the following: Step 100: Input the quadruple data of the current time slice corresponding to the continuous temporal knowledge graph stream and the historical knowledge graph corresponding to the current time slice into the dynamic knowledge graph update model, so that the dynamic knowledge graph update model performs the following: perform influence modeling on the quadruple data to generate structured temporal data containing new entity information; based on the structured temporal data, generate the initial features of the new entity by fusing the local association information and random initialization information of the new entity; construct entity propagation information according to the initial features of the new entity and its associated historical entity features; extract static periodic information representing the periodic evolution pattern based on the historically accumulated temporal signals generated by the dynamic knowledge graph update model in previous time slices; obtain the entity features to be updated from the historical knowledge graph, interact the entity features to be updated with the entity propagation information through a cross-attention mechanism, and fuse the interaction result with the static periodic information to generate updated entity features.

[0037] In step 100, a continuous temporal knowledge graph stream refers to a sequence of knowledge graph quadruples arriving sequentially according to their timestamps; it can also be simply referred to as a continuous stream. Specifically, a continuous temporal knowledge graph stream... ;in, As the main entity, For the relationship, As an object entity, For timestamps, It refers to the first time period in each time period. This refers to the last time period within a series of time intervals, where the time intervals are separated by time granularity. Intervals are made, and T is the total number of time periods. It is the input source of the dynamic knowledge graph update model (which can be simply referred to as the model in this embodiment).

[0038] Before step 100, you can first... By time granularity (Easily adjustable to 1 minute or 1 hour) Divided into time slices Each time slice contains all the quadruples for its corresponding time period. Then, slice each time period. Each slice is used as a current time slice and step 100 is executed accordingly.

[0039] A historical knowledge graph refers to all entities, relations, and their feature representations accumulated up to the current time slice, denoted as . The historical knowledge graph corresponding to the current time slice refers to the latest updated dynamic knowledge graph before the current time slice is input into the model. As another input to the model, it is used to provide historical context and semantic background for new entity initialization and feature updates.

[0040] It should be noted that impact modeling refers to a method for analyzing and quantifying the dynamic effects of newly emerging knowledge (quadruples) on the existing knowledge graph state. To further improve its modeling accuracy and provide a more reliable and effective data foundation for subsequent operations, impact modeling can be divided into immediate impact modeling (capturing the effect of new entities on their immediate neighbors) and persistent impact modeling (simulating the long-term, decaying impact of new entities on downstream entities through multi-hop relationships using RGCN), which will be explained in detail in subsequent embodiments.

[0041] The structured time-series data containing new entity information refers to the result data after impact modeling of quadruple data. This structured time-series data is no longer the original quadruple, but rather a feature representation that integrates impacts (such as immediate impacts (direct associations) and persistent impacts (indirect associations extracted through RGCN)), forming a standardized data block with a time dimension that can be used for subsequent module calculations.

[0042] The local association information of the new entity refers to the information that can be obtained from the new entity through a graph convolutional network (RGCN). Local subgraph The information obtained from aggregation is used to represent the new entity. Historical context semantics obtained from its neighboring historical entities.

[0043] The random initialization information can be a random initialization vector. It consists of a uniform distribution. Generate, where, This is a hyperparameter that controls the intensity of the disturbance (default 0.1). The random initialization information is used to ensure that new entities retain their uniqueness while integrating into the historical context.

[0044] The initial characteristics of the new entity It can also be called the initial feature of the new entity, which can be obtained by linear interpolation of local association information and random initialization information.

[0045] The historical entity characteristics refer to those derived from historical knowledge graphs. The feature vector of an existing entity that is directly or indirectly related to the current new entity, obtained from the previous state of the dynamic knowledge graph (i.e., the previous state of the dynamic knowledge graph).

[0046] The entity propagation information is a constructible message matrix M, which can be composed of direct propagation messages and indirect propagation messages, encoding the possible change paths that a new entity may bring to the knowledge graph.

[0047] The time-series signal refers to a data sequence stored in the periodic pattern cache, representing the evolution of entity relationship behavior over time. It is generated by the model through dynamic sampling, relationship-aware clustering, and MLP when processing historical time slices, and is the data foundation for periodic pattern analysis.

[0048] The static periodic information can refer to a tensor that encapsulates long-term periodic patterns. It can be obtained by performing a Fast Fourier Transform (FFT) and a two-dimensional convolution (Conv2D) on the historical time-series signal sequence S.

[0049] The entity features to be updated This refers to historical knowledge graphs The entity features obtained from the data that need to be updated by the model in the current time slice; these entity features to be updated can also be referred to as the affected entity features. It can include at least two types of entities: entities affected by the spread of new entities and identified long-term inactive entities.

[0050] Understandably, the Cross-Attention Mechanism first captures internal associations through self-attention computation, and then performs cross-attention computation between the output of self-attention and the original features of the entity to be updated, so as to achieve semantic space alignment between the newly propagated knowledge and the historical entity features and avoid catastrophic forgetting.

[0051] The updated entity features This refers to the final output of the entire model, also known as the final entity feature. It involves interacting with the entity feature to be updated using a cross-attention mechanism, followed by the entity propagation information, and then combining this with static periodic information. It is obtained by merging and then undergoing layer normalization (LayerNorm).

[0052] Step 200: Update the dynamic knowledge graph based on the updated entity features, and output the updated dynamic knowledge graph as the historical knowledge graph corresponding to the next time slice.

[0053] In step 200, updating the dynamic knowledge graph typically involves overwriting or replacing the old data structure in computer memory or storage with a data structure containing the updated entity features. For new entities, this is an insertion operation; for existing entities, it is a modification operation.

[0054] It is understandable that the dynamic knowledge graph updated after each time slice is predicted by the model is the historical knowledge graph corresponding to the next time slice. Specifically, in a real system, it may include: At time t, the model reads from storage the version of the dynamic knowledge graph corresponding to the previous time slice, which is updated by the model after prediction. Historical knowledge graph as a slice of the current time ; After processing the data at time t, the results are written back to storage, at which point the version in storage is updated to... .

[0055] When processing the data at time t+1, the system automatically... Read from storage as a new historical knowledge graph.

[0056] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application achieves collaborative updates of instant propagation and cyclical evolution through the cross-attention mechanism. This mechanism achieves semantic alignment between new knowledge and historical cycles, effectively avoiding catastrophic forgetting and improving the reliability of model applications. The adoption of a local update strategy reduces computational complexity to linear levels, thereby effectively improving knowledge graph update efficiency and significantly reducing the resource consumption of computing devices. Through hybrid initialization and cyclical modeling, the challenges of cold start and feature obsolescence are systematically addressed, ensuring the completeness and accuracy of the knowledge graph in dynamic environments. This meets the millisecond or minute-level real-time requirements for knowledge graph updates in scenarios such as real-time fault diagnosis and dynamic supply chain adjustments.

[0057] To address the challenge of reliably and efficiently updating dynamic knowledge graphs through a highly collaborative and clearly defined dedicated system architecture, this application provides a dynamic knowledge graph updating method based on cross-attention, see [link to relevant documentation]. Figure 2 The dynamic knowledge graph update model in the cross-attention-based dynamic knowledge graph update method specifically includes the following: (1) Data processing module 1 is configured to perform impact modeling on the quadruple data of the current time slice and generate structured time series data containing new entity information.

[0058] (2) New entity initialization module 2, configured to generate the initial features of the new entity based on the structured time series data by fusing the local association information and random initialization information of the new entity.

[0059] (3) Propagation information preprocessing module 3, configured to construct entity propagation information based on the initial characteristics of the new entity and the characteristics of its associated historical entities.

[0060] (4) Historical information update module 4, configured to extract static periodic information representing the periodic evolution pattern based on the time-series signal generated by the dynamic knowledge graph update model in previous time slices accumulated in history.

[0061] (5) Cross-attention update layer 5, configured to obtain entity features to be updated from the historical knowledge graph, and through the cross-attention mechanism, interact the entity features to be updated with the entity propagation information, and fuse the interaction result with the static periodic information to generate updated entity features.

[0062] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application forms an efficient feature computation and fusion pipeline system (hereinafter referred to as the system) with five modules in the model. This collaborative architecture allows each module to process its specific tasks in parallel or pipelined. Compared with a single model with ambiguous functions, it can significantly reduce the computational latency of computing devices and improve system throughput, thereby meeting the needs of higher-frequency real-time stream processing. Since each module undertakes a specific function (such as data processing, cold start processing, periodic extraction, etc.), when the system deviates or needs optimization, the corresponding module can be accurately located and adjusted without making black-box adjustments to the entire model, which can also greatly reduce the debugging and maintenance costs of the system. It can also suppress the negative impact of one type of update (such as a large influx of new entities) on another type of update (such as periodic pattern learning) at the system level, and ensure the stable output of the system under various data scenarios. In addition, the clear module division allows the system to use dedicated hardware acceleration units (such as GPUs and FPGAs) to optimize different computationally intensive modules (such as RGCN computation and FFT transformation), laying the foundation for realizing hardware and software co-design and further improving efficiency.

[0063] To capture both short-term, direct, immediate impact and long-term, indirect, persistent impact, and to avoid confusion between these two impact characteristics, while effectively injecting historical context information into the current data processing flow, this application provides a dynamic knowledge graph update method based on cross-attention. Specifically, data processing module 1 is the entry component of the dynamic knowledge graph update model, responsible for converting the original four-tuple data stream into semantically rich structured time-series data. This module captures both the immediate and long-term effects of new knowledge through a dual-impact modeling mechanism.

[0064] See Figure 3 The data processing module 1 in the dynamic knowledge graph update model specifically includes the following: (1) Instant impact modeling unit 11, configured to perform instant impact modeling on the quadruple data of the current time slice and extract the direct association quadruple features of the new entity.

[0065] Specifically, the immediate impact modeling unit 11 scans all quadruples in the current time slice, identifies all quadruples containing new entities, extracts the feature representations of these quadruples, and for each new entity... These respectively form direct association quadruple features that encode the immediate influence strength of a new entity on its direct neighbors. The direct association quadruple features are derived from the direct association quadruples. The extracted feature vectors are directly associated with quadruples to capture new entities. For adjacent entities Its immediate effect; It is a new entity Adjacent entities; It is a new entity Its adjacent entities The i-th type of relationship between them, such as connection type or interaction method.

[0066] (2) Persistent impact modeling unit 12, configured to perform persistent impact modeling on the quadruple data of the current time slice, wherein the persistent impact modeling includes: based on the historical knowledge graph, extracting the indirect association information generated by the new entity on the downstream entity through its associated entities in the historical window through the relational graph convolutional network RGCN, and aggregating the indirect association information to obtain indirect association features.

[0067] Specifically, for each new entity Find the directly related object entity. Based on historical knowledge graphs The process is performed using a single-layer relational graph convolutional network (RGCN): extracting directly related object entities. All indirectly related quadruples within the history window ; It is a new entity and its downstream entities The j-th type of relationship between them, such as connection type or interaction method; history window Configurable (default: 5 time steps); features are propagated along the relationship path to capture... through propagation to downstream entities The long-term effect is observed; an influence attenuation factor is applied, which decreases exponentially with propagation distance, simulating the real-world phenomenon of "echoes weakening with change," and outputting indirect correlation features. It can be understood that the indirect correlation features are obtained by aggregating one dimension from the indirect correlation quadruple. d The feature vector is a distributed representation obtained by RGCN from multiple historical quadruples describing indirect association paths.

[0068] (3) Feature fusion unit 13, configured to fuse the directly associated quadruple features and the indirectly associated features to generate the structured time series data.

[0069] Specifically, the structured time-series data for the current time slice can be generated by fusing the direct association quadruple features, which serve as immediate impact features, with the indirect association features, which serve as persistent impact features. In practice, the structured time-series data can be represented as an initial entity feature matrix with dimensions N×D, where N×D is the number of entities involved in the current time slice, and D is the dimension of the feature vectors.

[0070] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application, by processing directly associated quadruple features and indirectly associated features separately, ensures that the feature representations of short-term and long-term effects are clearly separated, providing cleaner and more targeted input features for subsequent modules, thereby improving the feature representation quality of each model. Through RGCN aggregation based on historical knowledge graphs, the indirectly associated features contain potential future impact paths that new entities may trigger. This ensures that the generated structured time-series data not only includes the current state but also predictive information about the future evolution of the system, significantly enhancing the value of the data. The immediate impact modeling (which relies on current data) and the continuous impact modeling (which relies on historical data) are logically relatively independent. This structural design allows for potential parallel computing, further improving the data processing speed of the hardware.

[0071] To further address the issue of how the new entity initialization module specifically implements cold-start entity feature generation, this application provides a dynamic knowledge graph update method based on cross-attention. For the cold-start problem of new entities, this module generates initial features through three steps: "local subgraph construction - relation feature aggregation - hybrid representation enhancement".

[0072] See Figure 3 The new entity initialization module 2 in the dynamic knowledge graph update model specifically includes the following: (1) Local subgraph construction unit 21 is used to extract all historical entities connected to the object entity through historical relationships from the historical knowledge graph based on the new entity and the object entity associated with it in the current time slice, and construct a local subgraph with the new entity, the object entity and the historical entity as nodes and the relationship between the new entity, the object entity and the historical entity as nodes as edges.

[0073] Among them, historical entities refer to those in the historical knowledge graph. In China, through historical relationships and objective entities A connected set of entities can be denoted as (i.e., through relationships) With the object entity (Connected historical entities). Historical entities provide rich historical semantic context for new entities. That is, for new entities... Related object entities (From the quadruple) Extract all related to Entities with historical connections.

[0074] Local subgraph It refers to the local network structure built for new entities. , where the node set edge set for The subgraph contains all relational edges between entities. This subgraph forms the computational basis for initializing new entities.

[0075] (2) Relationship graph convolutional network layer 22, used to perform multi-hop relationship feature aggregation on the local subgraph based on the relationship graph convolutional network RGCN, so as to generate entity features based on historical context.

[0076] Among them, multi-hop relationship feature aggregation refers to the information propagation and aggregation process carried out on local subgraphs through relation graph convolutional networks (RGCN) to capture the indirect associations between new entities and historical entities through multi-hop relationship paths.

[0077] Specifically, RGCN is used for local subgraphs. Multi-hop feature propagation is performed, where, For new entities A set of related relationships To pass through relationships With new entities The set of all directly connected neighboring entities (i.e., adjacent entities). For relational parameter matrices, For bias, for Activation function.

[0078] The formula for the aggregation of multi-hop relationship features is as follows: in, These are entity features based on historical context obtained after aggregation via RGCN.

[0079] (3) Hybrid representation enhancement unit 23, used to perform hybrid representation enhancement on the entity features based on historical context and a random initialization vector as random initialization information to generate the initial features of the new entity.

[0080] Specifically, linear interpolation Aggregated features and randomly initialized vectors are used to balance historical association information with the uniqueness of new entities; the initial features of the new entities are... The calculation formula is as follows: in, [0, 1] represents adaptive weights (learned through data). for A uniformly distributed random initialization vector on the surface.

[0081] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application constructs a local subgraph containing new entities, object entities, and their historically related entities, providing rich historical semantic context for new entities. This results in generated initial features with stronger semantic rationality and practicality. Multi-hop relation feature aggregation based on RGCN can capture second-order and higher-order indirect association information of new entities, generating deeper and richer feature representations compared to methods using only direct associations. Hybrid representation enhancement by combining entity features based on historical context with randomly initialized vectors leverages the guiding role of historical association information while maintaining the uniqueness and distinguishability of new entities, effectively preventing excessive assimilation of new entity features by historical patterns.

[0082] To further address the issue of how the information preprocessing module specifically constructs and integrates knowledge dissemination messages, this application provides a dynamic knowledge graph update method based on cross-attention. In this method, the module achieves efficient dissemination and semantic alignment of new knowledge through "message passing-cross-attention".

[0083] See Figure 3 The propagation information preprocessing module 3 in the dynamic knowledge graph update model specifically includes the following: (1) The propagation message construction unit 31 is used to construct direct propagation messages and indirect propagation messages based on the initial characteristics of the new entity, the characteristics of related historical entities obtained from the historical knowledge graph and their relationship characteristics; wherein, the direct propagation message is used to characterize the influence of the new entity on its directly related entities; the indirect propagation message is used to characterize the indirect influence of the new entity on its downstream entities through multi-hop relationships.

[0084] Specifically, based on the translation invariance assumption of TransE, a direct propagation message is constructed. Indirect communication of messages. Direct communication of messages. (New Entity → Neighboring Entities) represents the immediate impact of a new entity on its directly related entities, reflecting the direct role of the new entity under a specific relationship and the direct propagation of messages. The calculation formula is as follows: in, It is the feature vector of the new entity; and It is the relation feature vector, where, It is the first jump relationship. It is the second jump relationship.

[0085] Indirect dissemination of messages The calculation formula for (new entity → neighbor → downstream entity) is as follows: in, Features of neighboring entities; For the MLP parameter matrix, To affect the scaling factor, ⊙ indicates element-wise multiplication. Multi-hop relationships refer to indirect association paths formed between a new entity and its downstream entities through adjacent entities.

[0086] (2) Message integration unit 32 is used to integrate the direct propagation message and the indirect propagation message to generate entity propagation information.

[0087] Understandably, message integration refers to the process of organizing various types of communication messages into a unified matrix form, ensuring that all communication information has a consistent representation format and dimensions.

[0088] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application separates and independently models the influence of different paths and properties in the knowledge propagation process by constructing direct propagation messages and indirect propagation messages respectively, providing fine-grained input information for subsequent accurate updates. Direct propagation messages capture local direct effects, while indirect propagation messages encode global indirect effects. The combination of the two provides a complete propagation context for affected entities, significantly enhancing the semantic richness of entity propagation information. Integrating the two types of propagation messages into a unified entity propagation information matrix not only maintains the characteristics of different types of influences but also provides a standardized interface for subsequent cross-attention update layers, thereby reducing the coupling complexity between modules.

[0089] To further address the issue of how the historical information update module can specifically capture and utilize periodic patterns, this application provides a dynamic knowledge graph update method based on cross-attention. In this method, the module captures the long-term evolution patterns of inactive entities through "FFT periodic extraction-2D convolution fusion".

[0090] See Figure 3 The historical information update module 4 in the dynamic knowledge graph update model specifically includes the following: (1) The timing signal unit generation unit 41 is used to sample the dynamic local subgraph centered on the relationship associated with the inactive entity, and generate the timing signal unit of the current time slice based on the dynamic local subgraph.

[0091] Specifically, relationships associated with inactive entities Centered on, random walk sampling dynamic local subgraph , The set of nodes representing a dynamic local subgraph; Represents the edge set of a dynamic local subgraph. A relation-aware classifier is used to... Divided into For each semantic cluster, calculate the feature mean of the semantic cluster. and relational features The time-series signal units are generated by splicing and using a multilayer perceptron (MLP).

[0092] in, This represents the number of entities contained in the k-th semantic cluster; Represents a single entity belonging to a semantic cluster; Represents a relation-specific weight matrix used for relation-specific linear transformations of entity features; Representing entities The eigenvector at the previous time t-1; This represents the time-series signal unit generated by the k-th semantic cluster at time t; Representing relations The eigenvector at time t; Let represent a d-dimensional real vector space.

[0093] (2) Data caching unit 42 is used to store the timing signal unit of the current time step into the periodic mode cache, and together with the historical signal, constitute the historical timing signal sequence as the timing signal.

[0094] (3) Data reading unit 43 is used to read the historical time sequence from the periodic pattern buffer, perform frequency domain transformation on the historical time sequence, select a target periodic pattern based on the energy intensity of the frequency component, and rearrange the signals in the historical time sequence according to the length corresponding to the target periodic pattern to obtain the rearranged two-dimensional signal.

[0095] Specifically, read recent data from the periodic pattern cache (also known as the history cache). Historical time series signal sequence at each time step This is concatenated with the current signal to obtain a complete sequence, and then... conduct Transformation, Extraction Significant frequency components (Corresponding to the target periodic pattern) as the rearranged two-dimensional signal .

[0096] The standard formula for the Fast Fourier Transform (FFT) is as follows: Where t represents the current time step, For time step index, For frequency. This indicates that at time t, the relationship... The value of the time-series signal at time point n; This represents a complex exponential basis function. Top-K refers to selecting the K frequency components with the highest energy, corresponding to the most prominent periodic patterns in the data.

[0097] (4) Convolution unit 44 is used to perform convolution operation on the rearranged two-dimensional signal structure to output static periodic information representing the periodic evolution pattern.

[0098] Specifically, rearranging sequences according to period length The dimension is obtained as tensor (B represents the batch size) For the first One cycle, (As a feature dimension). A 3×3 convolution kernel is used to capture the interactive features of "intra-cycle fluctuations - cross-cycle trends": in, This represents a two-dimensional convolution operation; C represents the number of output channels of the convolution operation; N represents the feature dimension.

[0099] As described above, the dynamic knowledge graph update method based on cross-attention provided in this application can accurately identify inherent patterns in the data through FFT frequency domain transformation and energy analysis, providing a basis for feature updates that conform to historical evolution for inactive entities; feature updates based on historical periodic patterns enable the system to have predictive maintenance capabilities, allowing it to adapt to expected periodic changes in advance and significantly improve the system's decision-making quality in periodic scenarios; through the complete process of "signal generation - frequency domain analysis - rearrangement - convolution fusion", complex temporal pattern recognition can be transformed into efficient and computable operations; the dynamically updated periodic pattern caching mechanism enables the module to adapt to the changes in the periodic patterns themselves over time, avoiding model degradation caused by fixed periodic assumptions.

[0100] To further address the issue of how the cross-attention update layer specifically implements multi-source information fusion and semantic alignment, this application provides a dynamic knowledge graph update method based on cross-attention, see [link to relevant documentation]. Figure 3 The cross-attention update layer 5 in the dynamic knowledge graph update model specifically includes the following: (1) Input matrix generation unit 51 is used to obtain entity features to be updated from the historical knowledge graph, and to concatenate the entity features to be updated with the entity propagation information to form an input matrix.

[0101] Specifically, it involves analyzing the updated entity characteristics (i.e., the affected entity characteristics) of the receiving entity. Information M is propagated from the entity to obtain the input matrix. ( (Number of messages).

[0102] (2) Self-attention calculation unit 52 is used to perform self-attention calculation on the input matrix to obtain the self-attention calculation result.

[0103] Specifically, generating query vectors , key vector Sum value vector ( (As a learnable matrix), the self-attention calculation results are obtained. .

[0104] (3) Cross-attention calculation unit 53 is used to extract the first part of data associated with the entity feature to be updated from the self-attention calculation result, generate a secondary key vector based on the first part of data, and use the remaining part of the self-attention calculation result except the first part of data as a query to perform cross-attention calculation with the secondary key vector to obtain the cross-attention calculation result.

[0105] Specifically, a first portion of data associated with the entity features to be updated is extracted from the self-attention calculation results. Generate a quadratic key vector ( (As independent matrices), the influence is propagated through cross-attention fusion. The data excluding the first part in the self-attention calculation result. The rest of the data.

[0106] (4) Weighted fusion unit 54 is used to weightedly fuse the cross attention calculation result with the static periodic information, and generate the updated entity feature after layer normalization.

[0107] Specifically, the updated entity features are obtained by fusing the cross-attention calculation results with the static periodic information. .in, Entity-specific weights (initially 1, adaptive learning). Representation layer normalization.

[0108] As can be seen from the above description, the dynamic knowledge graph update method based on cross-attention provided in this application, through the cross-attention mechanism, allows the cross-attention mechanism to represent the influence of the spread of new knowledge. With the original state of the entity Through interaction, the model can adaptively learn how to translate or map new information into the entity's existing feature space. This ensures that the incorporation of new knowledge does not distort or cover up important historical semantics, thus fundamentally avoiding catastrophic forgetting; and the final update is a weighted fusion of the results after cross-attention filtering and calibration with periodic information, rather than a simple feature replacement or superposition. This makes the evolution of features smooth, controllable, and semantically consistent.

[0109] It should be noted that the dynamic knowledge graph update based on cross-attention can be performed on the client device. The specific implementation can be chosen based on the processing power of the client device and the limitations of the user's usage scenario. This application does not impose any limitations on this. If all operations are performed on the client device, the client device may further include a processor for the specific processing of the dynamic knowledge graph update based on cross-attention.

[0110] The aforementioned client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side; in other implementation scenarios, it may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, a server cluster consisting of multiple servers, or a distributed server structure.

[0111] The server and the client device can communicate using any suitable network protocol, including those not yet developed as of the date of this application. Such network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Furthermore, such network protocols may also include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer Protocol) protocols used on top of the aforementioned protocols.

[0112] Based on the above embodiments of the dynamic knowledge graph update method based on cross-attention, this application also provides an embodiment of a dynamic knowledge graph update model training method, see [link to embodiment]. Figure 4The dynamic knowledge graph update model training method specifically includes the following: Step 010: Obtain the training dataset, which contains quadruple data of multiple time slices of the continuous temporal knowledge graph stream and their corresponding historical knowledge graphs.

[0113] Step 020: Input the quadruple data of each time slice in the training dataset and the corresponding historical knowledge graph into the dynamic knowledge graph update model in sequence, so that the dynamic knowledge graph update model generates updated entity features for each time slice.

[0114] Step 030: Calculate the contrastive learning loss based on the updated entity features and preset labels corresponding to each time slice.

[0115] Step 040: Based on the contrastive learning loss, optimize the parameters of the dynamic knowledge graph update model end-to-end using the backpropagation algorithm to obtain the trained dynamic knowledge graph update model for executing the cross-attention-based dynamic knowledge graph update method.

[0116] Specifically, a contrastive learning loss function is used to distinguish between true quadruplets and negatively sampled quadruplets, as shown in the following formula: in L This is the total loss value. Representing the main entity, Indicates relationship Represents the object entity; t represents the timestamp; Represents the main entity The embedding vector at time t (i.e., the updated entity features); Representing the object entity In time t The embedding vector; Indicates negative sampling entity In time t The embedding vector. This is the marginal parameter (default 1.0). For bilinear scoring function ( ), These are negatively sampled entities (randomly selected from the entity set), used to construct negatively sampled quadruples. , express That is, if the value inside the parentheses is greater than 0, the value is taken; otherwise, 0 is taken. Training process: Mini-batch gradient descent is used. Each iteration takes a quadruple of a time slice as input and updates the RGCN, attention, and FFT related parameters. The number of iterations is adjusted according to the amount of data (default 100-500 rounds).

[0117] The dynamic knowledge graph update method based on cross-attention provided in this application can be specifically referred to in the processing flow of the above-described embodiment of the dynamic knowledge graph update method based on cross-attention. Its functions will not be repeated here, but can be referred to in the detailed description of the above-described embodiment of the dynamic knowledge graph update method based on cross-attention.

[0118] As can be seen from the above description, the dynamic knowledge graph update method based on cross-attention provided in this application constructs a complete dynamic knowledge graph learning ecosystem. Through an end-to-end comparative learning framework, it can ensure the collaborative evolution of each component of the model under a unified goal, thereby achieving significant technical breakthroughs in efficiency, performance, and robustness, and providing a solid technical foundation for the actual deployment and application of dynamic knowledge graphs.

[0119] To further illustrate the above embodiments, this application also provides a specific application example of a dynamic knowledge graph update method based on cross-attention. This application example aims to solve the problem of dynamic updating of existing temporal knowledge graphs in continuous streaming scenarios. The core objectives include generating high-quality initial features for new entities to alleviate the cold start problem, achieved by analyzing the relationships between new and existing entities and combining relationship graph convolution with hybrid feature enhancement; capturing long-term cyclical evolution patterns in the data, such as equipment maintenance cycles or seasonal fluctuations in the supply chain, and dynamically updating the feature representations of long-term inactive entities; designing a lightweight knowledge propagation mechanism to avoid full graph retraining, thereby supporting real-time and efficient processing of continuous streaming data; and simultaneously ensuring feature space alignment between new and historical entities through cross-attention mechanisms and regularization constraints to avoid catastrophic forgetting. Formally, given a continuous temporal knowledge graph stream... (in For the main entity, For relationships, For object entities, (for timestamps), and historical knowledge graphs ( (Entities and Relationships), design functions This makes the new entity Features Generated based on existing entity relationships, long-term inactive entities Features It updates dynamically according to the periodic pattern and ensures that the update latency changes linearly with the increase of data volume, thereby meeting the needs of real-time scenarios.

[0120] The application examples in this application have several technical features, including a multi-module collaborative design that achieves full-scene coverage of new entities, active entities, and long-term inactive entities through end-to-end linkage of three modules: initialization, new knowledge dissemination, and periodic updates; a cross-attention enhancement mechanism to achieve semantic alignment between new entities and historical entities, avoiding feature space misalignment; explicit modeling of periodic patterns, which extracts periodic features from training data through Fast Fourier Transform (FFT) and two-dimensional convolution, thereby solving the problem of outdated features of long-term inactive entities; and the adoption of a lightweight update strategy that only calculates local subgraphs and periodically related entities, without the need to retrain the entire graph, thus improving update efficiency by several times compared to existing technologies.

[0121] The overall algorithm flow of the Cross-Attention for Dynamic Knowledge Graph Update with Periodic Pattern Learning (CA-DKG) method provided in the application examples of this application is shown in Table 1, Algorithm 1.

[0122] Table 1

[0123] In one experiment, see Figure 5 The method provided in this application example has been validated on the general datasets ICEWS-14 (Integrated Early Warning System Dataset for 2014 events), ICEWS-15 (Integrated Early Warning System Dataset for 2015 events), ICEWS-18 (Integrated Early Warning System Dataset for 2018 events), and YAGO (Open Knowledge Graph (a large multilingual knowledge base)). Its performance on multiple datasets is almost always superior to other methods. The slightly weaker performance on ICEWS-14 may be due to the smaller dataset size of ICEWS-14, while the method in this application, by incorporating three cross-attention layers and RGCN layers, results in higher model parameters and underfitting due to insufficient data.

[0124] exist Figure 5In this context, MRR stands for Reciprocal Ranking, which is the average of the reciprocals of the rankings of all correct answers. hit@1 represents the hit rate@1, which is the proportion of correct answers ranked 1st. hit@10 represents the hit rate@10, which is the proportion of correct answers ranked within the top 10. TransE represents a translation embedding model; TTransE represents a temporal translation embedding model; DistMulti represents a bilinear diagonal model; CompGCN represents a composite graph convolutional network; TA-DistMult represents a temporally aware bilinear diagonal model; TNTComplEx represents a temporal neural tensor decomposition model; CyGNet represents a recurrent graph network; R-GCN represents a relational graph convolutional network; xERTE represents explicitly modeling temporal relational transformations. StreamE represents a streaming embedding model.

[0125] In summary, the application examples of this application have the following beneficial effects: 1) A new entity initialization method based on RGCN and hybrid feature enhancement: By aggregating historical association information through local subgraphs and combining random perturbations to balance the uniqueness of new entities, the cold start problem is effectively solved.

[0126] 2) Lightweight knowledge propagation mechanism that integrates cross-attention: It only calculates on local subgraphs and aligns the new / old entity feature spaces through attention to achieve real-time updates and semantic consistency.

[0127] 3) Periodic pattern extraction module based on FFT and 2D convolution: Explicitly models the periodicity of data, dynamically updates inactive entity features, and avoids feature obsolescence.

[0128] 4) End-to-end dynamic knowledge graph update framework: integrates four major functions: data processing, initialization, propagation, and periodic update. It does not depend on external modules and can be directly deployed in continuous flow scenarios.

[0129] This application also provides an electronic device, which may include a processor, a memory, a receiver, and a transmitter. The processor is used to execute the dynamic knowledge graph update method and / or dynamic knowledge graph update model training method mentioned in the above embodiments, wherein the processor and the memory can be connected via a bus or other means, taking a bus connection as an example. The receiver can be connected to the processor and the memory via wired or wireless means.

[0130] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0131] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the dynamic knowledge graph update method and / or dynamic knowledge graph update model training method based on cross-attention in the embodiments of this application. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the dynamic knowledge graph update method and / or dynamic knowledge graph update model training method based on cross-attention in the above method embodiments.

[0132] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0133] The one or more modules are stored in the memory, and when executed by the processor, they execute the dynamic knowledge graph update method and / or dynamic knowledge graph update model training method based on cross-attention in the implementation embodiment.

[0134] In some embodiments of this application, the user equipment may include a processor, a memory, and a transceiver unit. The transceiver unit may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected via a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0135] As one implementation method, the functions of the receiver and transmitter in this application can be implemented by transceiver circuits or dedicated transceiver chips, and the processor can be implemented by dedicated processing chips, processing circuits or general-purpose chips.

[0136] As another implementation approach, the server provided in this application embodiment can be implemented using a general-purpose computer. That is, the program code implementing the processor, receiver, and transmitter functions is stored in memory, and the general-purpose processor implements the processor, receiver, and transmitter functions by executing the code in memory.

[0137] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of the aforementioned dynamic knowledge graph update method and / or dynamic knowledge graph update model training method based on cross-attention. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0138] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned dynamic knowledge graph update method and / or dynamic knowledge graph update model training method based on cross-attention.

[0139] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave.

[0140] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0141] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0142] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to the embodiments of this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A dynamic knowledge graph update method based on cross-attention, characterized in that, include: The quadruple data of the current time slice corresponding to the continuous temporal knowledge graph stream and the historical knowledge graph corresponding to the current time slice are input into the dynamic knowledge graph update model, so that the dynamic knowledge graph update model performs: influence modeling on the quadruple data to generate structured temporal data containing new entity information; based on the structured temporal data, the initial features of the new entity are generated by fusing the local association information and random initialization information of the new entity. Entity propagation information is constructed based on the initial characteristics of the new entity and the characteristics of its associated historical entities; static periodic information representing the periodic evolution pattern is extracted based on the historically accumulated time-series signals generated by the dynamic knowledge graph update model in previous time slices. The entity features to be updated are obtained from the historical knowledge graph. Through the cross-attention mechanism, the entity features to be updated are interacted with the entity propagation information. The interaction result is then fused with the static periodic information to generate the updated entity features. The dynamic knowledge graph is updated based on the updated entity features, and the updated dynamic knowledge graph is output as the historical knowledge graph corresponding to the next time slice.

2. The dynamic knowledge graph update method based on cross-attention according to claim 1, characterized in that, The dynamic knowledge graph update model includes: The data processing module is configured to perform impact modeling on the quadruple data of the current time slice and generate structured time-series data containing information about new entities. The new entity initialization module is configured to generate the initial features of the new entity based on the structured time-series data by fusing the local association information and random initialization information of the new entity. The propagation information preprocessing module is configured to construct entity propagation information based on the initial characteristics of the new entity and the characteristics of its associated historical entities; The historical information update module is configured to extract static periodic information representing the periodic evolution pattern based on the time-series signals generated by the dynamic knowledge graph update model in previous time slices, which are accumulated historically. A cross-attention update layer is configured to obtain entity features to be updated from the historical knowledge graph, and through a cross-attention mechanism, interact the entity features to be updated with the entity propagation information, and fuse the interaction result with the static periodic information to generate updated entity features.

3. The dynamic knowledge graph update method based on cross-attention according to claim 2, characterized in that, The data processing module is specifically configured to: Real-time impact modeling is performed on the quadruple data of the current time slice to extract the direct association quadruple features of new entities; Furthermore, the continuous impact modeling of the quadruple data in the current time slice includes: based on the historical knowledge graph, extracting the indirect association information of the new entity to downstream entities through its associated entities within the historical window using the relational graph convolutional network RGCN, and aggregating the indirect association information to obtain indirect association features; The structured time-series data is generated by fusing the directly associated quadruple features with the indirectly associated features.

4. The dynamic knowledge graph update method based on cross-attention according to claim 2, characterized in that, The new entity initialization module is specifically configured to: Based on the new entity and the object entities associated with it in the current time slice, extract all historical entities connected to the object entity through historical relationships from the historical knowledge graph. Construct a local subgraph with the new entity, the object entity, and the historical entities as nodes and the relationships between the new entity, the object entity, and the historical entities as nodes as edges. The local subgraph is aggregated with multi-hop relation features based on the relation graph convolutional network RGCN to generate entity features based on historical context. The entity features based on historical context are mixed with a random initialization vector as random initialization information to enhance the representation and generate the initial features of the new entity.

5. The dynamic knowledge graph update method based on cross-attention according to claim 2, characterized in that, The propagation information preprocessing module is specifically configured to: Based on the initial characteristics of the new entity, the characteristics of related historical entities obtained from the historical knowledge graph, and their relationship characteristics, direct propagation messages and indirect propagation messages are constructed; wherein, the direct propagation message is used to characterize the influence of the new entity on its directly related entities; the indirect propagation message is used to characterize the indirect influence of the new entity on its downstream entities through multi-hop relationships; The directly propagated messages and the indirectly propagated messages are integrated to generate entity propagation information.

6. The dynamic knowledge graph update method based on cross-attention according to claim 2, characterized in that, The historical information update module is specifically configured for: Centered on the relationships associated with inactive entities, a dynamic local subgraph is sampled, and a time-series signal unit for the current time slice is generated based on the dynamic local subgraph. The timing signal unit of the current time step is stored in the periodic pattern cache, and together with the historical signal, it constitutes the historical timing signal sequence as the timing signal. The historical time-series signal sequence is read from the periodic pattern buffer, the historical time-series signal sequence is transformed in the frequency domain, a target periodic pattern is selected based on the energy intensity of the frequency components, and the signals in the historical time-series signal sequence are rearranged according to the length corresponding to the target periodic pattern to obtain the rearranged two-dimensional signal. A convolution operation is performed on the rearranged two-dimensional signal structure to output static periodic information representing the periodic evolution pattern.

7. The dynamic knowledge graph update method based on cross-attention according to claim 2, characterized in that, The cross-attention update layer is specifically configured for: The entity features to be updated are obtained from the historical knowledge graph, and the entity features to be updated are concatenated with the entity propagation information to form an input matrix. Perform self-attention calculation on the input matrix to obtain the self-attention calculation result; Extract the first part of the data associated with the entity feature to be updated from the self-attention calculation result, generate a secondary key vector based on the first part of the data, and use the remaining data in the self-attention calculation result excluding the first part of the data as a query to perform cross-attention calculation with the secondary key vector to obtain the cross-attention calculation result. The cross-attention calculation result is weighted and fused with the static periodic information, and then subjected to layer normalization to generate the updated entity features.

8. A method for training a dynamic knowledge graph update model, characterized in that, include: Obtain a training dataset, which contains quadruple data of multiple time slices of a continuous temporal knowledge graph stream and their corresponding historical knowledge graphs. The quadruple data of each time slice in the training dataset and the corresponding historical knowledge graph are sequentially input into the dynamic knowledge graph update model so that the dynamic knowledge graph update model generates updated entity features for each time slice. Based on the updated entity features and preset labels corresponding to each time slice, the contrastive learning loss is calculated; Based on the contrastive learning loss, the parameters of the dynamic knowledge graph update model are optimized end-to-end using the backpropagation algorithm to obtain a trained dynamic knowledge graph update model for executing the dynamic knowledge graph update method based on cross-attention as described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the dynamic knowledge graph update method based on cross-attention as described in any one of claims 1 to 7, and / or implements the dynamic knowledge graph update model training method as described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the dynamic knowledge graph update method based on cross-attention as described in any one of claims 1 to 7, and / or implements the dynamic knowledge graph update model training method as described in claim 8.

Images