A dynamic knowledge graph construction method, system and device

Abstract

The application discloses a dynamic knowledge graph construction method, system and device, relates to the technical field of knowledge graph expansion, and comprises the following steps: calculating the similarity of external text and historical text; extracting entities and events in the external text, defining the entities and events contained in the predefined entity and event words as keywords, and calculating the keyword coverage; when the similarity is less than a similarity threshold value, or the similarity is greater than or equal to the similarity threshold value and the keyword coverage is greater than or equal to a coverage threshold value, the current external text is retained; reasoning the relationship between entities in the retained external text to obtain new knowledge; when the new knowledge conflicts with existing knowledge in the knowledge graph, embedding the finally retained knowledge into the knowledge graph according to the evaluation standards of intra-chain consistency and cross-chain consistency, and updating the knowledge graph. The application is suitable for adapting to the complex and changeable requirements of enterprise internal management and realizes efficient construction of a dynamic knowledge graph.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph expansion, in particular to a dynamic knowledge graph construction method, system and device. BACKGROUND

[0002] In today's era of digital information explosion, knowledge graph as a powerful knowledge organization and management tool, its importance is increasingly prominent. Especially in the internal management scene of enterprises, knowledge graph can integrate various scattered information, and provide strong support for decision making, business process optimization, etc.

[0003] In the actual scene of enterprise internal management, the construction of dynamic knowledge graph has irreplaceable necessity and high practicality. Taking the enterprise personnel organization structure as an example, employees need to know the changes of organization structure in real time in daily work, so as to clear the information such as reporting relationship and cooperation object. However, due to the traditional knowledge graph updating not timely, the organization structure message mastered by the staff cannot guarantee the immediacy, which may lead to poor communication and work connection failure in cross department cooperation project due to the wrong understanding of the information such as the department to which the relevant personnel belong and the responsibility, and further affect the overall promotion efficiency of the project. In the financial system, if the knowledge graph relied by the financial personnel cannot update the tax policy, preferential information and other contents in time, it will not ensure that the knowledge relied on in the financial processing process conforms to the latest standard, which may lead to the enterprise missing tax preferential treatment, and even face legal risks due to improper tax processing. In the field of intellectual property, enterprises need to keep abreast of the patent dynamics, trademark information change and other information in the industry, update the knowledge graph in time, so as to effectively protect the self intellectual property, avoid infringement disputes, and seize the possible cooperation and development opportunities. In the vertical field, such as the production process knowledge of chemical industry enterprises and the programming knowledge of software enterprises, new knowledge is integrated into the knowledge graph in time, which helps enterprises to improve the core competitiveness, promote product innovation and service optimization.

[0004] The knowledge updating of a knowledge graph usually requires a huge amount of human cost. On the one hand, the internal data sources of an enterprise are extensive and complex, covering personnel organization structure, financial system, intellectual property, and information in vertical fields closely related to the scope of business management. For example, in the personnel organization structure, information such as the change of employees' positions and department adjustment needs to be timely and accurately entered into the knowledge graph. In the financial system, new financial policy interpretation, tax information changes, such as tax category adjustment, preferential information, and other updates due to policy changes, need to be sorted out and integrated by professional personnel. However, manual processing is not only inefficient but also prone to human errors, resulting in low accuracy of knowledge updating. On the other hand, with the continuous expansion of business and dynamic changes in the market environment, new knowledge is constantly emerging, and the speed of human updating of the knowledge graph is far slower than the rate of knowledge generation, which makes the iteration efficiency of the entire knowledge graph extremely slow. For example, in the vertical field of a chemical enterprise, new production processes and new material research and development achievements are constantly emerging. In a software enterprise, new programming techniques and methods are also constantly updated. The traditional knowledge graph updating method is difficult to quickly integrate new knowledge into the knowledge graph, seriously affecting the timeliness and practicality of the knowledge graph in enterprise operation and management.

[0005] In related prior art, the screening of new knowledge expansion and updating in the knowledge graph currently mostly uses a single judgment standard (such as screening only by a text similarity threshold) or relies on fixed threshold static filtering, which has significant limitations:

[0006] (1) Insufficient screening accuracy: a single similarity threshold cannot balance the exclusion of redundant information and the retention of innovative knowledge. For example, if only a high similarity threshold is used for screening, valuable new knowledge may be missed due to slight differences (such as the addition of key details) between new texts and historical knowledge. If the threshold is too low, a large amount of repetitive or low-quality information will be introduced, increasing storage and maintenance costs.

[0007] (2) Poor adaptability: fixed thresholds cannot match the characteristics of diverse data sources (such as the credibility difference between authoritative sources and ordinary sources), system running states (such as the need to reduce data processing volume during high load), and knowledge importance (such as the need for more refined screening of core entity-related knowledge). For example, using a fixed threshold for a low credibility data source may lead to incorrect information entering the graph, and using a uniform standard for high priority knowledge may introduce noise due to loose screening.

[0008] (3) Risk of missing key information: traditional methods lack directional recognition of high-value content in texts, and some new texts may be mistakenly deleted if only similarity screening is used, even though they have a high similarity to historical knowledge, but contain key event words (such as breakthrough technologies in the industry, changes in core entity attributes), resulting in the graph missing important updates.

[0009] In addition, in the process of continuous expansion and update of the knowledge graph, conflicts between new knowledge and existing knowledge are inevitable, and the root causes include differences in data sources (such as different source reliabilities), changes in information timeliness (such as policy adjustments and fact updates), and description perspective deviations (such as differences in multi-party representations of the same event). The existing technologies have significant limitations in handling knowledge conflicts:

[0010] (1) Conflict resolution logic is simplified: Most methods use static rules such as "either this or that", such as retaining the latest knowledge, retaining authoritative source knowledge, or manually judging. This approach ignores the contextual relevance of knowledge, for example, a new piece of knowledge from an ordinary source that is highly self-consistent with a recent event chain may be incorrectly discarded due to low source priority; and an outdated authoritative source knowledge may be retained due to its authoritative nature, resulting in the graph containing invalid information.

[0011] (2) Lack of utilization of knowledge relevance: Existing technologies mostly compare the characteristics of conflicting knowledge (such as timestamps and source weights) in isolation, without considering the event chain structure in which the knowledge is located. For example, two conflicting "product release time" information, if detached from the "product development-test-release" event chain, it is difficult to determine the correctness by isolated comparison.

[0012] (3) Inappropriate handling of cross-event chain conflicts: When conflicting knowledge belongs to different event chains, existing technologies often use equal weight comparison without considering the quality difference of the event chains themselves. For example, when a piece of knowledge from a fragmented and low-trust event chain conflicts with a piece of knowledge from a complete and high-trust event chain, equal comparison may result in the correct knowledge of the high-quality chain being discarded.

[0013] In summary, the existing technologies have low accuracy, poor adaptability, and missing of key information in new information screening, and have problems of incorrect retention or correct discard in new knowledge fusion and conflict handling, which reduces the quality of the knowledge graph. SUMMARY

[0014] To solve the above problems, the present application provides a dynamic knowledge graph construction method, system and device, which adopts a dynamic double-threshold triggering mechanism to construct a hybrid architecture of lightweight change detection triggering, precise focus of large models and knowledge incremental cascade propagation, which can adapt to the complex and variable needs of enterprise internal management and realize efficient construction of dynamic knowledge graph.

[0015] To achieve the above purpose, the present application adopts the following technical solutions:

[0016] In a first aspect, the present application provides a dynamic knowledge graph construction method, comprising:

[0017] calculating the similarity between the obtained external text and the historical text of the corresponding type;

[0018] extracting entities and events in the external text, defining entities and events contained in predefined entity and event words as keywords, and calculating keyword coverage;

[0019] when the similarity is less than the similarity threshold, or the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold, the current external text is retained; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to system load, data source credibility and keyword importance;

[0020] reasoning the relationship between entities for the retained external text to obtain new knowledge;

[0021] when the new knowledge conflicts with existing knowledge in the knowledge graph, it is determined whether the new knowledge and the conflicting knowledge belong to the same event chain, if yes, a consistency score of the new knowledge and other knowledge in the event chain is calculated, and when the consistency score is greater than or equal to a consistency threshold, the current new knowledge is retained; if not, the consistency scores of the new knowledge and the conflicting knowledge and other knowledge in the event chains are respectively calculated, and the consistency scores of the new knowledge and the conflicting knowledge are weighted based on chain quality, and the knowledge with higher score after weighting is retained;

[0022] embedding the finally retained knowledge into the knowledge graph to complete the update of the knowledge graph.

[0023] As an optional implementation, the system load is calculated by weighted sum of CPU usage, memory usage and the ratio of the number of current processing requests to the maximum number of bearable requests;

[0024] The data source credibility is calculated by historical accuracy, update frequency and expert score of the data source;

[0025] The keyword importance is that different weights are given to the predefined entity and event words to determine the weights of each keyword in the external text, and the maximum weight among them is the keyword importance.

[0026] As an optional implementation, the data source credibility is:

[0027] ;

[0028] wherein, is the data source credibility; n is the amount of data to be evaluated in the data source; is the historical accuracy of the i-th data to be evaluated; is the update frequency of the i-th data to be evaluated; is the expert score; , and are weight coefficients of historical accuracy, update frequency and expert score respectively, and .

[0029] As an alternative embodiment, the update frequency is calculated as the ratio of the average interval of the publishing time of all historical data related to the data to be evaluated in the data source to the interval of the publishing time of the first historical data related to the data to be evaluated and the last data.

[0030] As an alternative embodiment, the historical accuracy is calculated as the ratio of the amount of data judged to be accurate to the amount of all historical data related to the data to be evaluated, by calculating the similarity between the data to be evaluated and the official data and considering it accurate if it is greater than a set threshold.

[0031] As an alternative embodiment, the process of dynamically adjusting the similarity threshold and the coverage threshold according to the system load, the data source credibility and the keyword importance is as follows:

[0032] ;

[0033] ;

[0034] wherein, and are the adjusted similarity threshold and the coverage threshold; and are the unadjusted similarity threshold and the coverage threshold; is a load threshold, is a credibility threshold, and are load coefficients, and are credibility coefficients, is an importance coefficient, , and are the system load, the data source credibility and the keyword importance respectively.

[0035] As an alternative embodiment, the process of reasoning further includes: first reasoning by using a predefined rule, and then reasoning by using a large model if reasoning cannot be achieved by using the predefined rule.

[0036] As an alternative embodiment, the events that arrive continuously in the knowledge graph are organized into event chains according to time, causality and co-reference relationships;

[0037] The conflict includes the same cause leading to opposite results and the same results being caused by opposite results.

[0038] As an alternative implementation, the chain quality is calculated according to the data source reliability of each event in the event chain and the length of the event chain.

[0039] In a second aspect, the present application provides a dynamic knowledge graph construction system, comprising:

[0040] A similarity calculation module is configured to calculate the similarity between the obtained external text and the historical text of the corresponding type;

[0041] A coverage calculation module is configured to extract entities and events in the external text, define the entities and events contained in the predefined entity and event words as keywords, and calculate the keyword coverage;

[0042] A screening module is configured to retain the current external text when the similarity is less than the similarity threshold, or the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to the system load, the data source reliability and the keyword importance;

[0043] A reasoning module is configured to reason the relationship between entities in the retained external text to obtain new knowledge;

[0044] A fusion module is configured to, when the new knowledge conflicts with the existing knowledge in the knowledge graph, determine whether the new knowledge and the conflicting knowledge belong to the same event chain, if yes, calculate the consistency score of the new knowledge and other knowledge in the event chain, and retain the current new knowledge when the consistency score is greater than or equal to the consistency threshold; if not, calculate the consistency score of the new knowledge and the conflicting knowledge and other knowledge in the respective event chains, and weight the consistency scores of the new knowledge and the conflicting knowledge based on the chain quality, and retain the knowledge with a higher score after weighting;

[0045] An updating module is configured to embed the finally retained knowledge into the knowledge graph to complete the updating of the knowledge graph.

[0046] In a third aspect, the present application provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the method of the first aspect is completed.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] The application provides a dynamic knowledge graph construction method, system and device, which can adapt to the complex and changeable needs of enterprise internal management, realize efficient construction of dynamic knowledge graph, break the mode of traditional large model full processing flow data, adopt a dynamic double threshold trigger mechanism, construct a hybrid architecture of lightweight change detection trigger, large model precise focus and knowledge incremental cascade propagation, efficiently filter and preliminarily process data by the dynamic double threshold trigger mechanism, quickly capture high-value fragments, greatly reduce the load of large models from the source, reduce the computational complexity, and apply time sequence event chain embedding and incremental fusion to the screened high-value data flow for deep event chain modeling, conflict resolution and fusion, reduce the computational complexity, and are suitable for data flood, uneven change distribution and scenes with high real-time requirements.

[0049] The application is based on time sequence event chain embedding and incremental fusion. Since the traditional method processes the update of each event / relationship in isolation, a large amount of repeated calculation and context loss are caused. Therefore, the application method explicitly models the time sequence dependency relationship of the event chain, greatly improves the information fusion efficiency and consistency through incremental event chain embedding and conflict resolution, improves the knowledge quality and reasoning ability, and is especially suitable for fields with dense events, complex logic chains and strong time sequence dependencies, such as news narrative analysis, business process monitoring, financial transaction monitoring, social media trend tracking and the like, which deepens the understanding while ensuring the efficiency.

[0050] To solve the problems of low accuracy, poor adaptability and missing of key information in screening new information in the prior art, the application provides a dual-dimension screening and dynamic adjustment mechanism of similarity threshold and keyword coverage threshold, which has the following beneficial effects:

[0051] (1) Improve the accuracy and comprehensiveness of new knowledge screening. The similarity threshold and the keyword coverage threshold are used for cooperative judgment, which can filter repeated or low-correlation information through similarity (reduce redundancy) and capture high-value content through keyword coverage (such as important knowledge containing key event words), avoiding the problems of missing valuable knowledge or adding redundant information caused by a single standard. For example, for a text with high similarity to historical knowledge but containing core entity attribute changes, the text can be retained through keyword coverage, ensuring that the graph timely incorporates key updates.

[0052] (2) Dynamically adapt to diversified scene requirements. The similarity threshold and the coverage threshold can be adjusted in real time according to system load, data source credibility and entity importance, improving the flexibility of the screening mechanism. This dynamic adaptation capability makes the knowledge graph update adaptable to different business scenarios, enhancing the practicality.

[0053] (3) Reduce the cost of manual intervention and improve the efficiency of updating. The quantitative judgment logic of double thresholds reduces the need for manual review of boundary data, and ensures consistency through clear screening rules, accelerating the iteration speed of the knowledge graph.

[0054] (4) Optimize the quality of the knowledge graph and enhance its application value. Through precise screening, the knowledge graph can effectively eliminate redundant and low-quality information, while enriching high-value knowledge, improving the knowledge density and accuracy of the graph. This enables applications based on the knowledge graph, such as intelligent retrieval and decision analysis, to rely on more reliable underlying data, reducing decision bias caused by incorrect or redundant knowledge, and enhancing the application value of the knowledge graph in various fields.

[0055] (5) Enhance the robustness of knowledge updating. The introduction of keyword coverage provides a supplementary judgment dimension for texts with similar semantic expressions but containing new information, avoiding the masking of knowledge increments due to similar text styles, and improving the ability of the knowledge graph to capture subtle knowledge changes.

[0056] To solve the problem of error retention or correct rejection in new knowledge fusion and conflict processing, the invention adopts an event chain attribution judgment-consistency score calculation-chain quality weighting conflict processing logic, which has the following beneficial effects:

[0057] (1) Improve the accuracy of conflict judgment. Introducing event chains as a reference framework for conflict processing upgrades the correctness of knowledge judgment from isolated comparison to contextual verification. For conflicts within the same event chain, the compatibility of new knowledge is determined by the consistency score of other knowledge in the chain, avoiding misjudgment due to a single feature. For cross-event chain conflicts, the chain quality is weighted to ensure that high-quality chain knowledge is retained first.

[0058] (2) Dynamically adapt to complex scenarios for conflict processing. Differentiate between same-event chain and cross-event chain conflicts, and adopt differentiated processing strategies: within the same event chain, use consistency thresholds to quickly filter compatible new knowledge, simplifying the conflict processing process for similar events; across event chains, balance the credibility of knowledge from different sources through chain quality weighting, solving the conflict problem between outdated knowledge from authoritative sources and the latest knowledge from non-authoritative sources.

[0059] The advantages of the additional aspects of the invention will be partially given in the following description, partially become apparent from the following description, or be learned through the practice of the invention. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0061] Figure 1 The dynamic knowledge graph construction method flowchart is provided for Embodiment 1 of the present application. DETAILED DESCRIPTION

[0062] The present application will be further described below in combination with the drawings and embodiments.

[0063] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0064] It should be noted that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form, and in addition, it should be understood that the terms "comprise" and "include" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0065] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0066] Embodiment 1

[0067] As Figure 1 shown, the present embodiment provides a dynamic knowledge graph construction method, comprising:

[0068] S101: calculating the similarity between the obtained external text and the historical text of the corresponding type;

[0069] S102: extracting entities and events in the external text, defining the entities and events contained in the pre-defined entity and event words as keywords, and calculating the keyword coverage;

[0070] S103: When the similarity is less than the similarity threshold, or the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold, the current external text is retained; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to system load, data source credibility and keyword importance;

[0071] S104: Inference of the relationship between entities is performed on the retained external text to obtain new knowledge;

[0072] S105: When the new knowledge conflicts with the existing knowledge in the knowledge graph, it is determined whether the new knowledge and the conflicting knowledge belong to the same event chain, if yes, the consistency score of the new knowledge and other knowledge in the event chain is calculated, and when the consistency score is greater than or equal to the consistency threshold, the current new knowledge is retained; if not, the consistency scores of the new knowledge and the conflicting knowledge and other knowledge in the event chains in which they are located are calculated respectively, and the consistency scores of the new knowledge and the conflicting knowledge are weighted based on the chain quality, and the knowledge with a higher score after weighting is retained;

[0073] S106: The finally retained knowledge is embedded into the knowledge graph to complete the update of the knowledge graph.

[0074] In the present embodiment, the external text can be obtained by crawling through a crawler engine, and can be domain dynamics, news or other external text information. For example, in a financial system, new financial policy interpretation, tax information change, tax category adjustment, preferential information and the like.

[0075] Then, after obtaining the external text, a lightweight Sentence-BERT model is used. Sentence-BERT is a pre-training model based on BERT (Bidirectional Encoder Representations from Transformers), which encodes the external text and the corresponding type of historical text into fixed-length embedding vectors. The lightweight Sentence-BERT model is used to generate semantic embedding vectors (dense vector representation of text), and the model output dimension can be set to 384. These embedding vectors can effectively capture the semantic information of the text data, providing a basis for subsequent similarity calculation.

[0076] For example, for the external text "a company releases a new mobile phone" and the corresponding historical text "a company launches an innovative product", they are converted into corresponding embedding vectors through the pre-training model. Each dimension in these embedding vectors contains features related to the semantic information of the text.

[0077] After the obtained external text and the historical text of the corresponding type are encoded into embedding vectors, the similarity between the two embedding vectors is calculated; the similarity ranges from -1 to 1, and the greater the value, the closer the semantics of the two texts, and -1 indicates opposite semantics.

[0078] It can be understood that the cosine similarity, Euclidean distance and other common similarity measurement methods are used to calculate the similarity between the embedding vector of the obtained external text and the embedding vector of the historical text of the corresponding type. Taking the cosine similarity as an example, the calculation formula is:

[0079] ;

[0080] Among them, is the similarity; is the embedding vector of the external text; is the embedding vector of the historical text of the corresponding type.

[0081] In this embodiment, the existing named entity recognition (Named Entity Recognition, NER) tool is used to identify entities (such as names, place names, organizations, etc.) in the external text;

[0082] For keyword extraction of non-named entities, TF-IDF (Term Frequency-Inverse Document Frequency, a commonly used weighting technique for information retrieval and data mining), RAKE (Rapid Automatic keyword extraction, rapid automatic keyword extraction) algorithm or graph-based method (Page Ran) can be used to extract event words.

[0083] For example, for the external text "A company holds a new product launch meeting in B city", the entities "A company" and "B city" can be extracted, and the event is "holding a new product launch meeting".

[0084] A set of high-value entities and events are predefined in the corresponding research field or application scenario, which usually refers to entities and activities, behaviors or state changes in a specific field. For example, in the financial field, high-value entities may include large financial institutions, important listed companies, etc.; high-value events may involve major policy adjustments, mergers and acquisitions, listing, etc.

[0085] Then, it is judged whether the extracted entity and event words are included in the predefined entity and event words; if so, the extracted entity and event words are defined as keywords;

[0086] According to the quotient of the number of keywords of the external text and the total number of predefined entity and event words, the keyword coverage is calculated ; that is .

[0087] In this embodiment, dynamic double thresholds are set, namely a similarity threshold and a coverage threshold.

[0088] Specifically:

[0089] For the similarity threshold : According to the similarity distribution between historical texts, a suitable percentile (such as 90th percentile) is set as the similarity threshold , which means that only the new external text with a similarity lower than this percentile compared with the corresponding type of historical text will be considered "sufficiently different" to trigger further processing.

[0090] Therefore, the specific judgment logic is: if the similarity between the obtained external text and the corresponding type of historical text is greater than or equal to the similarity threshold, it is considered that the new external text is highly similar to the historical text, and the current information is not updated, so it can not be retained or may only need to be simply updated, without starting a comprehensive downstream processing process.

[0091] If the similarity between the obtained external text and the corresponding type of historical text is lower than the similarity threshold, it is considered that the external text has sufficient novelty, and the importance of the key entities or events it contains needs to be further evaluated.

[0092] For the coverage threshold When the obtained external text contains high-value entities or events, even if the overall similarity is high, the downstream processing process should be triggered. This is because even a slight change may carry important information.

[0093] Therefore, the judgment logic according to the dynamic double thresholds is:

[0094] First, evaluate the similarity. If the similarity is less than the similarity threshold, directly retain the current external text, because the new external text data has sufficient novelty.

[0095] If the similarity is greater than or equal to the similarity threshold, evaluate the keyword coverage. If the keyword coverage is greater than or equal to the coverage threshold, still retain the current external text and trigger the next step of processing. Because even if the overall similarity is high, the new external text data may contain important updates or changes.

[0096] Other cases do not need special processing.

[0097] According to the above logical analysis, the following conditions are constructed :

[0098] .

[0099] In this embodiment, although a fixed threshold can be directly set for the setting of the similarity threshold and the coverage threshold based on the experience of domain experts or the needs of specific application scenarios, a fixed threshold cannot guarantee that high-value information is processed first during traffic peaks. Therefore, this embodiment designs dynamic adjustment of the dual thresholds based on system load, data source credibility, and entity importance to ensure that high-value information is processed first during traffic peaks.

[0100] The adjustment rules are as follows:

[0101] (1) System load.

[0102] The system load is calculated by CPU usage, memory usage, and the ratio of the number of currently processed requests to the maximum number of requests that can be handled:

[0103] ;

[0104] Among them, CPU represents CPU usage, memory represents memory usage, and request represents the ratio of the current number of processed requests to the maximum number of requests that can be handled. 、 and are the weight coefficients of the three respectively, and .

[0105] When the system load is higher than the set load threshold, increase and To reduce the amount of downstream processing data, thereby reducing the burden on the system; when the system load is lower than the load threshold, reduce and This allows more data to pass through and be processed further.

[0106] (2) Credibility of data source.

[0107] For example, if the external text obtained is data crawled from a website, the credibility of the data source can be evaluated based on its historical accuracy, update frequency, and expert ratings:

[0108] ;

[0109] Where n is the amount of data to be evaluated in the data source; represents the historical accuracy of the i-th data to be evaluated; Indicates the update frequency of the i-th data to be evaluated; Indicates expert rating, that is, an expert gives a rating on whether the data is important / authoritative; , and respectively represent the weight coefficients of historical accuracy, update frequency and expert score, and .

[0110] wherein the historical accuracy, update frequency and expert score of the data source can be normalized first and then calculated, and the unit of measurement is unified.

[0111] wherein the calculation of the update frequency can be: obtaining the publishing time of all historical data related to the to-be-evaluated data in the data source, calculating the average value according to the interval of each publishing time, taking the interval between the publishing time of the first historical data related to the to-be-evaluated data and the last data as the total duration, and calculating the update frequency according to the ratio of the average value to the total duration.

[0112] wherein based on objective and authoritative reference standards such as official data, industry standards, expert consensus, the information crawled on a certain third-party website is verified for accuracy, the to-be-evaluated data is calculated for similarity with official data (or industry standards, etc.), and if the similarity is greater than a set threshold, it is considered accurate, thereby determining the amount of data judged to be accurate, and the ratio of the amount of data to all historical data related to the to-be-evaluated data is the historical accuracy.

[0113] It can be understood that the above calculation process can be modified by personnel in the art according to specific fields and application scenarios, and is not specifically limited.

[0114] When the data source credibility is greater than a set credibility threshold, it is considered to be a high credibility data source, and and because these sources may contain more valuable information; on the contrary, for low credibility data sources, increase and only when the data is significantly different or contains high-value content.

[0115] (3) Keyword importance imp.

[0116] For predefined entities and event words, different weights are given according to their importance in the knowledge graph, and the number of attribute changes of the same entity in a set time period is compared, and if it exceeds a set number threshold, it is considered to be a high-frequency change entity, and a higher attention weight can be set, because the changes of these entities can indicate important information updates or trend changes, which can be designed according to actual application scenarios, and is not specifically limited.

[0117] wherein each entity generally contains multiple attributes, for example, for a tax type, the attributes are validity period, preferential treatment, reduction method, tax object, etc.

[0118] Then, the weight of each keyword in the obtained external text is determined, and the maximum weight thereof is the keyword importance.

[0119] If the keyword importance is greater than a set importance threshold, it indicates that the new data contains a high importance entity / event, and even if the similarity is high, it should be appropriately reduced to ensure that these data are processed in a timely manner. Conversely, for data that do not contain or only contain low importance entities, the similarity threshold and the coverage threshold can be appropriately increased to reduce unnecessary processing overhead.

[0120] Alternatively, the keyword importance can also be determined according to factors such as a role importance score of the keyword in the business, an impact range score, a user attention score, and the like. ; wherein RIB represents the role importance score of the keyword in the business, impact represents the impact range score of the keyword on other processes or entities, UA represents the user attention score of the keyword, 、 and represent weight coefficients, and .

[0121] It can be understood that the role importance score of the keyword in the business, the impact range score, the user attention score, and the like can all be custom evaluated by those skilled in the art according to the actual application scenario, and are not described in detail.

[0122] In summary, the process of dynamically adjusting the similarity threshold and the coverage threshold according to the system load, the data source credibility, and the keyword importance is as follows:

[0123] ;

[0124] ;

[0125] wherein and are the adjusted similarity threshold and the coverage threshold; represents a load threshold, represents a credibility threshold, and are load coefficients, and are credibility coefficients, is an importance coefficient.

[0126] The method of the embodiment constructs a lightweight change detection triggering system through the calculation of semantic similarity, the identification of key entities and events, and the calculation of keyword coverage. A double-threshold triggering mechanism is adopted to ensure that a large amount of repetitive or unimportant information can be effectively filtered out, while ensuring that truly valuable updates or changes are responded to in a timely manner. Not only can the method quickly respond to changes in data streams, but it is also relatively friendly in terms of resource consumption.

[0127] In the embodiment, a large model is called for reserved external text to perform deep reasoning verification. The purpose of reasoning is to obtain the relationship between entities and perform dynamic evolution of the relationship. It can be understood that a commonly used large model trained can be directly called for reasoning.

[0128] In addition, the reasoning part can also be designed as follows: first, reasoning is performed through a lightweight graph neural network or predefined rules, such as easily thought-of relationship pairing; for example, “A acquires B” automatically infers that “B becomes a subsidiary of A”, and thus “the CEO of B may resign”.

[0129] If reasoning cannot be achieved through a lightweight graph neural network or predefined rules, then a large model is used to perform reasoning between entities.

[0130] The flow can be designed as follows:

[0131] (1) Event triggering: detecting new relationship changes or influence propagation events.

[0132] (2) Preliminary classification:

[0133] Direct relevance assessment: determining whether the event involves a direct and clearly defined relationship (such as “A acquires B”). If so, proceed to step (3); otherwise, proceed directly to step (5).

[0134] (3) Simple processing: using predefined rules or lightweight graph neural networks to quickly process the event; if the processing result meets all the set expected outputs (for example, “B becomes a subsidiary of A”), the flow ends; if there is uncertainty or further verification is needed, proceed to step (4).

[0135] (4) Local verification: verifying the reasonableness of the result produced by simple processing through simple logical checks or comparisons with known cases. If the verification is passed, the flow ends; otherwise, mark it as a complex case and proceed to step (5).

[0136] (5) Complex case analysis:

[0137] Key entity involvement assessment: determining whether the event involves important entities or has the potential to have a wide impact. If it involves key entities or has the potential to have a significant impact, proceed directly to step (6); otherwise, proceed to the next step.

[0138] (6) Resource availability check: Evaluate the current system resource status (e.g., computing power, time constraints, etc.). If resources allow for more in-depth analysis, invoke the large model for deep reasoning; otherwise, record the event for later processing and attempt to make the best guess using existing information.

[0139] (7) Deep reasoning: Invoke the large model to perform detailed causal analysis, multi-level impact assessment, or other required advanced analysis tasks. Update the relationship graph or make corresponding decisions based on the insights provided by the large model.

[0140] (8) Feedback loop: Feedback the new knowledge or pattern learning results obtained from the large model to the lightweight graph neural network or rule base to optimize future simple processing capabilities.

[0141] In summary, if the event involves direct and well-defined relationships, use the lightweight graph neural network / rule-based reasoning engine, and the processing result is reasonable, ending the process; otherwise, if the processing result is uncertain, perform local verification; if it involves important entities or has significant impact potential and resources allow, invoke the large model; otherwise, record the event and attempt the best guess under limited resources; finally, use the newly learned knowledge to improve the simple processing tools. This process provides a basic framework that can be further refined and adjusted according to specific application scenarios and requirements.

[0142] In this embodiment, the events arriving in the knowledge graph are organized into event chains according to time, causality, and co-reference relationships. For example, a company releases financial reports leading to a sharp rise in stock prices, which in turn leads to analysts raising their ratings, forming an event chain; for example, company A releases a new product, indicating that the market demand for the product is increasing, and company A expands its production scale, forming an event chain according to the causal relationship. Lightweight temporal dependency parsers or small temporal relationship classification models can be used, and are not described in detail.

[0143] Temporal-aware chain embedding maintains a dynamic vector representation for each event chain, using a time-decay-based vector average, which is an incremental training lightweight RNN (Recurrent Neural Network) efficient vector operation method:

[0144] ;

[0145] where, is the average vector, represents the vector corresponding to the i-th event, represents the time decay factor, represents the timestamp corresponding to the i-th event, The reference event node is represented, and the earliest event time, the current time or other meaningful time points can be selected.

[0146] When new knowledge conflicts with existing knowledge in the knowledge graph, first determine whether the new knowledge and the conflicting knowledge belong to the same event chain.

[0147] Among them, contradiction is a relationship between events, usually including two cases, one is the same reason leading to opposite results, and the other is the same result caused by opposite results.

[0148] If it belongs to the same event chain, calculate the consistency score (i.e. similarity calculation) of the new knowledge and other knowledge in the event chain; if the consistency score is greater than or equal to the set consistency threshold (such as 80%), the current new knowledge is retained, indicating that even if it conflicts with isolated facts, it is preferred; for example, "A denies merger" in the "merger negotiation chain" may be a negotiation strategy, which is more credible than isolated reports.

[0149] If the consistency score is less than the set consistency threshold, the current new knowledge is not accepted.

[0150] If it does not belong to the same event chain, i.e. there is a cross-chain conflict, the consistency scores of the new knowledge and the conflicting knowledge and other knowledge in their respective event chains are calculated.

[0151] The quality of the event chain where the new knowledge is located and the event chain where the conflicting knowledge is located is evaluated respectively; specifically, the quality of the chain is calculated by weighting according to the data source credibility of each event in the event chain and the length of the event chain; among them, the data source credibility and the length of the event chain are normalized and weighted, and the data source credibility has been described in the dynamic adjustment of the similarity threshold and the coverage threshold, which will not be repeated here.

[0152] Based on the chain quality, the consistency score of the new knowledge and the conflicting knowledge is weighted to generate a comprehensive score, and the knowledge with a higher score after weighting is retained.

[0153] Through the above rules, the intra-chain consistency and cross-chain coordination can be considered in the updating process, and the accuracy and reliability of the entire system can be improved. It can be understood that these rules can be appropriately adjusted and optimized according to different specific application scenarios.

[0154] In this embodiment, the finally retained knowledge is embedded in the knowledge graph, which is used to efficiently update the temporal relationship in the knowledge graph and infer the potential impact. And the chain embedding itself can be used as a high-level feature for downstream tasks, such as predicting the next event of the chain.

[0155] The embodiment proposes to embed and incrementally fuse based on a time sequence event chain. Since the traditional method updates each event / relationship in isolation, it leads to a large amount of repeated calculation and context loss. Therefore, the embodiment method explicitly models the time sequence dependency of the event chain, greatly improves the information fusion efficiency and consistency through incremental event chain embedding and conflict resolution, improves the knowledge quality and reasoning ability, and is particularly suitable for fields with dense events, complex logical chains, strong time sequence dependencies, such as news narrative analysis, business process monitoring, financial transaction monitoring, social media trend tracking, etc. Deepen understanding while ensuring efficiency.

[0156] In addition, the embodiment breaks the traditional pattern of full processing of stream data by a large model, adopts a dynamic double-threshold triggering mechanism, constructs a hybrid architecture of lightweight change detection triggering, large model precise focusing, and knowledge incremental cascading propagation, adopts a dynamic double-threshold triggering mechanism for efficient data filtering and preliminary processing, quickly captures high-value fragments, greatly reduces the load of the large model from the source, reduces the computational complexity, applies time sequence event chain embedding and incremental fusion to the filtered high-value data stream for deep event chain modeling, conflict resolution and fusion, reduces the computational complexity, and is suitable for scenarios with data flood, uneven change distribution, and strict real-time requirements.

[0157] Embodiment 2

[0158] The embodiment provides a dynamic knowledge graph construction system, comprising:

[0159] The similarity calculation module is configured to calculate the similarity between the obtained external text and the historical text of the corresponding type;

[0160] The coverage calculation module is configured to extract entities and events in the external text, define the entities and events contained in the predefined entity and event words as keywords, and calculate the keyword coverage;

[0161] The screening module is configured to retain the current external text when the similarity is less than the similarity threshold, or the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to system load, data source credibility, and keyword importance;

[0162] The reasoning module is configured to reason the relationship between the retained external text and the retained external text, and obtain new knowledge;

[0163] The fusion module is configured to, when new knowledge conflicts with existing knowledge in the knowledge graph, determine whether the new knowledge and the conflicting knowledge belong to the same event chain, if yes, calculate a consistency score of the new knowledge and other knowledge in the event chain where the new knowledge is located, and retain the current new knowledge when the consistency score is greater than or equal to a consistency threshold; if not, calculate a consistency score of the new knowledge and the conflicting knowledge and other knowledge in the event chain where each of the new knowledge and the conflicting knowledge is located respectively, and weight the consistency scores of the new knowledge and the conflicting knowledge based on chain quality, and retain the knowledge with a higher score after weighting;

[0164] The updating module is configured to embed the finally retained knowledge into the knowledge graph, and complete updating of the knowledge graph.

[0165] It should be noted that the system provided in the above embodiments, in realizing its functions, only divides the above-mentioned various functional modules as examples, and in actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above-described functions. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0166] In more embodiments, there are also provided:

[0167] An electronic device includes a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the method described in embodiment 1 is completed. For brevity, it will not be repeated here.

[0168] It should be understood that in the present embodiment, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0169] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0170] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0171] The method in embodiment 1 can be directly embodied as being completed by a hardware processor or being completed by a combination of hardware and software modules in the processor. The software modules can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or the like. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the above method in combination with hardware. To avoid repetition, no longer detailed description is made herein.

[0172] A computer program product comprises a computer program, which, when executed by a processor, implements the method described in embodiment 1.

[0173] The present application also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer executable instructions, for example, instructions embodied in program modules, executed by devices at the destination, real or virtual processors, to perform processes / methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In various embodiments, the functions of the program modules can be combined or divided as desired in various embodiments. Machine executable instructions for program modules can be executed within a local or distributed device. In a distributed device, program modules can be located in local and remote storage media.

[0174] Computer program code for carrying out operations of the present application can be written in one or more programming languages. These computer program code can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the computer or other programmable data processing apparatus, causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be entirely on the computer, partially on the computer, as an independent software package, partially on the computer and partially on a remote computer, or entirely on a remote computer or server.

[0175] In the context of the present application, computer program code or related data can be carried by any suitable carrier to enable a device, apparatus or processor to perform various processes and operations described above. Examples of carriers include signals, computer readable media, and the like. Examples of signals can include electrical, optical, radio, sound or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0176] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0177] It should be noted that the collection and processing of relevant data in this invention, when applied in practice, should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0178] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A method for constructing a dynamic knowledge graph, characterized in that: include: Calculate the similarity between the obtained external text and the historical text of the corresponding type; Extract entities and events from external texts, define entities and events contained in predefined entity and event words as keywords, and calculate keyword coverage; When the similarity is less than the similarity threshold, or the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold, the current external text is retained; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to the system load, the credibility of the data source and the importance of the keyword; Reasoning about the relationships between entities in the retained external text to obtain new knowledge; When new knowledge conflicts with existing knowledge in the knowledge graph, determine whether the new knowledge and the conflicting knowledge belong to the same event chain. If so, calculate the consistency score between the new knowledge and other knowledge in the event chain, and retain the current new knowledge when the consistency score is greater than or equal to the consistency threshold; if not, calculate the consistency score between the new knowledge and the conflicting knowledge and other knowledge in their respective event chains, and weight the consistency scores of the new knowledge and the conflicting knowledge based on the chain quality, and retain the knowledge with the higher score after weighting; The final retained knowledge is embedded into the knowledge graph to complete the update of the knowledge graph.

2. A method for constructing a dynamic knowledge graph according to claim 1, characterized in that: The system load is calculated by weighted summing of CPU usage, memory usage, and the ratio of the number of currently processed requests to the maximum number of requests that can be handled. The data source credibility is calculated based on the historical accuracy, update frequency and expert rating of the data source; The keyword importance is as follows: different weights are given to predefined entities and event words to determine the weight of each keyword in the external text, and the maximum weight among them is taken as the keyword importance.

3. A method for constructing a dynamic knowledge graph according to claim 2, characterized in that: The data source credibility is: ； in, is the credibility of the data source; n is the amount of data to be evaluated in the data source; is the historical accuracy of the i-th data to be evaluated; is the update frequency of the i-th data to be evaluated; Rating the experts; 、 and are the weight coefficients of historical accuracy, update frequency and expert rating respectively, and .

4. A method for constructing a dynamic knowledge graph according to claim 3, characterized in that: The update frequency is calculated as follows: the average of the release time intervals of all historical data related to the data to be evaluated in the data source is calculated, and the interval between the release time of the first and last historical data related to the data to be evaluated is used as the total time interval. The update frequency is calculated as the ratio of the average value to the total time interval. The historical accuracy rate is as follows: the similarity between the data to be evaluated and the official data is calculated. If it is greater than the set threshold, it is considered accurate. The amount of data judged to be accurate is determined, and the ratio of the amount of data judged to be accurate to the amount of all historical data related to the data to be evaluated is the historical accuracy rate.

5. A method for constructing a dynamic knowledge graph according to claim 2, characterized in that: The process of dynamically adjusting the similarity threshold and coverage threshold based on system load, data source credibility, and keyword importance is as follows: ； ； in, and is the adjusted similarity threshold and coverage threshold; and are the similarity threshold and coverage threshold before adjustment; is the load threshold, is the credibility threshold, and are load factors, and are all credibility coefficients, is the importance coefficient, 、 and They are system load, data source credibility and keyword importance.

6. A method for constructing a dynamic knowledge graph according to claim 1, characterized in that: The reasoning process also includes: first using predefined rules for reasoning, and then using the large model for reasoning if reasoning cannot be achieved through the predefined rules.

7. A method for constructing a dynamic knowledge graph according to claim 1, characterized in that: Organize the consecutively arriving events in the knowledge graph into event chains according to time, causality, and coreference relationships; Conflict involves the same causes leading to opposite effects and the same effects being caused by opposite consequences.

8. A method for constructing a dynamic knowledge graph according to claim 1, characterized in that: Chain quality is obtained by weighted calculation based on the credibility of the data source of each event in the event chain and the length of the event chain.

9. A dynamic knowledge graph construction system, characterized in that: include: a similarity calculation module configured to calculate the similarity between the acquired external text and the historical text of the corresponding type; a coverage calculation module configured to extract entities and events from external text, define entities and events contained in predefined entity and event words as keywords, and calculate keyword coverage; The filtering module is configured to retain the current external text when the similarity is less than a similarity threshold, or when the similarity is greater than or equal to the similarity threshold and the keyword coverage is greater than or equal to the coverage threshold; wherein the similarity threshold and the coverage threshold are dynamically adjusted according to system load, data source credibility, and keyword importance; The reasoning module is configured to reason about the relationship between entities on the retained external text to obtain new knowledge; The fusion module is configured to determine whether the new knowledge and the conflicting knowledge belong to the same event chain when there is a conflict between the new knowledge and the existing knowledge in the knowledge graph. If so, the consistency score of the new knowledge and the other knowledge in the event chain is calculated, and the current new knowledge is retained when the consistency score is greater than or equal to the consistency threshold. If not, the consistency scores of the new knowledge and the conflicting knowledge with the other knowledge in their respective event chains are calculated separately, and the consistency scores of the new knowledge and the conflicting knowledge are weighted based on the chain quality, and the knowledge with the higher weighted score is retained; The update module is configured to embed the final retained knowledge into the knowledge graph to complete the update of the knowledge graph.

10. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 8 is completed.

Images