Knowledge base generation method and system based on AI agent

Abstract

The application relates to the field of knowledge base generation, in particular to a knowledge base generation method and system based on an AI agent, which comprises the following steps: acquiring metadata of multiple data sources, converting the metadata of the multiple data sources into semantic vectors by using a semantic mapping network; setting a time window, performing time sequence division on the semantic vectors of the multiple data sources, acquiring a dense coefficient of each time sequence window, and obtaining a drift factor of each time sequence window according to the continuous change of the dense coefficient of each time sequence window; calculating an accumulated curve of the drift factor of each time sequence window on the time sequence by using the change trend of the drift factor of each time sequence window, combining the correlation of the accumulated curve of each time sequence window, and constructing a value coefficient of each time sequence window; aligning the multiple data sources according to the value coefficient of each time sequence window, and completing the construction of the knowledge base according to the alignment result of the multiple data sources. The application improves the data alignment accuracy of the knowledge base in space and time.

Description

Technical Field

[0001] This invention relates to the field of knowledge base generation, and more particularly to a method and system for generating knowledge bases based on AI agents. Background Technology

[0002] Against the backdrop of rapid development in artificial intelligence, AI agents, as autonomous systems possessing perception, reasoning, learning, and decision-making capabilities, rely heavily on the quality and real-time performance of their underlying knowledge base. Traditional knowledge bases are mostly based on manual construction or static data extraction, resulting in long knowledge update cycles and high maintenance costs, making it difficult to meet the timeliness and contextual adaptability requirements of AI agents in dynamic and complex environments. Especially when facing diverse data sources such as sensors, log systems, databases, and documents, the fragmented nature of knowledge acquisition due to inconsistent data formats, varying update rhythms, and diverse semantic expressions severely restricts the knowledge fusion and continuous learning capabilities of AI agents.

[0003] While existing technologies have attempted to vectorize textual information using natural language processing models (such as BERT) to enhance semantic understanding and organize knowledge through graph modeling, these methods generally lack in-depth modeling of the dynamic evolution characteristics of data sources. Most solutions focus only on semantic matching at a single moment, neglecting the stability changes and trend drift of data over time. This makes it difficult to identify data sources with semantic divergence or topical shifts during knowledge fusion, leading to knowledge conflicts or association errors. Furthermore, traditional alignment methods rely on precise time synchronization or static similarity calculations, which cannot effectively handle the asynchronous time, rhythmic differences, and non-linear evolutionary relationships between multi-source data, affecting the structural consistency and inference reliability of the knowledge base.

[0004] More importantly, current knowledge base generation mechanisms lack a "value-oriented" alignment strategy tailored to the operational needs of AI agents. This means they fail to assess the knowledge contribution and relevance between different data sources from the perspective of the synergy of data change trends. As a result, while the generated knowledge base has a static structure, it struggles to reflect the dynamic interaction patterns of multi-source information in the real world, limiting the adaptive decision-making capabilities of AI agents in complex scenarios.

[0005] Therefore, there is an urgent need for a new knowledge base generation method that can integrate semantic understanding, temporal evolution analysis and trend-driven alignment, in order to achieve the construction of a high-quality, evolvable and strongly correlated knowledge system for AI agents. Summary of the Invention

[0006] To address the technical problems of existing knowledge base generation methods in AI agent applications, such as the difficulty in effectively integrating multi-source heterogeneous and dynamically evolving data, leading to inaccurate knowledge alignment, weak semantic association, and poor structural adaptability, this invention provides solutions in the following aspects.

[0007] In the first aspect, the knowledge base generation method based on AI intelligent agents includes: acquiring metadata from multiple data sources and using a semantic mapping network to convert the metadata from multiple data sources into semantic vectors;

[0008] Set a time window, perform temporal division on the semantic vectors of multiple data sources, obtain the density coefficient of each time window, and obtain the drift factor of each time window based on the continuous change of the internal density coefficient of each time window.

[0009] By utilizing the changing trend of the drift factor in each time window, the cumulative curve of the drift factor in time for each time window is calculated. Combining the correlation of the cumulative curves of each time window, the value coefficient of each time window is constructed.

[0010] Based on the value coefficient of each time series window, multiple data sources are aligned, and the knowledge base is constructed based on the alignment results of the multiple data sources.

[0011] Preferably, the setting of the time window includes: obtaining the longest value of the single data source sampling time interval among all data sources. The fixed time window length is set at times, and this fixed time window length can be adjusted by the implementer according to the specific implementation scenario.

[0012] Preferably, the density coefficient is calculated by: obtaining the semantic vectors corresponding to the metadata generated by a single data source from the time before the time window length to any time, forming a semantic vector sequence; calculating the variance between the semantic vectors in the semantic vector sequence, and using the reciprocal of the variance as the density coefficient of the single data source for the corresponding time window at any time.

[0013] Preferably, the semantic vector is calculated by: acquiring metadata collected from a single data source, inputting the metadata collected from the single data source into a pre-trained semantic mapping network, and obtaining the output value of the semantic mapping network, wherein the output value of the semantic mapping network is the semantic vector input to the corresponding semantic mapping network.

[0014] Preferably, the drift factor is calculated by: calculating the difference between the density coefficients of a single data source at two consecutive times, using the difference as the direction of change of the density coefficient, and multiplying the direction of change by the density coefficient to obtain the drift factor of the single data source.

[0015] Preferably, the calculation method of the value coefficient includes: obtaining the drift factor corresponding to all times within a fixed time window corresponding to a single data source at any given time; calculating the cumulative value of the drift factor corresponding to all times in consecutive times to obtain a cumulative curve composed of the cumulative drift factor values ​​corresponding to each time within the window corresponding to the given time; obtaining the cumulative curve corresponding to any other data source at any given time, and normalizing the cumulative curves by dividing the cumulative values ​​of the two cumulative curves by the maximum cumulative value in the cumulative curves; calculating the Pearson correlation coefficient between the two normalized cumulative curves, and taking the absolute value of the Pearson correlation coefficient as the value coefficient corresponding to the single data source and any other data source at any given time.

[0016] Preferably, the construction and generation of the knowledge base includes: obtaining the value coefficients of any two data sources at the same time; obtaining all pending data sources currently used to build the knowledge base; obtaining the value coefficients of any two data sources among the pending data sources at all time points; obtaining the value coefficient sequences corresponding to any two data sources respectively; using the DTW algorithm, obtaining the alignment relationship between the two data sources at different times; constructing the graph structure data corresponding to the two data sources at different times based on the alignment relationship; and thus obtaining the graph structure data composed of the pending data sources, representing the spatiotemporal relationship between the data source and other data sources at any time, thereby completing the generation of the knowledge base.

[0017] Preferably, the graph structure data includes: in the graph structure data, each moment data in two data sources is a node; for two moments corresponding to two nodes in the data sources with an alignment relationship, the weight of the connecting edge is 1; otherwise, the weight of the connecting edge is 0; the semantic vector cosine similarity value between the data sources corresponding to moments with an alignment relationship is obtained; the obtained cosine similarity value is multiplied by the edge weight between the corresponding two nodes to obtain the updated edge weight value, which is used as the new edge weight between the corresponding two nodes, thereby completing the construction of the graph structure data.

[0018] Secondly, a knowledge base generation system based on AI intelligent agents includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the aforementioned knowledge base generation method based on AI intelligent agents is implemented.

[0019] The present invention has the following effects:

[0020] 1. Introducing a density coefficient as a quantitative indicator of the concentration of semantic vector distribution, and using the reciprocal of variance to reflect the consistency level of data semantics within a certain period of time, can effectively identify abnormal data sources with drastic semantic fluctuations or obvious topic drift.

[0021] 2. By calculating the direction of change of the density coefficient at continuous time intervals and constructing a drift factor in combination with its amplitude, we can not only reflect the stability of the semantic distribution of data, but also capture its evolution trend (convergence or divergence). This can accurately characterize the behavioral evolution of the data source in the time dimension and provide key decision signals for the dynamic updating of the knowledge base.

[0022] 3. By introducing value coefficients and using the alignment results of value coefficient sequences to construct graph structure data, edge weight optimization is achieved in both time and space dimensions, from "formal alignment" to "semantic credibility". This effectively reduces the probability of misconnection between data that are similar in trend but semantically unrelated, improves the quality of connection relationships in the graph structure, and makes the final generated knowledge base semantically accurate, significantly enhancing the reliability and usability of knowledge representation. Attached Figure Description

[0023] Figure 1 This is a flowchart of steps S1-S4 in the knowledge base generation method based on AI intelligent agents according to an embodiment of the present invention.

[0024] Figure 2 This is a structural block diagram of the knowledge base generation system based on AI intelligent agents according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0026] Reference Figure 1 The knowledge base generation method based on AI intelligent agents includes steps S1-S4, as detailed below:

[0027] S1: Obtain metadata from multiple data sources and use a semantic mapping network to transform the metadata from multiple data sources into semantic vectors.

[0028] The information collection system acquires metadata from multiple data sources, using it as the data source for generating the AI ​​agent's knowledge base. This metadata is structured data description information, including at least a detailed description of the data, its creation time, and the data source. The data source in the first The metadata corresponding to the time is , An array contains a detailed description of the data, as well as information such as creation time and data source.

[0029] The collected metadata is used as the input to the semantic mapping network, and the output is the mapping vector of the metadata input to the semantic mapping network. The semantic mapping network can be a BERT (Bidirectional Encoder Representations from Transformers) pre-trained network. BERT pre-trained network is a well-known technology and will not be described in detail in this solution.

[0030] Will After inputting values ​​into the semantic mapping network, the corresponding semantic vector output value of the semantic mapping network is obtained. , for The semantic vector.

[0031] S2: Set a time window, perform temporal division on the semantic vectors of multiple data sources, obtain the density coefficient of each time window, and obtain the drift factor of each time window based on the continuous change of the inner density coefficient of each time window.

[0032] When building a knowledge base, it is necessary to obtain the amount of change between different data in the knowledge base to represent the differences and approximations between different data.

[0033] To obtain the amount of change between different data in the knowledge base, but since the data is constantly generated in chronological order, the knowledge base should be dynamically updated.

[0034] When a fixed time window length is set, the longest sampling time interval value from all data sources is obtained. The fixed time window length is set at a multiple of [number], which can be adjusted by the implementer according to the specific implementation scenario. This is a hyperparameter and can be adjusted according to the implementation scenario.

[0035] Get the The data source is the time from before the time window length to the [number]th [time]. The metadata generated at each time step is used to obtain the semantic vectors corresponding to this metadata, thus obtaining the first... Semantic vector sequences corresponding to data sources within a fixed time window .

[0036] exist In the middle, if If the semantic vectors in the vectors are relatively scattered, then it represents the first... The data source itself is an unstable data type with significant information changes, which is not conducive to alignment with other data sources and leads to the problem of data drift during knowledge base generation. Conversely, it is conducive to alignment with other data sources.

[0037] To measure The dispersion of semantic vectors in the calculation The variance between semantic vectors is used as the inverse of the variance. Corresponding to the Dense coefficient of time window corresponding to time moment .

[0038] Calculate the first The difference in the density coefficients of a data source at two consecutive time points is used as the direction of change of the density coefficients. The data source in the first The direction of change of time is In the formula, For step function, when hour, ;when , ; , .

[0039] The first The data source in the first Multiplying the direction of change at time by the density coefficient yields the corresponding drift factor. , As a for A vector of magnitude.

[0040] S3: Utilize the changing trend of the drift factor in each time window to calculate the cumulative curve of the drift factor in time for each time window. Combine the correlation of the cumulative curves of each time window to construct the value coefficient of each time window.

[0041] After obtaining the number The data source in the first Drift factor at time After that, it can represent the first The data source in the first The goal is to determine whether the data at any given moment is trending towards stability and convergence, or beginning to change and diverge. This is to obtain information on stable data source changes and to measure the trend of data changes in the data source.

[0042] Get in the first The drift factor corresponding to each time point corresponds to a fixed time window. The cumulative value of the drift factor corresponding to all times is calculated to obtain the first drift factor. The cumulative curve is composed of the cumulative drift factor values ​​corresponding to each time point within the time window. .

[0043] Get the The data source in the first The cumulative curves corresponding to the nth time point are normalized by dividing the cumulative values ​​of the two cumulative curves by the maximum cumulative value of each curve. The Pearson correlation coefficient between the two normalized cumulative curves is then calculated, and the absolute value of the Pearson correlation coefficient is taken as the nth time point. The data and the first The data source in the first Value coefficient at each time point .

[0044] S4: Based on the value coefficient of each time series window, perform multi-data source alignment, and based on the alignment results of the multi-data source, complete the construction and generation of the knowledge base.

[0045] Step S3 can obtain the value coefficients of any two data sources at the same time, obtain all the data sources to be used in the current knowledge base construction, and obtain the value coefficients of any two data sources at all time points.

[0046] Given the value coefficient sequences corresponding to any two data sources, use the DTW (Dynamic Time Warping) algorithm to calculate the cumulative distance matrix of the value coefficient sequences corresponding to the two data sources. Perform path backtracking on the cumulative distance matrix to obtain the matching point of the value coefficient sequences corresponding to the two data sources, which is the alignment point.

[0047] Obtain the alignment relationship between two data sources at different times. Based on the alignment relationship, construct a graph structure of the data between the two data sources at different times. In this graph structure, each time point in the two data sources represents a node. For nodes in two data sources with an alignment relationship, there is a connecting edge with a weight of _____. Otherwise, the weight of the connecting edge is .

[0048] Obtain the semantic vector cosine similarity value between the data sources corresponding to the moments with alignment relationships. Multiply the obtained cosine similarity value with the edge weight between the corresponding two nodes to obtain the updated edge weight value, which is used as the new edge weight between the corresponding two nodes, thereby completing the construction of the graph structure data.

[0049] Logically, the graph structure data composed of the data sources to be used represents the spatiotemporal relationship between the data source and other data sources at any given time, thus completing the knowledge base generation.

[0050] This invention also provides a knowledge base generation system based on AI intelligent agents. For example... Figure 2As shown, the system includes a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement the knowledge base generation method based on AI intelligent agents according to the first aspect of the present invention. The system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, the settings and functions of which are known in the art and will not be described further here.

[0051] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An AI agent-based knowledge base generation method, characterized by, include: Obtain metadata from multiple data sources and use a semantic mapping network to transform the metadata from multiple data sources into semantic vectors; A time window is set, and the semantic vectors of multiple data sources are divided into temporal sequences. The density coefficient of each time window is obtained. The calculation method of the density coefficient includes: obtaining the semantic vectors corresponding to the metadata generated by a single data source from the time before the time window length to any time, forming a semantic vector sequence; calculating the variance between the semantic vectors in the semantic vector sequence, and taking the reciprocal of the variance as the density coefficient of the time window corresponding to the single data source at any time. The drift factor of each time window is obtained based on the continuous change of the density coefficient within each time window. The calculation method of the drift factor includes: calculating the difference of the density coefficient corresponding to a single data source at two consecutive times, which is taken as the direction of change of the corresponding density coefficient, and multiplying the direction of change by the density coefficient to obtain the drift factor of a single data source. By utilizing the changing trend of the drift factor in each time series window, the cumulative curve of the drift factor in time series for each time series window is calculated. Combining the correlation of the cumulative curves of each time series window, a value coefficient for each time series window is constructed. The calculation method for the value coefficient includes: obtaining the drift factor corresponding to all times within a fixed time window for a single data source at any given time; calculating the cumulative value of the drift factor corresponding to all times in consecutive times to obtain a cumulative curve composed of the cumulative values ​​of the drift factor corresponding to each time within the window at any given time; obtaining the cumulative curve corresponding to any other data source at any given time; normalizing the cumulative curves by dividing the cumulative values ​​of the two cumulative curves by the maximum cumulative value in each cumulative curve; calculating the Pearson correlation coefficient between the two normalized cumulative curves; and taking the absolute value of the Pearson correlation coefficient as the value coefficient corresponding to the single data source and any other data source at any given time. Based on the value coefficient of each time series window, multiple data sources are aligned, and the knowledge base is constructed based on the alignment results of the multiple data sources. 2.The AI agent-based knowledge base generation method of claim 1, wherein, The set time window includes: obtaining a maximum value of single data source sampling time interval values in all data sources times as the fixed time window length, wherein the fixed time window length can be adjusted by an implementer according to a specific implementation scenario. 3.The AI agent based knowledge base generation method of claim 1, wherein, The semantic vector is calculated in the following ways: Obtain metadata collected from a single data source, input the metadata collected from the single data source into a pre-trained semantic mapping network, and obtain the output value of the semantic mapping network. The output value of the semantic mapping network is the semantic vector corresponding to the input of the semantic mapping network. 4.The AI agent based knowledge base generation method of claim 1, wherein, The construction and generation of the knowledge base includes: Obtain the value coefficients of any two data sources at the same time. Retrieve all available data sources currently being used to build the knowledge base; Obtain the value coefficients of any two data sources in the pool of available data sources at all points in time; Obtain the value coefficient sequences corresponding to any two data sources, use the DTW algorithm to obtain the alignment relationship between the two data sources at different times, and construct the graph structure data between the two data sources at different times based on the alignment relationship. Logically, the graph structure data composed of the data sources to be used represents the spatiotemporal relationship between the data source and other data sources at any given time, thus completing the knowledge base generation. 5.The AI agent based knowledge base generation method of claim 4, wherein, The graph structure data includes: In graph-structured data, the data at a single moment in two data sources are each a node. For two moments with an alignment relationship, there is a connecting edge between the two nodes of the corresponding data sources with a weight of 1; otherwise, the connecting edge has a weight of 0. Obtain the semantic vector cosine similarity value between the data sources corresponding to the moments with alignment relationships. Multiply the obtained cosine similarity value with the edge weight between the corresponding two nodes to obtain the updated edge weight value, which is used as the new edge weight between the corresponding two nodes, thereby completing the construction of the graph structure data.

6. An AI-agent based knowledge base generation system characterized by, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the knowledge base generation method based on an AI agent according to any one of claims 1-5.

Images