A knowledge database updating method for an AI agent

Abstract

The application relates to the technical field of data processing, in particular to a knowledge database updating method for an AI intelligent agent, which comprises the following steps: acquiring updating data and each keyword of the updating data each time when a user uses the AI intelligent agent; screening each candidate keyword by the historical difference degree of each keyword of the current updating data; acquiring the unique feature value of the keyword of the current updating data after eliminating each candidate keyword; acquiring the keyword uniqueness of the current updating data; extracting each feature keyword; acquiring the unique identifier of the current updating data based on the feature keyword, and storing the unique identifier in the knowledge database of the AI intelligent agent; and searching for data and updating by the unique identifier. The application aims to reduce the calculation time of the unique identifier and improve the uniqueness of the unique identifier, so as to improve the updating efficiency and management efficiency of the knowledge database.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a method for updating a knowledge database for AI agents. Background Technology

[0002] An AI agent is an intelligent system capable of perceiving its environment and achieving specific goals through autonomous decision-making. Its core component is a knowledge database, used to store and manage various information, including facts, rules, and experiences, providing a basis for the AI ​​agent's decision-making. The method of updating the knowledge database is crucial to the performance of the AI ​​agent. Incremental updates are an efficient update strategy that allows the AI ​​agent to gradually add, modify, or delete information based on existing knowledge, rather than rebuilding the entire database with each update. This approach improves update efficiency, reduces computational resource consumption, ensures the continuity and consistency of knowledge, and enables the AI ​​agent to quickly adapt to environmental changes and integrate new knowledge in a timely manner, thereby enhancing the AI ​​agent's adaptability and flexibility.

[0003] In incremental updates, existing technologies typically employ content hashing, retrieving data by calculating the hash value of each record or file. However, due to the large volume of data stored by AI agents and the long length of individual data entries, the complexity of hash calculations is closely related to the data length; the longer the data, the more resources are required for computation. Existing technologies that directly perform hash calculations on the original data suffer from time consumption, leading to wasted computing resources and inefficiency. Summary of the Invention

[0004] In view of the above, it is necessary to provide a knowledge database update method for AI intelligent agents. Compared with traditional knowledge database update methods for AI intelligent agents, this method improves the update efficiency and management efficiency of the knowledge database by reducing the computation time of unique identifiers and improving the uniqueness of unique identifiers.

[0005] The knowledge database updating method for AI intelligent agents proposed in this application adopts the following technical solution:

[0006] One embodiment of this application provides a method for updating a knowledge database for an AI agent, the method comprising the following steps:

[0007] When a user uses the AI ​​agent, acquire each update of data and each keyword in each update of data;

[0008] The historical difference degree of each keyword is obtained by measuring the percentage of times each keyword in the current update data does not appear in the other update data, and by combining the difference in the frequency of each keyword in the current update data with the other update data.

[0009] filter each candidate keyword from all candidate keywords to obtain each candidate keyword set, and obtain a unique feature value of the keyword of the current update data after the each candidate keyword is filtered out by comparing the number of elements in each candidate keyword set that do not appear in the remaining each update data and the frequency of each element in the current update data and the remaining each update data;

[0010] obtain the keyword uniqueness of the current update data by the number of all candidate keywords filtered out and the difference between the frequency of each candidate keyword in the current update data and the remaining each update data;

[0011] extract each feature keyword from the filtered candidate keywords by the difference between the unique feature value and the keyword uniqueness, obtain a unique identifier of the current update data based on the feature keyword, and store the unique identifier in a knowledge database of an AI agent, and find data and update by the unique identifier.

[0012] In one embodiment, the process of obtaining the historical difference degree is:

[0013] calculate the difference between the frequency of each keyword in the current update data and the remaining each update data;

[0014] calculate the weighted sum value of the difference between the frequency of each keyword in the current update data and the remaining each update data, wherein the weights of all the difference values are equal and the sum of the weights is 1;

[0015] The historical difference degree can be further obtained by the number of times ratio and the weighted sum value, wherein the number of times ratio is not 0.

[0016] In one embodiment, the historical difference degree is the product of the number of times ratio and the weighted sum value.

[0017] In one embodiment, the method of filtering each candidate keyword from the keyword of the current update data is:

[0018] According to the size of the historical difference degree of all keywords of the current update data, all keywords are divided into two categories, and each keyword in the category with the largest historical difference degree is used as each candidate keyword.

[0019] In one embodiment, the process of obtaining the unique feature value is:

[0020] calculate the cumulative sum of the number of elements and a preset positive integer;

[0021] The difference between the frequency of each element in each candidate keyword set in the current update data and the rest of each update data is recorded as the frequency difference value. The mean value of all positive values in the frequency difference value of all elements in each candidate keyword set between the current update data and the rest of each update data is calculated.

[0022] The uniqueness feature value can be further obtained by the sum of the rest of all update data and the mean value.

[0023] In one embodiment, the calculation method of the uniqueness feature value is:

[0024] The product of the sum of the rest of each update data and the mean value is calculated.

[0025] The uniqueness feature value is the sum of the product of the rest of all update data.

[0026] In one embodiment, the process of obtaining the keyword uniqueness is:

[0027] The sum of the number and a predetermined positive integer is calculated.

[0028] The difference between the frequency of each candidate keyword between the current update data and the rest of each update data is calculated. The average value of the positive values in the difference calculation result corresponding to all candidate keywords screened is calculated.

[0029] The product of the sum of the rest of each update data and the average value is calculated.

[0030] The keyword uniqueness is the sum of the product calculation result of the rest of all update data.

[0031] In one embodiment, the extraction method of the feature keyword is: the difference between the uniqueness feature value and the keyword uniqueness is recorded as a unique difference value. Each feature keyword is extracted from the screened candidate keywords by the size of the unique difference value.

[0032] In one embodiment, the extraction of each feature keyword from the screened candidate keywords includes: when the unique difference value is less than or equal to 0, the corresponding candidate keyword is removed as a feature keyword.

[0033] In one embodiment, the acquisition method of the unique identifier is:

[0034] The feature keyword and the occurrence frequency of the feature keyword in the current update data are arranged in descending order of occurrence frequency, a feature keyword string is obtained, an update date of the current update data is added at the end of the feature keyword string, a feature unique string is obtained, and a unique identifier is obtained by combining a hash algorithm.

[0035] The present application has at least the following beneficial effects:

[0036] The present application can quantify the difference degree of the keyword in the current update data and the historical data by calculating the historical difference degree, helping to identify the keywords with significant changes; the candidate keywords with representativeness and discrimination are obtained by screening the historical difference degree, which are used to extract the feature keywords to generate the unique identifier, avoiding the problem that the generation of the unique identifier is too complex and difficult to guarantee the uniqueness when all keywords are directly used; further, by eliminating the uniqueness difference of the keywords of the current update data before and after each candidate keyword, the uniqueness of each candidate keyword is evaluated, providing a more accurate basis for generating the unique identifier; the unique identifier of the current update data is generated by the most representative feature keywords extracted, reducing the time-consuming of hash calculation, and at the same time, the unique identifier generated can accurately reflect the core features of the current update data, which can effectively avoid the repetition and confusion of data, ensure the uniqueness of the data in the knowledge database in each update, and quickly locate and update the data when updating the data in the knowledge database, improving the update efficiency and management efficiency of the knowledge database. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0038] Figure 1 A step flow chart of a knowledge database updating method for an AI agent provided by the present application;

[0039] Figure 2 A flowchart for obtaining the uniqueness of the keyword;

[0040] Figure 3 A flowchart for extracting the feature keyword. DETAILED DESCRIPTION

[0041] In the description of the embodiments of the present application, the words "exemplary", "or", "for example" are used to mean serving as an example, instance, or illustration, and not to imply any preference or superiority. In fact, the use of "exemplary", "or", "for example" is intended to present concepts in a particular manner. The embodiments described as "exemplary" or "for example" in the present application should not be construed as being more preferred or advantageous than other embodiments or design schemes.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. It is to be understood that the use of "or" in the present application is meant to encompass both a and b as well as a or b. The use of "including" in the information summary is meant to encompass both a and b as well as a or b.

[0043] In addition, it should be pointed out that the terms "first", "second" in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0044] The specific scheme of the knowledge database updating method for AI agent provided by the present application will be specifically described below in combination with the drawings.

[0045] An embodiment of the present application provides a knowledge database updating method for AI agent, specifically, a knowledge database updating method for AI agent is provided as follows, please refer to Figure 1 The method comprises the following steps:

[0046] Step 1, when the user uses the AI agent, the update data each time and each keyword of the update data each time are obtained.

[0047] When the user uses the AI agent, the AI agent will input files and data. After obtaining the use data of the user each time, the use data of the user obtained is preprocessed by using a regular expression to remove meaningless characters such as ".", ",", "de", etc. Then, the data after preprocessing is subjected to a word segmentation operation by using a jieba.cut() function of a python database. After the word segmentation is completed, the keywords are extracted by using an extract_tags() function. Finally, the update data each time and each keyword of the update data each time are obtained. The regular expression, the jieba.cut() function and the extract_tags() function are all known technologies, and will not be described herein.

[0048] Step 2, the historical difference degree of each keyword is obtained by combining the proportion of the number of times that each keyword of the current update data does not appear in the rest of the update data and the difference in the appearance frequency of each keyword in the current update data and the rest of the update data.

[0049] In the implementation of the incremental update operation on the database, the data or file to be updated is usually accurately located by means of a unique identifier. With the accurate guidance of the unique identifier, the update process only modifies the target data, strictly ensuring that other data contents are not disturbed and the data order remains unchanged. This method effectively avoids redundant scanning and processing of the entire database, significantly reducing computer operation time, improving data update efficiency, and ensuring data integrity and consistency, thereby providing strong support for efficient management and maintenance of the database.

[0050] The unique identifier has uniqueness. Due to the large amount of data contained in the AI agent database, the unique identifier may be repeated during the extraction process due to the complexity and diversity of the data, as well as potential uncertainties in the data generation and storage process, thereby causing different data items to be assigned the same unique feature and further causing data confusion or update errors.

[0051] Based on the above analysis, the frequency of each keyword in the current update data in the remaining update data is counted to form a keyword frequency matrix, which represents the repeatability of the keyword and the number of repeated occurrences. Each row in the keyword frequency matrix represents the frequency of a keyword in all the remaining update data. The frequency of each keyword in different update data is arranged in the order of update implementation, and each column in the keyword frequency matrix represents the frequency of all keywords in the current update data in the remaining update data.

[0052] Further, the historical difference degree of each keyword in the current update data is obtained by the proportion of the number of times that each keyword in the current update data does not appear in the remaining update data, and the difference in the frequency of each keyword in the current update data and the remaining update data. The expression is:

[0053] In the formula, represents the historical difference degree of the keyword corresponding to the i-th row of the keyword frequency matrix in the current update data; represents the number of zero elements in the i-th row of the keyword frequency matrix; a represents a predetermined positive integer, which is used to avoid the numerator being zero, resulting in the historical difference degree of the keyword being directly zero. The value of a is preset by human, and the implementer can set it himself. In this embodiment, the value of a is 1; n represents the number of columns of the keyword frequency matrix; represents the element in the i-th row and the j-th column of the keyword frequency matrix; represents the frequency of the keyword corresponding to the i-th row of the keyword frequency matrix in the current update data in the current update data; represents the absolute value operation.

[0054] It should be noted that: the greater the difference between the occurrence frequency of any keyword in the current update data and the occurrence frequency of the keyword in the historical update data, the higher the uniqueness of the keyword; at the same time, the more times the keyword does not appear in the historical update data, i.e. the less or never appears in the historical data, the higher the uniqueness of the keyword; in the above case, the historical difference degree of the keyword will be greater, indicating that the keyword has more distinguishing degree in the current update, and by optimizing the incremental update strategy through the keyword, the uniqueness of the data can be ensured and the management efficiency can be improved.

[0055] Step 3, filtering each candidate keyword from the keywords of the current update data through the historical difference degree; removing each candidate keyword from all candidate keywords in turn to obtain each candidate keyword set, obtaining the unique feature value of the keyword of the current update data after removing the each candidate keyword by the number of elements in each candidate keyword set that do not appear in the rest of the update data, and comparing the occurrence frequency of each element in the current update data and the rest of the update data.

[0056] When selecting keywords to calculate the unique identifier of the update data, keywords with large differences from historical update data need to be selected, therefore, according to the size of the historical difference degree of all keywords of the current update data, all keywords are divided into two categories, and each keyword in the category with the largest historical difference degree is used as a candidate keyword. Among them, when all keywords are divided into two categories, algorithms such as mean segmentation method, threshold segmentation algorithm, clustering algorithm can be used, and in this embodiment, the mean segmentation method is used, specifically: calculating the mean of the historical difference degree of all keywords of the current update data, and grouping all keywords of the current update data with a historical difference degree greater than or equal to the mean into a category, and grouping all keywords of the current update data with a historical difference degree less than the mean into a category. The corresponding row elements of all candidate keywords filtered in the keyword frequency matrix are arranged in the original order to form a feature frequency matrix.

[0057] The occurrence frequency of all keywords filtered in the current update data is composed into a frequency column vector, wherein the same row of the frequency column vector and the feature frequency matrix corresponds to the same candidate keyword. The frequency column vector is subtracted from each column element in the feature frequency matrix to obtain a frequency difference value column vector of each column element, and a frequency difference matrix is formed, which is used to represent the keyword information difference degree between the current update data and the rest of the update data. The elements in the frequency difference matrix are denoted as frequency difference values.

[0058] In addition, since a certain keyword can affect the difference between the current update data and the rest of the update data, the uniqueness of the keyword of the current update data is calculated after removing the certain keyword, and the expression is:

[0059] wherein, represents the uniqueness of the keyword of the current update data after removing the kth candidate keyword; m represents the number of the rest of the update data; represents the number of 0 elements in the jth column element after removing the corresponding row of the kth candidate keyword in the feature frequency matrix; b represents a preset positive integer, which is used to avoid 0, the value of b is artificially preset, and the implementer can set it by himself / herself, and the value of b in the embodiment is 1; represents the average value of positive values in the jth column element after removing the corresponding row of the kth candidate keyword in the frequency difference matrix.

[0060] It should be noted that the frequency change of the remaining candidate keywords after removing the kth candidate keyword from the current update data can reflect the contribution of the kth candidate keyword to the overall uniqueness. If the uniqueness of the keyword of the current update data is smaller after removing the kth candidate keyword, it means that the kth candidate keyword has a more important role in maintaining the data uniqueness. All the remaining candidate keywords after removing the kth candidate keyword form a candidate keyword set.

[0061] Step 4, the keyword uniqueness of the current update data is obtained by screening the number of all candidate keywords that do not appear in the rest of the update data, and the difference between the appearance frequency of each candidate keyword in the current update data and the rest of the update data.

[0062] Further, the keyword uniqueness of the current update data is obtained by screening the number of all candidate keywords that do not appear in the rest of the update data, and the difference between the appearance frequency of each candidate keyword in the current update data and the rest of the update data, and the expression is:

[0063] wherein, represents the keyword uniqueness of the current update data; m represents the number of the rest of the update data; represents the number of 0 elements in the jth column element in the feature frequency matrix; represents the average value of positive values in the jth column element in the frequency difference matrix; f represents a preset positive integer, which is used to avoid 0, the value of f is artificially preset, and the implementer can set it by himself / herself, and the value of f in the embodiment is 1.

[0064] It should be noted that the greater the difference between the current update data and the rest of the update data, the greater the difference in the frequency of occurrence of the keywords, and when a certain keyword does not appear in the rest of the update data, the higher the uniqueness of the certain keyword, which increases the uniqueness of the keywords of the current update data, and the unique identifier calculated using the part of the keywords is more unique, and the similarity between the unique identifier of the rest of the update data is lower. The acquisition process of the keyword uniqueness is shown in the schematic diagram of Figure 2 .

[0065] Step 5, extracting each feature keyword from the screened candidate keywords by the difference between the uniqueness feature value and the keyword uniqueness; based on the feature keyword, obtaining the unique identifier of the current update data, and storing it in the knowledge database of the AI agent, and searching for data and updating through the unique identifier.

[0066] After eliminating each candidate keyword, the difference between the uniqueness feature value of the keywords of the current update data and the keyword uniqueness of the current update data is recorded as the unique difference value, and when the unique difference value is less than or equal to 0, the corresponding eliminated candidate keyword is taken as a feature keyword. Arrange all feature keywords and their occurrence frequencies in the current update data in descending order of occurrence frequency to obtain a feature keyword string. For example, the feature keywords are: "management", "production", "yield", and "date", and the occurrence frequencies in the current update data are 4, 35, 3, and 21, respectively, so the feature keyword string is: "production 35 date 21 management 4 yield 3". Further, to avoid repetition of the feature keyword string, considering the uniqueness of time, the feature keyword string is added with the update date to obtain a feature unique string, such as the time being August 10, 2024, 12:08, then the feature unique string is "production 35 date 21 management 4 yield 3202408101208". The extraction process of the feature keyword is shown in the schematic diagram of Figure 3 .

[0067] Further, the feature unique string is taken as the input of the hash algorithm, and the output is the unique identifier. Then, the current update data and its unique identifier are stored in the knowledge database of the AI agent to realize incremental update of the data. When updating the data stored in the knowledge database, the data is identified by searching the unique identifier of the data, and the storage location of the data in the knowledge database is obtained through the unique identifier, and then the data is added, deleted or modified. The calculation process of the hash algorithm is a known technology, which will not be described herein.

[0068] To sum up, by calculating the historical difference degree, the keyword with significant change can be identified by quantifying the difference degree of the keyword between the current update data and the historical data; by the historical difference degree, the candidate keywords with representativeness and discrimination are screened out, which are used for extracting the feature keywords to generate the unique identifier, avoiding the problem that the generation of the unique identifier is too complex and difficult to guarantee the uniqueness if all the keywords are directly used; further, by eliminating the uniqueness difference of the keywords of the current update data before and after each candidate keyword, the uniqueness of each candidate keyword is evaluated, which provides a more accurate basis for generating the unique identifier; by the feature keywords with the most representativeness, the unique identifier of the current update data is generated, reducing the time consumption of the hash calculation, and meanwhile, the unique identifier can accurately reflect the core features of the current update data, which can effectively avoid the repetition and confusion of the data, ensure the uniqueness of the data in the knowledge database in each update, and quickly locate and update the data when the data in the knowledge database is updated, thereby improving the update efficiency and management efficiency of the knowledge database.

[0069] The flowcharts and block diagrams in the drawings show the possible implementation architecture, function and operation of the system, method and computer program product according to the embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logic function. In some alternative implementations, the functions noted in the blocks can also occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. In the description corresponding to the flowcharts and block diagrams in the drawings, the operations or steps corresponding to different blocks can also occur in different order from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. Each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0070] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the essential characteristics of the present application. Therefore, the above-described embodiments of the present application should be regarded as exemplary and non-limiting in any respect.

Claims

1. A method for updating a knowledge database for AI agents, characterized in that, The method includes the following steps: When a user uses the AI ​​agent, acquire each update of data and each keyword in each update of data; The historical difference degree of each keyword is obtained by measuring the percentage of times each keyword in the current update data does not appear in the other update data, and by combining the difference in the frequency of each keyword in the current update data with the other update data. Based on the historical difference, candidate keywords are selected from the keywords of the current updated data; each candidate keyword is removed from all candidate keywords in turn to obtain a set of candidate keywords; by measuring the number of elements in each candidate keyword set that do not appear in the other updated data, and comparing the frequency of each element in the current updated data and the other updated data, the unique feature value of the keywords in the current updated data after removing each candidate keyword is obtained. The keyword uniqueness of the current update data is obtained by calculating the number of candidate keywords that do not appear in the remaining updated data, and the difference in the frequency of each candidate keyword between the current update data and the remaining updated data. By using the difference between the uniqueness feature value and the keyword uniqueness, each feature keyword is extracted from the selected candidate keywords; based on the feature keywords, a unique identifier for the current updated data is obtained and stored in the knowledge database of the AI ​​agent, and the data is retrieved and updated using the unique identifier; The method for selecting candidate keywords from the keywords of the currently updated data is as follows: Based on the historical differences of all keywords in the current update data, all keywords are divided into two categories, and the keywords in the category with the greatest historical differences are selected as candidate keywords.

2. The knowledge database update method for AI intelligent agents as described in claim 1, characterized in that, The process of obtaining the historical difference is as follows: Calculate the difference in the frequency of occurrence of each keyword in the current update data and in each of the other updates data; Calculate the weighted sum of the differences between each keyword in the current updated data and all other updated data, wherein all the differences have equal weights and the sum of the weights is 1; The historical difference can be further obtained by the percentage of occurrences and the weighted sum, wherein the percentage of occurrences is not 0.

3. The knowledge database update method for AI intelligent agents as described in claim 2, characterized in that, The historical difference is the product of the percentage of occurrences and the weighted sum.

4. The knowledge database update method for AI intelligent agents as described in claim 1, characterized in that, The process for obtaining the unique feature value is as follows: Calculate the sum of the number of elements and a preset positive integer; The difference in frequency of each element in each candidate keyword set between the current updated data and each of the other updated data is denoted as the frequency difference; the mean of all positive values ​​in the frequency difference between the current updated data and each of the other updated data is calculated for all elements in each candidate keyword set. The uniqueness feature value can be further obtained by summing the remaining updated data and the mean value.

5. The knowledge database update method for AI intelligent agents as described in claim 4, characterized in that, The method for calculating the uniqueness feature value is as follows: Calculate the product of the cumulative sum and the mean of the remaining updated data; The uniqueness feature value is the sum of the products of all other updated data.

6. The knowledge database update method for AI intelligent agents as described in claim 1, characterized in that, The process of obtaining the keyword uniqueness is as follows: Calculate the sum of the stated number and a preset positive integer; Calculate the difference in frequency of each candidate keyword between the current updated data and each of the other updated data; calculate the average of the positive values ​​in the difference calculation results between the current updated data and each of the other updated data for all the selected candidate keywords; Calculate the product of the accumulated value and the average value of the remaining updated data; The keyword uniqueness is the sum of the product calculation results of all other updated data.

7. The knowledge database update method for AI intelligent agents as described in claim 1, characterized in that, The method for extracting the feature keywords is as follows: the difference between the uniqueness feature value and the uniqueness of the keyword is recorded as the uniqueness difference, and each feature keyword is extracted from the selected candidate keywords based on the magnitude of the uniqueness difference.

8. The knowledge database update method for AI intelligent agents as described in claim 7, characterized in that, The step of extracting feature keywords from the selected candidate keywords includes: taking the candidate keywords that are removed when the unique difference is less than or equal to 0 as feature keywords.

9. A knowledge database update method for AI intelligent agents as described in claim 1, characterized in that, The method for obtaining the unique identifier is as follows: The feature keywords and their frequency of occurrence in the current updated data are sorted in descending order of frequency to obtain the feature keyword string. The update date of the current updated data is added to the end of the feature keyword string to obtain a unique feature string. Combined with a hash algorithm, a unique identifier is obtained.

Images