Dynamic updating method, system and equipment of AI intelligent question and answer knowledge base and medium

Abstract

The invention relates to the technical field of data processing, and particularly provides a dynamic updating method, system and device for an AI intelligent question and answer knowledge base and a medium, and the method comprises the steps: obtaining to-be-updated content, extracting effective information from the to-be-updated content through a semantic recognition technology, and storing the effective information in a temporary file; matching the temporary file with metadata of a file which is uploaded to a knowledge base in a pre-constructed metadatabase; if the metadata matched with the temporary file exists, not executing the operation of updating the temporary file to the knowledge base; if the metadata matched with the temporary file does not exist, calling an interface of a knowledge base platform to update the temporary file to a knowledge base; and obtaining metadata of the temporary file returned by the knowledge base platform in the knowledge base, and storing the metadata in the metadatabase. Effective information is extracted through semantic recognition, and a metadata matching mechanism is combined, so that intelligent dynamic updating of the knowledge base is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to a dynamic updating method, system, device and medium of an AI intelligent question-answering knowledge base. BACKGROUND

[0002] The content timeliness and accuracy of the knowledge base relied on by the current AI intelligent question-answering system directly affect the service quality. The traditional knowledge base updating method usually has significant defects: firstly, it relies on manual review to determine whether the content already exists or needs to be updated, which is low in efficiency and high in cost; secondly, it lacks an effective repeated content detection mechanism, which easily leads to the same or highly similar content being uploaded multiple times, causing waste of storage resources and redundancy expansion of the knowledge base; and thirdly, the updating process lacks intelligent screening, and often needs full comparison or direct coverage, which not only occupies a large amount of computing resources and network bandwidth, but also increases the calling burden of the knowledge base platform interface. Especially when facing a large amount of rapidly changing information sources to be updated, the existing method is difficult to efficiently and accurately identify and filter out the semantically equivalent information already existing in the knowledge base while ensuring the real-time updating, resulting in low efficiency of knowledge base maintenance and decreased retrieval performance. Therefore, there is an urgent need for a technical solution that can automatically and intelligently identify and filter repeated semantic content and realize efficient dynamic updating of the knowledge base. SUMMARY

[0003] In view of the above problems of the prior art, the application provides a dynamic updating method, system, device and medium of an AI intelligent question-answering knowledge base to solve the above technical problems.

[0004] In a first aspect, the application provides a dynamic updating method of an AI intelligent question-answering knowledge base, comprising: obtaining to-be-updated content, extracting effective information from the to-be-updated content by using a semantic recognition technology, and saving the effective information to a temporary file; matching the temporary file with metadata of files uploaded to the knowledge base in a pre-constructed metadata database; if there is metadata matching the temporary file, not performing the operation of updating the temporary file to the knowledge base; if there is no metadata matching the temporary file, calling an interface of the knowledge base platform to update the temporary file to the knowledge base; obtaining metadata of the temporary file in the knowledge base returned by the knowledge base platform, and saving the metadata to the metadata database.

[0005] In an optional embodiment, obtaining to-be-updated content, extracting effective information from the to-be-updated content by using a semantic recognition technology, and saving the effective information to a temporary file comprises: obtaining to-be-updated content edited by a user; de-duplicating the content to be updated; extracting a triple of entity and relation from the de-duplicated content to be updated; saving the triple to a temporary file.

[0006] In an optional embodiment, the method further comprises: saving the temporary file to a pre-configured file storage path; generating a file name of the temporary file in combination with a time stamp, a type of the content to be updated, and an operation user ID of editing the content to be updated.

[0007] In an optional embodiment, matching the temporary file with metadata of files uploaded to the knowledge base in the pre-constructed metadata database comprises: judging whether the file name of the temporary file is consistent with the file name in the metadata: if consistent, the temporary file matches the metadata; if inconsistent, the temporary file does not match the metadata.

[0008] In an optional embodiment, updating the temporary file to the knowledge base by calling an interface of the knowledge base platform comprises: constructing a request meeting the requirements of the knowledge base platform, the request comprising authentication information and file processing rules; updating the temporary file to the knowledge base by calling the interface of the knowledge base platform by sending the request to the knowledge base platform.

[0009] In an optional embodiment, the file processing rules comprise an indexing mode, pre-processing rules, and segmentation rules; recognizing the content type of the temporary file by using a pre-trained FastText model, the content type comprising technical documents, customer service dialogues, and legal texts; setting corresponding pre-processing rules and segmentation rules according to the content type recognized by the lightweight classification module.

[0010] In an optional embodiment, updating the temporary file to the knowledge base by calling an interface of the knowledge base platform comprises: obtaining a text by applying pre-set cleaning rules to the content of the temporary file; generating text blocks separated by a specified delimiter by adaptively segmenting the text based on semantic integrity and the segmentation rules by calling a language model; The preprocessed file data is sent to the edge node; the edge node attempts to upload the entire file to the knowledge base platform's file processing interface on its first attempt; if it fails, it dynamically selects a compression algorithm based on the file content entropy to perform block compression, and then performs block upload on the compressed data. In the file processing interface, based on the actual file type identification result, a preset mapping table is queried to determine the corresponding indexing mode and special processing strategy, and an index is built for the uploaded file accordingly. Edge nodes send metadata about the files uploaded to the knowledge base back to the client.

[0011] Secondly, the present invention provides a dynamic update system for an AI intelligent question-answering knowledge base, comprising: The generation module is used to obtain the content to be updated, extract valid information from the content to be updated using semantic recognition technology, and save the valid information to a temporary file; The deduplication module is used to match the temporary file with the metadata of the files uploaded to the knowledge base in the pre-built metadata database; The termination module is used to prevent the operation of updating the temporary file to the knowledge base from being performed if metadata matching the temporary file exists. The upload module is used to update the temporary file to the knowledge base by calling the knowledge base platform's interface if no metadata matching the temporary file exists. The update module is used to obtain the metadata of the temporary files returned by the knowledge base platform in the knowledge base, and save the metadata to the metadata database.

[0012] Thirdly, a device is provided, comprising: The memory is used to store the dynamic update program of the AI ​​intelligent question-answering knowledge base; The processor is configured to implement the steps of the dynamic update method for the AI ​​intelligent question-answering knowledge base as provided in the first aspect when executing the dynamic update program for the AI ​​intelligent question-answering knowledge base.

[0013] Fourthly, a computer-readable medium is provided, on which a dynamic update program for an AI intelligent question-answering knowledge base is stored, wherein when the dynamic update program for the AI ​​intelligent question-answering knowledge base is executed by a processor, the steps of the dynamic update method for the AI ​​intelligent question-answering knowledge base provided in the first aspect are implemented.

[0014] The AI intelligent question and answer knowledge base dynamic updating method, system, device and medium provided by the application has the beneficial effects that: 1) the repeated content is accurately filtered, the redundant storage is avoided, and the resources are saved; 2) the updating demand is automatically identified, the manual intervention is greatly reduced, and the efficiency is improved; 3) only the necessary information is incrementally updated, the knowledge base platform interface calling and the calculation load are significantly reduced; 4) the timeliness and the simplicity of the knowledge base are effectively maintained, and thus the response speed and the answer accuracy of the question and answer system are improved. The operation efficiency and the service quality of the knowledge base are comprehensively improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0016] Figure 1 is a schematic flow chart of the method of one embodiment of the present application.

[0017] Figure 2 is a schematic block diagram of the system of one embodiment of the present application.

[0018] Figure 3 is a structural schematic diagram of a device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the person skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort should belong to the scope of protection of the present application.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application.

[0021] The AI intelligent question and answer knowledge base dynamic updating method provided by the embodiment of the present application is executed by a computer device, and accordingly, the AI intelligent question and answer knowledge base dynamic updating system runs in the computer device.

[0022] Figure 1 is a schematic flow chart of the method of one embodiment of the present application. In which, Figure 1 The execution subject can be a dynamic updating system of an AI intelligent question and answer knowledge base. According to different needs, the order of the steps in the flow chart can be changed, and some can be omitted.

[0023] As Figure 1 shown, the method comprises: S1. Obtain the content to be updated, extract the effective information from the content to be updated by using the semantic recognition technology, and save the effective information to a temporary file; S2. Match the temporary file with the metadata of the file uploaded to the knowledge base in the pre-constructed metadata database; S3. If there is metadata matching the temporary file, do not perform the operation of updating the temporary file to the knowledge base; S4. If there is no metadata matching the temporary file, call the interface of the knowledge base platform to update the temporary file to the knowledge base; S5. Obtain the metadata of the temporary file in the knowledge base returned by the knowledge base platform, and save the metadata to the metadata database.

[0024] In one embodiment of the present application, based on step S1, a possible embodiment will be given below to non-restrictively explain the specific implementation scheme.

[0025] S101. Obtain the content to be updated edited by the user.

[0026] A multi-protocol fusion content access architecture is adopted to realize accurate capture of the content to be updated. For structured editing data (such as form submission), a standardized reception is performed through a RESTful API interface, and the data format strictly follows the JSON Schema specification; for a rich text editing scene, a full-duplex communication channel is constructed based on a Web Socket protocol, real-time content incremental synchronization is supported, and transmission redundancy is reduced; for offline editing needs, a file uploading module integrated with FTP / SFTP protocol is adopted to realize a breakpoint resume function.

[0027] In order to guarantee data security, a multi-level verification mechanism is introduced: through JWT (JSON Web Token), bidirectional authentication of user identity is realized, and based on TLS1.3 protocol, the transmission link is encrypted; a 10MB threshold is set for a single content, and the content exceeding the threshold automatically triggers a fragmentation transmission strategy, the fragmentation size is fixed at 2MB, the breakpoint resume is realized through the HTTP Range header, and the integrity of large file transmission is ensured.

[0028] S102. De-duplicate the content to be updated.

[0029] Build a two-level deduplication system to achieve precise filtering of content redundancy: Precise deduplication layer: use SHA-256 hashing algorithm to extract fingerprints of content, generate 256-bit digest value, and compare with historical content hash library. Through Bloom filter (Bloom Filter) for preliminary screening, the hash collision probability is controlled at 10 -18 Below, ensure the accurate identification of completely repeated content.

[0030] Semantic deduplication layer: for similar content that is not completely repeated, use pre-trained model BERT (Base version) to convert text into 768-dimensional dense vector, and calculate the cosine similarity to evaluate the similarity at the semantic level. Set dynamic threshold mechanism (default threshold 0.92, support adaptive adjustment based on domain corpus), when the similarity exceeds the threshold, it is judged as semantic repetition. For long text (character number > 5000), use sliding window segmentation strategy (window size 512token, step 256token), through local semantic matching to improve the accuracy of long text deduplication, recall rate optimization to 94.6%.

[0031] S103. Extract the triplets of entities and relationships from the updated content after deduplication processing.

[0032] Use a hybrid knowledge extraction framework to accurately extract structured information: Entity recognition module: based on BERT-BiLSTM-CRF model architecture, through the introduction of domain dictionary (such as UMLS term library in medical field) for transfer learning, entity type covers core entity class (person, organization, concept), attribute class (time, numerical value, enumeration value) and event class (operation, state, process), after domain corpus fine-tuning, F1 value reaches 0.913.

[0033] Relation extraction module: adopt two-stage extraction strategy, first align the recognized entity with domain knowledge graph through entity linking technology (linking accuracy 0.897), then use PCNN-ATT model (convolutional neural network based on attention mechanism) for relationship classification, support 134 kinds of general relationship types and 28 kinds of domain exclusive relationship types identification.

[0034] Triplet normalization: through CorefBERT model to realize entity coreference resolution, map relationship terms to domain relationship ontology for normalization processing, format conversion for attribute values (such as date unified to ISO8601 standard format), finally generate <subject, relationship, object> triplet set, ensure the consistency of extraction results.

[0035] S104. Save the triplets to temporary file.

[0036] The triple sequence is serialized and stored in JSON-LD (JSON for Linking Data) format, and the file structure includes three core areas: Data area: store the triple set in array form, each triple contains "subject" (entity IRI or local unique identifier), "predicate" (relationship IRI), and "object" (entity IRI or literal value with data type) fields; Metadata area: record extraction process parameters (model version, confidence threshold), content features (original text length, entity density), and quality evaluation indicators (average confidence of triples, relationship extraction accuracy); Context area: define namespace mapping table to realize abbreviated expression of IRI to reduce storage volume.

[0037] The file is optimized using GZIP compression algorithm, and the compression level is set to 6 (balance compression ratio and computational overhead), with a stable compression ratio of 35%±5%.

[0038] S105. Save the temporary file to the pre-configured file storage path.

[0039] Based on distributed file system (such as MinIO), build hierarchical storage architecture, pre-configure path using "root directory / year / month / day / hour" multi-level directory structure, support data sharding and life cycle management by time dimension.

[0040] Introduce access control list (ACL) mechanism, strictly limit path operation permission: only content processing service has read and write permission, other services are only given read-only permission. File storage process uses transactional processing: first write to temporary buffer, then atomic migration to target path after MD5 verification to ensure file integrity, avoid damaged files. At the same time, based on Elasticsearch, build file index service, record storage path, size, modification time, etc. Metadata, support millisecond-level retrieval by content features.

[0041] S106. Generate the file name of the temporary file by combining the timestamp, the type of content to be updated, and the operation user ID of the content to be updated.

[0042] Use structured naming strategy to generate globally unique identifier, file name format is defined as: [timestamp]-[content type code]-[user ID hash].[file suffix], where: Timestamp uses 13-bit millisecond-level Unix timestamp to ensure time differentiation of files generated at the same time; Content type code adopts 2-bit hexadecimal number (0x01-0xFF), which corresponds to the predefined content type dictionary (for example, 0x01 represents technical document, and 0x02 represents news information); User ID hash adopts MD5 algorithm to perform one-way hash on the unique identification of the user, and the first 8-bit hexadecimal value is intercepted, so that the identity distinguishing ability is retained while protecting the privacy of the user; The file suffix is fixed as ".jsonld.gz", which clearly identifies the file format and compression mode. The naming rule controls the file renaming probability to 10 -20 The following, while supporting direct parsing of core attributes through the file name.

[0043] In an embodiment of the application, based on step S2, a possible embodiment will be given below to non-restrictively explain the specific implementation scheme.

[0044] S201. File name parsing and field extraction.

[0045] The file name of the temporary file is parsed according to the predefined format ([timestamp]-[content type code]-[user ID hash].[file suffix]) to extract four core fields of timestamp (13-digit string), content type code (2-bit hexadecimal number), user ID hash (8-bit hexadecimal number) and file suffix (fixed as.jsonld.gz). The parsing process adopts regular expression matching (the mode is ^(\d{13})-([0-9A-Fa-f]{2})-([0-9A-Fa-f]{8})\.jsonld\.gz$), which ensures the accuracy of field extraction, and the abnormal format file is directly determined as not matching.

[0046] S202. Metadata database query and field comparison.

[0047] Based on the extracted fields, the metadata database query condition is constructed, and the records containing the same timestamp, content type code and user ID hash fields are searched in the metadata index table. The comparison process adopts the field-by-field accurate matching strategy: The timestamp field is checked by numerical equality (millisecond-level precision matching); The content type code is checked by hexadecimal string equality (case-sensitive, for example, 0A and 0a are determined as not matching); The user ID hash is checked by string equality (the original character sequence of the MD5 hash value is maintained); The file suffix is fixed as.jsonld.gz, and if the suffix recorded in the metadata is different, it is directly determined as not matching.

[0048] S203. Matching result determination rule.

[0049] When and only when the four fields parsed from the temporary file are completely consistent with the corresponding fields of a record in the metadata database, it is determined to be "matched"; if there is any field that does not match (including length inconsistency, character difference, format exception, etc.), it is determined to be "not matched". In order to avoid index deviation, the matching process simultaneously checks the association between the file storage path of the record in the metadata and the current temporary file path, and ensures the spatiotemporal consistency of the matching result.

[0050] In an embodiment of the present application, based on step S3, a possible embodiment will be given below to non-restrictively illustrate the specific implementation thereof.

[0051] When the matching degree exceeds the preset threshold, the redundant interception process is triggered. The system automatically generates a "duplicate content analysis report", which includes semantic overlap area highlighting, similarity heat map and duplicate reason classification (complete duplication / semantic approximation / structure reuse). The report is pushed to the content management background through the message queue (RabbitMQ), and the interception log is recorded to the distributed log system (ELKStack) at the same time. The log field includes interception time, file fingerprint, matching score and processing node ID, which supports subsequent audit and threshold optimization.

[0052] In an embodiment of the present application, based on step S4, a possible embodiment will be given below to non-restrictively illustrate the specific implementation thereof.

[0053] If there is no metadata matching the temporary file, the interface of the knowledge base platform is called to update the temporary file to the knowledge base, specifically including the following steps.

[0054] S401. Construct an HTTP request message conforming to the RESTful specification, to realize the standardized submission of the temporary file to the knowledge base. The request structure includes an authentication layer and a business configuration layer: Authentication information encapsulation: JWT (JSON Web Token) is used to realize identity authentication. The token includes a three-tuple information: iss (unique identification of the request initiation service), exp (validity period, default 30 minutes), sign (digital signature based on HMAC-SHA256 algorithm, key is dynamically allocated by the knowledge base platform). The token is passed through the request header Authorization: Bearer <token>Carrying, while adding time stamp Nonce prevents replay attacks, time stamp deviation threshold set to 5 minutes.

[0055] Dynamic configuration of file processing rules, adaptive generation of processing rule set based on content type, specific process as follows: Content type identification: use the pre-trained FastText model (word vector dimension 100, ngram window [1, 3], training corpus contains 100,000 labeled samples) to classify temporary files, output technical documents (F1=0.92), customer service dialogues (F1=0.89), and legal texts (F1=0.94) three kinds of results. In order to improve efficiency, the front light rule engine (based on keyword matching and regular expression) is used for preliminary screening, which reduces the model calling cost by 30%.

[0056] Preprocessing rule mapping: according to the classification results, call the domain-specific preprocessing strategy: Technical documents: keep formula blocks (based on $...$ markers) and code segments (based on markers), and remove version history annotations (match / / Version\d+\.\d+); Customer service dialogues: extract the core interaction content within the <user> and <customer service> tags, and filter greetings (such as "Hello" and "Goodbye"); Legal texts: keep clause numbers (such as Article X) and reference markers (such as [XX Law] Article X), and standardize legal terms (based on Legal Term Standard Dictionary mapping).

[0057] Segmentation rule adaptation: set segmentation thresholds according to content type characteristics: Technical documents: segmented by chapter title (such as ##3.2 Experimental Methods) and semantic coherence (based on BERT sentence vector cosine similarity ≥0.75), with single segment length controlled at 800-1200 characters; Customer service dialogues: segmented by dialogue turns (<user>-<customer service> as the minimum unit), and merged with continuous statements of the same intent; Legal texts: segmented by clause units, and kept. And; as a sentence terminator, ensuring the independence of legal provisions.

[0058] Index mode specification: associate content type and index strategy through a pre-set mapping table: technical documents use "keyword + vector" hybrid index (Elasticsearch + Milvus), customer service dialogues use FAQ-style question and answer index, and legal texts use entity-clause association index (based on Neo4j graph database).

[0059] S402. Update the temporary file to the knowledge base by sending the request to the knowledge base platform.

[0060] Send a request to the knowledge base platform interface ( / api / v2 / knowledge / upload) through the HTTPS protocol to realize the secure upload and structured processing of the file, and the specific steps are as follows: Text preprocessing and segmentation: Cleaning stage: Apply general rules to remove HTML tags, redundant white spaces, and invisible characters (Unicode encoding \u0000-\u001F), and unify the text encoding to UTF-8; Segmentation stage: Call the fine-tuned BERT-base model to calculate the sentence-level semantic vector, combine the segmentation rules defined in S401, and find the optimal segmentation point through the dynamic programming algorithm to ensure the semantic integrity of each segment (semantic coherence ≥ 0.8). The segmentation result is expressed in <seg>Tag separation, generating directly indexable text block sequences.

[0061] Adaptive upload strategy of edge node, using edge computing node as upload intermediary to optimize transmission efficiency: First attempt complete upload: transmit file through ChunkedEncoding mode, set timeout threshold 30 seconds, fail retry 2 times (interval 1 second); Chunked compression upload (triggered after failure): Based on Shannon entropy to calculate content complexity (entropy value H≥4.5 is judged as high complexity), high complexity content adopts LZ77 algorithm (compression level 5), low complexity content adopts Huffman encoding; Dynamic adjustment of chunk size (512KB-4MB), generate check value through file hash (MD5) to ensure the integrity of the slice; Use HTTPRange header to realize slice transmission, and perform slice recombination and check by the knowledge base platform after uploading is completed.

[0062] Index construction of the knowledge base platform, the file processing interface receives data and executes: Type secondary check: combined with file header characteristics (such as \text{} mark of technical document) and content fingerprint, correct possible classification deviation (accuracy is improved to 0.96); Index mode matching: query preset mapping table (JSON format), determine index field (such as "clause number" and "effective date" of legal text), weight configuration (set the weight of "keyword" of technical document to 1.5) and special processing (such as intention label attached to customer service dialogue); Distributed index construction: call Spark cluster to process text blocks in parallel, and synchronously generate inverted index (Elasticsearch) and vector index (Milvus), and the index construction delay is controlled within 10 seconds.

[0063] The edge node aggregates the metadata (including file unique identifier kb_id, version number version, index state index_status, storage path uri and check hash checksum) returned by the knowledge base platform, and synchronously feeds back to the client through the JSON format, and simultaneously backs up the metadata to the distributed cache (Redis, expiration time 24 hours) to support fast query.

[0064] In an embodiment of the present application, based on step S5, a possible embodiment will be given below to non-restrictively explain the specific implementation scheme.

[0065] The metadata returned by the knowledge base includes core metadata (storage location, access permission, creation time) and extended metadata (semantic tags, associated file ID, access popularity). The synchronization process adopts an incremental update strategy, and only updates the changed fields by comparing the historical version in the metadata database. After synchronization, an index reconstruction task is triggered to implement full-text indexing of metadata based on Lucene, and update the semantic vectors in the vector database (such as Milvus) to ensure the real-time performance and accuracy of subsequent matching queries. The metadata synchronization state is maintained through ZooKeeper to achieve state consistency among distributed nodes.

[0066] In some embodiments, the dynamic updating system of the AI intelligent question and answer knowledge base can include a plurality of functional modules composed of computer program segments. The computer programs of each program segment in the dynamic updating system of the AI intelligent question and answer knowledge base can be stored in the memory of the computer device and executed by at least one processor to perform the functions of (see Figure 1 described) dynamically updating the AI intelligent question and answer knowledge base.

[0067] In this embodiment, the dynamic updating system of the AI intelligent question and answer knowledge base can be divided into a plurality of functional modules according to the functions it performs, as shown in Figure 2 The module referred to in the present application refers to a series of computer program segments that can be executed by at least one processor and can complete a fixed function, which are stored in the memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0068] The generating module is configured to obtain the content to be updated, extract effective information from the content to be updated using semantic recognition technology, and save the effective information to a temporary file; The duplicate checking module is configured to match the temporary file with the metadata of the files uploaded to the knowledge base in the pre-constructed metadata database; The terminating module is configured to, if there is metadata matching the temporary file, not perform the operation of updating the temporary file to the knowledge base; The uploading module is configured to, if there is no metadata matching the temporary file, call the interface of the knowledge base platform to update the temporary file to the knowledge base; The updating module is configured to obtain the metadata of the temporary file in the knowledge base returned by the knowledge base platform, and save the metadata to the metadata database.

[0069] Figure 3 The dynamic updating method of the AI intelligent question and answer knowledge base provided in the embodiments of the present application can be applied to a device. Those skilled in the art can understand that the device structure involved in the embodiments of the present application does not constitute a limitation on the device, and the device can include more or fewer components than the illustration, or combine certain components, or different component arrangements. In the embodiments of the present application, the device includes but is not limited to a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0070] The device 300 can include a processor 310, a memory 320, and a communication unit 330. These components communicate through one or more buses, and those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation on the present application, and it can be a bus structure or a star structure, and can include more or fewer components than the illustration, or combine certain components, or different component arrangements.

[0071] The memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the device 300 can execute part or all of the steps in the following method embodiments.

[0072] The processor 310 is the control center of the storage device, connects various parts of the entire electronic device by using various interfaces and lines, and performs various functions of the electronic device and / or processes data by running or executing software programs and / or modules stored in the memory 320 and calling data stored in the memory. The processor can be composed of an integrated circuit (IC), for example, can be composed of a single packaged IC, or can be composed of a plurality of packaged ICs connected together. For example, the processor 310 can only include a central processing unit (CPU). In the embodiments of the present application, the CPU can be a single operation core or can include multiple operation cores.

[0073] The communication unit 330 is configured to establish a communication channel, so that the storage device can communicate with other devices. The communication unit 330 receives user data sent by other devices or sends user data to other devices.

[0074] The present application also provides a computer medium, wherein the computer medium can store a program, and the program can include some or all steps in the embodiments provided by the present application when executed. The medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0075] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be realized by means of software and necessary general hardware platforms. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which is stored in a medium such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disc or an optical disc, etc. The medium can store program codes, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application.

[0076] In the present specification, the same or similar parts among various embodiments can be referred to each other. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0077] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other manners. For example, the system embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There can be another division manner for the actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different modules can be indirect couplings or communication connections through some interfaces, and electrical, mechanical or other forms.

[0078] The above described modules can or can not be physically separate, and the components shown as modules can or can not be physical modules, i.e., can be located in one place or distributed on a plurality of network modules. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0079] In addition, the functional modules in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0080] Although the present application has been described in detail by referring to the preferred embodiments thereof, it is to be understood that the present application is not limited to the embodiments described above. Rather, it should be appreciated that those skilled in the art, upon consideration of the disclosure, can make modifications and variations to the embodiments of the present application which are also within the scope of the present application. Any modifications and variations which can be made to the present application by those skilled in the art are to be considered within the scope of the present application disclosed herein.< / seg> < / token>

Claims

1. A method for dynamically updating an AI-powered intelligent question-answering knowledge base, characterized in that, include: Obtain the content to be updated, use semantic recognition technology to extract valid information from the content to be updated, and save the valid information to a temporary file; The temporary file is matched with the metadata of files uploaded to the knowledge base in the pre-built metadata database; If metadata matching the temporary file exists, the operation of updating the temporary file to the knowledge base will not be performed; If no metadata matching the temporary file exists, the temporary file is updated to the knowledge base by calling the knowledge base platform's interface. Obtain the metadata of the temporary file returned by the knowledge base platform in the knowledge base, and save the metadata to the metadata database.

2. The method according to claim 1, characterized in that, Obtain the content to be updated, use semantic recognition technology to extract valid information from the content, and save the valid information to a temporary file, including: Retrieve user-edited content awaiting updates; The content to be updated is deduplicated; Extract triples of entities and relations from the deduplicated content to be updated; Save the triple to a temporary file.

3. The method according to claim 2, characterized in that, The method further includes: Save the temporary file to the pre-configured file storage path; The filename of the temporary file is generated by combining the timestamp, the type of content to be updated, and the user ID of the user who edited the content to be updated.

4. The method according to claim 1, characterized in that, The temporary file is matched with the metadata of files uploaded to the knowledge base in a pre-built metadata database, including: Determine whether the filename of the temporary file matches the filename in the metadata: If they match, then the temporary file matches the metadata; If they are inconsistent, then the temporary file does not match the metadata.

5. The method according to claim 1, characterized in that, The temporary file is updated to the knowledge base by calling the knowledge base platform's interface, including: Construct a request that conforms to the requirements of the knowledge base platform, the request including authentication information and file processing rules; By sending the request to the knowledge base platform, the temporary file is updated to the knowledge base by calling the knowledge base platform's interface.

6. The method according to claim 5, characterized in that, The file processing rules include indexing methods, preprocessing rules, and segmentation rules; The content type of the temporary file is identified using a pre-trained FastText model, and the content types include technical documents, customer service dialogues, and legal texts; Based on the content type identified by the lightweight classification module, set corresponding preprocessing rules and segmentation rules.

7. The method according to claim 6, characterized in that, The temporary file is updated to the knowledge base by calling the knowledge base platform's interface, including: After preprocessing the contents of the temporary file using preset cleaning rules, the text is obtained. The language model is invoked to adaptively segment the text based on semantic integrity and the segmentation rules, generating text blocks separated by specified delimiters; The preprocessed file data is sent to the edge node; the edge node attempts to upload the entire file to the knowledge base platform's file processing interface on its first attempt; if it fails, it dynamically selects a compression algorithm based on the file content entropy to perform block compression, and then performs block upload on the compressed data. In the file processing interface, based on the actual file type identification result, a preset mapping table is queried to determine the corresponding indexing mode and special processing strategy, and an index is built for the uploaded file accordingly. Edge nodes send metadata about the files uploaded to the knowledge base back to the client.

8. A dynamic update system for an AI-powered intelligent question-answering knowledge base, characterized in that, include: The generation module is used to obtain the content to be updated, extract valid information from the content to be updated using semantic recognition technology, and save the valid information to a temporary file; The deduplication module is used to match the temporary file with the metadata of the files uploaded to the knowledge base in the pre-built metadata database; The termination module is used to prevent the operation of updating the temporary file to the knowledge base from being performed if metadata matching the temporary file exists. The upload module is used to update the temporary file to the knowledge base by calling the knowledge base platform's interface if no metadata matching the temporary file exists. The update module is used to obtain the metadata of the temporary files returned by the knowledge base platform in the knowledge base, and save the metadata to the metadata database.

9. A device for dynamically updating an AI-powered intelligent question-answering knowledge base, characterized in that, include: The memory is used to store the dynamic update program of the AI ​​intelligent question-answering knowledge base; A processor, configured to implement the steps of the dynamic update method for the AI ​​intelligent question-answering knowledge base as described in any one of claims 1-7 when executing the dynamic update program for the AI ​​intelligent question-answering knowledge base.

10. A computer-readable medium storing a computer program, characterized in that, The readable medium stores a dynamic update program for an AI intelligent question-answering knowledge base. When the processor executes the dynamic update program for the AI ​​intelligent question-answering knowledge base, it implements the steps of the dynamic update method for the AI ​​intelligent question-answering knowledge base as described in any one of claims 1-7.

Images