Automatic construction method, device, electronic device and storage medium of knowledge base

Abstract

The present application relates to the field of artificial intelligence technology, and provides a method, device, electronic device and storage medium for automatically constructing a knowledge base. The method includes: in a large language model, obtaining an initial text through a dialogue system of the large language model; when the initial text is not in the form of a question-answer pair, using a pre-constructed first prompt template to clean the initial text to obtain a first usable text; using a pre-constructed second prompt template to extract structured knowledge from the first usable text to obtain the initial information required to construct a knowledge base; using a pre-constructed third prompt template to perform knowledge fusion and verification on the initial information to obtain target information, and construct a knowledge base based on the target information. The present application solves the problem of high cost of constructing a knowledge base in the prior art.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for automatically constructing a knowledge base. Background Art

[0002] Knowledge base is of great significance to the development of enterprises. It can improve work efficiency, promote collaboration and communication, and play an important role in improving customer service quality, supporting decision-making and innovation. At the same time, knowledge base is also very important for the implementation of large language models (LLM). Retrieval-Augmented Generation (RAG), as the core technology to improve the LLM hallucination problem, requires a high-quality external knowledge base for retrieval to enhance model capabilities.

[0003] However, building a knowledge base is a difficult task for general enterprises and individuals. The construction of a knowledge base requires a lot of resources and time investment. Enterprises need to collect, organize and update various information and incorporate them into the knowledge base. In addition, the construction of a knowledge base requires professional knowledge and skills. Generally, enterprises need to hire professionals or introduce commercial services to build and maintain the knowledge base. It can be seen that the construction cost of the knowledge base in the existing technology is relatively high. Summary of the invention

[0004] In view of this, the present application provides a method, device, electronic device and storage medium for automatically constructing a knowledge base to solve the problem of high cost of constructing a knowledge base in the prior art.

[0005] In a first aspect of the present application, a method for automatically constructing a knowledge base is provided, comprising: obtaining an initial text in a large language model through a dialogue system of the large language model; when the initial text is not in the form of a question-answer pair, using a pre-constructed first prompt template to clean the initial text to obtain a first usable text; using a pre-constructed second prompt template to extract structured knowledge from the first usable text to obtain initial information required for constructing a knowledge base; using a pre-constructed third prompt template to perform knowledge fusion and verification on the initial information to obtain target information, and constructing a knowledge base based on the target information.

[0006] According to a second aspect of the present application, there is provided an automatic construction device for a knowledge base, comprising: an acquisition module configured to acquire an initial text in a large language model through a dialogue system of the large language model; a cleaning module configured to clean the initial text using a pre-constructed first prompt template to obtain a first usable text when the initial text is not in the form of a question-answer pair; an extraction module configured to extract structured knowledge from the first usable text using a pre-constructed second prompt template to obtain initial information required for constructing a knowledge base; and a construction module configured to perform knowledge fusion and verification on the initial information using a pre-constructed third prompt template to obtain target information, and construct a knowledge base based on the target information.

[0007] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the present application, a storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] At least one of the above technical solutions adopted in this application can achieve the following beneficial effects:

[0010] The initial text input by the user is obtained through the large language model dialogue system. When the initial text is not in the form of a question-answer pair, the first prompt template is used to clean the initial text to obtain the first usable text, and then the second prompt template is used to extract structured knowledge from the first usable text to obtain the initial information required for building the knowledge base, which is represented here as a relationship triple. After that, the third prompt template can be used to fuse and verify the initial information to obtain accurate and complete target information. Finally, a database is generated based on the obtained target information. In this way, the prompt template required in the knowledge base creation process built in the large language model in advance can be used to perform a series of processing on the initial text to obtain the target information. The understanding ability of the large language model can be used to automatically clean and integrate the initial text, without the need for manual complex natural language processing processes. It can help enterprises or non-natural language processing practitioners to build knowledge bases, which not only improves work efficiency, but also effectively saves the cost of knowledge base construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 It is a flowchart of a method for automatically constructing a knowledge base provided in an embodiment of the present application;

[0013] Figure 2 It is a flowchart of another method for automatically constructing a knowledge base provided in an embodiment of the present application;

[0014] Figure 3 It is a structural schematic diagram of an automatic knowledge base construction device provided in an embodiment of the present application;

[0015] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0017] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0018] In addition, it should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "includes..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0019] A method and device for automatically constructing a knowledge base according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0020] Figure 1Schematic diagram of a method for automatically constructing a knowledge base provided in an embodiment of the present application. Figure 1 As shown, the automatic construction method of the knowledge base includes:

[0021] S101, in the large language model, obtaining initial text through the dialogue system of the large language model;

[0022] S102, when the initial text is not in the form of a question-answer pair, the initial text is cleaned using a pre-constructed first prompt template to obtain a first usable text;

[0023] S103, using the pre-constructed second prompt template to extract structured knowledge from the first available text to obtain initial information required for building a knowledge base;

[0024] S104, using the pre-constructed third prompt template to perform knowledge fusion and verification on the initial information, obtain target information, and build a knowledge base based on the target information.

[0025] Specifically, first, the initial text input by the user can be obtained through the dialogue system of the large language model, and then the large language model can be used to automatically perform data cleaning, structured knowledge extraction, knowledge verification and other steps on the initial text to help the user create a knowledge base, thereby avoiding the heavy manual processes and costs in knowledge base construction. In the large language model, the required prompt templates, namely the first prompt template, the second prompt template and the third prompt template, can be manually configured first, so that the large language model can efficiently process the initial text according to the pre-configured templates.

[0026] Furthermore, when configuring the prompt template, you can construct it according to the type of knowledge base you ultimately want to build and the data to be stored, and you can also train the configured prompt template by obtaining a training set to obtain a prompt template with accurate output results, thereby obtaining accurate target information to achieve efficient construction of the knowledge base.

[0027] In some embodiments, after obtaining the initial text through the dialogue system of the large language model, it also includes: when the initial text is in the form of a question-answer pair, using the first prompt template to clean the question-answer pair to obtain a second usable text, the second usable text includes the original question and the original answer; using a pre-constructed fourth prompt template to rewrite the original question to obtain a new question, and constructing the knowledge base based on the new question and the original answer, the new question corresponds to the original answer, and the rewriting includes synonym replacement, grammatical correction and domain expansion.

[0028] Specifically, the initial text may be a text paragraph or a question-answer pair. When the initial text is in the form of a question-answer pair, the question-answer pair can be cleaned using the first prompt template to obtain a second usable text: In this case, the initial text in the form of a question-answer pair can be cleaned and processed using the first prompt template to obtain a second usable text. This process can also remove junk information, correct grammatical errors, remove duplicate content, etc.

[0029] Furthermore, for the original question in the second available text, the pre-constructed fourth prompt template can be used to rewrite to generate a new question. The rewriting process may involve techniques such as synonym replacement, grammatical correction, and domain expansion, aiming to make the question clearer and more accurate or cover a wider range of domain knowledge. As an example, "Question: During which time period is sleep quality the best?" is rewritten as "Question: For people aged 20-30, which time period of the day has the best sleep quality?", but there is no restriction here, and it can be rewritten according to the actual situation, or it can be identified and rewritten into a more appropriate expression based on LLM.

[0030] Finally, a knowledge base can be constructed based on the rewritten new questions and original answers. This allows users to get the same answers through different forms of questions. This can help expand the coverage of the knowledge base and provide more angles of query. It can also improve the efficiency and quality of question-answering processing of formal text, making the knowledge base more comprehensive and easy to use.

[0031] In some embodiments, a pre-constructed first prompt template is used to clean the initial text to obtain a first usable text, including: using the first prompt template to identify junk information in the initial text and filter the junk information, the junk information includes garbled characters, emoticons, hypertext tags, and continuous punctuation marks exceeding a preset number; using the first prompt template to perform grammatical judgment on the initial text and correct incorrect grammar; using the first prompt template to perform content duplication judgment on the initial text, if there is repeated content, retaining the content that appears for the first time in the repeated content; executing the above processing steps through the first prompt template to obtain the first usable text.

[0032] Specifically, the initial text can be cleaned using the first prompt template pre-constructed in the large language model to remove noise, erroneous information or inaccurate information therein, so as to improve the quality of the initial text and thus build a higher quality knowledge base.

[0033] Furthermore, the first prompt template can guide the large language model to receive an initial text and output a first usable text obtained after cleaning. The first prompt template can indicate spam identification and filtering, and can also indicate grammatical judgment on the initial text and correction of incorrect grammar, and can also deduplicate repeated content. In the large language model, these steps can be performed each time the data is cleaned to output a clean and accurate first usable text.

[0034] The construction of the first prompt template may also have other indication functions, but in this application, it may at least include: 1. Remove special characters: including garbled characters, HTML web page tags, more than three consecutive punctuation marks, etc. 2. Remove duplicate content: when two paragraphs are completely repeated, only the first occurrence is retained. 3. Output the cleaned text.

[0035] As an example, a first prompt template for extracting keywords and topics can be constructed. The initial text cleaning process for the input is as follows:

[0036] Input (initial text): I recently read a book about artificial intelligence. . . The book introduces concepts such as machine learning, deep learning, and natural language processing. By reading this book , I have a deeper understanding of artificial intelligence. In the past few years, artificial intelligence technology has made great progress, and I think artificial intelligence will play an important role in the future and change our lives. In the past few years, artificial intelligence technology has made great progress, and I think artificial intelligence will play an important role in the future and change our lives.

[0037] Output (First Available Text): Recently I read a book about artificial intelligence, which introduced concepts such as machine learning, deep learning, and natural language processing. By reading this book, I have a deeper understanding of artificial intelligence. In the past few years, artificial intelligence technology has made great progress, and I think artificial intelligence will play an important role in the future and change our lives.

[0038] According to the method of the above embodiment, the initial text can be cleaned by using the pre-constructed first prompt text, interfering factors can be removed, and the data quality can be improved to obtain a more reliable and useful knowledge base.

[0039] In some embodiments, a pre-constructed second prompt template is used to perform structured knowledge extraction on the first available text to obtain initial information required for building a knowledge base, including: using the second prompt template to perform semantic understanding, entity classification, entity recognition, relationship extraction, entity linking and relationship disambiguation on the first available text to obtain initial information, wherein the initial information includes triples.

[0040] Specifically, the module where the second prompt template is located can receive the cleaned first available text as input. By constructing a special second prompt template, LLM can be directly used to perform processing steps such as semantic understanding, entity classification, entity recognition, relationship extraction, entity linking and relationship disambiguation without the participation of multiple traditional natural language processing models.

[0041] Furthermore, the task of the second prompt template can be expressed as information extraction, that is, receiving the first available text, analyzing and extracting the triples therein to obtain the first available text, and the triples can be relationship triples or entity-attribute triples. If the triples are relationship triples, the entity and relationship information is extracted from the first available text and converted into a structured form, such as entity relationship triples and categories. These structured information can better organize and manage knowledge, so that the system can query and respond to user requests more efficiently. In order to achieve clear and complete semantics of the triples, the second prompt template can be trained in advance using a large amount of data.

[0042] A triple can represent a relationship consisting of three elements, usually including a subject entity, a relationship type, and an object entity, such as (apple, belongs to, fruit). It can also include entities, attributes, and attribute values, such as (Yao, gender, male). Such structured information can help the system better understand the relationship between knowledge and provide users with accurate and relevant answers.

[0043] Through such a modular design, the system can more efficiently process the initial text input by the user and convert it into structured knowledge representation, thereby improving the overall performance and user experience of the dialogue system.

[0044] For example, the task of the second prompt template can be implemented by the following structure:

[0045] Input (first available text): Yao Moumou (September 12, 1980 -), male, native of Zhenze Town, Wujiang District, Suzhou City, Jiangsu Province, born in Shanghai, nicknamed "Little Giant" and "The Walking Great Wall".

[0046] Output (entity-attribute triples): [

[0047] (Yao, date of birth, September 12, 1980),

[0048] (Yao, gender, male),

[0049] (Yao, native place: Zhenze Town, Wujiang District, Suzhou City, Jiangsu Province),

[0050] (Yao, birthplace, Shanghai),

[0051] (Yao, nickname, Little Giant),

[0052] (Yao Moumou, nickname, Mobile Great Wall) ]

[0054] According to the method of the above embodiment, triples can be obtained by processing the first available text, so that clear initial information required for constructing a knowledge base can be obtained.

[0055] In some embodiments, a pre-constructed third prompt template is used to perform knowledge fusion and verification on the initial information to obtain target information, including: using the third prompt template to resolve conflicts for identical entities in each string of triples to obtain resolved initial information; and verifying the credibility of the resolved initial information to obtain target information whose credibility score exceeds a threshold.

[0056] Specifically, the third prompt template can be used to resolve conflicts and verify the credibility of the same entities in the triplet to ensure the consistency and accuracy of the final information.

[0057] Furthermore, conflict resolution for the same entity in a triple can be expressed as follows: if different instances of the same attribute and relationship for an entity are found, and if the attribute and relationship can be multi-valued, no processing is required. For example, two different work experiences of a person are both correct and there is no conflict.

[0058] Alternatively, if different instances of the same attribute or relationship are found for an entity, if the attribute or relationship has a unique value, it will be processed based on voting. For example, for a person's birth time, the value with the highest number of occurrences will be used.

[0059] As an example:

[0060] Input (initial information): [

[0061] (Yao, date of birth, September 12, 1980),

[0062] (Yao, date of birth, September 12, 1980),

[0063] (Yao, gender, male),

[0064] (Yao, birthplace, Shanghai),

[0065] (Yao, nickname, Little Giant),

[0066] (Yao, nickname, the mobile Great Wall),

[0067] (Yao Moumou, date of birth, September 8, 1980) ]

[0069] Output (initial information after solving): [

[0070] (Yao, date of birth, September 12, 1980),

[0071] (Yao, gender, male),

[0072] (Yao, birthplace, Shanghai),

[0073] (Yao, nickname, Little Giant),

[0074] (Yao Moumou, nickname, Mobile Great Wall) ]

[0076] Afterwards, the credibility of the resolved initial information can be verified. When verifying the credibility, the LLM system can be used to count the credibility score of each piece of information and compare it with the preset threshold. If the credibility score of the corresponding information exceeds the threshold, it will be used as the target information. For example, if the credibility score of (Yao Moumou, nickname, Mobile Great Wall) is 0.99, then this piece of information is the target information. Of course, the credibility verification can also be performed through manual review operations to mark the credibility score of the initial information and obtain the target information whose credibility score exceeds the threshold. Before the initial information is fused and verified through the third prompt template, a training set can also be constructed to train the third prompt template.

[0077] According to the method of the above embodiment, the credibility of the initial information can be verified through the third prompt template. The credibility verification can include an evaluation of the source, consistency and logical rationality of the initial information to determine its credibility score, so as to ensure that the extracted target information is accurate and reliable, and use it to build an accurate knowledge base.

[0078] In some embodiments, after building a knowledge base based on the target information, it also includes: obtaining a query text in the language form input by the user; if it is determined that the query text requires wrong word rewriting or entity alias normalization, then rewriting the wrong word or unifying the entity alias according to a predefined entity name normalization table to obtain a processed query text; extracting entities and relationships from the processed query text to obtain key information, and converting the key information into a database query statement; executing a knowledge base query according to the database query statement to obtain a knowledge query result.

[0079] Specifically, building a knowledge base is not just about creating a static knowledge base, but about making use of it, such as providing a knowledge base query API to facilitate users to use the knowledge base. When performing a knowledge base query, the LLM system can be used to help users convert query text into database query statements and execute knowledge base queries.

[0080] Of course, after receiving the query text input by the user, you can first check whether the query text needs to be rewritten or the entity alias normalized. If rewriting is required, the LLM system can rewrite the wrong words or unify the entity aliases according to the pre-defined entity name normalization table (such as entity_dict) to normalize the entity names in the query text.

[0081] Next, the LLM system can extract entities and relations from the processed query text and extract key information, such as the names of entities and relations.

[0082] The extracted key information can then be converted into standard SPARQL query statements, which is a standardized query language for querying information in knowledge graphs, and can accurately express the user's query requirements.

[0083] Finally, the LLM system can return the converted standard SPARQL query statements, and users can execute these query statements to obtain the required knowledge query results from the knowledge base. In this way, the system can accurately understand the user's needs and generate corresponding query statements, thereby providing accurate knowledge query results.

[0084] In some embodiments, a knowledge base query is executed according to a database query statement to obtain a knowledge query result, including: judging the knowledge query result and the query text through a large language model, outputting the knowledge query result if the knowledge query result matches the query text, and displaying the knowledge query result through a visual interface.

[0085] Specifically, after obtaining the knowledge query results in the LLM system, the knowledge query results can also be matched and diagnosed with the query text to determine whether the knowledge query results match the user's intention. If the knowledge query results match the query text, these knowledge query results are output and displayed to the user through a visual interface so that the user can intuitively understand and browse the knowledge query results. Of course, if they do not match, the query text can be searched again. If there is no knowledge query result matching the query text after searching many times (such as 6 times), it means that there is no corresponding knowledge in the knowledge base, and the query can be terminated. At this time, the output can be: Sorry, no relevant knowledge was retrieved. However, there is no restriction here, and the display results of relevant situations can be set according to the actual situation.

[0086] When judging the matching degree between the knowledge query result and the query text, factors such as keyword matching, semantic similarity, context matching, entity association, etc. can be considered, which will not be elaborated here.

[0087] Of course, there may be scattered or redundant knowledge query results. In this case, the knowledge query results can be automatically rewritten according to LLM to generate appropriate answers. In practical applications, the LLM system can be trained to achieve this effect, which will not be expanded here.

[0088] According to the method of the above embodiment, by judging the matching degree between the query text and the knowledge query result and displaying the matching knowledge query result, the user's accurate understanding and effective use of the query result can be improved.

[0089] Figure 2 This is a flow chart of another method for automatically constructing a knowledge base provided in an embodiment of the present application. Figure 2 The embodiments of the present application are further described as follows:

[0090] First, users can input data through the interactive system of LLM, that is, the initial text, and obtain usable text after LLM processing. Then, the structured knowledge or question-answer pairs in the usable text are extracted, and the corresponding structured knowledge or question-answer pairs are further integrated and credibility-verified through LLM to obtain credible structured data or question-answer pairs, that is, target information, or original questions and original answers. In this step, the data can be manually assisted to verify its credibility to increase data accuracy, and then the corresponding knowledge base is generated.

[0091] After obtaining the knowledge base, the knowledge in the knowledge base can be queried. Similarly, the user can input the query text through the LLM dialogue system structure and process it, retrieve the knowledge query results from the knowledge base, and finally process and rewrite the knowledge query results, and output the answers back to the user.

[0092] In this way, LLM can be used to automatically clean and integrate the initial text input by the user, automatically build a knowledge base, and perform knowledge retrieval on the knowledge base. This not only greatly saves the cost of building the knowledge base, but also improves the efficiency and accuracy of knowledge base construction by using the model. In addition, when configuring each prompt template for the large language model, the knowledge base that needs to be built can be configured in a targeted manner, thereby achieving personalization and flexibility in knowledge base construction.

[0093] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0094] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0095] Figure 3 Schematic diagram of an automatic knowledge base construction device provided in an embodiment of the present application. Figure 3 As shown, the automatic construction device of the knowledge base includes:

[0096] An acquisition module 301 is configured to acquire an initial text in the large language model through a dialogue system of the large language model;

[0097] A cleaning module 302 is configured to clean the initial text using a pre-constructed first prompt template to obtain a first usable text when the initial text is not in the form of a question-answer pair;

[0098] The extraction module 303 is configured to extract structured knowledge from the first available text using a pre-constructed second prompt template to obtain initial information required for constructing a knowledge base;

[0099] The construction module 304 is configured to use the pre-constructed third prompt template to perform knowledge fusion and verification on the initial information to obtain target information, and construct a knowledge base based on the target information.

[0100] In some embodiments, the acquisition module 301 is specifically used to, when the initial text is in the form of a question-answer pair, use the first prompt template to clean the question-answer pair to obtain a second usable text, the second usable text including the original question and the original answer; use a pre-constructed fourth prompt template to rewrite the original question to obtain a new question, and construct the knowledge base based on the new question and the original answer, the new question corresponds to the original answer, and the rewriting includes synonym replacement, grammatical correction and domain expansion.

[0101] In some embodiments, the cleaning module 302 is specifically used to use the first prompt template to identify junk information in the initial text and filter the junk information, which includes garbled characters, emoticons, hypertext tags, and continuous punctuation marks exceeding a preset number; use the first prompt template to perform grammatical judgment on the initial text and correct incorrect grammar; use the first prompt template to perform content duplication judgment on the initial text, and if there is repeated content, retain the content that appears for the first time in the repeated content; execute the above processing steps through the first prompt template to obtain the first usable text.

[0102] In some embodiments, the extraction module 303 is specifically used to use the second prompt template to perform semantic understanding, entity classification, entity recognition, relationship extraction, entity linking and relationship disambiguation processing on the first available text to obtain initial information, which includes triples.

[0103] In some embodiments, the construction module 304 is specifically used to use the third prompt template to resolve conflicts for the same entities in each string of triples to obtain resolved initial information; and to verify the credibility of the resolved initial information to obtain target information with a credibility score exceeding a threshold.

[0104] In some embodiments, the construction module 304 is specifically used to obtain a query text in the language form input by the user; if it is determined that the query text requires wrong word rewriting or entity alias normalization, the wrong word is rewritten or the entity alias is uniformly processed according to a predefined entity name normalization table to obtain a processed query text; entities and relationships are extracted from the processed query text to obtain key information, and the key information is converted into a database query statement; a knowledge base query is executed according to the database query statement to obtain a knowledge query result.

[0105] In some embodiments, the construction module 304 is specifically used to judge the knowledge query result and the query text through a large language model, and if the knowledge query result matches the query text, the knowledge query result is output and displayed through a visual interface.

[0106] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0107] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0108] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0109] The processor 401 may be a central processing unit (CPU) or a graphics processing unit (GPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0110] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0111] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0112] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0113] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for automatically constructing a knowledge base, characterized in that: include: In the large language model, obtaining initial text through a dialogue system of the large language model; When the initial text is not in the form of a question-answer pair, the initial text is cleaned using a first prompt template pre-constructed in the large language model to obtain a first usable text; The step of using the first prompt template pre-constructed in the large language model to clean the initial text to obtain the first usable text includes: using the first prompt template to identify junk information in the initial text and filtering the junk information, wherein the junk information includes garbled characters, emoticons, hypertext tags, and continuous punctuation marks exceeding a preset number; using the first prompt template to perform grammar judgment on the initial text and correct incorrect grammar; using the first prompt template to perform content repetition judgment on the initial text, and if there is repetitive content, retaining the content that appears for the first time in the repetitive content; performing the cleaning step through the first prompt template to obtain the first usable text; Using the second prompt template pre-constructed in the large language model to extract structured knowledge from the first available text to obtain initial information required for building a knowledge base; Using the second prompt template pre-constructed in the large language model to extract structured knowledge from the first available text to obtain initial information required for building a knowledge base includes: using the second prompt template to perform semantic understanding, entity classification, entity recognition, relationship extraction, entity linking and relationship disambiguation processing on the first available text to obtain the initial information, wherein the initial information includes triples; The initial information is subjected to knowledge fusion and verification using a third prompt template pre-constructed in the large language model to obtain target information, and a knowledge base is constructed based on the target information; the method of using the third prompt template pre-constructed in the large language model to perform knowledge fusion and verification on the initial information to obtain target information includes: using the third prompt template to resolve conflicts for identical entities in each string of triples to obtain resolved initial information; and performing credibility verification on the resolved initial information to obtain the target information whose credibility score exceeds a threshold.

2. The method according to claim 1, characterized in that After the initial text is acquired by the dialogue system of the large language model, the method further includes: When the initial text is in the form of a question-answer pair, the question-answer pair is cleaned using the first prompt template to obtain a second usable text, where the second usable text includes the original question and the original answer; The original question is rewritten using a pre-constructed fourth prompt template to obtain a new question, and the knowledge base is constructed based on the new question and the original answer. The new question corresponds to the original answer, and the rewriting includes synonym replacement, grammatical correction and domain expansion.

3. The method according to claim 1, characterized in that After building a knowledge base based on the target information, the method further includes: Get the query text in the language input by the user; If it is determined that the query text needs to rewrite the wrong word or normalize the entity alias, the wrong word is rewritten or the entity alias is uniformly processed according to a predefined entity name normalization table to obtain a processed query text; Extracting entities and relationships from the processed query text to obtain key information, and converting the key information into a database query statement; A knowledge base query is executed according to the database query statement to obtain a knowledge query result.

4. The method according to claim 3, characterized in that The step of executing a knowledge base query according to the database query statement to obtain a knowledge query result includes: The knowledge query result and the query text are judged by the large language model. If the knowledge query result matches the query text, the knowledge query result is output and displayed through a visual interface.

5. An automatic knowledge base construction device, characterized in that: include: An acquisition module is configured to acquire an initial text in a large language model through a dialogue system of the large language model; a cleaning module configured to, when the initial text is not in the form of a question-answer pair, clean the initial text using a first prompt template pre-constructed in the large language model to obtain a first usable text; The step of using the first prompt template pre-constructed in the large language model to clean the initial text to obtain the first usable text includes: using the first prompt template to identify junk information in the initial text and filtering the junk information, wherein the junk information includes garbled characters, emoticons, hypertext tags, and continuous punctuation marks exceeding a preset number; using the first prompt template to perform grammar judgment on the initial text and correct incorrect grammar; using the first prompt template to perform content repetition judgment on the initial text, and if there is repetitive content, retaining the content that appears for the first time in the repetitive content; performing the cleaning step through the first prompt template to obtain the first usable text; The extraction module is configured to use the second prompt template pre-constructed in the large language model to perform structured knowledge extraction on the first available text to obtain initial information required for building a knowledge base; the use of the second prompt template pre-constructed in the large language model to perform structured knowledge extraction on the first available text to obtain initial information required for building a knowledge base includes: using the second prompt template to perform semantic understanding, entity classification, entity recognition, relationship extraction, entity linking and relationship disambiguation processing on the first available text to obtain the initial information, wherein the initial information includes triples; The construction module is configured to use a third prompt template pre-constructed in the large language model to perform knowledge fusion and verification on the initial information to obtain target information, and construct a knowledge base based on the target information; the use of the third prompt template pre-constructed in the large language model to perform knowledge fusion and verification on the initial information to obtain the target information includes: using the third prompt template to resolve conflicts for the same entities in each string of triples to obtain resolved initial information; and performing credibility verification on the resolved initial information to obtain the target information whose credibility score exceeds a threshold.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images