Data processing method and device based on knowledge base, equipment and storage medium

Abstract

The invention discloses a data processing method and device based on a knowledge base, equipment and a storage medium. The data processing method based on the knowledge base comprises the steps that knowledge extraction is conducted on an original knowledge file imported by a user, and an initial knowledge queue composed of at least two pieces of knowledge is obtained; performing missing value identification on each piece of knowledge in the initial knowledge queue, and performing knowledge content supplementation based on the identified missing values to obtain a supplemented knowledge queue; calculating the similarity between every two knowledge in the supplemented knowledge queue, and merging repeated knowledge according to the similarity to obtain a duplicate-removed knowledge queue; identifying invalid knowledge in the knowledge queue subjected to duplicate removal, and deleting the invalid knowledge to obtain an updated knowledge queue; and forwarding the knowledge contained in the updated knowledge queue to an auditing unit one by one, and in response to an auditing passing result fed back by the auditing unit, issuing the audited knowledge. According to the technical scheme, the data quality of knowledge is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, device, equipment and storage medium based on a knowledge base. Background Art

[0002] In the era of knowledge economy, the establishment of knowledge base is an important means to achieve enterprise resource sharing, promote corporate culture construction and mobilize employee enthusiasm. Knowledge management system continuously accumulates personal knowledge and enterprise knowledge through the acquisition, creation, integration, updating and innovation of knowledge.

[0003] Knowledge bases can help companies or individuals respond to market changes in a timely manner and make correct decisions, which places high demands on the accuracy of the knowledge contained in the knowledge base. Therefore, the processing of knowledge data is very important in the process of establishing a knowledge base. Summary of the invention

[0004] The present invention provides a data processing method, device, equipment and storage medium based on a knowledge base to solve the problem that the data quality of knowledge cannot be guaranteed during the knowledge base establishment process.

[0005] According to one aspect of the present invention, a data processing method based on a knowledge base is provided, comprising:

[0006] Extracting knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge;

[0007] Identify missing values ​​for each piece of knowledge in the initial knowledge queue, and supplement the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue;

[0008] Calculating the similarity between two pieces of knowledge in the supplemented knowledge queue, merging duplicate knowledge according to the similarity, and obtaining a deduplicated knowledge queue;

[0009] Identifying invalid knowledge in the deduplicated knowledge queue and deleting the invalid knowledge to obtain an updated knowledge queue;

[0010] The knowledge contained in the updated knowledge queue is forwarded to the review unit one by one, and in response to the review result fed back by the review unit, the knowledge that has passed the review is published.

[0011] According to another aspect of the present invention, there is provided a data processing device based on a knowledge base, comprising:

[0012] The initial queue acquisition module is used to extract knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge;

[0013] A missing value supplementation module is used to identify missing values ​​for each piece of knowledge in the initial knowledge queue, and supplement the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue;

[0014] A knowledge deduplication module is used to calculate the similarity between two pieces of knowledge in the supplemented knowledge queue, merge duplicate knowledge according to the similarity, and obtain a deduplicated knowledge queue;

[0015] An invalid knowledge deletion module, used for identifying invalid knowledge in the deduplicated knowledge queue and deleting the invalid knowledge to obtain an updated knowledge queue;

[0016] The knowledge review and release module is used to forward the knowledge contained in the updated knowledge queue to the review unit one by one, and release the reviewed knowledge in response to the review result fed back by the review unit.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the knowledge base-based data processing method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the knowledge base-based data processing method described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the knowledge base-based data processing method of any embodiment of the present disclosure is implemented.

[0023] The technical solution of the embodiment of the present invention performs knowledge extraction on the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge, and then identifies missing values ​​for each piece of knowledge in the initial knowledge queue, and supplements the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue, calculates the similarity between each piece of knowledge in the supplemented knowledge queue, merges duplicate knowledge according to the similarity to obtain a deduplicated knowledge queue, identifies invalid knowledge in the deduplicated knowledge queue, and deletes the invalid knowledge to obtain an updated knowledge queue, and finally forwards the knowledge contained in the updated knowledge queue to the audit unit one by one, and publishes the audited knowledge in response to the audit result fed back by the audit unit, which can improve the data quality of the knowledge and reduce the impact of unqualified data on users.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a data processing method based on a knowledge base provided according to the first embodiment of the present invention;

[0027] Figure 2 is a flow chart of a data processing method based on a knowledge base provided according to Embodiment 2 of the present invention;

[0028] Figure 3 is a schematic diagram of the structure of a data processing device based on a knowledge base provided according to Embodiment 3 of the present invention;

[0029] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the knowledge base-based data processing method of an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 A flowchart of a data processing method based on a knowledge base is provided for the first embodiment of the present invention. This embodiment is applicable to data cleaning during the process of establishing a knowledge base. The method can be executed by a data processing device based on a knowledge base. The data processing device based on a knowledge base can be implemented in the form of hardware and / or software. The data processing device based on a knowledge base can be configured in various general-purpose computing devices. Figure 1 As shown, the method includes:

[0034] S110 , extracting knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0035] Knowledge is used to provide users of the knowledge base with information related to individuals or companies. Each piece of knowledge can contain multiple fields, such as knowledge classification, knowledge name, answer, and similar questions. In addition, related entries can be set for each piece of knowledge, and users can query knowledge through entries.

[0036] The original knowledge file is a file containing knowledge-related content imported by the user, and the knowledge-related content can be extracted from the original knowledge file and combined into knowledge. Exemplarily, the original knowledge file is in Word document format, table format, or PDF format.

[0037] In an embodiment of the present invention, knowledge is extracted from an original knowledge file imported by a user to obtain an initial knowledge queue consisting of at least two pieces of knowledge. Specifically, the original knowledge file imported by the user can be text-processed to extract key information, and knowledge creation can be performed based on the key information. The initial knowledge queue is composed of at least one piece of created knowledge. Knowledge extraction can be performed by paragraph extraction, standard extraction, question-and-answer extraction, and subject-predicate-object (SPO) triple extraction. In the process of knowledge extraction, a natural language processing (NLP) model can be used to process the information in the original knowledge file.

[0038] In addition, when the original knowledge file is edited by the user according to the knowledge upload template, the field values ​​corresponding to each field can be extracted according to the fields pre-set in the template, and the knowledge can be assembled from the field values.

[0039] S120, identifying missing values ​​for each piece of knowledge in the initial knowledge queue, and supplementing the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue.

[0040] In an embodiment of the present invention, each piece of knowledge in the initial knowledge queue is compared to identify missing values, and knowledge content is supplemented based on the identified missing values. Specifically, it is possible to identify whether there are missing fields in each piece of knowledge based on pre-set knowledge fields, or to identify missing values ​​in knowledge through an NLP model. After identifying the missing values, the knowledge content can be supplemented by model prediction based on the context; the knowledge content can also be supplemented by pre-set specific text or characters; when the field to which the missing value belongs is not a pre-set key field, the missing value can be directly ignored or the content of the field to which the missing value belongs can be deleted. If there is a serial number in the knowledge content, the knowledge point content will be displayed in a line break according to the serial number. After completing the supplementation of all missing values ​​in the initial knowledge queue, the supplemented queue will be used as the supplemented knowledge queue.

[0041] S130, calculating the similarity between each pair of knowledge in the supplemented knowledge queue, merging duplicate knowledge according to the similarity, and obtaining a deduplicated knowledge queue.

[0042] In an embodiment of the present invention, the similarity between each pair of knowledge in the supplemented knowledge queue is calculated, and duplicate knowledge is merged according to the similarity to obtain a deduplicated knowledge queue. Specifically, firstly, the word vector corresponding to each piece of knowledge is obtained through One-hot encoding or word embedding model, for example, Word2Vec, and then based on the word vector, the similarity between each pair of knowledge in the supplemented knowledge queue is calculated. Finally, the knowledge whose similarity is greater than the set similarity threshold is merged as duplicate knowledge to obtain a deduplicated knowledge queue. Among them, the merging process includes merging both the title and the text of the knowledge.

[0043] S140, identifying invalid knowledge in the deduplicated knowledge queue, and deleting the invalid knowledge to obtain an updated knowledge queue.

[0044] In an embodiment of the present invention, after obtaining the deduplicated knowledge queue, the invalid knowledge in the deduplicated knowledge queue is further identified. Specifically, it is possible to identify whether there is a field missing in each piece of knowledge based on the pre-set knowledge field. When a key field is missing, the knowledge is determined to be invalid knowledge, and the invalid knowledge is deleted to obtain an updated knowledge queue. It is also possible to identify the illegal information contained in the knowledge. When the knowledge contains illegal information, the current knowledge is determined to be invalid knowledge.

[0045] In a specific example, knowledge includes fields such as knowledge classification, knowledge name, answer, and similar questions, among which the answer is preset as a key field. When it is detected that the key field is missing, the current knowledge is determined to be invalid knowledge.

[0046] S150, forwarding the knowledge contained in the updated knowledge queue to the review unit one by one, and publishing the reviewed knowledge in response to the review result fed back by the review unit.

[0047] In the embodiment of the present invention, after the data cleaning operations of supplementing missing values, deduplicating knowledge, and deleting invalid knowledge, in order to further ensure the data quality of knowledge in the knowledge base, the knowledge contained in the updated knowledge queue is forwarded to the review unit one by one. The reviewer will conduct knowledge review through the review unit, and the review results can be passed, failed, transferred, communicated, and completed. After selecting the review opinion of "passed", click Submit, the knowledge is reviewed and officially released; if "failed" is selected, the knowledge is returned, and the record will not be deleted after the return, and the initiator will be informed, and the approval can be initiated again after modification; transfer, select "transfer", the address book pops up, you can choose to transfer to other people to do, select the corresponding user, and the corresponding user has the review authority of the knowledge; communicate, select "communication", the address book pops up, you can choose to communicate with other people, select the corresponding user, and the corresponding user will also receive the knowledge review content and the results to be communicated, and the communication is completed, and the review can be passed; select "completed" means that it has been read, but no results are processed, and the current knowledge will not take effect after submission.

[0048] The technical solution of the embodiment of the present invention performs knowledge extraction on the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge, and then identifies missing values ​​for each piece of knowledge in the initial knowledge queue, and supplements the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue, calculates the similarity between each piece of knowledge in the supplemented knowledge queue, merges duplicate knowledge according to the similarity to obtain a deduplicated knowledge queue, identifies invalid knowledge in the deduplicated knowledge queue, and deletes the invalid knowledge to obtain an updated knowledge queue, and finally forwards the knowledge contained in the updated knowledge queue to the audit unit one by one, and publishes the audited knowledge in response to the audit result fed back by the audit unit, which can improve the data quality of the knowledge and reduce the impact of unqualified data on users.

[0049] Embodiment 2

[0050] Figure 2 This is a flowchart of a data processing method based on a knowledge base provided in the second embodiment of the present invention. This embodiment further refines the above embodiment and provides specific steps before identifying missing values ​​for each piece of knowledge in the initial knowledge queue, as well as specific steps for extracting knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge. Figure 2 As shown, the method includes:

[0051] S210: Extract knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0052] Optionally, the original knowledge file is a non-template knowledge file;

[0053] The original knowledge file imported by the user is extracted to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including:

[0054] A non-template knowledge file imported by a user is obtained, and knowledge is extracted using at least one of a paragraph extraction model, a standard extraction model, and a question-answer extraction model to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0055] A non-template knowledge file is a type of original knowledge file, which is a file containing knowledge-related content imported by the user. The knowledge-related content can be extracted from the original knowledge file and combined into knowledge. The user does not need to use a template when editing the original knowledge file. Exemplarily, the original knowledge file is in Word document format, PDF format, or text document format, etc.

[0056] In this optional embodiment, when the original knowledge file is a non-template knowledge file, a method is provided for performing knowledge extraction on the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge: the non-template knowledge file imported by the user is obtained, and at least one of a paragraph extraction model, a standard extraction model, and a question-and-answer extraction model is used to perform knowledge extraction to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0057] Optionally, the non-template knowledge file is in Word document format or Portable Document Format PDF.

[0058] In this optional embodiment, the non-template knowledge file is limited to a Word document format or a portable document format PDF.

[0059] Optionally, the original knowledge file is a template knowledge file edited based on the knowledge upload template;

[0060] The original knowledge file imported by the user is extracted to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including:

[0061] A template knowledge file imported by a user and edited based on a knowledge upload template is obtained, and knowledge is extracted from the template knowledge file according to the arrangement of knowledge information in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0062] The template knowledge file is another type of original knowledge file. It is a file containing knowledge-related content imported by the user. The knowledge-related content can be extracted from the original knowledge file and combined into knowledge. The user needs to use the knowledge upload template when editing the original knowledge file. Exemplarily, the original knowledge file is in a table Excel format.

[0063] Among them, the knowledge upload template can limit multiple fields associated with the knowledge, such as knowledge classification, knowledge name, answer, and similar questions.

[0064] In this optional embodiment, when the original knowledge file is a template knowledge file, a method is provided for extracting knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge: obtaining the template knowledge file imported by the user and edited based on the knowledge upload template, and extracting knowledge from the template knowledge file according to the arrangement of the knowledge information in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0065] In a specific example, the knowledge fields corresponding to each column in the Excel file are predefined to obtain a knowledge upload template. The user fills in the knowledge information on the knowledge upload template according to the correspondence between the knowledge fields and the columns to obtain a template knowledge file. After receiving the target knowledge file uploaded by the user, knowledge is extracted according to the correspondence between the columns and the knowledge fields in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0066] Optional, knowledge upload templates include concise templates and complete templates;

[0067] The concise template includes knowledge classification, knowledge name, answer, and similar question fields;

[0068] The complete template includes knowledge classification, knowledge name, answers, similar questions, text content, knowledge relationship, answer configuration and dynamic knowledge attribute fields.

[0069] In this optional embodiment, the knowledge upload template can be divided into a simple template and a complete template according to different knowledge upload requirements. The simple template includes knowledge classification, knowledge name, answer, and similar question fields; the complete template includes knowledge classification, knowledge name, answer, similar question, text content, knowledge relationship, answer configuration, and dynamic knowledge attribute fields.

[0070] Optionally, the template knowledge file is in a table format;

[0071] The template knowledge file imported by the user and edited based on the knowledge upload template is obtained, and knowledge is extracted from the template knowledge file according to the knowledge information arrangement in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including:

[0072] Obtain the template knowledge file imported by the user and edited based on the knowledge upload template, and extract each line of information in the template knowledge file in turn;

[0073] According to the correspondence between columns and fields in the knowledge upload template, field values ​​are extracted for each row of information in turn to obtain the knowledge corresponding to each row of information, and the knowledge corresponding to each row of information in the template knowledge file constitutes an initial knowledge queue.

[0074] In this optional embodiment, when the target knowledge file is in a table format, a method is provided for obtaining a template knowledge file imported by a user and edited based on a knowledge upload template, and extracting knowledge from the template knowledge file according to the arrangement of knowledge information in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge: first, obtaining a template knowledge file imported by a user and edited based on a knowledge upload template, and extracting each line of information in the template knowledge file in turn. Furthermore, according to the correspondence between columns and fields in the knowledge upload template, field values ​​are extracted from each line of information in turn to obtain the knowledge corresponding to each line of information, and the initial knowledge queue is formed by the knowledge corresponding to each line of information in the template knowledge file.

[0075] In a specific example, in the Excel knowledge upload template, the first column is predefined as the knowledge identification number, the second column is the knowledge classification, the third column is the knowledge name, the fourth column is the answer, and the fifth column is the similar question. Obtain the template knowledge file that is imported by the user and edited based on the knowledge upload template, and extract each row of information in the template knowledge file in turn. For each row of information, according to the correspondence between the columns and fields in the knowledge upload template, extract the field values ​​of each row of information in turn, combine the field values ​​of each column in the current row, and obtain the knowledge corresponding to the current row of information. Finally, the knowledge corresponding to each row of information in the template knowledge file constitutes the initial knowledge queue.

[0076] S220, sequentially obtain the knowledge name of each piece of knowledge in the initial knowledge queue as the current knowledge name, and compare the current knowledge name with the knowledge names of the stored knowledge.

[0077] In an embodiment of the present invention, there is a duplicate name check during the knowledge creation process, and the knowledge names under all fields cannot be repeated. Before the initial knowledge queue is cleaned, the knowledge name of each knowledge in the initial knowledge queue is obtained in turn as the current knowledge name, and the current knowledge name is compared with the knowledge name of the stored knowledge. Specifically, the current knowledge name can be word embedded to obtain the word vector corresponding to the current knowledge name. Based on the word vector and the word vector of the knowledge name in the stored knowledge, the similarity between the current knowledge name and the knowledge name of the stored knowledge is calculated. When the similarity between the two is 100%, a duplicate name is determined.

[0078] S230: When the current knowledge name is the same as the knowledge name of the stored knowledge, update the current knowledge name.

[0079] In an embodiment of the present invention, when the current knowledge name is the same as the knowledge name of the stored knowledge, the current knowledge name is updated. Specifically, the current knowledge name can be updated by adding preset characters to the current knowledge name; or a set number of characters can be extracted from the knowledge content and added to the current knowledge name to update the current knowledge name.

[0080] S240, identifying missing values ​​for each piece of knowledge in the knowledge queue, and supplementing the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue.

[0081] S250, calculating the similarity between each pair of knowledge in the supplemented knowledge queue, merging duplicate knowledge according to the similarity, and obtaining a deduplicated knowledge queue.

[0082] S260: Identify invalid knowledge in the deduplicated knowledge queue and delete the invalid knowledge to obtain an updated knowledge queue.

[0083] S270, forwarding the knowledge contained in the updated knowledge queue to the review unit one by one, and publishing the reviewed knowledge in response to the review result fed back by the review unit.

[0084] The technical solution of the embodiment of the present invention forwards the knowledge contained in the updated knowledge queue to the review unit for review one by one after the data cleaning operations of missing value supplementation, knowledge deduplication and invalid knowledge deletion, so as to further ensure the data quality of the knowledge in the knowledge base, and performs knowledge duplicate verification before data cleaning, so as to avoid the inconvenience caused by duplicate knowledge to the knowledge base user query process.

[0085] Embodiment 3

[0086] Figure 3 This is a schematic diagram of the structure of a data processing device based on a knowledge base provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0087] The initial queue acquisition module 310 is used to extract knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge;

[0088] A missing value supplementation module 320 is used to identify missing values ​​for each piece of knowledge in the initial knowledge queue, and supplement the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue;

[0089] The knowledge deduplication module 330 is used to calculate the similarity between two pieces of knowledge in the supplemented knowledge queue, merge duplicate knowledge according to the similarity, and obtain a deduplicated knowledge queue;

[0090] The invalid knowledge deletion module 340 is used to identify the invalid knowledge in the deduplicated knowledge queue and delete the invalid knowledge to obtain an updated knowledge queue;

[0091] The knowledge review and release module 350 is used to forward the knowledge contained in the updated knowledge queue to the review unit one by one, and to release the reviewed knowledge in response to the review result fed back by the review unit.

[0092] The technical solution of the embodiment of the present invention performs knowledge extraction on the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge, and then identifies missing values ​​for each piece of knowledge in the initial knowledge queue, and supplements the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue, calculates the similarity between each piece of knowledge in the supplemented knowledge queue, merges duplicate knowledge according to the similarity to obtain a deduplicated knowledge queue, identifies invalid knowledge in the deduplicated knowledge queue, and deletes the invalid knowledge to obtain an updated knowledge queue, and finally forwards the knowledge contained in the updated knowledge queue to the audit unit one by one, and publishes the audited knowledge in response to the audit result fed back by the audit unit, which can improve the data quality of the knowledge and reduce the impact of unqualified data on users.

[0093] Optionally, the original knowledge file is a non-template knowledge file;

[0094] The initial queue acquisition module 310 includes:

[0095] The first queue acquisition unit is used to acquire the non-template knowledge file imported by the user, and use at least one of the paragraph extraction model, the standard extraction model and the question-answer extraction model to extract knowledge, so as to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0096] Optionally, the original knowledge file is a template knowledge file edited based on a knowledge upload template;

[0097] The initial queue acquisition module 310 includes:

[0098] The second queue acquisition unit is used to acquire the template knowledge file imported by the user and edited based on the knowledge upload template, and extract knowledge from the template knowledge file according to the arrangement of knowledge information in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

[0099] Optionally, the template knowledge file is in a table format;

[0100] The second queue acquisition unit is specifically used for:

[0101] Obtain a template knowledge file that is imported by a user and edited based on a knowledge upload template, and extract each line of information in the template knowledge file in sequence;

[0102] According to the correspondence between columns and fields in the knowledge upload template, field values ​​are extracted for each row of information in turn to obtain the knowledge corresponding to each row of information, and the knowledge corresponding to each row of information in the template knowledge file constitutes an initial knowledge queue.

[0103] Optional, knowledge upload templates include concise templates and complete templates;

[0104] The concise template includes knowledge classification, knowledge name, answer and similar question fields;

[0105] The complete template includes knowledge classification, knowledge name, answer, similar question method, text content, knowledge relationship, answer configuration and dynamic knowledge attribute fields.

[0106] Optionally, the non-template knowledge file is in Word document format or Portable Document Format PDF.

[0107] Optionally, the data processing device based on the knowledge base further includes:

[0108] A knowledge name comparison module is used to obtain the knowledge name of each piece of knowledge in the initial knowledge queue in turn as the current knowledge name before identifying missing values ​​for each piece of knowledge in the initial knowledge queue, and compare the current knowledge name with the knowledge name of the stored knowledge;

[0109] The knowledge name updating module is used to update the current knowledge name when the current knowledge name is the same as the knowledge name of the stored knowledge.

[0110] The knowledge base-based data processing device provided in the embodiment of the present invention can execute the knowledge base-based data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0111] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0112] Embodiment 4

[0113] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0114] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, applicators, blade applicators, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0117] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a knowledge base-based data processing method.

[0118] In some embodiments, the data processing method based on the knowledge base can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data processing method based on the knowledge base described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the data processing method based on the knowledge base in any other appropriate manner (for example, by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or application.

[0121] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data applier), or a computing system that includes a middleware component (e.g., an application applier), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0124] A computing system may include a client and an application server. The client and the application server are generally remote from each other and usually interact through a communication network. The relationship between the client and the application server is generated by computer programs running on corresponding computers and having a client-application server relationship with each other. The application server may be a cloud application server, also known as a cloud computing application server or a cloud host, which is a host product in a cloud computing application system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS applications.

[0125] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0126] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method based on a knowledge base, characterized in that: include: Extracting knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge; Identify missing values ​​for each piece of knowledge in the initial knowledge queue, and supplement the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue; Calculating the similarity between two pieces of knowledge in the supplemented knowledge queue, merging duplicate knowledge according to the similarity, and obtaining a deduplicated knowledge queue; Identifying invalid knowledge in the deduplicated knowledge queue and deleting the invalid knowledge to obtain an updated knowledge queue; The knowledge contained in the updated knowledge queue is forwarded to the review unit one by one, and in response to the review result fed back by the review unit, the knowledge that has passed the review is published.

2. The method according to claim 1, characterized in that The original knowledge file is a non-template knowledge file; The original knowledge file imported by the user is extracted to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including: A non-template knowledge file imported by a user is obtained, and knowledge is extracted using at least one of a paragraph extraction model, a standard extraction model, and a question-answer extraction model to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

3. The method according to claim 1, characterized in that The original knowledge file is a template knowledge file edited based on the knowledge upload template; The original knowledge file imported by the user is extracted to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including: A template knowledge file imported by a user and edited based on a knowledge upload template is obtained, and knowledge is extracted from the template knowledge file according to the arrangement of knowledge information in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge.

4. The method according to claim 3, characterized in that The template knowledge file is in a table format; A template knowledge file imported by a user and edited based on a knowledge upload template is obtained, and knowledge is extracted from the template knowledge file according to the knowledge information arrangement in the knowledge upload template to obtain an initial knowledge queue consisting of at least two pieces of knowledge, including: Obtain a template knowledge file imported by a user and edited based on a knowledge upload template, and extract each line of information in the template knowledge file in sequence; According to the correspondence between columns and fields in the knowledge upload template, field values ​​are extracted for each row of information in turn to obtain the knowledge corresponding to each row of information, and the knowledge corresponding to each row of information in the template knowledge file constitutes an initial knowledge queue.

5. The method according to claim 3, characterized in that: The knowledge upload template includes a concise template and a complete template; The concise template includes knowledge classification, knowledge name, answer and similar question fields; The complete template includes knowledge classification, knowledge name, answer, similar question method, text content, knowledge relationship, answer configuration and dynamic knowledge attribute fields.

6. The method according to claim 2, characterized in that The non-template knowledge file is in Word document format or portable document format PDF.

7. The method according to claim 1, characterized in that Before identifying missing values ​​for each piece of knowledge in the initial knowledge queue, the method further includes: Obtaining the knowledge name of each piece of knowledge in the initial knowledge queue in turn as the current knowledge name, and comparing the current knowledge name with the knowledge name of the stored knowledge; When the current knowledge name is the same as the knowledge name of the stored knowledge, the current knowledge name is updated.

8. A data processing device based on a knowledge base, characterized in that: include: The initial queue acquisition module is used to extract knowledge from the original knowledge file imported by the user to obtain an initial knowledge queue consisting of at least two pieces of knowledge; A missing value supplementation module is used to identify missing values ​​for each piece of knowledge in the initial knowledge queue, and supplement the knowledge content based on the identified missing values ​​to obtain a supplemented knowledge queue; A knowledge deduplication module is used to calculate the similarity between two pieces of knowledge in the supplemented knowledge queue, merge duplicate knowledge according to the similarity, and obtain a deduplicated knowledge queue; An invalid knowledge deletion module, used for identifying invalid knowledge in the deduplicated knowledge queue and deleting the invalid knowledge to obtain an updated knowledge queue; The knowledge review and release module is used to forward the knowledge contained in the updated knowledge queue to the review unit one by one, and release the reviewed knowledge in response to the review result fed back by the review unit.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the knowledge-based data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the knowledge base-based data processing method according to any one of claims 1 to 7 when executed.

Images