Knowledge base processing method and device suitable for multiple industries and storage medium

Abstract

The invention discloses a knowledge base processing method and device suitable for multiple industries and a storage medium, and the method comprises the steps: receiving first target data, and carrying out the knowledge extraction of the first target data, thereby generating first knowledge corresponding to the first target data; according to the first knowledge, a knowledge base system is retrieved and judged whether second knowledge associated with the first knowledge exists in the knowledge base system or not, the knowledge base system comprises a plurality of knowledge bases, and the plurality of knowledge bases correspond to different industries; under the condition that the knowledge base system does not have the second knowledge, filling the first knowledge into a preset prompt template, and generating a corresponding first query text; generating a corresponding first SQL (Structured Query Language) instruction based on the first query text and a preset generative language model, and determining corresponding first business data; and taking the first business data as second knowledge, and updating the knowledge base system. Therefore, the technical effect of meeting data use requirements of different industries can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge base processing, and in particular to a knowledge base processing method, device and storage medium applicable to multiple industries. Background Art

[0002] In today's era of rapid digital development, various industries have an increasingly urgent need for accurate, efficient, and comprehensive knowledge management. For example, since different industries have different characteristics, they need knowledge base systems that correspond to their characteristics, so that knowledge models and management strategies can be flexibly customized for different industries. However, existing knowledge base systems often have problems such as insufficient adaptability to various industries and single application scenarios.

[0003] For another example, for different industries, the update and dynamic expansion of the knowledge base system can ensure the timeliness and comprehensiveness of the acquired knowledge. However, the existing technology often has the problem that the data in the knowledge base system is updated slowly and cannot meet the needs of different industries.

[0004] The publication number is CN104679850A, and the name is address structuring method and device. The address structuring method includes: step 10, dividing the address text into address word sequences; step 20, performing part-of-speech tagging on each address word in the address word sequence according to a predefined part-of-speech tagging set reflecting the characteristics of the address words; step 30, performing dependency syntactic analysis on the annotated address word sequence according to predefined address word dependency rules, taking entity address words as nodes and dependency relationships between entity address words as edges, and generating a dependency syntactic graph structure reflecting the address structure. The present invention also provides an address structuring device.

[0005] The publication number is CN109492214A, and the name is attribute word identification and its hierarchical construction method, device, equipment and storage medium. It includes: obtaining a noun word list of the comment corpus; obtaining a hypernymy matrix, which is a matrix of the hypernymy relationship between the seed word and the hyponymy of the seed word; determining the predicted hypernym vector of each word in the noun word list according to the hypernymy matrix and each word in the noun word list, the predicted hypernym vector is the product of the hypernymy matrix and the word vector of the word; determining the attribute word of the seed word according to the predicted hypernym vector of each word.

[0006] Currently, no effective solution has been proposed to the technical problem that the above-mentioned prior art cannot guarantee the timeliness and comprehensiveness of the knowledge base system, resulting in slow data updating in the existing knowledge base system and failure to meet the needs of different industries. Summary of the invention

[0007] The embodiments of the present disclosure provide a knowledge base processing system, method, device and storage medium applicable to multiple industries, so as to at least solve the technical problem existing in the prior art that the timeliness and comprehensiveness of the knowledge base system cannot be guaranteed, thereby causing the data in the existing knowledge base system to be updated slowly and unable to meet the needs of different industries.

[0008] According to one aspect of an embodiment of the present disclosure, a knowledge base processing method applicable to multiple industries is provided, including: receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; searching in a knowledge base system based on the first knowledge, and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries; in the case that the second knowledge does not exist in the knowledge base system, filling the first knowledge into a preset prompt template, and generating a corresponding first query text; generating a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determining the corresponding first business data; and using the first business data as the second knowledge, and updating the knowledge base system.

[0009] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0010] According to another aspect of the embodiment of the present disclosure, a knowledge base processing device applicable to multiple industries is also provided, including: a knowledge extraction module, used to receive first target data, and extract knowledge from the first target data, so as to generate first knowledge corresponding to the first target data; a first retrieval module, used to search in a knowledge base system according to the first knowledge, and determine whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries; a first enhancement module, used to fill the first knowledge into a preset prompt template and generate a corresponding first query text when the second knowledge does not exist in the knowledge base system; a first generation module, used to generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine the corresponding first business data; and a knowledge base update module, used to use the first business data as the second knowledge and update the knowledge base system.

[0011] According to another aspect of the embodiment of the present disclosure, a knowledge base processing device applicable to multiple industries is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; searching in a knowledge base system based on the first knowledge, and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries; in the case that the second knowledge does not exist in the knowledge base system, filling the first knowledge into a preset prompt template, and generating a corresponding first query text; generating a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determining the corresponding first business data; and using the first business data as the second knowledge, and updating the knowledge base system.

[0012] The present application discloses a knowledge base processing method applicable to multiple industries. And referring to the above-mentioned contents, it can be known that, when receiving the first target data, the present application pre-determines whether there is second knowledge corresponding to the first knowledge in the knowledge base system based on the first knowledge corresponding to the first target data, so as to determine whether the knowledge base system has been updated based on the above steps, and determine whether the knowledge base system can meet the usage requirements of different industries.

[0013] And when the second knowledge does not exist in the knowledge base system, the first knowledge and the pre-set generative language model are used to generate the corresponding first business data, and the first business data is used as the second knowledge to update the knowledge base system. Therefore, since the first business data contains knowledge associated with the first knowledge, the first business data can be used as the second knowledge to update the target knowledge base corresponding to the first knowledge. And since after the knowledge base system is updated, the target knowledge base in the knowledge base system adds the knowledge data corresponding to the industry, so when the knowledge data corresponding to the industry is needed in the future, the data usage needs can be met.

[0014] This can achieve the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system, and further ensure the data update in the knowledge base system to meet the data usage needs of different industries. This solves the technical problem that the timeliness and comprehensiveness of the knowledge base system cannot be guaranteed in the prior art, which makes the data update in the existing knowledge base system slow and unable to meet the needs of different industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings: Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present disclosure; Figure 2 is a schematic diagram of a knowledge base processing system applicable to multiple industries according to Example 1 of the present application; Figure 3 It is a flowchart of the knowledge base processing method applicable to multiple industries according to Example 1 of the present application; Figure 4 is a flow chart of a method for updating a knowledge base system using first target data according to Embodiment 1 of the present application; Figure 5 This is a flow chart of a method for generating second service data by using first user data according to Embodiment 1 of the present application; Figure 6 is a schematic diagram of a knowledge base processing device applicable to multiple industries according to Embodiment 2 of the present application; and Figure 7 This is a schematic diagram of a knowledge base processing device applicable to multiple industries as described in Example 3 of the present application. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0018] Example 1 According to this embodiment, a method embodiment of knowledge base processing applicable to multiple industries is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0019] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a knowledge base processing method applicable to multiple industries. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0020] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computing device. As involved in the embodiments of the present disclosure, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0021] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the knowledge base processing method applicable to multiple industries in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the knowledge base processing method applicable to multiple industries of the above-mentioned application program is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0022] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0023] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0024] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0025] Figure 2 Schematic diagram of a knowledge base processing system applicable to multiple industries according to this embodiment. Figure 2 As shown, the system includes: a knowledge base management system, a knowledge base system and an intelligent agent system. The knowledge base management system and the intelligent agent system are connected to each other in communication. The intelligent agent system and the knowledge base system are connected to each other in communication. The knowledge base management system and the knowledge base system are connected to each other in communication. The intelligent agent system and the user terminal are connected to each other in communication.

[0026] And wherein, a preprocessing module, a semantic extraction module, a classification module, a knowledge extraction module and a knowledge update module are provided in the knowledge base management system. The preprocessing module is used to perform data cleaning on the second target data and generate the third target data. The semantic extraction module is used to perform vectorization processing on the third target data, perform semantic extraction on the third target data after vectorization processing, and generate the first target data. The classification module is used to classify the first target data or the second user data, or to cluster multiple third data samples. The knowledge extraction module is used to extract knowledge from the first target data or each third data sample. The knowledge update module is used to use the first business data as the third knowledge and update the knowledge base system.

[0027] The knowledge base system includes multiple knowledge bases corresponding to different industries. For example, knowledge base 1 corresponds to the technology industry, knowledge base 2 corresponds to the service industry, and knowledge base 3 corresponds to the sales industry.

[0028] The intelligent agent system includes an interface module, a retrieval module, an enhancement module and a generation module. The intelligent agent system is connected to the user terminal through the interface module, or is connected to the knowledge base management system through the interface module. The retrieval module, the enhancement module and the generation module are used to interact with the generative language model, and generate a first SQL language instruction corresponding to the first query text through the operation of RAG. The generation module is also used to determine the corresponding first business data using the generated first SQL language instruction and the database corresponding to the generative language model.

[0029] It should be noted that the knowledge base management system, knowledge base system and intelligent agent system in the system can all be applied to the hardware structure described above.

[0030] Under the above operating environment, according to the first aspect of this embodiment, a knowledge base processing method applicable to multiple industries is provided. Figure 2 The knowledge base processing system shown in is implemented. Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes: S302: receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; S304: searching in a knowledge base system according to the first knowledge, and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes a plurality of knowledge bases, and the plurality of knowledge bases correspond to different industries; S306: When the second knowledge does not exist in the knowledge base system, the first knowledge is filled into a preset prompt template, and a corresponding first query text is generated; S308: Generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine the corresponding first business data; and S310: Using the first business data as second knowledge and updating the knowledge base system.

[0031] Specifically, in the embodiment of the present application, there are two different stages. That is, the knowledge base system construction stage and the knowledge base system update stage. Among them, the knowledge base system construction stage is used to indicate that multiple knowledge bases in the knowledge base system are constructed based on multiple first data samples. The knowledge base system update stage is used to indicate that the target knowledge base in the knowledge base system is updated based on the first target data. And after the knowledge base management system has constructed the knowledge base system, the knowledge base management system can further update the knowledge base system based on the received first target data.

[0032] Specifically, Figure 4 is a flow chart of a method for updating a knowledge base system using first target data according to an embodiment of the present application. Figure 4 As shown, first, the knowledge base management system receives the second target data, and uses the preprocessing module to perform data cleaning on the second target data, thereby generating third target data.

[0033] And since the structured data with industry category labels can already determine its semantic information and the corresponding industry category, when the second target data is structured data and has industry category labels, the knowledge extraction module vectorizes the third target data and uses the vectorized third target data as the first target data. The semantic extraction module is then used to extract knowledge from the first target data, thereby generating first knowledge corresponding to the first target data.

[0034] Since the structured data without industry category labels cannot determine its semantic information and corresponding industry category, when the second target data is unstructured data and does not have industry category labels, the knowledge base management system needs to use the semantic extraction module to vectorize and semantically extract the third target data, and use the classification module to determine the industry category before it can extract knowledge from the first target data, thereby generating the first knowledge corresponding to the first target data (S302). It is worth noting that the first target data includes the third target data after vectorization and the third target data before vectorization. For example, if the third target data is a natural language text, the first target data includes a vector corresponding to the natural language text and also includes the natural language text. And the above is just an example of the first target data being a natural language text. It should be clear to those skilled in the art that the first target data can also be a picture or a table, etc. The above content will be described in detail later, so it will not be repeated here.

[0035] When the knowledge extraction module generates the corresponding first knowledge based on the first target data, the first knowledge is further sent to the retrieval module in the intelligent agent system through the interface module in the intelligent agent system.

[0036] When the search module receives the first knowledge, it searches the knowledge base system according to the first knowledge and determines whether the knowledge base system has corresponding second knowledge (S304). The knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries. Figure 4 As shown, for example, knowledge base 1 corresponds to the technology industry; knowledge base 2 corresponds to the service industry; and knowledge base 3 corresponds to the sales industry.

[0037] For another example, when the retrieval module in the intelligent system receives the first knowledge corresponding to the first target data, the retrieval module determines the corresponding target knowledge base based on its industry category. Then, the retrieval module determines whether there is second knowledge corresponding to the first knowledge in the target knowledge base. If there is second knowledge corresponding to the first knowledge in the target knowledge base, it means that the knowledge in the knowledge base system is relatively rich and can meet the query requirements; if there is no second knowledge corresponding to the first knowledge in the target knowledge base, it means that the corresponding knowledge is missing in the knowledge base system, and the knowledge base system needs to be further updated to ensure that the knowledge in the knowledge base system can meet industry needs and ensure the real-time and comprehensiveness of the knowledge base system update.

[0038] And wherein, the second knowledge includes but is not limited to information related to the concepts, events or objects contained in the first knowledge. For example, the first knowledge is "neural network models include convolutional neural network models, residual network models, RNN models, and LSTM models." Then the second knowledge corresponding to the first knowledge can be "convolutional neural network models and residual network models are feedforward neural network models, and RNN models and LSTM models are feedback neural network models." It should be noted that those skilled in the art are only giving examples of the first knowledge and the second knowledge, and the actual situation is not limited to this.

[0039] In the knowledge base system, when the target knowledge base corresponding to the first target data does not contain the second knowledge, the enhancement module fills the first knowledge into the preset prompt template and generates the corresponding first query text (S306). For example, the preset prompt template may be "first knowledge + what other related information is included according to the above example". Thus, the enhancement module fills the first knowledge into the above prompt template and generates "neural network models include convolutional neural network models, residual network models, RNN models, LSTM models, and what other related information is included according to the above example" (ie, the first query text).

[0040] Furthermore, the generation module inputs the first query text into a pre-trained generative language model and generates a corresponding first SQL language instruction. The generation module then inputs the first SQL language instruction into a business database corresponding to the generative language model for query, thereby obtaining first business data corresponding to the first SQL language instruction (S308).

[0041] Afterwards, when the generation module determines the first business data corresponding to the second knowledge, the first business data is returned to the knowledge extraction module of the knowledge base management system through the interface module. And the knowledge extraction module sends the first business data to the knowledge update module. Thus, when the knowledge update module receives the first business data, the first business data is used as the second knowledge, and the target knowledge base in the knowledge base system is updated using the second knowledge (S310).

[0042] As described in the background technology, in today's era of rapid digital development, various industries have an increasingly urgent need for accurate, efficient, and comprehensive knowledge management. For example, since different industries have different characteristics, a knowledge base system corresponding to their characteristics is required, so that knowledge models and management strategies can be flexibly customized for different industries. However, existing knowledge base systems often have problems such as insufficient adaptability to various industries and single application scenarios.

[0043] For another example, for different industries, the update and dynamic expansion of the knowledge base system can ensure the timeliness and comprehensiveness of the acquired knowledge. However, in the prior art, the data in the knowledge base system is often updated slowly and cannot meet the needs of different industries.

[0044] In view of this, the present application provides a knowledge base processing method applicable to multiple industries. And referring to the above-mentioned content, it can be known that, when receiving the first target data, the present application pre-determines whether there is second knowledge corresponding to the first knowledge in the knowledge base system based on the first knowledge corresponding to the first target data, so as to determine whether the knowledge base system has been updated based on the above steps, and determine whether the knowledge base system can meet the usage requirements of different industries.

[0045] And when the second knowledge does not exist in the knowledge base system, the first knowledge and the pre-set generative language model are used to generate the corresponding first business data, and the first business data is used as the second knowledge to update the knowledge base system. Therefore, since the first business data contains knowledge associated with the first knowledge, the first business data can be used as the second knowledge to update the target knowledge base corresponding to the first knowledge. And since after the knowledge base system is updated, the target knowledge base in the knowledge base system adds the knowledge data corresponding to the industry, so when the knowledge data corresponding to the industry is needed in the future, the data usage needs can be met.

[0046] This can achieve the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system, and further ensure the data update in the knowledge base system to meet the data usage needs of different industries. This solves the technical problem that the timeliness and comprehensiveness of the knowledge base system cannot be guaranteed in the prior art, which makes the data update in the existing knowledge base system slow and unable to meet the needs of different industries.

[0047] Optionally, when the second target data is structured data and carries an industry category label, the method also includes: receiving the second target data and performing data cleaning on the second target data to generate third target data; and vectorizing the third target data and using the vectorized third target data as the first target data to extract knowledge from the first target data.

[0048] Specifically, refer to Figure 4As shown, before the knowledge base management system uses the knowledge extraction module to extract knowledge from the first target data, it is necessary to predetermine the data type of the second target data, and perform data processing on the second target data according to the data type of the second target data. Thus, firstly, the knowledge base management system receives the second target data, and uses the preprocessing module to perform data cleaning on the second target data, thereby generating cleaned second target data (i.e., third target data). Thus, data cleaning of the second target data can achieve the technical effect of improving data quality and ensuring analysis accuracy.

[0049] Afterwards, when the second target data corresponding to the third target data is structured data and has an industry category label, the knowledge base management system uses the knowledge extraction module to vectorize the third target data, uses the vectorized third target data as the first target data, and uses the knowledge extraction module to extract knowledge from the first target data, thereby generating the first knowledge corresponding to the first target data. For example, if the second target data is short text data, has a high degree of structure, and the knowledge base management system can determine the industry category corresponding to the short text data, then after the third target data corresponding to the second target data is vectorized, the knowledge extraction module is directly used to extract knowledge from the vectorized third target data (i.e., the first target data).

[0050] For another example, if the second target data is tabular data, each column has a specific meaning and the relationship between the data can be directly seen. In addition, the tabular data carries an industry category label, and the knowledge base management system can determine the corresponding industry category based on the industry category label. Then, after vectorizing the third target data corresponding to the second target data, the knowledge extraction module is directly used to extract knowledge from the vectorized third target data (i.e., the first target data).

[0051] Optionally, when the second target data is unstructured data and does not carry an industry category label, the method also includes: receiving the second target data and performing data cleaning on the second target data to generate third target data; vectorizing the third target data and performing semantic extraction on the vectorized third target data to generate first target data; determining the industry category corresponding to the first target data and labeling the first target data with a corresponding industry category label; and performing knowledge extraction on the first target data.

[0052] Specifically, refer to Figure 4As shown, first, the knowledge base management system receives the second target data, and uses the preprocessing module to clean the second target data, thereby generating cleaned second target data (i.e., third target data). Thus, cleaning the second target data can achieve the technical effect of improving data quality and ensuring analysis accuracy.

[0053] Afterwards, when the second target data corresponding to the third target data is unstructured data and does not carry an industry category label, the knowledge base management system uses a semantic extraction module to vectorize the third target data and performs semantic extraction on the vectorized third target data to generate the first target data.

[0054] Furthermore, the knowledge base management system uses the classification module to determine the industry category corresponding to the first target data and tags the first target data with the corresponding industry category label. For example, the classification module determines the industry category corresponding to the first target data by determining the similarity between vectors.

[0055] Finally, the knowledge extraction module performs knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data.

[0056] Thus, by performing different data processing operations on the second target data based on the data type of the second target data (i.e., the second target data is structured data with an industry category label, or the second target data is unstructured data without an industry category label), a technical effect is achieved that is able to save the time required for extracting knowledge to the greatest extent while ensuring knowledge extraction from the second target data.

[0057] Optionally, the operation of taking the first business data as the second knowledge and updating the knowledge base system includes: determining the target knowledge base corresponding to the first target data based on the industry category label corresponding to the first target data; and when the target knowledge base corresponding to the first target data is determined, updating the target knowledge base using the second knowledge.

[0058] Specifically, referring to the above content, it can be known that no matter the second target data is structured data with an industry category label, or the second target data is unstructured data without an industry category label, the first target data received by the final knowledge extraction module has determined the corresponding industry category.

[0059] Therefore, when the knowledge extraction module receives the first business data corresponding to the first target data, it can determine the target knowledge base corresponding to the industry category based on the predetermined industry category corresponding to the first target data.

[0060] Finally, when the knowledge updating module determines the target knowledge base corresponding to the first business data, the first business data is used as the second knowledge, and the target knowledge base is updated using the second knowledge.

[0061] Therefore, even if there is no second knowledge corresponding to the first target data in the knowledge base system, the RAG operation can be used to generate the corresponding second knowledge using the first knowledge, and then the second knowledge can be used to update the knowledge base system, thereby ensuring the real-time and comprehensiveness of the knowledge base system update.

[0062] Optionally, it also includes: constructing a knowledge base system, wherein the operation of constructing the knowledge base system includes: receiving multiple first data samples, and performing data cleaning on the multiple first data samples to generate multiple second data samples; performing vectorization processing on the multiple second data samples, and performing semantic extraction on the vectorized multiple second data samples to generate corresponding multiple third data samples; using a clustering algorithm to determine the industry category corresponding to each third data sample, and generating corresponding industry category labels; performing knowledge extraction on each third data sample respectively to generate corresponding multiple fourth knowledge, and marking each fourth knowledge with a corresponding industry category label; and constructing a knowledge base system based on the multiple fourth knowledge.

[0063] Specifically, Figure 4 The content of the flowchart shown is a flowchart corresponding to the knowledge base system update phase, but the multiple knowledge bases in the knowledge base system can only be updated when the multiple knowledge bases in the knowledge base system are constructed.

[0064] So first, refer to Figure 4 As shown, the knowledge base management system receives multiple first data samples and uses a preprocessing module to clean the multiple first data samples, thereby generating cleaned first data samples (ie, multiple second data samples). The multiple first data samples may be, for example, multi-source heterogeneous data.

[0065] Afterwards, the knowledge base management system uses the semantic extraction module to vectorize the multiple second data samples, and performs semantic extraction on the multiple second data samples after vectorization, thereby generating corresponding multiple third data samples. Among them, the semantic extraction module can, for example, use the BERT model or the XLNet model to perform semantic extraction on the second data samples, which will not be repeated here. And it is worth noting that the above-mentioned third data sample includes the second data sample after vectorization and the second data sample before vectorization. For example, the second data sample is a natural language text, and the natural language text is "The main function of the service industry is service", then the third data sample includes the natural language text "The main function of the service industry is service" and the corresponding vector.

[0066] Furthermore, in the case that the first data sample received by the knowledge base management system is a multi-source heterogeneous data sample corresponding to multiple different industries, the knowledge base management system can utilize a classification module and cluster the third data samples corresponding to the multiple first data samples through a clustering algorithm, thereby determining the industry category corresponding to each third data sample and generating corresponding industry category labels.

[0067] Afterwards, the knowledge base management system uses the knowledge extraction module to extract knowledge from each third data sample, thereby generating corresponding fourth knowledge, and labels each fourth knowledge with a corresponding industry category label. The knowledge extraction model includes but is not limited to methods that can use statistical models or methods based on deep learning to extract knowledge from each third data sample. It should be clear to those skilled in the art that any method that can generate fourth knowledge based on the third data sample is not limited here.

[0068] Finally, the knowledge update module constructs multiple knowledge bases based on multiple fourth knowledge and industry category labels corresponding to each fourth knowledge. Among them, knowledge base 1 corresponds to the technology industry, and the industry category label of the fourth knowledge included in knowledge base 1 is "technology industry"; knowledge base 2 corresponds to the service industry, and the industry category label of the fourth knowledge included in knowledge base 2 is "service industry"; knowledge base 3 corresponds to the sales industry, and the industry category label of the fourth knowledge included in knowledge base 3 is "sales industry".

[0069] Thus, the knowledge base management system achieves the technical effect of providing the necessary foundation and corresponding guarantee for subsequent industries to acquire relevant knowledge by utilizing the received multiple first data samples to construct the operation of the knowledge base system.

[0070] Optionally, it also includes: receiving the first user data corresponding to the user terminal, and performing vectorization processing and semantic extraction on the first user data to generate the second user data; and classifying the second user data and determining the industry category corresponding to the second user data to generate the third user data, wherein the third user data includes the second user data and the industry category label; and based on the third user data, acquiring and returning the corresponding first business data to the user terminal. Further optionally, based on the third user data, the operation of acquiring and returning the corresponding second business data to the user terminal includes: searching in the knowledge base system according to the third user data, and determining whether the knowledge base system has the corresponding fifth knowledge; if the knowledge base system does not have the fifth knowledge, filling the third user data into the pre-set prompt template and generating the corresponding second query text; generating the corresponding second SQL language instruction based on the second query text, and generating the corresponding second business data using the generative language model and the second SQL language instruction; and returning the second business data to the user terminal.

[0071] Specifically, Figure 5 is a flow chart of a method for generating second service data by using first user data according to an embodiment of the present application. Figure 5 As shown, the knowledge base management system and the intelligent agent system can not only update the target knowledge base in the knowledge base system, but also generate corresponding second business data based on the first user data sent by the user terminal.

[0072] First, the interface module of the agent system receives the first user data sent by the user terminal, and then the interface module sends the first user data to the semantic extraction module of the knowledge base management system, and uses the semantic extraction module to vectorize and extract semantics from the first user data, thereby generating corresponding second user data. It is worth noting that the second user data includes the first user data and the vector corresponding to the first user data.

[0073] Then, the knowledge base management system uses the classification module and the clustering algorithm to classify the second user data, thereby further determining the industry category corresponding to the second user data and generating the third user data. The third user data includes the second user data and the corresponding industry category label. For example, the second user data is "what aspects does the service industry mainly include + the corresponding vector", so the third user data is "what aspects does the service industry mainly include + the service industry + the corresponding vector".

[0074] Furthermore, the classification module sends the third user data to the retrieval module through the interface module in the agent system. The retrieval module then searches the knowledge base system based on the third user data and determines whether the corresponding fifth knowledge exists in multiple knowledge bases in the knowledge base system. If the retrieval module determines that the fifth knowledge exists in the knowledge base system, the fifth knowledge is used as the second business data and returned to the user terminal through the interface module.

[0075] When the search module determines that the corresponding fifth knowledge does not exist in the knowledge base system, the enhancement module fills the third user data into the preset prompt template and generates the corresponding second query text. For example, the preset prompt template may be "third user data + what other related information is included according to the above example". Thus, the enhancement module fills the third user data into the above prompt template and generates "what aspects of the service industry are included + what other related information is included according to the above example" (i.e., the second query text).

[0076] Furthermore, the generation module inputs the second query text into a pre-trained generative language model and generates a corresponding second SQL language instruction. The generation module then inputs the second SQL language instruction into a business database corresponding to the generative language model for query, thereby obtaining second business data corresponding to the second SQL language instruction.

[0077] Finally, when the generation module generates the second service data, the second service data is sent to the interface module, and the second service data is returned to the user terminal through the interface module, so that the user can determine the second service data corresponding to the first user data through the user terminal.

[0078] Therefore, referring to the above-mentioned contents, it can be known that the present application can not only update the knowledge data in the knowledge base system, but also query the first user data corresponding to the user terminal, thereby meeting the user's query needs.

[0079] Therefore, according to the first aspect of this embodiment, the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system and thus meeting the data query needs of different industries is achieved.

[0080] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0081] Therefore, according to this embodiment, the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system and satisfying the data query needs of different industries is achieved.

[0082] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0084] Example 2 Figure 6 FIG. 6 shows a knowledge base processing device 600 applicable to multiple industries according to this embodiment, and the device 600 corresponds to the method according to embodiment 1. Figure 6 As shown, the device 600 includes: a knowledge extraction module 610, which is used to receive first target data and extract knowledge from the first target data, so as to generate first knowledge corresponding to the first target data; a first retrieval module 620, which is used to search in the knowledge base system according to the first knowledge, and determine whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries; a first enhancement module 630, which is used to fill the first knowledge into a preset prompt template when the second knowledge does not exist in the knowledge base system, and generate a corresponding first query text; a first generation module 640, which is used to generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine the corresponding first business data; and a knowledge base update module 650, which is used to use the first business data as the second knowledge and update the knowledge base system.

[0085] Optionally, when the second target data is structured data and carries an industry category label, the device 600 also includes: a first data cleaning module, used to receive the second target data and perform data cleaning on the second target data to generate third target data; and a first knowledge extraction submodule, used to vectorize the third target data when the second target data is structured data and carries an industry category label, use the vectorized third target data as the first target data, and perform knowledge extraction.

[0086] Optionally, when the second target data is unstructured data and does not carry an industry category label, the device 600 also includes: a second data cleaning module, used to receive the second target data and perform data cleaning on the second target data to generate third target data; a first semantic extraction module, used to perform vectorization processing on the third target data and perform semantic extraction on the vectorized third target data to generate first target data; a first classification module, used to determine the industry category corresponding to the first target data and label the first target data with a corresponding industry category label; and a second knowledge extraction submodule, used to perform knowledge extraction on the first target data.

[0087] Optionally, the knowledge base update module 650 includes: a target knowledge base determination module, which is used to determine the target knowledge base corresponding to the first target data based on the industry category label corresponding to the first target data; and a knowledge base update sub-module, which is used to update the target knowledge base using the second knowledge when the target knowledge base corresponding to the first target data is determined.

[0088] Optionally, the device 600 also includes: a knowledge base system construction module, which is used to construct a knowledge base system, wherein the knowledge base system construction module includes: a third data cleaning module, which is used to receive multiple first data samples, and perform data cleaning on the multiple first data samples, so as to generate multiple second data samples; a second semantic extraction module, which is used to vectorize the multiple second data samples, and perform semantic extraction on the vectorized multiple second data samples, so as to generate corresponding multiple third data samples; a second classification module, which is used to use a clustering algorithm to determine the industry category corresponding to each third data sample, and generate a corresponding industry category label; a third knowledge extraction sub-module, which is used to perform knowledge extraction on each third data sample respectively, generate a corresponding multiple fourth knowledge, and label each fourth knowledge with a corresponding industry category label; and a knowledge base system construction sub-module, which is used to construct a knowledge base system based on multiple fourth knowledge.

[0089] Optionally, the device 600 also includes: a third semantic extraction module, which is used to receive first user data corresponding to the user terminal, and perform vectorization processing and semantic extraction on the first user data to generate second user data; and a third classification module, which is used to classify the second user data and determine the industry category corresponding to the second user data to generate third user data, wherein the third user data includes the second user data and the industry category label; and a business data return module, which is used to obtain and return the corresponding second business data to the user terminal based on the third user data.

[0090] Optionally, the business data return module includes: a second retrieval module, used to search in the knowledge base system according to the third user data, and determine whether the knowledge base system has corresponding fifth knowledge; a second enhancement module, used to fill the third user data into a preset prompt template when the fifth knowledge does not exist in the knowledge base system, and generate a corresponding second query text; a second generation module, used to generate a corresponding second SQL language instruction based on the second query text and a preset generative language model, and determine the corresponding second business data; and a business data return sub-module, used to return the second business data to the user terminal.

[0091] Therefore, according to this embodiment, the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system and satisfying the data query needs of different industries is achieved.

[0092] Example 3 Figure 7 FIG. 7 shows a knowledge base processing device 700 applicable to multiple industries according to this embodiment, and the device 700 corresponds to the method according to embodiment 1. Figure 7 As shown, the device 700 includes: a processor 710; and a memory 720, which is connected to the processor 710 and is used to provide the processor 710 with instructions for processing the following processing steps: receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; searching in a knowledge base system based on the first knowledge, and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes multiple knowledge bases, and the multiple knowledge bases correspond to different industries; in the case that the knowledge base system does not have the second knowledge, filling the first knowledge into a preset prompt template and generating a corresponding first query text; generating a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determining the corresponding first business data; and using the first business data as the second knowledge and updating the knowledge base system.

[0093] Optionally, when the second target data is structured data and carries an industry category label, the device 700 also includes: receiving the second target data and performing data cleaning on the second target data to generate third target data; and vectorizing the third target data and using the vectorized third target data as the first target data, thereby extracting knowledge from the first target data.

[0094] Optionally, when the second target data is unstructured data and does not carry an industry category label, the device 700 also includes: receiving the second target data and performing data cleaning on the second target data to generate third target data; performing vectorization processing on the third target data and performing semantic extraction on the vectorized third target data to generate first target data; determining the industry category corresponding to the first target data and marking the first target data with a corresponding industry category label; and performing knowledge extraction on the first target data.

[0095] Optionally, the operation of taking the first business data as the second knowledge and updating the knowledge base system includes: determining the target knowledge base corresponding to the first target data based on the industry category label corresponding to the first target data; and when the target knowledge base corresponding to the first target data is determined, updating the target knowledge base using the second knowledge.

[0096] Optionally, the device 700 also includes: constructing a knowledge base system, wherein the operation of constructing the knowledge base system includes: receiving multiple first data samples, and performing data cleaning on the multiple first data samples to generate multiple second data samples; performing vectorization processing on the multiple second data samples, and performing semantic extraction on the vectorized multiple second data samples to generate corresponding multiple third data samples; using a clustering algorithm to determine the industry category corresponding to each third data sample, and generating a corresponding industry category label; performing knowledge extraction on each third data sample respectively to generate a corresponding multiple fourth knowledge, and marking each fourth knowledge with a corresponding industry category label; and constructing a knowledge base system based on the multiple fourth knowledge.

[0097] Optionally, the device 700 also includes: receiving first user data corresponding to the user terminal, and performing vectorization processing and semantic extraction on the first user data to generate second user data; and classifying the second user data and determining the industry category corresponding to the second user data to generate third user data, wherein the third user data includes the second user data and an industry category label; and based on the third user data, obtaining and returning the corresponding second business data to the user terminal.

[0098] Optionally, the operation of acquiring and returning corresponding second business data to the user terminal based on the third user data includes: searching in the knowledge base system according to the third user data, and determining whether the knowledge base system has corresponding fifth knowledge; if the fifth knowledge does not exist in the knowledge base system, filling the third user data into a preset prompt template and generating a corresponding second query text; generating a corresponding second SQL language instruction based on the second query text and a preset generative language model, and determining the corresponding second business data; and returning the second business data to the user terminal.

[0099] Therefore, according to this embodiment, the technical effect of ensuring the timeliness and comprehensiveness of the knowledge base system and satisfying the data query needs of different industries is achieved.

[0100] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0101] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0106] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A knowledge base processing method applicable to multiple industries, characterized in that: include: Receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; According to the first knowledge, searching in a knowledge base system and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes a plurality of knowledge bases, and the plurality of knowledge bases correspond to different industries; In the case that the second knowledge does not exist in the knowledge base system, the first knowledge is filled into a preset prompt template, and a corresponding first query text is generated; Generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine the corresponding first business data; as well as The first business data is used as the second knowledge, and the knowledge base system is updated.

2. The method according to claim 1, characterized in that When the second target data is structured data and has industry category labels, the method further includes: receiving second target data, and performing data cleaning on the second target data to generate third target data; and The third target data is vectorized, and the vectorized third target data is used as the first target data, thereby performing knowledge extraction on the first target data.

3. The method according to claim 2, characterized in that When the second target data is unstructured data and does not carry an industry category label, the method further includes: receiving the second target data, and performing data cleaning on the second target data, thereby generating third target data; performing vectorization processing on the third target data, and performing semantic extraction on the vectorized third target data, thereby generating the first target data; Determine an industry category corresponding to the first target data, and label the first target data with a corresponding industry category label; and Perform knowledge extraction on the first target data.

4. The method according to claim 2 or 3, characterized in that: The operation of using the first business data as second knowledge and updating the knowledge base system includes: Determining a target knowledge base corresponding to the first target data based on the industry category label corresponding to the first target data; and When a target knowledge base corresponding to the first target data is determined, the target knowledge base is updated using the second knowledge.

5. The method according to claim 1, characterized in that Also includes: Constructing a knowledge base system, wherein the operations of constructing the knowledge base system include: Receiving a plurality of first data samples, and performing data cleaning on the plurality of first data samples, thereby generating a plurality of second data samples; Performing vectorization processing on the plurality of second data samples, and performing semantic extraction on the vectorized plurality of second data samples, thereby generating a corresponding plurality of third data samples; Determine the industry category corresponding to each third data sample by using a clustering algorithm, and generate a corresponding industry category label; Performing knowledge extraction on each of the third data samples to generate corresponding plurality of fourth knowledge, and labeling each of the fourth knowledge with a corresponding industry category label; and Based on the plurality of fourth knowledge, a knowledge base system is constructed.

6. The method according to claim 1, characterized in that Also includes: Receiving first user data corresponding to a user terminal, and performing vectorization processing and semantic extraction on the first user data, thereby generating second user data; as well as Classifying the second user data and determining an industry category corresponding to the second user data, thereby generating third user data, wherein the third user data includes the second user data and an industry category label; as well as Based on the third user data, corresponding second service data is acquired and returned to the user terminal.

7. The method according to claim 6, characterized in that The operation of acquiring and returning corresponding second service data to the user terminal based on the third user data includes: According to the third user data, searching in the knowledge base system, and determining whether the knowledge base system has corresponding fifth knowledge; In the case that the fifth knowledge does not exist in the knowledge base system, filling the third user data into a preset prompt template and generating a corresponding second query text; Generate a corresponding second SQL language instruction based on the second query text and a preset generative language model, and determine the corresponding second business data; and The second service data is returned to the user terminal.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 5.

9. A knowledge base processing device applicable to multiple industries, characterized in that: include: A knowledge extraction module, configured to receive first target data and perform knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; A first retrieval module is used to search in a knowledge base system according to the first knowledge, and determine whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes a plurality of knowledge bases, and the plurality of knowledge bases correspond to different industries; A first enhancement module, configured to fill the first knowledge into a preset prompt template and generate a corresponding first query text if the second knowledge does not exist in the knowledge base system; A first generating module, configured to generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine corresponding first business data; as well as A knowledge base updating module is used to use the first business data as the second knowledge and update the knowledge base system.

10. A knowledge base processing device applicable to multiple industries, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Receiving first target data, and performing knowledge extraction on the first target data, thereby generating first knowledge corresponding to the first target data; According to the first knowledge, searching in a knowledge base system and determining whether the knowledge base system has second knowledge associated with the first knowledge, wherein the knowledge base system includes a plurality of knowledge bases, and the plurality of knowledge bases correspond to different industries; In the case that the second knowledge does not exist in the knowledge base system, the first knowledge is filled into a preset prompt template, and a corresponding first query text is generated; Generate a corresponding first SQL language instruction based on the first query text and a preset generative language model, and determine the corresponding first business data; as well as The first business data is used as the second knowledge, and the knowledge base system is updated.

Images