Question and answer knowledge base system knowledge processing method, device, equipment and medium

Abstract

The present application relates to the technical field of natural language processing, in particular to a knowledge processing method and device of a question and answer knowledge base system, equipment and medium, the method comprising: acquiring all knowledge contained in the question and answer knowledge base system, and performing fragmentation processing on the acquired knowledge to obtain a plurality of knowledge fragments; using a preset text similarity algorithm to calculate the similarity between each two knowledge fragments, and comparing each calculated similarity with a preset similarity threshold; determining any similarity greater than the similarity threshold as a target similarity, and placing the two knowledge fragments based on which the target similarity is calculated into the same knowledge block, thereby realizing block processing of the knowledge. The present application can make the correlation of the fragments contained in each block higher, and the information in each block as complete as possible, thereby effectively improving the efficiency, accuracy and comprehensiveness of the search.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a knowledge processing method, apparatus, equipment, and medium for a question-and-answer knowledge base system. Background Technology

[0002] A question-answering knowledge base system is a natural language processing technology based on a knowledge base. It can receive natural language questions from users and search the knowledge base for answers that meet the user's needs.

[0003] Existing search implementations are typically based on knowledge fragmentation, chunking, and vectorization. Specifically, fragmentation breaks down the original knowledge base into several independent, shorter knowledge points. Each knowledge point serves as the minimum record for a question and answer, matching the answer to the question. Chunking has a broader scope; for example, regulations on cross-regional medical insurance include several detailed rules, such as the reimbursement process, reimbursement ratio, reimbursement scope, and required documents. Assuming these detailed rules are the minimum knowledge points, cross-regional medical insurance is summarized into a large chunk, which is provided along with the question when the user asks about cross-regional medical insurance.

[0004] In traditional question-answering knowledge base systems, knowledge is broken down into fixed blocks. If the blocks are too small, the answers may be incomplete; if the blocks are too large, they may contain redundant information, affecting search efficiency and accuracy. Summary of the Invention

[0005] In view of this, the present invention provides a knowledge processing method, apparatus, device and medium for a question-and-answer knowledge base system, which can effectively improve the efficiency, accuracy and comprehensiveness of the search.

[0006] In a first aspect, the present invention provides a knowledge processing method for a question-and-answer knowledge base system, the method comprising:

[0007] Obtain all the knowledge contained in the question-and-answer knowledge base system, and then process the obtained knowledge into multiple knowledge fragments.

[0008] The similarity between each two knowledge segments is calculated using a preset text similarity algorithm, and each calculated similarity is compared with a preset similarity threshold.

[0009] Any similarity greater than the aforementioned similarity threshold is determined as the target similarity, and the two knowledge segments on which the target similarity is calculated are placed into the same knowledge block to achieve block processing of the knowledge.

[0010] The knowledge processing method for a question-and-answer knowledge base system provided by this invention has the following advantages:

[0011] Unlike traditional question-and-answer knowledge base systems that divide knowledge into fixed blocks, this invention uses a text similarity algorithm to calculate the similarity between different blocks, and then dynamically divides the blocks according to the similarity between different blocks. This makes the blocks within each block highly relevant and the information within each block as complete as possible, effectively improving the efficiency, accuracy, and comprehensiveness of the search.

[0012] In an optional implementation, before placing the two knowledge segments on which the target similarity is based into the same knowledge block, the method further includes:

[0013] Each of the two knowledge segments on which the target similarity is based is sequentially determined as the current knowledge segment;

[0014] If the similarity between the current knowledge segment and only one other knowledge segment is greater than the similarity threshold, then the step of placing the two knowledge segments on which the target similarity is based into the same knowledge block is executed; otherwise, the maximum value among all similarities calculated based on the current knowledge segment is determined, and the current knowledge segment and the other knowledge segment on which the maximum value is based are placed into the same knowledge block.

[0015] In one alternative implementation, it further includes:

[0016] During the process of placing the knowledge fragments into the knowledge blocks, if the number of knowledge fragments contained in any knowledge block reaches the corresponding capacity threshold, it is determined that the knowledge block has been divided into blocks, and it is prohibited to place any more knowledge fragments into the knowledge block.

[0017] In one alternative implementation, it further includes:

[0018] In the process of placing the knowledge fragments into the knowledge blocks, if the two knowledge fragments on which the similarity calculation is based when any similarity is greater than the similarity threshold cannot be placed into the same knowledge block, then the placement of the corresponding knowledge fragments is achieved based on the principle of prioritizing placing the two knowledge fragments on which any similarity calculation is based when any similarity is greater than the similarity threshold in adjacent positions.

[0019] In one optional implementation, the preset text similarity algorithm is specifically a TF-IDF algorithm and a cosine similarity algorithm of word frequency vectors.

[0020] In one optional implementation, after performing block processing of the knowledge, the method further includes:

[0021] The search identifies the target knowledge segment as the specified knowledge segment. Based on the specified knowledge segment, a sliding window is used to slide forward and backward over a specified number of knowledge segments, and both the specified knowledge segment and the knowledge segment that has been slid over are used as search results.

[0022] In one optional implementation, before the sliding window slides forward and backward over a specified number of knowledge fragments starting from the specified knowledge fragment, the method further includes:

[0023] Receive a window adjustment instruction and respond to the window adjustment instruction by adjusting the sliding window to a corresponding value; wherein, the value of the sliding window is the number of knowledge fragments that the sliding window can slide through in a single time.

[0024] Secondly, the present invention provides a knowledge processing apparatus for a question-and-answer knowledge base system, the apparatus comprising:

[0025] The sharding module is used to: acquire all the knowledge contained in the question-and-answer knowledge base system, and shard the acquired knowledge to obtain multiple knowledge shards;

[0026] The calculation module is used to: calculate the similarity between each two knowledge segments using a preset text similarity algorithm, and compare each calculated similarity with a preset similarity threshold;

[0027] The block segmentation module is used to: determine any similarity greater than the similarity threshold as the target similarity, and put the two knowledge segments on which the target similarity is calculated into the same knowledge block, thereby realizing the block processing of the knowledge.

[0028] The knowledge processing device for a question-and-answer knowledge base system provided by this invention has the following advantages:

[0029] Unlike traditional question-and-answer knowledge base systems that divide knowledge into fixed blocks, this invention uses a text similarity algorithm to calculate the similarity between different blocks, and then dynamically divides the blocks according to the similarity between different blocks. This makes the blocks within each block highly relevant and the information within each block as complete as possible, effectively improving the efficiency, accuracy, and comprehensiveness of the search.

[0030] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the knowledge processing method of the question-and-answer knowledge base system of the first aspect or any corresponding embodiment described above.

[0031] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the knowledge processing method of the question-and-answer knowledge base system of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a flowchart illustrating a knowledge processing method for a question-and-answer knowledge base system according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the block-based process in a knowledge processing method of a question-and-answer knowledge base system provided by an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the block data obtained in the knowledge processing method of a question-and-answer knowledge base system according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram illustrating the feedback of block data after a successful search using existing technology.

[0037] Figure 5 This is a schematic diagram of a sliding window in a knowledge processing method of a question-and-answer knowledge base system according to an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram illustrating the problem that can be solved by the sliding window scheme in a knowledge processing method of a question-and-answer knowledge base system according to an embodiment of the present invention;

[0039] Figure 7 This is a structural block diagram of a knowledge processing device for a question-and-answer knowledge base system according to an embodiment of the present invention;

[0040] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Question-and-answer knowledge base systems have a wide range of applications, such as chatbots and intelligent customer service. A vivid example is deploying a medical insurance question-and-answer knowledge base in a hospital. When a patient asks how to use out-of-town medical insurance in Beijing, the system searches the knowledge base, finds the relevant provisions, and provides the correct answer.

[0043] The development of question-answering knowledge base systems has gone through the following stages:

[0044] 1. Rule-based question-answering knowledge base system. This is the most traditional type of question-answering knowledge base system. It uses a set of rules to determine the user's question and selects the most suitable answer from the knowledge base. The advantage of this method is its high controllability, but the disadvantages are that it requires a lot of time and effort to design the rules and cannot handle complex questions.

[0045] 2. Statistical Question-Answering Knowledge Base System. This system analyzes a large number of questions and answers to build a statistical model, then finds the most suitable answer from the model based on the user's question. The advantage of this method is its ability to handle complex problems; the disadvantages are that it requires a large amount of labeled data, and the accuracy is affected by data quality.

[0046] 3. Deep learning-based question-answering knowledge base system. This system trains a deep neural network model using deep learning algorithms on a large number of questions and answers, then finds the most suitable answer from the model based on the user's question. The advantage of this method is its ability to handle complex questions and its high accuracy; the disadvantage is that it requires a large amount of labeled data and computational resources.

[0047] 4. Large-scale model-based question-answering knowledge base system. This system trains a large model on questions and answers in a knowledge base, leveraging the model's excellent understanding and interpretability to achieve more efficient and accurate question-answering functionality. The drawbacks are the exceptionally high time and computational resource costs of training the large model, and the fact that, compared to the conservative nature of traditional methods, the large model's unchecked nature often produces unexpected or uncontrollable answers.

[0048] Besides their respective advantages and disadvantages, the above-mentioned solutions share a common problem: once one knowledge base is built, building another requires retraining of rules, statistical models, deep learning models, and large-scale models, except for some reusable deep learning network structures. This demands sufficient professional knowledge and a significant amount of annotation and design work from the relevant personnel. For example, building a medical insurance question-and-answer knowledge base and then a stock market question-and-answer knowledge base would almost require repeating the same work.

[0049] To meet the need for rapidly building question-answering knowledge bases across different knowledge domains, a popular approach is to use word embedding to transform the original knowledge text into word vectors, which are then stored as a dictionary. When a question is asked, it is similarly transformed into word vectors, and then sophisticated search algorithms are used to search for relevant knowledge points in the knowledge base dictionary and assemble them into an answer.

[0050] There's a problem here: because the knowledge sources are controllable and processed, the content of the knowledge base is relatively standardized. However, the questions asked by users are relatively arbitrary, and the matching degree with the content in the knowledge base may not be ideal. For example, in the medical insurance Q&A knowledge base, there is content about out-of-town medical insurance indexed as "out-of-town medical insurance." If a patient asks "How do I use out-of-town medical insurance?", the content can be accurately searched and an answer can be given. However, if a patient asks "I'm from Shanxi, how do I get reimbursed?", it is difficult to accurately identify the question and give an answer. Therefore, traditional solutions usually require users to standardize their questions (e.g., specifying which questions can be asked, or only questions with a fixed format), or the algorithm continuously adds rules or trains models to adapt to various "unexpected" questions, in order to identify Shanxi as out-of-town (relative to Beijing) and reimbursement as medical insurance.

[0051] With the emergence of large-scale models, their superior understanding capabilities have led to new solutions. This involves transforming user-generated, arbitrary, and complex questions into simpler, more standardized ones, or assigning them to questions within a question bank. Furthermore, large-scale models offer a novel approach to answer generation: after retrieving content from the knowledge base, they utilize their generative capabilities to produce more easily understood and coherent answers. The integration of large-scale models with question-answering knowledge bases will not be elaborated upon here.

[0052] Regardless of the technology used (rules, statistics, deep learning, large models) combined with search, the solution is based on knowledge fragmentation, chunking, and vectorization of the content.

[0053] The tokenizer is generated to divide the text into individual tokens, ensuring that each token has relatively complete and independent semantics for use in subsequent tasks such as embedding. Several encoding methods are available, such as r50k_base, p50k_base, and cl100k_base. For models targeting OpenAI, the cl100k_base encoding method is typically used.

[0054] Segmentation involves breaking down the original knowledge base into several independent, shorter knowledge points. Each knowledge point serves as the minimum record for a question and answer, matching it with the question when answering. The scope of a segment is broader. For example, regulations on out-of-town medical insurance include several detailed rules, such as the reimbursement process, reimbursement ratio, reimbursement scope, and required documents. Assuming these details are the minimum knowledge points, out-of-town medical insurance is summarized into a large segment, which is provided along with the question when the questioner asks about out-of-town medical insurance. If the knowledge points are not closely related, this segmentation step may be omitted; in this case, one segment equals one segment. Most knowledge base systems involve segmentation and segmentation processes. A segment consists of a predetermined number of segments, and a general principle is that providing more answers is better than providing fewer.

[0055] In traditional question-answering knowledge base systems, knowledge is broken down into fixed blocks. If the blocks are too small, the answers may be incomplete; if the blocks are too large, they may contain redundant information, affecting search efficiency and accuracy.

[0056] This embodiment provides a knowledge processing method for a question-and-answer knowledge base system, which can be used in terminal devices such as mobile phones and tablets. Figure 1 This is a flowchart of a knowledge processing method for a question-and-answer knowledge base system according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0057] S11: Obtain all the knowledge contained in the question-and-answer knowledge base system, and perform fragmentation processing on the obtained knowledge to obtain multiple knowledge fragments.

[0058] The question-answering knowledge base system contains all the knowledge, which constitutes the original knowledge base and is typically text-based. This knowledge is then segmented to prepare for subsequent partitioning and searching. In this embodiment of the invention, the segments are made as small as possible to ensure search accuracy.

[0059] S12: Calculate the similarity between each two knowledge segments using a preset text similarity algorithm, and compare each calculated similarity with a preset similarity threshold.

[0060] The similarity threshold can be preset according to actual needs. If the similarity between two knowledge segments is greater than the similarity threshold, the two knowledge segments are considered to be highly related and are likely to be the answers to the same question. Otherwise, the two knowledge segments are considered to be less related and are likely to be the answers to different questions.

[0061] In addition, text similarity algorithms can be selected according to actual needs, such as the deep learning algorithm BERT (Bidirectional Encoder Representations from Transformers), or TF-IDF (Term Frequency-Inverse Document Frequency) + cosine similarity of frequency vectors, etc.

[0062] S13: Determine any similarity greater than the similarity threshold as the target similarity, and put the two knowledge segments on which the target similarity is calculated into the same knowledge block to realize the block processing of the knowledge.

[0063] Existing technologies, after segmentation, divide the data into blocks according to a predetermined number, for example, 3 segments are divided into 1 block. Generally, the smaller the block, the higher the score given by the search algorithm during the search, and the more accurate the search; the larger the block, the more complete the answer. With a predetermined number of blocks, smaller segments result in smaller blocks, and larger segments result in larger blocks. To solve this problem, this invention replaces the fixed block division of the existing technology with dynamic block division. Specifically, a preset text similarity algorithm is used to calculate the similarity between two knowledge segments. If the obtained similarity is greater than a set threshold, the two knowledge segments are placed in the same knowledge block; otherwise, they are placed in different blocks. This ensures that the knowledge segments within the same knowledge block have higher relevance, such as... Figure 2 As shown.

[0064] Unlike traditional question-and-answer knowledge base systems that divide knowledge into fixed blocks, this invention uses a text similarity algorithm to calculate the similarity between different blocks, and then dynamically divides the blocks according to the similarity between different blocks. This makes the blocks within each block highly relevant and the information within each block as complete as possible, effectively improving the efficiency, accuracy, and comprehensiveness of the search.

[0065] The knowledge processing method of the question-answering knowledge base system provided in this embodiment may further include, before placing the two knowledge segments on which the target similarity is based into the same knowledge block:

[0066] Each of the two knowledge segments on which the target similarity is based is sequentially determined as the current knowledge segment;

[0067] If the similarity between the current knowledge segment and only one other knowledge segment is greater than the similarity threshold, then the step of placing the two knowledge segments on which the target similarity is based into the same knowledge block is executed; otherwise, the maximum value among all similarities calculated based on the current knowledge segment is determined, and the current knowledge segment and the other knowledge segment on which the maximum value is based are placed into the same knowledge block.

[0068] If the similarity between any knowledge segment and multiple other knowledge segments is greater than a similarity threshold, then to improve the comprehensiveness of the search, the knowledge segment can be copied into multiple copies, and each copy can be placed into the same knowledge block with each of the other knowledge segments whose similarity is greater than the similarity threshold. For example, if knowledge segment a has similarity greater than the similarity threshold with knowledge segments b, c, and d, then a can be copied into three copies, and then a and b can be placed into the same knowledge block, a and c into the same knowledge block, and a and d into the same knowledge block.

[0069] However, to improve search accuracy, if the similarity between any knowledge segment and multiple other knowledge segments is greater than the similarity threshold, only that knowledge segment and the other knowledge segments corresponding to the maximum similarity can be placed in the same knowledge block. For example, if knowledge segment a has similarities greater than the similarity thresholds with knowledge segments b, c, and d, and has the highest similarity with b, then only a and b can be placed in the same knowledge block.

[0070] Of course, other settings can be made according to actual needs, all of which are within the protection scope of this invention, and will not be elaborated here.

[0071] The knowledge processing method of the question-and-answer knowledge base system provided in this embodiment may further include:

[0072] During the process of placing the knowledge fragments into the knowledge blocks, if the number of knowledge fragments contained in any knowledge block reaches the corresponding capacity threshold, it is determined that the knowledge block has been divided into blocks, and it is prohibited to place any more knowledge fragments into the knowledge block.

[0073] To avoid an excessive number of fragments within a single knowledge block, this embodiment of the invention allows setting a maximum number of fragments that a single knowledge block can contain, i.e., a capacity threshold. Specifically, to improve search comprehensiveness, this embodiment of the invention can appropriately increase the number of fragments in each block, for example, each block can contain a maximum of 5 fragments. After using a text similarity algorithm to calculate the similarity between fragments, if the similarity is greater than the set threshold, the two fragments are placed in the same block. When the number of fragments in a block reaches the set number of fragments per block (capacity threshold), it is prohibited to add any more fragments to that block, thus placing the corresponding fragments in a block according to the set number of fragments.

[0074] The knowledge processing method of the question-and-answer knowledge base system provided in this embodiment may further include:

[0075] In the process of placing the knowledge fragments into the knowledge blocks, if the two knowledge fragments on which the similarity calculation is based when any similarity is greater than the similarity threshold cannot be placed into the same knowledge block, then the placement of the corresponding knowledge fragments is achieved based on the principle of prioritizing placing the two knowledge fragments on which any similarity calculation is based when any similarity is greater than the similarity threshold in adjacent positions.

[0076] If, due to factors such as the setting of the block capacity threshold, two knowledge segments based on which the similarity is greater than the similarity threshold cannot be placed in the same knowledge block, these two knowledge segments can be placed in adjacent positions as much as possible, so that the adjacent knowledge segments have a high correlation, thereby further improving the comprehensiveness of the search when supplementing the search.

[0077] After the process described in the above embodiment is completed, the knowledge is divided into a series of fragments, each fragment is appended with the index of the block it belongs to, and then word embedding and database storage are performed. Compared to the processing without dynamic segmentation, the resulting segmented data is as follows: Figure 3 As shown.

[0078] This embodiment provides a knowledge processing method for a question-and-answer knowledge base system. After segmenting the knowledge into blocks, it may further include:

[0079] The search identifies the target knowledge segment as the specified knowledge segment. Based on the specified knowledge segment, a sliding window is used to slide forward and backward over a specified number of knowledge segments, and both the specified knowledge segment and the knowledge segment that has been slid over are used as search results.

[0080] Before the sliding window slides forward and backward over a specified number of knowledge fragments starting from the specified knowledge fragment, it may also include:

[0081] Receive a window adjustment instruction and respond to the window adjustment instruction by adjusting the sliding window to a corresponding value; wherein, the value of the sliding window is the number of knowledge fragments that the sliding window can slide through in a single time.

[0082] After the knowledge base is fully stored, users of the question-and-answer knowledge base system ask questions. The system converts the questions into word vectors using the same method, and then uses a sophisticated search algorithm to search for relevant knowledge points in the knowledge base dictionary, concatenates them into an answer, and provides the result. In existing technologies, this is implemented as follows: Figure 4 As shown, the search algorithm hits s0, s1, and s10, and then returns the blocks where these fragments are located, namely c0 and c2, which are then concatenated to form the answer.

[0083] After adding a dynamic segmentation scheme in this embodiment of the invention, the accuracy and comprehensiveness of the answer are improved even when using existing search methods. Furthermore, based on the aforementioned dynamic segmentation scheme, a sliding window adjustment scheme is added; this is implemented by setting a sliding window that slides over a specified number of segments, typically with the window size set to be the same as the segment size (capacity threshold). For example... Figure 5 As shown, assuming the partition sn is hit by the search algorithm, and the sliding window size is set to 5, sliding from {Sn-4,...,Sn} to {Sn,...,Sn+4}. By adjusting the sliding window, the following problem can be solved: Figure 6 The questions shown further enhance the completeness of the answers.

[0084] In practice, certain parameters, such as the threshold setting for the dynamic segmentation scheme and the size of the sliding window, can be determined through experiments to obtain the best results. Furthermore, users can adjust the size of the sliding window using window adjustment commands to better suit their current needs.

[0085] This invention addresses the shortcomings of existing question-answering knowledge base systems in terms of accuracy and comprehensiveness when processing user questions through a dynamic segmentation scheme and a sliding window adjustment scheme. The dynamic segmentation scheme enables the system to dynamically segment knowledge based on text similarity, improving search accuracy. The sliding window adjustment scheme further enhances the comprehensiveness of the answer by mining information surrounding the hit knowledge point. Specifically, traditional knowledge base systems divide knowledge into fixed small blocks. If the blocks are too small, the answer may be incomplete; if the blocks are too large, they may contain redundant information, affecting search efficiency and accuracy. The dynamic segmentation scheme utilizes a text similarity calculation algorithm to dynamically segment based on the similarity between different blocks, ensuring that each block is highly relevant and contains as much complete information as possible, thereby improving search accuracy and comprehensiveness. Traditional knowledge base systems directly connect all relevant knowledge points found after a search to form an answer. However, this may overlook some related knowledge segments that are adjacent to the target segment but were not found in the search. The sliding window adjustment scheme uses a fixed window to slide around the found segments, bringing in those overlooked but still important contents. This ensures the quality of the answer while improving its comprehensiveness. Both schemes are designed to better suit real-world application scenarios and search needs, thus this new technical solution effectively addresses existing problems.

[0086] This embodiment also provides a knowledge processing apparatus for a question-and-answer knowledge base system. This apparatus is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0087] This embodiment provides a knowledge processing device for a question-and-answer knowledge base system, such as... Figure 7 The demonstration may include:

[0088] The sharding module 11 is used to: acquire all the knowledge contained in the question-and-answer knowledge base system, and shard the acquired knowledge to obtain multiple knowledge shards;

[0089] The calculation module 12 is used to: calculate the similarity between each two knowledge segments using a preset text similarity algorithm, and compare each calculated similarity with a preset similarity threshold;

[0090] The block segmentation module 13 is used to: determine any similarity greater than the similarity threshold as the target similarity, and put the two knowledge segments on which the target similarity is calculated into the same knowledge block to realize the block processing of the knowledge.

[0091] In this embodiment, the knowledge processing device of the question-and-answer knowledge base system is presented in the form of functional modules. Here, a module refers to an application-specific integrated circuit (ASIC), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0092] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0093] This invention also provides a computer device having the above-described features. Figure 7 The knowledge processing device of the question-and-answer knowledge base system shown.

[0094] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a set of sharded servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0095] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include hardware chip shards. These hardware chip shards may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The programmable logic devices may be complex programmable logic devices (CLPs), field-programmable gate arrays (FPGAs), general-purpose array logic (GDAs), or any combination thereof.

[0096] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0097] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0099] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0100] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0101] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0102] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A knowledge processing method of a question and answer knowledge base system, characterized by, include: Obtain all the knowledge contained in the question-and-answer knowledge base system, and then process the obtained knowledge into multiple knowledge fragments. The similarity between each two knowledge segments is calculated using a preset text similarity algorithm, and each calculated similarity is compared with a preset similarity threshold. Any similarity greater than the aforementioned similarity threshold is determined as the target similarity, and the two knowledge segments on which the target similarity is calculated are placed into the same knowledge block to achieve block processing of the knowledge. The search hits a specified knowledge segment. Based on the specified knowledge segment, a sliding window is used to slide forward and backward a specified number of knowledge segments, starting from the specified knowledge segment. The specified knowledge segment and the knowledge segments that have been slid over are both used as search results. The size of the sliding window is the same as the capacity threshold of the knowledge segments contained in the knowledge segment.

2. The method of claim 1, wherein, Before placing the two knowledge segments on which the target similarity is based into the same knowledge block, the method further includes: Each of the two knowledge segments on which the target similarity is based is sequentially determined as the current knowledge segment; If the similarity between the current knowledge segment and only one other knowledge segment is greater than the similarity threshold, then the step of placing the two knowledge segments on which the target similarity is based into the same knowledge block is executed; otherwise, the maximum value among all similarities calculated based on the current knowledge segment is determined, and the current knowledge segment and the other knowledge segment on which the maximum value is based are placed into the same knowledge block.

3. The method according to claim 2, characterized in that, Also includes: During the process of placing the knowledge fragments into the knowledge blocks, if the number of knowledge fragments contained in any knowledge block reaches the corresponding capacity threshold, it is determined that the knowledge block has been divided into blocks, and it is prohibited to place any more knowledge fragments into the knowledge block.

4. The method according to claim 3, characterized in that, Also includes: In the process of placing the knowledge fragments into the knowledge blocks, if the two knowledge fragments on which the similarity calculation is based when any similarity is greater than the similarity threshold cannot be placed into the same knowledge block, then the placement of the corresponding knowledge fragments is achieved based on the principle of prioritizing placing the two knowledge fragments on which any similarity calculation is based when any similarity is greater than the similarity threshold in adjacent positions.

5. The method according to claim 4, characterized in that, The preset text similarity algorithm is specifically TF-IDF and cosine similarity algorithm of word frequency vectors.

6. The method according to claim 1, characterized in that, Before the sliding window slides forward and backward a specified number of knowledge fragments from the specified knowledge fragment, it also includes: Receive a window adjustment instruction and respond to the window adjustment instruction by adjusting the sliding window to a corresponding value; wherein, the value of the sliding window is the number of knowledge fragments that the sliding window can slide through in a single time.

7. A knowledge processing device for a question-and-answer knowledge base system, characterized in that, include: The sharding module is used to: acquire all the knowledge contained in the question-and-answer knowledge base system, and shard the acquired knowledge to obtain multiple knowledge shards; The calculation module is used to: calculate the similarity between each two knowledge segments using a preset text similarity algorithm, and compare each calculated similarity with a preset similarity threshold; The block segmentation module is used to: determine any similarity greater than the similarity threshold as the target similarity, and put the two knowledge segments on which the target similarity is calculated into the same knowledge block, thereby realizing the block segmentation processing of the knowledge; The feedback module is used to: determine that the knowledge segment hit by the search is a specified knowledge segment, slide a specified number of knowledge segments forward and backward based on the specified knowledge segment, and provide feedback on the specified knowledge segment and the knowledge segment slid to as search results, wherein the size of the sliding window is the same as the capacity threshold of the knowledge segments contained in the knowledge segment.

8. A computer device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the knowledge processing method of the question-and-answer knowledge base system according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the knowledge processing method of the question-and-answer knowledge base system according to any one of claims 1 to 6.

Images