Text processing method, computing device and computer readable storage medium

Through text processing methods collaboratively with multilingual model, problem text is optimized and vectorized, and multi-level matching detection is carried out, the problem of insufficient accuracy of knowledge text output in the existing technology is solved, and more efficient and accurate knowledge acquisition is achieved.

CN119088955APending Publication Date: 2024-12-06ZHONGKE YUNGU TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410861886.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When the prior art uses large-scale language model (LLM) to achieve knowledge text output, the accuracy is insufficient, making it difficult to quickly and accurately obtain the required knowledge from complicated network texts.

Method used

By setting up multiple language models to collaborate on problem text, including optimizing the problem based on the preset first language model, vectorizing the problem to match the knowledge base, using the second language model for matching detection, and if necessary, the The trilingual model provides supplementary processing to ensure the accuracy of the knowledge text of the output.

Benefits of technology

It improves the accuracy and fluency of the knowledge text, enhances the accuracy of understanding and answers to user questions, and reduces the possibility of wrong matches.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088955A_ABST
    Figure CN119088955A_ABST
Patent Text Reader

Abstract

The invention discloses a text processing method, computing equipment and a computer readable storage medium, and the method comprises the steps: carrying out the problem optimization of a to-be-processed problem text based on a preset first language model, and obtaining the optimized to-be-processed problem text; carrying out vectorization processing on the optimized to-be-processed problem text, then matching the to-be-processed problem text with a preset knowledge base, and determining a target knowledge vector; the preset knowledge base comprises at least one knowledge vector; and performing matching detection between the target knowledge vector and the to-be-processed question text based on a preset second language model, and outputting the target knowledge vector when the detection is passed. In this way, the to-be-processed problem text is collaborated by setting the multiple language models, and the accuracy of the output knowledge text can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text processing method, a computing device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of the Internet, a massive text knowledge base has been formed on the Internet, which has provided great convenience for people to learn and acquire knowledge. However, how to quickly and accurately obtain the required knowledge from the complex network text is still a difficult problem to be solved.

[0003] In recent years, large-scale language model (LLM) technology has made great progress and demonstrated strong capabilities in natural language understanding tasks. LLM is a natural language processing model based on deep learning, which can learn the grammar and semantics of natural language and generate human-readable text. However, the current use of LLM to achieve knowledge text output is still not accurate enough. Summary of the invention

[0004] The purpose of the present application is to provide a text processing method, a computing device and a computer-readable storage medium, which help improve the accuracy of the output knowledge text by setting multiple language models to collaborate on the problem text to be processed.

[0005] To achieve the above objectives:

[0006] In a first aspect, an embodiment of the present application provides a text processing method, comprising:

[0007] Optimizing the question text to be processed based on the preset first language model to obtain the optimized question text to be processed;

[0008] The optimized problem text to be processed is vectorized and matched with a preset knowledge base to determine a target knowledge vector; the preset knowledge base includes at least one knowledge vector;

[0009] A matching detection is performed between the target knowledge vector and the question text to be processed based on a preset second language model, and the target knowledge vector is output when the detection passes.

[0010] In one embodiment, the step of optimizing the question text to be processed based on the preset first language model to obtain the optimized question text to be processed includes:

[0011] Based on the first language model, the intent of the question text to be processed is classified to obtain a target intent;

[0012] The problem text to be processed is optimized according to the target intention to obtain the optimized problem text to be processed.

[0013] In one embodiment, the optimized question text to be processed is vectorized and matched with a preset knowledge base to determine a target knowledge vector, including:

[0014] Performing vectorization processing on the optimized question text to be processed to obtain a corresponding question vector;

[0015] Matching the question vector with a preset knowledge base to obtain a similarity between the question vector and at least one knowledge vector in the preset knowledge base;

[0016] The knowledge vector with the highest similarity and greater than a first preset threshold is determined as a target knowledge vector.

[0017] In one embodiment, the vectorization processing of the optimized question text to be processed to obtain a corresponding question vector includes:

[0018] Slicing the optimized question text to be processed to obtain a word set;

[0019] Encode the word set, and input the encoded word set into an embedding layer of a preset word embedding model to obtain a feature word vector output by the preset word embedding model;

[0020] Obtaining the cosine similarity between the feature word vector and at least one question vector in a preset question vector library;

[0021] According to the cosine similarity, a question vector corresponding to the feature word vector is determined.

[0022] In one embodiment, the slicing process is performed on the optimized question text to be processed to obtain a word set, including:

[0023] Inputting the optimized problem text to be processed into the embedding layer of the word embedding model to obtain the word embedding vector output by the preset word embedding model;

[0024] Based on the word embedding vector, the optimized contextual relationship in the question text to be processed is calculated by the first language model to obtain the optimized global interaction characteristics of the question text to be processed;

[0025] Based on the global interaction characteristics, a word set constituting at least one feature word of the optimized question text to be processed is determined.

[0026] In one embodiment, the method further comprises:

[0027] If the similarities are all less than or equal to the first preset threshold, the optimized question text to be processed is processed by a preset third language model based on a third-party knowledge base, and the obtained processing result is output.

[0028] In one embodiment, the matching detection between the target knowledge vector and the question text to be processed based on a preset second language model includes:

[0029] Detecting the similarity between at least one knowledge data in a local database or a third-party knowledge base and the target knowledge vector, and determining that the target knowledge vector and the problem text to be processed have passed the detection when at least one of the similarities is greater than a second preset threshold; and / or,

[0030] The similarity between the knowledge data and the problem text to be processed is detected, and when at least one of the similarities is greater than a third preset threshold, it is determined that the target knowledge vector and the problem text to be processed have passed the detection.

[0031] In one embodiment, the method further comprises:

[0032] When the detection fails, the question text to be processed is re-optimized based on a preset first language model.

[0033] In a second aspect, an embodiment of the present application provides a computing device, specifically comprising:

[0034] processor;

[0035] a memory for storing instructions executable by the processor;

[0036] The processor is configured to execute the instructions to perform the text processing method as described in the first aspect.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the instructions in the computer-readable storage medium are executed by a processor of a computing device, the computing device is enabled to implement the text processing method described in the first aspect.

[0038] The text processing method, computing device and computer-readable storage medium provided by the embodiment of the present application include: optimizing the problem text to be processed based on a preset first language model to obtain the optimized problem text to be processed; vectorizing the optimized problem text to be processed and matching it with a preset knowledge base to determine a target knowledge vector; the preset knowledge base includes at least one knowledge vector; matching detection between the target knowledge vector and the problem text to be processed based on a preset second language model, and outputting the target knowledge vector when the detection passes. In this way, by setting multiple language models to collaborate with the problem text to be processed, it helps to improve the accuracy of the output knowledge text. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a text processing method provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a process for adjusting matching accuracy in an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a specific flow chart of a text processing method provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0044] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0045] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0046] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0047] It should be noted that, in this article, step codes such as S101, S102, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the scope of protection of this application.

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0050] See also Figure 1 , is a text processing method provided in an embodiment of the present application. The text processing method can be implemented in software and / or hardware. In this embodiment, the text processing method is applied to a server as an example. The text processing method provided in this embodiment includes:

[0051] Step S101: optimizing the question text to be processed based on a preset first language model to obtain an optimized question text to be processed.

[0052] The preset first language model may be a large-scale language model, which is mainly used to provide clear and accurate prompts according to user needs. At the same time, the preset first language model can understand the user's needs and convert them into specific tasks.

[0053] Optionally, when obtaining the text of the problem to be processed, the demand or intention referred to by the text of the problem to be processed can be determined based on the preset first language model according to the characteristic words or prompt words contained in the text of the problem to be processed, and after determining the demand or intention, the obtained text of the problem to be processed is further optimized. The text of the problem to be processed can be obtained in the form of word, pdf, excel, txt, csv, json, web page, etc.

[0054] In one embodiment, optimizing the question text to be processed based on the preset first language model to obtain the optimized question text to be processed includes:

[0055] Based on the preset first language model, the intent of the question text to be processed is classified to obtain the target intent;

[0056] The problem text to be processed is optimized according to the target intention to obtain the optimized problem text to be processed.

[0057] First, the preset first language model is used to analyze the question text to be processed through natural language processing (NLP) technologies such as deep learning and machine learning, and the user question is classified according to the categories in the predefined category name to find out the business opportunities that may be contained therein. For example, when the user inputs the question "How to buy a mixer truck", the main intention in the question input by the user is classified, such as whether it is: consulting to buy the main product, consulting to buy the accessory product, and other intentions; and the secondary intention of the question input by the user is classified, such as whether it is: consulting to buy the main product: concrete equipment, mining equipment, emergency equipment; and determine whether the consultation to buy the accessory product is: concrete equipment accessories, mining equipment accessories, emergency equipment accessories; and determine whether other intentions are: non-business opportunity intentions. Then, after clarifying the intention of the question, the question text is optimized according to the determined intention through NLP technologies such as text generation and semantic analysis. This includes reorganizing or rewriting the question to make the optimized question text to be processed clearer, easier to understand and answer.

[0058] Optionally, the user question can also be optimized based on specific cases in the local data or examples in the sample. For example, when the user inputs the question "Do you have hydraulic oil pipes for pump trucks?", the input question is classified into: consulting on purchasing accessory products - concrete equipment accessories. Then, combined with the specific cases in the local data or examples contained in the sample, the optimized question text to be processed is determined to be: "consulting on hydraulic oil pipe accessories".

[0059] Step S102: vectorizing the optimized question text to be processed and matching it with a preset knowledge base to determine a target knowledge vector; the preset knowledge base includes at least one knowledge vector.

[0060] First, the optimized text of the problem to be processed is vectorized to determine the vector corresponding to the features contained in the optimized text of the problem to be processed. Then, according to some tools for calculating the similarity between vectors contained in the preset knowledge base, such as cosine similarity, Euclidean distance, etc., the vectorized vector is matched with the preset knowledge base to determine the target knowledge vector that answers the problem to be processed. Here, in order to efficiently retrieve and match vectors, the knowledge base is usually set with an index mechanism, such as a tree structure, a hash table or a dedicated vector database technology.

[0061] In one embodiment, the optimized question text to be processed is vectorized and matched with a preset knowledge base to determine a target knowledge vector, including:

[0062] Perform vectorization processing on the optimized question text to be processed to obtain the corresponding question vector;

[0063] Matching the question vector with a preset knowledge base to obtain a similarity between the question vector and at least one knowledge vector in the preset knowledge base;

[0064] The knowledge vector with the highest similarity and greater than a first preset threshold is determined as the target knowledge vector.

[0065] First, the word embedding model in natural language processing (NLP), such as the BERT (Bidirectional Encoder Representations from Transformers) model, can be used. Here, the BERT model is a language model pre-trained based on a bidirectional Transformer encoder, which can effectively capture representations at the word and sentence levels. Convert the question text into a mathematically processable vector form, namely the question vector. Here, the question vector contains the optimized semantic information of the question text to be processed. Secondly, compare the question vector with each knowledge vector in the preset knowledge base to calculate the similarity between them. Similarity calculation can use a variety of methods, such as cosine similarity, Euclidean distance or other measurement methods, the purpose of which is to quantify the similarity between the question vector and each knowledge vector. Then, based on the calculated similarity, filter out the knowledge vectors with a similarity higher than the first preset threshold. Finally, select a knowledge vector with the highest similarity from the filtered knowledge vectors as the target knowledge vector to determine the knowledge point that is most likely to represent the user's question intention for subsequent question answering or information retrieval. In this way, determining the most relevant target knowledge vector in the preset knowledge base by calculating the similarity helps to quickly and accurately provide the information required by the user, further improving the accuracy of the output knowledge text. In one embodiment, vectorization is performed on the optimized question text to be processed to obtain a corresponding question vector, including:

[0066] Slice the optimized question text to be processed to obtain a word set;

[0067] Encode the word set, and input the encoded word set into the embedding layer of the preset word embedding model to obtain the feature word vector output by the preset word embedding model;

[0068] Obtaining the cosine similarity between the feature word vector and at least one question vector in a preset question vector library;

[0069] According to the cosine similarity, the question vector corresponding to the feature word vector is determined.

[0070] First, the optimized text of the question to be processed is sliced, that is, the text is divided into separate word sets, which usually involves word segmentation operations. Secondly, the obtained word set is encoded. During the encoding process, each word is converted into a unique digital identifier based on a predefined vocabulary. Here, the encoded word set can be used as the input of the model. Next, the encoded word set is input into the embedding layer of the BERT (Bidirectional EncoderRepresentations from Transformers) model. Through the embedding layer of the BERT model, the feature word vectors contained in the optimized text of the question to be processed can be determined, and these feature word vectors contain rich semantic information. Then, the cosine similarity between the feature word vector obtained and at least one question vector in the preset question vector library is calculated. Specifically, the similarity of different sentences can be measured by COSINE similarity, and the formula is expressed as follows:

[0071]

[0072] Among them, A represents the feature word vector, and B represents a question vector in the preset question vector library. Here, when calculating the cosine value of the angle between two vectors through cosine similarity to determine their similarity, if the two vectors are in the same direction, then their cosine similarity is 1, indicating that they are very similar; if the directions are opposite, then the cosine similarity is -1, indicating that they are completely dissimilar; if the two vectors are orthogonal, then the cosine similarity is 0, indicating that there is no correlation between them. Here, the matching accuracy can also be adjusted through the Cross-encoder model. Here, the Cross-encoder model is a vectorized retrieval model that can encode and judge multiple text pairs at one time. Figure 2 As shown in the figure, the Cross-encoder model is a neural network architecture. Different from the model that processes a single text, the Cross-encoder needs to concatenate two sentences (usually including a query and a document, or two sentences that need to be compared) when the model is input, and input them into the BERT model. BERT analyzes and processes the two sentences and inputs them into the classifier to score the similarity or relevance of the two sentences, and then outputs a score between 0 and 1. The output score reflects the similarity or relevance between the input texts. This allows the model to interact with the two sentences at a deeper level, so as to better understand and compare their semantic content.

[0073] Finally, the question vector corresponding to the feature word vector is determined based on the calculated cosine similarity. This usually involves selecting a preset question vector that is most similar to the feature word vector, or generating the final question vector by weighted averaging based on similarity. Here, the language understanding ability of the BERT model can efficiently process natural language data.

[0074] Exemplarily, first, after determining the optimized problem text to be processed, the sentence text converted from the prompt word is sliced, and CLS tags are added to determine the semantic information of the text, and SEP tags are added to divide the different text parts in the text. For example, when processing the relationship between sentences and corresponding labels, SEP tags can help the model distinguish them. Secondly, determine the word set T1 = {[t1a1, t1a2, t1a3,...]}, {[t2a1, t2a2, t2a3,...]},..., {[tia1, tia2, tia3,...]}. Then, the set T1 is encoded, and each word or tag in T1 is converted into a unique integer ID to generate a coding set T2. Next, the obtained coding collection T2 is input into the embedding layer of the BERT model for embedding processing to output the feature word vector T3. Among them, T3 is a vectorized representation set in a high-dimensional space, which contains the embedding vectors of multiple words generated by the input encoding. Finally, the cosine similarity between the feature word vector T3 and each question vector in the preset question vector library is compared to determine the matching question vector.

[0075] In one embodiment, the optimized question text to be processed is sliced ​​to obtain a word set, including:

[0076] Input the optimized problem text to be processed into the embedding layer of the word embedding model to obtain the word embedding vector output by the preset embedding model;

[0077] Based on the word embedding vector, the contextual relationship in the optimized question text to be processed is calculated through the first language model to obtain the global interaction characteristics of the optimized question text to be processed;

[0078] Based on the global interaction characteristics, a word set constituting at least one characteristic word of the optimized question text to be processed is determined.

[0079] First, the optimized question text to be processed is input into the embedding layer of a word embedding model (such as a BERT model). Here, the embedding layer of the BERT model is responsible for converting each word in the text into a high-dimensional word embedding vector. These vectors not only contain the semantic information of the word, but also can capture the contextual relationship between words through the pre-training process. Then, these word embedding vectors are input into the sparse multi-head attention layer of the first language model. In Transformer models such as BERT, the multi-head attention mechanism allows the model to focus on different information on different heads, so as to better understand the complex patterns and relationships in the text. In this process, the contextual relationship in the optimized question text is calculated to obtain the global interaction characteristics of the output. These global interaction characteristics are a deep understanding of the semantics and contextual information of the entire question text. Finally, key information is extracted from the global interaction characteristics, and the feature words or phrases that best represent the meaning of the question text are identified to determine the word set of at least one feature word that constitutes the optimized question text. Here, by deeply understanding the optimized question text and extracting the most representative word set from it, it helps to more accurately reflect the user's question intention, thereby improving the effect and accuracy of subsequent processing steps.

[0080] In one embodiment, it further comprises:

[0081] If the similarities are all less than or equal to the first preset threshold, the preset third language model processes the optimized question text to be processed based on the third-party knowledge base and outputs the obtained processing result.

[0082] Among them, the preset third language model can be a large-scale language model with a strong knowledge reserve, can answer various questions, and provide detailed information about a specific topic. Specifically, the third language model can push questions and answers to users based on an external knowledge base (a third party, such as an open knowledge base on the Internet). Here, when the first language model cannot determine the target knowledge vector (i.e., when the existing knowledge base cannot satisfy the user's question), the third language model can return a more likely accurate target knowledge vector, and then the user can give feedback by evaluating the target knowledge vector (satisfied, dissatisfied), and the dissatisfied target knowledge vector is maintained after manual review. Here, after the third language model processes the optimized problem text to be processed based on the third-party knowledge base, the processed target knowledge vector can be updated in the preset knowledge base.

[0083] Step S103: performing a matching detection between the target knowledge vector and the question text to be processed based on a preset second language model, and outputting the target knowledge vector when the detection passes.

[0084] The preset second language model can be a large-scale language model, which is mainly used to check and verify the obtained information. In the present application, the second language model is mainly used to verify the target knowledge vector determined by the first language model or the third language model, and specifically can be determined by matching the target knowledge vector with the problem text to be processed to determine whether the target knowledge vector is qualified.

[0085] In one embodiment, matching detection is performed between the target knowledge vector and the question text to be processed based on a preset second language model, including:

[0086] Detecting the similarity between at least one knowledge data in a local database or a third-party knowledge base and the target knowledge vector, and determining that the target knowledge vector and the problem text to be processed have passed the detection when at least one similarity is greater than a second preset threshold; and / or,

[0087] The similarity between the knowledge data and the question text to be processed is detected, and when at least one similarity is greater than a third preset threshold, it is determined that the target knowledge vector and the question text to be processed have passed the detection.

[0088] First, using a preset second language model, detect the similarity between at least one knowledge data in a local database or a third-party knowledge base and the above-determined target knowledge vector, and obtain a corresponding numerical value of at least one similarity. Then, when it is determined that the similarity between the knowledge data and the target knowledge vector is greater than the second preset threshold, it is considered that the matching detection between the target knowledge vector and the text of the problem to be processed has passed, which means that the target knowledge vector is considered to be an effective match and can be used as an answer or solution to the problem. In addition, the similarity between at least one knowledge data in a local database or a third-party knowledge base and the text of the problem to be processed can also be directly detected to determine whether the target knowledge vector is qualified. Here, if there is a similarity between the knowledge data and the text of the problem to be processed that is greater than the third preset threshold, it is also considered that the matching detection between the target knowledge vector and the text of the problem to be processed has passed. In this way, verifying the matching degree between the target knowledge vector and the text of the problem to be processed from multiple angles helps to reduce the possibility of false matching and ensure that the questions provided by the user can be answered accurately and reliably.

[0089] Exemplarily, assuming that the user input question is "mixer truck price", and the target knowledge vector given by the first language model is "mixer truck prices vary according to cities and models. Please further provide intended purchase locations and purchase models". If the second threshold and the third threshold are both set to 0.8, the second language model is used to detect whether there is a knowledge vector in the knowledge base whose question vector matches "mixer truck price" more than 0.8. If so, it is determined that the target knowledge vector has passed the detection of the question text to be processed; if not, the second language model can also be used to detect whether there is an answer vector in the knowledge base whose answer vector matches "mixer truck prices vary according to cities and models. Please further provide intended purchase locations and purchase models" more than 0.8. If so, it is determined that the target knowledge vector has passed the detection of the question text to be processed.

[0090] In one embodiment, it further comprises:

[0091] When the detection fails, the question text to be processed is re-optimized based on a preset first language model.

[0092] Optionally, when the matching test fails, it means that the correlation between the target knowledge vector and the question text to be processed may not be strong enough, or there is ambiguity and misunderstanding. In this case, the question text to be processed will be analyzed again through the first language model, and the corresponding target knowledge vector will be determined. In this way, the matching success rate can be improved by iteratively optimizing the question text, which helps to deal with complex or ambiguous questions and ensures that users can get satisfactory answers.

[0093] In summary, in the text processing method provided in the above embodiment, by setting multiple language models to collaborate on the problem text to be processed, it is helpful to improve the accuracy of the output knowledge text, while improving the accuracy and fluency of knowledge expression.

[0094] Based on the same inventive concept as the above embodiments, the text processing method provided by the present application is described in detail below through a specific example. Figure 3 As shown, the preset first language model is LLM No. 1, which plays the role of prompt engineer; the preset second language model is LLM No. 2, which plays the role of omniscient; the preset third language model is LLM No. 3, which plays the role of knowledge prosecutor. The existing text processing method includes the following processes: 1) vectorize the knowledge base as a whole to obtain multiple knowledge base vectors; 2) vectorize the questions input by the user; 3) match the question vector and the knowledge vector to obtain the N knowledge vectors with the highest matching degree; 4) text the N knowledge vectors with the highest matching degree to obtain the corresponding question text as prompt, and after inputting it into the LLM model, obtain the output text; 5) return the output to the user as the output result after the knowledge base question and answer.

[0095] However, the accuracy of existing knowledge base retrieval technology, query and output depends on the slice matching of knowledge base documents. Generally speaking, if the slice is too small or too large, the search results with higher matching degree will have semantic understanding deviation or one-sidedness with the user input questions.

[0096] Existing technologies usually solve this problem by adjusting the appropriate slice size, but this method has limitations; some other technologies adjust the matching degree by adding user evaluation afterwards, but this method relies on continuous iteration of accurate user data input, which requires time and data costs for training or fine-tuning. In addition, the existing knowledge base lacks a verification mechanism in the process of asking and answering questions, which may cause a large difference between the question and answer results and expectations, and needs to be improved by improving the quality of the original data of the knowledge base.

[0097] To solve the above problems, the text processing method provided in this embodiment specifically includes:

[0098] Step S201: Obtain the text of the question to be answered.

[0099] Optionally, the problem text to be processed may be obtained in the form of word, pdf, excel, txt, csv, json, web page, etc.

[0100] Step S202: LLM No. 1 performs text segmentation to determine vectorized text.

[0101] Optionally, the question is converted through the prompt word through LLM No. 1, and the converted question is vectorized to obtain a question vector. Here, during the vectorization process, due to the token length limit of the text when encoding the text, the text needs to be segmented. The traditional mode mainly performs mechanical segmentation, which is easy to segment the context key content during segmentation, resulting in unsatisfactory performance of vector search results and question and answer results. Therefore, in this application, a method of customizing sparse attention is used to adjust the attention parameters according to needs, so that good contextual coherence can be achieved in long texts. The specific operation process includes: first, adding LoRA adjustment to each attention layer of Bert, using the class LoRA_BERT; then, customizing the LoRA Adjustment class to adjust the LoRA parameters; finally, according to the calculation results, adjust the query layer, key layer and value layer of Bert.

[0102] Step S203: Combine the question vector with the context prompt to perform vectorized query.

[0103] Optionally, the question vector is combined with contextual cues, which usually involves vector operations such as concatenation, weighted summation, or the use of neural network models to integrate multi-source information. In this way, the combined vector contains both the direct intent of the question and relevant contextual information. Then, based on the combined vector, a query is performed in a knowledge base, database, or any other dataset, and the similarity between the combined vector and the vector representation of each candidate answer in the dataset is calculated. Here, the similarity can be measured using various methods, such as cosine similarity, Euclidean distance, etc. Finally, the answer that is most similar to the query vector (i.e., has the highest similarity score) is returned.

[0104] Step S204: indexing in a preset knowledge base.

[0105] Step S205: Calculate the similarity and determine the target knowledge vector.

[0106] Optionally, the question vector is matched with a preset knowledge base, the similarity between the question vector and at least one knowledge vector in the preset knowledge base is obtained, and the knowledge vector with the highest similarity and greater than a first preset threshold is determined as the target knowledge vector.

[0107] Step S206: Combine the target knowledge vector, the question text to be processed and the context prompt, and send them to LLM No. 2.

[0108] Optionally, the target knowledge vector, the question text to be processed and the context prompt are combined and packaged to determine a data structure including multiple parts so as to be input into LLM No. 2 together. In this way, the determined target knowledge vector is detected by LLM No. 2.

[0109] Step S207: If the target knowledge vector cannot be determined, the LLM No. 3 omniscient person gives an answer.

[0110] Optionally, LLM No. 3 will push questions and answers to users based on an external knowledge base (a third party, such as an open knowledge base on the Internet). The large model here can be the same as LLM No. 1 and No. 2, or different NLP models can be selected. Its essence is that when the existing knowledge base cannot satisfy the user's questions, it can return a more likely accurate answer to the user. Subsequent users can provide feedback by evaluating the answer (satisfied or dissatisfied). Unsatisfactory answers will be maintained after manual review.

[0111] Step S208: LLM Prosecutor No. 2 verifies the knowledge. If the verification is qualified, execute step S209; if the verification is unqualified, return to execute step S202.

[0112] Optionally, determine whether the newly proposed question already has similar questions in the existing knowledge base, and determine whether the existing answers are similar to the answers given by this model; if not, match the newly proposed question with the answers in the existing knowledge base to determine whether there is a high degree of similarity; if still not satisfied, you can choose to determine whether it is similar through the public knowledge base on the Internet, and the internal knowledge base of the enterprise generally chooses manual review to determine whether to maintain the existing knowledge base. When judging the similarity, choose BERT based on the Transformer structure to calculate the text similarity.

[0113] Step S209: After verification, the qualified knowledge is output.

[0114] In summary, in the text processing method provided in the above embodiment, by setting multiple language models to collaborate on the problem text to be processed, it is helpful to improve the accuracy of the output knowledge text, and at the same time improve the accuracy and fluency of knowledge expression.

[0115] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides a computing device, such as Figure 4 As shown, the computing device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 4 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 4 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the above text processing method is implemented.

[0116] The computing device may also include: at least one network interface 312. The various components in the computing device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not used in the example above. Figure 4 Various buses are labeled as bus system 313.

[0117] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0118] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the computing device. Examples of these data include: any computer program used to operate on the computing device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player, a browser, etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0119] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. The computer-readable storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); it may also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer-readable storage medium is executed by the processor, the text processing method applied to the above-mentioned device is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0120] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0122] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A text processing method, characterized in that: include: Optimizing the question text to be processed based on the preset first language model to obtain the optimized question text to be processed; The optimized problem text to be processed is vectorized and matched with a preset knowledge base to determine a target knowledge vector; the preset knowledge base includes at least one knowledge vector; A matching detection is performed between the target knowledge vector and the question text to be processed based on a preset second language model, and the target knowledge vector is output when the detection passes.

2. The method according to claim 1, characterized in that: The step of optimizing the problem text to be processed based on the preset first language model to obtain the optimized problem text to be processed includes: Based on the first language model, the intent of the question text to be processed is classified to obtain a target intent; The problem text to be processed is optimized according to the target intention to obtain the optimized problem text to be processed.

3. The method according to claim 1, characterized in that The optimized problem text to be processed is vectorized and matched with a preset knowledge base to determine a target knowledge vector, including: Performing vectorization processing on the optimized question text to be processed to obtain a corresponding question vector; Matching the question vector with a preset knowledge base to obtain a similarity between the question vector and at least one knowledge vector in the preset knowledge base; The knowledge vector with the highest similarity and greater than a first preset threshold is determined as a target knowledge vector.

4. The method according to claim 3, characterized in that The step of vectorizing the optimized question text to be processed to obtain a corresponding question vector includes: Slicing the optimized problem text to be processed to obtain a word set; encoding the word set, and inputting the encoded word set into a preset word embedding model to obtain a feature word vector output by the preset word embedding model; Obtaining the cosine similarity between the feature word vector and at least one question vector in a preset question vector library; According to the cosine similarity, a question vector corresponding to the feature word vector is determined.

5. The method according to claim 4, characterized in that The step of slicing the optimized problem text to be processed to obtain a word set includes: Inputting the optimized problem text to be processed into the word embedding model to obtain the word embedding vector output by the preset word embedding model; Based on the word embedding vector, the optimized contextual relationship in the question text to be processed is calculated by the first language model to obtain the optimized global interaction characteristics of the question text to be processed; Based on the global interaction characteristics, a word set constituting at least one feature word of the optimized question text to be processed is determined.

6. The method according to claim 3, characterized in that: The method further comprises: If the similarities are all less than or equal to the first preset threshold, the optimized question text to be processed is processed by a preset third language model based on a third-party knowledge base, and the obtained processing result is output.

7. The method according to claim 1, characterized in that The matching detection between the target knowledge vector and the question text to be processed based on the preset second language model includes: Detecting the similarity between at least one knowledge data in a local database or a third-party knowledge base and the target knowledge vector, and determining that the target knowledge vector and the problem text to be processed have passed the detection when at least one of the similarities is greater than a second preset threshold; and / or, The similarity between the knowledge data and the problem text to be processed is detected, and when at least one of the similarities is greater than a third preset threshold, it is determined that the target knowledge vector and the problem text to be processed have passed the detection.

8. The method according to claim 1, characterized in that The method further comprises: When the detection fails, the question text to be processed is re-optimized based on a preset first language model.

9. A computing device, characterized in that include: A processor and a memory for storing executable instructions; wherein the processor is configured to execute the instructions to implement the text processing method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor, the text processing method according to any one of claims 1 to 8 is implemented.

Citation Information

Cited By

  • Knowledge base updating method and device, equipment and storage medium

    CN121327081A