A model reasoning method, device, equipment and medium suitable for question-answering scenarios

Abstract

The present application discloses a model reasoning method, device, equipment and medium suitable for question-answering scenarios, which relates to the field of artificial intelligence technology. The method includes utilizing the prior knowledge in the target corpus data and, through a preset word segmentation algorithm, constructing a mapping relationship between a new vocabulary consisting of generated target long words and the original vocabulary; when the pre-trained model infers and generates specified content, the final output result of the pre-trained model is quickly determined through a probability transfer model, which solves the problems of low precision, high model training cost and high computing cost in existing model acceleration methods. During the entire reasoning process, only an additional probability transfer model is added, which effectively reduces the pressure on computing resources and improves the generation speed of the entire system. In addition, making full use of the prior knowledge in domain question-answering can also improve the accuracy of domain-specific vocabulary generation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model reasoning method, apparatus, device, and medium suitable for question-answering scenarios. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, large language models (LLMs) have rapidly penetrated industrial and consumer scenarios from laboratory research. They have been deployed in a wide range of industries, including intelligent customer service, contract review, financial report generation, virtual assistants, medical consultations, and code generation. The speed and efficiency of model inference have become key factors influencing the implementation of this technology. In specific application scenarios, LLMs are generally deployed in cloud computing or edge devices (such as mobile phones). For scenarios with high real-time requirements, such as conversations, higher requirements are placed on inference latency and energy efficiency. However, the inference latency of models with hundreds of billions of parameters on conventional hardware can reach seconds to minutes. Therefore, how to quickly and efficiently perform model inference has become a pressing technical challenge. Summary of the Invention

[0003] The present application provides a model reasoning method, apparatus, device and medium suitable for question-answering scenarios, so as to at least solve the problems of low model acceleration accuracy, high model training cost and high computing cost in related technologies.

[0004] This application provides a model reasoning method suitable for question-answering scenarios, including:

[0005] Obtain the generated content of the pre-trained model; the pre-trained model is the model applied to the target question-answering scenario;

[0006] Matching the generated content with a pre-generated mapping table to obtain a matching result; wherein the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and the original vocabulary of the pre-trained model; the new vocabulary is a vocabulary generated using target long words, and the target long words are words whose length meets the preset long word determination criteria after word segmentation processing of the target corpus data in the target question-answering scenario based on the original vocabulary using a preset word segmentation algorithm;

[0007] When the matching result representation generates content that matches the mapping table, the pre-trained probability transfer model is called to determine the transfer probability of the target long word, and the output result of the pre-trained model is determined based on the transfer probability.

[0008] This application also provides a model reasoning device suitable for question-answering scenarios, including:

[0009] A generated content acquisition module is used to obtain the generated content of the pre-trained model; the pre-trained model is the model applied to the target question-answering scenario;

[0010] Generate a content matching module, which is used to match the generated content with a pre-generated mapping table to obtain a matching result; wherein the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and the original vocabulary of the pre-trained model; the new vocabulary is a vocabulary generated using target long words, and the target long words are words whose length meets the preset long word determination criteria after word segmentation processing of the target corpus data in the target question-answering scenario based on the original vocabulary using a preset word segmentation algorithm;

[0011] The output result determination module is used to call a pre-trained probability transfer model to determine the transfer probability of the target long word when the matching result representation generated content matches the mapping table, and determine the output result of the pre-trained model according to the transfer probability.

[0012] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned model reasoning methods applicable to question-answering scenarios when executing the computer program.

[0013] The present application also provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-mentioned model reasoning methods applicable to question-answering scenarios.

[0014] Through this application, since the pre-training model’s landing application projects are mostly vertical field questions and answers, the field data itself is a kind of prior knowledge. Therefore, the prior knowledge in the field question and answer can be fully utilized, and a mapping relationship is established between the new vocabulary composed of the generated target long words and the original vocabulary through a preset word segmentation algorithm; when the pre-training model infers and generates specified content, the final output result of the pre-training model is quickly determined through the probability transfer model. It can be seen that in the entire reasoning process, only an additional probability transfer model is added. Since the probability transfer model is a discriminant model, the parameter scale is much smaller than the language model. Therefore, it can effectively reduce the pressure on computing resources and increase the generation speed of the entire system. In addition, making full use of the prior knowledge in the field question and answer can also improve the accuracy of the generation of domain-specific vocabulary. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 This is a flowchart of a model reasoning method applicable to question-answering scenarios disclosed in this application;

[0017] Figure 2 A schematic diagram of a mapping relationship disclosed in this application;

[0018] Figure 3 A schematic diagram of the process of constructing a mapping relationship between a new vocabulary and an original vocabulary disclosed in this application;

[0019] Figure 4 A schematic diagram of the process of constructing a probabilistic transfer judgment model training set disclosed in this application;

[0020] Figure 5 A schematic diagram of a probabilistic transfer model training disclosed in this application;

[0021] Figure 6 This is a LLM accelerated reasoning flowchart disclosed in this application;

[0022] Figure 7 This is a schematic diagram of the structure of a model reasoning device suitable for question-answering scenarios disclosed in this application. DETAILED DESCRIPTION

[0023] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0025] In the current context of AI development, LLM models are widely used in various fields. However, as LLM capabilities improve, their parameter count is growing exponentially. For example, the GPT (Generative Pre-trained Transformer) series, a pre-trained language model, has seen model size increase from 117 million parameters in GPT-1 (2018) to approximately 1.8 trillion parameters in GPT-4 (2023). Consequently, LLM computational complexity (FLOPs) and memory usage have grown superlinearly, and inference costs have also risen sharply. For example, a single training session of GPT-3 costs over $4.6 million, and a single inference session consumes the equivalent of a 20-hour phone charge.

[0026] As LLM models become increasingly complex, the speed and efficiency of model reasoning have become key factors affecting the implementation of technology applications. Especially for scenarios with high real-time requirements such as conversations, how to quickly and efficiently perform model reasoning has become a technical challenge that needs to be solved urgently.

[0027] Currently, many inference acceleration technologies exist, including quantization, pruning, and distillation. Quantization reduces model weights and activations from FP32 / FP32 to INT8 / INT4, reducing memory usage and computational complexity. However, this method can result in a significant loss of accuracy. Pruning reduces the computational complexity of forward inference by trimming the model's inherent structure and removing redundant weights. This method also results in reduced accuracy and weakened model generalization. Knowledge distillation involves training a small model to fit the output distribution of a large model, and then using the small model instead of the large model for inference. However, this method is limited by the number of parameters in the small model, resulting in limited improvement.

[0028] Another method for accelerating inference is speculative sampling (Speculative Decoding). Its core idea is to use a small model to generate multiple tokens (basic units in text) and use a large model for verification and modification. This can reduce the number of calls to the large model and thus increase generation speed. In this method, the large model's computational workload is concentrated on long sequence verification rather than token-by-token generation. The final output is completely determined by the probability distribution of the large model, consistent with the results generated directly using the large model, without sacrificing generation quality. However, speculative sampling schemes have the following disadvantages:

[0029] (1) If the accuracy of the candidate generated by the small model is low, the rejection rate of the large model will increase, and it will be necessary to frequently fall back to the large model generation, resulting in a decrease in the acceleration effect or even negative optimization. Therefore, in order to ensure that the distribution probability of the two models is close, the current optimization solution is to train the two models on the same batch of data, but this will lead to additional training costs.

[0030] (2) Since the small model is also a generative model, too few parameters will directly affect the generation accuracy. Therefore, the parameter scale of the small model is only smaller than that of the large model. In fact, it still requires more than 6B parameters. The small model inference itself introduces additional computing costs, which may offset some of the acceleration benefits. In addition, the small model and the large model need to be loaded simultaneously during inference, which increases the memory usage and requires more computing resources.

[0031] To this end, this application provides a model reasoning solution suitable for question-answering scenarios, which can effectively reduce computing resource pressure, increase the generation speed of the entire system, and improve the accuracy of domain-specific vocabulary generation. To enable those skilled in the art to better understand this application solution, the application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0032] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the model reasoning method suitable for question-answering scenarios depends, the specific application environment architecture or specific hardware architecture is described here.

[0033] The embodiment of the present application provides a model reasoning method applicable to question-answering scenarios. Figure 1 As shown, the method is described in detail, and the method includes:

[0034] Step S11: Obtain the generated content of the pre-trained model; the pre-trained model is a model applied to the target question-answering scenario.

[0035] In the examples of this application, the pre-trained model is illustrated using a large language model (LLM) as an example. In real-world LLM applications, most are vertical question-answering. Therefore, the target question-answering scenarios are those targeting vertical fields, i.e., those within specific industries, professional fields, or niche scenarios, typically focusing on specific knowledge areas.

[0036] Step S12: Match the generated content with a pre-generated mapping table to obtain a matching result; wherein, the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and the original vocabulary of the pre-trained model; the new vocabulary is a vocabulary generated using target long words, and the target long words are words whose length meets the preset long word judgment conditions after word segmentation processing of the target corpus data in the target question and answer scenario based on the original vocabulary and using a preset word segmentation algorithm.

[0037] In an embodiment of the present application, a mapping table is pre-constructed, and the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and an original vocabulary of a pre-trained model.

[0038] In natural language processing (NLP) tasks, a raw vocabulary is a set of words used to represent the smallest unit of a corpus. It typically contains all possible words or characters and forms the basis for constructing word vectors, vocabulary, and subsequent model training. The raw vocabulary of LLM comes from the text corpus used in its pre-training phase, which typically comes from large-scale unlabeled data such as the internet, books, and articles.

[0039] The new vocabulary is a vocabulary generated using target long words, wherein the target long words are words whose lengths meet the preset long word determination conditions after the target corpus data in the target question-answering scenario is segmented based on the original vocabulary and using a preset word segmentation algorithm. Since the target question-answering scenario is a question-answering scenario for a vertical field, in the embodiment of the present application, the target corpus data is usually a vertical field corpus, that is, in the current question-answering scenario, a set of text data for a specific industry, professional field or segmented scenario is a priori knowledge. This type of corpus usually has a high degree of domain expertise, semantic concentration and application pertinence. By using a word segmentation algorithm to generate target long words based on the original vocabulary according to domain data, the construction of a new vocabulary is achieved.

[0040] Currently, mainstream word segmentation algorithms include BPE (Byte Pair Encoding) and BBPE (Byte-level Byte Pair Encoding). BPE is essentially a data compression method. It first requires a predefined character set and a set vocabulary size. New tokens are then generated by continuously merging the most frequent consecutive token pairs (the basic unit of text) in the corpus until the set vocabulary size is reached. BBPE, on the other hand, constructs sentences as sequences of UTF-8-encoded bytes (each byte is 8 bits) rather than character sequences. It uses a 256-byte set as its vocabulary and employs the same strategy as BPE, generating new tokens by continuously merging the most frequent consecutive token pairs in the corpus until the set vocabulary size is reached.

[0041] In the embodiment of the present application, BPE is used as an example to generate target long words. The implementation of other word segmentation algorithms is similar to this and will not be described in detail later.

[0042] In the embodiment of the present application, a newly constructed long word vocabulary is used to construct a mapping relationship between the tokens of the original vocabulary and the long words generated in the new vocabulary. Specifically, the words in the original vocabulary are used as keys, and the list of target long words prefixed with the keys in the new vocabulary is used as values ​​to construct a mapping relationship between the new vocabulary and the original vocabulary, so as to obtain a mapping table from the words in the original vocabulary to the target long words in the new vocabulary. Its format is: key: list <value>, where the key is the token in the original vocabulary, and the value is the list of generated long words in the new vocabulary, all of which are prefixed with the key.

[0043] It should be noted that the token (generated long word) in the new vocabulary must start with the token in the original vocabulary, and the long words in the new vocabulary must not exist in the original vocabulary. For example, if a certain token in the new vocabulary is "shadow bank", then there must be any one of "shadow", "shadow bank", or "shadow silver" in the original vocabulary, otherwise the long word will be discarded. As Figure 2 As shown is a simple mapping relationship. Among them, "cross" is the token in the original vocabulary, and "cross currency basis" etc. are the vocabulary in the new vocabulary.

[0044] Step S13: When the generated content of the matching result representation is matched with the mapping table, call the pre-trained probability transition model to determine the transition probability of the target long word, and determine the output result of the pre-trained model according to the transition probability.

[0045] In the embodiment of this application, the generated content of the matching result representation being matched with the mapping table means that the generated content of the LLM model matches the key in the mapping table. Further, it is necessary to judge through the probability transition model whether a certain value can be matched. That is, call the pre-trained probability transition model to determine the transition probability of the target long word. Then compare the transition probability with the classification decision threshold of the probability transition model. This classification decision threshold is used for the classification judgment of the model. If it is greater than this threshold, it means that the possibility of belonging to the positive sample is high enough, so it can be judged as a positive sample. [[ID=,13]]

[0046] Further, if the transition probability is greater than the classification decision threshold, it means that the model matches successfully, and it is considered that the model outputs the entire target long word at one time. Then directly generate the target long word and output the target long word as the output result of the pre-trained model, and then continue the subsequent reasoning; if the transition probability is not greater than the classification decision threshold, continue the reasoning through the pre-trained model, and when the generated content of the pre-trained model is matched with the key in the mapping table, jump to the step of calling the pre-trained probability transition model to determine the transition probability of the target long word.

[0047] In one feasible implementation, multimodal data can be leveraged to enhance the depth of domain knowledge understanding. For example, this can be achieved by incorporating modal information beyond text (e.g., diagrams, formulas, and symbols specific to specific domains) into probabilistic transfer models. For example, in the medical field, incorporating medical records with images and laboratory data as auxiliary features can enhance the model's decision-making process for long word generation, avoiding misjudgments caused by relying solely on textual context. This approach is applicable to complex scenarios requiring comprehensive information judgment and fills the gaps inherent in single text. In another feasible implementation, adversarial training mechanisms can be employed, introducing generative adversarial networks (GANs) or adversarial examples to enhance the robustness of probabilistic transfer models. For example, generating adversarial examples (e.g., deliberately obfuscated domain terms) forces the model to learn to distinguish between true long words and distractors, improving its accuracy in the presence of noisy input. This can enhance the model's adaptability to complex inputs and reduce the risk of inference errors caused by data bias or malicious input. Furthermore, a human-computer collaborative interface can be designed to allow domain experts to manually annotate and adjust the long word list or the decision logic of the probabilistic transfer model. By integrating expert knowledge, data or algorithm limitations can be overcome, improving the system's interpretability and customization capabilities.

[0048] Through this application, since the pre-training model’s landing application projects are mostly vertical field questions and answers, the field data itself is a kind of prior knowledge. Therefore, the prior knowledge in the field question and answer can be fully utilized, and a mapping relationship is established between the new vocabulary composed of the generated target long words and the original vocabulary through a preset word segmentation algorithm; when the pre-training model infers and generates specified content, the final output result of the pre-training model is quickly determined through the probability transfer model. It can be seen that in the entire reasoning process, only an additional probability transfer model is added. Since the probability transfer model is a discriminant model, the parameter scale is much smaller than the language model. Therefore, it can effectively reduce the pressure on computing resources and increase the generation speed of the entire system. In addition, making full use of the prior knowledge in the field question and answer can also improve the accuracy of the generation of domain-specific vocabulary.

[0049] Based on the above embodiment, this embodiment will specifically explain the process of generating target long words in the above embodiment. Figure 3 As shown, the following steps are included:

[0050] Based on the original vocabulary of the pre-trained model and using the preset word segmentation algorithm, the target corpus data is segmented to obtain different word segmentation results;

[0051] Count the frequency of each unit pair consisting of adjacent word segmentation results in the target corpus data, and iteratively merge the unit pairs with the highest frequency to generate new long words;

[0052] The new long words are screened according to the preset screening rules, and after the generated new word table reaches the preset new word table size based on the number of new long words, the target long word is determined.

[0053] When training an LLM, a tokenizer converts text into tokens, which are then encoded into vectors for subsequent calculations. The tokenizer primarily consists of two parts: a vocabulary (a mapping between token_ids and tokens) and a word segmentation algorithm. In this embodiment, the default word segmentation algorithm is described using the BPE algorithm as an example.

[0054] First, input the target corpus data, vocabulary size, and original vocabulary. The vocabulary size is the pre-set initial vocabulary size. Then, based on the original vocabulary, the target corpus data is broken down into smallest units, yielding different segmentation results. "Smallest units" here refer to the target corpus data being broken down into irreparable characters / subwords, based on the tokens in the original vocabulary.

[0055] Furthermore, according to the implementation logic of the BPE algorithm, the frequency of adjacent unit pairs within a word in the corpus is counted, and the most frequent token pairs are selected and merged to generate new long words. Taking the medical field as an example, suppose a medical text in the target corpus data is: "Immune cell therapy can improve diabetes." The original vocabulary is basic medical vocabulary, such as {"cell", "immunity", "treatment", "diabetes", "disease"}. The corpus is then split into {"immunity", "cell", "treatment", "diabetes", "disease", "disease"}, and the adjacent unit pairs include {"immunity", "cell"}, {"cell", "treatment"}, {"treatment", "can"}, {"can", "improve"}, {"improve", "diabetes"}, {"diabetes", "disease"}, and {"disease", "disease"}. If {"immunity", "cell"} has the highest occurrence frequency, {"immunity", "cell"} are merged to generate "immune cell", and the subsequent split result becomes {"immune cell", "treatment", "diabetes", "disease", "disease"}.

[0056] After merging to generate a new long word, the merged tokens are filtered and used as tokens in the new vocabulary. The filtering rules are as follows:

[0057] Determine whether the new long word appears in the original vocabulary in turn to filter out the new long words that are not in the original vocabulary; that is, the newly generated token must not be in the original vocabulary;

[0058] Determine whether the unit pairs contain punctuation symbols in turn to filter out new long words that do not contain punctuation symbols in the unit pairs; that is, the newly generated token cannot cross sentences and cannot contain punctuation symbols such as (.,?!;).

[0059] Repeat the above iterative merging and filtering process until the size of the current vocabulary reaches the pre-set vocabulary size. It should be noted that the new vocabulary is merged based on the original vocabulary, and based on the new vocabulary, a mapping relationship between the original vocabulary and the new vocabulary can be established.

[0060] In the embodiments of the present application, the number of new vocabularies is now predetermined and is strongly correlated with the data in the vertical domain. Given the data scale and data quality of the vertical domain, if the number of new vocabularies is set too large, a large number of words will be generated, and the probability of matching the generated words when using LLM reasoning will increase. However, the length of the generated words will be relatively short, and the acceleration effect during model reasoning may not be obvious. For example, when constructing the vocabulary, some commonly used two-word combinations or three-word combinations are easily included in the vocabulary because of their high frequency. In this way, even if there are long words, they may be composed of several short parts. High-frequency short words and some commonly used combinations will be preferentially selected, making it difficult for long words to become longer. However, if the vocabulary is set too small, the model will have a limited range of options when generating words. In order to express various semantics, it can only form long words by combining more small words. This may cause the long words to be too long, reducing the probability of matching long words during LLM reasoning, and negatively affecting the reasoning speed.

[0061] Therefore, based on the above considerations, when setting the size of the new vocabulary, it is necessary to comprehensively consider the inference speed and matching success rate. Therefore, the recommended reference values ​​are: , that is, 1 / 2 of the number of new vocabularies after the second iteration screening.

[0062] Specifically, execute two iterations of the preset word segmentation algorithm, and record the number of words contained in the new word list when the new word list is generated after the second iteration; set half of the number of words as the size of the new word list, and continue to execute the iterations of the preset word segmentation algorithm. When the new word list generated based on the number of new long words reaches the size of the new word list, determine the target long word.

[0063] In the embodiment of the present application, after setting the initial size of the new vocabulary, a second screening is performed after the first screening. The number of words left after the second screening is the number of words in the new vocabulary after the second iterative screening. The vocabulary size at this time is the recommended final vocabulary size.

[0064] As can be seen, this embodiment fully leverages prior knowledge from vertical domain question-answering. Using word segmentation algorithms like BPE, a mapping relationship (tokens to long words, phrases, and proprietary terms) is established between the domain data and the existing LLM vocabulary. This allows LLM to quickly determine whether a match is successful using a probabilistic transition judgment model when generating a specific key. This allows for the generation of multiple tokens simultaneously, improving the overall system's generation speed.

[0065] Based on the above embodiments, in a feasible implementation manner, the process of training the probability transfer model may include the following steps:

[0066] Traverse the target corpus data based on the mapping table to extract samples, so as to obtain the context of the key and value appearing in the target corpus data;

[0067] Replace the position where the key appears in the context with the target long word in the value to generate positive samples, and keep the position where the key appears in the context unchanged to generate negative samples;

[0068] Count the number of positive samples and negative samples generated under each mapping relationship in the mapping table, and determine the proportional coefficient of the negative samples based on the number of positive samples and negative samples;

[0069] If the proportional coefficient is lower than the preset threshold, extract data containing the same key from the general domain corpus to supplement the negative samples until the proportional coefficient is not lower than the preset threshold, and then determine the current negative samples; the general domain corpus is a collection of text data for different domains in different question-and-answer scenarios;

[0070] Construct a binary classification data set using the current negative samples and positive samples, and train the probability transfer model based on the binary classification data set.

[0071] Figure 4 In the embodiments of the present application, the training data is first constructed, and the construction process of the entire training set is as shown. Based on the long word table mapping, samples are extracted from the vertical corpus; that is, the context of the mapping relationship key and value is searched. Taking {"shadow": ["shadow bank"]} as an example, from the target corpus data, find all the context containing "shadow" and "shadow bank". The specific examples are as follows:

[0072] The darker area formed because the object blocks the propagation of light and cannot pass through the opaque object is what we often call a shadow;

[0073] All in all, all deposit and loan-like businesses that游离于银行监管体系之外 can be called shadow banks.

[0074] Replace the key with the value to construct a binary classification data set. The example is shown in Table 1:

[0075] Table 1 Binary classification data set

[0076]

[0077] Understandably, when constructing a binary classification dataset, negative samples are original samples without replacing the key (that is, retaining the key without replacing it with the value). For highly domain-specific data, the key is often a domain-specific term. These terms may appear too frequently (e.g., "cell"), resulting in insufficient negative sample richness. For example, a corpus may only contain descriptions of shadow banking, with no descriptions of shadows, images, or other related terms. Alternatively, if the key is "cell" and the value is "immune cell," in medical data, the context of "cell" may frequently contain domain terms such as "immunity" or "treatment." This results in negative samples (where "cell" is not replaced) naturally occurring very rarely in the domain data (because domain expressions tend to use the full term "immune cell"). Therefore, to increase the richness of negative samples, additional negative samples need to be sampled from general data. General domain corpora have a more balanced vocabulary distribution, cover multiple domains, and support common scenarios (such as search engines and social media).

[0078] It is important to note that for any mapping relationship, the ratio of positive and negative samples should be saved and used as the threshold for subsequent discriminant model judgment. In other words, if the number of negative samples is 4:1, then the probability y output by the model should be thresholded at 4 / (4+1), that is:

[0079] ;

[0080] An example of a threshold mapping format is: {shadow bank: 0.8}. Traversing all mapping relationships constructs a binary classification dataset; in addition, all samples should be within 510 tokens.

[0081] Furthermore, considering the accuracy and computational efficiency, the Bert-base structure is selected in this embodiment to fine-tune the probability transfer model. However, this is not the only implementation method of the present invention, and the implementation of other models should all be within the scope of protection of the present invention. The training process is shown in the attached figure. Figure 5 As shown, the sample is prefixed with [CLS] and suffixed with [SEP]. [CLS] is the abbreviation of "classification", which usually indicates the beginning of a sentence or document. In BERT, [CLS] corresponds to the first word in the input text; [SEP] is the abbreviation of "separator", which usually indicates the end of a sentence or document. In BERT, [SEP] corresponds to the word vector of the last word in the input text, and its function is to separate different sentences. For example, when processing sentence pairs in BERT, a [SEP] is usually inserted between the two sentences to indicate their dividing point. The final output is the output corresponding to [CLS]. After passing through the pooler layer and softmax, it becomes the final probability. The category can be determined using the threshold saved when building the training set.

[0082] Specifically, the probability transfer model is called to determine the transfer probability of the target long word, including: determining the input sequence of the probability transfer model according to the target long word, and according to the mark symbols in the input sequence, determining the transfer probability of the target long word after the input sequence is sequentially passed through the pooling layer and the activation function calculation.

[0083] Furthermore, the entire process of LLM acceleration generation is as shown in the attached Figure 6 As shown in the figure, when performing vertical field question answering, when a token generated by the LLM matches the key of the mapping table, the above context and the corresponding word are thrown to the probability judgment model for judgment. If the probability is greater than the specified threshold, it is considered that the model has output the entire target long word at one time. Otherwise, it is thrown to the LLM for further reasoning.

[0084] It can be seen that in this embodiment, the training data of the probability transfer judgment model is fully constructed based on the domain data, so that the model can quickly judge whether the next token can be the target long word based on the previous context, and because the text of the domain data is obviously different from that of the general data, the accuracy of the judgment model is very high, which greatly improves the accuracy of LLM in domain question answering. During the entire reasoning process, only an additional probability transfer model is added. Since the probability transfer model is a discriminant model, the parameter scale is much smaller than that of the language model, which can effectively reduce the pressure on computing resources. It can be used in scenarios such as question recommendation, customer service Q&A, article writing, and information extraction.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0086] The embodiment of the present application also provides a model reasoning device suitable for question-answering scenarios, see Figure 7 As shown, the device includes:

[0087] Generated content acquisition module 11, used to obtain the generated content of the pre-trained model; the pre-trained model is a model applied to the target question-answering scenario;

[0088] A content matching module 12 is generated, which is used to match the generated content with a pre-generated mapping table to obtain a matching result; wherein the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and an original vocabulary of a pre-trained model; the new vocabulary is a vocabulary generated using target long words, and the target long words are words whose length meets the preset long word determination condition after word segmentation processing of the target corpus data in the target question-answering scenario based on the original vocabulary using a preset word segmentation algorithm;

[0089] The output result determination module 13 is used to call a pre-trained probability transfer model to determine the transfer probability of the target long word when the matching result representation generated content matches the mapping table, and determine the output result of the pre-trained model according to the transfer probability.

[0090] For the description of the features in the embodiment corresponding to the model reasoning device applicable to the question-answering scenario, please refer to the relevant description of the embodiment corresponding to the model reasoning method applicable to the question-answering scenario, which will not be repeated here.

[0091] In a feasible implementation, the model reasoning device applicable to the question-answering scenario further includes:

[0092] Target long word generation module, used for:

[0093] Based on the original vocabulary of the pre-trained model and using the preset word segmentation algorithm, the target corpus data is segmented to obtain different word segmentation results;

[0094] Count the frequency of each unit pair consisting of adjacent word segmentation results in the target corpus data, and iteratively merge the unit pairs with the highest frequency to generate new long words;

[0095] The new long words are screened according to the preset screening rules, and after the generated new word table reaches the preset new word table size based on the number of new long words, the target long word is determined.

[0096] In a feasible implementation, the target long word generation module further includes:

[0097] The rule screening unit is specifically used to: determine in turn whether the new long word appears in the original vocabulary to screen out the new long words that are not in the original vocabulary; determine in turn whether the unit pair contains a punctuation symbol to screen out the new long words that do not contain a punctuation symbol in the unit pair.

[0098] The long word generation unit is specifically used to: execute two iterative operations of the preset word segmentation algorithm, and record the number of words contained in the new word list when the new word list is generated after the second iteration; set half of the number of words as the size of the new word list, and continue to execute the iterative operation of the preset word segmentation algorithm, and determine the target long word when the new word list generated based on the number of new long words reaches the size of the new word list.

[0099] In a feasible implementation, the model reasoning device applicable to the question-answering scenario further includes:

[0100] The mapping relationship construction module is used to: use the words in the original vocabulary as keys and the list of target long words in the new vocabulary with the keys as prefixes as values ​​to construct a mapping relationship between the new vocabulary and the original vocabulary, so as to obtain a mapping table that maps the words in the original vocabulary to the target long words in the new vocabulary.

[0101] Model training module, used for:

[0102] Traverse the target corpus data based on the mapping table to extract samples to obtain the context in which the key and value appear in the target corpus data;

[0103] Replace the position where the key appears in the above text with the target long word in the value to generate a positive sample, and keep the position where the key appears in the above text unchanged to generate a negative sample;

[0104] Counting the number of positive samples and negative samples generated under each mapping relationship in the mapping table, and determining the negative sample ratio coefficient based on the number of positive samples and the number of negative samples;

[0105] If the ratio coefficient is lower than the preset threshold, data containing the same key is extracted from the general domain corpus to supplement the negative sample. This process is repeated until the ratio coefficient is no lower than the preset threshold, and the current negative sample is determined. The general domain corpus is a collection of text data for different fields in different question-and-answer scenarios.

[0106] Use the current negative samples and positive samples to construct a binary classification data set, and train the probability transfer model based on the binary classification data set;

[0107] Accordingly, the output result determination module 13 is specifically configured to:

[0108] The input sequence of the probability transfer model is determined according to the target long word, and the input sequence is sequentially passed through the pooling layer and the activation function according to the mark symbols in the input sequence to determine the transfer probability of the target long word.

[0109] In a feasible implementation, the output result determination module 13 is specifically configured to:

[0110] When the generated content of the matching result representation matches the key in the mapping table, the pre-trained probability transfer model is called to determine the transfer probability of the target long word;

[0111] determining a classification decision threshold of the probabilistic transition model and comparing the transition probability with the classification decision threshold;

[0112] If the transition probability is greater than the classification decision threshold, the target long word is output as the output result of the pre-training model;

[0113] If the transition probability is not greater than the classification decision threshold, the reasoning continues through the pre-trained model, and when the generated content of the pre-trained model matches the key in the mapping table, the process jumps to the step of calling the pre-trained probability transition model to determine the transition probability of the target long word.

[0114] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned model reasoning method embodiments applicable to question-answering scenarios.

[0115] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned model reasoning method embodiments applicable to question-answering scenarios when running.

[0116] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0117] An embodiment of the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned model reasoning method embodiments applicable to question-answering scenarios.

[0118] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned model reasoning method embodiments applicable to question-answering scenarios.

[0119] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] The above is a detailed introduction to the model reasoning method, device, equipment and medium provided by this application and suitable for question-answering scenarios. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.< / value>

Claims

1. A model reasoning method suitable for question-answering scenarios, characterized in that: include: Get the generated content of the pre-trained model; The pre-trained model is a model applied to the target question-answering scenario; Matching the generated content with a pre-generated mapping table to obtain a matching result; wherein the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and an original vocabulary of the pre-trained model; the new vocabulary is a vocabulary generated using target long words, and the target long words are words whose lengths meet preset long word determination conditions after word segmentation processing of the target corpus data in the target question-answering scenario based on the original vocabulary and using a preset word segmentation algorithm; the target long words in the new vocabulary start with words in the original vocabulary, and the target long words do not exist in the original vocabulary; When the matching result indicates that the generated content matches the mapping table, calling a pre-trained probability transition model to determine the transition probability of the target long word, and determining an output result of the pre-trained model according to the transition probability; Calling the probability transition model to determine the transition probability of the target long word includes: An input sequence of the probability transfer model is determined according to the target long word, and according to the marking symbols in the input sequence, the input sequence is sequentially passed through a pooling layer and an activation function to determine the transfer probability of the target long word.

2. The model reasoning method applicable to question-answering scenarios according to claim 1, characterized in that: The process of generating the target long word includes: Based on the original vocabulary of the pre-trained model and using a preset word segmentation algorithm, the target corpus data is segmented to obtain different word segmentation results; Counting the frequency of occurrence of each unit pair consisting of adjacent word segmentation results on the target corpus data, and iteratively merging the unit pairs with the highest occurrence frequency to generate new long words; The new long words are screened according to a preset screening rule, and after the generated new word table is determined to have reached a preset new word table size based on the number of the new long words, the target long word is determined.

3. The model reasoning method applicable to question-answering scenarios according to claim 2, characterized in that: The screening of the new long words according to the preset screening rules includes: determining in sequence whether the new long words appear in the original word list, so as to filter out the new long words that are not in the original word list; It is determined in turn whether the unit pairs contain a punctuation symbol, so as to filter out new long words that do not contain the punctuation symbol in the unit pairs.

4. The model reasoning method applicable to question-answering scenarios according to claim 2, characterized in that: After the new word list generated based on the number of the new long words reaches a preset new word list size, determining the target long word includes: Executing two iterations of the preset word segmentation algorithm, and recording the number of words included in the new word list when the new word list is generated after the second iteration; One half of the number of words is set as the size of the new vocabulary table, and the iterative operation of the preset word segmentation algorithm is continued to be executed. When the new vocabulary table generated based on the number of the new long words reaches the new vocabulary table size, the target long word is determined.

5. The model reasoning method applicable to question-answering scenarios according to claim 1, characterized in that: The process of constructing the mapping table includes: Using the words in the original vocabulary as keys and the list of target long words in the new vocabulary with the keys as prefixes as values, a mapping relationship between the new vocabulary and the original vocabulary is constructed to obtain a mapping table that maps the words in the original vocabulary to the target long words in the new vocabulary.

6. The model reasoning method applicable to question-answering scenarios according to claim 5, characterized in that: The process of training the probabilistic transfer model includes: Traversing the target corpus data based on the mapping table to perform sample extraction to obtain contexts where the key and the value appear in the target corpus data; Replacing the position where the key appears in the above text with the target long word in the value to generate a positive sample, and keeping the position where the key appears in the above text unchanged to generate a negative sample; Counting the number of positive samples and the number of negative samples generated under each mapping relationship in the mapping table, and determining a proportion coefficient of the negative samples based on the number of positive samples and the number of negative samples; If the proportional coefficient is lower than a preset threshold, data containing the same key is extracted from a general domain corpus to supplement the negative sample, and the current negative sample is determined until the proportional coefficient is not lower than the preset threshold; the general domain corpus is a collection of text data for different fields in different question-and-answer scenarios; A binary classification data set is constructed using the current negative sample and the positive sample, and the probability transfer model is trained based on the binary classification data set.

7. The model reasoning method applicable to question-answering scenarios according to any one of claims 1 to 6, characterized in that: When the matching result indicates that the generated content matches the mapping table, calling a pre-trained probability transition model to determine the transition probability of the target long word, and determining an output result of the pre-trained model according to the transition probability, includes: When the matching result indicates that the generated content matches the key in the mapping table, calling a pre-trained probability transition model to determine the transition probability of the target long word; determining a classification decision threshold of the probabilistic transition model, and comparing the transition probability with the classification decision threshold; If the transition probability is greater than the classification decision threshold, outputting the target long word as the output result of the pre-training model; If the transition probability is not greater than the classification decision threshold, the reasoning continues through the pre-trained model, and when the generated content of the pre-trained model matches the key in the mapping table, the process jumps to the step of calling the pre-trained probability transition model to determine the transition probability of the target long word.

8. A model reasoning device suitable for question-answering scenarios, characterized in that: include: Generated content acquisition module, used to obtain the generated content of the pre-trained model; The pre-trained model is a model applied to the target question-answering scenario; Generate a content matching module, configured to match the generated content with a pre-generated mapping table to obtain a matching result; wherein the mapping table is a table generated by constructing a mapping relationship between a new vocabulary and an original vocabulary of the pre-trained model; the new vocabulary is a vocabulary generated using target long words, wherein the target long words are words whose lengths meet preset long word determination conditions after word segmentation processing of the target corpus data in the target question-answering scenario based on the original vocabulary and using a preset word segmentation algorithm; the target long words in the new vocabulary begin with words in the original vocabulary, and the target long words do not exist in the original vocabulary; an output result determination module, configured to, when the matching result indicates that the generated content matches the mapping table, call a pre-trained probability transition model to determine a transition probability of the target long word, and determine an output result of the pre-trained model based on the transition probability; Accordingly, the output result determination module is specifically used to: An input sequence of the probability transfer model is determined according to the target long word, and according to the marking symbols in the input sequence, the input sequence is sequentially passed through a pooling layer and an activation function to determine the transfer probability of the target long word.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the model reasoning method applicable to the question-answering scenario as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the model reasoning method applicable to the question-answering scenario as described in any one of claims 1 to 7 are implemented.

Images