Vertical domain model reasoning acceleration method and device

Abstract

The present application relates to the field of artificial intelligence technology, and provides a method, device, electronic device and readable storage medium for accelerating the reasoning of a vertical domain model. The method includes: obtaining a target text; obtaining a data text corresponding to the target text in a pre-set text library; establishing a prefix index library based on the target text and the data text; reasoning the target text and the data text through a large language model, and when the number of reasoning is equal to the preset number, determining a matching prefix based on the result of the reasoning; matching the matching prefix with the prefix index library, and if the match is successful, obtaining a data fragment consistent with the prefix of the matching prefix as the target data fragment; decomposing the data fragment into a reasoning sequence based on the matching prefix and the target data fragment; performing concurrent reasoning on the reasoning sequence to obtain a character set; and determining the currently output text based on the character set. The present application solves the technical problem of the slow reasoning process of the vertical domain model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a vertical domain model reasoning acceleration method, device, electronic device and readable storage medium. Background Art

[0002] The Large Language Model (LLM) is a generative model based on Transformer. Although LLM has good problem-solving and dialogue response capabilities within the general knowledge range, it is usually unable to directly obtain good response effects in vertical scenarios in specific fields. Existing technologies are mainly divided into further training and style alignment in the vertical domain, and providing enough examples to enable LLM to obtain good enough expressiveness in the vertical domain by learning in the context. However, the data cost required for training in the vertical domain is very high, and because of the number of parameters and the characteristics of the autoregressive structure of LLM, each decoding step depends on the token outputted previously, which results in each step having to wait for the result of the previous reasoning, resulting in a slow reasoning process.

[0003] It can be seen that the existing technology has the problem of slow vertical domain model reasoning process. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a vertical domain model reasoning acceleration method, device, electronic device and readable storage medium to solve the problem of slow vertical domain model reasoning process in the prior art.

[0005] A first aspect of an embodiment of the present application provides a method for accelerating vertical domain model reasoning, including:

[0006] Get the target text;

[0007] Obtain the data text corresponding to the target text in the pre-set text library;

[0008] Based on the target text and the data text, a prefix index library is established, which includes the data fragments corresponding to the data text and the target text;

[0009] The target text and the data text are inferred by the large language model. When the number of inferences is equal to the preset number, the matching prefix is ​​determined based on the inference result. The preset number is a positive integer, and the inference result is all the target characters output within the preset number.

[0010] The matching prefix is ​​matched with the prefix index library. If the match is successful, the data segment that is consistent with the prefix of the matching prefix is ​​obtained as the target data segment;

[0011] Decomposing the data fragment into an inference sequence according to the matching prefix and the target data fragment;

[0012] Perform concurrent reasoning on the reasoning sequence to obtain a character set, where the character set includes the characters obtained by reasoning in each reasoning sequence;

[0013] Determine the text currently being output based on the character set.

[0014] A second aspect of an embodiment of the present application provides a vertical domain model reasoning acceleration device, comprising:

[0015] A first acquisition module is used to acquire the target text;

[0016] The second acquisition module is used to obtain the data text corresponding to the target text in the pre-set text library;

[0017] A determination module is used to establish a prefix index library based on the target text and the data text;

[0018] The first inference module is used to infer the target text and the data text using the large language model. When the number of inferences is equal to a preset number, a matching prefix is ​​determined based on the inference result. The preset number is a positive integer, and the inference result is all target characters output within the preset number.

[0019] The matching module is used to match the matching prefix with the prefix index library. If the match is successful, the data segment that is consistent with the prefix of the matching prefix is ​​obtained as the target data segment;

[0020] a decomposition module for decomposing the data fragment into an inference sequence according to the matching prefix and the target data fragment;

[0021] The second reasoning module is used to perform concurrent reasoning on the reasoning sequence to obtain a character set, where the character set includes the characters obtained by reasoning in each reasoning sequence;

[0022] The output module is used to determine the current output text based on the character set.

[0023] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0024] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0025] The beneficial effects of the embodiments of the present application include at least:

[0026] Ensure that the model responds to the latest user input and provides immediate reasoning results; use pre-trained encoders to quickly retrieve data text that is highly relevant to the user's current input, which can improve the relevance and accuracy of the model's response; the prefix index library can quickly retrieve matching data fragments during the generation process, thereby improving reasoning efficiency; using a large language model for reasoning generation can make full use of its pre-trained knowledge to obtain high-quality text output; when the reasoning process reaches a preset number of times, determine whether the string matches the prefix of a certain data fragment, which can reduce unnecessary reasoning and quickly obtain reliable generation continuation; continuing to generate according to the data fragment can reduce the amount of text that needs to be generated each time, further improving the reasoning speed; using the concurrent processing capabilities of the hardware, multiple reasoning sequence sets can be processed simultaneously, thereby significantly reducing the overall time required; the final output text is obtained by integrating the concurrent reasoning results, which means that the output text can be generated faster, solving the technical problem of the slow reasoning process of the vertical field model. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 This is a flowchart of a vertical domain model reasoning acceleration method provided by an embodiment of the present application;

[0029] Figure 2 This is a schematic diagram of the concurrent reasoning process of a vertical domain model reasoning acceleration method provided in an embodiment of the present application;

[0030] Figure 3 This is a structural diagram of a vertical domain model reasoning acceleration device provided in an embodiment of the present application;

[0031] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0033] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the data used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects connected before and after are in an "or" relationship.

[0034] In addition, it should be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, the elements defined by the phrase "comprises..." do not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the elements.

[0035] A vertical domain model reasoning acceleration method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 This is a flow chart of a vertical domain model reasoning acceleration method provided by an embodiment of the present application. This method can be executed by a terminal device or a server. Figure 1 As shown, the vertical domain model reasoning acceleration method includes:

[0037] Step 101: Obtain target text.

[0038] The target text refers to the text that needs to be inferred and matched. It is the text that we hope to obtain relevant information or draw conclusions through this method.

[0039] Specifically, the target text can be a question, a query, a requirement, or a task to be solved. For example, for an intelligent search engine, the search term entered by the user is the target text; for an automatic question-answering system, the question asked by the user is the target text.

[0040] Step 102: Obtain the document text corresponding to the target text in a preset text library.

[0041] The purpose of obtaining the reference text corresponding to the target text in the text library is to provide further information and context to support reasoning and matching of the target text.

[0042] By obtaining the source text corresponding to the target text, richer contextual information can be provided for the reasoning process. The content in the source text can be used to supplement, explain, or expand the information in the target text, making the reasoning results more accurate and complete. The source text provides an additional data source that can be used in the reasoning and matching process. By matching with the target text, relevant data fragments can be found to better understand and answer user questions or achieve other specific tasks. The source text is crucial for verifying and confirming the accuracy and reliability of the reasoning results. By introducing the support of the source text, the reasoning process can be verified and validated, ensuring that the output results are more accurate and reliable. Obtaining the source text corresponding to the target text can provide more comprehensive and detailed answers or solutions, thereby improving the quality of the model output content.

[0043] By obtaining reference text corresponding to the target text in the text library, the effectiveness and quality of the inference process are enhanced, accuracy and reliability are improved, and a better user experience is provided. It provides more contextual information and data support for the model, thereby enhancing its practicality and effectiveness.

[0044] Step 103: Create a prefix index library based on the target text and the document text.

[0045] The prefix index library includes the data fragments corresponding to the source text and the target text. For each keyword, a prefix index is constructed. A prefix index is a data structure that helps quickly find keywords that begin with a prefix. Common prefix index structures include the Trie and the Compressed Trie.

[0046] Map each keyword's prefix index to the corresponding text fragment. This allows you to quickly locate text fragments containing the keyword based on the keyword's prefix index during searches. Store all text fragments and their index mappings in an index repository. For example, create a prefix tree or prefix matching array for the target document.

[0047] Establishing a prefix index library can significantly accelerate the matching and retrieval of target text and reference text. Using a prefix index, you can quickly locate potentially matching text segments based on keyword prefixes, reducing the time and computational cost of traversing the entire text library. A prefix index library provides more accurate matching results. By mapping keyword prefix indexes to text segments, you can precisely find text segments containing keywords, rather than relying on fuzzy matching or full-text search to retrieve relevant text. Using a prefix index library can better supplement the contextual information of the target text. By matching keywords, you can find reference text segments related to the target text, providing more background knowledge and detailed information, making the reasoning process more complete and accurate. Establishing a prefix index library provides fundamental support for reasoning and application tasks. Through fast retrieval and precise matching, you can better understand user questions or needs and provide relevant answers, solutions, or support. Establishing a prefix index library can significantly improve the efficiency and performance of matching and retrieval. Compared to traditional full-text searches or traversal matching methods, a prefix index library can complete matching tasks in a shorter time, improving system responsiveness and overall efficiency. It can be seen that establishing a prefix index library can speed up matching and retrieval, provide accurate matching results, supplement contextual information, support reasoning and application tasks, and improve the efficiency and performance of the system.

[0048] Step 104 : Inferring the target text and the data text using the large language model. When the number of inferences is equal to a preset number, a matching prefix is ​​determined based on the inference result.

[0049] The preset number of times is a positive integer, and the inference result is all target characters output within the preset number of times. The Large Language Model (LLM) is an autoregressive language model. During the inference phase, due to the dependencies of autoregressive decoding, each decoding step depends on the previously output token. Therefore, only one token is output per time step. The inference frequency is one per time step.

[0050] By performing reasoning through a large language model and determining matching prefixes based on the inference results, we can improve matching accuracy, accelerate search speed, improve system efficiency, and enhance the practicality and benefits of various application scenarios.

[0051] Step 105: Match the matching prefix with the prefix index library. If the match is successful, a data segment with the same prefix is ​​obtained as the target data segment.

[0052] Specifically, the prefix index is a data structure used to store text snippets and their prefix information. Each text snippet is associated with its corresponding prefix and stored in the prefix index. When a matching prefix is ​​found, the prefix index can be searched for to quickly find the matching text snippet.

[0053] If the match is successful, the required information can be extracted from the matched text segment or further processed. In this way, by matching the prefix and the prefix index library, it is possible to conveniently obtain the data segments that are consistent with the target prefix and perform subsequent operations.

[0054] Step 106: Decompose the data segment into an inference sequence according to the matching prefix and the data segment.

[0055] The purpose of decomposing the data segment into an inference sequence is to split the longer text into smaller parts so that the hardware concurrency capabilities can be utilized during the inference process to input the model for concurrent processing.

[0056] As an example, assume that when the LLM is performing inference, at the first time step, the inferred token is "light", and at the second time step, the inferred current token is "water". We take the output of the previous n - 1 time steps to form a string (for example, when n = 2, this string is "fresh water"). By utilizing the characteristics of the prefix tree or prefix array, we perform a match in the pre-maintained prefixes of the data segments to confirm whether there is a prefix that is consistent with this string. If a match is found, we will consider that this data segment is very likely to be fully reused by the LLM as highly credible and highly relevant information. Then we copy this segment completely (for example, "Fresh water is a limited resource") one by one into the subsequent input of the LLM. Each time a new token is added, it will form a new example in the batch, such as ["Fresh water", "Fresh water is", "Fresh water is a"...]. In this way, the LLM can complete the operation of续写 (continuing to write) one token for all the texts in this batch in one processing step through the hardware batch processing (batch) calculation ability.

[0057] Step 107: Perform concurrent inference on the inference sequence to obtain a character set.

[0058] Among them, the character set includes the characters inferred from each inference sequence; the character set can be further used for subsequent processing, such as combining the characters of different sequences to form longer text segments or sentences.

[0059] By inputting the inference sequence into the large language model for concurrent inference, this concurrent inference method can improve the inference efficiency and ensure that each inference sequence can obtain corresponding prediction results.

[0060] During the concurrent inference process, multiple inference sequences can be input into the large language model simultaneously. The model will make predictions based on the context information of each sequence to generate the next character. Through concurrent inference, the computing power of the hardware can be fully utilized to process multiple inference sequences in parallel, accelerating the inference speed.

[0061] Step 108: Determine the text currently being output according to the character set.

[0062] Specifically, based on the generated character set, the current output text can be determined. The predicted characters of each inference sequence can be combined according to their order in the sequence to form a complete text segment or sentence.

[0063] In addition, after the reasoning is completed for all input texts, the output text will be output as a complete sentence. After the output is completed, the currently established index library will be deleted.

[0064] According to the technical solution provided in the embodiment of the present application, by utilizing the characteristics of continuous training in the vertical domain, semantic coding is obtained for automatic retrieval of relevant information, and when there is data input, LLM reuses and overlaps the data, and performs a skip-step design during reasoning, giving full play to the characteristics of parallel computing of current hardware resources, and accelerating the reasoning of single-case data. The method of the present application can achieve fast and accurate text reasoning in vertical fields. The present application achieves efficient processing and reasoning of target text and data text by using technical means such as reasoning with a large language model, establishing a prefix index library to accelerate matching, and concurrent reasoning; it solves the technical problem of the slow reasoning process of the vertical field model.

[0065] In some embodiments, before obtaining the document text corresponding to the target text in the preset text library, the method further includes:

[0066] Obtain first document text in a vertical field; segmentally encode the first document text using a pre-trained encoder to obtain a first vector, wherein the first vector corresponds to a segment of the first document text; and obtain a text library based on the document text and the first vector.

[0067] First, prepare a database of data texts for the specific vertical domain. This database does not require any annotation, but must be high-quality, referenceable, and suitable for external use without any permissions issues. An encoder model, such as BERT, is then required to perform further vertical domain pre-training on this prepared database. This involves using MLM mask training to allow encoders like BERT to gain a deeper understanding of the language of the relevant vertical domain, enabling them to adjust their pre-trained semantic space and appropriately encode the vertical domain text. The pre-trained encoder is then used to segment the database, generating semantic vectors for the segmented texts and storing them in the database along with the original text.

[0068] By introducing relevant first-source text and segmented coding, this embodiment allows the model to better understand contextual information and generate more accurate and coherent text. Building a text library also provides more references and materials for subsequent tasks, helping to generate more informative and logical text results.

[0069] In some embodiments, the process of obtaining a document text corresponding to a target text in a preset text library includes:

[0070] The target text is vector-encoded by the encoder to obtain a second vector; the data text is obtained based on the second vector and the text library.

[0071] The document text includes a text segment corresponding to a target vector whose similarity to the second vector is greater than a preset threshold.

[0072] Specifically, the target text is input into the encoder model to obtain a vector representation of the target text, called the second vector. Similarity is calculated between the second vector and the vectors of each text segment in the text library, using cosine similarity or other similarity metrics. The text segments corresponding to the target vector whose similarity to the second vector exceeds a preset threshold are obtained.

[0073] This embodiment uses an encoder to vectorize the target text and selects text segments with high similarity to the target text as reference text based on a similarity calculation method. This can provide more comprehensive, coherent, and high-quality text information, which helps improve text generation efficiency.

[0074] In some embodiments, when the number of times is equal to a preset number, after matching the inferred character string with the prefix index library, the method further includes:

[0075] When the prefix index library does not match the string or the matching fragment cannot be successfully inferred, sequential inference is performed directly; the number of times is reset, and the current target text and data text are inferred through the large language model. When the number of inferences is equal to the preset number, the steps for matching the prefix are determined based on the inference results.

[0076] Specifically, if the inferred string doesn't match the prefix index, meaning no matching fragment can be found or the matching fragment can't successfully infer the next step, sequential inference can be performed. Sequential inference generates the next character or word step by step according to the text order, without relying on the prefix index.

[0077] The purpose of resetting the number of inferences is to control the duration of inferences and avoid infinite loops or long waits. By limiting the number of inferences, we can balance the inference effect and operation efficiency to a certain extent.

[0078] By resetting the number of inferences and re-finding matching prefixes, the reuse of relevant text materials has been maximized.

[0079] In some embodiments, decomposing the target data segment into an inference sequence includes:

[0080] According to the target data segment, obtain the first string, which is the prefix of the target data segment plus the next character at the corresponding position of the prefix match in the target data segment; according to the target data segment, obtain the second string, which is the first string plus the next character at the corresponding position of the first string in the target data segment; decompose the target data segment into an inference sequence in turn.

[0081] As an example, when the LLM is performing inference, at the first time step, the inferred token is "light", and at the second time step, the inferred current token is "water". When the preset number is 2, the matching prefix is "fresh water". By utilizing the characteristics of the prefix tree or prefix array, match in the pre-maintained prefixes of the data segments to confirm whether there is a prefix that is the same as this matching prefix. If a consistent target data segment is found, such as "Fresh water is a limited resource", then this segment is copied one by one in its entirety into the subsequent input of the LLM. Each time a new token is added, it will form a new example in the batch, such as ["fresh water", "fresh water is", "fresh water is a"...], thus completing the decomposition of the target data segment.

[0082] In this embodiment, the data segment is gradually decomposed into an inference sequence for subsequent text generation or inference tasks. This can maintain the coherence of the generated text and gradually infer the next character or word based on the content of the data segment to generate a text result that meets expectations.

[0083] In some embodiments, determining the current output text according to the character set includes:

[0084] According to the character set, match the characters generated by each inference sequence with the characters at the corresponding positions in the data segment to obtain the matching length of successful matching; if the matching length is less than the length of the data segment, select the longest segment with successful matching for output; if the matching length is equal to the length of the data segment, output the data segment.

[0085] Specifically, match the characters generated by each inference sequence with the characters at the corresponding positions in the data segment to obtain the matching length of successful matching. This means comparing the characters generated by the inference sequence with the characters at the corresponding positions in the data segment, and if they are the same, it is considered a successful match.

[0086] If the characters generated in the inference sequence match a part of the characters in the data segment but not completely, then select the longest segment with successful matching for output as the current output text. If the characters generated by the inference sequence completely match the data segment, then the entire data segment can be directly output as the current output text.

[0087] As an example, such as Figure 2As shown in the figure, the target data text is "Fresh water is a limited resource". After the output of the constructed batch is calculated in parallel through LLM, assuming the inference result is "Fresh water is a limited natural resource", it can be found that the longest matching prefix is ​​up to the word "limit", so the 6 tokens indicated by the dotted line can be output at one time.

[0088] This embodiment matches the characters generated from the inference sequence with the characters at corresponding positions in the data fragment, and selects the longest matching fragment or the entire data fragment as the current output text. This ensures that the output text is relevant to the data fragment and complies with the matching rules and logic. The data skipping operation is reused during inference, ensuring that inference speed does not slow down as the input length increases.

[0089] In some embodiments, after determining the text currently being output according to the character set, the method further includes:

[0090] Analyze the text and obtain analysis results, which are used to verify the accuracy of the text; based on the analysis results, obtain the response text corresponding to the target text; and output the response text through the large language model.

[0091] Analyzing the text means analyzing the results of the reasoning, including semantic and logical rationality, consistency, etc. Through in-depth analysis of the reasoning results, possible errors or inaccuracies can be discovered.

[0092] Design appropriate processing strategies based on the analysis results. These strategies can include text error correction, logic revision, and information completion. Based on the needs of specific application scenarios, design appropriate processing strategies to correct errors and inaccuracies in the inference results and generate response text corresponding to the target text input by the model.

[0093] In addition, the response text can be verified to ensure that the correction algorithm can effectively improve the accuracy and usability of the inference results.

[0094] This embodiment can further improve the accuracy and usability of the inference results by analyzing the currently output text. This embodiment can correct errors or inaccuracies that may exist in the inference process, making the final inference results more reliable and accurate.

[0095] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0096] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0097] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0098] Figure 3 This is a schematic diagram of a vertical field reasoning acceleration device provided by an embodiment of the present application. Figure 3 As shown, the device includes:

[0099] A first acquisition module 301 is used to acquire a target text;

[0100] The second acquisition module 302 is used to obtain the data text corresponding to the target text in a preset text library;

[0101] Determination module 303, for establishing a prefix index library based on the target text and the document text;

[0102] A first reasoning module 304 is configured to perform reasoning on the target text and the reference text using a large language model. When the number of inferences reaches a preset number, a matching prefix is ​​determined based on the inference result. The preset number is a positive integer, and the inference result is all target characters output within the preset number of inferences.

[0103] The matching module 305 is used to match the matching prefix with the prefix index library. If the match is successful, a data segment that is consistent with the prefix of the matching prefix is ​​obtained as the target data segment.

[0104] a decomposition module 306 for decomposing the data segment into an inference sequence according to the matching prefix and the target data segment;

[0105] The second reasoning module 307 is used to perform concurrent reasoning on the reasoning sequence to obtain a character set, where the character set includes the characters obtained by reasoning in each reasoning sequence;

[0106] The output module 308 is used to determine the text to be outputted according to the character set.

[0107] In some embodiments, the second acquisition module is also used to obtain a first data text in a vertical field, segment-encode the first data text through a pre-trained encoder, and obtain a first vector, wherein the first vector corresponds to a segment of the first data text; and obtain a text library based on the data text and the first vector.

[0108] In some embodiments, the second acquisition module is specifically used to perform vector encoding on the target text through an encoder to obtain a second vector, and obtain a data text based on the second vector and the text library, where the data text includes a text segment corresponding to the target vector whose similarity with the second vector is greater than a preset threshold.

[0109] In some embodiments, the matching module is also used to directly perform sequential reasoning, reset the number of times, and execute reasoning on the current target text and data text through the large language model when the prefix index library does not match the string or the matching fragment cannot be successfully inferred. When the number of reasoning is equal to the preset number of times, the step of matching the prefix is ​​determined according to the result of the reasoning.

[0110] In some embodiments, the inference module is specifically used to obtain a first string based on a data fragment, where the first string is the prefix of the data fragment plus the next character in the data fragment corresponding to the matching prefix; obtain a second string based on the data fragment, where the second string is the first string plus the next character in the data fragment corresponding to the first string; and decompose the data fragment into an inference sequence in sequence.

[0111] In some embodiments, the output module is specifically used to match the characters generated by each inference sequence with the characters at the corresponding position in the data fragment according to the character set to obtain the matching length of the successful match. If the matching length is less than the length of the data fragment, the longest segment of the successful match is selected for output; if the matching length is equal to the length of the data fragment, the data fragment is output.

[0112] In some embodiments, the output module is further used to analyze the text and obtain analysis results, which are used to verify the accuracy of the text. Based on the analysis results, a reply text corresponding to the target text is obtained, and the reply text is output through a large language model.

[0113] The vertical field reasoning acceleration device provided in the embodiment of the present application can realize all the processes and beneficial effects realized by the above-mentioned method embodiment. To avoid repetition, they will not be described here.

[0114] Figure 4 Schematic diagram of the electronic device 4 provided in the embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable by the processor 401. When the processor 401 executes the computer program 403, the steps of the above-described method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-described device embodiments are implemented.

[0115] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include but is not limited to a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 This is merely an example of the electronic device 4 and does not limit the electronic device 4 . The electronic device 4 may include more or fewer components than shown in the figure, or different components.

[0116] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0117] Memory 402 can be an internal storage unit of electronic device 4, such as a hard disk or memory of electronic device 4. Memory 402 can also be an external storage device of electronic device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on electronic device 4. Memory 402 can also include both an internal storage unit of electronic device 4 and an external storage device. Memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0118] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (such as a computer-readable storage medium). Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0120] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A vertical domain model reasoning acceleration method, characterized by: include: Get the target text; Obtaining the data text corresponding to the target text in a pre-set text library; Establishing a prefix index library based on the target text and the document text, wherein the prefix index library includes document segments corresponding to the document text and the target text; Performing reasoning on the target text and the data text using a large language model, and determining a matching prefix based on the result of the reasoning when the number of reasoning equals a preset number, wherein the preset number is a positive integer, and the result of the reasoning is all target characters output within the preset number; Matching the matching prefix with the prefix index library, if the match is successful, obtaining the data segment consistent with the prefix of the matching prefix as the target data segment; decomposing the data segment into an inference sequence according to the matching prefix and the target data segment; Performing concurrent reasoning on the reasoning sequence to obtain a character set, wherein the character set includes characters obtained by reasoning in each reasoning sequence; The text currently being output is determined according to the character set.

2. The vertical domain model reasoning acceleration method according to claim 1 is characterized in that: Before obtaining the document text corresponding to the target text in the preset text library, the method further includes: Obtain the first-hand information text in the vertical field; Segment-encoding the first document text using a pre-trained encoder to obtain a first vector, wherein the first vector corresponds to a segment of the first document text; The text library is obtained according to the document text and the first vector.

3. The vertical domain model reasoning acceleration method according to claim 2 is characterized in that: The process of obtaining the data text corresponding to the target text in the preset text library includes: Performing vector encoding on the target text by the encoder to obtain a second vector; The document text is obtained according to the second vector and the text library, and the document text includes a text segment corresponding to a target vector whose similarity with the second vector is greater than a preset threshold.

4. The vertical domain model reasoning acceleration method according to claim 1, characterized in that: When the number of times is equal to the preset number of times, after matching the inferred character string with the prefix index library, the method further includes: When the prefix index library does not match the string or the matching fragment cannot be successfully inferred, sequential reasoning is performed directly; Reset the number of times, execute the step of reasoning the target text and the data text by using the large language model, and when the number of times of reasoning is equal to the preset number of times, determine the matching prefix according to the result of the reasoning.

5. The vertical domain model reasoning acceleration method according to claim 1, characterized in that: Decomposing the data segment into an inference sequence according to the matching prefix and the target data segment includes: Obtaining a first character string according to the target data segment, where the first character string is a prefix of the target data segment plus a next character at a position corresponding to the matching prefix in the target data segment; Obtaining a second character string according to the target data segment, where the second character string is the first character string plus the next character in the target data segment corresponding to the first character string; The target data segments are sequentially decomposed into the inference sequences.

6. The vertical domain model reasoning acceleration method according to claim 1, characterized in that: The process of determining the text currently being output according to the character set includes: According to the character set, matching the characters generated by each inference sequence with the characters at corresponding positions in the target data segment to obtain a matching length of a successful match; If the matching length is less than the target data segment length, the longest segment that successfully matches is selected and output; If the matching length is equal to the target data segment length, the target data segment is output.

7. The vertical domain model reasoning acceleration method according to any one of claims 1 to 6, characterized in that: After determining the text currently being output according to the character set, the method further includes: Analyzing the text to obtain an analysis result, wherein the analysis result is used to verify the accuracy of the text; Obtaining a reply text corresponding to the target text according to the analysis result; The reply text is outputted through the large language model.

8. A vertical domain model reasoning acceleration device, characterized in that: include: A first acquisition module is used to acquire the target text; The second acquisition module is used to acquire the data text corresponding to the target text in a preset text library; A determination module, configured to establish a prefix index library based on the target text and the document text, wherein the prefix index library includes document segments corresponding to the document text and the target text; a first reasoning module, configured to perform reasoning on the target text and the data text using a large language model, and determine a matching prefix based on the reasoning result when the number of reasonings is equal to a preset number, wherein the preset number is a positive integer, and the reasoning result is all target characters output within the preset number; a matching module, configured to match the matching prefix with the prefix index library, and if the match is successful, obtain the data segment consistent with the prefix of the matching prefix as the target data segment; a decomposition module, configured to decompose the data fragment into an inference sequence according to the matching prefix and the target data fragment; a second reasoning module, configured to perform concurrent reasoning on the reasoning sequence to obtain a character set, wherein the character set includes characters obtained by reasoning in each reasoning sequence; The output module is used to determine the text currently output according to the character set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images