A dynamic reasoning method and system under a large language model

Abstract

In view of the diversity of user input and the difference in resource consumption of large language models when processing complex and simple queries, the present invention proposes a dynamic reasoning method and system under a large language model, which performs word segmentation on the prompt sentence input by the user, and uses two different recognition models to calculate the matching degree of each word segmentation respectively, and then calculates the reasoning control factor. The reasoning control factor is used to guide the dynamic reasoning process within the large language model, specifically by adjusting the management strategy of the Key-Value cache. When the relative distance between the current token and the context token exceeds the range determined by the reasoning control factor, the relevant Key and Value will be removed from the cache. This method effectively balances the use of storage space and reasoning efficiency, allowing the large language model to dynamically adjust its internal resource allocation according to different types of user input, thereby improving the overall response speed and accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of large language model optimization, and in particular relates to a dynamic reasoning method and system under a large language model. Background Art

[0002] In today's artificial intelligence field, large language models have become an important force driving technological progress. These models are able to process and understand large amounts of natural language data, thereby providing a variety of intelligent services such as machine translation, question-answering systems, text generation, etc. However, with the diversification of user needs, large language models face different challenges, especially in processing different types of user input and optimizing resource usage.

[0003] On the one hand, the prompt sentences input by users may be more inclined to professional knowledge acquisition or more inclined to daily communication, which leads to differences in the consumption of inference computing power within the large language model. Professional knowledge acquisition usually requires the model to perform in-depth analysis and reasoning to provide accurate and detailed answers. For example, when users ask complex medical questions or technical details, the model needs to access a large number of professional databases for precise information matching and logical reasoning. This type of query tends to consume more computing resources because they involve deep semantic understanding and the application of professional knowledge.

[0004] In contrast, life-oriented communication focuses more on daily conversations and relaxed exchanges. Such interactions usually do not require deep reasoning, but rely on common sense and a broad range of knowledge. Although such conversations may appear simple on the surface, they require the model to have a wide range of knowledge coverage and fast response capabilities in order to interact smoothly with users. Therefore, although the computational requirements of a single conversation may not be as high as those of a professional query, in order to maintain the coherence and real-time nature of the conversation, the model still needs to continuously update its knowledge base and optimize the response speed.

[0005] On the other hand, dynamic reasoning technology based on kv cache is an effective way to handle large language models. Key-value (kv) cache is a commonly used data storage method that allows the model to store previously calculated results for quick retrieval during subsequent reasoning. This method can significantly improve the processing speed of the model, especially when dealing with repeated or similar queries. However, when the prompt statement is long or the content is complex, the amount of kv cache data generated will become very large. This not only takes up a lot of storage space, but also adds additional overhead when managing and maintaining this data.

[0006] As the number of users increases, this problem becomes more serious. Each user's query may generate new cache data. Without sufficient optimization measures, the cache growth rate may far exceed the system's processing capacity. In addition, queries from different users may overlap, resulting in the same information being stored multiple times, which not only wastes storage resources but also increases the complexity of data processing. How to better optimize the dynamic reasoning methods related to large language models based on users' personalized input needs is a typical demand in the current large language model optimization scenario. Summary of the invention

[0007] In order to solve the problems of the prior art, the present invention proposes a dynamic reasoning method and system under a large language model.

[0008] Specifically, the present application relates to a dynamic reasoning method under a large language model, characterized in that it includes the following steps:

[0009] Performing word segmentation processing on the prompt sentence input by the user to obtain a plurality of word segments, respectively inputting the plurality of word segments into a first recognition model to obtain a first matching degree of the prompt sentence, further respectively inputting the plurality of word segments into a second recognition model to obtain a second matching degree of the prompt sentence, and then calculating the inference control factor according to the first matching degree and the second matching degree of the prompt sentence;

[0010] The multiple word segments are converted into vector representations and then input into the large language model in word order. Each word segment generates a key and a value through linear transformation at each layer of the large language model. The key and the value are cached and calculated to obtain a token.

[0011] The relative distance between the current token and the context token is calculated, and when the relative distance exceeds the distance range mapped by the inference control factor, the Key and Value corresponding to the context token are removed from the cache.

[0012] Preferably, the first recognition model is a pre-trained professional matching neural network model, which is used to evaluate the professionalism of the prompt statement, and the second recognition model is a pre-trained interactivity matching neural network model, which is used to evaluate the interactivity of the prompt statement.

[0013] Preferably, the first recognition model and the second recognition model are dynamically updated, and the triggering conditions of the dynamic update include: model sample update, framework adjustment based on user feedback.

[0014] Preferably, the inference control factor is calculated based on the first matching degree and the second matching degree of the prompt sentence, and is specifically calculated using the following formula:

[0015]

[0016] Among them, F is the inference control factor, M1 is the first matching degree, M2 is the second matching degree, α and γ are preset weight coefficients, and β and δ are preset adjustment factors.

[0017] Preferably, α, γ, β and δ are used as the preset configuration of the large language model after the debugging phase, and a management platform interface is provided to adjust them.

[0018] Preferably, the caching method adopts distributed memory database caching.

[0019] Preferably, the relative distance between the current token and the context token is calculated by using the following formula:

[0020]

[0021] Among them, D is the relative distance between the current token and the context token, t is the position of the current token in the text, c is the position of the context token in the text, max(T) is the maximum length of the text, ε and u are preset adjustment factors, M1 is the first matching degree, and M2 is the second matching degree.

[0022] Preferably, when the relative distance exceeds the distance range mapped by the inference control factor, removing the Key and Value corresponding to the context token from the cache includes:

[0023] Determine the corresponding relative distance value range in the mapping table according to the range of the calculated inference control factor, and when the calculated relative distance is greater than the relative distance value range, remove the Key and Value corresponding to the context token from the cache;

[0024] The mapping table records the relative distance value ranges corresponding to different inference control factor value ranges.

[0025] The present application also proposes a large language model system, which is characterized by comprising the following functional components for dynamic reasoning:

[0026] A word segmentation processing module is used to receive the prompt sentence input by the user and perform word segmentation processing, and output multiple word segments;

[0027] A first recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a first matching degree of the prompt sentence;

[0028] A second recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a second matching degree of the prompt sentence;

[0029] An inference control factor calculation module, used for calculating the inference control factor according to the first matching degree and the second matching degree of the prompt sentence;

[0030] A vector representation conversion module, used to convert the multiple word segments into vector representations;

[0031] A large language model, used for receiving the vector representation output by the vector representation conversion module in word order, and generating Key and Value through linear transformation at each layer;

[0032] The distance calculation module is used to calculate the relative distance between the current token and the context token;

[0033] The cache management module is used to cache the Key and Value and adjust the cache content according to the inference control factor. When the relative distance exceeds the distance range mapped by the inference control factor, the Key and Value corresponding to the context token are removed from the cache.

[0034] The present application also relates to a computer-readable storage medium, on which a program code is stored. When the program code is executed by a processor, the steps of the dynamic reasoning method under a large language model as described above are executed.

[0035] The beneficial technical effects of the patent of this invention include: calculating the matching degree based on multiple recognition models according to the prompt sentence input by the user, obtaining the tendency of the user input, and then determining the reasoning control factor to affect the subsequent dynamic reasoning means, and dynamically adjusting the difference in reasoning computing power consumption within the large language model through different input requirements, thereby improving the efficiency of dynamic reasoning. Furthermore, combining the personalized input requirements of users with the cache elimination mechanism in the kv cache technology, the strategy is convenient and reliable, which can improve the requirements for storage and computing power in the reasoning process under the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 : Framework diagram of the method flow according to an embodiment of the present invention.

[0037] Figure 2 : A system composition framework diagram according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] As Figure 1 shown in the method flow framework diagram of this solution, first, the prompt statement input by the user is tokenized to obtain multiple tokens.

[0040] In one embodiment, the prompt statement input by the user is received, and the prompt statement is tokenized using a predefined dictionary and grammar rules. For example, according to the lexical boundaries of Chinese, such as single-character words, two-character words, multi-character words, etc., and punctuation marks to divide the vocabulary. Apply stop word filtering to remove some common but meaningless words for analysis, such as "de", "shi", etc. Output the tokenization result, with each token as an independent element. Optionally, the tokenization method includes, for example, the n-gram model or the Hidden Markov Model (HMM), and the most likely tokenization method is determined according to the statistical data in the corpus.

[0041] The first recognition model is a pre-trained neural network model for professional matching degree, which is used to evaluate the professional degree of the prompt statement. The second recognition model is a pre-trained neural network model for interaction matching degree, which is used to evaluate the interaction degree of the prompt statement.

[0042] In one embodiment, the basic steps for establishing the first and second recognition models include:

[0043] Data collection. For the first recognition model, collect vocabulary and article samples in the professional field through various channels, which can include academic databases, professional journals, industry reports, technical documents, etc., to ensure that the collected data covers a wide range of professional fields, such as medicine, law, engineering, finance, etc. For the second recognition model, collect communication vocabulary and article samples in daily life scenarios through various channels, which can include social media, forums, blogs, news websites, daily conversation records, etc. Ensure that the collected data covers a wide range of life scenarios, such as family, work, entertainment, shopping, etc.

[0044] Data preprocessing and feature extraction. Clean the collected communication vocabulary and articles in the life scenario, remove noise data and non-text information, tokenize the text data, and use a tokenization tool or algorithm suitable for the life scenario. Then extract features from the tokenized text. Methods such as TF-IDF, Word2Vec, GloVe, etc. can be used to convert the vocabulary into vector representations.

[0045] Model training and evaluation optimization. Use deep learning models, such as Recurrent Neural Network (RNN), Bidirectional Long Short-Term Memory Network (BiLSTM), etc., to train the extracted features. Evaluate the model through an independent test set, and calculate metrics such as accuracy, recall rate, F1 score, etc. According to the evaluation results, adjust and optimize the model, such as adjusting the network structure, adding regularization terms, etc.

[0046] In one embodiment, when it is determined that there is a need to update the model samples, the first recognition model and the second recognition model are dynamically updated. For example, data sources in professional fields and daily life scenarios are continuously monitored, such as academic databases, social media, news websites, etc. New data samples are captured regularly using crawler technology or API interfaces. In addition, the model structure can also be adjusted based on user feedback, such as modifying the number of network layers, adding regularization terms, etc., and the adjusted model is verified using an independent test set to ensure that the improvement is effective and does not introduce new problems.

[0047] The inference control factor is calculated based on the first matching degree and the second matching degree of the prompt sentence, and is specifically calculated using the following formula:

[0048]

[0049] Among them, F is the reasoning control factor, M1 is the first matching degree, M2 is the second matching degree, α and γ are preset weight coefficients, which are used to adjust the relative importance of professional matching degree and interactive matching degree in the final reasoning control factor. These two parameters can be adjusted according to the needs of actual applications to ensure the performance of the model in different scenarios; β and δ are preset adjustment factors, which are used to control the speed of the influence of professional matching degree and interactive matching degree on the reasoning control level. A larger β or δ value means that as the matching degree increases, the reasoning control level will approach its maximum value faster.

[0050] After the debugging phase, α, γ, β, and δ are used as the preset configurations of the large language model, and a management platform interface is provided to adjust them.

[0051] In one embodiment, a suitable distributed memory database, such as Redis, Memcached, etc., is selected according to the performance requirements, scalability, and reliability requirements of the system.

[0052] The relative distance between the current token and the context token is calculated by the following formula:

[0053]

[0054] Among them, D is the relative distance between the current token and the context token, t is the position of the current token in the text, c is the position of the context token in the text, max(T) is the maximum length of the text, ε and u are preset adjustment factors, M1 is the first matching degree, and M2 is the second matching degree.

[0055] The absolute distance between the current token and the context token is calculated and normalized by dividing it by the maximum length of the text. This ensures that the distance value is between 0 and 1, which is convenient for subsequent processing.

[0056] is a dynamic adjustment item. As the first matching degree M1 and the second matching degree M2 increase, this item will gradually decrease, thereby reducing the relative distance. This design makes the reasoning process focus more on the current token and its nearby context rather than distant tokens when the prompt sentence is highly professional and interactive.

[0057] In one embodiment, a mapping table search is performed to determine whether the calculated relative distance exceeds the distance range mapped by the inference control factor. The mapping table is constructed using a large amount of experimental data during the model training or tuning phase, and each record thereof mainly includes two records: the range interval of the control factor value, and the range interval of the relative distance value. Specifically, in the mapping table, the corresponding relative distance value range is determined based on the range of the calculated inference control factor. When the calculated relative distance is greater than the relative distance value range, the Key and Value corresponding to the context token are removed from the cache.

[0058] In one embodiment, it also relates to a large language model system, characterized in that: Figure 2 As shown, it contains the following functional components for dynamic reasoning:

[0059] A word segmentation processing module is used to receive the prompt sentence input by the user and perform word segmentation processing, and output multiple word segments;

[0060] A first recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a first matching degree of the prompt sentence;

[0061] A second recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a second matching degree of the prompt sentence;

[0062] An inference control factor calculation module, used for calculating the inference control factor according to the first matching degree and the second matching degree of the prompt sentence;

[0063] A vector representation conversion module, used to convert the multiple word segments into vector representations;

[0064] A large language model, used for receiving the vector representation output by the vector representation conversion module in word order, and generating Key and Value through linear transformation at each layer;

[0065] The distance calculation module is used to calculate the relative distance between the current token and the context token;

[0066] The cache management module is used to cache the Key and Value and adjust the cache content according to the inference control factor. When the relative distance exceeds the distance range mapped by the inference control factor, the Key and Value corresponding to the context token are removed from the cache.

[0067] The embodiment of the present disclosure provides a non-volatile computer storage medium, wherein the computer-readable storage medium stores program code; the computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute the method steps described in the above embodiment.

[0068] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above. The above-mentioned computer-readable medium may be contained in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device. The computer program code for performing the operation of the present disclosure may be written in one or more programming languages ​​or a combination thereof, and the above-mentioned programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional procedural programming languages—such as "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network including a Local Area Network (AN) or a Wide Area Network (WAN), or may be connected to an external computer.

[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions. The units involved in the embodiments described in the present disclosure can be implemented by software or by hardware. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.

[0070] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A dynamic reasoning method under a large language model, characterized in that: The following steps are involved: Performing word segmentation processing on the prompt sentence input by the user to obtain a plurality of word segments, respectively inputting the plurality of word segments into a first recognition model to obtain a first matching degree of the prompt sentence, further respectively inputting the plurality of word segments into a second recognition model to obtain a second matching degree of the prompt sentence, and then calculating the inference control factor according to the first matching degree and the second matching degree of the prompt sentence; The multiple word segments are converted into vector representations and then input into the large language model in word order. Each word segment generates a key and a value through linear transformation at each layer of the large language model. The key and the value are cached and calculated to obtain a token. The relative distance between the current token and the context token is calculated, and when the relative distance exceeds the distance range mapped by the inference control factor, the Key and Value corresponding to the context token are removed from the cache.

2. The dynamic reasoning method under a large language model as claimed in claim 1, characterized in that: The first recognition model is a pre-trained professional matching neural network model, which is used to evaluate the professionalism of the prompt statement, and the second recognition model is a pre-trained interactivity matching neural network model, which is used to evaluate the interactivity of the prompt statement.

3. The dynamic reasoning method under a large language model as claimed in claim 2, characterized in that: The first recognition model and the second recognition model are dynamically updated, and the triggering conditions of the dynamic update include: model sample update and framework adjustment based on user feedback.

4. The dynamic reasoning method under a large language model as claimed in claim 1, characterized in that: The inference control factor is calculated based on the first matching degree and the second matching degree of the prompt sentence, and is specifically calculated using the following formula: Among them, F is the inference control factor, M1 is the first matching degree, M2 is the second matching degree, α and γ are preset weight coefficients, and β and δ are preset adjustment factors.

5. The dynamic reasoning method under a large language model as claimed in claim 4, characterized in that: , , and After the debugging phase is completed, it is used as the preset configuration of the large language model, and a management platform interface is provided to adjust it.

6. The dynamic reasoning method under a large language model as claimed in claim 1, characterized in that: The caching method adopts distributed memory database caching.

7. The dynamic reasoning method under a large language model as claimed in claim 1, characterized in that: The relative distance between the current token and the context token is calculated using the following formula: Among them, D is the relative distance between the current token and the context token, t is the position of the current token in the text, c is the position of the context token in the text, max(T) is the maximum length of the text, ε and u are preset adjustment factors, M1 is the first matching degree, and M2 is the second matching degree.

8. The dynamic reasoning method under a large language model as claimed in claim 1, characterized in that: When the relative distance exceeds the distance range mapped by the inference control factor, removing the Key and Value corresponding to the context token from the cache includes: Determine the corresponding relative distance value range in the mapping table according to the range of the calculated inference control factor, and when the calculated relative distance is greater than the relative distance value range, remove the Key and Value corresponding to the context token from the cache; The mapping table records the relative distance value ranges corresponding to different inference control factor value ranges.

9. A large language model system, characterized in that: Contains the following functional components for dynamic reasoning: A word segmentation processing module is used to receive a prompt sentence input by a user and perform word segmentation processing, and output multiple word segments; A first recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a first matching degree of the prompt sentence; A second recognition model, used for receiving a plurality of segmented words output by the segmentation processing module, and calculating a second matching degree of the prompt sentence; An inference control factor calculation module, used for calculating the inference control factor according to the first matching degree and the second matching degree of the prompt sentence; A vector representation conversion module, used to convert the multiple word segments into vector representations; A large language model, used for receiving the vector representation output by the vector representation conversion module in word order, and generating Key and Value through linear transformation at each layer; Distance calculation module, used to calculate the relative distance between the current token and the context token; The cache management module is used to cache the Key and Value and adjust the cache content according to the inference control factor. When the relative distance exceeds the distance range mapped by the inference control factor, the Key and Value corresponding to the context token are removed from the cache.

10. A computer-readable storage medium having program code stored thereon, wherein the program code, when executed by a processor, executes the dynamic reasoning method under a large language model as described in any one of claims 1-8.

Images