Adaptive control method and system of large language model, computer storage medium

Abstract

The application discloses an adaptive control method of a large language model, comprising: based on an input business conversation, performing multi-modal feature extraction to obtain multi-dimensional conversation state features; inputting the multi-dimensional conversation state features into a multi-layer perception machine, taking the multi-layer perception machine as a fusion network to output a control gain matrix, the control gain matrix comprising gain weights corresponding to each dimension of the conversation state features; based on the values of each gain weight element of the control gain matrix, performing policy mapping through a hierarchical mapping table, and mapping each gain weight element into a corresponding natural language control instruction; injecting the natural language control instruction into a structured prompt word template to construct an enhanced prompt word; and sending the enhanced prompt word to the large language model to output a reply to the business conversation through the large language model. The method can realize quantitative adjustment and adaptive control of the large language model to generate a reply, and improve the accuracy, controllability and adaptability of the conversation response in a vertical business scenario.

Description

Technical Field

[0001] This application relates to the field of large language model control, and in particular to an adaptive control method and system for large language models, and a computer storage medium. Background Technology

[0002] Existing technologies typically employ a "large language model + retrieval-enhanced generation (RAG)" architecture, relying on static prompts, keyword-based rule engines, or offline reinforcement learning (RLHF) to control the generation behavior of the large language model.

[0003] However, the evaluation metrics of the aforementioned existing technologies are subjective and mutually coupled, relying on subjective human definitions and exhibiting serious conflicts between dimensions (such as empathy and focus), lacking objective quantitative standards that can be mathematically decoupled. Secondly, the control strategies are static and rigid, relying on static prompts or fixed rules, and cannot adaptively adjust according to real-time dynamic changes in user emotions and business processes. Finally, there is a lack of effective closed-loop correction mechanisms, belonging to an open-loop control mode, which cannot perceive model generation deviations in real time and perform online self-correction, making it difficult to achieve accurate, transparent, and adaptive control of model generation behavior in complex and ever-changing vertical business scenarios. Summary of the Invention

[0004] This application provides an adaptive control method and system for a large language model, as well as a computer storage medium. The method can realize quantitative adjustment and adaptive control of the responses generated by the large language model.

[0005] Firstly, this application provides an adaptive control method for a large language model, the method comprising:

[0006] Based on the input business dialogue, multimodal feature extraction is performed to obtain multidimensional dialogue state features;

[0007] The multidimensional dialogue state features are input into the multilayer perceptron, and the multilayer perceptron is used as the output control gain matrix of the fusion network. The control gain matrix includes the gain weight corresponding to each dimension of dialogue state features.

[0008] Based on the values ​​of each gain weight element of the control gain matrix, a hierarchical mapping table is used to map the strategy, and each gain weight element is mapped to the corresponding natural language control command.

[0009] Natural language control commands are injected into structured prompt word templates to construct enhanced prompt words that include system variables, knowledge base retrieval results, and natural language control commands. System variables are a dynamic parameter set that records and maintains the dialogue context and logical state in real time. Knowledge base retrieval results are obtained by retrieving the input business dialogue content in the knowledge base.

[0010] The enhanced prompts and the input business dialogue are sent to the large language model, which then outputs a response to the business dialogue.

[0011] In one embodiment, the adaptive control method further includes:

[0012] Based on the control gain matrix, the expected response standard for this round is calculated by dynamic weighted interpolation control between the preset ideal state and the preset bottom line state;

[0013] The actual state vector is obtained by vectorizing the business dialogue response output by the large language model and then transforming the vectorized data through the evaluation coordinate system. The evaluation coordinate system is a three-dimensional spatial mapping matrix obtained by extracting the semantic dimension of historical expert high-quality response text through orthogonal semantic decomposition technology.

[0014] The deviation between the expected response standard and the actual state vector is calculated. If the deviation between the expected response standard and the actual state vector is greater than a preset threshold, a correction instruction is added to the enhanced prompts in the next round of business dialogue so that the response output by the large language model in the next round converges.

[0015] In one embodiment, the adaptive control method further includes training a multilayer perceptron, including:

[0016] Construct training sample pairs, which include high-quality response texts from historical experts and baseline response texts generated by the original large language model in the same business dialogue context as the high-quality response texts from historical experts.

[0017] By evaluating the coordinate system, the high-quality response texts from historical experts and the baseline response texts are mapped to the target state vector and the initial state vector, respectively.

[0018] Starting with the initial state vector, the control gain matrix is ​​used as the adjustment ratio to correct the initial state vector toward the target state vector, thus obtaining the simulated corrected state vector.

[0019] The error between the simulated corrected state vector and the target state vector is calculated, and the weight parameters of the multilayer perceptron are updated using the backpropagation algorithm.

[0020] In one embodiment, the evaluation coordinate system is obtained by extracting semantic dimensions from high-quality historical expert responses using orthogonal semantic decomposition technology, including:

[0021] Using the BERT model as an encoder, the high-quality historical expert response texts are vectorized.

[0022] Principal component analysis is used to reduce the dimensionality of high-quality historical expert responses after data vectorization. The three projection directions with the largest data variance are found to obtain a three-dimensional spatial mapping matrix, which is then used as the evaluation coordinate system.

[0023] In one embodiment, the adaptive control method further includes defining indices for each axis of the evaluation coordinate system, including:

[0024] Using high-quality responses from historical experts as samples, we calculated the projection values ​​of all samples onto each axis of the evaluation coordinate system.

[0025] For each axis of the evaluation coordinate system, a feature set is constructed. Each feature set includes a preset number of samples with the largest projection value on the corresponding axis.

[0026] For each feature set, the top N most frequent feature words in the feature set are calculated. Based on the most frequent feature words in each feature set, a large language model is called to summarize and obtain the index definitions corresponding to the three dimensions of the evaluation coordinate system. Here, N is a natural number greater than or equal to 1. The index definitions include focus, flexibility and progress.

[0027] In one embodiment, the multidimensional dialogue state features include business features, and the adaptive control method further includes constructing a business feature extraction model to extract business features. Constructing the business feature extraction model includes:

[0028] Convert the standard business process diagram into a topology diagram, which includes a set of business nodes and a set of jump logic between nodes;

[0029] Perform random walks on the topology graph to simulate user actions and generate node sequences;

[0030] Based on the node sequence, the Skip-Gram algorithm is used to train the initial vector of each business node, so that adjacent business nodes on the topology graph have similar vector representations;

[0031] Using the initial vector as input, a pre-trained graph neural network (GCN) aggregates neighborhood information in the topology graph, outputting an enhanced business feature vector for each business node, thereby obtaining a business feature extraction model.

[0032] In one embodiment, the multidimensional dialogue state features include user features, and the adaptive control method further includes extracting user features through the following method:

[0033] The DistilBERT language model pre-trained on a general corpus is used as the base model;

[0034] Collect historical customer service dialogue logs, which include emotion tags and intent tags. The emotion tags include at least the categories of anger, anxiety, or calmness, and the intent tags represent the user's dialogue goals.

[0035] Based on historical customer service dialogue logs, the DistilBERT language model was fine-tuned under supervision. During fine-tuning, the cross-entropy loss function was used for optimization. After fine-tuning, the top-level classifier of the DistilBERT language model was removed, and the output of the last Transformer encoder layer in the DistilBERT language model was retained as the output user features. The cross-entropy loss function is expressed by the following formula:

[0036] ;

[0037] In the formula, y label For the sample distribution of emotion labels and intent labels, P ( y pred ) represents the predicted probability distribution output by the DistilBERT language model, L user This represents the loss value of the cross-entropy loss function. The goal is to perform supervised fine-tuning of the DistilBERT language model by minimizing the loss value of the cross-entropy loss function.

[0038] In one embodiment, the multidimensional dialogue state features include temporal features, and the adaptive control method further includes extracting the temporal features through the following method:

[0039] For the current dialogue round and the preset maximum dialogue round in a multi-round business dialogue, the sinusoidal position encoding method in the Transformer architecture is used to generate a temporal feature vector, thereby obtaining the temporal feature representing the temporal position of the current dialogue round in the overall dialogue sequence.

[0040] Secondly, this application also provides an adaptive control system for a large language model, which applies the adaptive control method for a large language model as described in the first aspect.

[0041] Thirdly, this application also provides a computer storage medium storing a processing program, which executes the adaptive control method of the large language model of the first aspect when it runs.

[0042] The aforementioned adaptive control method for a large language model extracts multi-dimensional dialogue state features and inputs these features into a multilayer perceptron to generate a control gain matrix that includes weights for attention, flexibility, and progress. Based on each gain weight value, a hierarchical mapping table is used to convert each gain weight into corresponding natural language control commands. These natural language control commands, along with system variables and knowledge base retrieval results, are injected into a structured prompt template to form enhanced prompt words. The enhanced prompt words and the input business dialogue are then sent to the large language model to generate a response. This method enables quantitative adjustment and adaptive control of the responses generated by the large language model, improving the accuracy, controllability, and adaptability of dialogue responses in vertical business scenarios. Attached Figure Description

[0043] Figure 1 A flowchart of the adaptive control method for a large language model provided in the embodiments of this application;

[0044] Figure 2 A flowchart illustrating the construction of the evaluation coordinate system provided in the embodiments of this application;

[0045] Figure 3 A flowchart for business dialogue correction provided in the embodiments of this application;

[0046] Figure 4 This is a flowchart of training a multilayer perceptron in one embodiment;

[0047] Figure 5 This is a schematic diagram of an adaptive control system for a large language model in one embodiment. Detailed Implementation

[0048] The present application will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application. Any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the protection scope of the present application.

[0049] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0050] In one embodiment, such as Figure 1 As shown, an adaptive control method for a large language model is provided, which includes the following steps:

[0051] Step 101: Based on the input business dialogue, perform multimodal feature extraction to obtain multidimensional dialogue state features;

[0052] Specifically, multimodal feature extraction is performed based on the input business dialogue to obtain multidimensional dialogue state features. For example, the multidimensional dialogue state can include three dimensions of dialogue state features, which can be business features, user features, and temporal features, respectively. Each dimension of dialogue state features has its own corresponding feature extraction method.

[0053] For example, the method for extracting business features is as follows: Based on predefined business process rules and topology, business features are extracted. These features can be used to quantify the correlation between the current business dialogue and standard business specifications, thereby identifying the boundaries of the business scenario. The method for extracting user features is as follows: Through semantic analysis and sentiment recognition of the business dialogue text, user features are extracted. These features are used to perceive the user's real-time intent, emotional state, and personalized needs. The method for extracting business features is as follows: Based on the turn information of the business dialogue, temporal features are extracted. These temporal features are used to characterize the stages and urgency of the dialogue process, reflecting the pressure of progress over time.

[0054] Step 102: Input the multi-dimensional dialogue state features into the multilayer perceptron, and use the multilayer perceptron as the output control gain matrix of the fusion network. The control gain matrix includes the gain weight corresponding to each dimension of the dialogue state features.

[0055] Specifically, multi-dimensional dialogue state features are input into a fusion network constructed using a multilayer perceptron. This fusion network processes and nonlinearly transforms the input multi-dimensional dialogue state features through internal fully connected layers and a sigmoid activation function, outputting three independent scalar parameters with values ​​ranging from (0,1), defined as attention gain weight α, flexibility gain weight β, and propulsion gain weight γ. Each dimension of the multi-dimensional dialogue state feature corresponds to one of these gain weights. Furthermore, a control gain matrix is ​​constructed based on the attention gain weight α, flexibility gain weight β, and propulsion gain weight γ.

[0056] Step 103: Based on the values ​​of each gain weight element of the control gain matrix, perform policy mapping through a hierarchical mapping table, mapping each gain weight element to the corresponding natural language control command.

[0057] Specifically, the values ​​of the focus gain weight α, flexibility gain weight β, and propulsion gain weight γ in the control gain matrix are compared with the corresponding threshold intervals in the hierarchical mapping table to determine the control intensity level corresponding to each gain weight, and the corresponding natural language instruction template is selected based on the control intensity level corresponding to each gain weight.

[0058] Specifically, the focus gain weight α generates focus control instructions based on a hierarchical mapping table. Similarly, the flexibility gain weight β generates flexibility control instructions, and the progress gain weight γ generates progress control instructions.

[0059] Step 104: Inject natural language control instructions into the structured prompt word template to construct an enhanced prompt word containing system variables, knowledge base retrieval results and natural language control instructions. The system variables are a dynamic parameter set that records and maintains the dialogue context and logical state in real time. The knowledge base retrieval results are obtained by retrieving the input business dialogue content in the knowledge base.

[0060] Specifically, focus control instructions, flexibility control instructions, and progress control instructions are input into a preset structured prompt word template to construct enhanced prompt words that include system variables, knowledge base retrieval results, and natural language control instructions.

[0061] System variables can be a dynamic set of parameters that are recorded and maintained in real time to define the dialogue context and logical state, such as user identity, business process, and dialogue history. Knowledge base retrieval results can refer to relevant business knowledge fragments that are retrieved and returned in real time from the enterprise knowledge base based on the current user's question.

[0062] Step 105: Send the enhanced prompts and the input business dialogue to the large language model, and output the response to the business dialogue through the large language model.

[0063] Specifically, after receiving the enhanced prompt word and the input business dialogue, the large language model performs constrained reasoning based on the enhanced prompt word. Under the premise of following the focus control instructions, flexibility control instructions, and progress control instructions, it generates a response to the input business dialogue and forces the response to include the specific format of the two fields "think" and "result" to generate the output.

[0064] The "think" field needs to output the large language model's own analysis, decision-making logic, and self-checking chain (e.g., analyzing user intent, retrieving and matching control commands, and explaining response strategies). The "result" field needs to include the final response text (reply_text) and the dialogue status indicator (topic_status) of the business dialogue. Furthermore, based on the response text (reply_text) and the dialogue status indicator (topic_status) of the business dialogue, the text content of the response text (reply_text) in the "result" field is parsed and extracted as the final response to the current round of business dialogue.

[0065] In this embodiment, the method extracts multi-dimensional dialogue state features and inputs these features into a multilayer perceptron to generate a control gain matrix including attention gain weights, flexibility gain weights, and propulsion gain weights. Based on the values ​​of each gain weight, a hierarchical mapping table is used to convert each gain weight into a corresponding natural language control command. The natural language control command, along with system variables and knowledge base retrieval results, is injected into a structured prompt template to form enhanced prompt words. These enhanced prompt words, along with the input business dialogue, are sent to a large language model to generate a response to the business dialogue. This method enables quantitative adjustment and adaptive control of the responses generated by the large language model, improving the accuracy, controllability, and adaptability of dialogue responses in vertical business scenarios.

[0066] It should be noted that in the adaptive control method for the large language model provided in this application embodiment, the multi-dimensional dialogue state features can include three dimensions: business features, user features, and temporal features. These three dimensions of dialogue state features can be used as extraction targets, rather than based on subjective human evaluation. An objective quantitative evaluation standard is established based on orthogonal semantic decomposition technology. In the adaptive control method provided in this application embodiment, by constructing an orthogonal semantic space based on unsupervised PCA, subjective human definitions are abandoned. Principal component analysis is used to decouple unstructured dialogue into three non-interfering mathematical coordinates: focus, flexibility, and progress, thereby achieving an objective quantitative measurement of the quality of the large model generation.

[0067] Specifically, such as Figure 2As shown, in the control method provided in this application embodiment, an evaluation coordinate system is constructed in advance during the offline stage. This evaluation coordinate system includes three objectively existing, non-interfering (orthogonal) key semantic dimensions (i.e., the coordinate axes of the evaluation coordinate system) extracted from massive historical high-quality dialogue data through mathematical transformation. By assigning physical meaning to each axis of the evaluation coordinate system, the index definition of the evaluation coordinate system is realized. Under the guidance of this index definition, the adaptive control method provided in this application extracts multi-dimensional dialogue state features from the dialogue, which can effectively solve the problems of ambiguous index definition, poor dynamic adaptability, and uncontrollable generation behavior of existing large language models in vertical business scenarios. For example, the construction of the evaluation coordinate system includes the following steps:

[0068] Step 21: Using the BERT model as an encoder, the high-quality historical expert response text is vectorized. Then, the principal component analysis algorithm is used to reduce the dimensionality of the vectorized high-quality historical expert response text, find the three projection directions with the largest data variance, obtain the three-dimensional spatial mapping matrix, and use the three-dimensional spatial mapping matrix as the evaluation coordinate system.

[0069] Step 22: Using the high-quality responses from historical experts as samples, calculate the projection values ​​of all samples onto each axis of the evaluation coordinate system;

[0070] Step 23: Construct feature sets for each axis of the evaluation coordinate system. Each feature set includes a preset number of samples with the largest projection value on the corresponding axis.

[0071] Step 24: For each feature set, calculate the top N most frequent feature words in the feature set. Based on the most frequent feature words in each feature set, call the large language model to summarize and obtain the index definitions corresponding to the three dimensions of the evaluation coordinate system. Here, N is a natural number greater than or equal to 1. The index definitions include focus, flexibility and progress.

[0072] Specifically, in step 21, high-quality responses from historical experts in historical business dialogue scenarios are collected, and these high-quality responses from historical experts are used to construct a high-quality dialogue dataset of historical experts. ,in This is a text from a high-quality dialogue dataset of historical experts.

[0073] Using a pre-trained BERT model as the encoder, a high-quality dialogue dataset of historical experts is generated. Each text in Data vectorization is performed to obtain sample vectors. .

[0074] Furthermore, principal component analysis (PCA) algorithm is used to analyze the sample vectors. Dimensionality reduction is performed, and its covariance matrix is ​​calculated: And perform eigenvalue decomposition. .

[0075] Furthermore, we find the three eigenvectors with the largest data variance and label these three eigenvectors as follows: At this point, the three eigenvectors These are orthogonal vectors, used to construct a three-dimensional spatial mapping matrix. and the three-dimensional space mapping matrix As an evaluation coordinate system.

[0076] In step 22, using high-quality responses from historical experts as samples, a vector for each sample is calculated. The evaluation considers the projection values ​​on the three axes of the coordinate system, that is, for each sample vector. The unit vectors of the three axes in the evaluation coordinate system Perform dot product operation:

[0077] , , .

[0078] This leads to the acquisition of each sample vector. Quantitative projection values ​​on the three semantic dimensions of focus, flexibility, and progress. , , .

[0079] In step 23, feature sets are constructed for each axis of the evaluation coordinate system. For example, for... Projection value corresponding to the axis Select the M largest samples (e.g., 100) to form Feature set corresponding to the axis Similarly, for Projection value corresponding to the axis The 100 largest samples can be selected to construct a feature set. ,for Projection value corresponding to the axis The 100 largest samples can be selected to construct a feature set. After obtaining the feature sets corresponding to each coordinate axis. , , Then, the TF-IDF algorithm can be used to calculate the top N most frequent feature words in each feature set. (N is a natural number greater than or equal to 1).

[0080] In step 24, the list of the top N most frequent feature words in each feature set is used as input, and the large language model is invoked. The large language model summarizes and conceptually generalizes the lexical semantics of the high-frequency feature words in each feature set. Based on the lexical semantics of the high-frequency feature words in each feature set, the large language model outputs core indicator definitions for each of the three feature sets, thereby obtaining the explicit semantics corresponding to the three axes of the evaluation coordinate system. In this embodiment, the definition is... The axis corresponds to the focus axis, reflecting the semantic overlap with the standard knowledge base. The axis corresponds to the flexibility axis: reflecting the responsiveness to personalized user input and emotions. The axis corresponds to the progress axis: reflecting the driving force on business process nodes.

[0081] Based on the above description, the method provided in this application does not rely on subjective human definition. Instead, it extracts three objectively existing and non-interfering (orthogonal) key semantic dimensions from massive historical high-quality dialogue data through mathematical transformation, and assigns them physical meaning to construct an evaluation coordinate system. Under the guidance of this evaluation coordinate system, the method provided in this application extracts features from dialogue data from three dimensions: business features, user features, and temporal features. This allows for objective quantitative measurement in the subsequent large language model execution deviation feedback stage (the specific processing procedure will be described later).

[0082] As an optional implementation, multidimensional dialogue state features include business features, user features, and temporal features. In the adaptive control method of the large language model provided in this application embodiment, three independent feature extraction models are constructed to extract features from the three dimensions of business features, user features, and temporal features, respectively.

[0083] In one embodiment, obtaining business characteristics based on input from a business dialogue includes the following steps:

[0084] The standard business process diagram is converted into a topology graph, which includes a set of business nodes and a set of transition logic between nodes. Random walks are performed on the topology graph to simulate user operations and generate node sequences. Based on the node sequences, the Skip-Gram algorithm is used to train an initial vector for each business node, so that adjacent business nodes on the topology graph have similar vector representations. The initial vectors are used as input, and a pre-trained Graph Neural Network (GCN) is used to aggregate neighborhood information on the topology graph to output an enhanced business feature vector for each business node, thereby obtaining a business feature extraction model.

[0085] Specifically, the Standard Operating Procedure (SOP) is constructed as a computer-processable topology graph G=(V,E). Here, V is the set of business nodes (such as "user authentication", "appointment time confirmation", etc. defined in the knowledge base), and E is the set of jump logic between nodes (i.e. the process sequence specified in the standard business procedure).

[0086] Furthermore, in the constructed topology graph G Random walks are performed to simulate the paths that users might take in actual business operations, thereby generating a series of node sequences composed of business nodes.

[0087] Furthermore, the Skip-Gram algorithm is used to train the model on the node sequence, so that adjacent business nodes on the topology graph obtain similar distributed representations in the vector space, thereby obtaining the initial vector of each business node.

[0088] The initial vector of each business node is used as input, and a pre-trained Graph Convolutional Network (GCN) is used to aggregate neighborhood information on the topology graph. That is, each layer of the GCN updates the node representation by aggregating the feature information of the node itself and its one-hop neighbors. After multiple layers of propagation, the output is an enhanced business feature vector that can simultaneously encode the semantics of the node itself and its local topological structure information. Thus, a business feature extraction model is obtained.

[0089] In one embodiment, obtaining user characteristics based on input from a business dialogue includes the following steps:

[0090] A DistilBERT language model pre-trained on a general corpus is used as the base model. Historical customer service dialogue logs are collected, including emotion and intent labels. Emotion labels include at least the categories of anger, anxiety, or calm, while intent labels represent the user's dialogue goal. Based on these historical logs, the DistilBERT language model is fine-tuned under supervision. The cross-entropy loss function is used for optimization during the fine-tuning process. After fine-tuning, the top-level classifier of the DistilBERT language model is removed, and the output of the last Transformer encoder is retained as the user feature output. The cross-entropy loss function is expressed by the following formula:

[0091] ;

[0092] In the formula, y label For the sample distribution of emotion labels and intent labels, P ( y pred ) represents the predicted probability distribution output by the DistilBERT language model, L userThis represents the loss value of the cross-entropy loss function. The goal is to perform supervised fine-tuning of the DistilBERT language model by minimizing the loss value of the cross-entropy loss function.

[0093] Specifically, the lightweight Transformer model DistilBERT, pre-trained on a general corpus, is used as the base model. Historical customer service dialogue logs are collected, which can include manually labeled emotion tags (e.g., anger, anxiety, calm) and intent tags (e.g., inquiry, processing, complaint, query). The intent tags can be used to represent the user's dialogue goal.

[0094] Furthermore, based on historical customer service dialogue logs with emotion and intent labels, the DistilBERT model is subjected to supervised fine-tuning to adapt it to user state recognition tasks in business scenarios. The fine-tuning process employs a cross-entropy loss function for optimization, which measures the difference between the probability distribution predicted by the DistilBERT model and the true label distribution.

[0095] After fine-tuning, the top-level classifier of the DistilBERT model is removed, and the output of the last Transformer encoder layer is used as the extracted user features. User characteristics It can integrate the deep semantic, emotional, and intent information of the current user's statement.

[0096] In one embodiment, obtaining temporal features based on the input business dialogue includes: generating a temporal feature vector using the sinusoidal positional encoding method in the Transformer architecture for the current dialogue round and the preset maximum dialogue round in a multi-round business dialogue, thereby obtaining temporal features representing the temporal position of the current dialogue round in the overall dialogue sequence.

[0097] Specifically, the turn number of the currently input business dialogue is obtained in real time. And the preset maximum allowed number of dialogue rounds, with the round number as follows. Substituting the values ​​into the sinusoidal position encoding function of the Transformer architecture, the expression of which is:

[0098] and .

[0099] Where i is the index of the feature dimension, This represents the total dimension of the time-series feature vectors.

[0100] This sinusoidal position coding function maps a one-dimensional temporal scalar to a high-dimensional temporal feature vector. The time-series feature vector It can represent the temporal characteristics and urgency of the current dialogue turn's position in the overall dialogue sequence.

[0101] To further illustrate the adaptive control method for the large language model provided in this application embodiment, the following describes the specific process of generating dialogue responses in the online phase of the method provided in this application embodiment.

[0102] In step 101, based on the input business dialogue (such as natural language dialogues of user online input questions and requests), multidimensional dialogue state features are obtained by extracting features from three dimensions: business features, user features, and temporal features, according to the three independent feature extraction models mentioned above. .

[0103] In step 102, the multidimensional dialogue state feature vector is... The input is a multilayer perceptron, which is used as a fusion network to perform nonlinear feature fusion through fully connected layers. Finally, the sigmoid activation function outputs three scalar parameters with values ​​between (0,1), defined as attention gain weight α, flexibility gain weight β, and propulsion gain weight γ. A control gain matrix is ​​constructed based on these three weights. :

[0104] .

[0105] In step 103, based on the control gain matrix The values ​​of each gain weight element are mapped to a policy using a hierarchical mapping table, assigning each gain weight element to a corresponding natural language control command. For example, a hierarchical mapping table is provided, as shown below:

[0106]

[0107] It should be noted that the hierarchical mapping table defines a correspondence between each gain weight and a control strength level. By looking up the table, the high, medium, and low control strength levels corresponding to each gain weight can be found, thereby obtaining the pre-made natural language control commands bound to the control strength level corresponding to the value of the gain weight.

[0108] For example, suppose the current scenario is that the user is somewhat impatient (User characteristic) and the process is severely delayed (Time characteristic). The following results can be obtained based on the table lookup operation.

[0109] The fusion network automatically calculates the output:

[0110] α (Focus Gain) = 0.9 (extremely high, SOP must be strictly followed);

[0111] β (flexibility gain) = 0.1 (extremely low, no idle chatter allowed);

[0112] γ (propulsion gain) = 0.8 (high, powerful propulsion).

[0113] In step 104, natural language control instructions are injected into the structured prompt word template to construct enhanced prompt words containing system variables, knowledge base retrieval results, and natural language control instructions. In step 105, the enhanced prompt words and the input business dialogue are sent to the large language model, which then outputs a response to the business dialogue. Based on the above method, the adaptive control method provided in this embodiment of the application achieves online response to business dialogues.

[0114] Furthermore, the adaptive control method provided in this application embodiment also includes a large language model execution deviation closed-loop feedback loop. This loop uses the evaluation coordinate system constructed in the offline stage to objectively quantify the response output by the system in step 105, calculates the deviation between the actual response and the expected response, and thus executes enhanced prompt words to correct deviations online according to the magnitude of the deviation, forming a closed-loop optimization.

[0115] Specifically, such as Figure 3 As shown, the adaptive control method provided in this application includes the following steps in its closed-loop feedback mechanism based on execution deviation:

[0116] Step 301: Based on the control gain matrix, calculate the expected recovery standard for this round by using dynamic weighted interpolation control between the preset ideal state and the preset bottom line state;

[0117] Specifically, before each round of dialogue, a pre-trained multilayer perceptron is invoked in real time, and the multilayer perceptron acts as a fusion network to output the current business features. User characteristics and time series characteristics This then outputs the current control gain matrix:

[0118] .

[0119] Furthermore, two baseline state vectors are predefined, which can represent the preset ideal state vectors of the optimal response standard under the current business node. (Typically, the values ​​are [1, 1, 1]) and a preset bottom-line state vector representing the minimum acceptable standard. .

[0120] In each round of dialogue decision-making, the control gain matrix output by the fusion network will be used. As a dynamic sliding weight, the expected response standard for this round is calculated using a weighted interpolation formula. The weighted interpolation formula is shown below:

[0121] ;

[0122] in, I It is an identity matrix.

[0123] It should be noted that the method provided in the embodiments of this application expects a response standard. Instead of using a fixed threshold, it dynamically calculates the control gain matrix between the "ideal state" and the "bottom line state". As a sliding weight, it is used to find a balance between the 'ideal target' and the 'safety bottom line'.

[0124] Step 302: Obtain the actual state vector. The actual state vector is the state vector obtained by transforming the vectorized data through the evaluation coordinate system after the business dialogue response output by the large language model is data vectorized.

[0125] Specifically, the pre-trained BERT encoder is invoked to process one or more candidate response texts generated in real time by the large language model. Data vectorization into sample vectors .

[0126] Furthermore, an evaluation coordinate system constructed based on the offline phase. Through matrix multiplication: Project the sample vectors onto this three-dimensional spatial mapping matrix. This allows us to calculate the current response as a transformed actual state vector. .

[0127] in, To generate a candidate response text for the dialogue system. The result is a scalar, representing the sample vector corresponding to the candidate response text content on the focus axis. The higher the projection value, the more the candidate response text conforms to the standard business process diagram specification; The result is a scalar, representing the sample vector corresponding to the candidate response text content on the flexibility axis. The projection value on the screen indicates that the higher the projection value, the more the candidate response text focuses on the user's personalized emotions. Let be a scalar, representing the sample vector corresponding to the candidate response text content on the progress axis. The projection value is the highest value, indicating that the candidate response text has a stronger driving force in the process.

[0128] Step 303: Calculate the deviation between the expected response standard and the actual state vector. If the deviation between the expected response standard and the actual state vector is greater than a preset threshold, add a correction instruction to the enhanced prompts in the next round of business dialogue so that the response output by the large language model in the next round converges.

[0129] Specifically, calculate the expected response standard. With the actual state vector Deviation vector between , .

[0130] Assuming a preset deviation threshold When the absolute value of the deviation vector is detected When this occurs, it is determined that the current response has not met control expectations. The deviation vector is then analyzed. The system automatically generates clear, targeted natural language correction instructions based on the component signs and magnitudes across various dimensions (focus, flexibility, and progress). If the value is negative, the instruction "The previous response deviated from the core business scope; please strictly adhere to the standard business process diagram specifications in this round" is generated. This natural language correction instruction is then dynamically appended to the enhanced prompts in the next round of dialogue, forming online real-time correction and ensuring that the output response of the large language model aligns with the expected response standard vector. convergence.

[0131] If the deviation vector Less than or equal to the threshold If the current response meets the control expectation, the data from this round is stored as a successful sample in the experience replay pool. Data is sampled periodically and the parameters of the fusion network are fine-tuned through reinforcement learning algorithms (such as PPO) to achieve offline optimization.

[0132] Based on the above description, the adaptive control method provided in this application embodiment utilizes the deviation between the actual executed response and the expected response to endow the system with the ability to correct errors online in real time and for the strategy to self-evolve.

[0133] As an optional implementation method, such as Figure 4 As shown, the control method provided in this application embodiment further includes training a multilayer perceptron, comprising the following steps:

[0134] Step 401: Construct training sample pairs, which include high-quality response texts from historical experts and baseline response texts generated by the original large language model in the same business dialogue context as the high-quality response texts from historical experts.

[0135] Step 402: By evaluating the coordinate system, map the high-quality response texts from historical experts and the baseline response texts to the target state vector and the initial state vector, respectively;

[0136] Step 403: Starting from the initial state vector, use the control gain matrix as the adjustment ratio to correct the initial state vector towards the target state vector, and obtain the simulated corrected state vector.

[0137] Step 404: Calculate the error between the simulated corrected state vector and the target state vector, and update the weight parameters of the multilayer perceptron using the backpropagation algorithm.

[0138] Specifically, high-quality response texts from history experts are defined as follows: Calculate its three-dimensional index score as the target state vector. The baseline response text generated by the original large language model in the same business dialogue context as the high-quality response text from historical experts is defined as follows: Calculate its three-dimensional index score as the initial state vector. .

[0139] With the initial state vector Starting from this point, the control gain matrix is... To adjust the scale, the initial state vector To the target state vector The control gain matrix obtained from the simulated modified state vector fusion network output can be corrected to... ,Right now:

[0140] )).

[0141] in, This is the control function.

[0142] Calculate the simulated corrected state vector With the target state vector Error between:

[0143] .

[0144] With the goal of minimizing the error loss, the backpropagation algorithm is used to fix the parameters of the front-end feature extractor and update the weight parameters of the multilayer perceptron. This enables the fusion network to learn how to output the optimal gain in order to eliminate the gap between the model and the expert.

[0145] For example, the system may include a multimodal feature extraction module 501, a fusion network module 502, a policy mapping module 503, a prompt word construction module 504, and a large language model interface module 505.

[0146] The multimodal feature extraction module 501 can be configured to extract multidimensional dialogue state features from the input business dialogue; the fusion network module 502 can be constructed using a multilayer perceptron and receive multidimensional dialogue state feature vectors to output a control gain matrix; the policy mapping module 503 can convert each gain parameter into natural language control instructions through a built-in hierarchical mapping table; the prompt word construction module 504 can integrate system variables, knowledge base retrieval results, and control instructions into a structured template to form enhanced prompt words; and the large language model interface module 505 can send the enhanced prompt words to the large language model and obtain a response.

[0147] This application also provides a computer storage medium. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0148] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0149] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The above are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for adaptive control of a large language model, the method comprising: The adaptive control method includes: Based on the input business dialogue, multimodal feature extraction is performed to obtain multidimensional dialogue state features; The multidimensional dialogue state features are input into a multilayer perceptron, and the multilayer perceptron is used as a fusion network to output a control gain matrix, which includes the gain weight corresponding to each dimension of the dialogue state features. Based on the values ​​of each gain weight in the control gain matrix, a hierarchical mapping table is used to perform policy mapping, mapping each gain weight to a corresponding natural language control command. The natural language control instructions are injected into the structured prompt word template to construct an enhanced prompt word that includes system variables, knowledge base retrieval results and the natural language control instructions. The system variables are a dynamic parameter set that records and maintains the dialogue context and logical state in real time. The knowledge base retrieval results are obtained by retrieving the input business dialogue content in the knowledge base. The enhanced prompt words and the input business dialogue are sent to the large language model, which then outputs a response to the business dialogue.

2. The adaptive control method of a large language model according to claim 1, characterized in that, The adaptive control method further includes: Based on the control gain matrix, the expected response standard for this round is calculated by dynamic weighted interpolation control between the preset ideal state and the preset bottom line state. The actual state vector is obtained by vectorizing the business dialogue response output by the large language model and then transforming the vectorized data through the evaluation coordinate system. The evaluation coordinate system is a three-dimensional spatial mapping matrix obtained by extracting the semantic dimension of historical expert high-quality response text through orthogonal semantic decomposition technology. Calculate the deviation between the expected response standard and the actual state vector. If the deviation between the expected response standard and the actual state vector is greater than a preset threshold, add a correction instruction to the enhanced prompt words in the next round of business dialogue so that the response output by the large language model in the next round converges.

3. The adaptive control method for a large language model according to claim 2, characterized in that, The adaptive control method further includes training the multilayer perceptron, including: Construct training sample pairs, which include high-quality response texts from historical experts and baseline response texts generated by the original large language model in the same business dialogue context as the high-quality response texts from historical experts. Using the evaluation coordinate system, the historical expert high-quality response text and the benchmark response text are respectively mapped to the target state vector and the initial state vector; Starting from the initial state vector, the control gain matrix is ​​used as the adjustment ratio to correct the initial state vector toward the target state vector, thereby obtaining a simulated corrected state vector. The error between the simulated corrected state vector and the target state vector is calculated, and the weight parameters of the multilayer perceptron are updated using the backpropagation algorithm.

4. The adaptive control method for a large language model according to claim 2, characterized in that, The semantic dimensions of high-quality responses from historical experts are extracted using orthogonal semantic decomposition techniques to obtain the evaluation coordinate system, which includes: Using the BERT model as an encoder, the high-quality historical expert response texts are vectorized. Principal component analysis is used to reduce the dimensionality of the vectorized historical expert response texts, and the three projection directions with the largest data variance are found to obtain a three-dimensional spatial mapping matrix, which is then used as the evaluation coordinate system.

5. The adaptive control method of a large language model according to claim 4, characterized in that, The adaptive control method further includes defining indices for each axis of the evaluation coordinate system, including: Using the high-quality responses from historical experts as samples, calculate the projection values ​​of all samples on each axis of the evaluation coordinate system. A feature set is constructed for each axis of the evaluation coordinate system, and each feature set includes a preset number of samples with the largest projection value on the corresponding axis; For each feature set, the top N most frequent feature words in the feature set are calculated. Based on the most frequent feature words in each feature set, a large language model is called to summarize and obtain the index definitions corresponding to the three dimensions of the evaluation coordinate system, where N is a natural number greater than or equal to 1.

6. The adaptive control method of a large language model according to claim 1, characterized in that, The multidimensional dialogue state features include business features, and the adaptive control method further includes constructing a business feature extraction model, extracting the business features through the business feature extraction model, and constructing the business feature extraction model includes: Convert the standard business process diagram into a topology diagram, which includes a set of business nodes and a set of jump logic between nodes; A random walk is performed on the topology graph to simulate user actions and generate a node sequence. Based on the node sequence, the Skip-Gram algorithm is used to train the initial vector of each business node, so that adjacent business nodes on the topology graph have similar vector representations; Using the initial vector as input, a pre-trained graph neural network (GCN) is used to aggregate neighborhood information in the topology graph, outputting an enhanced business feature vector for each business node, thereby obtaining the business feature extraction model.

7. The method of adaptive control of a large language model according to claim 1, wherein, The multidimensional dialogue state features include user features, and the adaptive control method further includes extracting the user features through the following method: The DistilBERT language model pre-trained on a general corpus is used as the base model; Collect historical customer service dialogue logs, which include emotion tags and intent tags. The emotion tags include at least the categories of anger, anxiety, or calmness, and the intent tags represent the user's dialogue goals. Based on the historical customer service dialogue logs, the DistilBERT language model is subjected to supervised fine-tuning. During the fine-tuning process, the cross-entropy loss function is used for optimization. After fine-tuning, the top-level classifier of the DistilBERT language model is removed, and the output of the last layer Transformer encoder in the DistilBERT language model is retained as the output user feature. The cross-entropy loss function is expressed by the following formula: ； wherein, y label is a sample distribution of the emotion label and the intent label, P y pred is a prediction probability distribution output by the DistilBERT language model, L user represents a loss value of the cross-entropy loss function, and the DistilBERT language model is supervised and fine-tuned with a target of minimizing the loss value of the cross-entropy loss function.​ 8. The adaptive control method for a large language model according to claim 1, characterized in that, The multidimensional dialogue state features include temporal features, and the adaptive control method further includes extracting the temporal features using the following method: For the current dialogue round and the preset maximum dialogue round in a multi-round business dialogue, a temporal feature vector is generated using the sinusoidal position encoding method in the Transformer architecture, thereby obtaining the temporal feature representing the temporal position of the current dialogue round in the overall dialogue sequence.

9. An adaptive control system for a large language model, characterized in that, The adaptive control system applies the adaptive control method of the large language model as described in any one of claims 1 to 8.

10. A computer storage medium, characterized in that, The computer storage medium stores a processing program, which, when executed, performs the adaptive control method for a large language model as described in any one of claims 1 to 8.

Images