Collaborative response method, device and equipment for large and small models, medium and program

Abstract

The invention discloses a collaborative response method and device for large and small models, equipment, a medium and a program. The method comprises the steps of calling a low-rank adaptation small model to generate an initial response through an edge node when an input instruction of a user is received; calculating the whole sentence confidence coefficient in the initial response; if the whole sentence confidence coefficient is smaller than a sentence-level confidence coefficient threshold value, a collaborative response instruction is sent to the cloud server cluster; selecting a specified number of target large models from the plurality of large models, and correcting at least one lexical element based on a lexical element level confidence coefficient threshold to obtain a plurality of collaborative responses; and selecting the target collaborative response with the highest confidence as a real response, and returning the real response to the user. According to the embodiment of the invention, the large model and the small model are utilized to cooperatively work to quickly give a preliminary reply to a simple question, the large model intervenes in optimization when a complex question is encountered, a satisfactory answer is provided for a user, the user experience is improved, and the instruction processing capability is enhanced by performing fine tuning optimization on the low-rank adaptive small model.

Description

Technical Field

[0001] The present invention is applicable to the field of natural language processing technology, and in particular relates to a collaborative response method, device, equipment, medium and program for large and small models. Background Art

[0002] In recent years, with the continuous development of artificial intelligence technology, deep learning-based natural language processing models have made significant progress in areas such as intelligent voice assistants and machine translation. However, due to their large number of parameters and computational complexity, large-scale pre-trained models are usually deployed in the cloud. This requires frequent communication between end devices and the cloud, and user experience is often affected by latency, especially in poor network conditions.

[0003] To improve inference performance, collaborative inference methods for large and small models have been proposed. This approach offloads some computing tasks to edge nodes or devices, reducing the frequency of cloud communication and thus improving response speed. However, existing collaborative inference methods for large and small models still suffer from low inference accuracy for the small edge model, frequent feedback from the small edge model to the large cloud model, inconsistent response results between the large cloud model and the small edge model, and the inability to continuously optimize the small model. Summary of the Invention

[0004] Based on this, the present invention provides a collaborative response method, device, equipment, medium and program for large and small models to solve the problem that it is difficult to strike a balance between response efficiency and quality in existing large and small models.

[0005] In a first aspect, an embodiment of the present invention provides a collaborative response method for large and small models, the method comprising:

[0006] When receiving a user input instruction through the edge node, the low-rank adaptive small model is called to perform small model calculation on the input instruction to generate an initial response;

[0007] Through the edge nodes, the generation probability of each word in the initial response is calculated, and the generation probability of each word is multiplied to obtain the confidence of the whole sentence corresponding to the input instruction;

[0008] The edge node compares the confidence of the entire sentence with the sentence-level confidence threshold. If the confidence of the entire sentence is less than the sentence-level confidence threshold, a coordinated response instruction is sent to the cloud server cluster.

[0009] A specified number of target large models are selected from multiple large models through a cloud server cluster, and each target large model is called to modify at least one word in the initial response based on a word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses;

[0010] Through the cloud server cluster, the target collaborative response with the highest confidence is selected as the true response from multiple collaborative responses, and the target large model that generates the true response is instructed to return the true response to the user.

[0011] In a second aspect, an embodiment of the present invention provides a collaborative response device for large and small models, the device comprising:

[0012] An initial response generation module is configured to, upon receiving a user input instruction through an edge node, call a low-rank adaptive small model to perform small model calculation on the input instruction and generate an initial response;

[0013] The whole sentence confidence calculation module is used to calculate the generation probability of each word in the initial response through the edge node, and multiply the generation probability of each word to obtain the whole sentence confidence corresponding to the input instruction;

[0014] A collaborative response instruction sending module is used to compare the confidence of the entire sentence with the sentence-level confidence threshold through the edge node, and if the confidence of the entire sentence is less than the sentence-level confidence threshold, send a collaborative response instruction to the cloud server cluster;

[0015] A collaborative response generation module is used to select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to modify at least one word in the initial response based on the word-level confidence threshold according to the input instructions and initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses;

[0016] The real response returning module is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through the cloud server cluster, and instruct the target large model that generates the real response to return the real response to the user.

[0017] In a third aspect, an embodiment of the present invention provides an electronic device, comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the collaborative response method of large and small models described in any embodiment of the present invention.

[0021] In a fourth aspect, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a collaborative response method of large and small models as described in any embodiment of the present invention when executed.

[0022] In a fifth aspect, a computer program product is also provided, which includes a computer program, and when the computer program is executed by a processor, it implements the collaborative response method of large and small models described in any embodiment of the present invention.

[0023] The technical solution of the embodiments of the present invention deploys a dynamically optimized, low-rank adaptive small model at the edge node, enabling rapid initial responses to user input commands. Compared to directly using a large model, the small model requires less computation and has faster processing speed, enabling rapid local generation of initial responses, reducing user wait time. When a user issues a simple command, the small model can quickly provide preliminary results without having to immediately rely on the large cloud model, improving system responsiveness. When the confidence level of the initial response generated by the edge node falls below a threshold, multiple large models in the cloud server cluster perform collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large model modifies the initial response based on a word-level confidence threshold. The highest-confidence collaborative response from the multiple large models is selected and returned to the user as the true response, ensuring that the user ultimately receives a high-quality, more accurate answer. The cloud server cluster constructs a set of triples based on the true response generated by the large model, the input command, and the initial response. Once a sufficient number of triples are accumulated, a dataset is constructed to optimize the low-rank adaptive small model at the edge node. Through this continuous learning and optimization mechanism, the small model continuously improves its performance, better handling various user commands and adapting to different application scenarios.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 This is a flowchart of a collaborative response method for large and small models provided in accordance with the first embodiment of the present invention;

[0027] Figure 2is a flowchart of another collaborative response method for large and small models provided according to embodiment 2 of the present invention;

[0028] Figure 3 2 is a schematic structural diagram of a large-small model collaborative response device provided according to a third embodiment of the present invention;

[0029] Figure 4 3 is a structural diagram of an electronic device according to a collaborative response method of large and small models provided in accordance with a fourth embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a collaborative response method for large and small models provided in the first embodiment of the present invention. This embodiment is applicable to situations where large and small models are used to collaboratively respond to user input instructions. The method can be executed by a collaborative response device for large and small models. The device can be implemented in the form of hardware and / or software and can be configured in the architecture system of edge servers and cloud servers. Figure 1 As shown, the method includes:

[0034] S110. When receiving a user input instruction through an edge node, a low-rank adaptive small model is called to perform small model calculation on the input instruction to generate an initial response.

[0035] In the embodiment of the present invention, an edge node is a computing node close to the data source or user, providing services at the edge of the network. It has certain computing and storage capabilities and can quickly process some tasks locally, reducing data transmission delays and dependence on the cloud. In the embodiment of the present invention, it is responsible for the initial processing of user commands. For example, the smart speaker in a smart home device can initially process user voice commands locally and is an edge node. The low-rank adaptive small model is a small model optimized by low-rank adaptation technology. Low-rank adaptation reduces the number of parameters while ensuring model performance by performing low-rank decomposition on the model parameters, reducing computational complexity and memory requirements, and enabling the small model to run efficiently on the edge node.

[0036] Specifically, in an embodiment of the present invention, in the self-attention layer and feedforward network layer of the small model, the low-rank adaptation technology introduces two trainable low-rank matrices LR1 and LR2 to parameterize the weight matrix. During the fine-tuning optimization process, only the weight parameters of these two low-rank matrices need to be learned, and the original main parameters of the small model are frozen. This greatly reduces the number of parameters that need to be trained, reduces the complexity of training, and enables the model to adapt to new command-response patterns more efficiently. Compared with directly adjusting the huge original model parameters, the low-rank adaptation technology significantly reduces the amount of calculation and improves the training speed. The small model is small in size and has a small amount of calculation, making it suitable for quickly processing simple tasks at edge nodes.

[0037] The low-rank adaptive small model deployed on the edge node is responsible for receiving user input instructions, and then performing preprocessing, word embedding, instruction encoding and decoding operations in sequence, and finally generating a preliminary response result.

[0038] S120 , calculating the generation probability of each word unit in the initial response through the edge node, and multiplying the generation probabilities of each word unit to obtain the confidence level of the entire sentence corresponding to the input instruction.

[0039] A word is the basic unit in text processing and is usually the smallest meaningful unit obtained after word segmentation of the text. In an embodiment of the present invention, the model performs processing and probability calculations on a word-by-word basis. The confidence of the entire sentence is a numerical value obtained by multiplying the generation probability of each word in the initial response. It is used to measure the credibility of the entire initial response. The higher the value, the more reliable the initial response. The edge node analyzes the initial response just generated, calculates the probability of each word being generated, and then multiplies these probabilities to obtain a numerical value that reflects the credibility of the entire initial response, namely the confidence of the entire sentence.

[0040] S130. Compare the whole sentence confidence with the sentence-level confidence threshold through the edge node. If the whole sentence confidence is less than the sentence-level confidence threshold, send a coordinated response instruction to the cloud server cluster.

[0041] The sentence-level confidence threshold is a pre-set standard value used to determine whether the confidence of the initial response reaches an acceptable level. The edge node compares the calculated sentence confidence with the pre-set sentence-level confidence threshold. If the sentence confidence falls below the threshold, indicating that the initial response is not trustworthy enough, the edge node sends a coordinated response instruction to the cloud server cluster, requesting assistance from the large model.

[0042] S140. Select a specified number of target large models from multiple large models through the cloud server cluster, and call each target large model to correct at least one word in the initial response based on the word-level confidence threshold according to the input instructions and initial response in the collaborative response instruction to obtain multiple collaborative responses.

[0043] A cloud server cluster is a collection of multiple cloud servers that work together through cluster technology to provide powerful computing and storage capabilities. In this embodiment of the present invention, it handles complex tasks that edge nodes cannot complete independently, optimizing the initial response with the help of the large model. The word-level confidence threshold is a confidence standard value set for each word. When the large model makes a correction to the word in the initial response, if the word-level generation probability falls below this threshold, the large model will regenerate the word.

[0044] After receiving the coordinated response command from the edge node, the cloud server cluster selects a specified number of target large models from its deployed large models. Each target large model then checks and corrects some or all of the initial responses based on the input command and initial response in the coordinated response command, according to the word-level confidence threshold. Ultimately, each target large model generates a coordinated response, resulting in multiple coordinated responses.

[0045] Optionally, through a cloud server cluster, each target large model is called to modify at least one word in the initial response based on the word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses, which may include:

[0046] Through the cloud server cluster, call each target large model to perform the following operations:

[0047] Obtain the generation probability of all word units in the initial response, and make correction judgments for each word unit in turn;

[0048] When it is determined that the generation probability of the target word currently being processed is greater than or equal to the word-level threshold, the target word is retained as a retained word;

[0049] When it is determined that the generation probability of the target word currently being processed is less than the word-level threshold, regenerating a modified word corresponding to the current word according to the input instruction;

[0050] The retained word and the modified word are combined to form a coordinated response.

[0051] Upon receiving a collaboration request from an edge node, the cloud server cluster calls each previously selected target large model, allowing them to execute subsequent processing operations on the initial response in parallel to improve processing efficiency. Each target large model obtains the generation probability of each token in the initial response generated by the edge node. The token generation probability reflects the likelihood that the model will generate the token. Each token is analyzed and evaluated sequentially to determine whether it needs to be modified.

[0052] The word-level threshold is a pre-set standard value. When the target large model determines that the generation probability of a word currently being processed reaches or exceeds this threshold, the word-level generation in the initial response is considered relatively reliable and is retained without modification as part of the subsequent collaborative response. Conversely, if the generation probability of the target word-level is less than the word-level threshold, the large model will consider the word-level to be unreliable. At this time, the large model will regenerate a more appropriate word-level based on the user's original input instruction to replace the word-level in the initial response. The newly generated word-level is the revised word-level.

[0053] After processing all the tokens in the initial response, the target model sequentially combines the retained tokens with the newly generated, modified tokens to form a complete, optimized text response, known as a collaborative response. The collaborative responses generated by multiple target models are then returned to the cloud server cluster for further processing.

[0054] S150. Select the target collaborative response with the highest confidence as the true response from multiple collaborative responses through the cloud server cluster, and instruct the target large model that generates the true response to return the true response to the user.

[0055] The cloud server cluster selects the most confident response from the collaborative responses generated by the multiple target large models and determines it as the final true response. The cloud server cluster also instructs the target large model that generated the true response to send the response back to the user, completing the collaborative response process for the large and small models.

[0056] Furthermore, after obtaining the current actual responses obtained after the current initial responses are corrected by each test large model through the cloud server cluster, the following steps may also be included:

[0057] The cloud server cluster is used to calculate the current average confidence of the current true response in all the test models, and the current true response with the largest current average confidence is determined as the benchmark response of the current initial response.

[0058] The cloud server cluster obtains the current true response generated by each test large model after correcting the current initial response. For each current true response, the cluster calculates its average confidence among all test large models. This calculation process is to statistically average the confidence assigned by each test large model when generating the current true response. By calculating the average confidence, the degree of recognition of the same current true response by different test large models can be comprehensively considered, and the reliability of the response can be evaluated as a whole. After calculating the current average confidence of all current true responses, the cloud server cluster will compare these average confidences, select the current true response with the largest average confidence value, and determine it as the baseline response for the current initial response. The baseline response represents the most reliable and expected response result under the comprehensive evaluation of multiple test large models. It will be used as a reference to calculate the model loss and fine-tune the low-rank adaptive small model.

[0059] Embodiments of the present invention deploy dynamically optimized, low-rank adaptive small models at edge nodes to quickly generate initial responses to user input commands. Compared to directly using a large model, the small model requires less computation and has faster processing speed. It can quickly generate initial responses locally, reducing user wait time. When a user issues a simple command, the small model can quickly provide preliminary results without having to immediately rely on the large cloud model, improving the system's responsiveness. When the confidence level of the initial response generated by the edge node falls below a threshold, multiple large models in the cloud server cluster perform collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large model modifies the initial response based on a word-level confidence threshold. The highest-confidence collaborative response from the multiple large models is selected and returned to the user as the true response, ensuring that the user ultimately receives a high-quality, more accurate answer. The cloud server cluster constructs a set of ternary mappings based on the true response generated by the large model, the input command, and the initial response. Once a certain number of ternary mappings are accumulated, a dataset is constructed to optimize the low-rank adaptive small model at the edge node. Through this continuous learning and optimization mechanism, the small model continuously improves its performance, better handling various user commands and adapting to different application scenarios.

[0060] Example 2

[0061] Figure 2 This is a flow chart of another collaborative response method for large and small models provided in the second embodiment of the present invention. This embodiment is refined based on the above embodiment. Accordingly, Figure 2 As shown, the method specifically includes:

[0062] S210. Through the edge node, call the low-rank adaptive small model to perform word segmentation and standardization preprocessing operations on the input instructions, and convert the preprocessed instructions into corresponding word element embedding vectors.

[0063] Word segmentation is a fundamental operation in natural language processing. It breaks down continuous text sequences into individual words or tokens according to specific rules. These tokens serve as the fundamental units for subsequent text analysis and processing. Normalization aims to standardize text formats and eliminate interference caused by formatting differences. Common operations include converting text to lowercase and removing punctuation and stop words to improve the accuracy and consistency of model processing. Token embedding is a technique that represents tokens as numerical vectors. Using a specific algorithm, each token is mapped into a fixed-dimensional vector space, allowing the token's semantic information to be represented in a vector format, facilitating model computation and learning. The position and distance of the embedding vectors of different tokens in the vector space reflect their semantic similarity.

[0064] After receiving user input, the edge node activates a small low-rank adaptive model. This model first preprocesses the instruction, segmenting the continuous text instruction into individual tokens (a process known as word segmentation). These tokens are then standardized, such as by capitalizing them and removing special symbols, to ensure a more standardized format. After preprocessing, these instruction tokens are converted into computer-readable numerical vectors, known as token embedding vectors, to facilitate subsequent model computation.

[0065] S220. Through the edge node, call the low-rank adaptive small model to encode the word embedding vector through the attention mechanism, and use the decoder to decode and continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end mark or reach the specified length, all generated responses are used as initial responses.

[0066] In deep learning models, the encoder is responsible for extracting and transforming features from the input data, mapping the raw data into a latent representation space. This latent representation contains key feature information of the input data and prepares it for subsequent processing. In an embodiment of the present invention, the encoder processes the word embedding vectors to extract key information. The decoder corresponds to the encoder and converts the latent representation output by the encoder into the final output. In natural language processing tasks, the decoder generates a textual response based on the encoded information.

[0067] The end mark is a special mark set in advance to indicate that the response content generated by the model has been completed. When this end mark appears in the text generated by the model, it means that the response generation is finished. For example, you can set " <eos>"(End Of Sentence)" is used as the end mark. The specified length is the maximum length limit of the response generated by the model. If the model does not encounter the end mark during the response generation process, but the length of the generated text reaches this specified length, the generation will stop and the generated content will be used as the final response to prevent the model from generating text indefinitely.

[0068] After obtaining the word-element embedding vectors, the low-rank adaptive small model at the edge nodes encodes these vectors using an attention mechanism. This attention mechanism allows the model to assign different attention weights to different word-element components based on their importance when processing them, thereby better capturing key information in the text. The encoded information is then decoded by the decoder, which continuously generates responses corresponding to the encoded instructions. When a generated response reaches a preset end marker or reaches a pre-defined length, the entire response from the start of generation to that point is output as the initial response.

[0069] S230 , calculating the generation probability of each word unit in the initial response through the edge node, and multiplying the generation probabilities of each word unit to obtain the confidence level of the entire sentence corresponding to the input instruction.

[0070] S240. Compare the whole sentence confidence with the sentence-level confidence threshold through the edge node. If the whole sentence confidence is less than the sentence-level confidence threshold, send a coordinated response instruction to the cloud server cluster.

[0071] S250. Select a specified number of target large models from multiple large models through the cloud server cluster, and call each target large model to correct at least one word in the initial response based on the word-level confidence threshold according to the input instructions and initial response in the collaborative response instruction to obtain multiple collaborative responses.

[0072] S260. Select the target collaborative response with the highest confidence as the true response from multiple collaborative responses through the cloud server cluster, and instruct the target large model that generates the true response to return the true response to the user.

[0073] S270. Obtain the true response, the input instruction corresponding to the true response, and the initial response from the target large model through the cloud server cluster, construct a ternary mapping group, and store the ternary mapping group in the response database on the cloud.

[0074] After the actual response is returned to the user, the cloud server cluster begins to obtain the actual response content, the original input instructions that generated the actual response, and the initial response originally generated by the edge node from the target large model that generated the actual response. These three types of data are combined into a three-element mapping group and stored in the cloud-based response database to accumulate data for subsequent model optimization.

[0075] S280. Monitor the cumulative amount of ternary mapping groups in the response database in real time through the cloud server cluster, and when the cumulative amount of currently stored ternary mapping groups reaches a data storage threshold, construct the currently stored ternary mapping groups into instruction and response data sets.

[0076] The cloud server cluster continuously monitors changes in the number of triplet mappings in the response database. A pre-set data storage threshold is set. When the number of triplet mappings in the database reaches this threshold, all currently stored triplet mappings are organized into a command and response dataset. This dataset contains a large number of user commands and corresponding model response data, forming an important foundation for subsequent model optimization.

[0077] S290. Perform data screening, data weighting, and format conversion operations on the command and response data sets through a cloud server cluster to obtain a standard data set that meets the requirements for optimizing a low-rank adaptive small model.

[0078] The cloud server cluster must perform a series of processing on the constructed instruction and response data sets. Data screening is to remove duplicate, erroneous or low-quality data; data weighting assigns different weights to different data based on the importance or reliability of the data; format conversion adjusts the data into a format suitable for the optimization of low-rank adaptive small models, and finally forms a standard data set, which is a data set specifically used to optimize low-rank adaptive small models.

[0079] S2100. When the cloud server cluster detects that a new standard data set is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working condition, a model optimization instruction is sent to the low-rank adaptive small model.

[0080] Idle working conditions refer to when the low-rank adaptive model is not processing tasks such as user input and is ready to accept and execute model optimization operations. The cloud server cluster regularly checks the response database. Once a newly generated standard dataset is detected and the low-rank adaptive model on the edge node is confirmed to be idle and will not affect normal business processing, it sends a model optimization instruction to the low-rank adaptive model, initiating the model optimization process.

[0081] S2110. When the edge node receives the model optimization instruction, it suspends the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster.

[0082] After receiving the model optimization instruction, the edge node temporarily stops the current low-rank adaptive small model to ensure that the optimization process is not interrupted. It then reports relevant information about the small model, such as model structure and parameters, to the cloud server cluster, allowing the cloud server cluster to understand the current status of the small model and provide necessary information for subsequent optimization.

[0083] S2120. Through the cloud server cluster, the low-rank adaptive small model is fine-tuned and optimized according to the new standard data set and the preset weighted loss function, and the optimized low-rank adaptive small model is transmitted back to the edge node for small model optimization.

[0084] The cloud server cluster fine-tunes the low-rank adaptive mini-model based on the new standard dataset and a preset weighted loss function. The weighted loss function measures the difference between the model's predictions and the actual results. Model optimization is achieved by adjusting model parameters to minimize the loss. Once optimized, the optimized low-rank adaptive mini-model is transferred back to the edge node, replacing the original model. This process improves the mini-model's performance.

[0085] Optionally, fine-tuning and optimizing the low-rank adaptive small model according to the new standard dataset and the preset weighted loss function may include:

[0086] Obtaining current standard data in the standard data set in sequence through a cloud server cluster, and inputting the current standard data into the low-rank adaptive small model to obtain a current initial response corresponding to the current standard data;

[0087] Through the cloud server cluster, a specified number of test large models are obtained from the multiple deployed large models, and the current standard data and the current initial response are input into each test large model respectively, and the current true response is obtained after the current initial response is corrected by each test large model;

[0088] Through the cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross entropy loss between the current initial response and the benchmark response of the current standard data;

[0089] Through the cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross entropy loss, and the low-rank adaptive small model is fine-tuned and optimized based on the current weighted total loss value;

[0090] Through the cloud server cluster, the operation of obtaining the current standard data in the standard data set is returned and executed in sequence until the end of the iteration condition is met.

[0091] In an embodiment of the present invention, the cloud server cluster sequentially extracts data from the standard data set, and the data extracted each time is the current standard data. The current standard data is input into the low-rank adaptive small model, and the small model processes it to generate a corresponding current initial response. The initial response will serve as the basis for subsequent comparison with the correction result of the large model. A specified number of large models are selected from the many large models deployed by the cloud server cluster as test large models. The current standard data and the current initial response generated by the low-rank adaptive small model are simultaneously input into each test large model. Each test large model will correct the current initial response based on its own capabilities, and finally obtain multiple current true responses.

[0092] For ease of understanding, the weighted total loss value of the small model fine-tuning optimization is composed of cross-entropy loss and KL divergence loss. Cross-entropy loss measures the difference between the initial response output by the small model and the baseline response that has not been modified by the large model, reflecting the degree of deviation of the small model from the correct result in terms of basic output. KL divergence loss measures the difference between the output response of the small model and the actual response after modification by the j-th large model in the cloud server cluster, reflecting the guiding role of large model correction in the optimization of the small model. The calculated KL divergence loss and cross-entropy loss are weighted and summed according to certain weights to obtain the current weighted total loss value. The current weighted total loss value comprehensively reflects the overall difference between the current initial response generated by the small model and multiple reference responses.

[0093] The parameters of the low-rank adaptive mini-model are adjusted based on the weighted total loss. Optimization algorithms such as gradient descent can be used to adjust the parameters of LR1 and LR2. When the loss is high, indicating a significant discrepancy between the mini-model's output and the true response, the algorithm increases the adjustment of the low-rank matrix parameters, aiming to reduce the loss. As the loss decreases, the adjustment is reduced accordingly, allowing the model to converge to a more optimal state. By continuously adjusting the low-rank matrix parameters based on the loss calculation results, the mini-model can better fit the true response and improve output quality.

[0094] The previous process of acquiring data from a standard dataset, calculating the loss, and optimizing the model is repeated in a continuous loop. During training, the weighted total loss continuously provides feedback on the performance of the small model, and low-rank adaptation promptly adjusts the model structure and parameters based on this feedback. As training progresses, the output of the small model gradually approaches the high-quality response of the large model, the loss value continuously decreases, and the model's generalization, robustness, and inference performance are significantly improved.

[0095] Each iteration will improve the performance of the low-rank adaptive small model. The optimization process stops when pre-set conditions are met, such as when the loss value drops to a certain level and no longer changes, or when the model accuracy reaches the target value.

[0096] The technical solution of the embodiment of the present invention focuses on the optimization of the low-rank adaptive small model, refines the overall solution, and builds a complete data-driven optimization system. Specifically, by collecting and processing real response data, a standard data set for optimizing the low-rank adaptive small model is constructed, and fine-tuning optimization is performed based on this, so that the small model can continuously learn and improve. With the accumulation of new data and the iterative optimization of the model, the small model's understanding and processing capabilities of various input instructions are gradually enhanced, and the generated initial response is more accurate and reliable, thereby improving the performance of the entire large and small model collaborative response system. In addition, by strictly controlling the optimization process, such as optimizing the low-rank adaptive small model of the edge node only when it meets the idle working conditions, the optimization process is avoided from interfering with normal business processing. At the same time, by setting the end iteration condition, the model is prevented from being over-optimized, and the rationality and stability of the model optimization process are guaranteed. In a real-time interactive system, this ensures that while optimizing the model, the system can still provide services to users stably, and there will be no service interruption or instability due to model optimization.

[0097] Example 3

[0098] Figure 3 This is a schematic diagram of the structure of a large and small model collaborative response device provided in the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0099] The initial response generation module 310 is configured to, upon receiving a user input instruction through an edge node, call a low-rank adaptive small model to perform small model calculation on the input instruction and generate an initial response;

[0100] The whole sentence confidence calculation module 320 is used to calculate the generation probability of each word in the initial response through the edge nodes, and multiply the generation probabilities of each word to obtain the whole sentence confidence corresponding to the input instruction;

[0101] A collaborative response instruction sending module 330 is configured to compare the whole sentence confidence with the sentence-level confidence threshold through an edge node, and send a collaborative response instruction to the cloud server cluster if the whole sentence confidence is less than the sentence-level confidence threshold;

[0102] The collaborative response generation module 340 is configured to select a specified number of target large models from the plurality of large models through the cloud server cluster, and invoke each target large model to modify at least one word in the initial response based on a word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, thereby obtaining a plurality of collaborative responses.

[0103] The real response returning module 350 is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through the cloud server cluster, and instruct the target large model that generates the real response to return the real response to the user.

[0104] Embodiments of the present invention deploy dynamically optimized, low-rank adaptive small models at edge nodes to quickly generate initial responses to user input commands. Compared to directly using a large model, the small model requires less computation and has faster processing speed. It can quickly generate initial responses locally, reducing user wait time. When a user issues a simple command, the small model can quickly provide preliminary results without having to immediately rely on the large cloud model, improving the system's responsiveness. When the confidence level of the initial response generated by the edge node falls below a threshold, multiple large models in the cloud server cluster perform collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large model modifies the initial response based on a word-level confidence threshold. The highest-confidence collaborative response from the multiple large models is selected and returned to the user as the true response, ensuring that the user ultimately receives a high-quality, more accurate answer. The cloud server cluster constructs a set of ternary mappings based on the true response generated by the large model, the input command, and the initial response. Once a certain number of ternary mappings are accumulated, a dataset is constructed to optimize the low-rank adaptive small model at the edge node. Through this continuous learning and optimization mechanism, the small model continuously improves its performance, better handling various user commands and adapting to different application scenarios.

[0105] Optionally, based on the above embodiments, the initial response generation module 310 may include:

[0106] An input instruction preprocessing unit is used to call a low-rank adaptive small model through an edge node to perform word segmentation and standardization preprocessing operations on the input instruction, and convert the preprocessed instruction into a corresponding word element embedding vector;

[0107] The response collection unit is used to call the low-rank adaptive small model through the edge node to encode the word embedding vector through the attention mechanism, and then use the decoder to decode and continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end mark or reach the specified length, all generated responses are used as the initial response.

[0108] Optionally, based on the above embodiments, the collaborative response generation module 340 may include:

[0109] The large model calling unit is used to call each target large model through the cloud server cluster to perform the following operations:

[0110] A correction determination unit is used to obtain the generation probability of all word units in the initial response and perform correction determination on each word unit in turn;

[0111] a word-unit retaining unit, configured to retain the target word-unit as a retained word-unit when determining that the generation probability of the target word-unit currently being processed is greater than or equal to the word-unit level threshold;

[0112] a word-gram regeneration unit, configured to regenerate a modified word-gram corresponding to the current word-gram according to the input instruction when determining that the generation probability of the target word-gram currently being processed is less than the word-gram level threshold;

[0113] The collaborative response construction unit is used to combine the retained word unit and the modified word unit to form a collaborative response.

[0114] Optionally, based on the above embodiments, after the target large model generating the real response is instructed by the cloud server cluster to return the real response to the user, the method may further include: a cluster monitoring unit configured to obtain the real response, the input instruction corresponding to the real response, and the initial response from the target large model via the cloud server cluster, construct a ternary mapping group, and store the ternary mapping group in a response database on the cloud;

[0115] The accumulated amount of the ternary mapping groups in the response database is monitored in real time by a cloud server cluster, and when the accumulated amount of the currently stored ternary mapping groups reaches a data storage threshold, the currently stored ternary mapping groups are constructed into a command and response data set;

[0116] The cloud server cluster performs data screening, data weighting, and format conversion on the command and response datasets to obtain a standard dataset that meets the requirements for optimizing low-rank adaptive small models.

[0117] When the cloud server cluster detects that a new standard data set is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working condition, a model optimization instruction is sent to the low-rank adaptive small model;

[0118] When the edge node receives the model optimization instruction, it suspends the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster;

[0119] Through the cloud server cluster, the low-rank adaptive small model is fine-tuned and optimized according to the new standard data set and the preset weighted loss function, and the optimized low-rank adaptive small model is transmitted back to the edge node for small model optimization.

[0120] Optionally, based on the above embodiments, the cluster monitoring unit can be further used to sequentially obtain current standard data from the standard data set through the cloud server cluster, and input the current standard data into the low-rank adaptive small model to obtain a current initial response corresponding to the current standard data;

[0121] Through the cloud server cluster, a specified number of test large models are obtained from the multiple deployed large models, and the current standard data and the current initial response are input into each test large model respectively, and the current true response is obtained after the current initial response is corrected by each test large model;

[0122] Through the cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross entropy loss between the current initial response and the benchmark response of the current standard data;

[0123] Through the cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross entropy loss, and the low-rank adaptive small model is fine-tuned and optimized based on the current weighted total loss value;

[0124] Through the cloud server cluster, the operation of obtaining the current standard data in the standard data set is returned and executed in sequence until the end of the iteration condition is met.

[0125] Optionally, based on the above embodiments, after obtaining the current true responses obtained after the current initial response is corrected by each test large model through the cloud server cluster, it can also include: a benchmark response determination unit, which is used to calculate the current average confidence of the current true response in all test large models through the cloud server cluster, and determine the current true response with the largest current average confidence as the benchmark response of the current initial response.

[0126] A collaborative response device for large and small models provided in an embodiment of the present invention can execute a collaborative response method for large and small models provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0127] Example 4

[0128] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0129] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0131] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors for running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a collaborative response method for large and small models.

[0132] That is, when receiving the user's input instruction through the edge node, the low-rank adaptive small model is called to perform small model calculation on the input instruction to generate an initial response;

[0133] Through the edge nodes, the generation probability of each word in the initial response is calculated, and the generation probability of each word is multiplied to obtain the confidence of the whole sentence corresponding to the input instruction;

[0134] The edge node compares the confidence of the entire sentence with the sentence-level confidence threshold. If the confidence of the entire sentence is less than the sentence-level confidence threshold, a coordinated response instruction is sent to the cloud server cluster.

[0135] A specified number of target large models are selected from multiple large models through a cloud server cluster, and each target large model is called to modify at least one word in the initial response based on a word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses;

[0136] Through the cloud server cluster, the target collaborative response with the highest confidence is selected as the true response from multiple collaborative responses, and the target large model that generates the true response is instructed to return the true response to the user.

[0137] In some embodiments, a collaborative response method for a size model can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the collaborative response method for a size model described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform a collaborative response method for a size model by any other appropriate means (e.g., by means of firmware).

[0138] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0143] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0144] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0145] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.< / eos>

Claims

1. A collaborative response method for large and small models, characterized in that: The method is collaboratively executed by an edge node deployed with a dynamically optimized low-rank adaptive small model and a cloud server cluster deployed with multiple large models, and includes: When receiving a user input instruction through the edge node, the low-rank adaptive small model is called to perform small model calculation on the input instruction to generate an initial response; Through the edge nodes, the generation probability of each word in the initial response is calculated, and the generation probability of each word is multiplied to obtain the confidence of the whole sentence corresponding to the input instruction; The edge node compares the confidence of the entire sentence with the sentence-level confidence threshold. If the confidence of the entire sentence is less than the sentence-level confidence threshold, a coordinated response instruction is sent to the cloud server cluster. A specified number of target large models are selected from multiple large models through a cloud server cluster, and each target large model is called to modify at least one word in the initial response based on a word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses; Through the cloud server cluster, the target collaborative response with the highest confidence is selected as the true response from multiple collaborative responses, and the target large model that generates the true response is instructed to return the true response to the user.

2. The method according to claim 1, characterized in that When receiving a user input instruction through the edge node, the low-rank adaptive small model is called to perform small model calculation on the input instruction to generate an initial response, including: Through the edge node, a low-rank adaptive small model is called to perform word segmentation and standardization preprocessing operations on the input instruction, and the preprocessed instruction is converted into a corresponding word element embedding vector; Through the edge node, the low-rank adaptive small model is called to encode the word embedding vector through the attention mechanism, and then the decoder is used to decode and continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end mark or reach the specified length, all generated responses are used as the initial response.

3. The method according to claim 1, characterized in that Through the cloud server cluster, each target large model is called to modify at least one word in the initial response based on the word-level confidence threshold according to the input instruction and the initial response in the collaborative response instruction, and multiple collaborative responses are obtained, including: Through the cloud server cluster, call each target large model to perform the following operations: Obtain the generation probability of all word units in the initial response, and make correction judgments for each word unit in turn; When it is determined that the generation probability of the target word currently being processed is greater than or equal to the word-level threshold, the target word is retained as a retained word; When it is determined that the generation probability of the target word currently being processed is less than the word-level threshold, regenerating a modified word corresponding to the current word according to the input instruction; The retained word and the modified word are combined to form a coordinated response.

4. The method according to any one of claims 1 to 3, characterized in that After the target large model generating the true response is instructed by the cloud server cluster to return the true response to the user, the following steps are also included: Obtaining the true response, the input instruction corresponding to the true response, and the initial response from the target large model through the cloud server cluster, constructing a ternary mapping group, and storing the ternary mapping group in a response database in the cloud; The accumulated amount of the ternary mapping groups in the response database is monitored in real time by a cloud server cluster, and when the accumulated amount of the currently stored ternary mapping groups reaches a data storage threshold, the currently stored ternary mapping groups are constructed into a command and response data set; The cloud server cluster performs data screening, data weighting, and format conversion on the command and response datasets to obtain a standard dataset that meets the requirements for optimizing low-rank adaptive small models. When the cloud server cluster detects that a new standard data set is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working condition, a model optimization instruction is sent to the low-rank adaptive small model; When the edge node receives the model optimization instruction, it suspends the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster; Through the cloud server cluster, the low-rank adaptive small model is fine-tuned and optimized according to the new standard data set and the preset weighted loss function, and the optimized low-rank adaptive small model is transmitted back to the edge node for small model optimization.

5. The method according to claim 4, characterized in that Through the cloud server cluster, the low-rank adaptive small model is fine-tuned and optimized according to the new standard dataset and the preset weighted loss function, including: Obtaining current standard data in the standard data set in sequence through a cloud server cluster, and inputting the current standard data into the low-rank adaptive small model to obtain a current initial response corresponding to the current standard data; Through the cloud server cluster, a specified number of test large models are obtained from the multiple deployed large models, and the current standard data and the current initial response are input into each test large model respectively, and the current true response is obtained after the current initial response is corrected by each test large model; Through the cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross entropy loss between the current initial response and the benchmark response of the current standard data; Through the cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross entropy loss, and the low-rank adaptive small model is fine-tuned and optimized based on the current weighted total loss value; Through the cloud server cluster, the operation of obtaining the current standard data in the standard data set is returned and executed in sequence until the end of the iteration condition is met.

6. The method according to claim 5, characterized in that After obtaining the current actual responses obtained by correcting the current initial responses through each test large model through the cloud server cluster, the following steps are also included: The cloud server cluster is used to calculate the current average confidence of the current true response in all the test models, and the current true response with the largest current average confidence is determined as the benchmark response of the current initial response.

7. A collaborative response device for large and small models, characterized in that: Deployed in a large model cluster deployed in the cloud, including: An initial response generation module is configured to, upon receiving a user input instruction through an edge node, call a low-rank adaptive small model to perform small model calculation on the input instruction and generate an initial response; The whole sentence confidence calculation module is used to calculate the generation probability of each word in the initial response through the edge node, and multiply the generation probability of each word to obtain the whole sentence confidence corresponding to the input instruction; A collaborative response instruction sending module is used to compare the confidence of the entire sentence with the sentence-level confidence threshold through the edge node, and if the confidence of the entire sentence is less than the sentence-level confidence threshold, send a collaborative response instruction to the cloud server cluster; A collaborative response generation module is used to select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to modify at least one word in the initial response based on the word-level confidence threshold according to the input instructions and initial response in the collaborative response instruction, thereby obtaining multiple collaborative responses; The real response returning module is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through the cloud server cluster, and instruct the target large model that generates the real response to return the real response to the user.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the coordinated response method of a large and small model according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a coordinated response method for large and small models according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements a coordinated response method of a size model according to any one of claims 1 to 6.

Images