A method, apparatus, device, medium and program for cooperative response of a large model

Abstract

The application discloses a kind of size model's collaborative response method, device, equipment, medium and program.The method comprises: through edge node, when receiving the input instruction of user, call low rank adaptive small model to generate initial response;Calculate the confidence of whole sentence in initial response;If the confidence of whole sentence is less than sentence level confidence threshold, send collaborative response instruction to cloud server cluster;Select specified number of target large model in multiple large models, at least one word element is corrected based on word element level confidence threshold, obtain multiple collaborative responses;Select the highest confidence target collaborative response as real response, and return real response to user.The application embodiment utilizes the collaborative work of size model to quickly give preliminary reply to simple problem, when complex problem is encountered, large model is involved optimization, to provide satisfactory answer for user, improve user experience, and the ability of processing instruction is enhanced by fine tuning optimization to low rank adaptive small model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a collaborative response method, apparatus, device, medium and program for large and small models. Background Technology

[0002] In recent years, with the continuous development of artificial intelligence technology, deep learning-based natural language processing models have made significant progress in fields such as intelligent voice assistants and machine translation. However, large-scale pre-trained models are usually deployed in the cloud due to their huge number of parameters and computational complexity. The frequent communication between terminal devices and the cloud, especially when network conditions are poor, often affects the user experience due to latency issues.

[0003] To improve inference performance, large-scale model collaborative inference methods have been proposed. These methods reduce the frequency of communication with the cloud by offloading some computational tasks to edge nodes or devices, thereby improving response speed. However, existing large-scale model collaborative inference methods still suffer from problems such as low inference accuracy of edge small models, frequent feedback from edge small models to cloud large models, poor connection between response results of cloud large models and edge small models, and the inability of small models to be continuously optimized. Summary of the Invention

[0004] Based on this, the present invention provides a method, apparatus, device, medium and program for coordinated response of large and small models, in order to solve the problem that existing large and small models are difficult to balance in terms of response efficiency and quality.

[0005] In a first aspect, embodiments of the present invention provide a collaborative response method for size-based models, the method comprising:

[0006] When a user input command is received via an edge node, a low-rank adaptive small model is invoked to perform small model calculations on the input command and generate an initial response.

[0007] By using edge nodes, the generation probability of each word in the initial response is calculated, and the generation probabilities of each word are multiplied together to obtain the confidence score of the whole sentence corresponding to the input command.

[0008] By using edge nodes, the confidence score of the whole sentence is compared with the sentence-level confidence score threshold. If the confidence score of the whole sentence is less than the sentence-level confidence score threshold, a collaborative response instruction is sent to the cloud server cluster.

[0009] By selecting a specified number of target large models from multiple large models through a cloud server cluster, and calling each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, multiple collaborative responses are obtained.

[0010] By using a cloud server cluster, the target collaborative response with the highest confidence level is selected as the real response from among multiple collaborative responses, and the target large model that generated the real response is instructed to return the real response to the user.

[0011] Secondly, embodiments of the present invention provide a cooperative response device for size models, the device comprising:

[0012] The initial response generation module is used to generate an initial response by calling a low-rank adaptive small model to perform small model calculations on the input command when the user's input command is received through the edge node.

[0013] The whole sentence confidence calculation module is used to calculate the generation probability of each word in the initial response through edge nodes, and multiply the generation probabilities of each word to obtain the whole sentence confidence corresponding to the input command.

[0014] The collaborative response instruction sending module is used to compare the confidence of the whole sentence with the sentence-level confidence threshold through edge nodes. If the confidence of the whole sentence is less than the sentence-level confidence threshold, a collaborative response instruction is sent to the cloud server cluster.

[0015] The collaborative response generation module is used to select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, to obtain multiple collaborative responses.

[0016] The Real Response Return module is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through a cloud server cluster, and instruct the target large model that generated the real response to return the real response to the user.

[0017] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a cooperative response method for a size model as described in any embodiment of the present invention.

[0021] Fourthly, a computer-readable storage medium is also provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a cooperative response method for a size model as described in any embodiment of the present invention.

[0022] Fifthly, a computer program product is also provided, the computer program product comprising a computer program, which, when executed by a processor, implements a cooperative response method for a size model as described in any embodiment of the present invention.

[0023] The technical solution of this invention, by deploying dynamically optimizable low-rank adaptive small models at edge nodes, can quickly provide initial responses to user input commands. Compared to directly using large models, small models have lower computational requirements and faster processing speeds, and can rapidly generate initial responses locally, reducing user waiting time. When a user issues a simple command, the small model can provide preliminary results in a short time, without immediately relying on large models in the cloud, thus improving the system's responsiveness. When the confidence score of the initial response generated by the edge node is lower than a threshold, multiple large models in the cloud server cluster are used for collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large models correct the lexical units in the initial response based on a lexical-level confidence threshold, selecting the highest-confidence response from the collaborative responses generated by multiple large models as the true response and returning it to the user, ensuring that the user ultimately receives a high-quality and more accurate answer. The cloud server cluster constructs triplet mapping groups based on the true responses generated by the large models, input commands, and initial responses. After accumulating a certain number, a dataset is built to optimize the low-rank adaptive small models at the edge nodes. Through this continuous learning and optimization mechanism, the small models can continuously improve their performance, better handle various user commands, and adapt to different application scenarios.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a collaborative response method for a size model according to Embodiment 1 of the present invention;

[0027] Figure 2This is a flowchart of another collaborative response method for a size model provided in Embodiment 2 of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a cooperative response device with a large and small model according to Embodiment 3 of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of an electronic device according to a cooperative response method of a size model provided in Embodiment 4 of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a large-small model collaborative response method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where large-small models collaboratively respond to user input commands. The method can be executed by a large-small model collaborative response device, which can be implemented in hardware and / or software. This device can be configured in an edge server and cloud server architecture system. Figure 1 As shown, the method includes:

[0034] S110. When a user's input instruction is received via an edge node, a low-rank adaptive small model is invoked to perform small model calculations on the input instruction, generating an initial response.

[0035] In this embodiment of the invention, an edge node is a computing node located close to the data source or user, providing services at the network edge. It possesses certain computing and storage capabilities, enabling it to quickly process some tasks locally, reducing data transmission latency and dependence on the cloud. In this embodiment, it is responsible for the initial processing of user commands. For example, a smart speaker in a smart home device can initially process user voice commands locally, thus belonging to the category of edge nodes. The low-rank adaptive small model is a small model optimized using low-rank adaptation technology. Low-rank adaptation reduces the number of parameters while maintaining model performance, lowering computational complexity and memory requirements, enabling the small model to run efficiently on edge nodes.

[0036] Specifically, in this embodiment of the invention, in the self-attention layer and feedforward network layer of the small model, the low-rank adaptation technique introduces two trainable low-rank matrices, LR1 and LR2, to parameterize the weight matrix. During fine-tuning and optimization, only the weight parameters of these two low-rank matrices need to be learned, while the original master parameters of the small model are frozen. This significantly reduces the number of parameters that need to be trained, lowers the complexity of training, and enables the model to adapt to new instruction-response patterns more efficiently. Compared to directly adjusting the large number of original model parameters, the low-rank adaptation technique significantly reduces computation and improves training speed. Furthermore, the small model, with its small size and low computational cost, is suitable for quickly processing simple tasks at edge nodes.

[0037] The low-rank adaptive small model deployed on the edge node is responsible for receiving user input instructions, and then performing preprocessing, word embedding, instruction encoding and decoding operations in sequence to finally generate a preliminary response result.

[0038] S120. Calculate the generation probability of each word in the initial response through edge nodes, and multiply the generation probabilities of each word together to obtain the confidence score of the whole sentence corresponding to the input instruction.

[0039] A word unit is the basic unit in text processing, typically the smallest meaningful unit obtained after word segmentation. In this embodiment of the invention, the model processes and calculates probabilities at the word unit level. The overall sentence confidence score is a value obtained by multiplying the generation probability of each word unit in the initial response; it measures the credibility of the entire initial response, with a higher value indicating a more reliable initial response. Edge nodes analyze the newly generated initial response, calculate the generation probability of each word unit, and then multiply these probabilities together to obtain a value that reflects the credibility of the entire initial response, i.e., the overall sentence confidence score.

[0040] S130. Through edge nodes, compare the confidence of the whole sentence with the sentence-level confidence threshold. If the confidence of the whole sentence is less than the sentence-level confidence threshold, send a collaborative response instruction to the cloud server cluster.

[0041] The sentence-level confidence threshold is a pre-defined standard value used to determine whether the overall confidence of the initial response reaches an acceptable level. Edge nodes compare the calculated overall confidence with the pre-defined sentence-level confidence threshold. If the overall confidence is lower than this threshold, it indicates that the initial response is not sufficiently credible. In this case, the edge node sends a collaborative response command to the cloud server cluster, requesting assistance from the larger model.

[0042] S140. Select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, to obtain multiple collaborative responses.

[0043] A cloud server cluster is a collection of multiple cloud servers that work collaboratively through cluster technology to provide powerful computing and storage capabilities. In this embodiment of the invention, it is responsible for handling complex tasks that edge nodes cannot complete independently, and optimizes the initial response using a large model. The word-level confidence threshold is a standard confidence value set for each word. When the large model corrects the word probability of the initial response, if the generation probability of a word is lower than this threshold, the large model will regenerate that word.

[0044] After receiving the collaborative response instruction from the edge node, the cloud server cluster selects a specified number of target large models from its deployed large models. Then, each target large model checks and corrects some or all of the tokens in the initial response according to the input instructions and initial response in the collaborative response instruction, based on the token-level confidence threshold. Finally, each target large model generates a collaborative response, thus obtaining multiple collaborative responses.

[0045] Optionally, through a cloud server cluster, each target large model is invoked to modify at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, using a word-level confidence threshold, to obtain multiple collaborative responses, which may include:

[0046] Using a cloud server cluster, each target large model is invoked to perform the following operations:

[0047] Obtain the generation probability of all words in the initial response, and then make correction judgments for each word in turn;

[0048] When the generation probability of the target word being processed is determined to be greater than or equal to the word-level threshold, the target word is retained as a reserved word.

[0049] When it is determined that the generation probability of the target word being processed is less than the word-level threshold, the corrected word corresponding to the current word is regenerated according to the input instruction.

[0050] Combine retained lexical units with modified lexical units to form a collaborative response.

[0051] Upon receiving a collaboration request from the edge nodes, the cloud server cluster invokes the previously selected target models as required, allowing them to execute a series of subsequent processing operations on the initial response in parallel, thereby improving processing efficiency. Each target model obtains the generation probability of each word in the initial response generated by the edge nodes, where the word generation probability reflects the likelihood of the model generating that word. These words are then analyzed and judged sequentially to determine whether correction is needed.

[0052] The lexical-level threshold is a pre-defined standard value. When the target model determines that the generation probability of a lexical currently being processed reaches or exceeds this threshold, it considers the generation of that lexical in the initial response to be relatively reliable, and thus retains it without modification as part of subsequent collaborative responses. Conversely, if the generation probability of the target lexical is less than the lexical-level threshold, the model considers that lexical unreliable. In this case, the model will regenerate a more suitable lexical based on the user's initial input command to replace the lexical in the initial response; the newly generated lexical is the correction lexical.

[0053] After processing all the tokens in the initial response, the target large model combines the retained tokens and the newly generated corrected tokens in sequence to form a complete and optimized text response, also known as the collaborative response. The collaborative responses generated by multiple target large models are then returned together to the cloud server cluster for further processing.

[0054] S150: Through a cloud server cluster, select the target collaborative response with the highest confidence level from multiple collaborative responses as the real response, and instruct the target large model that generated the real response to return the real response to the user.

[0055] The cloud server cluster selects the response with the highest confidence from the collaborative responses generated by multiple target large models and identifies it as the final real response. Simultaneously, the cloud server cluster instructs the target large model that generated this real response to send it back to the user, completing this large-scale model collaborative response process.

[0056] Furthermore, after obtaining the current true responses from the initial response through the cloud server cluster and correcting them for each large test model, it may also include:

[0057] The current average confidence level of the current real response in the entire test model is calculated using a cloud server cluster, and the current real response with the highest current average confidence level is determined as the baseline response for the current initial response.

[0058] The cloud server cluster obtains the current true response generated by each large test model after correcting the initial response. For each current true response, the cluster calculates its average confidence across all large test models. This calculation involves statistically averaging the confidence levels assigned to the current true response by each large test model. By calculating the average confidence, the cluster comprehensively considers the degree of acceptance of the same current true response by different large test models, thus evaluating the overall reliability of the response. After calculating the current average confidence for each of the current true responses, the cloud server cluster compares these average confidence levels and selects the current true response with the highest average confidence value, defining it as the baseline response for the current initial response. The baseline response represents the most reliable and expected response result under the comprehensive evaluation of multiple large test models. This baseline response will be used as a reference to calculate the model loss and fine-tune the low-rank adaptive small model.

[0059] This invention, through the deployment of dynamically optimizable low-rank adaptive small models at edge nodes, enables rapid initial responses to user input commands. Compared to directly using large models, small models have lower computational requirements and faster processing speeds, allowing for rapid local generation of initial responses and reducing user waiting time. When a user issues a simple command, the small model can provide preliminary results quickly, without immediately relying on large models in the cloud, thus improving the system's responsiveness. When the confidence level of the initial response generated by the edge node is below a threshold, multiple large models in the cloud server cluster are used for collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large models correct the lexical units in the initial response based on a lexical-level confidence threshold, selecting the highest-confidence response from the collaborative responses generated by multiple large models as the true response and returning it to the user, ensuring that the user ultimately receives a high-quality and more accurate answer. The cloud server cluster constructs triplet mapping groups based on the true responses generated by the large models, the input commands, and the initial responses. After accumulating a certain number, a dataset is built to optimize the low-rank adaptive small models at the edge nodes. Through this continuous learning and optimization mechanism, the small models can continuously improve their performance, better handle various user commands, and adapt to different application scenarios.

[0060] Example 2

[0061] Figure 2 This is a flowchart of another cooperative response method for a size model provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiment, and correspondingly, as shown below. Figure 2 As shown, the method specifically includes:

[0062] S210. Using edge nodes, the low-rank adaptive small model is invoked to perform word segmentation and standardization preprocessing operations on the input instruction, and the preprocessed instruction is converted into the corresponding word embedding vector.

[0063] Word segmentation is a fundamental operation in natural language processing (NLP). It divides a continuous text sequence into individual words or lexical units according to certain rules. These lexical units are the basic units for subsequent text analysis and processing. Standardization aims to unify text format and eliminate interference caused by format differences. Common operations include converting text to lowercase, removing punctuation marks, and removing stop words to improve the accuracy and consistency of model processing. Lexical embedding vectors (LEVVs) are a technique that represents lexical units as numerical vectors. Through specific algorithms, each lexical unit is mapped to a fixed-dimensional vector space, allowing the semantic information of lexical units to be represented in vector form. This facilitates model computation and learning, and the position and distance of the embedding vectors of different lexical units in the vector space reflect their semantic similarity.

[0064] Upon receiving user input, the edge node initiates a low-rank adaptive mini-model. The mini-model first preprocesses the instruction, segmenting the continuous text into individual tokens—a process known as word segmentation. Then, it standardizes these tokens, such as by unifying capitalization and removing special characters, to ensure a more standardized token format. After preprocessing, these instruction tokens are converted into numerical vectors that the computer can understand—terminus embedding vectors—facilitating subsequent computational processing by the model.

[0065] S220. Through edge nodes, the low-rank adaptive small model is called to encode the word embedding vector through the attention mechanism. The decoder is used to continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end marker or reach the specified length, all generated responses are used as the initial response.

[0066] In deep learning models, the encoder is responsible for extracting and transforming features from the input data, mapping the raw data to a hidden representation space. This hidden representation contains key feature information of the input data, preparing it for subsequent processing. In this embodiment of the invention, the encoder processes the word embedding vectors to extract key information. The decoder corresponds to the encoder; its role is to convert the hidden representation output by the encoder into the final output result. In natural language processing tasks, the decoder generates a text-based response based on the encoded information.

[0067] The end marker is a predefined special symbol used to indicate that the model-generated response content has been completed. When this end marker appears in the text generated by the model, it indicates that the response generation process has ended. For example, it can be set to "...". <eos>The "(End Of Sentence)" field serves as the end marker. The specified length is the maximum length limit for the model's generated response. If the model does not encounter the end marker during response generation, but the generated text reaches this specified length, generation will stop, and the already generated content will be used as the final response, preventing the model from generating text indefinitely.

[0068] After obtaining the word embedding vectors, the low-rank adaptive small model of the edge nodes encodes these vectors using an attention mechanism. This attention mechanism allows the model to assign different attention weights based on the importance of different words when processing them, thus better capturing key information in the text. The encoded information is then decoded by a decoder, continuously generating response content corresponding to the encoded instructions. When a preset end marker appears in the generated response, or when a pre-defined length is reached, the entire response content from the start of generation up to that point is used as the initial response output.

[0069] S230. Calculate the generation probability of each word in the initial response through edge nodes, and multiply the generation probabilities of each word together to obtain the confidence score of the whole sentence corresponding to the input instruction.

[0070] S240. Through edge nodes, compare the confidence of the whole sentence with the sentence-level confidence threshold. If the confidence of the whole sentence is less than the sentence-level confidence threshold, send a collaborative response instruction to the cloud server cluster.

[0071] S250. Select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, to obtain multiple collaborative responses.

[0072] S260: Through a cloud server cluster, select the target collaborative response with the highest confidence level from multiple collaborative responses as the real response, and instruct the target large model that generated the real response to return the real response to the user.

[0073] S270. Obtain the real response, the input command corresponding to the real response, and the initial response from the target large model through the cloud server cluster, construct a ternary mapping group, and store the ternary mapping group in the response database in the cloud.

[0074] After the actual response is returned to the user, the cloud server cluster begins to retrieve the actual response content, the original input instructions that generated the actual response, and the initial response initially generated by the edge nodes from the target large model that generated the actual response. These three types of data are combined into a ternary mapping group and then stored in the cloud response database to accumulate data for subsequent model optimization.

[0075] S280. Monitor the cumulative amount of ternary mapping groups in the response database in real time through the cloud server cluster, and when the cumulative amount of the currently stored ternary mapping groups reaches the data storage threshold, construct the currently stored ternary mapping groups into an instruction and response dataset.

[0076] The cloud server cluster continuously monitors the changes in the number of triplet mapping groups in the response database. A pre-set data storage threshold is used; when the number of triplet mapping groups in the database reaches this threshold, all currently stored triplet mapping groups are reorganized into a command and response dataset. This dataset contains a large amount of user commands and corresponding model response data, serving as a crucial foundation for subsequent model optimization.

[0077] S290. Perform data filtering, data weighting, and format conversion operations on the instruction and response datasets through a cloud server cluster to obtain a standard dataset that conforms to the requirements for optimizing low-rank adaptive small models.

[0078] The cloud server cluster performs a series of processes on the constructed instruction and response dataset. Data filtering removes duplicate, erroneous, or low-quality data. Data weighting assigns different weights to different data based on their importance or reliability. Format conversion adjusts the data to a format suitable for optimization of low-rank adaptive small models, ultimately forming a standard dataset, which is a dataset specifically designed for optimizing low-rank adaptive small models.

[0079] S2100: When the cloud server cluster detects that a new standard dataset is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working conditions, it sends a model optimization instruction to the low-rank adaptive small model.

[0080] Idle working conditions refer to a state where the low-rank adaptive small model is not processing user input commands or other tasks, and is in an acceptable state to perform model optimization operations. The cloud server cluster periodically checks the response database. Once it finds a newly generated standard dataset and confirms that the low-rank adaptive small model on the edge node is in an idle state and will not affect normal business processing, it sends a model optimization command to the low-rank adaptive small model to start the model optimization process.

[0081] S2110. When the edge node receives the model optimization instruction, it pauses the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster.

[0082] Upon receiving the model optimization instruction, the edge node temporarily halts its current work on the low-rank adaptation small model to ensure the optimization process remains undisturbed. Then, it reports relevant information about the small model, such as its structure and parameters, to the cloud server cluster, allowing the cluster to understand the current status of the small model and providing necessary information for subsequent optimization.

[0083] S2120. Through the cloud server cluster, the low-rank adaptive small model is fine-tuned and optimized according to the new standard dataset and the preset weighted loss function, and the optimized low-rank adaptive small model is transmitted back to the edge node for small model optimization.

[0084] The cloud server cluster fine-tunes the low-rank adaptive small model based on a new standard dataset and a pre-defined weighted loss function. The weighted loss function measures the difference between the model's predictions and the actual results. By adjusting the model parameters to minimize the loss value, model optimization is achieved. After optimization, the optimized low-rank adaptive small model is transmitted back to the edge nodes to replace the original model, thereby improving the performance of the small model.

[0085] Optionally, fine-tuning the low-rank adaptive small model based on a new standard dataset and a preset weighted loss function may include:

[0086] By using a cloud server cluster, the current standard data is sequentially obtained from the standard dataset and then input into the low-rank adaptive small model to obtain the current initial response corresponding to the current standard data.

[0087] By using a cloud server cluster, a specified number of test large models are obtained from multiple deployed large models. The current standard data and the current initial response are input into each test large model respectively, and the current real responses are obtained after the current initial response is corrected by each test large model.

[0088] Using a cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross-entropy loss between the current initial response and the benchmark response of the current standard data.

[0089] Using a cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross-entropy loss. The low-rank adaptive small model is then fine-tuned and optimized based on the current weighted total loss value.

[0090] The cloud server cluster returns the operation of sequentially retrieving the current standard data from the standard dataset until the end of the iteration condition is met.

[0091] In this embodiment of the invention, the cloud server cluster sequentially retrieves data entries from a standard dataset, with each retrieved entry serving as the current standard data. This current standard data is input into a low-rank adaptive small model, which processes it to generate a corresponding initial response. This initial response serves as the basis for subsequent comparison with the corrected results of the large model. A specified number of large models are selected from the numerous large models deployed on the cloud server cluster as test large models. The current standard data and the initial responses generated by the low-rank adaptive small models are simultaneously input into each test large model. Each test large model corrects its initial response based on its own capabilities, ultimately yielding multiple current true responses.

[0092] For ease of understanding, the weighted total loss value for small model fine-tuning optimization consists of cross-entropy loss and KL divergence loss. Cross-entropy loss measures the difference between the initial response output by the small model and the baseline response without modification by the large model, reflecting the degree of deviation of the small model from the correct result in its basic output. KL divergence loss measures the difference between the output response of the small model and the true response after modification by the j-th large model in the cloud server cluster, reflecting the guiding role of large model correction in small model optimization. The calculated KL divergence loss and cross-entropy loss are weighted and summed according to certain weights to obtain the current weighted total loss value. The current weighted total loss value comprehensively reflects the overall difference between the current initial response generated by the small model and multiple reference responses.

[0093] The parameters of the low-rank adaptive small model are adjusted based on the weighted total loss value. Optimization algorithms such as gradient descent can be used to adjust the parameters of LR1 and LR2. When the loss value is large, it indicates a significant discrepancy between the small model's output and the true response. In this case, the algorithm will increase the adjustment magnitude of the low-rank matrix parameters, guiding the small model towards reducing the loss. As the loss value gradually decreases, the adjustment magnitude also decreases accordingly, allowing the model to gradually converge to a better state. By continuously adjusting the low-rank matrix parameters based on the loss calculation results, the small model can better fit the true response, improving output quality.

[0094] Repeat the process of acquiring data from the standard dataset, calculating the loss, and optimizing the model, continuously iterating throughout the training process. During training, the weighted total loss value continuously reflects the performance of the smaller model, while low-rank adaptation adjusts the model structure and parameters in a timely manner based on the feedback. As training progresses, the output of the smaller model gradually approaches the high-quality response of the larger model, the loss value continuously decreases, and the model's generalization ability, robustness, and inference performance are significantly improved.

[0095] Each iteration improves the performance of the low-rank adaptive small model to some extent. The optimization process stops when pre-defined termination conditions are met, such as the loss value decreasing to a certain level and no longer changing, or the model accuracy reaching the target value.

[0096] The technical solution of this invention focuses on the optimization of low-rank adaptive small models, refining the overall solution and constructing a complete data-driven optimization system. Specifically, by collecting and processing real response data, a standard dataset for optimizing low-rank adaptive small models is constructed, and fine-tuning is performed based on this dataset, allowing the small model to continuously learn and improve. With the accumulation of new data and iterative optimization of the model, the small model's understanding and processing capabilities for various input commands gradually improve, resulting in more accurate and reliable initial responses, thereby improving the performance of the entire large and small model collaborative response system. Furthermore, by strictly controlling the optimization process, such as only optimizing when the low-rank adaptive small model at the edge node meets the idle working conditions, interference with normal business processing during the optimization process is avoided. Simultaneously, by setting termination iteration conditions, over-optimization of the model is prevented, ensuring the rationality and stability of the model optimization process. In a real-time interactive system, this ensures that while optimizing the model, the system can still stably provide services to users, without service interruption or instability due to model optimization.

[0097] Example 3

[0098] Figure 3 This is a schematic diagram of the structure of a cooperative response device with a large and small model provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0099] The initial response generation module 310 is used to generate an initial response by calling a low-rank adaptive small model to perform small model calculations on the input instruction when the user's input instruction is received through the edge node;

[0100] The whole sentence confidence calculation module 320 is used to calculate the generation probability of each word in the initial response through edge nodes, and multiply the generation probabilities of each word to obtain the whole sentence confidence corresponding to the input command.

[0101] The collaborative response instruction sending module 330 is used to compare the confidence of the whole sentence with the sentence-level confidence threshold through the edge node. If the confidence of the whole sentence is less than the sentence-level confidence threshold, a collaborative response instruction is sent to the cloud server cluster.

[0102] The collaborative response generation module 340 is used to select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, to obtain multiple collaborative responses.

[0103] The Real Response Return Module 350 is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through a cloud server cluster, and instruct the target large model that generated the real response to return the real response to the user.

[0104] This invention, through the deployment of dynamically optimizable low-rank adaptive small models at edge nodes, enables rapid initial responses to user input commands. Compared to directly using large models, small models have lower computational requirements and faster processing speeds, allowing for rapid local generation of initial responses and reducing user waiting time. When a user issues a simple command, the small model can provide preliminary results quickly, without immediately relying on large models in the cloud, thus improving the system's responsiveness. When the confidence level of the initial response generated by the edge node is below a threshold, multiple large models in the cloud server cluster are used for collaborative processing. Leveraging their powerful language understanding and generation capabilities, the large models correct the lexical units in the initial response based on a lexical-level confidence threshold, selecting the highest-confidence response from the collaborative responses generated by multiple large models as the true response and returning it to the user, ensuring that the user ultimately receives a high-quality and more accurate answer. The cloud server cluster constructs triplet mapping groups based on the true responses generated by the large models, the input commands, and the initial responses. After accumulating a certain number, a dataset is built to optimize the low-rank adaptive small models at the edge nodes. Through this continuous learning and optimization mechanism, the small models can continuously improve their performance, better handle various user commands, and adapt to different application scenarios.

[0105] Optionally, based on the above embodiments, the initial response generation module 310 may include:

[0106] The input instruction preprocessing unit is used to call the low-rank adaptive small model through edge nodes to perform word segmentation and standardization preprocessing operations on the input instruction, and to convert the preprocessed instruction into the corresponding word embedding vector.

[0107] The response collection unit is used to call the low-rank adaptive small model through the edge node to encode the word embedding vector through the attention mechanism, and then use the decoder to continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end marker or reach a specified length, all generated responses are used as the initial response.

[0108] Optionally, based on the above embodiments, the collaborative response generation module 340 may include:

[0109] The large model invocation unit is used to invoke each target large model through a cloud server cluster to perform the following operations:

[0110] The correction and determination unit is used to obtain the generation probability of all words in the initial response and to make correction and determination for each word in turn.

[0111] The lexical retention unit is used to retain the target lexical as a reserved lexical when the generation probability of the target lexical being processed is greater than or equal to the lexical-level threshold.

[0112] The lexical regeneration unit is used to regenerate the corrected lexical corresponding to the current lexical according to the input instruction when the generation probability of the target lexical being processed is less than the lexical level threshold.

[0113] The collaborative response building unit is used to combine retained lexical units and modified lexical units to form a collaborative response.

[0114] Optionally, based on the above embodiments, after the target large model that generates the real response returns the real response to the user through the cloud server cluster, it may further include: a cluster monitoring unit, used to obtain the real response, the input instruction corresponding to the real response and the initial response from the target large model through the cloud server cluster, construct a ternary mapping group, and store the ternary mapping group in the response database in the cloud.

[0115] The cumulative amount of ternary mapping groups in the response database is monitored in real time by cloud server clusters, and when the cumulative amount of the currently stored ternary mapping groups reaches the data storage threshold, the currently stored ternary mapping groups are constructed into an instruction and response dataset.

[0116] By using a cloud server cluster to perform data filtering, weighting, and format conversion on the instruction and response datasets, a standard dataset suitable for optimizing low-rank adaptive small models is obtained.

[0117] When the cloud server cluster detects that a new standard dataset is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working conditions, it sends a model optimization instruction to the low-rank adaptive small model.

[0118] When the edge node receives the model optimization instruction, it pauses the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster.

[0119] The low-rank adaptive small model is fine-tuned and optimized using a cloud server cluster based on a new standard dataset and a preset weighted loss function. The optimized low-rank adaptive small model is then transmitted back to the edge nodes for small model optimization.

[0120] Optionally, based on the above embodiments, the cluster monitoring unit can be further configured to sequentially obtain the current standard data from the standard dataset through the cloud server cluster, and input the current standard data into the low-rank adaptive small model to obtain the current initial response corresponding to the current standard data;

[0121] By using a cloud server cluster, a specified number of test large models are obtained from multiple deployed large models. The current standard data and the current initial response are input into each test large model respectively, and the current real responses are obtained after the current initial response is corrected by each test large model.

[0122] Using a cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross-entropy loss between the current initial response and the benchmark response of the current standard data.

[0123] Using a cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross-entropy loss. The low-rank adaptive small model is then fine-tuned and optimized based on the current weighted total loss value.

[0124] The cloud server cluster returns the operation of sequentially retrieving the current standard data from the standard dataset until the end of the iteration condition is met.

[0125] Optionally, based on the above embodiments, after obtaining the current true responses obtained by each test model after the current initial response is corrected by the cloud server cluster, it may further include: a benchmark response determination unit, used to calculate the current average confidence of the current true response in all test models through the cloud server cluster, and determine the current true response with the largest current average confidence as the benchmark response of the current initial response.

[0126] The size model cooperative response device provided in this embodiment of the invention can execute the size model cooperative response method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0127] Example 4

[0128] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a cooperative response method for a size model.

[0132] That is, when a user input command is received through an edge node, a low-rank adaptive small model is invoked to perform small model calculations on the input command and generate an initial response.

[0133] By using edge nodes, the generation probability of each word in the initial response is calculated, and the generation probabilities of each word are multiplied together to obtain the confidence score of the whole sentence corresponding to the input command.

[0134] By using edge nodes, the confidence score of the whole sentence is compared with the sentence-level confidence score threshold. If the confidence score of the whole sentence is less than the sentence-level confidence score threshold, a collaborative response instruction is sent to the cloud server cluster.

[0135] By selecting a specified number of target large models from multiple large models through a cloud server cluster, and calling each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, multiple collaborative responses are obtained.

[0136] By using a cloud server cluster, the target collaborative response with the highest confidence level is selected as the real response from among multiple collaborative responses, and the target large model that generated the real response is instructed to return the real response to the user.

[0137] In some embodiments, a size-model cooperative response method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the size-model cooperative response method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a size-model cooperative response method by any other suitable means (e.g., by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0142] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0143] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0145] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / eos>

Claims

1. A cooperative response method for a size model, characterized in that, The method, executed collaboratively by edge nodes deployed with dynamically optimizable low-rank adaptive small models and a cloud server cluster deployed with multiple large models, includes: When a user input command is received via an edge node, a low-rank adaptive small model is invoked to perform small model calculations on the input command and generate an initial response. By using edge nodes, the generation probability of each word in the initial response is calculated, and the generation probabilities of each word are multiplied together to obtain the confidence score of the whole sentence corresponding to the input command. By using edge nodes, the confidence score of the whole sentence is compared with the sentence-level confidence score threshold. If the confidence score of the whole sentence is less than the sentence-level confidence score threshold, a collaborative response instruction is sent to the cloud server cluster. By selecting a specified number of target large models from multiple large models through a cloud server cluster, and calling each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, multiple collaborative responses are obtained. By using a cloud server cluster, the target collaborative response with the highest confidence level is selected as the real response from among multiple collaborative responses, and the target large model that generated the real response is instructed to return the real response to the user.

2. The method according to claim 1, characterized in that, Upon receiving a user input command via an edge node, a low-rank adaptive small model is invoked to perform small model calculations on the input command, generating an initial response, including: By using edge nodes, a low-rank adaptive small model is invoked to perform word segmentation and standardization preprocessing on the input instructions, and the preprocessed instructions are converted into corresponding word embedding vectors. By using edge nodes, the low-rank adaptive small model is called to encode the word embedding vector through the attention mechanism. The decoder is then used to continuously generate responses corresponding to the encoding results of each instruction. When the continuously generated responses carry an end marker or reach a specified length, all generated responses are used as the initial response.

3. The method according to claim 1, characterized in that, Through a cloud server cluster, each target large model is invoked to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, using a word-level confidence threshold, resulting in multiple collaborative responses, including: Using a cloud server cluster, each target large model is invoked to perform the following operations: Obtain the generation probability of all words in the initial response, and then make correction judgments for each word in turn; When the generation probability of the target word being processed is determined to be greater than or equal to the word-level threshold, the target word is retained as a reserved word. When it is determined that the generation probability of the target word being processed is less than the word-level threshold, the corrected word corresponding to the current word is regenerated according to the input instruction. Combine retained lexical units with modified lexical units to form a collaborative response.

4. The method according to any one of claims 1-3, characterized in that, After the target large model, which generates the realistic response via a cloud server cluster, returns the real response to the user, it also includes: The system obtains the actual response, the corresponding input instructions, and the initial response from the target large model through a cloud server cluster, constructs a ternary mapping group, and stores the ternary mapping group in the response database in the cloud. The cumulative amount of ternary mapping groups in the response database is monitored in real time by cloud server clusters, and when the cumulative amount of the currently stored ternary mapping groups reaches the data storage threshold, the currently stored ternary mapping groups are constructed into an instruction and response dataset. By using a cloud server cluster to perform data filtering, weighting, and format conversion on the instruction and response datasets, a standard dataset suitable for optimizing low-rank adaptive small models is obtained. When the cloud server cluster detects that a new standard dataset is stored in the response database and the low-rank adaptive small model deployed on the edge node meets the idle working conditions, it sends a model optimization instruction to the low-rank adaptive small model. When the edge node receives the model optimization instruction, it pauses the current running state of the low-rank adaptive small model and reports the model information of the low-rank adaptive small model to the cloud server cluster. The low-rank adaptive small model is fine-tuned and optimized using a cloud server cluster based on a new standard dataset and a preset weighted loss function. The optimized low-rank adaptive small model is then transmitted back to the edge nodes for small model optimization.

5. The method according to claim 4, characterized in that, The low-rank adaptive small model is fine-tuned and optimized using a cloud server cluster based on a new standard dataset and a pre-defined weighted loss function, including: By using a cloud server cluster, the current standard data is sequentially obtained from the standard dataset and then input into the low-rank adaptive small model to obtain the current initial response corresponding to the current standard data. By using a cloud server cluster, a specified number of test large models are obtained from multiple deployed large models. The current standard data and the current initial response are input into each test large model respectively, and the current real responses are obtained after the current initial response is corrected by each test large model. Using a cloud server cluster, calculate the KL divergence loss between the current initial response and each current true response, and calculate the cross-entropy loss between the current initial response and the benchmark response of the current standard data. Using a cloud server cluster, the current weighted total loss value corresponding to the current standard data is calculated based on the calculated KL divergence loss and cross-entropy loss. The low-rank adaptive small model is then fine-tuned and optimized based on the current weighted total loss value. The cloud server cluster returns the operation of sequentially retrieving the current standard data from the standard dataset until the end of the iteration condition is met.

6. The method according to claim 5, characterized in that, After obtaining the current true responses from the initial response obtained through the cloud server cluster and corrected by each large test model, the following is also included: The current average confidence level of the current real response in the entire test model is calculated using a cloud server cluster, and the current real response with the highest current average confidence level is determined as the baseline response for the current initial response.

7. A cooperative response device for large and small models, characterized in that, Deployed within a large cluster of models located in the cloud, including: The initial response generation module is used to generate an initial response by calling a low-rank adaptive small model to perform small model calculations on the input command when the user's input command is received through the edge node. The whole sentence confidence calculation module is used to calculate the generation probability of each word in the initial response through edge nodes, and multiply the generation probabilities of each word to obtain the whole sentence confidence corresponding to the input command. The collaborative response instruction sending module is used to compare the confidence of the whole sentence with the sentence-level confidence threshold through edge nodes. If the confidence of the whole sentence is less than the sentence-level confidence threshold, a collaborative response instruction is sent to the cloud server cluster. The collaborative response generation module is used to select a specified number of target large models from multiple large models through a cloud server cluster, and call each target large model to correct at least one word in the initial response based on the input instructions and initial response in the collaborative response instructions, and based on the word-level confidence threshold, to obtain multiple collaborative responses. The Real Response Return module is used to select the target collaborative response with the highest confidence as the real response from multiple collaborative responses through a cloud server cluster, and instruct the target large model that generated the real response to return the real response to the user.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform a cooperative response method for a size model according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a cooperative response method for a size model according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a cooperative response method for a size model according to any one of claims 1-6.

Images