Multi-model collaborative reasoning method, device and equipment and storage medium
Through the multi-model collaborative reasoning method, the synergy between the first and second models is used to process data and generate intermediate marks, the problem of low memory access bandwidth in the inference process of intelligent model is solved, and a more efficient inference process is achieved.
Patent Information
- Application Number
- CN202411748375.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-07
AI Technical Summary
During the inference process of intelligent model, the inference process is slower due to the low memory access bandwidth.
The multi-model collaborative inference method is adopted to generate a first mark through the first model processing data, and an intermediate mark is generated in association with the preset mark. The second model checksum the intermediate mark until the stop condition is met, and a multi-model collaborative inference result is generated.
The efficiency of intelligent model inference is improved through the multi-model collaborative reasoning method, and the synergy of multi-models is used to optimize data processing and memory access, which improves the speed of the inference process.
Smart Images
Figure CN119578552A_ABST
Abstract
Description
Background Art
[0002] With the development of the times, intelligent model reasoning has gradually attracted widespread attention in the industry. Model reasoning usually requires the use of autoregressive sampling methods. Autoregressive sampling methods usually require serial decoding of data, and each generated label requires all parameters to be transferred from the storage unit to the technical unit. In the case of low memory access bandwidth, the reasoning process will be slow. Summary of the invention
[0003] The present disclosure provides a multi-model collaborative reasoning method, apparatus, device and storage medium, which improves the efficiency of intelligent model reasoning at least to a certain extent.
[0004] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0005] According to one aspect of the present disclosure, a multi-model collaborative reasoning method is provided, comprising:
[0006] Processing the data based on the first model to obtain a first label;
[0007] generating a first intermediate mark based on the association data between the first mark and the preset mark;
[0008] generating a second intermediate mark based on the first intermediate mark and the associated data of the preset mark;
[0009] Verifying the first intermediate marker and the second intermediate marker based on the second model;
[0010] If the verification fails, at least one of the first mark, the first intermediate mark, and the second intermediate mark is processed based on the second model to obtain a multi-model collaborative reasoning result.
[0011] In one embodiment of the present disclosure, the method further includes:
[0012] generating a third intermediate mark based on the second intermediate mark and the associated data of the preset mark;
[0013] Repeat the above steps until the stopping condition is met, and determine the third intermediate mark finally generated as the multi-model collaborative reasoning result.
[0014] In one embodiment of the present disclosure, the method further includes:
[0015] When the first intermediate marker and / or the second intermediate marker meets the preset condition, the intermediate marker that meets the preset condition is determined as the multi-model collaborative reasoning result.
[0016] In one embodiment of the present disclosure, the method further includes:
[0017] Determine a delay function based on the amount of data transmission required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data;
[0018] Determine a load balancing function based on decision variables for generating the first intermediate marker and / or generating the second intermediate marker and computing power data provided for generating the first intermediate marker and / or generating the second intermediate marker;
[0019] The computing power data allocated to the first model and the second model are determined based on the delay function, the load balancing function and the preset algorithm.
[0020] In one embodiment of the present disclosure, the delay function includes:
[0021]
[0022] Token x The first middle mark or the second middle mark, V (Token x ) is the verification token x The required data transmission amount, j is the first model, j′ is the second model, R j,j′ represents the transmission speed between the jth and j'th nodes, C(Token x ) is the verification token x Required computing power data, H j′ Indicates j' verification Token x Capability data, R(Token x ) is the acceptable delay ratio data for users, T accept Accept delayed data for users.
[0023] In one embodiment of the present disclosure, the load balancing function includes:
[0024]
[0025] Among them, L j is the load balancing data, j is the first model, C j The first model computing power data, Token x is the first middle mark or the second middle mark, Indicates the generation of Token x decision variables.
[0026] In one embodiment of the present disclosure, the computing power data allocated to the first model and the second model is determined based on the delay function, the load balancing function and the preset algorithm, including:
[0027] Determine the objective function based on the delay function and the load balancing function;
[0028] Determine the computing power data allocated to the first model and the second model based on the objective function and the greedy algorithm;
[0029] The objective function includes:
[0030]
[0031] Among them, w1 and w2 are weight values, and M is Token x Total number, N is the total amount of computing power data.
[0032] According to another aspect of the present disclosure, a multi-model collaborative reasoning device is provided, comprising:
[0033] A first processing module, configured to process data based on a first model to obtain a first label;
[0034] A first generating module, configured to generate a first intermediate mark based on the association data between the first mark and the preset mark;
[0035] A second generating module, configured to generate a second intermediate mark based on the first intermediate mark and associated data of the preset mark;
[0036] A verification module, configured to verify the first intermediate marker and the second intermediate marker based on the second model;
[0037] The second processing module is used to process at least one of the first mark, the first intermediate mark and the second intermediate mark based on the second model to obtain a multi-model collaborative reasoning result when the verification fails.
[0038] In one embodiment of the present disclosure, the device further includes:
[0039] A third generating module, configured to generate a third intermediate mark based on the second intermediate mark and associated data of the preset mark;
[0040] The iteration module is used to repeat the above steps until the stopping condition is met, and determine the finally generated third intermediate mark as the multi-model collaborative reasoning result.
[0041] In one embodiment of the present disclosure, the device further includes:
[0042] A determination module is used to determine the intermediate marker that meets the preset condition as the multi-model collaborative reasoning result when the first intermediate marker and / or the second intermediate marker meets the preset condition.
[0043] In one embodiment of the present disclosure, the device further includes:
[0044] A second determination module is used to determine the delay function based on the data transmission volume required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data;
[0045] A third determination module, configured to determine a load balancing function based on a decision variable for generating the first intermediate mark and / or generating the second intermediate mark, and computing power data provided for generating the first intermediate mark and / or generating the second intermediate mark;
[0046] The fourth determination module is used to determine the computing power data allocated to the first model and the second model based on the delay function, the load balancing function and the preset algorithm.
[0047] In one embodiment of the present disclosure, the delay function includes:
[0048]
[0049] Token x The first middle mark or the second middle mark, V (Token x ) is the verification token x The required data transmission amount, j is the first model, j′ is the second model, R j,j′ represents the transmission speed between the jth and j'th nodes, C(Token x ) is the verification token x Required computing power data, H j′ Indicates j' verification Token x Capability data, R(Token x ) is the acceptable delay ratio data for users, T accept Accept delayed data for users.
[0050] In one embodiment of the present disclosure, the load balancing function includes:
[0051]
[0052] Among them, L j is the load balancing data, j is the first model, C j The first model computing power data, Token x is the first middle mark or the second middle mark, Indicates the generation of Tokenx decision variables.
[0053] In one embodiment of the present disclosure, the fourth determining module includes:
[0054] A first determining unit, configured to determine an objective function based on a delay function and a load balancing function;
[0055] A second determining unit, configured to determine computing power data allocated to the first model and the second model based on an objective function and a greedy algorithm;
[0056] The objective function includes:
[0057]
[0058] Among them, w1 and w2 are weight values, and M is Token x Total number, N is the total amount of computing power data.
[0059] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned multi-model collaborative reasoning method by executing the executable instructions.
[0060] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the multi-model collaborative reasoning method described above is implemented.
[0061] The multi-model collaborative reasoning method, apparatus, device and storage medium provided by the embodiments of the present disclosure process data based on a first model to obtain a first label, generate a first intermediate label based on associated data of the first label and a preset label, generate a second intermediate label based on the associated data of the first intermediate label and the preset label, verify the first intermediate label and the second intermediate label based on a second model, and if the verification fails, process at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0062] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0064] Figure 1 A flowchart of a multi-model collaborative reasoning method in an embodiment of the present disclosure is shown;
[0065] Figure 2 Another flowchart of a multi-model collaborative reasoning method in an embodiment of the present disclosure is shown;
[0066] Figure 3 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown;
[0067] Figure 4 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown;
[0068] Figure 5 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown;
[0069] Figure 6 An architecture diagram of a multi-model collaborative reasoning device in an embodiment of the present disclosure is shown;
[0070] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0071] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0072] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0073] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0074] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0075] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0076] In the process of intelligent model reasoning, the lack of full utilization of GPU computing power and video memory resources is an important problem currently faced by intelligent models. The video memory of mainstream GPUs is usually above 32GB. One GPU card can carry 2-3 6B large models, but related technologies usually only deploy one. In the process of model reasoning, serial efficiency is low.
[0077] In order to solve the above problems, the embodiments of the present disclosure provide a multi-model collaborative reasoning method, apparatus, device and storage medium.
[0078] Figure 1 A flow chart of a multi-model collaborative reasoning method in an embodiment of the present disclosure is shown.
[0079] like Figure 1 As shown, the method may include:
[0080] S110, processing data based on a first model to obtain a first label.
[0081] In some embodiments, the first model may include a small model. Different from a large model, a small model usually has relatively weak expressive power due to fewer parameters and simpler structure, and is difficult to handle complex data patterns and relationships. However, a small model is easier to interpret and understand, and is also easier to avoid problems such as overfitting and underfitting. In specific tasks and scenarios, a small model can also achieve satisfactory accuracy.
[0082] In some embodiments, the second model can be a large model. Due to the large number of parameters and complex structure, the large model can handle more complex data patterns and relationships, and has higher prediction accuracy and stronger generalization ability. However, this also makes the large model more prone to overfitting, requiring more training data and computing resources for training and optimization.
[0083] For example, the large model may be Llama-2-7B, and the small model may be Llama-2-1B.
[0084] In some embodiments, the first token may be a word-gram token.
[0085] In some embodiments, processing the data based on the first model to obtain the first label may include:
[0086] The small model generates tokens based on the model reasoning Prefill process.
[0087] In some embodiments, before S110, the method may further include:
[0088] Computing resources are allocated to the first model and the second model based on the parameters of the first model and the second model, and the computing resources may be GPU resources.
[0089] S120: Generate a first intermediate mark based on association data between the first mark and the preset mark.
[0090] In some embodiments, the first intermediate marker may include information associated with the first marker.
[0091] In some embodiments, the associated data may include a vocabulary.
[0092] In some embodiments, generating the first intermediate mark based on the association data between the first mark and the preset mark may include:
[0093] Based on the Q, K, and V values of the large model’s attention mechanism, the context attention vector is calculated, and the words associated with the first tag are retained.
[0094] In some embodiments, the words associated with the first tag may include words with the highest probability of being associated with the first tag.
[0095] For example, the probability distribution in the vocabulary may be queried, and the first two words with the highest probability are retained.
[0096] S130: Generate a second intermediate marker based on the first intermediate marker and associated data of the preset marker.
[0097] In some embodiments, the context attention vector can be calculated based on the Q, K, and V values of the large model attention mechanism, and the words associated with the first intermediate token can be retained.
[0098] Exemplarily, the first intermediate tokens may be used as probability distribution in the target query vocabulary, and the first two words with the highest probability of being associated with each first intermediate token are retained.
[0099] In some embodiments, S130 may be repeated until a stop condition is triggered.
[0100] S140: Verify the first intermediate marker and the second intermediate marker based on the second model.
[0101] In some embodiments, the second model may be a large model.
[0102] In some embodiments, before S140, the method may further include:
[0103] The first intermediate marker and the second intermediate marker are constructed in the form of a binary tree.
[0104] In some embodiments, verifying the first intermediate marker and the second intermediate marker may include verifying the first intermediate marker and the second intermediate marker in parallel.
[0105] S150: If the verification fails, at least one of the first mark, the first intermediate mark, and the second intermediate mark is processed based on the second model to obtain a multi-model collaborative reasoning result.
[0106] In some embodiments, processing the first mark, the second intermediate mark, and at least one of the second intermediate mark based on the second model may include:
[0107] In the case of a failed check, the data that failed the check can be determined in the nodes of the generated binary tree, and the nodes before the nodes of the binary tree that failed the check and their check results can be first sent to the second model, so that the second model determines the nodes that passed the check based on the nodes and the check results of the nodes, and then generates a multi-model collaborative processing result based on the nodes that passed the check. The nodes that passed the check can be the first mark, the first intermediate mark, and the second intermediate mark.
[0108] The multi-model collaborative reasoning method provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0109] Figure 2 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown.
[0110] like Figure 2 As shown, the method may include:
[0111] S210, processing data based on a first model to obtain a first label;
[0112] S220, generating a first intermediate mark based on the association data between the first mark and the preset mark;
[0113] S230, generating a second intermediate mark based on the first intermediate mark and associated data of the preset mark;
[0114] S240: Generate a third intermediate marker based on the second intermediate marker and associated data of the preset marker.
[0115] In some embodiments, a method for generating a third intermediate marker based on the second intermediate marker is the same as a method for generating a second intermediate marker based on the first intermediate marker, and is not described in detail herein.
[0116] S250, repeat the above steps until the stop condition is met, and determine the finally generated third intermediate mark as the multi-model collaborative reasoning result.
[0117] In some embodiments, the stop condition may include that the number of iterations reaches the number of iterations or the number of the obtained second intermediate marks and the third intermediate marks reaches a preset number.
[0118] In some embodiments, the stop condition may further include that a probability of association between obtaining the third intermediate marker and the first intermediate marker for obtaining the third intermediate marker exceeds a preset threshold.
[0119] Exemplarily, the preset threshold may be 90%.
[0120] In some embodiments, before determining the third intermediate marker as the multi-model collaborative reasoning result, it is necessary to ensure that the first intermediate marker, the second intermediate marker, and the third intermediate marker all pass the verification of the large model.
[0121] The multi-model collaborative reasoning method provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0122] Figure 3 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown.
[0123] like Figure 3 As shown, the method may include:
[0124] S310, processing data based on a first model to obtain a first label;
[0125] S320, generating a first intermediate mark based on the association data between the first mark and the preset mark;
[0126] S330, generating a second intermediate mark based on the first intermediate mark and the associated data of the preset mark;
[0127] S340: When the first intermediate marker and / or the second intermediate marker meets a preset condition, determine the intermediate marker that meets the preset condition as a multi-model collaborative reasoning result.
[0128] In some embodiments, the preset condition includes that the probability of association between the first intermediate marker and the first marker exceeds a preset threshold and / or the probability of association between the first intermediate marker and the second intermediate marker exceeds a preset threshold.
[0129] Exemplarily, the preset threshold may be 90%.
[0130] The multi-model collaborative reasoning method provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0131] Figure 4 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown.
[0132] like Figure 4 As shown, the method may include:
[0133] S410, determining the delay function based on the amount of data transmission required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data.
[0134] In some embodiments, the delay function may include:
[0135]
[0136] Token x The first middle mark or the second middle mark, V (Token x ) is the verification token x The required data transmission amount, j is the first model, j′ is the second model, R j,j′ represents the transmission speed between the jth and j'th nodes, C(Tokenx ) is the verification token x Required computing power data, H j′ Indicates j' verification Token x Capability data, R(Token x ) is the acceptable delay ratio data for users, T accept Accept delayed data for users.
[0137] S420: Determine a load balancing function based on decision variables for generating the first intermediate marker and / or generating the second intermediate marker and computing power data provided for generating the first intermediate marker and / or generating the second intermediate marker.
[0138] In some embodiments, the load balancing function includes:
[0139]
[0140] Among them, L j is the load balancing data, j is the first model, C j The first model computing power data, Token x is the first middle mark or the second middle mark, Indicates the generation of Token x decision variables.
[0141] S430, determining computing power data allocated to the first model and the second model based on the delay function, the load balancing function and the preset algorithm.
[0142] S440, processing the data based on the first model to obtain a first label;
[0143] S450, generating a first intermediate mark based on the association data between the first mark and the preset mark;
[0144] S460, generating a second intermediate mark based on the first intermediate mark and the associated data of the preset mark;
[0145] S470, verifying the first intermediate marker and the second intermediate marker based on the second model;
[0146] S480: If the verification fails, at least one of the first mark, the first intermediate mark, and the second intermediate mark is processed based on the second model to obtain a multi-model collaborative reasoning result.
[0147] The multi-model collaborative reasoning method provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0148] Figure 5 A flowchart of another multi-model collaborative reasoning method in an embodiment of the present disclosure is shown.
[0149] like Figure 5 As shown, the method may include:
[0150] S510, determining a delay function based on the amount of data transmission required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data;
[0151] S520, determining a load balancing function based on a decision variable for generating the first intermediate marker and / or generating the second intermediate marker and computing power data provided for generating the first intermediate marker and / or generating the second intermediate marker;
[0152] S530, determining the computing power data allocated to the first model and the second model based on the delay function, the load balancing function and the preset algorithm.
[0153] S540, determining an objective function based on the delay function and the load balancing function;
[0154] S550, determining computing power data allocated to the first model and the second model based on the objective function and the greedy algorithm;
[0155] The objective function includes:
[0156]
[0157] Among them, w1 and w2 are weight values, and M is Token x Total number, N is the total amount of computing power data.
[0158] The multi-model collaborative reasoning method provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0159] Based on the same inventive concept, the present disclosure also provides a multi-model collaborative reasoning processing device, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.
[0160] Figure 6 A structural diagram of a multi-model collaborative reasoning device in an embodiment of the present disclosure is shown.
[0161] like Figure 6 As shown, the apparatus 600 may include:
[0162] A first processing module 610, configured to process data based on a first model to obtain a first label;
[0163] A first generating module 620, configured to generate a first intermediate mark based on the association data between the first mark and the preset mark;
[0164] A second generating module 630, configured to generate a second intermediate mark based on the first intermediate mark and associated data of the preset mark;
[0165] A verification module 640, configured to verify the first intermediate mark and the second intermediate mark based on the second model;
[0166] The second processing module 650 is used to process at least one of the first mark, the first intermediate mark and the second intermediate mark based on the second model to obtain a multi-model collaborative reasoning result when the verification fails.
[0167] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0168] In one embodiment of the present disclosure, the device further includes:
[0169] A third generating module, configured to generate a third intermediate mark based on the second intermediate mark and associated data of the preset mark;
[0170] The iteration module is used to repeat the above steps until the stopping condition is met, and determine the finally generated third intermediate mark as the multi-model collaborative reasoning result.
[0171] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0172] In one embodiment of the present disclosure, the device further includes:
[0173] A determination module is used to determine the intermediate marker that meets the preset condition as the multi-model collaborative reasoning result when the first intermediate marker and / or the second intermediate marker meets the preset condition.
[0174] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0175] In one embodiment of the present disclosure, the device further includes:
[0176] A second determination module is used to determine the delay function based on the data transmission volume required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data;
[0177] A third determination module, configured to determine a load balancing function based on a decision variable for generating the first intermediate mark and / or generating the second intermediate mark, and computing power data provided for generating the first intermediate mark and / or generating the second intermediate mark;
[0178] The fourth determination module is used to determine the computing power data allocated to the first model and the second model based on the delay function, the load balancing function and the preset algorithm.
[0179] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0180] In one embodiment of the present disclosure, the delay function includes:
[0181]
[0182] Token x The first middle mark or the second middle mark, V (Token x ) is the verification token x The required data transmission amount, j is the first model, j′ is the second model, R j,j′ represents the transmission speed between the jth and j'th nodes, C(Token x ) is the verification token x Required computing power data, H j′ Indicates j' verification Token x Capability data, R(Token x ) is the acceptable delay ratio data for users, T accept Accept delayed data for users.
[0183] In one embodiment of the present disclosure, the load balancing function includes:
[0184]
[0185] Among them, L j is the load balancing data, j is the first model, C j The first model computing power data, Token x is the first middle mark or the second middle mark, Indicates the generation of Token xdecision variables.
[0186] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0187] In one embodiment of the present disclosure, the fourth determining module includes:
[0188] A first determining unit, configured to determine an objective function based on a delay function and a load balancing function;
[0189] A second determining unit, configured to determine computing power data allocated to the first model and the second model based on an objective function and a greedy algorithm;
[0190] The objective function includes:
[0191]
[0192] Among them, w1 and w2 are weight values, and M is Token x Total number, N is the total amount of computing power data.
[0193] The multi-model collaborative reasoning device provided by the embodiments of the present disclosure processes data based on a first model to obtain a first label, generates a first intermediate label based on associated data of the first label and a preset label, generates a second intermediate label based on the associated data of the first intermediate label and the preset label, verifies the first intermediate label and the second intermediate label based on a second model, and if the verification fails, processes at least one of the first label, the first intermediate label and the second intermediate label based on the second model to obtain a multi-model collaborative processing result, thereby improving the efficiency of model reasoning through multi-model collaborative processing.
[0194] The multi-model collaborative reasoning device provided in the embodiment of the present disclosure can be used to execute the positioning methods provided in the above-mentioned method embodiments. The implementation principles and technical effects are similar and will not be described here for the sake of simplicity.
[0195] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0196] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0197] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).
[0198] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps of the above method embodiment:
[0199] Processing the data based on the first model to obtain a first label;
[0200] generating a first intermediate mark based on the association data between the first mark and the preset mark;
[0201] generating a second intermediate mark based on the first intermediate mark and the associated data of the preset mark;
[0202] Verifying the first intermediate marker and the second intermediate marker based on the second model;
[0203] If the verification fails, at least one of the first mark, the first intermediate mark, and the second intermediate mark is processed based on the second model to obtain a multi-model collaborative reasoning result.
[0204] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .
[0205] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0206] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0207] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0208] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.
[0209] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.
[0210] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0211] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0212] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0213] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).
[0214] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0215] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0216] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the example implementation modes described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation mode of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation mode of the present disclosure.
[0217] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.
Claims
1. A multi-model collaborative reasoning method, characterized in that: include: Processing the data based on the first model to obtain a first label; generating a first intermediate mark based on the association data between the first mark and the preset mark; generating a second intermediate mark based on the first intermediate mark and the associated data of the preset mark; verifying the first intermediate marker and the second intermediate marker based on a second model; If the verification fails, at least one of the first mark, the first intermediate mark, and the second intermediate mark is processed based on the second model to obtain a multi-model collaborative reasoning result.
2. The method according to claim 1, characterized in that The method further comprises: generating a third intermediate mark based on the second intermediate mark and the associated data of the preset mark; Repeat the above steps until the stopping condition is met, and determine the third intermediate mark finally generated as the multi-model collaborative reasoning result.
3. The method according to claim 1, characterized in that The method further comprises: When the first intermediate mark and / or the second intermediate mark meets a preset condition, the intermediate mark that meets the preset condition is determined as a multi-model collaborative reasoning result.
4. The method according to claim 1, characterized in that: The method further comprises: Determine a delay function based on the amount of data transmission required for verifying the first intermediate mark and / or the second intermediate mark, the transmission speed between the first model and the second model, the computing power data required for the second model to verify the first intermediate mark and / or the second intermediate mark, the capability data of the second model to verify the first intermediate mark and / or the second intermediate mark, the user acceptance delay data, and the user acceptable delay ratio data; Determine a load balancing function based on decision variables for generating the first intermediate marker and / or generating the second intermediate marker and computing power data provided for generating the first intermediate marker and / or generating the second intermediate marker; The computing power data allocated to the first model and the second model are determined based on the delay function, the load balancing function and the preset algorithm.
5. The method according to claim 4, characterized in that The delay function includes: Token x The first middle mark or the second middle mark, V (Token x ) is the verification token x The required data transmission amount, j is the first model, j′ is the second model, R j，j′ represents the transmission speed between the jth and j′th nodes, C(Token x ) is the verification token x Required computing power data, H j′ Indicates j′ verification Token x Capability data, R(Token x ) is the acceptable delay ratio data for users, T accept Accept delayed data for users.
6. The method according to claim 4, characterized in that The load balancing function includes: Among them, L j is the load balancing data, j is the first model, C j The first model computing power data, Token x is the first middle mark or the second middle mark, Indicates the generation of Token x decision variables.
7. The method according to claim 5 or 6, characterized in that: The determining the computing power data allocated to the first model and the second model based on the delay function, the load balancing function and the preset algorithm includes: Determine an objective function based on the delay function and the load balancing function; Determine computing power data allocated to the first model and the second model based on the objective function and the greedy algorithm; The objective function includes: Among them, w1 and w2 are weight values, and M is Token x Total number, N is the total amount of computing power data.
8. A multi-model collaborative reasoning device, characterized in that: include: A first processing module, configured to process data based on a first model to obtain a first label; A first generating module, configured to generate a first intermediate mark based on association data between the first mark and the preset mark; A second generating module, configured to generate a second intermediate mark based on the first intermediate mark and associated data of the preset mark; a verification module, configured to verify the first intermediate mark and the second intermediate mark based on a second model; The second processing module is used to process at least one of the first mark, the first intermediate mark and the second intermediate mark based on the second model to obtain a multi-model collaborative reasoning result when the verification fails.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the multi-model collaborative reasoning method described in any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-model collaborative reasoning method described in any one of claims 1 to 7 is implemented.