Information processing method and device, computer readable storage medium and computer program product

Abstract

The invention relates to an information processing method and device, a computer readable storage medium and a computer program product, and relates to the technical field of computers. The information processing method comprises the steps that first information is input into a first machine learning model to output candidate processing results, the first machine learning model is composed of a plurality of designated layers in a second machine learning model, and the difference between input and output of each designated layer in the designated layers is larger than a threshold value; and outputting a final processing result according to the candidate processing result by using the second machine learning model. According to the technical scheme, resources required by processing can be reduced, so that the information processing performance is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an information processing method, an information processing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Machine learning models based on artificial intelligence technology have been widely used in various practical applications of information processing. Machine learning models generally require a lot of inference costs. In the inference process of machine learning models, the main performance bottlenecks include the autoregressive decoding process, etc.

[0003] In related technologies, in order to alleviate the inference pressure of machine learning models, acceleration methods such as model pruning, model quantization, and knowledge distillation can be used. Summary of the invention

[0004] According to some embodiments of the present disclosure, there is provided an information processing method, comprising: inputting first information into a first machine learning model to output a candidate processing result, the first machine learning model being composed of a plurality of specified layers in a second machine learning model, the difference between the input and output of each of the plurality of specified layers being greater than a threshold; and using the second machine learning model to output a final processing result based on the candidate processing results.

[0005] In some embodiments, the angular distance between the input vector and the output vector of each specified layer is greater than a threshold.

[0006] In some embodiments, the information processing method also includes: inputting the second information into a second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model, which are used to determine multiple specified layers from the multiple layers.

[0007] In some embodiments, the second information is obtained from multiple types of subtask data sets.

[0008] In some embodiments, inputting the first information into the first machine learning model to output candidate processing results includes: inputting the first information into the first machine learning model to determine candidate outputs of multiple specified layers; using the second machine learning model to output the final processing result based on the candidate processing results includes: using the second machine learning model to determine the final processing result based on the candidate outputs of multiple specified layers.

[0009] In some embodiments, using a second machine learning model, determining the final processing result based on candidate outputs of multiple designated layers includes: inputting a designated candidate output among the candidate outputs into the non-designated layer that is most closely ranked in the second machine learning model to determine the final processing result, wherein the designated candidate output is the candidate output of the previous designated layer of the most closely ranked non-designated layer.

[0010] In some embodiments, inputting the first information into a first machine learning model to output candidate processing results includes: inputting the first information into a second machine learning model to output cached values ​​of the key vector and value vector of each specified layer; and using the first machine learning model to output candidate processing results based on the cached values.

[0011] According to other embodiments of the present disclosure, there is provided an information processing device, including: a first processing unit, used to input first information into a first machine learning model to output a candidate processing result, the first machine learning model being composed of a plurality of specified layers in a second machine learning model, the difference between the input and output of each of the plurality of specified layers being greater than a threshold; and a second processing unit, used to output a final processing result based on the candidate processing results using the second machine learning model.

[0012] In some embodiments, the angular distance between the input vector and the output vector of each specified layer is greater than a threshold.

[0013] In some embodiments, the second processing unit inputs the second information into the second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model for determining multiple specified layers from the multiple layers.

[0014] In some embodiments, the second information is obtained from multiple types of subtask data sets.

[0015] In some embodiments, the first processing unit inputs the first information into a first machine learning model to determine candidate outputs of multiple specified layers; the second processing unit uses the second machine learning model to determine the final processing result based on the candidate outputs of multiple specified layers.

[0016] In some embodiments, the second processing unit inputs a specified candidate output among the candidate outputs into the non-specified layer that is ranked closest to the front in the second machine learning model to determine the final processing result, and the specified candidate output is the candidate output of the previous specified layer of the non-specified layer that is ranked closest to the front.

[0017] In some embodiments, the second processing unit inputs the first information into a second machine learning model to output cached values ​​of the key vector and value vector of each specified layer; and the first processing unit uses the first machine learning model to output candidate processing results based on the cached values.

[0018] According to some further embodiments of the present disclosure, there is provided an information processing device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the information processing method in any one of the above embodiments based on instructions stored in the memory.

[0019] According to some further embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the information processing method in any of the above embodiments is implemented.

[0020] According to some further embodiments of the present disclosure, a computer program product is provided, comprising instructions, which, when executed by a processor, enable the processor to execute the information processing method according to any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0023] Figure 1 Flowcharts showing some embodiments of the information processing method of the present disclosure;

[0024] Figure 2 Schematic diagrams showing some embodiments of the information processing method of the present disclosure;

[0025] Figure 3 A block diagram showing some embodiments of the information processing device of the present disclosure;

[0026] Figure 4 A block diagram showing some other embodiments of the information processing device of the present disclosure;

[0027] Figure 5 A block diagram showing still some embodiments of the information processing device of the present disclosure. DETAILED DESCRIPTION

[0028] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0029] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0030] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0031] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0032] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0033] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0034] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: the implementation process of the accelerated processing is complicated, resulting in a decrease in information processing performance.

[0035] As mentioned earlier, although acceleration methods such as model pruning, model quantization, and knowledge distillation improve computational efficiency to a certain extent, these methods usually require changing the model architecture, changing the training procedure, retraining or fine-tuning the model, and cannot keep the output before and after acceleration the same.

[0036] To address the above technical issues, the speculative decoding method can use a relatively small model (called a draft model) to quickly generate candidate outputs; then, using the relatively large original model to be deployed (called the target model, verification model), the accuracy of the candidate outputs is checked through forward transmission to ensure that the outputs of the draft model and the target model are consistent.

[0037] However, speculative inference methods require a suitable draft model to ensure that the draft model and the target model have similar outputs. If the target model is a fine-tuned model, the above acceleration process will become more complicated.

[0038] Therefore, how to find or train a draft model that can effectively mimic the output of the target model is an important task. In addition, introducing additional draft models, especially on devices with limited memory capacity, will also increase the memory overhead of devices such as GPUs (Graphics Processing Units) and increase deployment difficulties.

[0039] In view of this, the present disclosure proposes an information processing technical solution, which can simplify the implementation process of accelerated processing, thereby improving information processing performance. For example, the technical solution of the present disclosure can be implemented through the following embodiments.

[0040] Figure 1Flowcharts showing some embodiments of the information processing method of the present disclosure.

[0041] like Figure 1 As shown, in step 110, the first information is input into the first machine learning model to output a candidate processing result. The first machine learning model is composed of multiple specified layers in the second machine learning model, and the difference between the input and output of each specified layer in the multiple specified layers is greater than a threshold. For example, the first information can be input into the first machine learning model multiple times to output multiple candidate processing results.

[0042] For example, the first information may include image information, natural language information, audio information, text information, etc. to be processed; the first information may be information used to realize interaction between people and computers, such as information to be processed input by a user to a computer through an interface such as an interactive platform, and the processing result output by the computer is reply information corresponding to the first information.

[0043] In some embodiments, the number of layers included in the first machine learning model is less than the number of layers of the second machine learning model. The first machine learning model can be called a draft model, and the second machine learning model can be called a target model. The first machine learning model can be generated offline in advance, and the first machine learning model is composed of multiple specified layers among the multiple layers of the second machine learning model.

[0044] In step 120, the second machine learning model is used to output a final processing result based on the candidate processing results. For example, the second machine learning model can be used to verify multiple candidate processing results to determine the final processing result based on the multiple candidate processing results.

[0045] In the above embodiment, a part of the layers of the second machine learning model is used to form the first machine learning model, which is used to assist the second machine learning model in reasoning. In this way, by sharing part of the layers of the same machine learning model, the complexity of the implementation process of computer acceleration processing is reduced, thereby reducing the computing resources and storage resources required for computer processing, so as to improve the information processing performance of the computer.

[0046] The following is an illustrative example of a method for generating a first machine learning model (i.e., a draft model) through some embodiments.

[0047] In some embodiments, the second information is input into a second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model for determining multiple specified layers from the multiple layers. For example, the second information is obtained from multiple types of subtask data sets.

[0048] For example, the second information may include image information, natural language information, audio information, text information, etc. to be processed; the second information may be information used to realize interaction between people and computers, such as information to be processed input by a user to a computer through an interface such as an interactive platform, and the processing result output by the computer is reply information corresponding to the first information.

[0049] For example, extract a part of the data from each type of sub-task dataset to build a more complete dataset, called a calibration dataset; use each piece of data in the calibration dataset as the second information and input it into the target model to obtain the output activation value of each layer in the target model.

[0050] In this way, by constructing a complete data set, important layers can be accurately screened out from the second machine learning model to construct the first machine learning model, thereby improving the accuracy of information processing.

[0051] In some embodiments, the angular distance between the input vector and the output vector of each designated layer is greater than a threshold. For example, the angular distance is the angle between the input vector and the output vector. The designated layer can also be determined based on other physical quantities (such as vector distance, etc.) that can characterize the difference between the input vector and the output vector.

[0052] For example, the output of layer l of the second machine learning model is a (l) , the input of the lth layer is the output a of the l-1th layer (l -1) , a (l) with a (l-1) The angular distance is:

[0053]

[0054] “·” represents the vector dot product, and “‖‖” represents the L2 norm of the vector.

[0055] For example, the angular distance θ corresponding to each layer of the second machine learning model can be calculated (l) , to obtain the angular distance sequence {θ corresponding to the 1st to Lth layers in the second machine learning model arranged in the processing order (1) ,θ (2) ,…,θ (l) ,…,θ (L) ,}. In response to θ (l) ≤τ, the lth layer is marked as an “unimportant layer” and is not included in the first machine learning model. τ is a preset threshold. In response to θ (l) >τ, the lth layer is marked as an “important layer” and is included in the first machine learning model.

[0056] In this way, through the above offline preprocessing, the draft model is determined by a part of the layers in the target model. On the one hand, the draft model can be generated without training additional machine learning models, and on the other hand, only the target model needs to be deployed during the deployment phase, thereby reducing the deployment cost of the machine learning model and improving the computer's information processing effect and the efficiency of human-computer interaction.

[0057] For example, you can Figure 2 In the embodiment, a first machine learning model is generated.

[0058] Figure 2 Schematic diagrams showing some embodiments of the information processing method of the present disclosure.

[0059] like Figure 2 As shown, the second machine learning model includes layers 210 to 270, and the solid arrows in the figure represent the reasoning process of the second machine learning model, namely layer 210-layer 220-layer 230-layer 240-layer 250-layer 260-layer 270; by calculating the angular distance corresponding to each layer of the second machine learning model, it can be confirmed that the layers with angular distances greater than the threshold include layer 210, layer 220, layer 240 and layer 270; it can be determined that the first machine learning model is composed of layer 210, layer 220, layer 240 and layer 270, and the dotted arrows in the figure represent the reasoning process of the first machine learning model, namely layer 210-layer 220-layer 240-layer 270.

[0060] For example, the output of layer l of the second machine learning model is a (l) , the input of the lth layer is the output a of the l-1th layer (l -1) , a (l) with a (l-1) The angular distance is θ (l) ; The angular distance θ corresponding to each layer of the second machine learning model can be calculated (l) , to obtain the angular distance sequence {θ corresponding to layers 210 to 270 in the second machine learning model arranged in the processing order (210) ,θ (220) ,…,θ (l) ,…,θ (270) ,}. In response to θ (l) ≤τ, the lth layer is marked as an “unimportant layer” (such as layer 230, layer 250, layer 260, etc.), and these layers are not included in the first machine learning model, and τ is a preset threshold; in response to θ (l)>τ, the lth layer is marked as an “important layer” (such as layer 210, layer 220, layer 240, layer 270, etc.), and these layers are included in the first machine learning model.

[0061] In this way, through the above offline preprocessing, the draft model is determined by a part of the layers in the target model. On the one hand, the draft model can be generated without training an additional machine learning model, and on the other hand, only the target model needs to be deployed during the deployment phase, thereby reducing the deployment cost of the machine learning model and improving the information processing effect. For example, by using the angular distance difference of each layer output in the offline preprocessing, the important layers are selected as the draft model, and there is no need to train an additional model as the draft model, which reduces the implementation complexity of accelerated reasoning; there is no need to deploy two independent models, which alleviates the computing resources and storage resources required for model deployment.

[0062] For example, the input information is obtained from multiple types of subtask datasets. A portion of data can be extracted from each type of subtask dataset to construct a more complete dataset, called a calibrate dataset; each piece of data in the calibrate dataset is used as input information and input into the second machine learning model to obtain the output activation values ​​of layers 210 to 270 in the second machine learning model, which are used to determine that layers 210, 220, 240, and 270 are important layers to generate the first machine learning model.

[0063] In this way, by constructing a complete data set, important layers can be accurately screened out from the second machine learning model to construct the first machine learning model, thereby improving the accuracy of information processing.

[0064] The following uses some embodiments to exemplarily illustrate the reasoning process of the first machine learning model (i.e., the draft model).

[0065] In some embodiments, the first information is input into a second machine learning model to output cache values ​​(i.e., KV-cache) of the key vector (Key) and value vector (Value) of each specified layer; and the candidate processing results are output based on the cache values ​​using the first machine learning model.

[0066] For example, the second machine learning model can first be used to infer the information to be processed to determine the KV-cache corresponding to each layer in the second machine learning model; then, in the reasoning stage when the second machine learning model generates candidate processing results (also called draft processing results), each layer in the second machine learning model can use the predetermined KV-cache corresponding to the layer to output the candidate processing results.

[0067] In this way, by utilizing the fact that the layers included in the draft model are a subset of the layers included in the target model, each layer of the draft model can reuse the KV-cache of the corresponding layer in the target model, thereby reducing the amount of calculation and improving the efficiency of information processing.

[0068] For example, you can Figure 2 In the embodiment, the first machine learning model reuses the cached values ​​of the key vector and the value vector.

[0069] like Figure 2 As shown, the first machine learning model includes layers 210, 220, 240, and 270 in sequence, and the second machine learning model includes layers 210 to 270 in sequence, that is, the layers included in the first machine learning model are a subset of the layers included in the second machine learning model. The second machine learning model can be used to reason about the information to be processed to determine the KV-cache corresponding to each layer in layers 210 to 270; then, in the reasoning stage of the second machine learning model generating candidate processing results, the KV-caches corresponding to layers 210, 220, 240, and 270 are reused to output candidate processing results.

[0070] The following uses some embodiments to illustrate the reasoning process of the second machine learning model (i.e., target model).

[0071] In some embodiments, the first information is input into a first machine learning model to determine candidate outputs of multiple designated layers (also referred to as draft outputs); and a second machine learning model is used to determine a final processing result based on the candidate outputs of the multiple designated layers. For example, a designated candidate output among the candidate outputs is input into the most forward non-designated layer in the second machine learning model to determine the final processing result, and the designated candidate output is the candidate output of the previous designated layer of the most forward non-designated layer.

[0072] For example, multiple layers in the second machine learning model are arranged in the processing order of the reasoning process, with the layers closer to the input end of the second machine learning model being arranged in the front, and the layers closer to the output end being arranged in the back. The second machine learning model can reuse the draft output of the previous layer of the non-specified layer that is arranged in the front.

[0073] In this way, the deviation of the intermediate processing results of the draft model reused by the target model can be guaranteed to be small, which not only improves the processing efficiency of the target model but also ensures the processing accuracy of the target model.

[0074] For example, you can Figure 2In the embodiment, the second machine learning model reuses the outputs of some specified layers.

[0075] like Figure 2 As shown, the first machine learning model includes layers 210, 220, 240, and 270 in sequence, and the second machine learning model includes layers 210 to 270 in sequence, that is, the layers included in the first machine learning model are a subset of the layers included in the second machine learning model. Layers 210 and 220 are two shared layers (i.e., designated layers) of the first machine learning model and the second machine learning model, and layer 230 is the most forward non-shared layer (i.e., non-designated layer) in the second machine learning model. Layer 220 is the previous designated layer of non-designated layer 230, and the second machine learning model can reuse the draft output of layer 220 as the input of layer 230.

[0076] For example, during the inference process of the second machine learning model, the draft output of layer 220 generated during the inference process of the first machine learning model can be input into layer 230 of the second machine learning model to achieve output sharing of the specified layer.

[0077] For example, in the reasoning process of the second machine learning model, the reasoning process of layer 210 to layer 220 of the first machine learning model is directly reused, that is, the draft output of layer 220 is used as the input of layer 230 of the second machine learning model; then the reasoning process of the second machine learning model is completed through layer 230-layer 240-layer 250-layer 260-layer 270 to determine the final processing result.

[0078] In the above embodiment, there is no need to train an additional model as the draft model, but directly use some layers of the target model as the draft model. In this way, the computing and storage resources required to deploy two models are saved. Moreover, the draft model can share inter-layer output and input with the target model. In the process of verifying the candidate processing results of the target model, some layers that overlap with the draft model and the target model can be used, thereby improving the information processing efficiency of the computer.

[0079] Figure 3 A block diagram showing some embodiments of the information processing device of the present disclosure.

[0080] like Figure 3As shown, the information processing device 3 includes: a first processing unit 31, used to input the first information into the first machine learning model to output a candidate processing result, the first machine learning model is composed of multiple specified layers in the second machine learning model, and the difference between the input and output of each specified layer in the multiple specified layers is greater than a threshold; a second processing unit 32, used to use the second machine learning model to output a final processing result based on the candidate processing results.

[0081] In some embodiments, the angular distance between the input vector and the output vector of each specified layer is greater than a threshold.

[0082] In some embodiments, the second processing unit 32 inputs the second information into the second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model for determining multiple specified layers from the multiple layers.

[0083] In some embodiments, the second information is obtained from multiple types of subtask data sets.

[0084] In some embodiments, the first processing unit 31 inputs the first information into the first machine learning model to determine candidate outputs of multiple specified layers; the second processing unit 32 uses the second machine learning model to determine the final processing result based on the candidate outputs of multiple specified layers.

[0085] In some embodiments, the second processing unit 32 inputs a designated candidate output among the candidate outputs into the frontmost non-designated layer in the second machine learning model to determine the final processing result, and the designated candidate output is the candidate output of the previous designated layer of the frontmost non-designated layer.

[0086] In some embodiments, the second processing unit 32 inputs the first information into the second machine learning model to output cache values ​​of the key vector and value vector of each specified layer; the first processing unit 31 uses the first machine learning model to output candidate processing results based on the cache values.

[0087] In the above embodiment, the importance of each layer is ranked by calculating and analyzing the difference between the input and output of each layer of the second machine learning model; according to a preset threshold, the layer whose angular distance change is less than the threshold is retained to generate the first machine learning model; the original second machine learning model to be deployed is used as a verification model, and the first machine learning model is used as a draft model to perform self-speculative sampling processing. In this way, the reasoning speed can be effectively improved and computing resources can be saved without training additional draft models.

[0088] Figure 4 A block diagram showing some other embodiments of the information processing device of the present disclosure.

[0089] like Figure 4As shown, the information processing device 4 of this embodiment includes: a memory 41 and a processor 42 coupled to the memory 41 , and the processor 42 is configured to execute the information processing method in any one of the embodiments of the present disclosure based on the instructions stored in the memory 41 .

[0090] The memory 41 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0091] In some embodiments, the processor 42 is configured to input the first information into a first machine learning model to output a candidate processing result, where the first machine learning model is composed of multiple specified layers in a second machine learning model, and the difference between the input and output of each of the multiple specified layers is greater than a threshold; and use the second machine learning model to output a final processing result based on the candidate processing results.

[0092] In some embodiments, the angular distance between the input vector and the output vector of each specified layer is greater than a threshold.

[0093] In some embodiments, the processor 42 is configured to input the second information into the second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model for determining multiple specified layers from the multiple layers.

[0094] In some embodiments, the second information is obtained from multiple types of subtask data sets.

[0095] In some embodiments, the processor 42 is configured to input the first information into a first machine learning model to determine candidate outputs of multiple specified layers; and use a second machine learning model to determine the final processing result based on the candidate outputs of multiple specified layers.

[0096] In some embodiments, the processor 42 is configured to input a specified candidate output among the candidate outputs into the most forward non-specified layer in the second machine learning model to determine the final processing result, wherein the specified candidate output is the candidate output of the previous designated layer of the most forward non-specified layer.

[0097] In some embodiments, the processor 42 is configured to input the first information into the second machine learning model to output cached values ​​of the key vector and value vector of each specified layer; and use the first machine learning model to output candidate processing results based on the cached values.

[0098] In the above embodiment, the importance of each layer is ranked by calculating and analyzing the difference between the input and output of each layer of the second machine learning model; according to a preset threshold, the layer whose angular distance change is less than the threshold is retained to generate the first machine learning model; the original second machine learning model to be deployed is used as a verification model, and the first machine learning model is used as a draft model to perform self-speculative sampling processing. In this way, the reasoning speed can be effectively improved and computing resources can be saved without training additional draft models.

[0099] Figure 5 A block diagram showing still some embodiments of the information processing device of the present disclosure.

[0100] like Figure 5 As shown, the information processing device 5 of this embodiment includes: a memory 510 and a processor 520 coupled to the memory 510 , and the processor 520 is configured to execute the information processing method in any of the aforementioned embodiments based on the instructions stored in the memory 510 .

[0101] The memory 510 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0102] The information processing device 5 may also include an input / output interface 530, a network interface 540, a storage interface 550, etc. These interfaces 530, 540, 550, the memory 510, and the processor 520 may be connected, for example, via a bus 560. The input / output interface 530 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 540 provides a connection interface for various networked devices. The storage interface 550 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0103] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] In some embodiments, computer instructions are stored on a computer-readable storage medium, and when the instructions are executed by a processor, an information processing method is implemented, including: inputting first information into a first machine learning model to output a candidate processing result, the first machine learning model is composed of multiple specified layers in a second machine learning model, and the difference between the input and output of each specified layer in the multiple specified layers is greater than a threshold; using the second machine learning model, outputting a final processing result based on the candidate processing results.

[0105] In some embodiments, the angular distance between the input vector and the output vector of each specified layer is greater than a threshold.

[0106] In some embodiments, the second information is input into a second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model for determining multiple specified layers from the multiple layers.

[0107] In some embodiments, the second information is obtained from multiple types of subtask data sets.

[0108] In some embodiments, the first information is input into a first machine learning model to determine candidate outputs of multiple specified layers; and a second machine learning model is used to determine a final processing result based on the candidate outputs of multiple specified layers.

[0109] In some embodiments, a designated candidate output among the candidate outputs is input into the most forward non-designated layer in the second machine learning model to determine the final processing result, and the designated candidate output is the candidate output of the previous designated layer of the most forward non-designated layer.

[0110] In some embodiments, the first information is input into a second machine learning model to output cached values ​​of a key vector and a value vector for each specified layer; and the candidate processing results are output based on the cached values ​​using the first machine learning model.

[0111] In the above embodiment, the importance of each layer is ranked by calculating and analyzing the difference between the input and output of each layer of the second machine learning model; according to a preset threshold, the layer whose angular distance change is less than the threshold is retained to generate the first machine learning model; the original second machine learning model to be deployed is used as a verification model, and the first machine learning model is used as a draft model to perform self-speculative sampling processing. In this way, the reasoning speed can be effectively improved and computing resources can be saved without training additional draft models.

[0112] So far, the information processing method, information processing device, computer-readable storage medium and computer program product according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0113] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0114] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An information processing method, comprising: Inputting the first information into a first machine learning model to output a candidate processing result, wherein the first machine learning model is composed of a plurality of designated layers in the second machine learning model, and a difference between an input and an output of each designated layer in the plurality of designated layers is greater than a threshold value; Utilize the second machine learning model to output a final processing result based on the candidate processing results.

2. The information processing method according to claim 1, wherein: The angular distance between the input vector and the output vector of each designated layer is greater than the threshold.

3. The information processing method according to claim 1, further comprising: The second information is input into the second machine learning model for processing to obtain inputs and outputs of multiple layers in the second machine learning model, which is used to determine the multiple specified layers from the multiple layers.

4. The information processing method according to claim 3, wherein: The second information is obtained from multiple types of subtask data sets.

5. The information processing method according to any one of claims 1 to 4, wherein: The step of inputting the first information into the first machine learning model to output a candidate processing result includes: Inputting the first information into the first machine learning model to determine candidate outputs for the plurality of designated layers; The using the second machine learning model to output a final processing result according to the candidate processing result comprises: The final processing result is determined based on the candidate outputs of the multiple specified layers using the second machine learning model.

6. The information processing method according to claim 5, wherein: The using the second machine learning model to determine the final processing result according to the candidate outputs of the multiple designated layers comprises: Input a designated candidate output among the candidate outputs into the frontmost non-designated layer in the second machine learning model to determine the final processing result, wherein the designated candidate output is the candidate output of the previous designated layer of the frontmost non-designated layer.

7. The information processing method according to any one of claims 1 to 4, wherein: The step of inputting the first information into the first machine learning model to output a candidate processing result includes: Inputting the first information into the second machine learning model to output cached values ​​of the key vector and the value vector of each specified layer; Utilize the first machine learning model to output the candidate processing result according to the cache value.

8. An information processing device, comprising: a first processing unit, configured to input the first information into a first machine learning model to output a candidate processing result, wherein the first machine learning model is composed of a plurality of designated layers in the second machine learning model, and a difference between an input and an output of each designated layer in the plurality of designated layers is greater than a threshold; The second processing unit is used to use the second machine learning model to output a final processing result based on the candidate processing results.

9. An information processing device, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the information processing method according to any one of claims 1 to 7 based on instructions stored in the memory.

10. A computer-readable storage medium having computer instructions stored thereon, wherein when the instructions are executed by a processor, the information processing method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising instructions, which, when executed by a processor, cause the processor to perform the information processing method according to any one of claims 1 to 7.

Images