Task processing method and computer device
By building the attention layer to be deleted for selecting tasks and deleting the target big model, the problem of high computing resources after the big model is compressed is solved, and more efficient resource utilization is achieved.
Patent Information
- Application Number
- CN202510571601.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-08-19
AI Technical Summary
In the prior art, large models still require high computing resources after compression, and the degree of compression of quantization methods is low, resulting in high computing resources consumption of processors.
By constructing the selection task, the attention result of the attention layer in the target big model is determined using the selection prompt information, and the corresponding attention layer to be deleted is deleted to achieve compression of the big model.
It improves the compression level of the large model, reduces the computing resource consumption of the processor, and improves the computing efficiency of the processor.
Smart Images
Figure CN120509451A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a task processing method and a computer device. Background Art
[0002] Large models generally need to be compressed before they can be deployed on the end-side processor due to their large number of parameters and high computing resource requirements.
[0003] Currently, quantization is the mainstream method for compressing large models. This method compresses the model parameters from higher precision to lower precision. However, quantization provides a low degree of compression for large models, resulting in high processor resource consumption when running the compressed large model. Summary of the Invention
[0004] An embodiment of the present application provides a task processing method, including: obtaining a target large model to be compressed, the target large model being capable of running on at least one processor, the target large model including multiple attention layers; inputting selection prompt information constructed based on a selection task into the target large model for processing to determine an attention result of each of the attention layers relative to the selection prompt information; determining an attention layer to be deleted in each of the attention layers based on each of the attention results; and in response to a call of a target task, running the target large model on at least one processor to delete the attention layer to be deleted.
[0005] In some embodiments, the selection prompt information includes a title and multiple options, wherein at least one option is the same as a key field in the title; each of the attention results includes the option attention of each of the attention layers relative to each of the options, and determining the attention layer to be deleted in each of the attention layers based on each of the attention results includes: for each of the attention layers, determining the option with the highest option attention among the multiple options as the target option; and determining the attention layer to be deleted based on the target option corresponding to each of the attention layers.
[0006] In some embodiments, each of the attention layers includes multiple attention heads. Before determining the option with the highest option attention among the multiple options as the target option for each of the attention layers, it also includes: obtaining multiple fields of options corresponding to the selection prompt information; determining the self-attention score of each attention head in each of the attention layers relative to each of the fields; and determining the option attention of each of the attention layers relative to each of the options based on the multiple self-attention scores of each of the attention layers relative to each of the fields.
[0007] In some embodiments, determining the option attention of each attention layer relative to each option based on the multiple self-attention scores of each attention layer relative to each field includes: for each option, determining the option field corresponding to the option from each field; determining the option attention score of each attention layer relative to each option field based on the multiple self-attention scores of each attention layer relative to each field; and determining the option attention of each attention layer relative to each option based on the option attention score of each attention layer relative to each option field.
[0008] In some embodiments, there are multiple selection prompt information, and different selection prompt information corresponds to different questions, multiple options are different, or the corresponding questions are the same, but the option distributions are different. The process of inputting the selection prompt information constructed based on the selection task into the target large model for processing is performed multiple times; the attention layer to be deleted is determined according to the target options corresponding to each attention layer, including: determining the target options corresponding to each attention layer in each selection task; determining the distribution discreteness of the multiple target options corresponding to each attention layer; determining at least one attention layer to be deleted from each attention layer according to the distribution discreteness corresponding to each attention layer, and the distribution discreteness corresponding to the attention layer to be deleted is greater than the distribution discreteness corresponding to the remaining attention layers in the target large model.
[0009] In some embodiments, the construction process of each of the selection prompt information includes: obtaining a selection prompt template including positions to be filled in, the positions to be filled in include the filling positions of the key fields in the title and the filling positions corresponding to the key fields in the options; obtaining multiple fields to be filled in from a database related to the target large model; filling each of the fields to be filled in into the filling positions respectively to generate each of the selection prompt information.
[0010] In some embodiments, the options corresponding to the correct answers are randomly or evenly distributed among all the options corresponding to the selection prompt information.
[0011] In some embodiments, after determining at least one of the attention layers to be deleted from each of the attention layers according to the distribution discreteness corresponding to each of the attention layers, it also includes: deleting the attention layer to be deleted from the target large model; determining the attention layer to be adjusted from the remaining attention layers in the target large model; keeping the parameters of the target large model other than the parameters of the attention layer to be adjusted unchanged, and adjusting the parameters of the attention layer to be adjusted based on the target fine-tuning task.
[0012] In some embodiments, the attention layer to be adjusted satisfies at least one of the following conditions: the number of the attention layers to be adjusted does not exceed the number of the attention layers to be deleted; the attention layer to be adjusted belongs to the adjacent attention layer of the attention layer to be deleted; the attention layer to be adjusted belongs to the remaining candidate attention layers among the candidate attention layers, and the attention layer to be deleted is determined from each of the candidate attention layers, and the distribution discreteness corresponding to each of the candidate attention layers is greater than the distribution discreteness corresponding to other attention layers in the target large model except each of the candidate attention layers.
[0013] An embodiment of the present application also proposes a computer device, comprising a memory, at least one processor and a computer program stored on the memory, wherein at least one of the processors executes the computer program to achieve: obtaining a target large model to be compressed, wherein the target large model can run on at least one of the processors, and wherein the target large model includes multiple attention layers; inputting selection prompt information constructed based on a selection task into the target large model for processing to determine the attention results of each of the attention layers relative to the selection prompt information; determining an attention layer to be deleted in each of the attention layers based on each of the attention results; and in response to the call of the target task, running the target large model to delete the attention layer to be deleted. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0015] Figure 1 The process of the task processing method of the embodiment of this application Figure 1 ;
[0016] Figure 2 The process of determining the attention layer to be deleted in each attention layer in the embodiment of the present application Figure 1 ;
[0017] Figure 3 The process of determining the option attention of each attention layer relative to each option in the embodiment of the present application Figure 1 ;
[0018] Figure 4 The process of determining the option attention of each attention layer relative to each option in the embodiment of the present application Figure 2 ;
[0019] Figure 5The process of determining the attention layer to be deleted in each attention layer in the embodiment of the present application Figure 2 ;
[0020] Figure 6 A schematic diagram of selection prompt information 1 according to an embodiment of the present application;
[0021] Figure 7 This is a schematic diagram of selection prompt information 2 according to an embodiment of the present application;
[0022] Figure 8 A schematic diagram of selection prompt information 3 according to an embodiment of the present application;
[0023] Figure 9 A flowchart of constructing each selection prompt information in an embodiment of the present application;
[0024] Figure 10 A schematic diagram of a selection prompt template according to an embodiment of the present application;
[0025] Figure 11 The process of the task processing method of the embodiment of this application Figure 2 ;
[0026] Figure 12 This is a structural block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION
[0027] Various aspects and features of the present application are described herein with reference to the accompanying drawings.
[0028] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.
[0029] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0030] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.
[0031] It should also be understood that although the present application has been described with reference to certain specific examples, those skilled in the art will readily be able to implement many other equivalent forms of the present application.
[0032] The above and other aspects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
[0033] Specific embodiments of the present application will be described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments described are merely examples of the present application and may be implemented in a variety of ways. Familiar and / or repetitive functions and structures are not described in detail to avoid obscuring the present application with unnecessary or redundant details. Therefore, the specific structural and functional details described herein are not intended to be limiting, but rather serve merely as a basis and representative basis for the claims to teach those skilled in the art to variously utilize the present application with substantially any suitable detailed structure.
[0034] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may all refer to one or more of the same or different embodiments according to the present application.
[0035] A task processing method according to an embodiment of the present application inputs selection prompt information constructed based on a selection task into a target large model for processing, determines the attention results of each attention layer in the target large model relative to the selection prompt information, determines the attention layer to be deleted from each attention layer based on each attention result, and executes the target large model on at least one processor to delete the attention layer to be deleted in response to the call of the target task. By deleting the attention layer to be deleted from the target large model, the target large model is compressed, thereby improving the degree of compression of the large model, thereby reducing the computing resources consumed by the processor and improving the computing efficiency of the processor.
[0036] like Figure 1 As shown, the task processing method includes the following steps:
[0037] Step S101: obtain a target large model to be compressed, where the target large model can run on at least one processor and includes multiple attention layers.
[0038] In this embodiment, the large model is also called an artificial intelligence large model, which is a type of artificial intelligence model with a large number of parameters constructed by an artificial neural network. The large model may include a large language model, a visual large model, a multimodal large model, and a basic science large model. The target large model includes a multi-layer attention layer. The attention layer is a mechanism in deep learning, inspired by human visual attention, which allows the model to dynamically focus on different parts of the input when processing information and ignore irrelevant information, thereby improving processing efficiency and accuracy. The target large model can run on at least one processor, which can be a dedicated processor specifically for running the large model, such as an NPU (Neural Network Processing Unit, neural network processor), or a CPU (Central Processing Unit, central processing unit), GPU (Graphics Processing Unit, graphics processing unit) and other processors. The target large model to be compressed is a large model to be deployed to a computer device, which includes the above-mentioned at least one processor.
[0039] Step S102: input the selection prompt information constructed based on the selection task into the target large model for processing to determine the attention results of each attention layer relative to the selection prompt information.
[0040] In this embodiment, selection prompt information is constructed based on the selection task, and the selection prompt information is input into the target large model for processing, so that the target large model performs the selection task. During the process of the target large model processing the selection prompt information, each attention layer responds to the input selection prompt information and generates an attention result for the selection prompt information. The selection prompt information can be, for example, one or more correct options selected from multiple options. The type of option can be text, image, or both text and image. In some embodiments of the present application, the selection prompt information is a prompt word. The target large model is used to determine the calculation results of each attention layer relative to the selection prompt information during the execution of the selection task, thereby determining the attention result of each attention layer relative to the selection prompt information.
[0041] Step S103: determining the attention layer to be deleted in each of the attention layers according to each of the attention results.
[0042] In this embodiment, the attention layer to be deleted is determined from each attention layer based on each attention result. There may be one or more attention layers to be deleted. Subsequently, the attention layer to be deleted is deleted from the target large model to achieve compression of the target large model, and the compressed target large model is deployed to the computer device corresponding to the processor.
[0043] Step S104, in response to the call of the target task, running the target large model to delete the attention layer to be deleted on at least one processor.
[0044] In this embodiment, the invocation of the target task can be triggered by a user or by another computer device. In response to the invocation of the target task, the target large model for deleting the attention layer to be deleted is executed on at least one processor, so that the target large model for deleting the attention layer to be deleted executes the target task. Because the target large model for deleting the attention layer to be deleted is a compressed large model, the computing resources consumed by the at least one processor when executing the target task can be reduced.
[0045] It can be understood that steps S101 to S103 and step S104 belong to two different stages. Steps S101 to S103 are generally executed once. After completing steps S101 to S103, step S104 can be repeatedly executed to enable the target large model to perform different target tasks.
[0046] The task processing method of an embodiment of the present application obtains a target large model to be compressed, the target large model being capable of running on at least one processor, and including multiple attention layers; inputs selection prompt information constructed based on a selection task into the target large model for processing to determine the attention results of each attention layer relative to the selection prompt information; determines the attention layer to be deleted from each attention layer based on each attention result; and, in response to a call to the target task, runs the target large model on at least one processor to delete the attention layer to be deleted. By deleting the attention layer to be deleted from the target large model, compression of the target large model is achieved, thereby improving the degree of compression of the large model, thereby reducing the computing resources consumed by the processor and improving the computing efficiency of the processor.
[0047] In some embodiments of the present application, the selection prompt information includes a title and multiple options, wherein at least one option is the same as a key field in the title;
[0048] Each of the attention results includes the option attention of each of the attention layers relative to each of the options, and the attention layers to be deleted in each of the attention layers are determined according to each of the attention results, such as Figure 2 As shown, the following steps are included:
[0049] Step S1031: For each of the attention layers, the option with the highest attention among the multiple options is determined as the target option.
[0050] In this embodiment, the selection prompt information includes a title and multiple options, at least one of which is the same as a key field in the title. For example, the title may be to determine which option from multiple options is consistent with the key field in the title.
[0051] Each attention result includes the option attention of each attention layer relative to each option, for the selection prompt information including the title and multiple options. For each attention layer, its option attention to each option is determined, and the option with the highest option attention among the multiple options is determined as the target option. For example, if each attention layer includes attention layer 1, attention layer 2 and attention layer 3, and the multiple options include four options A, B, C, and D, the option with the highest option attention corresponding to attention layer 1 is A, the option with the highest option attention corresponding to attention layer 2 is B, and the option with the highest option attention corresponding to attention layer 3 is B, then the target option corresponding to attention layer 1 is A, and the target options corresponding to attention layer 2 and attention layer 3 are B.
[0052] Step S1032: Determine the attention layer to be deleted according to the target options corresponding to each attention layer.
[0053] The attention layers that can be deleted are determined according to the target options corresponding to each attention layer, and are determined as the attention layers to be deleted.
[0054] By adopting selection prompt information including questions and multiple options, the attention layer to be deleted is determined according to the target option with the highest attention corresponding to each attention layer, so that the ability characteristics of each attention layer can be efficiently determined, and the attention layer to be deleted can be determined efficiently and accurately.
[0055] In some embodiments of the present application, each of the attention layers includes a plurality of attention heads. Before determining the option with the highest attention among the plurality of options as the target option for each of the attention layers, as shown in FIG. Figure 3 As shown, the following steps are also included:
[0056] Step S201: obtaining a plurality of fields corresponding to the options of the selection prompt information.
[0057] In this embodiment, each attention layer includes multiple attention heads, and the data input to the attention layer is first linearly mapped to three different matrices: query, key, and value. Each attention head operates on these matrices independently, and the outputs of all attention heads are spliced or averaged to obtain the final output. Optionally, the multiple fields corresponding to the selection prompt information can be the multiple fields included in each option. For example, if option A is "A:abc", the fields corresponding to option A may include A, a, b, and c. Alternatively, the multiple fields corresponding to the selection prompt information can also be the token fields corresponding to each field in each option, and the token field belongs to the vocabulary (tokenizer) of the target large model.
[0058] Step S202: Determine the self-attention score of each attention head in each of the attention layers relative to each of the fields.
[0059] In this embodiment, the attention of each attention head relative to each field is represented by an attention score, and the self-attention score of each attention head relative to each field in each attention layer is determined. In some embodiments of the present application, the self-attention matrix corresponding to each attention layer can be determined based on the number of attention heads in each attention layer and the length of the input data. Based on the respective attention scores of each attention head in each attention layer relative to each field in the last row of the self-attention matrix, the self-attention score of each attention head in each attention layer relative to each field is determined.
[0060] Step S203: Determine the option attention of each attention layer relative to each option based on the multiple self-attention scores of each attention layer relative to each field.
[0061] In this embodiment, each option includes multiple fields, and the option attention of each attention layer relative to each option can be determined based on the multiple self-attention scores of each attention layer relative to each field.
[0062] By first determining the self-attention scores of each attention head of each attention layer relative to each field of the determined option, and then determining the option attention of each attention layer relative to each option based on the self-attention scores corresponding to each field of the option, the option attention of each attention layer relative to each option can be determined efficiently and accurately.
[0063] In some embodiments of the present application, the option attention of each attention layer relative to each option is determined based on the multiple self-attention scores of each attention layer relative to each field, such as Figure 4 As shown, the following steps are included:
[0064] Step S2031: for each of the options, determine the option field corresponding to the option from the fields.
[0065] In this embodiment, the option field is composed of the fields in the options. For example, if the options are A:aaa, B:bbb, and C:ccc, the option field corresponding to option A is "A:aaa", the option field corresponding to option B is "B:bbb", and the option field corresponding to option C is "C:ccc".
[0066] Step S2032: Determine the option attention score of each attention layer relative to each option field based on the multiple self-attention scores of each attention layer relative to each field.
[0067] Each option field includes multiple fields, and the option attention score of each attention layer relative to each option field is determined based on the multiple self-attention scores of each attention layer relative to each field. In some embodiments of the present application, the average value of the respective attention scores of each attention layer relative to each field can be determined first, and then for each option field, the average value of each attention layer relative to each field in the option field is added to determine the option attention score of each attention layer relative to each option field. For example, the average value of the respective attention scores of each attention layer relative to each field can be determined according to Formula 1, which is:
[0068]
[0069] Among them, attn i,j represents the average of the respective attention scores of the jth field in the i-th row of the self-attention matrix, i represents the last row of the self-attention matrix, H represents the number of attention heads in each attention layer, represents the self-attention score of the h-th attention head in each attention layer relative to the j-th field in the i-th row of the self-attention matrix.
[0070] Step S2033: Determine the option attention of each attention layer relative to each option based on the option attention score of each attention layer relative to each option field.
[0071] In this embodiment, for each attention layer, the option attention of the attention layer relative to each option is determined according to the corresponding attention score of each option. For example, if the option field corresponding to option A is "A:aaa", the option field corresponding to option B is "B:bbb", and the option field corresponding to option C is "C:ccc", then the option attention of attention layer 1 relative to option A is determined by the attention score of attention layer 1 relative to "A:aaa", the option attention of attention layer 1 relative to option B is determined by the attention score of attention layer 1 relative to "B:bbb", and the option attention of attention layer 1 relative to option C is determined by the attention score of attention layer 1 relative to "C:ccc".
[0072] Subsequently, based on the attention of each option, the option with the highest option attention among multiple options is determined as the target option. The option attention score corresponding to the target option can be determined by Formula 2, which is:
[0073] Score i,S =max(attn i,k ),k∈tokens(S);
[0074] Among them, Score i,Srepresents the option attention score corresponding to the target option, S represents the set of option fields, k represents the kth option field in S, attn i,k represents the option attention score of the k-th option field.
[0075] By determining the option attention score of each option field and determining the option attention of each attention layer relative to each option based on the option attention score of each option field, the accuracy of the option attention of each attention layer relative to each option is improved.
[0076] In some embodiments of the present application, there are multiple selection prompts, and different selection prompts correspond to different questions, different options, or the same question but different option distributions. The process of inputting the selection prompts constructed based on the selection task into the target large model for processing is performed multiple times.
[0077] The attention layer to be deleted is determined according to the target options corresponding to each attention layer, such as Figure 5 As shown, the following steps are included:
[0078] Step S10321, determining the target options corresponding to each of the attention layers in each of the selection tasks.
[0079] In this embodiment, there are multiple selection prompts, and the selection prompts constructed based on the selection task are input into the target large model for processing multiple times. For each attention layer, multiple target options can be determined. In some cases, different selection prompts correspond to different questions and different options. For example, Figure 6 The diagram shown is a schematic diagram of selecting prompt information 1, as shown in FIG. Figure 7 The diagram shows a selection prompt 2. Compared with the selection prompt 1, the selection prompt 2 has different questions and multiple options. In other cases, different selection prompts have the same questions but different option distributions. For example, Figure 8 The diagram shown is a schematic diagram of selection prompt information 3. Compared with selection prompt information 1, selection prompt information 3 has the same question but a different option distribution.
[0080] Step S10322: determine the distribution discreteness of the multiple target options corresponding to each of the attention layers.
[0081] In this embodiment, since the selection prompt information constructed based on the selection task is input into the target large model for processing multiple times, each attention layer can correspond to multiple target options, and the distribution dispersion of the multiple target options is determined based on the number of times each target option belongs to each option. For example, if the multiple options include four options A, B, C, and D, the distribution dispersion of the multiple target options is determined based on the number of times the target option is A, the number of times the target option is B, the number of times the target option is C, and the number of times the target option is D. Optionally, the distribution dispersion can be determined by any one of the indicators such as the variance, standard deviation, and range of the multiple target options.
[0082] Step S10323: Determine at least one attention layer to be deleted from each attention layer according to the distribution discreteness corresponding to each attention layer, and the distribution discreteness corresponding to the attention layer to be deleted is greater than the distribution discreteness corresponding to the remaining attention layers in the target large model.
[0083] In this embodiment, the options corresponding to the correct answers are randomly or evenly distributed among all the options corresponding to the selection prompt information. For example, if the correct answers are evenly distributed, there are 100 selection prompts, and the options include A, B, C, and D, then the correct answers are located in the four options A, B, C, and D 25 times each. For different attention layers, the greater the distribution discreteness of the target options corresponding to the attention layer, the more susceptible the attention layer is to the influence of option position offset rather than the semantics of the selection prompt information, and the attention layer focuses on the wrong options. Therefore, its contribution to the overall capability of the target large model is also smaller. Determine at least one attention layer to be deleted from each attention layer, and the distribution discreteness of the target options corresponding to the attention layer to be deleted is greater than the distribution discreteness of the target options corresponding to the remaining attention layers in the target large model, that is, the attention layer to be deleted is the one or more attention layers with the largest distribution discreteness of the target options among each attention layer.
[0084] By inputting the selection prompt information into the target large model for processing multiple times, and making the options corresponding to the correct answers randomly or evenly distributed among all the options corresponding to each selection prompt information, the influence of the distribution of different correct answers on the target large model can be reduced or eliminated. Moreover, by determining the distribution discreteness of multiple target options corresponding to each attention layer, the attention layer with poor self-consistency and easily affected by the option position offset can be accurately determined from each attention layer, thereby achieving efficient and accurate determination of the attention layer to be deleted, further improving the accuracy of model compression.
[0085] In some embodiments of the present application, at least one attention layer to be deleted is determined from each attention layer based on the distribution discreteness of the target options corresponding to each attention layer, including: determining multiple candidate attention layers with the largest distribution discreteness of the target options from each attention layer based on the number of target candidates; and determining the attention layer to be deleted from each candidate attention layer based on the target screening condition. The target screening condition can be determined based on one or more data including the total number of parameters of the target large model, the total number of layers, the target parameter amount, the degree of model capability degradation, the computing resources of the processor, whether it is the early attention layer of the target large model, the performance of the processor, etc. For example, the early attention layer is generally the basic attention layer of the large model, and its output result will directly affect the capability of the later attention layer. Therefore, if the candidate attention layer belongs to the early attention layer of the target large model, it will not be determined as the attention layer to be deleted, so as to avoid a significant impact on the performance of the target large model due to the deletion of the early attention layer.
[0086] By first determining multiple candidate attention layers and then determining the attention layer to be deleted from the multiple candidate attention layers, the accuracy of the attention layer to be deleted is improved.
[0087] In some embodiments of the present application, Figure 9 As shown, the process of constructing each selection prompt information includes the following steps:
[0088] Step S301: obtaining a selection prompt template including positions to be filled in, wherein the positions to be filled in include the filling positions of the key fields in the question and the filling positions corresponding to the key fields in the options.
[0089] In this embodiment, a selection prompt template including positions to be filled in is pre-established. Since at least one option is the same as the key field in the question, the positions to be filled in include the filling position of the key field in the question and the filling position corresponding to the key field in the option. Figure 10 The figure shows a schematic diagram of the selection prompt template. The filling position of the key field in the question and the filling position corresponding to the key field in the option are the positions of "Token" in the selection prompt template.
[0090] Step S302: obtaining a plurality of fields to be filled in from a database related to the target large model.
[0091] The database associated with the target large model can be a database built into the target large model or a database plug-in of the target large model. In some embodiments of the present application, the database associated with the target large model is a vocabulary (Tokenizer) of the target large model, through which the input text of the model can be segmented into discrete word units (tokens), which are then used in the training and reasoning process of the model. A plurality of fields to be filled in are obtained from the database associated with the target large model. In the case where the database is a vocabulary of the target large model, the plurality of fields to be filled in are a plurality of tokens determined from the vocabulary.
[0092] Step S303: Fill in each of the fields to be filled in into the filling position respectively, and generate each of the selection prompt information.
[0093] Fill in each field to be filled in the filling position respectively, instantiate the selection prompt template, and generate each selection prompt information. Each selection prompt information can correspond to one or more fields to be filled in.
[0094] By obtaining multiple fields to be filled in from a database related to a target large model and filling in the to-be-filled positions in a selection prompt template, the efficiency of constructing selection prompt information is improved.
[0095] In some embodiments of the present application, after determining at least one attention layer to be deleted from each attention layer according to the distribution discreteness of the target options corresponding to each attention layer, Figure 11 As shown, the following steps are also included:
[0096] Step S401: Delete the attention layer to be deleted from the target large model.
[0097] In this embodiment, the attention layer to be deleted is deleted from the target large model to compress the target large model.
[0098] Step S402: determine the attention layer to be adjusted from the remaining attention layers in the target large model.
[0099] In this embodiment, after deleting the attention layer to be deleted from the target large model, in order to reduce the impact of deleting the attention layer to be deleted on the target large model, the parameters of the remaining attention layers are fine-tuned. Specifically, the attention layer to be adjusted is first determined from the remaining attention layers in the target large model.
[0100] Step S403: Keep the parameters of the target large model except the parameters of the attention layer to be adjusted unchanged, and adjust the parameters of the attention layer to be adjusted based on the target fine-tuning task.
[0101] In this embodiment, after determining the attention layer to be adjusted, all parameters of the target large model, except for the parameters of the attention layer to be adjusted, are kept unchanged. The parameters of the attention layer to be adjusted are then adjusted based on the target fine-tuning task. Specifically, all parameters of the target large model, except for the parameters of the attention layer to be adjusted, are frozen, and the parameters of the attention layer to be adjusted are set to receive backpropagation updates, so that the target large model performs the target fine-tuning task. Optionally, the target fine-tuning task can be the same as or different from the selection task.
[0102] By adjusting the remaining attention layers, the impact of deleting the attention layers to be deleted on the target large model is reduced, thereby reducing the performance loss of the target large model.
[0103] In some embodiments of the present application, the attention layer to be adjusted satisfies at least one of the following conditions:
[0104] The number of attention layers to be adjusted does not exceed the number of attention layers to be deleted;
[0105] The attention layer to be adjusted belongs to an adjacent attention layer of the attention layer to be deleted;
[0106] The attention layer to be adjusted belongs to the remaining candidate attention layers among the candidate attention layers, and the attention layer to be deleted is determined from the candidate attention layers. The distribution discreteness of the target options corresponding to each candidate attention layer is greater than the distribution discreteness of the target options corresponding to other attention layers in the target large model except the candidate attention layers.
[0107] In this embodiment, by ensuring that the number of attention layers to be adjusted does not exceed the number of attention layers to be deleted, it is possible to avoid the number of attention layers to be adjusted being too large, and to reduce the impact of deleting the attention layers to be deleted on the target large model. By ensuring that the attention layer to be adjusted belongs to the adjacent attention layer of the attention layer to be deleted, the impact of deleting the attention layer to be deleted on the target large model can be more effectively reduced. By ensuring that the attention layer to be adjusted belongs to the remaining candidate attention layers among the candidate attention layers, it is possible to ensure that only the attention layers with a higher degree of discreteness in the distribution of the target options are adjusted, ensuring that the adjustment of the attention layer to be adjusted does not significantly affect the performance of the target large model. In some embodiments of the present application, the set of candidate attention layers is determined by Formula 3, and Formula 3 is:
[0108] DropCandidateSet=Top K (SD1,SD2,...,SD n );
[0109] The set of attention layers to be adjusted is determined by Formula 4, which is:
[0110]
[0111] Among them, DropCandidateSet represents the set of candidate attention layers, SD n Indicates the standard deviation of multiple target options corresponding to the nth attention layer, Top K Indicates the selection of the largest K, FinetuningLayerSet indicates the set of attention layers to be adjusted, Represents the remaining candidate attention layers after deleting the attention layer to be deleted from the set of candidate attention layers. 2K Indicates selecting the largest 2K.
[0112] The present application also provides a computer device, such as Figure 12 As shown, it includes a memory, at least one processor and a computer program stored in the memory, and at least one processor executes the computer program to implement:
[0113] Obtain a target large model to be compressed, the target large model being capable of running on at least one of the processors, the target large model comprising multiple attention layers; inputting selection prompt information constructed based on a selection task into the target large model for processing to determine an attention result of each of the attention layers relative to the selection prompt information; determining an attention layer to be deleted in each of the attention layers based on each of the attention results; and in response to a call of a target task, running the target large model to delete the attention layer to be deleted.
[0114] The computer device in the embodiments of the present application may be a terminal or other device other than a terminal. For example, the computer device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), an ATM, or an kiosks, etc., and the embodiments of the present disclosure do not specifically limit this.
[0115] The memory may include RAM (Random Access Memory) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0116] The above-mentioned processor can be a general-purpose processor, including a CPU, NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0117] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).
[0118] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.
Claims
1. A task processing method, comprising: Obtaining a target large model to be compressed, where the target large model can be run on at least one processor, and the target large model includes multiple attention layers; Inputting the selection prompt information constructed based on the selection task into the target large model for processing to determine the attention results of each attention layer relative to the selection prompt information; Determining an attention layer to be deleted from each of the attention layers according to each of the attention results; In response to the call of the target task, the target large model that deletes the attention layer to be deleted is run on at least one processor.
2. The task processing method according to claim 1, wherein the selection prompt information includes a title and multiple options, wherein at least one option is the same as a key field in the title; Each of the attention results includes the option attention of each of the attention layers relative to each of the options, and determining the attention layer to be deleted from each of the attention layers according to each of the attention results includes: For each of the attention layers, determining the option with the highest option attention among the multiple options as the target option; The attention layer to be deleted is determined according to the target option corresponding to each attention layer.
3. The task processing method of claim 2, wherein each of the attention layers comprises a plurality of attention heads, and before determining the option with the highest attention among the plurality of options as the target option for each of the attention layers, the method further comprises: Obtaining multiple fields of options corresponding to the selection prompt information; Determining a self-attention score for each attention head in each of the attention layers relative to each of the fields; The option attention of each attention layer relative to each option is determined based on the multiple self-attention scores of each attention layer relative to each field.
4. The task processing method according to claim 3, wherein determining the option attention of each attention layer relative to each option based on the plurality of self-attention scores of each attention layer relative to each field comprises: For each of the options, determining an option field corresponding to the option from the fields; Determining an option attention score of each of the attention layers relative to each of the option fields according to a plurality of the self-attention scores of each of the attention layers relative to each of the fields; The option attention of each attention layer relative to each option is determined based on the option attention score of each attention layer relative to each option field.
5. The task processing method according to claim 2, wherein the selection prompt information is multiple, and different selection prompt information corresponds to different questions, multiple options, or corresponds to the same question, but the option distribution is different, and the process of inputting the selection prompt information constructed based on the selection task into the target large model for processing is performed multiple times; The determining the attention layer to be deleted according to the target options corresponding to each attention layer includes: Determining the target option corresponding to each of the attention layers in each of the selection tasks; Determining the distribution discreteness of the plurality of target options corresponding to each of the attention layers; According to the distribution discreteness corresponding to each of the attention layers, at least one of the attention layers to be deleted is determined from each of the attention layers, and the distribution discreteness corresponding to the attention layer to be deleted is greater than the distribution discreteness corresponding to the remaining attention layers in the target large model.
6. The task processing method according to claim 5, wherein the process of constructing each selection prompt information comprises: Obtaining a selection prompt template including positions to be filled in, wherein the positions to be filled in include positions for filling in the key fields in the question and positions in the options corresponding to the key fields; Obtain a plurality of fields to be filled in from a database related to the target large model; Fill in each of the fields to be filled in into the filling position respectively to generate each of the selection prompt information.
7. The task processing method according to claim 5, wherein the options corresponding to the correct answers are randomly or evenly distributed among all the options corresponding to the selection prompt information.
8. The task processing method according to claim 5, further comprising: after determining at least one attention layer to be deleted from each of the attention layers based on the distribution dispersion corresponding to each of the attention layers; Deleting the to-be-deleted attention layer from the target large model; Determining an attention layer to be adjusted from the remaining attention layers in the target large model; Keep the parameters of the target large model except the parameters of the attention layer to be adjusted unchanged, and adjust the parameters of the attention layer to be adjusted based on the target fine-tuning task.
9. The task processing method according to claim 8, wherein the attention layer to be adjusted satisfies at least one of the following conditions: The number of attention layers to be adjusted does not exceed the number of attention layers to be deleted; The attention layer to be adjusted belongs to an adjacent attention layer of the attention layer to be deleted; The attention layer to be adjusted belongs to the remaining candidate attention layers among the candidate attention layers, and the attention layer to be deleted is determined from the candidate attention layers. The distribution discreteness corresponding to each candidate attention layer is greater than the distribution discreteness corresponding to other attention layers in the target large model except the candidate attention layers.
10. A computer device comprising a memory, at least one processor, and a computer program stored in the memory, wherein at least one processor executes the computer program to implement: Obtaining a target large model to be compressed, the target large model being capable of running on at least one of the processors, the target large model comprising multiple attention layers; inputting selection prompt information constructed based on the selection task into the target large model for processing to determine an attention result of each of the attention layers relative to the selection prompt information; Determining an attention layer to be deleted from each of the attention layers according to each of the attention results; In response to the call of the target task, the target large model is run to delete the attention layer to be deleted.