Task-based model reasoning method and device

Abstract

The invention provides a model reasoning method and device based on tasks, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining to-be-processed data and a task identifier corresponding to the to-be-processed data; obtaining an index set corresponding to the to-be-processed data by utilizing the task identifier; acquiring a task matrix corresponding to the to-be-processed data by utilizing the task identifier; and performing transformation processing on the to-be-processed data by using the layer task matrixes corresponding to all the reasoning layers in the reasoning model to obtain a reasoning result of the to-be-processed data. In the invention, the preset index and the layer task matrix are both related to the task identifier, the reasoning performance under different task scenes can be ensured, the layer task matrix of the reasoning layer with the preset index is utilized to process the to-be-processed data, all model layers are not required to be used for processing, unnecessary calculation can be effectively reduced, and the reasoning efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a task-based model reasoning method and device. Background Art

[0002] With the rapid development of artificial intelligence technology, large models are used in various task scenarios. A large model is a neural network model with a very large number of parameters. Due to its large number of parameters, there is an excessive burden of reasoning calculation when it is applied, which limits the practicality and scalability of the large model, resulting in low reasoning efficiency when applying the large model.

[0003] At present, the computational cost of reasoning can be reduced by methods such as model compression. Model compression refers to operations such as parameter compression and dimension reduction on the original network structure to improve the reasoning speed of the network. However, model compression is usually for a single task scenario, and a compressed model is saved in each task scenario, which is inconvenient to maintain and use. In order to pursue the versatility of the model, some model compressions will be balanced in multiple task scenarios, resulting in certain performance losses in different task scenarios. Therefore, there is an urgent need for an inference method that can support multiple task scenarios and take into account both inference efficiency and inference performance. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a task-based model reasoning method and device, which can support multiple task scenarios and take into account both reasoning efficiency and reasoning performance.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a task-based model reasoning method, comprising:

[0007] Acquire the data to be processed and the task identifier corresponding to the data to be processed;

[0008] Using the task identifier, obtaining an index set corresponding to the data to be processed, wherein the index set includes a plurality of preset indexes;

[0009] Using the task identifier, obtaining a task matrix corresponding to the data to be processed, wherein the task matrix includes a layer task matrix associated with the preset index;

[0010] The data to be processed is transformed using the layer task matrices corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed, and the layer index of the inference layer in the inference model is the preset index.

[0011] Optionally, the transforming the data to be processed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed includes:

[0012] For each model layer in the inference model, if layer input data corresponding to the model layer is received, determine whether the model layer is an inference layer according to the layer index of the model layer in the inference model and the index set, and the layer input data is related to the data to be processed;

[0013] If the model layer is an inference layer, the layer input data is transformed into layer output data through a layer task matrix corresponding to the inference layer, and the layer output data is passed to the next model layer of the model layer;

[0014] If the model layer is not an inference layer, using the layer input data as layer output data, and passing the layer output data to the next model layer of the model layer;

[0015] The layer output data of the last model layer in the inference model is used as the inference result of the data to be processed.

[0016] Optionally, the transforming the data to be processed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed includes:

[0017] Determine, in the inference model, a model layer whose layer index is consistent with the preset index as the inference layer;

[0018] For each of the inference layers, if layer input data corresponding to the inference layer is obtained, the layer input data is related to the data to be processed;

[0019] Transforming the layer input data into layer output data through the layer task matrix corresponding to the reasoning layer, and passing the layer output data to the next reasoning layer of the reasoning layer;

[0020] The layer output data of the last inference layer in the inference model is used as the inference result of the data to be processed.

[0021] Optionally, the layer task matrix includes a layer dimension increase matrix and a layer dimension reduction matrix, and the layer input data is transformed into layer output data by using the layer task matrix corresponding to the reasoning layer, including:

[0022] Obtaining an initial parameter matrix of the inference layer;

[0023] Calculate the product of the layer dimension increase matrix and the layer dimension reduction matrix to obtain an intermediate parameter matrix;

[0024] Summing the initial parameter matrix and the intermediate parameter matrix to obtain an inference parameter matrix;

[0025] The layer input data is calculated and processed using the inference parameter matrix to obtain layer output data.

[0026] Optionally, before using the task identifier to obtain an index set corresponding to the data to be processed, the method further includes:

[0027] Obtaining a first task data set corresponding to a preset task identifier;

[0028] Performing reasoning on the first task data set using the reasoning model to obtain specified performance parameters of the reasoning model and a layer input vector of each model layer;

[0029] Calculate the importance parameter of each model layer under the first task data set using the specified performance parameter and the layer input vector of each model layer;

[0030] Calculate the number of targets based on the layer input vectors and inference resource consumption of each model layer;

[0031] The target number of inference layers is determined from the inference model according to the importance parameter, and the layer indexes of all the inference layers are used as a preset index set corresponding to the preset task identifier.

[0032] Optionally, the performing reasoning on the first task data set using the reasoning model to obtain specified performance parameters of the reasoning model and a layer input vector of each model layer includes:

[0033] Performing a first inference on the first task data set using all model layers in the inference model to obtain a baseline performance parameter of the inference model and a layer input vector of each model layer;

[0034] For each of the model layers in the reasoning model, use other model layers except the model layer to perform second reasoning on the first task data set to obtain layer performance parameters corresponding to the model layer;

[0035] The baseline performance parameters and the layer performance parameters are used as designated performance parameters of the inference model.

[0036] Optionally, the calculating, using the specified performance parameter and the layer input vector of each model layer, an importance parameter of each model layer under the first task data set includes:

[0037] Calculating the relative weight of the model layer using the initial parameter matrix corresponding to the model layer;

[0038] Calculating a performance change parameter of the model layer using the baseline performance parameter and the layer performance parameter;

[0039] Calculating the feature similarity of the model layer using the layer input vector of the model layer and the layer input vector of the next model layer;

[0040] The importance parameter corresponding to the model layer is obtained by subtracting the feature similarity from the sum of the relative weight and the performance change parameter.

[0041] Optionally, the calculating the target quantity based on the layer input vector of each model layer and the inference resource consumption includes:

[0042] Obtaining resource consumption of the first inference and total resources deployed;

[0043] Calculating the ratio of the resource consumption to the total resource amount to obtain a resource consumption ratio;

[0044] Calculating the task processing difficulty corresponding to the preset task identifier using the layer input vector of each model layer;

[0045] The target quantity is calculated according to the resource consumption ratio and the task processing difficulty.

[0046] Optionally, after determining a target number of inference layers from a plurality of model layers in the inference model according to the importance parameter and using the index of the inference layer as a preset index set corresponding to the preset task identifier, the method further includes:

[0047] For each inference layer, insert the initial dimension reduction matrix and the initial dimension increase matrix into the initial parameter matrix corresponding to the inference layer;

[0048] Fine-tune the inference model using a second task data set corresponding to a preset task identifier, and perform gradient update on the initial dimension reduction matrix and the initial dimension increase matrix to obtain a layer dimension reduction matrix and a layer dimension increase matrix corresponding to the inference layer;

[0049] The layer index corresponding to the reasoning layer is used as a preset index, and the layer dimension reduction matrix and the layer dimension increase matrix are used as layer task matrices;

[0050] The layer task matrices corresponding to all preset indexes are used as preset task matrices, and an association relationship between the preset task matrix and the preset task identifier is established.

[0051] On the other hand, the present invention also provides a task-based model reasoning device for implementing any of the above methods, the device comprising:

[0052] A data acquisition module, used to acquire the data to be processed and the task identifier corresponding to the data to be processed;

[0053] An index acquisition module, used to acquire an index set corresponding to the data to be processed by using the task identifier, wherein the index set includes a plurality of preset indexes;

[0054] A matrix acquisition module, used to acquire a task matrix corresponding to the data to be processed by using the task identifier, wherein the task matrix includes a layer task matrix associated with the preset index;

[0055] The inference module is used to transform the data to be processed with the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed, and the layer index of the inference layer in the inference model is the preset index.

[0056] On the other hand, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps in any task-based model reasoning method provided by the present invention.

[0057] On the other hand, the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any task-based model reasoning method provided by the present invention.

[0058] On the other hand, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in any one of the task-based model reasoning methods provided by the present invention.

[0059] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0060] In an embodiment of the present invention, the data to be processed and the task identifier corresponding to the data to be processed are obtained, and the index set and the task matrix of the data to be processed are obtained using the task identifier. The index set includes multiple preset indexes, and the task matrix includes a layer task matrix associated with each preset index. Both the preset index and the layer task matrix are related to the task identifier, which can ensure the reasoning performance in different task scenarios. The layer task matrix of the reasoning layer with the preset index is used to process the data to be processed, without using all model layers for processing, which can effectively reduce unnecessary calculations and improve reasoning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 is a schematic diagram of an application scenario of a task-based model reasoning method provided by an embodiment of the present invention;

[0063] Figure 2 is a flowchart of a task-based model reasoning method provided by an embodiment of the present invention;

[0064] Figure 3 is a schematic diagram of a first task data set for reasoning provided by an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of a process for reasoning about data to be processed provided by an embodiment of the present invention;

[0066] Figure 5 is another schematic diagram of a flow chart of reasoning data to be processed provided by an embodiment of the present invention;

[0067] Figure 6 is a schematic diagram of the structure of a task-based model reasoning device provided by an embodiment of the present invention;

[0068] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0070] It is understandable that in the specific implementation of the present invention, data related to user information, etc., needs to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0071] The present invention proposes a task-based model reasoning method and device, which can support multiple task scenarios while taking into account both reasoning efficiency and reasoning performance. Figure 1, shows a schematic diagram of an application scenario of a task-based model reasoning method. The application scenario may include a terminal 101 and a server 102, and data can be exchanged between the terminal 101 and the server 102 via a network, and a corresponding application program may be installed on the terminal 101. The terminal 101 may be a mobile phone, a tablet computer, a smart Bluetooth device, a computer, a large screen device, a robot, etc.; the server 102 may be a single server or a server cluster consisting of multiple servers.

[0072] The user can send the data to be processed and the corresponding task identifier to the server 102 through the terminal 101. The server 102 can use the task identifier to obtain the index set corresponding to the data to be processed, and the index set includes multiple preset indexes; use the task identifier to obtain the task matrix corresponding to the data to be processed, and the task matrix includes a layer task matrix associated with the preset index; use the layer task matrix corresponding to all reasoning layers in the reasoning model to transform the data to be processed to obtain the reasoning result of the data to be processed, wherein the layer index of the reasoning layer in the reasoning model is the preset index.

[0073] The server 102 may send the inference result to the terminal 101 so that the terminal 101 displays the inference result to the user.

[0074] In this embodiment, a task-based model reasoning method is provided, such as Figure 2 As shown, the specific process of the task-based model reasoning method can be as follows:

[0075] S110: Acquire data to be processed and a task identifier corresponding to the data to be processed.

[0076] The data to be processed is the data input by the user that needs to be processed by the model. The data to be processed can be one or more of text, image, and video. The type of data to be processed can be determined based on the type of data that the model can process.

[0077] The task ID is the task information associated with the data to be processed. It can be input by the user. One task ID corresponds to one task. For example, a question-and-answer task in the health care field corresponds to one task ID, a policy question-and-answer task corresponds to one task ID, and a question-and-answer task in the legal field corresponds to one task ID. The correspondence between task IDs and tasks can be agreed upon in advance.

[0078] S120: Using the task identifier, obtain an index set corresponding to the data to be processed.

[0079] After obtaining the task identifier, the task identifier can be used to obtain an index set corresponding to the data to be processed, wherein the index set includes multiple preset indexes. Optionally, a first association relationship between the preset task identifier and the preset index set can be first obtained, and then the preset index set corresponding to the task identifier can be obtained based on the first association relationship as the index set corresponding to the data to be processed.

[0080] Before using the task identifier to obtain the index set corresponding to the data to be processed, a first association relationship between the preset task identifier and the preset index set can be established. The first association relationship between the preset task identifier and the preset index set is stored in a designated location, and can be obtained by directly accessing the designated location when needed. Based on the first association relationship, the task identifier can be searched in the preset task identifier, and the preset index set corresponding to the task identifier can be used as the index set.

[0081] In some embodiments, when establishing a first association relationship between a preset task identifier and a preset index set, the first task data set corresponding to the preset task identifier can be obtained; the first task data set is inferred using the inference model to obtain specified performance parameters of the inference model and the layer input vector of each model layer; the importance parameters of each model layer under the first task data set are calculated using the specified performance parameters and the layer input vector of each model layer; the target number is calculated based on the layer input vector of each model layer and the inference resource consumption; the target number of inference layers is determined from the inference model according to the importance parameters, and the layer indexes of all the inference layers are used as the preset index set corresponding to the preset task identifier.

[0082] The preset task identifier can be set according to actual needs, and one preset task identifier corresponds to one task. There can be multiple preset task identifiers. For each preset task identifier, the corresponding first task data set can be obtained, wherein the first task data set is a small sample calibration data set, which can usually contain 1024 samples. The first task data set is inferred using the inference model to obtain the performance parameters of the inference model and the layer input vector of each model layer in the inference model.

[0083] The inference model is a large model, which may include an embedding layer and multiple attention layers. The model layer referred to in the embodiment of the present invention is the attention layer. The specified performance parameters may be accuracy, perplexity, etc., which may be set according to actual needs. When reasoning about the first task data set, reasoning may be performed multiple times, and the model layers used for each reasoning are different to evaluate the impact of each model layer on the reasoning. In the embodiment of the present invention, the number of reasoning times is related to the total number of model layers in the reasoning model.

[0084] When the inference model is used to infer the first task data set, each data in the first task data set will pass through the model layer, so that the layer input vector corresponding to each model layer can be obtained. Using the specified performance parameters and the layer input vector of each model layer, the importance parameter of each model layer under the first task data set can be calculated. The importance parameter can reflect the importance of each model layer in processing the first task data set. The larger the importance parameter, the more important the model layer. Under different tasks, the degree of participation of each model layer in the large model in reasoning is different. Evaluating the importance parameters of each model layer based on the first task data set is more suitable for the specific task referred to by the preset task identifier.

[0085] The target number refers to the number of model layers required to use when reasoning about the task indicated by the preset task identifier. The model reasoning process is affected by computing resources and task difficulty, so the target number can be determined based on the layer input vector of each model layer and the consumption of reasoning resources. Then, according to the importance parameter, the target number of reasoning layers are selected from the reasoning model, and the indexes of all reasoning layers are used as the preset index set corresponding to the preset task identifier.

[0086] In order to comprehensively evaluate the contribution of each model layer to reasoning, when using the inference model to reason on the first task data set, the first task data set can be inferred using all model layers in the inference model to obtain the baseline performance parameters of the inference model and the layer input vector of each model layer; for each model layer in the inference model, the first task data set is inferred using other model layers except the model layer to obtain the layer performance parameters corresponding to the model layer; the baseline performance parameters and layer performance parameters are used as the specified performance parameters of the inference model.

[0087] For example, see Figure 3 , shows a schematic diagram of reasoning the first task data set. The first reasoning refers to reasoning the first task data set using all model layers in the reasoning model. The task data set can be processed by multiple model layers in sequence to obtain the final reasoning result. In this process, the layer input vector corresponding to each model layer and the performance parameters of the reasoning model can be obtained, and the performance parameters can be used as benchmark performance parameters.

[0088] The second reasoning refers to removing one of the model layers in the reasoning model and using the remaining model layers to reason about the task data set, where removal can refer to shielding, skipping, or culling. For example, Figure 3In the figure, the dotted box represents the model layer that has been removed, and the data enters the subsequent model layer directly without being processed by the model layer. If there are n model layers in the inference model, n second inferences are required, and one of the model layers will be removed during each second inference. During each second inference, the performance parameters of the inference model can be recorded. Since one model layer is removed in each second inference, the performance parameters can be recorded as the layer performance parameters of the model layer. After n second inferences, the layer performance parameters of each model layer can be obtained. Finally, the baseline performance parameters and the layer performance parameters of each model layer can be used as the specified performance parameters of the inference model.

[0089] The specified performance parameters may include a baseline performance parameter and multiple layer performance parameters, and the layer performance parameters correspond one-to-one to the model layers in the inference model. Using the specified performance parameters and the layer input vector of each model layer, the importance parameters of each model layer under the first task data set can be calculated. When calculating the importance parameters of the model layer, the relative weight of the model layer can be calculated using the initial parameter matrix corresponding to the model layer; the performance change parameters of the model layer can be calculated using the baseline performance parameters and the layer performance parameters; the feature similarity of the model layer is calculated using the layer input vector of the model layer and the layer input vector of the next model layer; the importance parameter corresponding to the model layer is obtained by subtracting the feature similarity from the sum of the relative weight and the performance change parameter.

[0090] Each model layer in the inference model has its corresponding initial parameter matrix, also called weight matrix, which is obtained through a large amount of training. The model weight accuracy range defines the value range of the weight value in the model. The model weight accuracy range may include the minimum accuracy and the maximum accuracy. The relative weight of the model layer can be calculated using the minimum accuracy, the maximum accuracy and the initial parameter matrix of the model layer.

[0091] The aforementioned baseline performance parameters are the performance parameters obtained when reasoning using all model layers, while the layer performance parameters are the performance parameters obtained when reasoning when the corresponding model layer is missing. The degree of impact on the performance of the reasoning model after the corresponding model layer is missing can be calculated to obtain the performance change parameters.

[0092] Among them, the performance change parameter can first use the baseline performance parameter to subtract the layer performance parameter to obtain the specific value of the performance change, and then use the specific value of the performance change to divide the baseline performance parameter to obtain the performance degradation ratio as the performance change parameter. Since there are positive and negative indicators for performance parameters, for example, accuracy is a positive indicator, that is, the greater the accuracy, the better the performance, and confusion is a negative indicator, the greater the confusion, the worse the performance. Therefore, a sign parameter can be introduced. When the performance parameter adopts a positive indicator, the sign parameter is positive, and when the performance parameter adopts a negative indicator, the sign parameter is negative.

[0093] In the first reasoning, the layer input vector corresponding to each model layer can be obtained, and the average cosine similarity can be calculated based on the layer input vector of the model layer and the layer input vector of the next model layer as the feature similarity of the model layer. The higher the cosine similarity, the higher the feature similarity, indicating that the model layer processes less data content, the smaller the gap between the input vector of the model layer and the input vector of the next model layer, and the lower the importance of the model layer.

[0094] Then, the relative weight and the performance change parameter can be summed, and the feature similarity can be subtracted from the sum to obtain the importance parameter corresponding to the model layer.

[0095] Specifically, the importance parameter of the model layer can be calculated according to the following formula:

[0096] ;

[0097] Among them, LS j Characterize the importance parameter of the j-th model layer; The L2 norm of the initial parameter matrix representing the j-th model layer; w min Minimum characterization accuracy; w max Characterize the maximum accuracy; represents the difference between the baseline performance parameter and the layer performance parameter of the j-th model layer; metric0 represents the baseline performance parameter; β is the symbol parameter; x j The layer input vector representing the j-th model layer; x j+1 The layer input vector representing the j+1th model layer. The first item is the relative weight. Logarithmic mapping is performed in the process of calculating the relative weight to prevent the data from being too skewed. Linear normalization can map the relative weight to the range of [0, 1], which is consistent with the value range of the performance change parameter and feature similarity.

[0098] Thus, the importance parameter of each model layer can be calculated, and the model layers can be sorted in descending order according to the importance parameter to obtain the model layer sequence.

[0099] In different tasks, the contribution of each model layer is different. In order to reduce unnecessary computational burden, some model layers can be adaptively activated for reasoning during reasoning tasks. As an implementation method, an importance threshold can be set, and the model layer with an importance parameter greater than the importance threshold is used as the reasoning layer. The reasoning layer is the model layer that needs to be activated when processing data.

[0100] As another implementation, the number of layers activated during reasoning can be determined in combination with the actual reasoning resources and task difficulty. For example, the resource consumption of the first reasoning and the total amount of resources deployed can be obtained; the ratio of the resource consumption to the total amount of resources can be calculated to obtain the resource consumption ratio; the task processing difficulty corresponding to the preset task identifier can be calculated using the layer input vector of each model layer; and the target number can be calculated based on the resource consumption ratio and the task processing difficulty.

[0101] In the first reasoning and multiple second reasonings, the inference model consumes different resources due to the different model layers used. In the first reasoning, all model layers are used for reasoning, and the resource consumption of the first reasoning can be obtained for subsequent calculations. The resources in the embodiment of the present invention can be understood as computing resources. The total amount of resources deployed refers to the total amount of computing resources on the device for reasoning. The resource consumption ratio can be obtained by dividing the resource consumption by the total amount of resources.

[0102] Different task identifiers correspond to different tasks. Different tasks have different difficulties, consume different computing resources, and use different numbers of model layers. In the first reasoning, the layer input vector of each model layer can be obtained. The more similar the layer input vectors of each adjacent model layer are, the less difficult the task is. Taking into account the consumption of computing resources and the difficulty of task processing, the target number can be calculated based on the following formula:

[0103] ;

[0104] Among them, k i represents the number of targets; R represents the total amount of resources deployed on the device; R0 represents the resource consumption of the first inference; n represents the total number of model layers in the inference model; x j The layer input vector representing the j-th model layer; x j+1 The layer input vector representing the j+1th model layer.

[0105] Then, we can select the top k from the model layer sequence. i model layers and convert the k i The model layers are used as inference layers, and then the layer index corresponding to each inference layer in the inference model is obtained, the layer index is stored as a preset index set, and the preset index set and the preset task identifier are associated and stored.

[0106] S130: Using the task identifier, obtain a task matrix corresponding to the data to be processed.

[0107] Based on the task identifier of the data to be processed, the task matrix corresponding to the task identifier can be obtained, and the task matrix contains the layer task matrix associated with the preset index. Similarly, a second association relationship between the preset task identifier and the preset task matrix can be established in advance, and each preset task matrix contains the layer task matrix associated with the preset index. Based on the second association relationship, the task identifier is used to find the corresponding preset task matrix, which can be used as the task matrix of the data to be processed.

[0108] Optionally, after establishing the first association relationship between the preset task identifier and the preset index set, in order to further improve the performance of the reasoning model under a specific task, the second task data set may be used to fine-tune the reasoning layer and save related parameters.

[0109] Based on the aforementioned first association relationship between the preset task identifier and the preset index set, the layer index of the reasoning layer can be obtained for a specific task identifier. During fine-tuning, for each reasoning layer, the initial dimension reduction matrix and the initial dimension increase matrix can be inserted into the initial parameter matrix corresponding to the reasoning layer; the reasoning model can be fine-tuned using the second task data set corresponding to the preset task identifier, and the initial dimension reduction matrix and the initial dimension increase matrix are gradient updated to obtain the layer dimension reduction matrix and the layer dimension increase matrix corresponding to the reasoning layer; the layer index corresponding to the reasoning layer is used as the preset index, and the layer dimension reduction matrix and the layer dimension increase matrix are used as the layer task matrix; the layer task matrices corresponding to all preset indexes are used as the preset task matrix, and an association relationship between the preset task matrix and the preset task identifier is established.

[0110] The initial parameter matrix is ​​the parameter matrix of each model layer in the inference model before fine-tuning. The initial dimension reduction matrix and the initial dimension increase matrix are a set of learnable parameter matrices. If the initial parameter matrix is dimension, then the initial dimension reduction matrix is The initial dimension-enhanced matrix is dimensional, where r is much smaller than the minimum value of d and k. The initial dimension reduction matrix is ​​initialized with a random Gaussian distribution, and the initial dimension increase matrix is ​​initially a zero matrix. After inserting the initial dimension reduction matrix and the initial dimension increase matrix, the forward propagation process can be changed from y = W0x to . Where x is the d-dimensional input vector, y is the k-dimensional output vector, W0 is the initial parameter matrix, A is the initial dimension reduction matrix, and B is the initial dimension increase matrix.

[0111] The second task data set corresponding to the preset task identifier refers to the training data set under the task scenario. The difference from the first task data set is that the second task data set has a larger data volume. During the fine-tuning process, each initial parameter matrix in the inference model needs to be frozen, that is, set to be non-differentiable, and then the second task data set is used to train the inference model, calculate the gradient of each set of initial dimensionality reduction matrix and initial dimensionality increase matrix, and update the initial dimensionality reduction matrix and initial dimensionality increase matrix according to the gradient until the convergence condition is met. The training is completed, and the layer dimensionality reduction matrix and layer dimensionality increase matrix corresponding to each inference layer can be obtained.

[0112] The layer dimension reduction matrix and the layer dimension increase matrix can be recorded as a layer task matrix. The layer task matrix corresponds to the reasoning layer. The layer index of the reasoning layer can be used as a preset index to establish an association relationship between the preset index and the layer task matrix, i.e., a second association relationship. Since there are multiple reasoning layers, the association relationship between all preset indexes and the layer task matrix can be used as a preset task matrix, and a second association relationship can be established and stored between the preset task matrix and the preset task identifier.

[0113] Therefore, after obtaining the task identifier of the data to be processed, the preset task matrix corresponding to the task identifier can be found as the task matrix of the data to be processed, and the task matrix includes the layer task matrices corresponding to each preset index.

[0114] S140: transform the data to be processed using the layer task matrices corresponding to all the reasoning layers in the reasoning model to obtain the reasoning results of the data to be processed.

[0115] The inference layer refers to the model layer in the inference model used to process the data to be processed. The layer index of the inference layer in the inference model is in the preset index set. That is, in the inference model, the model layer with the preset index is the inference layer. Since the preset index is associated with the layer task matrix, the layer task matrix of the corresponding inference layer can be obtained.

[0116] The data to be processed is input into the inference model, and after being processed by all model layers in the inference model, the inference result can be obtained. That is, the layer task matrix corresponding to all inference layers can be used to transform the data to be processed to obtain the inference result.

[0117] As an implementation mode, the data to be processed is transformed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed. For each model layer in the inference model, if the layer input data corresponding to the model layer is received, determine whether the model layer is an inference layer according to the layer index of the model layer in the inference model and the index set, and the layer input data is related to the data to be processed; if the model layer is an inference layer, transform the layer input data into layer output data through the layer task matrix corresponding to the inference layer, and pass the layer output data to the next model layer of the model layer; if the model layer is not an inference layer, use the layer input data as layer output data, and pass the layer output data to the next model layer of the model layer; use the layer output data of the last model layer in the inference model as the inference result of the data to be processed.

[0118] The data to be processed is input into the inference model, and the data to be processed can pass through each model layer in each inference model in turn to realize the inference of the data to be processed using the inference layer. Figure 4 , shows a flow chart of reasoning about data to be processed. For each model layer, if the model layer receives corresponding layer input data, it can determine what operation to perform on the layer input data based on the layer index and index set of the model layer.

[0119] Layer input data refers to the data received by the model layer, which is obtained by transforming the data to be processed. When the model layer receives the layer input data, it can determine whether the layer index of the model layer in the inference model is in the index set to determine whether the model layer is an inference layer.

[0120] If the layer index of the model layer in the inference model is in the index set, it can be determined that the model layer is an inference layer; if the layer index of the model layer in the inference model is not in the index set, it can be determined that the model layer is not an inference layer.

[0121] If the model layer is an inference layer, the model layer needs to process data. The layer input data can be transformed into layer output data through the layer task matrix corresponding to the inference layer. Then, the layer output data can be passed to the next model layer as the layer input data of the next model layer. If the model layer is not an inference layer, the model layer does not need to process data. The layer input data can be directly used as layer output data and passed to the next model layer as the layer input data of the next model layer. In this case, the layer input data of the next model layer is consistent with the layer input data of the model layer. For example, Figure 4In the example, data 1 is the input data of model layer 1; if 1 is in the index set, data 1 can be processed by model layer 1 to obtain data 2, which is used as the input data of model layer 2; if 1 is not in the index set, data 1 is directly passed to model layer 2, then data 1 and data 2 are the same.

[0122] In this way, by traversing each model layer, the layer output data of the last model layer in the inference model can be obtained, and the layer output data is the inference result of the data to be processed.

[0123] A task selection mechanism can be introduced before the embedding layer in the reasoning model, and the core components include the first association relationship and a message sending program. When a task identifier is received, an index set can be obtained based on the first association relationship, and the index set can be sent to each model layer. A routing mechanism can be introduced before each model layer, and the core components include a message receiving program and a gating program. The message receiving program can be used to receive an index set, and the gating program is used to determine whether the index of the model layer is in the index set; if so, the layer input data is routed to the model layer; if not, the layer input data is routed to the next model layer.

[0124] As another implementation, the data to be processed is transformed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed. The model layer in the inference model whose layer index is consistent with the preset index can be determined as the inference layer; for each of the inference layers, the layer input data corresponding to the inference layer is obtained, and the layer input data is related to the data to be processed; the layer input data is transformed into layer output data through the layer task matrix corresponding to the inference layer, and the layer output data is passed to the next inference layer of the inference layer; and the layer output data of the last inference layer in the inference model is used as the inference result of the data to be processed.

[0125] All inference layers can be identified from multiple model layers in the inference model based on the index set. For example, see Figure 5 , showing another flow diagram of reasoning the data to be processed. Among them, the layer indexes 1, 3, 4, and n are all reasoning layers. After reasoning layer 1 processes data 1, the layer output data is directly passed to reasoning layer 3 for processing, and will not be passed to reasoning layer 2. Specifically, the layer index of each model layer in the reasoning model can be obtained first, and the model layer whose layer index is consistent with the preset index in the index set is determined as the reasoning layer.

[0126] For each inference layer, the layer input data corresponding to the inference layer can be obtained, and the layer input data can be obtained by transforming the data to be processed. Using the layer task matrix of the inference layer, the layer input data of the inference layer can be transformed into layer output data, and the layer output data is passed to the next inference layer of the inference layer, that is, the layer input data of the next inference layer is the layer output data of the inference layer. Finally, the layer output data of the last inference layer in the inference model can be obtained as the inference result of the data to be processed. For example Figure 5 In the example, it is assumed that the preset indexes are 1, 3, 4, and n, and data 1 is the layer input data of model layer 1.

[0127] Among them, when using the layer task matrix of the inference layer to transform the layer input data into the layer output data, it can be to obtain the initial parameter matrix of the inference layer; calculate the product of the layer dimensionality increase matrix and the layer dimensionality reduction matrix to obtain the intermediate parameter matrix; sum the initial parameter matrix and the intermediate parameter matrix to obtain the inference parameter matrix; and calculate and process the layer input data with the inference parameter matrix to obtain the layer output data.

[0128] Among them, the layer task matrix contains the layer dimension increase matrix and the layer dimension reduction matrix. After the inference model is trained, each model layer has its corresponding initial parameter matrix, so the initial parameter matrix of the inference layer can be obtained. The product of the layer dimension increase matrix and the layer dimension reduction matrix is ​​used as the intermediate parameter matrix, and then the intermediate parameter matrix and the initial parameter matrix are added to obtain the inference parameter matrix. Finally, the layer input data is calculated and processed using the inference parameter matrix to obtain the layer output data.

[0129] When the task identifier has not changed, but there is new data input to be processed, the above S140 can be repeated to continue the reasoning. If the task identifier has changed, it is necessary to re-acquire the index set corresponding to the new task identifier to continue the subsequent process. Among them, since the task matrix corresponding to the previous task identifier has been loaded, it is necessary to unload the task matrix first, and then load the task matrix corresponding to the new task identifier. For each reasoning layer, the process of loading the task matrix is ​​to add the product of the layer dimensionality reduction matrix and the layer dimensionality increase matrix to the initial parameter matrix, and the process of unloading is to subtract the product of the layer dimensionality reduction matrix and the layer dimensionality increase matrix on the basis of loading.

[0130] The model layer and the reasoning layer in the embodiment of the present invention are both transformer layers. In the transformer architecture, the sub-attention layer captures the dependencies between elements by calculating the attention score of each element in the layer input data, which involves a large number of matrix multiplication operations. In the text generation task, the model will generate each token of the text one by one. In this process, the key and value vectors of the processed tokens can be stored, that is, the KV cache, so that they can be reused when generating subsequent tokens to avoid repeated calculations. In the embodiment of the present invention, each token is processed using the same reasoning layer, which can ensure the effectiveness of the KV cache, reduce repeated calculations, and improve reasoning efficiency.

[0131] The task-based model reasoning solution provided by the embodiment of the present invention can be applied to various scenarios where large models are used for reasoning. For example, taking health question and answer as an example, the solution provided by the embodiment of the present invention can adapt to the health question and answer task, and only activates the corresponding model layer from the reasoning model without using all the model layers, which can effectively reduce the reasoning cost and improve the reasoning efficiency.

[0132] The method provided by the embodiment of the present invention can be used to infer the task data set through the inference model in advance to calculate the importance parameters of each model layer of the inference model layer, and then determine the number of model layers that need to be activated based on the computational cost of the inference and the difficulty of the task, and use the importance parameters to determine the model layer used for the inference to obtain the inference layer. When the reasoning is actually applied, the corresponding inference layer inference data can be selected based on the task identifier, which can effectively improve the reasoning efficiency. Since the reasoning layer is determined based on the importance parameters, the reasoning performance can also be ensured, so that it can be adapted to a variety of different task scenarios and can also take into account the reasoning performance while improving the reasoning efficiency.

[0133] In order to better implement the above method, the embodiment of the present invention also provides a task-based model reasoning device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0134] For example, in this embodiment, the method of the embodiment of the present invention is described in detail by taking the task-based model reasoning device specifically integrated in the server as an example.

[0135] For example, Figure 6 As shown, the task-based model reasoning device 200 may include a data acquisition module 210 , an index acquisition module 220 , a matrix acquisition module 230 and a reasoning module 240 .

[0136] The data acquisition module 210 is used to acquire the data to be processed and the task identifier corresponding to the data to be processed;

[0137] An index acquisition module 220, configured to acquire an index set corresponding to the data to be processed by using the task identifier, wherein the index set includes a plurality of preset indexes;

[0138] A matrix acquisition module 230, used to acquire a task matrix corresponding to the data to be processed by using the task identifier, wherein the task matrix includes a layer task matrix associated with the preset index;

[0139] The reasoning module 240 is used to transform the data to be processed using the layer task matrix corresponding to all the reasoning layers in the reasoning model to obtain the reasoning result of the data to be processed, and the layer index of the reasoning layer in the reasoning model is the preset index.

[0140] In some embodiments, the reasoning module 240 is specifically used to:

[0141] For each model layer in the inference model, if layer input data corresponding to the model layer is received, determine whether the model layer is an inference layer according to the layer index of the model layer in the inference model and the index set, and the layer input data is related to the data to be processed;

[0142] If the model layer is an inference layer, the layer input data is transformed into layer output data through a layer task matrix corresponding to the inference layer, and the layer output data is passed to the next model layer of the model layer;

[0143] If the model layer is not an inference layer, using the layer input data as layer output data, and passing the layer output data to the next model layer of the model layer;

[0144] The layer output data of the last model layer in the inference model is used as the inference result of the data to be processed.

[0145] In some embodiments, the reasoning module 240 is specifically used to:

[0146] Determine, in the inference model, a model layer whose layer index is consistent with the preset index as the inference layer;

[0147] For each of the inference layers, if layer input data corresponding to the inference layer is obtained, the layer input data is related to the data to be processed;

[0148] Transforming the layer input data into layer output data through the layer task matrix corresponding to the reasoning layer, and passing the layer output data to the next reasoning layer of the reasoning layer;

[0149] The layer output data of the last inference layer in the inference model is used as the inference result of the data to be processed.

[0150] In some embodiments, the layer task matrix includes a layer dimension increase matrix and a layer dimension reduction matrix, and the reasoning module 240 is specifically used for:

[0151] Obtaining an initial parameter matrix of the inference layer;

[0152] Calculate the product of the layer dimension increase matrix and the layer dimension reduction matrix to obtain an intermediate parameter matrix;

[0153] Summing the initial parameter matrix and the intermediate parameter matrix to obtain an inference parameter matrix;

[0154] The layer input data is calculated and processed using the inference parameter matrix to obtain layer output data.

[0155] In some embodiments, the task-based model reasoning device 200 further includes a preparation module. Before obtaining the index set corresponding to the data to be processed using the task identifier, the preparation module is specifically used to:

[0156] Obtaining a first task data set corresponding to a preset task identifier;

[0157] Performing reasoning on the first task data set using the reasoning model to obtain specified performance parameters of the reasoning model and a layer input vector of each model layer;

[0158] Calculate the importance parameter of each model layer under the first task data set using the specified performance parameter and the layer input vector of each model layer;

[0159] Calculate the number of targets based on the layer input vectors and inference resource consumption of each model layer;

[0160] The target number of inference layers is determined from the inference model according to the importance parameter, and the layer indexes of all the inference layers are used as a preset index set corresponding to the preset task identifier.

[0161] In some embodiments, the preparation module is specifically configured to:

[0162] Performing a first inference on the first task data set using all model layers in the inference model to obtain a baseline performance parameter of the inference model and a layer input vector of each model layer;

[0163] For each of the model layers in the reasoning model, use other model layers except the model layer to perform second reasoning on the first task data set to obtain layer performance parameters corresponding to the model layer;

[0164] The baseline performance parameters and the layer performance parameters are used as designated performance parameters of the inference model.

[0165] In some embodiments, the preparation module is specifically configured to:

[0166] Calculating the relative weight of the model layer using the initial parameter matrix corresponding to the model layer;

[0167] Calculating a performance change parameter of the model layer using the baseline performance parameter and the layer performance parameter;

[0168] Calculating the feature similarity of the model layer using the layer input vector of the model layer and the layer input vector of the next model layer;

[0169] The importance parameter corresponding to the model layer is obtained by subtracting the feature similarity from the sum of the relative weight and the performance change parameter.

[0170] In some embodiments, the preparation module is specifically configured to:

[0171] Obtaining resource consumption of the first inference and total resources deployed;

[0172] Calculating the ratio of the resource consumption to the total resource amount to obtain a resource consumption ratio;

[0173] Calculating the task processing difficulty corresponding to the preset task identifier using the layer input vector of each model layer;

[0174] The target quantity is calculated according to the resource consumption ratio and the task processing difficulty.

[0175] In some embodiments, after determining a target number of inference layers from multiple model layers in the inference model according to the importance parameter and using the index of the inference layer as a preset index set corresponding to the preset task identifier, the preparation module is specifically used to:

[0176] For each inference layer, insert the initial dimension reduction matrix and the initial dimension increase matrix into the initial parameter matrix corresponding to the inference layer;

[0177] Fine-tune the inference model using a second task data set corresponding to a preset task identifier, and perform gradient update on the initial dimension reduction matrix and the initial dimension increase matrix to obtain a layer dimension reduction matrix and a layer dimension increase matrix corresponding to the inference layer;

[0178] The layer index corresponding to the reasoning layer is used as a preset index, and the layer dimension reduction matrix and the layer dimension increase matrix are used as layer task matrices;

[0179] The layer task matrices corresponding to all preset indexes are used as preset task matrices, and an association relationship between the preset task matrix and the preset task identifier is established.

[0180] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0181] As can be seen from the above, the task-based model reasoning device of this embodiment can obtain the data to be processed and the task identifier corresponding to the data to be processed, and use the task identifier to obtain the index set and task matrix of the data to be processed, wherein the index set contains multiple preset indexes, and the task matrix contains the layer task matrix associated with each preset index. Both the preset index and the layer task matrix are related to the task identifier, which can ensure the reasoning performance in different task scenarios. The layer task matrix of the reasoning layer with the preset index is used to process the data to be processed, without using all model layers for processing, which can effectively reduce unnecessary calculations and improve reasoning efficiency.

[0182] The embodiment of the present invention further provides an electronic device, which may be a terminal, a server, or the like. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, or the like; the server may be a single server or a server cluster composed of multiple servers, or the like.

[0183] In some embodiments, the task-based model reasoning device can also be integrated into multiple electronic devices. For example, the task-based model reasoning device can be integrated into multiple servers, and the task-based model reasoning method of the present invention can be implemented by multiple servers.

[0184] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 7 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0185] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0186] The processor 310 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 310.

[0187] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0188] The electronic device also includes a power supply 330 for supplying power to various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 330 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0189] The electronic device may further include an input module 340, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0190] The electronic device may further include a communication module 350. In some embodiments, the communication module 350 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0191] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 310 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 320 according to the following instructions, and the processor 310 will run the application programs stored in the memory 320, thereby implementing the steps in the methods of the embodiments of the present invention.

[0192] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0193] As can be seen from the above, the electronic device provided by the embodiment of the present invention can obtain the data to be processed and the task identifier corresponding to the data to be processed, and use the task identifier to obtain the index set and task matrix of the data to be processed, wherein the index set contains multiple preset indexes, and the task matrix contains the layer task matrix associated with each preset index. Both the preset index and the layer task matrix are related to the task identifier, which can ensure the reasoning performance in different task scenarios. The layer task matrix of the reasoning layer with the preset index is used to process the data to be processed, without using all model layers for processing, which can effectively reduce unnecessary calculations and improve reasoning efficiency.

[0194] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0195] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which multiple instructions are stored. The instructions can be loaded by a processor to execute the steps in any one of the task-based model reasoning methods provided by the embodiment of the present invention.

[0196] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0197] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations of the preparation aspect or the application reasoning aspect provided in the above-mentioned embodiments.

[0198] Since the instructions stored in the storage medium can execute the steps in any one of the task-based model reasoning methods provided in the embodiments of the present invention, the beneficial effects that can be achieved by any one of the task-based model reasoning methods provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0199] The above is a detailed introduction to a task-based model reasoning method and device provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A task-based model reasoning method, characterized in that: The method comprises: Acquire the data to be processed and the task identifier corresponding to the data to be processed; Using the task identifier, obtaining an index set corresponding to the data to be processed, wherein the index set includes a plurality of preset indexes; Using the task identifier, obtaining a task matrix corresponding to the data to be processed, wherein the task matrix includes a layer task matrix associated with the preset index; The data to be processed is transformed using the layer task matrices corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed, and the layer index of the inference layer in the inference model is the preset index.

2. The method according to claim 1, characterized in that The transforming process of the data to be processed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed includes: For each model layer in the inference model, if layer input data corresponding to the model layer is received, determine whether the model layer is an inference layer according to the layer index of the model layer in the inference model and the index set, and the layer input data is related to the data to be processed; If the model layer is an inference layer, the layer input data is transformed into layer output data through a layer task matrix corresponding to the inference layer, and the layer output data is passed to the next model layer of the model layer; If the model layer is not an inference layer, using the layer input data as layer output data, and passing the layer output data to the next model layer of the model layer; The layer output data of the last model layer in the inference model is used as the inference result of the data to be processed.

3. The method according to claim 1, characterized in that The transforming process of the data to be processed using the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed includes: Determine, in the inference model, a model layer whose layer index is consistent with the preset index as the inference layer; For each of the inference layers, if layer input data corresponding to the inference layer is obtained, the layer input data is related to the data to be processed; Transforming the layer input data into layer output data through the layer task matrix corresponding to the reasoning layer, and passing the layer output data to the next reasoning layer of the reasoning layer; The layer output data of the last inference layer in the inference model is used as the inference result of the data to be processed.

4. The method according to claim 2 or 3, characterized in that: The layer task matrix includes a layer dimension increase matrix and a layer dimension reduction matrix, and the layer input data is transformed into layer output data by using the layer task matrix corresponding to the reasoning layer, including: Obtaining an initial parameter matrix of the inference layer; Calculate the product of the layer dimension increase matrix and the layer dimension reduction matrix to obtain an intermediate parameter matrix; Summing the initial parameter matrix and the intermediate parameter matrix to obtain an inference parameter matrix; The layer input data is calculated and processed using the inference parameter matrix to obtain layer output data.

5. The method according to claim 1, characterized in that Before using the task identifier to obtain the index set corresponding to the data to be processed, the method further includes: Obtaining a first task data set corresponding to a preset task identifier; Performing reasoning on the first task data set using the reasoning model to obtain specified performance parameters of the reasoning model and a layer input vector of each model layer; Calculate the importance parameter of each model layer under the first task data set using the specified performance parameter and the layer input vector of each model layer; Calculate the number of targets based on the layer input vectors and inference resource consumption of each model layer; The target number of inference layers is determined from the inference model according to the importance parameter, and the layer indexes of all the inference layers are used as a preset index set corresponding to the preset task identifier.

6. The method according to claim 5, characterized in that The step of performing reasoning on the first task data set using the reasoning model to obtain specified performance parameters of the reasoning model and a layer input vector of each model layer includes: Performing a first inference on the first task data set using all model layers in the inference model to obtain a baseline performance parameter of the inference model and a layer input vector of each model layer; For each of the model layers in the reasoning model, use other model layers except the model layer to perform second reasoning on the first task data set to obtain layer performance parameters corresponding to the model layer; The baseline performance parameters and the layer performance parameters are used as designated performance parameters of the inference model.

7. The method according to claim 6, characterized in that The step of calculating the importance parameter of each model layer under the first task data set by using the specified performance parameter and the layer input vector of each model layer includes: Calculating the relative weight of the model layer using the initial parameter matrix corresponding to the model layer; Calculating a performance change parameter of the model layer using the baseline performance parameter and the layer performance parameter; Calculating the feature similarity of the model layer using the layer input vector of the model layer and the layer input vector of the next model layer; The importance parameter corresponding to the model layer is obtained by subtracting the feature similarity from the sum of the relative weight and the performance change parameter.

8. The method according to claim 6, characterized in that The target quantity is calculated based on the layer input vector of each model layer and the inference resource consumption, including: Obtaining resource consumption of the first inference and total resources deployed; Calculating the ratio of the resource consumption to the total resource amount to obtain a resource consumption ratio; Calculating the task processing difficulty corresponding to the preset task identifier using the layer input vector of each model layer; The target quantity is calculated according to the resource consumption ratio and the task processing difficulty.

9. The method according to claim 5, characterized in that After determining a target number of inference layers from a plurality of model layers in the inference model according to the importance parameter, and using the index of the inference layer as a preset index set corresponding to the preset task identifier, the method further includes: For each inference layer, insert the initial dimension reduction matrix and the initial dimension increase matrix into the initial parameter matrix corresponding to the inference layer; Fine-tune the inference model using a second task data set corresponding to a preset task identifier, and perform gradient update on the initial dimension reduction matrix and the initial dimension increase matrix to obtain a layer dimension reduction matrix and a layer dimension increase matrix corresponding to the inference layer; The layer index corresponding to the reasoning layer is used as a preset index, and the layer dimension reduction matrix and the layer dimension increase matrix are used as layer task matrices; The layer task matrices corresponding to all preset indexes are used as preset task matrices, and an association relationship between the preset task matrix and the preset task identifier is established.

10. A task-based model reasoning device, the device being used to implement the method according to any one of claims 1 to 9, characterized in that: The device comprises: A data acquisition module, used to acquire the data to be processed and the task identifier corresponding to the data to be processed; An index acquisition module, used to acquire an index set corresponding to the data to be processed by using the task identifier, wherein the index set includes a plurality of preset indexes; A matrix acquisition module, used to acquire a task matrix corresponding to the data to be processed by using the task identifier, wherein the task matrix includes a layer task matrix associated with the preset index; The inference module is used to transform the data to be processed with the layer task matrix corresponding to all the inference layers in the inference model to obtain the inference result of the data to be processed, and the layer index of the inference layer in the inference model is the preset index.

Images