Task processing method and device, electronic equipment and storage medium
By dynamically selecting and optimizing the weight calculation method of the LORA module in the large language model, the problem of low computational efficiency of low-rank adaptation in the inference stage is solved, improving computational efficiency and GPU memory resource utilization, and enhancing user experience.
Patent Information
- Application Number
- CN202511260660.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-04
- Publication Date
- 2025-12-30
AI Technical Summary
Low-rank adaptation (LORA) is computationally inefficient during the inference phase of large language models, leading to increased computational complexity and resource requirements, which negatively impacts user experience.
By dynamically selecting target LORA modules for fine-tuning, and using the product of the first LORA weight and the second LORA weight to optimize the calculation method, the amount of computation is reduced. Furthermore, the utilization of video memory resources is optimized through dynamic allocation of task level and video memory region.
It improves computing efficiency and inference speed, enhances user experience, and optimizes the utilization of video memory resources.
Smart Images
Figure CN121233249A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of large language model technology, and includes, but is not limited to, a task processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] Low-Rank Adaptation (LORA), as an efficient parameter fine-tuning method, is widely used in multi-task adaptation of large language models. This method approximates parameter updates by introducing a trainable low-rank matrix, reducing the number of parameters that need to be trained while keeping the parameters of the pre-trained base model frozen. However, this method suffers from significant computational efficiency bottlenecks, especially during the inference phase. The multiplication operations of the low-rank matrix introduce additional computational burdens, leading to increased computational complexity and resource requirements, thereby reducing the system's response speed and impacting user experience. Therefore, improving computational efficiency has become a pressing technical problem to be solved. Summary of the Invention
[0003] In view of this, embodiments of this application provide a task processing method, apparatus, electronic device, and storage medium, which can optimize the calculation method of low-rank matrices in the model inference stage and improve computational efficiency.
[0004] In a first aspect, embodiments of this application provide a task processing method, the method comprising:
[0005] Based on the target task type corresponding to the user-input text to be processed, the target LORA module corresponding to the target task type is determined. Different LORA modules correspond to different task types. The target LORA module is used to fine-tune at least one target layer of each transport layer of the basic large language model. The weight of each target layer includes the basic weight of each target layer and the weight of the target LORA module. The weight of the target LORA module includes the first LORA weight and the second LORA weight. The basic large language model also includes an input layer and an output layer.
[0006] The input layer is used to process the text to be processed to obtain text features;
[0007] Based on the text features, at least one transport layer is processed to obtain a transport layer result. The input of the first transport layer is the text features, and the output of the last transport layer is the processing result of the text to be processed. The output of any transport layer other than the last transport layer is the input of the next transport layer. The output result of each target layer of each transport layer includes a variable result and a basic result. The basic result is obtained based on the input parameters of each target layer and the basic weights of each target layer. The variable result is obtained based on the product of the input parameters of each target layer and the first LORA weight and the second LORA weight.
[0008] The output layer is used to process the results of the transport layer to obtain the processing result of the text to be processed.
[0009] In the above embodiments, by dynamically selecting and loading the corresponding target LORA module based on the target task type corresponding to the user-input text to be processed, the target layer in the basic large language model is fine-tuned. This enables support for multiple task processing on a single basic large language model, significantly reducing the GPU memory overhead required to deploy independent models for different tasks and improving the system's resource utilization. It can be seen that the above embodiments configure a corresponding LORA module for each task type. For a target task type, a target LORA module can be determined, and the weights of the target LORA module include a first LORA weight and a second LORA weight. Therefore, the corresponding low-rank matrix (the weights of the LORA module) can be called according to the task type, enhancing the basic large language model's processing capability for specific task types. Furthermore, during the inference process, firstly, the input layer processes the text to be processed to obtain text features, and then the transport layer processes the text layer by layer to obtain the transport layer results. The output of each target layer in each transport layer includes variable results and basic results. The variable results are obtained by multiplying the input parameters and the first LORA weights by the second LORA weights. Specifically, the product of the input parameters and the first LORA weights is calculated first, and then multiplied by the second LORA weights. This optimized calculation method for variable results during the inference phase effectively reduces the computational load, thereby improving inference speed. Finally, the output layer processes the transport layer results to obtain the processed text. The above embodiment improves computational efficiency and inference speed by optimizing the calculation method of the weights (low-rank matrix) of the LORA module during the model inference phase, thus enhancing the user experience.
[0010] In some embodiments, the first LORA weight is an up-projection matrix, and the second LORA weight is a down-projection matrix.
[0011] In the above embodiments, the first LORA weights serve as the uplink projection matrix, and the second LORA weights serve as the downlink projection matrix, together constituting an efficient parameter update mechanism. Specifically, the downlink projection matrix maps the input feature vectors to a low-dimensional space, achieving effective compression of the feature representation; the uplink projection matrix reprojects the feature vectors in the low-dimensional space back to the original dimension. By constraining parameter updates to a low-dimensional subspace, the learning enables the basic large language model to adapt to the minimum and most effective feature transformations of the target task, significantly reducing the number of trainable parameters and computational complexity, and improving computational efficiency.
[0012] In some embodiments, before processing the text to be processed using the input layer to obtain text features, the method further includes:
[0013] Based on the parameter information of the target task type, the task level of the target task type is determined. The parameter information includes at least one of call frequency, latency requirement, and video memory requirement. The call frequency is used to characterize the number of calls of the target task type. The latency requirement is used to characterize the response latency of the target task type. The video memory requirement is used to characterize the video memory capacity occupied by the target LORA module corresponding to the target task type. The task level includes a first level and other levels lower than the first level.
[0014] Based on the task level of the target task type, the target video memory area corresponding to the target LORA module is determined. The target video memory area includes a dedicated video memory area or a shared video memory area. The dedicated video memory area is used to store the LORA module corresponding to the first level of task type, and the shared video memory area is used to store the LORA module corresponding to the other levels of task type.
[0015] If the target LORA module does not exist in the target video memory region, the target LORA module is loaded into the target video memory region.
[0016] In the above embodiments, a task-level-based dynamic memory allocation mechanism improves memory resource utilization efficiency and task response performance in multi-LORA module scenarios. Specifically, based on the parameter information of the target task type (including at least one of call frequency, latency requirements, and memory requirements), the task level of the target task type is determined, enabling a quantitative assessment of the resource requirements of different task types and providing a reliable basis for subsequent differentiated memory allocation strategies. Then, based on the task level of the target task type, a corresponding target memory region is allocated to the target LORA module. LORA modules with higher task levels are allocated to a dedicated memory region to ensure their high-frequency call and low-latency response requirements; while LORA modules with other task levels are allocated to a shared memory region, improving memory resource reuse and reducing redundant occupation. Finally, if no target LORA module exists in the target memory region, the target LORA module is loaded into the target memory region. On-demand loading effectively avoids unnecessary memory pre-occupancy and performance fluctuations caused by frequent I / O operations.
[0017] In some embodiments, determining the task level of the target task type based on parameter information of the target task type includes:
[0018] The priority value of the target task type is determined based on the parameter information of the target task type;
[0019] If the priority value of the target task type is greater than or equal to the first threshold, the task level of the target task type is determined as the first level;
[0020] If the priority value corresponding to the target task type is less than the first threshold, the task level of the target task type is determined to be one of the other levels.
[0021] In the above embodiments, priority values are calculated based on parameter information of task types, and then the priority values are compared with preset thresholds to classify task types into first level or other levels. By further refining the calculation method for determining task levels based on priority values, a more accurate quantitative assessment of resource requirements for different task types is achieved, providing a reliable basis for subsequent differentiated memory allocation strategies.
[0022] In some embodiments, the other levels include a second level and a third level that is lower than the second level;
[0023] If the priority value corresponding to the target task type is less than the first threshold, the task level of the target task type is determined to be one of the other levels, including:
[0024] If the priority value corresponding to the target task type is less than the first threshold and greater than or equal to the second threshold, the task level of the target task type is determined to be the second level.
[0025] If the priority value corresponding to the target task type is less than the second threshold, the task level of the target task type is determined as the third level.
[0026] In the above embodiments, other levels are further subdivided into second and third levels based on priority values, which realizes a more refined division of task levels. This allows for a more precise quantification of the differences in resource requirements for different task types, providing a more reliable basis for subsequent differentiated memory allocation strategies.
[0027] In some embodiments, the capacity of the dedicated video memory region accounts for a first proportion of the total video memory capacity, the capacity of the shared video memory region used to store the LoRA modules corresponding to the second level of task types accounts for a second proportion of the total video memory capacity, and the capacity of the shared video memory region used to store the LoRA modules corresponding to the third level of task types accounts for a third proportion of the total video memory capacity, wherein the first proportion is greater than the second proportion, and the second proportion is greater than the third proportion.
[0028] In the above embodiments, memory capacity is allocated to LORA modules of different task levels according to differentiated ratios, which realizes the matching of memory resources with task levels. This ensures that LORA modules with higher task levels obtain more exclusive memory resources. At the same time, more memory resources are reserved for second-level LORA modules in the shared memory area than for third-level LORA modules, avoiding excessive memory occupation by third-level LORA modules, thereby improving the overall utilization rate of memory resources.
[0029] In some embodiments, loading the target LoRa module into the target video memory region when the target LoRa module is not present in the target video memory region includes:
[0030] If the target LORA module is not present in the target video memory region and the occupancy rate of the target video memory region is higher than a preset threshold, remove the currently unused portion of the LORA modules stored in the target video memory region.
[0031] The target LORA module is loaded into the target video memory area.
[0032] In the above embodiments, when there is no target LORA module in the target video memory region and the occupancy rate of the target video memory region is too high, the currently unused LORA modules are removed first, and then the target LORA module is loaded. This solves the loading conflict problem when video memory resources are scarce, avoids the target LORA module loading failure due to insufficient video memory space, and enables the target LORA module to be loaded normally into the target video memory region to support subsequent task inference, thereby improving the flexibility and reliability of dynamic scheduling of video memory resources.
[0033] Secondly, embodiments of this application provide a task processing apparatus, the apparatus comprising:
[0034] Determination Module: Based on the target task type corresponding to the user-input text to be processed, determine the target LORA module corresponding to the target task type. Different LORA modules correspond to different task types. The target LORA module is used to fine-tune at least one target layer of each transport layer of the basic large language model. The weight of each target layer includes the basic weight of each target layer and the weight of the target LORA module. The weight of the target LORA module includes the first LORA weight and the second LORA weight. The basic large language model also includes an input layer and an output layer.
[0035] First processing module: used to process the text to be processed using the input layer to obtain text features;
[0036] The second processing module is used to process the at least one transport layer based on the text features to obtain the transport layer result. The input of the first transport layer is the text features, and the output of the last transport layer is the processing result of the text to be processed. The output of any transport layer other than the last transport layer is the input of the next transport layer. The output result of each target layer of each transport layer includes variable results and basic results. The basic results are obtained based on the input parameters of each target layer and the basic weights of each target layer. The variable results are obtained based on the product of the input parameters of each target layer and the first LORA weight and the second LORA weight.
[0037] The third processing module is used to process the results of the transmission layer using the output layer to obtain the processing result of the text to be processed.
[0038] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the program to implement the method described in embodiments of this application.
[0039] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in embodiments of this application. Attached Figure Description
[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.
[0041] Figure 1 This is a flowchart illustrating a task processing method disclosed in an embodiment of this application;
[0042] Figure 2 This is a flowchart illustrating a process for determining the target task type corresponding to user-inputted text in an embodiment of this application.
[0043] Figure 3 This is a flowchart illustrating the calculation of variable results disclosed in an embodiment of this application;
[0044] Figure 4 This is a schematic diagram of a process for calculating the output result of a target layer, as disclosed in an embodiment of this application.
[0045] Figure 5 This is a flowchart illustrating a method for determining the target memory region corresponding to a target LORA module, as disclosed in an embodiment of this application.
[0046] Figure 6 This is a flowchart illustrating a method for determining the task level of a target task type, as disclosed in an embodiment of this application.
[0047] Figure 7 This is a schematic flowchart of a method for loading a target LORA module disclosed in an embodiment of this application;
[0048] Figure 8 This is a schematic diagram of the structure of a task processing device disclosed in an embodiment of this application;
[0049] Figure 9 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.
[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0052] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0053] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0054] With the widespread adoption of Large Language Models (LLMs) in natural language processing, from generative pre-trained transformers (GPTs) with hundreds of billions of parameters to open-source Large Language Model MetaAI (LLaMA), these models, while accumulating massive amounts of general knowledge during pre-training, still require "fine-tuning" to adapt to specific task scenarios (such as customer service dialogues, industry document question answering, and sentiment analysis). Traditional full-parameter fine-tuning methods retrain and adjust all model parameters to suit specific task requirements. However, this approach faces significant limitations. For LLMs with tens of billions of parameters, a full parameter update not only requires terabytes of GPU memory but also consumes substantial computational resources. Furthermore, different tasks require separate storage of the complete fine-tuned model, resulting in extremely high deployment and maintenance costs.
[0055] Low-Rank Adaptation (LORA), as an efficient parameter fine-tuning method, is widely used in multi-task adaptation of large language models. This method approximates parameter updates by introducing trainable low-rank matrices, reducing the number of parameters that need to be trained while keeping the parameters of the pre-trained base model frozen, thus significantly reducing memory usage during the training phase. However, LORA's efficiency is mainly reflected in the training phase, while it faces significant computational efficiency bottlenecks during inference. The multiplication operations of low-rank matrices introduce additional computational burdens, leading to increased computational complexity and resource requirements, thereby reducing system response speed and impacting user experience. Therefore, improving computational efficiency has become an urgent technical problem to be solved.
[0056] In view of this, embodiments of this application provide a task processing method, apparatus, electronic device, and storage medium. The method includes: determining a target LORA module corresponding to the target task type based on the target task type corresponding to the user-input text to be processed; different LORA modules correspond to different task types; the target LORA module is used to fine-tune at least one target layer of each transport layer of a basic large language model; the weight of each target layer includes the basic weight of each target layer and the weight of the target LORA module; the weight of the target LORA module includes a first LORA weight and a second LORA weight; the basic large language model also includes an input layer and an output layer; and processing the text to be processed using the input layer. The text features are obtained; based on these text features, at least one transport layer is applied to obtain the transport layer result. The input of the first transport layer is the text features, and the output of the last transport layer is the processed text. The output of any transport layer other than the last one is the input of the next transport layer. The output of each target layer in each transport layer includes variable results and basic results. The basic results are obtained based on the input parameters of each target layer and the basic weights of each target layer. The variable results are obtained based on the product of the input parameters of each target layer and the first LORA weight, plus the second LORA weight. The output layer processes the transport layer results to obtain the processed text. This embodiment optimizes the calculation method of the low-rank matrix in the inference stage, improving computational efficiency.
[0057] The task processing method provided in this application is applied to an electronic device, which may be a mobile phone, tablet computer, laptop computer, mobile internet device (MID), terminal in industrial control, etc. This application does not limit the application to this type of device.
[0058] To make the purpose and technical solution of this application clearer and more intuitive, a task processing method disclosed in this application will be described below with reference to the accompanying drawings.
[0059] It should be understood that the execution subject of the task processing method of this application can be an electronic device, or a processor or chip in the electronic device. For ease of description, the electronic device is used as the execution subject to describe the embodiments of this application.
[0060] Please see Figure 1 , Figure 1 This is a flowchart illustrating a task processing method disclosed in an embodiment of this application. Figure 1The method shown may include the following steps:
[0061] Step 101: The electronic device determines the target LORA module corresponding to the target task type based on the target task type corresponding to the text to be processed input by the user.
[0062] In some embodiments, see Figure 2 , Figure 2 This is a flowchart illustrating a method for determining the target task type corresponding to user-inputted text, as disclosed in an embodiment of this application. The user-inputted text can be processed according to a recognition model to determine the target task type corresponding to the user-inputted text, and one user input corresponds to one task type.
[0063] For example, a pre-trained lightweight language model, such as qwen2.5-0.5b-instruct, can be used as the recognition model. When a user inputs text to be processed, the recognition model first processes the text and outputs the task type. For instance, if a user inputs "What's the weather like in Shanghai today?", the recognition model will output the corresponding task type as "weather query task".
[0064] In some embodiments, different LORA modules correspond to different task types.
[0065] In some embodiments, multiple LORA modules corresponding to multiple task types can be obtained through model training, a mapping relationship between task types and LORA modules can be established, and then the target LORA module corresponding to the target task type can be determined based on the different LORA modules corresponding to different task types.
[0066] For example, based on the target task type corresponding to the user-input text to be processed, and according to the pre-established mapping relationship between task types and LORA modules, the target LORA module is determined from multiple LORA modules. For instance, if the user inputs "What's the weather like in Shanghai today?", the corresponding target task type is a weather query task. Based on the pre-established mapping relationship between weather query tasks and LORA module 1, the corresponding target LORA module is determined to be LORA module 1.
[0067] In some embodiments, the target LORA module is used to fine-tune at least one target layer for each transport layer of at least one transport layer of the underlying large language model. In some embodiments, the underlying large language model further includes an input layer and an output layer.
[0068] For example, the core of the basic large language model is the Transformer architecture. Its input layer is an input embedding layer, used to convert discrete text words or tokens into numerical vectors. The output layer is a linear projection layer, used to map the output of the last transport layer back to the vocabulary space, generating a probability distribution to predict the next token. At least one transport layer of the basic large language model is a Transformer layer, and each transport layer may include multiple sub-layers. At least one target layer among these sub-layers can be the object of fine-tuning the LORA model. For example, each Transformer layer includes a multi-head self-attention layer, a feedforward neural network layer, residual connections, and layer normalization. At least one target layer of each transport layer can be the multi-head self-attention layer and the feedforward neural network layer in the Transformer layer. The target LORA module can fine-tune the multi-head self-attention layer and the feedforward neural network layer of the Transformer layer of the basic large language model.
[0069] In some embodiments, the weight of each target layer includes the base weight of each target layer and the weight of the target LORA module.
[0070] For example, the formula for calculating the weight of the target layer after fine-tuning is: weight of target layer = W0 + ΔW, where W0 is the base weight and ΔW is the weight of the target LORA module.
[0071] In some embodiments, the weights of the target LORA module include a first LORA weight and a second LORA weight. For example, the first LORA weight is an uplink projection matrix, and the second LORA weight is a downlink projection matrix.
[0072] For example, the first LORA weight is matrix B, the second LORA weight is matrix A, and the weight ΔW of the target LORA module is B × A. The downlink projection matrix (matrix A) maps the input feature vector to a low-dimensional space, achieving effective compression of the feature representation; the uplink projection matrix (matrix B) reprojects the feature vector in the low-dimensional space back to the original dimension.
[0073] For example, taking at least one target layer including a multi-head self-attention layer as an example, in the self-attention calculation process of the multi-head self-attention layer of the Transformer layer, in related technologies, the projection matrices (such as W_Q, W_K, W_V) of the query (Q), key (K), and value (V) are originally updated directly through a full-parameter fine-tuning method. However, in this embodiment, the LORA module can decompose the weight update ΔW into the product of matrix B and matrix A through low-rank decomposition. Taking the query (Q) projection matrix W_Q of the multi-head self-attention layer of the Transformer layer as an example, the full-parameter fine-tuning method directly updates all parameters of the W_Q matrix. Compared with the above-mentioned related technologies, in this embodiment, W_Q is not directly updated, but the weight update ΔW is represented by the product of two low-rank matrices, ΔW = B·A.
[0074] As can be seen, by implementing the above embodiments, the first LORA weights serve as the uplink projection matrix, and the second LORA weights serve as the downlink projection matrix, together constituting an efficient parameter update mechanism. Specifically, the downlink projection matrix maps the input feature vectors to a low-dimensional space, achieving effective compression of the feature representation; the uplink projection matrix reprojects the feature vectors in the low-dimensional space back to the original dimension. By constraining parameter updates to a low-dimensional subspace, the learning enables the basic large language model to adapt to the minimum and most effective feature transformation of the target task, significantly reducing the number of trainable parameters and computational complexity, and improving computational efficiency.
[0075] Step 102: The electronic device uses the input layer to process the text to be processed and obtain text features.
[0076] In some embodiments, text features are represented in matrix form.
[0077] For example, the input layer first performs word segmentation on the text to be processed, splitting the continuous text sequence into discrete tokens (such as words and sub-words). For example, "How is the weather in Shanghai today?" is split into ["today", "Shanghai", "weather", "how is", "?"]. Subsequently, each token is mapped to a fixed-dimensional vector through an input embedding layer (such as Word Embedding), while incorporating positional encoding to represent the token's positional information in the text sequence, ultimately obtaining the text feature matrix.
[0078] Step 103: The electronic device performs at least one transmission layer processing based on text features to obtain the transmission layer result.
[0079] In some embodiments, the input to the first transport layer is text features, the output of the last transport layer is the processing result of the text to be processed, and the output of any transport layer other than the last transport layer is the input to the next transport layer.
[0080] For example, the input to the first transport layer is the text features, the input to the second transport layer is the output of the first transport layer, the output of the second transport layer is the input of the third transport layer, and so on. In the last transport layer, the output of the last transport layer is the processing result of the text to be processed.
[0081] In some embodiments, the output of each target layer of each transport layer includes variable results and basic results. The basic results are obtained based on the input parameters of each target layer and the basic weights of each target layer. The variable results are obtained based on the product of the input parameters of each target layer and the first LORA weights and the second LORA weights.
[0082] For example, the output result h of a target layer can be calculated using method 1. The variable result is obtained based on the product of the input parameters and the first LORA weights of each target layer, plus the second LORA weights. That is, the product of the input parameters and the first LORA weights is calculated first, and then the resulting product is multiplied by the second LORA weights. For example, the formula is h = x × W0 + (x × B) × A, where x is the input parameter with dimensions [b, s, d], W0 is the base weight, B and A are the first LORA weights and the second LORA weight, respectively, with dimensions [r, h] for A and [d, r] for B, and x × B is the product of the input parameters and the first LORA weights.
[0083] Please see Figure 3 , Figure 3 This is a flowchart illustrating a calculation of variable results disclosed in an embodiment of this application. The calculation method shown in the figure refers to the method for calculating variable results in method 1.
[0084] Please see Figure 4 , Figure 4 This is a flowchart illustrating the calculation of the output of a target layer according to an embodiment of this application. In the figure, x is the input parameter, W0 is the basic weight, B and A are the first LORA weight and the second LORA weight, respectively, and h is the output of the target layer.
[0085] For example, the original calculation method 2 can also be used to calculate the output result h of a target layer, where the variable result is obtained based on the product of the input parameters of each target layer and the first LORA weight and the second LORA weight. That is, the product of the first LORA weight and the second LORA weight is calculated first, and then the resulting product is multiplied by the input parameters. Taking the formula as an example, h = x × W0 + x × (B × A).
[0086] Comparing Method 1 and Method 2, taking r=32, d=1024, h=4096, s as the length of the input text=1024, and b as the batch_size=4 as an example, the computational cost of (x×B)×A in Method 1 is b*s*d*r+b*s*r*h=4*1024*1024*32+4*1024*32*4096=671088640, while the computational cost of x×(B×A) in Method 2 is d*r*h+b*s*d*h=1024*32*4096+4*1024*1024*4096=1731408691. It can be seen that the computational cost of the optimized Method 1 is 3.8% of that of the original Method 2. Thus, the computational cost of the reasoning process can be effectively reduced, thereby improving the reasoning speed.
[0087] Step 104: The electronic device uses the output layer to process the transmission layer result to obtain the processing result of the text to be processed.
[0088] For example, the output layer is a linear projection layer, used to map the high-dimensional feature tensor of the output of the last Transformer layer to a dimensional space matching the size of the model vocabulary. Subsequently, the output layer performs Softmax activation on the raw scores of these tokens to obtain a probability distribution, where each probability value corresponds to the likelihood of "generating this token as the next token". For example, in a weather query task, the output layer will select the token with the highest probability based on the probability distribution to generate a coherent natural language response such as "Today's weather in Shanghai is cloudy turning sunny, with a temperature of 18-25℃".
[0089] As can be seen, by implementing the above embodiments, dynamically selecting and loading the corresponding target LORA module according to the target task type corresponding to the user-input text to be processed, the target layer in the basic large language model is fine-tuned. This enables support for multiple task processing on a single basic large language model, significantly reducing the GPU memory overhead required to deploy independent models for different tasks and improving the system's resource utilization. It can be seen that the above embodiments configure a corresponding LORA module for each task type. For the target task type, a target LORA module can be determined, and the weights of the target LORA module include the first LORA weight and the second LORA weight. Therefore, the corresponding low-rank matrix (the weights of the LORA module) can be called according to the task type, enhancing the basic large language model's processing capability for specific task types. Furthermore, in the inference process, firstly, the input layer processes the text to be processed to obtain text features, and then the transport layer processes the text layer by layer to obtain the transport layer results. The output of each target layer in each transport layer includes variable results and basic results. The variable results are obtained by multiplying the input parameters and the first LORA weights by the second LORA weights. Specifically, the product of the input parameters and the first LORA weights is calculated first, and then multiplied by the second LORA weights. This optimized calculation method for variable results during the inference phase effectively reduces the computational load, thereby improving inference speed. Finally, the output layer processes the transport layer results to obtain the processed text. The above embodiment improves computational efficiency and inference speed by optimizing the calculation method of the weights (low-rank matrix) of the LORA module during the model inference phase, thus enhancing the user experience.
[0090] In some embodiments, before processing the text to be processed using the input layer to obtain text features, the method further includes determining the target video memory region corresponding to the target LORA module. See also Figure 5 , Figure 5 This is a flowchart illustrating a method for determining the target memory region corresponding to a target LORA module, as disclosed in an embodiment of this application. Figure 5 The method shown may include the following steps:
[0091] Step 201: The electronic device determines the task level of the target task type based on the parameter information of the target task type.
[0092] In some embodiments, the parameter information includes at least one of call frequency, latency requirement, and video memory requirement. The call frequency is used to characterize the number of calls for the target task type, the latency requirement is used to characterize the response latency of the target task type, and the video memory requirement is used to characterize the video memory capacity occupied by the target LORA module corresponding to the target task type.
[0093] In some embodiments, the call frequency in the parameter information can be the number of calls to the target task type, or the ratio of the number of calls to the target task type to the number of calls to all task types. The latency requirement in the parameter information can be the response latency of the target task type, or the ratio of the response latency of the target task type to the maximum latency allowed by the system. The video memory requirement in the parameter information can be the video memory capacity occupied by the target LoRA module corresponding to the target task type, or the ratio of the video memory capacity occupied by the target LoRA module corresponding to the target task type to the total video memory capacity of the GPU.
[0094] In some embodiments, the task level includes a first level and other levels below the first level.
[0095] For example, the task level of the target task type can be directly determined based on at least one of the following: call frequency, latency requirements, and video memory requirements. For instance, if the call frequency of the target task type is greater than or equal to a preset threshold, the task level of the target task type is the first level; if the call frequency of the target task type is less than the preset threshold, the task level of the target task type is another level.
[0096] Step 202: The electronic device determines the target video memory region corresponding to the target LORA module based on the task level of the target task type.
[0097] In some embodiments, the target video memory region includes a dedicated video memory region or a shared video memory region. The dedicated video memory region is used to store the LOA modules corresponding to the first level of task types, and the shared video memory region is used to store the LOA modules corresponding to other levels of task types.
[0098] For example, if the task level of the target task type is Level 1, the target LOA module is stored in the dedicated video memory area; if the task level of the target task type is another level, the target LOA module is stored in the shared video memory area. For instance, the task level of a weather query task is Level 1, and the corresponding target video memory area is the dedicated video memory area.
[0099] Step 203: If the target LORA module does not exist in the target video memory area, the electronic device loads the target LORA module into the target video memory area.
[0100] For example, following the above embodiments, if a query reveals that there is no LORA module 1 corresponding to the weather query task in the dedicated video memory area, then the LORA module 1 is loaded from the storage medium (such as a hard disk) into the dedicated video memory area.
[0101] As can be seen, implementing the above embodiments improves the efficiency of memory resource utilization and task response performance in multi-LORA module scenarios through a task-level-based dynamic memory allocation mechanism. Specifically, based on the parameter information of the target task type (including at least one of the following: call frequency, latency requirements, and memory requirements), the task level of the target task type is determined, enabling a quantitative assessment of the resource requirements of different task types and providing a reliable basis for subsequent differentiated memory allocation strategies. Then, based on the task level of the target task type, corresponding target memory regions are allocated to the target LORA modules. LORA modules with higher task levels are allocated to dedicated memory regions to ensure their high-frequency call and low-latency response requirements; while LORA modules with other task levels are allocated to shared memory regions to improve memory resource reuse and reduce redundant occupation. Finally, when no target LORA module exists in the target memory region, the target LORA module is loaded into the target memory region. On-demand loading effectively avoids unnecessary memory pre-occupancy and performance fluctuations caused by frequent I / O operations.
[0102] In some embodiments, a priority value for the target task type is determined based on parameter information of the target task type, and a task level for the target task type is determined based on the priority value. See also... Figure 6 , Figure 6 This is a flowchart illustrating a method for determining the task level of a target task type, as disclosed in an embodiment of this application. Figure 6 The method shown may include the following steps:
[0103] Step 301: The electronic device determines the priority value of the target task type based on the parameter information of the target task type.
[0104] For the specific implementation of the parameter information in step 301, please refer to the content in step 201 above, which will not be repeated here.
[0105] In some embodiments, the parameter information may include call frequency, latency requirements, and video memory requirements. In this embodiment, for example, the priority value for the task type is... i The calculation formula is: Where i represents the task type, C i For the number of calls to task type i, C max For the number of calls for all task types, Let T be the average response latency for task type i. max M is the maximum allowable delay for the system. i M represents the amount of video memory occupied by the LoRa module corresponding to task type i. totalThis represents the total capacity of the GPU's video memory. α, β, and γ are weighting coefficients, and α + β + γ = 1. For example, α = 0.6, β = 0.3, and γ = 0.1.
[0106] In other embodiments, the parameter information may include at least two of the following: call frequency, latency requirements, and video memory requirements, with each parameter corresponding to a weight. For example, taking parameter information including call frequency and video memory requirements as an example, the priority value for the task type is... i The calculation formula is: α and γ are weighting coefficients and α+γ=1, for example, α=0.6, γ=0.4.
[0107] In other embodiments, the parameter information may include any one of the following: call frequency, latency requirements, and video memory requirements.
[0108] Step 302: If the priority value of the target task type is greater than or equal to the first threshold, the task level of the target task type is determined as the first level.
[0109] For example, if the priority value of the target task type is 1 and the first threshold is 0.8, then if the priority value of the target task type is greater than 0.8, the task level of the target task type is the first level.
[0110] Step 303: If the priority value corresponding to the target task type is less than the first threshold, the task level of the target task type is determined to be another level.
[0111] For example, if the priority value of the target task type is 0.6 and the first threshold is 0.8, and the priority value of the target task type is less than 0.8, then the task level of the target task type is other levels.
[0112] As can be seen, by implementing the above embodiments, priority values are calculated based on the parameter information of task types, and then the priority values are compared with preset thresholds to classify task types into first level or other levels. By further refining the calculation method for determining task levels based on priority values, a more accurate quantitative assessment of the resource requirements of different task types is achieved, providing a reliable basis for subsequent differentiated memory allocation strategies.
[0113] In some embodiments, other levels include a second level and a third level which is lower than the second level.
[0114] In some embodiments, if the priority value corresponding to the target task type is less than the first threshold and greater than or equal to the second threshold, the task level of the target task type is determined to be the second level; if the priority value corresponding to the target task type is less than the second threshold, the task level of the target task type is determined to be the third level.
[0115] For example, the first threshold is 0.8 and the second threshold is 0.5. If the priority value of the target task type is less than 0.8 and greater than or equal to 0.5, the task level of the target task type is the second level. If the priority value of the target task type is less than 0.5, the task level of the target task type is the third level.
[0116] As can be seen, by implementing the above embodiments, other levels are further subdivided into second and third levels based on priority values, which achieves a more refined division of task levels. This allows for a more precise quantification of the differences in resource requirements for different task types, providing a more reliable basis for subsequent differentiated memory allocation strategies.
[0117] Of course, in other embodiments, the classification of other levels is not limited to the second and third levels in the examples above, and there may be more levels. In this way, the task level can be classified more finely, thereby allowing for a more accurate and precise quantification of the differences in resource requirements for different task types.
[0118] In some embodiments, the capacity of the dedicated video memory region accounts for a first proportion of the total video memory capacity, the capacity of the shared video memory region used to store the LoRA modules corresponding to the second-level task types accounts for a second proportion of the total video memory capacity, and the capacity of the shared video memory region used to store the LoRA modules corresponding to the third-level task types accounts for a third proportion of the total video memory capacity. The first proportion is greater than the second proportion, and the second proportion is greater than the third proportion.
[0119] For example, the first, second, and third ratios can be dynamically changed according to task requirements. For instance, the capacity of the dedicated video memory area accounts for 45% of the total video memory capacity, the capacity of the shared video memory area used to store the LoRa modules corresponding to the second-level task type accounts for 30% of the total video memory capacity, and the capacity of the shared video memory area used to store the LoRa modules corresponding to the third-level task type accounts for 15% of the total video memory capacity.
[0120] As can be seen, by implementing the above embodiments, the video memory capacity is allocated to LORA modules of different task levels according to differentiated ratios, thus achieving the matching of video memory resources with task levels. This ensures that LORA modules with higher task levels obtain more exclusive video memory resources. At the same time, more video memory resources are reserved for second-level LORA modules in the shared video memory area than for third-level LORA modules, preventing third-level LORA modules from over-consuming video memory, thereby improving the overall utilization rate of video memory resources.
[0121] In some embodiments, if the target LORA module is not present in the target video memory region and the occupancy rate of the target video memory region is higher than a preset threshold, the currently unused portion of the LORA modules stored in the target video memory region are removed, and the target LORA module is loaded into the target video memory region. See also... Figure 7 , Figure 7 This is a schematic flowchart illustrating a method for loading a target LORA module disclosed in an embodiment of this application. Figure 7 The method shown may include the following steps:
[0122] Step 401: If the electronic device does not have a target LORA module in the target video memory area and the occupancy rate of the target video memory area is higher than a preset threshold, remove the currently unused LORA modules stored in the target video memory area.
[0123] In some embodiments, if there are no target LORA modules in the target video memory region and the occupancy rate of the target video memory region is higher than a preset threshold, the least recently used LORA modules can also be removed.
[0124] For example, if there is no corresponding LORA module 1 for the weather query task in the dedicated video memory area, and the occupancy rate of the dedicated video memory area is higher than a preset threshold of 75%, and the currently unused LORA modules stored in the dedicated video memory area are LORA module 3, LORA module 4, and LORA module 7, one or more of these three LORA modules can be removed based on the video memory requirements of LORA module 1. Furthermore, a "least recently used" rule can be used to prioritize removing the LORA module that has not been called for the longest time. For example, if LORA module 7 has not been called for 5 consecutive hours and has the longest idle time among the three, then LORA module 7 will be removed first.
[0125] In other embodiments, unused LoRa modules stored in dedicated or shared memory regions can be removed based on the total video memory utilization. When the total video memory utilization is greater than a third threshold and less than or equal to a fourth threshold, third-level LoRa modules stored in the shared memory region are removed; when the total video memory utilization is greater than the fourth threshold and less than or equal to a fifth threshold, second-level LoRa modules stored in the shared memory region are removed; and when the total video memory utilization is greater than the fifth threshold, first-level LoRa modules stored in the dedicated video memory region are removed.
[0126] For example, the third threshold is 75%, the fourth threshold is 90%, and the fifth threshold is 95%.
[0127] Please refer to the table below, which is a table of removal operations in the embodiments of this application.
[0128] Total video memory usage range Pressure level Remove operation object 0-75％ normal none 75％-90％ warn Level 3 LORA module 90％-95％ serious Level 2 LORA module 95％-100％ critical Level 1 LORA module
[0129] Step 402: The electronic device loads the target LORA module into the target video memory area.
[0130] For example, following the above embodiments, after removing part of the LORA module from the dedicated video memory area, the LORA module 1 is loaded from the storage medium (such as a hard disk) into the dedicated video memory area.
[0131] As can be seen, by implementing the above embodiments, when there is no target LORA module in the target video memory region and the occupancy rate of the target video memory region is too high, the unused LORA modules are removed first, and then the target LORA module is loaded. This solves the loading conflict problem when video memory resources are scarce, avoids the target LORA module loading failure due to insufficient video memory space, and enables the target LORA module to be loaded normally into the target video memory region to support subsequent task inference, thereby improving the flexibility and reliability of dynamic scheduling of video memory resources.
[0132] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps. In addition, the above embodiments can be implemented independently or in combination with each other, without limitation.
[0133] Based on the foregoing embodiments, this application provides a task processing device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.
[0134] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a task processing device disclosed in an embodiment of this application, as shown below. Figure 8 The apparatus shown includes a determining module 501, a first processing module 502, a second processing module 503, and a third processing module 504.
[0135] Determining Module: Based on the target task type corresponding to the user-input text to be processed, determine the target LORA module corresponding to the target task type. Different LORA modules correspond to different task types. The target LORA module is used to fine-tune at least one target layer of each transport layer of the basic large language model. The weight of each target layer includes the basic weight of each target layer and the weight of the target LORA module. The weight of the target LORA module includes the first LORA weight and the second LORA weight. The basic large language model also includes an input layer and an output layer.
[0136] The first processing module is used to process the text to be processed from the input layer to obtain text features;
[0137] The second processing module is used to perform at least one transport layer processing based on text features to obtain the transport layer results. The input of the first transport layer is the text features, and the output of the last transport layer is the processing result of the text to be processed. The output of any transport layer other than the last transport layer is the input of the next transport layer. The output result of each target layer of each transport layer includes variable results and basic results. The basic results are obtained based on the input parameters of each target layer and the basic weights of each target layer. The variable results are obtained based on the product of the input parameters of each target layer and the first LORA weight and the second LORA weight.
[0138] The third processing module is used to process the results of the transmission layer using the output layer to obtain the processing result of the text to be processed.
[0139] It is understandable that the first processing module can be understood as the input layer of the basic large language model, the second processing module can be understood as the transmission layer of the basic large language model, and the third processing module can be understood as the output layer of the basic large language model.
[0140] In some embodiments, the first LORA weight is an up-projection matrix, and the second LORA weight is a down-projection matrix.
[0141] In some embodiments, before processing the text to be processed using the input layer to obtain text features, the method further includes:
[0142] Based on the parameter information of the target task type, determine the task level of the target task type. The parameter information includes at least one of the following: call frequency, latency requirement, and video memory requirement. The call frequency is used to characterize the number of calls of the target task type, the latency requirement is used to characterize the response latency of the target task type, and the video memory requirement is used to characterize the video memory capacity occupied by the target LORA module corresponding to the target task type. The task level includes the first level and other levels below the first level.
[0143] Based on the task level of the target task type, determine the target video memory area corresponding to the target LOA module. The target video memory area includes a dedicated video memory area or a shared video memory area. The dedicated video memory area is used to store the LOA modules corresponding to the first level of task type, and the shared video memory area is used to store the LOA modules corresponding to other levels of task type.
[0144] If the target LORA module does not exist in the target video memory region, load the target LORA module into the target video memory region.
[0145] In some embodiments, determining the task level of the target task type based on parameter information of the target task type includes:
[0146] Based on the parameter information of the target task type, determine the priority value of the target task type;
[0147] If the priority value of the target task type is greater than or equal to the first threshold, the task level of the target task type is determined as the first level;
[0148] If the priority value corresponding to the target task type is less than the first threshold, the task level of the target task type will be determined as another level.
[0149] In some embodiments, other levels include a second level and a third level which is lower than the second level;
[0150] If the priority value corresponding to the target task type is less than the first threshold, the task level of the target task type will be determined to another level, including:
[0151] If the priority value corresponding to the target task type is less than the first threshold and greater than or equal to the second threshold, the task level of the target task type will be determined as the second level.
[0152] If the priority value corresponding to the target task type is less than the second threshold, the task level of the target task type will be determined as the third level.
[0153] In some embodiments, the capacity of the dedicated video memory region accounts for a first proportion of the total video memory capacity, the capacity of the shared video memory region used to store the LoRA modules corresponding to the second-level task types accounts for a second proportion of the total video memory capacity, and the capacity of the shared video memory region used to store the LoRA modules corresponding to the third-level task types accounts for a third proportion of the total video memory capacity. The first proportion is greater than the second proportion, and the second proportion is greater than the third proportion.
[0154] In some embodiments, loading the target LORA module into the target video memory region when the target LORA module does not exist in the target video memory region includes:
[0155] If no target LORA module exists in the target memory region and the occupancy rate of the target memory region is higher than a preset threshold, remove the currently unused LORA modules stored in the target memory region.
[0156] Load the target LORA module into the target video memory area.
[0157] It should be noted that the division of modules in the task processing device shown in the embodiments of this application is illustrative and is only a logical functional division. In actual implementation, there may be other division methods.
[0158] An electronic device provided in this application embodiment is described below. Figure 9 , Figure 9 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. For example... Figure 9 As shown, the electronic device includes:
[0159] Memory 601 storing executable program code;
[0160] Processor 602 coupled to memory 601;
[0161] The processor 602 calls the executable program code stored in the memory 601 to execute any one of the task processing methods in the above method embodiments.
[0162] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in any of the task processing methods provided in the above embodiments.
[0163] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0164] It should be noted that the descriptions of the above embodiments of the apparatus, electronic devices, and computer-readable storage media are similar to the descriptions of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of the apparatus, electronic devices, and computer-readable storage media of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0165] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.
[0166] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.
[0167] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0168] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and electronic devices can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules above is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.
[0169] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0170] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0171] The features disclosed in the several device embodiments provided in this application can be arbitrarily combined without conflict to obtain new device embodiments.
[0172] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A task processing method characterized by, The method comprises: According to the target task type corresponding to the user input to-be-processed text, a target LORA module corresponding to the target task type is determined, different LORA modules correspond to different task types, the target LORA module is used for fine-tuning at least one target layer of each transmission layer of at least one transmission layer of a basic large language model, the weight of each target layer includes a basic weight of each target layer and a weight of the target LORA module, the weight of the target LORA module includes a first LORA weight and a second LORA weight, and the basic large language model further includes an input layer and an output layer; The input layer is used to process the to-be-processed text to obtain text features; Based on the text features, processing of the at least one transmission layer is performed to obtain a transmission layer result, the input of a first transmission layer is the text features, the output of a last transmission layer is a processing result of the to-be-processed text, and the output of any transmission layer other than the last transmission layer in the at least one transmission layer is the input of a next transmission layer, wherein the output result of each target layer of each transmission layer includes a variable result and a basic result, the basic result is obtained based on an input parameter of each target layer and a basic weight of each target layer, and the variable result is obtained based on a product of the input parameter of each target layer and the first LORA weight and the second LORA weight; The output layer is used to process the transmission layer result to obtain the processing result of the to-be-processed text.
2. The method of claim 1, wherein, The first LORA weight is an uplink projection matrix, and the second LORA weight is a downlink projection matrix.
3. The method of claim 1, wherein, Before the input layer is used to process the to-be-processed text to obtain text features, the method further comprises: According to the parameter information of the target task type, a task level of the target task type is determined, the parameter information includes at least one of a call frequency, a time delay requirement and a video memory requirement, the call frequency is used to represent the number of calls of the target task type, the time delay requirement is used to represent the response time delay of the target task type, and the video memory requirement is used to represent the capacity of the video memory occupied by the target LORA module corresponding to the target task type, the task level includes a first level and other levels less than the first level; According to the task level of the target task type, a target video memory area corresponding to the target LORA module is determined, the target video memory area includes a dedicated video memory area or a shared video memory area, the dedicated video memory area is used to store the LORA module corresponding to the task type of the first level, and the shared video memory area is used to store the LORA module corresponding to the task type of the other levels; In the case that the target LORA module does not exist in the target video memory area, the target LORA module is loaded in the target video memory area.
4. The method of claim 3, wherein, The determination of the task level of the target task type according to the parameter information of the target task type comprises: determine a priority value of the target task type according to the parameter information of the target task type; determine a task level of the target task type as the first level if the priority value of the target task type is greater than or equal to a first threshold value; determine the task level of the target task type as the other level if the priority value corresponding to the target task type is less than the first threshold value.
5. The method of claim 3, wherein, The other level includes a second level and a third level less than the second level. The determination of the task level of the target task type as the other level if the priority value corresponding to the target task type is less than the first threshold value includes: determine the task level of the target task type as the second level if the priority value corresponding to the target task type is less than the first threshold value and greater than or equal to a second threshold value; determine the task level of the target task type as the third level if the priority value corresponding to the target task type is less than the second threshold value.
6. The method of claim 5, wherein, The capacity of the dedicated GPU region accounts for a first proportion of the total GPU capacity, the capacity of the shared GPU region for storing the LORA module corresponding to the task type of the second level accounts for a second proportion of the total GPU capacity, and the capacity of the shared GPU region for storing the LORA module corresponding to the task type of the third level accounts for a third proportion of the total GPU capacity. The first proportion is greater than the second proportion, and the second proportion is greater than the third proportion.
7. The method of claim 3, wherein, The loading of the target LORA module in the target GPU region in the case that the target LORA module does not exist in the target GPU region includes: In the case that the target LORA module does not exist in the target GPU region and the occupancy rate of the target GPU region is higher than a preset threshold value, removing a part of the LORA modules that are not currently used and stored in the target GPU region; loading the target LORA module in the target GPU region.
8. A task processing apparatus characterized by comprising: The device includes: A determination module is configured to determine a target LORA module corresponding to a target task type of a user-inputted to-be-processed text according to the target task type. Different LORA modules correspond to different task types. The target LORA module is used to fine-tune at least one target layer of each transmission layer of at least one transmission layer of a basic large language model. The weight of each target layer includes a basic weight of each target layer and a weight of the target LORA module. The weight of the target LORA module includes a first LORA weight and a second LORA weight. The basic large language model further includes an input layer and an output layer. A first processing module is configured to process the to-be-processed text by using the input layer to obtain text features. The second processing module is configured to perform processing on the at least one transmission layer based on the text features to obtain a transmission layer result, wherein an input of a first transmission layer is the text features, an output of a last transmission layer is a processing result of the text to be processed, an output of any transmission layer other than the last transmission layer in the at least one transmission layer is an input of a next transmission layer, wherein an output result of each target layer of each transmission layer comprises a variable result and a basic result, the basic result is obtained based on an input parameter of the each target layer and a basic weight of the each target layer, and the variable result is obtained based on a product of the input parameter of the each target layer and the first LORA weight and the second LORA weight; The third processing module is configured to perform processing on the transmission layer result by using the output layer to obtain the processing result of the text to be processed.
9. An electronic device, comprising: A processor and a memory are included, the memory stores a computer program, and the processor implements the steps of the method in any one of claims 1-7 when executing the computer program.
10. A computer-readable storage medium, characterized in that, A computer program is stored, and the program is executed by a processor to implement the steps of the method in any one of claims 1-7.