Unified base model multi-task inference method for resource-sensitive deployment environment

Abstract

The application provides a unified base model multi-task reasoning method for a resource-sensitive deployment environment. The application generates a rank configuration instruction according to a hardware resource load parameter, which indicates an effective rank dimension to be activated in a low-rank enhancement branch. Different rank levels correspond to different numbers of activated ranks, so that the reasoning device can adjust the number of data channels actually participating in calculation of the low-rank enhancement branch in real time according to the current resource status. In response to the instruction, the low-rank enhancement branch is switched to a low-rank configuration corresponding to the current effective rank dimension, thereby changing the calculation intensity of the enhancement branch without changing the base model. In this way, the reasoning device can dynamically adjust the calculation amount of the low-rank enhancement branch under fluctuating resource deployment conditions, solving the defect that the fixed low-rank enhancement path in the related art cannot be adjusted with resource changes, and taking into account the reasoning requirements under different resource conditions.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a multi-task reasoning method for a unified base model for resource-sensitive deployment environments. Background Technology

[0002] In multi-task deployment scenarios, large language models typically employ low-rank adaptation techniques as the mainstream solution for parameter fine-tuning. For example, low-rank branches are set next to the original weight matrix to achieve rapid adaptation to specific tasks. However, these solutions generally use fixed low-rank enhancement paths during the inference phase. That is, the structure and computational cost of the low-rank branches are fixed after training and cannot be adjusted according to changes in the actual deployment environment.

[0003] In resource-sensitive deployment environments, hardware resources often struggle to support all inference requests using the highest precision mode, making it difficult for the system to balance inference efficiency and quality under fluctuating resource conditions. For example, during peak request periods or when hardware resources are scarce, fixed paths in related technologies cannot reduce the computational load of enhanced branches to ensure real-time inference, potentially leading to increased service latency or even system crashes. Conversely, when hardware resources are plentiful, fixed paths cannot fully utilize idle resources to improve inference quality. Summary of the Invention

[0004] In view of this, this application provides a unified base model multi-task inference method for resource-sensitive deployment environments, so as to balance the inference efficiency and inference quality of multi-task.

[0005] The technical solution provided in this application is as follows: According to an embodiment of the first aspect of this application, a multi-task inference method for a unified base model for resource-sensitive deployment environments is provided. The method is applied to an inference device configured to run a unified base model, which includes an original weight matrix and independently configured low-rank enhancement branches. The method includes: Based on the hardware resource load parameters of the inference device and the task to be executed, a rank configuration instruction is generated. The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. The low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation. The effective rank dimension includes a first target rank level in the shared low-rank base and a second target rank level in the task-specific low-rank compensation. Different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated. In response to the rank configuration instruction, the low-rank enhancement branch in the unified base model is switched to a low-rank configuration corresponding to the effective rank dimension; The target inference result is obtained by superimposing the output of the original weight matrix to the input data in the unified base model and the calculation result of the low-rank configuration to the input data.

[0006] According to an embodiment of the second aspect of this application, a multi-task inference apparatus for a unified base model oriented to a resource-sensitive deployment environment is provided. The apparatus is applied to an inference device configured to run a unified base model, the unified base model including an original weight matrix and independently configured low-rank enhancement branches. The apparatus includes: A generation unit is configured to generate a rank configuration instruction based on the hardware resource load parameters of the inference device and the task to be executed. The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. The low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation. The effective rank dimension includes a first target rank level in the shared low-rank base and a second target rank level in the task-specific low-rank compensation. Different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated. A switching unit is configured to, in response to the rank configuration instruction, switch the low-rank enhancement branch in the unified base model to a low-rank configuration corresponding to the effective rank dimension. The inference unit is used to superimpose the output of the original weight matrix of the unified base model on the input data and the calculation result of the low-rank configuration on the input data to obtain the target inference result.

[0007] According to an embodiment of a third aspect of this application, an electronic device is provided, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; the processor being configured to execute the machine-executable instructions to implement the method as described in the first aspect.

[0008] As can be seen from the above technical solution, this application generates a rank configuration instruction based on hardware resource load parameters. This instruction indicates the effective rank dimension to be activated in the low-rank enhancement branch, and different rank levels correspond to different numbers of rank to be activated. This allows the inference device to adjust the number of data channels actually involved in the computation of the low-rank enhancement branch in real time according to the current resource status. In response to this instruction, the low-rank enhancement branch is switched to the low-rank configuration corresponding to the current effective rank dimension, thereby changing the computational intensity of the enhancement branch without changing the base model. Thus, the inference device can dynamically adjust the computational load of the low-rank enhancement branch under deployment conditions with fluctuating resources, solving the defect in related technologies where fixed low-rank enhancement paths cannot be adjusted with resource changes, and taking into account the inference needs under different resource conditions. Attached Figure Description

[0009] Figure 1 A flowchart of a multi-task inference method for a unified base model for resource-sensitive deployment environments provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the low-rank enhancement branch provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating intra-group and inter-group handover scenarios provided in the embodiments of this application; Figure 4 This application provides a structural diagram of a unified base model multi-task inference device for resource-sensitive deployment environments. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0011] In multi-task deployment scenarios, large language models typically employ low-rank adaptation techniques as the mainstream solution for parameter fine-tuning. For example, low-rank branches are set next to the original weight matrix to achieve rapid adaptation to specific tasks. However, these solutions generally use fixed low-rank enhancement paths during the inference phase. That is, the structure and computational cost of the low-rank branches are fixed after training and cannot be adjusted according to changes in the actual deployment environment.

[0012] In related technologies, the rank of the low-rank branch is a preset, fixed hyperparameter. Once the model is trained and deployed to the inference device, the number of data channels actually involved in the computation of the low-rank augmentation branch is determined. Under this fixed-path mechanism, regardless of the current hardware resource load of the inference device, all inference requests perform low-rank augmentation operations with the same amount of computation, and cannot be dynamically adjusted according to resource availability.

[0013] In resource-sensitive deployment environments, the shortcomings of the fixed-path mechanism are particularly pronounced. Resource-sensitive deployment environments refer to scenarios where inference devices have relatively limited or high-cost hardware resources, such as edge computing devices, edge inference devices, or cloud inference devices in high-concurrency scenarios. In these environments, hardware resources cannot guarantee that all inference requests will be computed using the highest precision mode. During peak request periods or when hardware resources are scarce, executing high-precision, low-rank enhancements according to the fixed path may significantly increase the latency of a single inference, leading to request timeouts or even system service unavailability. Conversely, during off-peak request periods or when hardware resources are abundant, executing low-precision, low-rank enhancements according to the fixed path cannot fully utilize idle hardware computing power to improve inference quality.

[0014] It is evident that the fixed low-rank enhancement path in the relevant technologies lacks elastic scheduling capabilities, making it difficult for the system to balance inference efficiency and inference quality under deployment conditions with fluctuating resources, thus reducing the availability and adaptability of inference devices in resource-sensitive scenarios.

[0015] Based on this, this application provides a unified base model multi-task inference method for resource-sensitive deployment environments, so as to balance the inference efficiency and inference quality of multi-task.

[0016] Please refer to Figure 1 , Figure 1 A flowchart illustrating the multi-task inference process of a unified base model for resource-sensitive deployment environments provided in this application embodiment.

[0017] In this embodiment, the method can be applied to an inference device configured to run a unified base model, which includes an original weight matrix and independently configured low-rank enhancement branches.

[0018] For example, the inference device here can be an electronic device with relatively limited hardware resources or high cost, such as an edge computing device, an end-side inference device, or a cloud inference device in a high-concurrency scenario.

[0019] In this embodiment, the inference device is configured to run a unified base model (such as a large language model or a multimodal model) as a backbone network shared by multiple tasks.

[0020] At the model architecture level, the linear transformation layer in the unified base model contains the original weight matrix W, which has fixed parameters obtained during large-scale pre-training. Based on this, the base model also has an independently set low-rank augmentation branch. That is, the low-rank augmentation branch and the original weight matrix W are two independent computational paths. The original weight matrix W forms one path, and the low-rank augmentation branch forms another parallel path. Both transform the input data and are then superimposed at the output.

[0021] Specifically, the low-rank enhancement branch adopts a low-rank adaptation structure. Taking LoRA (Low-Rank Adaptation) as an example, this branch contains the product of two low-rank matrices, with a parameter count much smaller than the original weight matrix. During inference, the output of the low-rank branch is superimposed on the output of the original weight matrix W, thereby achieving the effect of injecting specific task capabilities into the base model with a minimal number of parameters.

[0022] It should be noted that, unlike the solutions in related technologies, the low-rank enhancement branch in this application is not a single fixed branch, but a composite structure that includes a shared low-rank base and task-specific low-rank compensation. Its specific composition and activation method will be detailed below.

[0023] like Figure 1 As shown, the method may include the following steps: Step 101: Generate a rank configuration instruction based on the hardware resource load parameters of the inference device and the task to be executed.

[0024] The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch; the low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation, and the effective rank dimension includes the first target rank level in the shared low-rank base and the second target rank level in the task-specific low-rank compensation; different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated.

[0025] In this embodiment, hardware resource load parameters are used to characterize the current resource usage of the inference device, and may include, but are not limited to, at least one of the following: GPU utilization, CPU utilization, video memory usage, RAM usage, current concurrent requests, or available computing power. These hardware resource load parameters can be obtained in real time by the operating system or resource monitoring module of the inference device and used as input for the generation of subsequent rank configuration instructions.

[0026] In this embodiment, the low-rank enhancement branch includes a shared low-rank basis and a task-specific low-rank compensation, both of which are implemented in the form of a low-rank matrix.

[0027] Specifically, the enhancement term sharing a low-rank basis can be expressed as: Where As is the shared parameter matrix and Bs is the shared mapping matrix. The task-specific low-rank compensation enhancement term can be expressed as: , where At is the task-specific parameter matrix and Bt is the task-specific mapping matrix.

[0028] In practical deployments, to further reduce the number of parameters, At can be set to As, meaning all tasks share the same shared parameter matrix As. Therefore, the task-specific low-rank compensation enhancement term can be rewritten as follows: In other words, the task-specific low-rank compensation structurally comprises a cascaded shared parameter matrix As and a task-specific mapping matrix Bt. Similarly, the shared low-rank basis comprises a cascaded shared parameter matrix As and a shared mapping matrix Bs.

[0029] In this embodiment, the shared parameter matrix As is used to compress the input data into low-dimensional intermediate features, that is, to perform dimensionality reduction and common feature extraction on the input data; the shared mapping matrix Bs is used to map the intermediate features into shared output features; and the task-specific mapping matrix Bt is used to map the intermediate features into task-specific output features for a specific task.

[0030] In other words, the input data x is first compressed into intermediate features by the shared parameter matrix As. These intermediate features are then fed into the shared mapping matrix Bs or the corresponding task-specific mapping matrix Bt for mapping, ultimately yielding the corresponding output features. Since As is shared by all tasks, in multi-task deployments, only one copy of the shared parameter matrix As and the task-specific mapping matrices Bt for each task need to be stored, significantly reducing parameter storage and making it more suitable for resource-constrained deployment environments.

[0031] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the low-rank enhancement branch provided in an embodiment of this application.

[0032] like Figure 2 As shown, the input x flows into three computation paths simultaneously: The first path is the original weight matrix W path, where the input x is transformed by W and directly participates in the superposition output.

[0033] The second path is a shared low-rank basis path. The input x passes through the shared parameter matrix A_s and the shared mapping matrix B_s in sequence, and the output is... This path is shared by all tasks.

[0034] The third path is a task-specific low-rank compensation path. The input x passes through the shared parameter matrix A_s and the task-specific mapping matrix B_t in sequence, and the output is... This path is only used by the current task.

[0035] Furthermore, if the target task is determined to be a difficult task during the training phase and a private low-rank reinforcement subbasis (which can be denoted as a private subbasis) is added, then there is a fourth path: input x via the private subbasis. Transformation, output is This path is unique to difficult tasks, and the private subbases here will be detailed below, and will not be repeated here.

[0036] The outputs of the four paths are superimposed at the end to obtain the final inference result. .

[0037] In the above formula, λ_s, λ_t, and λ_p represent the activation coefficients of the shared low-rank basis, the task-specific low-rank compensation, and the private low-rank enhancement sub-basis, respectively. These activation coefficients are used to control whether the corresponding low-rank branch participates in the superposition calculation of the current inference: when the activation coefficient is 1, it means that the branch is activated and its output participates in the superposition; when the activation coefficient is 0, it means that the branch is closed and its output does not participate in the superposition.

[0038] Here, λ_s and λ_t can be determined by the effective rank dimension in the rank configuration instruction: when the effective rank dimension is non-zero, the corresponding activation coefficient is 1; when the effective rank dimension is zero, the corresponding activation coefficient is 0. λ_p can be determined by whether the current task has a private low-rank augmenting subbase: if the current task has a private subbase added during the training phase, then λ_p=1; otherwise, λ_p=0.

[0039] When the private low-rank enhancer subbase is not activated or does not exist When the term is zero, the formula degenerates into: This means that it consists of three superimposed paths: the original weight matrix path, the shared low-rank basis path, and the task-specific low-rank compensation path.

[0040] It should be noted that the activation coefficient and the rank are two independent control dimensions: the activation coefficient determines whether the corresponding low-rank branch is on or off, and the rank determines how many channels are used when the branch is on. Together, they enable flexible control over the low-rank enhancement branch.

[0041] In this embodiment, the rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. Specifically, the low-rank enhancement branch includes a shared low-rank base and task-specific low-rank compensation, therefore the effective rank dimension further includes a first target rank level corresponding to the shared low-rank base and a second target rank level corresponding to the task-specific low-rank compensation. Here, the rank level refers to the number of data channels actually involved in the calculation of the low-rank matrix, and different rank levels correspond to different numbers of ranks to be activated.

[0042] For example, if the preset candidate ranks include 2, 4, 8, and 16, then rank 2 means that only 2 data channels are activated for computation, and rank 16 means that all 16 data channels are activated for computation. The lower the rank, the fewer parameter channels are involved in the computation, and the less computation is required; the higher the rank, the more parameter channels are involved in the computation, and the more computation is required.

[0043] By assigning the shared low-rank basis and the task-specific low-rank compensation to independent target rank levels, the computational intensity of both can be adjusted independently, thus providing finer control granularity in resource scheduling.

[0044] It should be noted that after training, the data channels in the shared low-rank basis and task-specific low-rank compensation can be sorted according to their contribution to inference accuracy, with channels contributing more being sorted first. When using a lower rank, only the data channels sorted first are activated to participate in the calculation, thereby reducing the amount of computation while preserving the feature dimensions that contribute the most to inference accuracy as much as possible.

[0045] As one embodiment, a specific method for generating rank configuration instructions based on the hardware resource load parameters of the inference device and the task to be executed may include: Based on the task attributes of the task to be executed, determine whether the task to be executed is a general task or a special task; When the task to be executed is a general task, generate a rank configuration instruction indicating that the effective rank dimension is zero to close the low-rank enhancement branch; When the task to be executed is a special task, generate a rank configuration instruction that indicates the effective rank dimension of the low-rank enhancement branch to be a non-zero value.

[0046] Specifically, based on the task attributes of the task to be executed, it is determined whether the task is a general task or a special task. General tasks refer to tasks that can be completed using only the original general capabilities of the unified base model, such as general question answering and general text generation; special tasks refer to tasks that require specific task capabilities to complete, such as contract clause extraction, sentiment classification, and summary generation in a specific format.

[0047] The determination of task attributes can be achieved by reading the task identifier carried in the inference request, or by using a preset task type classifier to identify the request content in real time. This application does not impose any restrictions on this.

[0048] When it is determined that the task to be executed belongs to a general task, the inference device generates a rank configuration instruction indicating that the effective rank dimension is zero. This instruction is used to disable low-rank enhancement branches, so that the inference process only transforms the input data and outputs it using the original weight matrix.

[0049] In other words, for general tasks, the model can degenerate into the original base model without adding any low-rank enhancements. Thus, the original general capabilities of the base model are fully preserved, eliminating the need to deploy an untuned base model.

[0050] When it is determined that the task to be executed belongs to a special task, the inference device generates a rank configuration instruction that indicates that the effective rank dimension of the low-rank enhancement branch is a non-zero value, that is, the low-rank enhancement branch is enabled, and the output of the low-rank enhancement branch is superimposed on the output of the original weight matrix.

[0051] In this embodiment, determining the specific effective rank dimension for a dedicated task can further include the following two different implementation methods: (1) Determine the current load level of the inference device based on the hardware resource load parameters; determine the target rank from multiple preset candidate rank levels based on the load level, with different candidate rank levels corresponding to different numbers of rank to be activated; wherein the load level and the target rank level are negatively correlated; and use the target rank level as the first target rank level and the second target rank level.

[0052] In this embodiment, the inference device determines the current load level based on hardware resource load parameters, and further determines the target rank from a plurality of preset candidate rank levels based on the load level. The target rank is used as both the first target rank (i.e., the rank sharing the low rank base) and the second target rank (i.e., the rank with task-specific low rank compensation).

[0053] There is a negative correlation between load level and target rank; that is, the higher the load level (the more strained the resources), the lower the target rank; and the lower the load level (the more abundant the resources), the higher the target rank.

[0054] In this embodiment, the inference device can pre-configure a mapping table between load levels and candidate rank levels. For example, when the load level is high, the corresponding candidate rank level is 2; when the load level is medium, the corresponding candidate rank level is 8; and when the load level is low, the corresponding candidate rank level is 16. It should be noted that the above values ​​are only illustrative examples, and in actual applications, they can be flexibly set according to the hardware configuration of the inference device and the specific deployment scenario. This application does not impose any limitations on this.

[0055] In this implementation, the shared low-rank base and the task-specific low-rank compensation use the same rank level, making the control logic relatively simple and suitable for most conventional deployment scenarios.

[0056] (2) Determine the current load level of the inference device based on the hardware resource load parameters; determine the first target rank from a set of multiple candidate rank levels based on the load level, with different candidate rank levels corresponding to different numbers of rank levels to be activated; wherein the load level is negatively correlated with the first target rank level; determine the complexity level of the task to be executed based on the task complexity; determine the second target rank from a set of multiple candidate rank levels based on the complexity level, with the complexity level being positively correlated with the second target rank level.

[0057] In this embodiment, the inference device determines the current load level based on hardware resource load parameters, and determines a first target rank based on the load level (the load level and the first target rank are negatively correlated). At the same time, it determines the complexity level of the task to be executed based on the task complexity, and determines a second target rank based on the complexity level (the complexity level and the second target rank are positively correlated).

[0058] In this approach, the first objective rank and the second objective rank are determined independently of each other; they can be the same or different.

[0059] As an example, task complexity can be determined based on at least one of the following factors: the length of the input data (e.g., the number of tokens in the input sequence), the number of inference steps required by the task, and the required level of output accuracy. For instance, for short text classification tasks, the complexity is low, and the corresponding second objective rank can be set to a smaller value; while for complex information extraction tasks from long documents, the complexity is high, and the corresponding second objective rank can be set to a larger value.

[0060] This approach enables independent adjustment of the shared low-rank basis and task-specific low-rank compensation, making it suitable for scenarios with high requirements for the fine-grained nature of resource scheduling.

[0061] As one embodiment, the generation of the rank configuration instruction is further adjusted based on the task priority and / or context state of the task to be executed. The task priority indicates the importance of the current inference request. When the task priority is higher than a preset priority threshold, the effective rank dimension can be adjusted upwards from the candidate rank levels corresponding to the load level or complexity level to ensure the inference quality of high-priority tasks. When the task priority is lower than the preset priority threshold, the effective rank dimension can be adjusted downwards from the candidate rank levels to further save computational resources. The context state indicates whether the current inference request shares input prefixes or context information with previous inference requests. When reusable context information exists, the effective rank dimension can be adjusted downwards from the candidate rank levels to utilize already computed intermediate features and reduce redundant computation.

[0062] After determining the first target rank and the second target rank, the inference device generates a rank configuration instruction containing the aforementioned target rank information and outputs the instruction to the switching module of the low-rank enhancement branch to trigger subsequent low-rank configuration switching operations.

[0063] This concludes the description of step 101.

[0064] Step 102: In response to the rank configuration instruction, switch the low-rank enhancement branch in the unified base model to the low-rank configuration corresponding to the effective rank dimension.

[0065] In this embodiment, the rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch, including the first target rank level corresponding to the shared low-rank base and the second target rank level corresponding to the task-specific low-rank compensation. In response to this instruction, the inference device switches the current low-rank enhancement branch configuration to the target low-rank configuration corresponding to the effective rank dimension. Here, the low-rank configuration refers to the combination of data channels actually involved in the calculation of each parameter matrix in the low-rank enhancement branch. Since different rank levels correspond to different numbers of data channels to be activated, switching the low-rank configuration essentially adjusts the number of parameter channels actually involved in the calculation in the low-rank enhancement branch.

[0066] In this embodiment, the enhancement term of the low-rank enhancement branch can be represented as: , where r represents the currently active rank.

[0067] Here, A can be a shared parameter matrix A_s, and B can be a shared mapping matrix B_s or a task-specific mapping matrix B_t. The symbol ":" indicates taking all elements of that dimension, and ":r" indicates taking the first r elements of that dimension. Therefore, B[:,:r] means taking all rows of matrix B but only the first r columns, and A[:r,:] means taking the first r rows of matrix A but all columns. Multiplying the two yields a low-rank enhancement term ΔW(r) with an effective rank of r. When r equals the preset total rank, all channels participate in the calculation; when r is less than the preset total rank, only the first r channels participate in the calculation, and the remaining channels are truncated and do not participate in this inference.

[0068] Specifically, as r changes, the actual matrix dimension involved in the calculation changes accordingly. The smaller r is, the fewer parameter channels are involved in the calculation, and the less computation is required; the larger r is, the more parameter channels are involved in the calculation, and the more computation is required. Therefore, switching to the low-rank configuration corresponding to the effective rank dimension means adjusting the number of channels actually activated in the shared low-rank base to the number corresponding to the first target rank, and adjusting the number of channels actually activated in the task-specific low-rank compensation to the number corresponding to the second target rank.

[0069] During continuous operation of the inference device, when a new inference request is received, the device generates a corresponding rank configuration instruction based on the task type and current hardware resource status, and switches the current low-rank enhancement branch configuration to the target configuration. This switching process is performed dynamically during the operation of the inference device, without stopping the service or reloading the entire model. Specifically, taking the switching of a shared low-rank base as an example, if the current rank of the shared low-rank base is r1 and the target rank is r2, then the switching operation is to switch the number of effective channels in the shared low-rank base from r1 to r2, that is, from the current configuration ΔW(r1) to the target configuration ΔW(r2). After the switching is completed, subsequent inference computations will use the target configuration to perform low-rank enhancements.

[0070] It should be noted that in scenarios involving multi-task switching, switching of low-rank configurations also involves replacing the shared low-rank base and task-specific low-rank compensation.

[0071] Combined with the output formula ; As can be seen from the formula, the original weight matrix W and the shared parameter matrix A_s are shared by multiple tasks, while the task-specific mapping matrix B_t is independent of each task. Therefore, whether the shared parameter matrix A_s needs to be changed when switching between different tasks depends on whether the two tasks belong to the same shared low-rank basis set.

[0072] Specifically, when switching from the first task to the second task, the inference device first determines whether the first shared low-rank basis corresponding to the first task and the second shared low-rank basis corresponding to the second task are the same shared low-rank basis. If they are the same shared low-rank basis, that is, the two tasks share the same shared parameter matrix A_s and the same shared mapping matrix B_s, then the shared low-rank basis remains unchanged, and only the task-specific low-rank compensation is switched from the first-specific compensation (i.e., B_t1) corresponding to the first task to the second-specific compensation (i.e., B_t2) corresponding to the second task. At this time, the change in the inference formula is only reflected in the replacement of the task-specific mapping matrix: B_t changes from B_t1 to B_t2, while A_s and B_s remain unchanged.

[0073] If the two tasks use different shared low-rank bases—that is, the first task uses the first shared parameter matrix A_s1 and the first shared mapping matrix B_s1, while the second task uses the second shared parameter matrix A_s2 and the second shared mapping matrix B_s2—then the switching operation requires changing the shared low-rank base from the first shared low-rank base to the second shared low-rank base (i.e., A_s changes from A_s1 to A_s2, and B_s changes from B_s1 to B_s2), and simultaneously changing the task-specific low-rank compensation from the first specific compensation to the second specific compensation (i.e., B_t changes from B_t1 to B_t2). In this case, the shared parameter matrix A_s, the shared mapping matrix B_s, and the task-specific mapping matrix B_t in the inference formula are all replaced.

[0074] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating intra-group and cross-group switching scenarios provided in the embodiments of this application.

[0075] like Figure 3 As shown, Task 1 and Task 2 share the same shared low-rank basis A_s1 (containing the shared mapping matrix B_s1). Task 1 has its own dedicated mapping matrix B_t1, and Task 2 has its own dedicated mapping matrix B_t2. Task 3 and Task 4 belong to different shared low-rank basis sets. Task 3 uses the shared low-rank basis A_s1 (shared mapping matrix B_s1) and its own dedicated mapping matrix B_t3, while Task 4 uses the shared low-rank basis A_s2 (shared mapping matrix B_s2) and its own dedicated mapping matrix B_t4.

[0076] When switching from Task 1 to Task 2, it is a switch within the same group. In this case, the shared low-rank basis A_s1 remains unchanged, and only the task-specific low-rank compensation is switched from B_t1 to B_t2.

[0077] When switching from Task 1 to Task 3, which is a cross-group switch, the shared low-rank basis A_s1 is switched to A_s2 (the corresponding shared mapping matrix B_s1 is switched to the shared mapping matrix B_s2), and the task-specific low-rank compensation is switched from B_t1 to B_t3.

[0078] During the aforementioned switching process, regardless of whether it's a switch within the same group or across groups, the original weight matrix W remains unchanged and does not require reloading. In the case of a switch within the same group, the shared low-rank basis (including the shared parameter matrix A_s and the shared mapping matrix B_s) also remains unchanged; only the task-specific mapping matrix B_t needs to be replaced, minimizing the switching overhead. This mechanism makes multi-task switching within the same shared low-rank basis group quite efficient, especially suitable for scenarios with shared inputs, shared prefixes, or shared contexts. In the case of a switch across groups, although it is necessary to switch both the shared low-rank basis and the task-specific low-rank compensation simultaneously, the original weight matrix W remains unchanged, still avoiding the significant data transfer and reloading overhead associated with switching the entire model.

[0079] As can be seen, the above switching mechanism makes multi-task switching no longer a whole model switching, but a low-rank branch configuration switching and intensity switching on a unified base, which significantly reduces task switching latency and resource consumption.

[0080] This concludes the description of step 102.

[0081] Step 103: The output of the original weight matrix to the input data in the unified base model and the calculation results of the low-rank configuration to the input data are superimposed to obtain the target inference result.

[0082] In this embodiment, after the inference device switches the low-rank enhancement branch to the low-rank configuration corresponding to the effective rank dimension according to the rank configuration instruction, it can perform the inference calculation of the current task.

[0083] The output of the inference device consists of two superimposed parts: the first part is the transformation result of the original weight matrix in the unified base model on the input data, and the second part is the operation result of the low-rank configuration on the same input data.

[0084] In this embodiment, the output of a certain linear transformation layer in the base model can be expressed as: , where Wx is the transformation result of the original weight matrix, and ΔW(r)·x is the enhancement result of the low-rank enhancement branch at rank r.

[0085] As before, the low-rank enhancement branch includes a shared low-rank base and task-specific low-rank compensation, therefore the above formula can be further expanded as follows: λ_s and λ_t are the activation coefficients of the shared low-rank basis and the task-specific low-rank compensation, respectively, determined by the effective rank dimension in the rank configuration instruction. When the effective rank is zero, the corresponding activation coefficient is zero, and this branch does not participate in the stacking. When the effective rank is non-zero, the corresponding branch participates in the stacking with the corresponding rank. Through the above stacking method, the model can flexibly inject task-specific enhancement capabilities while retaining its original general capabilities.

[0086] In the implementation, the original weight matrix W transforms the input x into Wx. This path always exists and remains unchanged regardless of task switching or rank changes, thus ensuring the original general capabilities of the base model are always available. The low-rank configuration operates on the input x as ΔW(r)·x, and this path is activated only when the effective rank dimension indicated by the rank configuration instruction is non-zero. When the effective rank dimension is zero, ΔW(r)·x is a zero vector, and the model output degenerates to y=Wx, meaning only the original weight matrix is ​​used for transformation, without any added enhancement capabilities.

[0087] In one implementation, the shared low-rank basis and task-specific low-rank compensation described above are pre-trained in the following manner.

[0088] Acquire training data for each of the multiple tasks; use the training data from the multiple tasks to train a shared low-rank basis, and obtain the trained shared low-rank basis.

[0089] The goal of this stage is to enable the shared low-rank basis to learn common augmentation capabilities shared by multiple tasks, forming a common augmentation foundation, which can be represented as follows: .

[0090] Once the shared low-rank basis has been trained to a stable state, the parameter values ​​of the shared parameter matrix A_s in the shared low-rank basis are fixed, and the updating of this matrix is ​​stopped. Subsequently, using the training data of each task, while keeping the parameter values ​​of the shared parameter matrix A_s fixed, the task-specific low-rank compensation for each task is trained.

[0091] The form of task-specific low-rank compensation can be represented as follows: In this training method, each task learns its own specific mapping matrix B_t, while the shared parameter matrix A_s remains unchanged. Through this training approach, the shared parameter matrix A_s becomes a feature extractor common to all tasks, while each task's specific mapping matrix B_t learns its own task-specific mapping based on this shared feature extractor, thus achieving a balance between parameter reuse and task isolation.

[0092] Furthermore, during the training of low-rank compensation specific to each task, there may be situations where some tasks still cannot achieve the expected results by relying solely on a shared base plus specific compensation.

[0093] Therefore, this application also provides a preferred implementation. During the low-rank compensation process specific to the training task, the inference device or training device continuously monitors the performance metrics of each task on a preset validation set. These performance metrics may include, but are not limited to, loss function values, accuracy, or F1 scores.

[0094] When the performance indicators of a target task fail to improve to the preset threshold after multiple iterations, it indicates that the target task differs significantly from the common base, or that the target task is too difficult and cannot be compensated by the current shared base plus dedicated compensation alone.

[0095] At this point, a private low-rank augmentation sub-basis is added to the target task. This private low-rank augmentation sub-basis contains an independent parameter matrix and is not shared with other tasks. This private low-rank augmentation sub-basis is superimposed on the task-specific low-rank compensation corresponding to the target task, and together they serve as the final task-specific low-rank compensation for the target task. In formulaic form, after adding the private sub-basis, the final task-specific low-rank compensation can be written as: ,in This is the newly added private low-rank enhancer subbase.

[0096] Based on this, when the task-specific low-rank compensation corresponding to the task to be executed is superimposed with a private low-rank enhancement sub-basis, the output of superimposed inference is correspondingly expanded to a superposition of four parts: the output of the original weight matrix, the output of the shared low-rank basis, the output of the task-specific low-rank compensation, and the output of the private low-rank enhancement sub-basis.

[0097] In formulaic form, this reasoning process can be written as follows: .

[0098] Here, λ_p is the activation coefficient of the private low-rank augmenting subbase. λ_p is non-zero when the task has a private subbase and it is activated; otherwise, λ_p is zero. Through the above superposition method, difficult tasks containing private subbases can obtain stronger augmentation capabilities, while other tasks remain unaffected. This maintains the stability of the shared base while providing differentiated augmentation support for special tasks.

[0099] In this embodiment, regarding the effective rank adjustment method of the private low-rank augmentation sub-base, in one implementation, the private low-rank augmentation sub-base does not participate in the dynamic adjustment of the effective rank dimension, and it maintains full-rank activation during inference. Since the private low-rank augmentation sub-base is an independent augmentation path specifically added for difficult tasks during the training phase, its purpose is to provide maximum additional augmentation capabilities for the task. Therefore, in this embodiment, rank selection is not required to ensure that the difficult task can obtain complete private augmentation support.

[0100] In another implementation, a private low-rank enhancement subbase can also participate as an independent branch in the dynamic adjustment of the effective rank dimension, and its effective rank is determined independently based on the hardware resource load parameters.

[0101] This concludes the discussion on... Figure 1 The description.

[0102] This application generates a rank configuration instruction based on hardware resource load parameters. This instruction indicates the effective rank dimensions to be activated in the low-rank enhancement branch, with different rank levels corresponding to different numbers of rank dimensions to be activated. This allows the inference device to adjust the number of data channels actually involved in the computation of the low-rank enhancement branch in real time according to the current resource conditions. In response to this instruction, the low-rank enhancement branch is switched to the low-rank configuration corresponding to the current effective rank dimension, thereby changing the computational intensity of the enhancement branch without changing the base model. Therefore, the inference device can dynamically adjust the computational load of the low-rank enhancement branch under deployment conditions with fluctuating resources, solving the deficiency in related technologies where fixed low-rank enhancement paths cannot be adjusted with resource changes, and taking into account the inference needs under different resource conditions.

[0103] Please refer to Figure 4 , Figure 4 This is a structural diagram of a unified base model multi-task inference device for resource-sensitive deployment environments proposed in this application. The device is applied to an inference device, which is configured to run a unified base model. The unified base model includes an original weight matrix and independently configured low-rank enhancement branches. Figure 4 As shown, the device may include a generation unit 401, a switching unit 402, and an inference unit 403. Specifically, the device includes: The generation unit 401 is used to generate a rank configuration instruction based on the hardware resource load parameters of the inference device and the task to be executed. The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. The low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation. The effective rank dimension includes a first target rank level in the shared low-rank base and a second target rank level in the task-specific low-rank compensation. Different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated. Switching unit 402 is used to switch the low-rank enhancement branch in the unified base model to a low-rank configuration corresponding to the effective rank dimension in response to the rank configuration command. The inference unit 403 is used to superimpose the output of the original weight matrix to the input data in the unified base model and the calculation results of the low-rank configuration to the input data to obtain the target inference result.

[0104] As one embodiment, the generation unit 401 is specifically used for: Based on the task attributes of the task to be executed, determine whether the task to be executed is a general task or a special task; When the task to be executed is a general task, generate a rank configuration instruction indicating that the effective rank dimension is zero to close the low-rank enhancement branch; When the task to be executed is a special task, generate a rank configuration instruction that indicates the effective rank dimension of the low-rank enhancement branch to be a non-zero value.

[0105] As one embodiment, the generation unit 401 is specifically used for: Determine the current load level of the inference device based on the hardware resource load parameters; Based on the load level, the target rank is determined from a set of multiple candidate rank levels. Different candidate rank levels correspond to different numbers of ranks to be activated. The load level and the target rank are negatively correlated. The target rank is used as the first target rank and the second target rank.

[0106] As an example, generation unit 401: Determine the current load level of the inference device based on the hardware resource load parameters; Based on the load level, a first target rank is determined from a set of multiple candidate rank levels. Different candidate rank levels correspond to different numbers of ranks to be activated. The load level is negatively correlated with the first target rank. Determine the complexity level of the task to be executed based on its complexity. Based on the complexity level, a second target rank is determined from multiple pre-set candidate rank levels, and there is a positive correlation between the complexity level and the second target rank.

[0107] As one embodiment, the switching unit 402 is specifically used for: When switching from the first task to the second task, if the first shared low-rank basis corresponding to the first task and the second shared low-rank basis corresponding to the second task are the same shared low-rank basis, then the shared low-rank basis remains unchanged, and the task-specific low-rank compensation is switched from the first-specific compensation corresponding to the first task to the second-specific compensation corresponding to the second task. If the first shared low-rank basis and the second shared low-rank basis are different shared low-rank basis, then the shared low-rank basis will be switched from the first shared low-rank basis to the second shared low-rank basis, and the task-specific low-rank compensation will be switched from the first specific compensation to the second specific compensation.

[0108] As one example, the shared low-rank basis includes a cascaded shared parameter matrix and a shared mapping matrix; the task-specific low-rank compensation includes a cascaded shared parameter matrix and a task-specific mapping matrix. Among them, the shared parameter matrix is ​​used to compress the input data into intermediate features, the shared mapping matrix is ​​used to map the intermediate features into shared output features, and the task-specific mapping matrix is ​​used to map the intermediate features into task-specific output features.

[0109] As an example, the shared low-rank basis and task-specific low-rank compensation are trained in the following way: By using training data from multiple tasks, a shared low-rank basis is trained together to obtain the trained shared low-rank basis. The parameter values ​​of the shared parameter matrix in the fixed shared low-rank basis are used to train the task-specific low-rank compensation for each task using the training data of each task, so as to obtain the trained task-specific low-rank compensation for each task.

[0110] As one example, in the low-rank compensation specific to the training task, the following is also included: Monitor the performance metrics of each task on the preset validation set; When the performance metrics of the target task fail to reach the preset threshold after multiple iterations, a private low-rank enhancement subbase is added to the target task. The private low-rank enhancement subbase is then superimposed on the task-specific low-rank compensation corresponding to the target task, serving as the final task-specific low-rank compensation.

[0111] As an example, when a private low-rank enhancement subbase is superimposed on the task-specific low-rank compensation corresponding to the task to be executed, the inference unit is specifically used for: The outputs of the original weight matrix, the shared low-rank basis, the task-specific low-rank compensation, and the private low-rank enhancement sub-basis are superimposed to obtain the target inference result.

[0112] This concludes the discussion on... Figure 4 Description of the device.

[0113] This application also provides embodiments that... Figure 4 Hardware structure description of the illustrated device. This hardware structure is... Figure 5 The structure in the illustrated electronic device. Please refer to [link / reference]. Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0114] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0115] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, computer-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

Claims

1. A multi-task inference method for a unified base model in resource-sensitive deployment environments, characterized in that, The method is applied to an inference device configured to run a unified base model, which includes an original weight matrix and independently configured low-rank enhancement branches. The method includes: Based on the hardware resource load parameters of the inference device and the task to be executed, a rank configuration instruction is generated. The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. The low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation. The effective rank dimension includes a first target rank level in the shared low-rank base and a second target rank level in the task-specific low-rank compensation. Different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated. In response to the rank configuration instruction, the low-rank enhancement branch in the unified base model is switched to a low-rank configuration corresponding to the effective rank dimension; The target inference result is obtained by superimposing the output of the original weight matrix to the input data in the unified base model and the calculation result of the low-rank configuration to the input data.

2. The method according to claim 1, characterized in that, The step of generating a rank configuration instruction based on the hardware resource load parameters of the inference device and the task to be executed includes: Based on the task attributes of the task to be executed, determine whether the task to be executed is a general task or a special task; When the task to be executed belongs to the general task, a rank configuration instruction indicating that the effective rank dimension is zero is generated to close the low-rank enhancement branch; When the task to be executed belongs to the dedicated task, a rank configuration instruction is generated that indicates the effective rank dimension of the low-rank enhancement branch is a non-zero value.

3. The method according to claim 2, characterized in that, The generation of rank configuration instructions that indicate the effective rank dimension of the low-rank enhancement branch is a non-zero value includes: Based on the hardware resource load parameters, determine the current load level of the inference device; Based on the load level, a target rank is determined from a plurality of preset candidate rank levels, with different candidate rank levels corresponding to different numbers of ranks to be activated; wherein, the load level and the target rank are negatively correlated. The target rank is used as the first target rank and the second target rank.

4. The method according to claim 2, characterized in that, The generation of rank configuration instructions that indicate the effective rank dimension of the low-rank enhancement branch is a non-zero value includes: Based on the hardware resource load parameters, determine the current load level of the inference device; Based on the load level, the first target rank is determined from a plurality of preset candidate rank levels, and different candidate rank levels correspond to different numbers of ranks to be activated; wherein, the load level is negatively correlated with the first target rank. Based on the task complexity of the task to be executed, determine the complexity level of the task to be executed; Based on the complexity level, the second target rank is determined from a plurality of preset candidate rank levels, wherein the complexity level and the second target rank are positively correlated.

5. The method according to claim 1, characterized in that, The step of switching the low-rank enhancement branch in the unified base model to a low-rank configuration corresponding to the effective rank dimension in response to the rank configuration instruction includes: When switching from the first task to the second task, if the first shared low-rank basis corresponding to the first task and the second shared low-rank basis corresponding to the second task are the same shared low-rank basis, then the shared low-rank basis is kept unchanged, and the task-specific low-rank compensation is switched from the first-specific compensation corresponding to the first task to the second-specific compensation corresponding to the second task. If the first shared low-rank basis and the second shared low-rank basis are different shared low-rank basis, then the shared low-rank basis is switched from the first shared low-rank basis to the second shared low-rank basis, and the task-specific low-rank compensation is switched from the first specific compensation to the second specific compensation.

6. The method according to claim 1, characterized in that, The shared low-rank basis comprises a shared parameter matrix and a shared mapping matrix connected in series; the task-specific low-rank compensation comprises the shared parameter matrix and the task-specific mapping matrix connected in series. The shared parameter matrix is ​​used to compress input data into intermediate features, the shared mapping matrix is ​​used to map the intermediate features into shared output features, and the task-specific mapping matrix is ​​used to map the intermediate features into task-specific output features.

7. The method according to claim 6, characterized in that, The shared low-rank basis and the task-specific low-rank compensation are obtained through training in the following manner: The shared low-rank basis is trained using training data from multiple tasks to obtain the trained shared low-rank basis. The parameter values ​​of the shared parameter matrix in the shared low-rank basis are fixed, and the task-specific low-rank compensation corresponding to each task is trained using the training data of each task, so as to obtain the trained task-specific low-rank compensation corresponding to each task.

8. The method according to claim 7, characterized in that, When training the task-specific low-rank compensation, the method further includes: Monitor the performance metrics of each task on the preset validation set; When the performance index of the target task fails to reach the preset threshold after multiple iterations, a private low-rank enhancement sub-base is added to the target task. The private low-rank enhancement sub-base is then superimposed on the task-specific low-rank compensation corresponding to the target task, serving as the final task-specific low-rank compensation for the target task.

9. The method according to claim 8, characterized in that, When the private low-rank enhancement sub-basis is superimposed in the task-specific low-rank compensation corresponding to the task to be executed, the superposition of the output of the original weight matrix of the unified base model to the input data and the calculation result of the low-rank configuration on the input data includes: The outputs of the original weight matrix, the shared low-rank basis, the task-specific low-rank compensation, and the private low-rank enhancement sub-basis are superimposed to obtain the target inference result.

10. A unified base model multi-task inference device for resource-sensitive deployment environments, characterized in that, The device is applied to an inference device configured to run a unified base model, which includes an original weight matrix and independently configured low-rank enhancement branches. The device includes: A generation unit is configured to generate a rank configuration instruction based on the hardware resource load parameters of the inference device and the task to be executed. The rank configuration instruction is used to indicate the effective rank dimension to be activated in the low-rank enhancement branch. The low-rank enhancement branch includes a shared low-rank base and a task-specific low-rank compensation. The effective rank dimension includes a first target rank level in the shared low-rank base and a second target rank level in the task-specific low-rank compensation. Different rank levels in the shared low-rank base correspond to different numbers of ranks to be activated, and different rank levels in the task-specific low-rank compensation correspond to different numbers of ranks to be activated. A switching unit is configured to, in response to the rank configuration instruction, switch the low-rank enhancement branch in the unified base model to a low-rank configuration corresponding to the effective rank dimension. The inference unit is used to superimpose the output of the original weight matrix of the unified base model on the input data and the calculation result of the low-rank configuration on the input data to obtain the target inference result.

11. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute the machine-executable instructions to implement the method as described in any one of claims 1 to 9.

Images