Task processing methods and devices, storage media, electronic devices

Abstract

This application discloses a task processing method, apparatus, storage medium, and electronic device. The method includes: acquiring joint perception information of the task to be reasoned and its corresponding environment; when the joint perception information of the environment and task meets preset conditions, analyzing the task to be reasoned using a target small member model within a pre-trained family model to obtain an intermediate handover packet output by the target key intermediate layer of the target small member model; analyzing the intermediate handover packet using a target large member model within the family model to obtain a first answer output by the task output header of the target large member model, and outputting the first answer as the target answer for the task to be reasoned. This application solves the technical problems of low analysis efficiency and serious waste of computing resources when using an edge-cloud architecture to analyze edge-side reasoning tasks in related technologies.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a task processing method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] In the existing three-tiered "cloud-edge-device" architecture, lightweight models such as convolutional neural networks are typically deployed on the device side for rapid initial judgment of real-time video frames or sensor data; while high-performance models such as large language models or residual neural networks are typically deployed on the edge side or in the cloud for high-precision re-examination. However, this architecture faces the following core technical challenges in practical applications (such as fall detection for the elderly):

[0003] (1) Due to the limited number of model parameters, the edge-side model has a high false alarm rate. To achieve high-precision re-inspection in the cloud, the edge must completely upload the original video frames, sensor time-series data (such as acceleration, attitude angle, body temperature, etc.) or their high-dimensional feature representations to the edge-side or cloud-side model. On the one hand, because these data contain highly sensitive personal physiological states, behavioral trajectories and life scene information, there is a serious risk of privacy leakage during transmission. Even with encrypted transmission or differential privacy technology, it is still difficult to avoid the potential risks of unauthorized access, data abuse or leakage on the edge-side or cloud-side, especially in scenarios involving vulnerable groups such as home security and elderly care. On the other hand, after receiving the original video frames, sensor time-series data or their high-dimensional feature representations, the edge-side or cloud-side model will repeatedly calculate the spatiotemporal features extracted by the edge-side, resulting in bandwidth waste and response delay.

[0004] (2) The end-side model and the edge / cloud model have a fragmented structure and inconsistent feature space. Therefore, the output of the end-side model cannot be directly reused by the edge / cloud model and must be re-encoded, resulting in low efficiency of collaborative processing.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides a task processing method and apparatus, storage medium, and electronic device to at least solve the technical problems of low analysis efficiency and serious waste of computing resources when using an edge-cloud architecture to analyze inference tasks on the edge side in related technologies.

[0007] According to one aspect of the embodiments of this application, a task processing method is provided, comprising: acquiring a task to be reasoned and environmental task joint perception information of the task to be reasoned, wherein the environmental task joint perception information includes at least one of the following: network state information, local available computing resource information, and task meta-information; when the environmental task joint perception information meets preset conditions, analyzing the task to be reasoned using a target small member model within a pre-trained family model to obtain an intermediate handover packet output by the target key intermediate layer of the target small member model, wherein the intermediate handover packet includes at least: intermediate hidden states output by the target key intermediate layer, multiple member models in the family model share the same task output head, and the types of member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model; analyzing the intermediate handover packet using a target large member model within the family model to obtain a first answer output by the task output head of the target large member model, and outputting the first answer as the target answer of the task to be reasoned, wherein the local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer.

[0008] According to another aspect of the embodiments of this application, a task processing apparatus is also provided, comprising: an acquisition module, configured to acquire a task to be reasoned and environmental task joint perception information of the task to be reasoned, wherein the environmental task joint perception information includes at least one of the following: network state information, local available computing resource information, and task meta-information; a first analysis module, configured to, when the environmental task joint perception information meets preset conditions, analyze the task to be reasoned using a target small member model within a pre-trained family model, and obtain an intermediate handover packet output by the target key intermediate layer of the target small member model, wherein the intermediate handover packet includes at least: an intermediate hidden state output by the target key intermediate layer, multiple member models in the family model share the same task output head, and the types of member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model; and a second analysis module, configured to analyze the intermediate handover packet using a target large member model within the family model, obtain a first answer output by the task output head of the target large member model, and output the first answer as the target answer of the task to be reasoned, wherein the local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer.

[0009] In an exemplary embodiment, the above-described apparatus is further configured to train a family model by means of: constructing a base model; decomposing multiple intermediate networks within the base model to obtain multiple extended branch networks corresponding to each intermediate network, and determining multiple homologous models by the multiple extended branch networks corresponding to each intermediate network, wherein the intermediate network is a sub-network deployed between the exit point and the task output head of the key intermediate layer of the base model; determining the first predicted answer obtained by the base model from analyzing multiple inference samples and the second predicted answer obtained by the multiple homologous models from analyzing multiple inference samples, and constructing a target loss function based on the standard answer of each of the multiple inference samples, the corresponding first predicted answer and the second predicted answer; adjusting the parameters of the base model and the multiple homologous models using the target loss function, and using the trained multiple homologous models and the trained base model as multiple member models to form a family model.

[0010] In an exemplary embodiment, the above-described apparatus is further configured to decompose multiple intermediate networks within a base model to obtain multiple extended branch networks corresponding to each intermediate network by the following method: for each intermediate network within the base model, determining the input feature vector of the inference sample in each set of training sample data at the key intermediate layer, and forming the input activation matrix of the intermediate network by each input feature vector; performing centering and covariance whitening on the input activation matrix, and performing linear regression processing in combination with the initial weight matrix of the intermediate network to obtain the regression matrix corresponding to the intermediate network; performing singular value decomposition on the regression matrix to obtain multiple singular values, sorting the multiple singular values ​​in descending order, and determining the first r target singular values ​​in the sorting result, forming multiple low-rank weight matrices by the r target singular values ​​and the feature vector corresponding to each target singular value, where r is a positive integer greater than or equal to 1; and constructing multiple extended branch networks corresponding to the intermediate network based on the multiple low-rank weight matrices.

[0011] In an exemplary embodiment, the above-described apparatus is further configured to construct a target loss function based on the standard answers, first predicted answers, and second predicted answers of multiple inference samples by the following method: constructing a global loss function based on the standard answers of each of the multiple inference samples and the corresponding first predicted answers and second predicted answers; for each intermediate network, constructing a branch loss function corresponding to the intermediate network based on the standard answers of each of the multiple training samples and the second predicted answers output by each of the multiple homologous models obtained by decomposing the same intermediate network; and performing a weighted summation of the global loss function and the multiple branch loss functions to obtain the target loss function.

[0012] In an exemplary embodiment, the intermediate handover packet further includes: identification information of the exit point of the target key intermediate layer. The aforementioned apparatus is further configured to analyze the intermediate handover packet using the target large member model within the family model, and obtain a first response output by the task output head of the target large member model, including: determining a target sub-model within the target large member model based on the identification information, wherein the target sub-model consists of multiple extended branch networks and a task output head directly connected to the exit point of the target key intermediate layer within the target large member model; and inputting the intermediate hidden state from the intermediate handover packet into the target sub-model to obtain the first response output by the task output head of the target sub-model.

[0013] In an exemplary embodiment, the above-described apparatus is further configured to input the intermediate hidden state in the intermediate handover packet into the target sub-model to obtain the first answer output by the task output head of the target sub-model by means of: randomly inputting the intermediate hidden state in the intermediate handover packet into any extended branch network directly connected to the exit point of the target key intermediate layer in the target sub-model, and inputting the intermediate hidden state output by the extended branch network into the task output head to obtain the first answer output by the task output head.

[0014] In an exemplary embodiment, the above-described apparatus is further configured to, when the environmental task joint perception information does not meet preset conditions, analyze the reasoning task using the target small member model within the family model, obtain the second answer output by the task output head of the target small member model, and output the second answer as the target answer for the reasoning task.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein the computer-readable storage medium, when executed by a processor, performs the steps in any of the above method embodiments.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the steps of any of the above method embodiments through the computer program.

[0017] In this embodiment, a distributed intelligent inference paradigm of "fast front-end output, deep back-end inference, and lossless intermediate connection" is constructed to achieve accurate reuse and collaborative inference of intermediate states between models of different scales. While ensuring inference quality, it significantly reduces edge-side energy consumption and cloud load, realizing a highly efficient, scalable, and low-latency inference system with cloud-edge-device collaboration. This solves the technical problems of low analysis efficiency and serious waste of computing resources when using an edge-cloud architecture to analyze edge-side inference tasks in related technologies. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a flowchart illustrating an optional task processing method according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the training process of an optional family model according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of an optional exit point according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of an optional first answer output process according to an embodiment of this application;

[0023] Figure 5 This is a structural block diagram of an optional task processing device according to an embodiment of this application;

[0024] Figure 6 This is a structural block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] According to an embodiment of this application, a task processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0028] Figure 1 This is a flowchart illustrating a task processing method according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0029] Step S102: Obtain the joint perception information of the task to be reasoned and its environment, wherein the joint perception information of the environment includes at least one of the following: network status information, local available computing resource information, and task meta information.

[0030] Step S104: When the joint perception information of the environment and task meets the preset conditions, the target small member model in the pre-trained family model is used to analyze the reasoning task to be performed, and the intermediate handover packet of the target key intermediate layer output of the target small member model is obtained. The intermediate handover packet includes at least: the intermediate hidden state output by the target key intermediate layer. Multiple member models in the family model share the same task output head. The types of member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model.

[0031] Step S106: Analyze the intermediate handover packet using the target large member model within the family model to obtain the first answer output by the task output head of the target large member model, and output the first answer as the target answer for the reasoning task. The local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer.

[0032] Optionally, in the embodiments of this application, the above-mentioned reasoning task is an intelligent perception or generation task initiated by the terminal device, which includes, but is not limited to, human action recognition in video frames, anomaly detection of sensor time-series data, intent understanding of voice commands, semantic segmentation of image content, or real-time response requests for multimodal scenes.

[0033] Optionally, in this embodiment, the aforementioned environmental task joint perception information is a comprehensive representation of the external and internal operating conditions faced by the terminal device when performing the task to be processed. This includes, but is not limited to, network status information, locally available computing resource information, and task meta-information. Specifically: network status information reflects the communication capability between the terminal device and cloud nodes or the edge, specifically covering communication bandwidth, transmission latency, data packet loss rate, and network type; locally available computing resource information reflects the local computing and energy carrying capacity of the terminal device, specifically covering remaining computing power, memory occupancy, CPU utilization, GPU utilization, and battery level; task meta-information is structured descriptive information related to the task to be reasoned about, not input content, used to characterize the semantic complexity, quality requirements, and processing constraints of the task to be reasoned about, specifically covering task type identifiers (such as "closed-ended question answering," "open generation," etc.). The functions include: "summary extraction", "code completion", "multi-turn dialogue", "instruction compliance", etc.; historical confidence records (the output confidence of this task or similar tasks at previous exit points, such as maximum probability value, entropy value, self-consistency score, etc.); privacy sensitivity level (i.e. whether the task contains sensitive information, such as medical or financial text, which determines whether it is allowed to be uploaded to the cloud or processed only on the edge); latency budget constraint (i.e. the maximum tolerable end-to-end response time of this task, such as less than or equal to 300ms); number of generated tokens (in incremental generation scenarios, used to reflect inference progress and assist in judging the remaining computation amount), etc.

[0034] Optionally, in this embodiment, the aforementioned preset conditions are a set of decision thresholds for determining whether to trigger the relay-style reasoning mechanism, including but not limited to: (1) if the network latency is lower than the first threshold (e.g., less than or equal to 50ms) and the communication bandwidth is higher than the second threshold (e.g., greater than or equal to 5Mbps), it is determined that the network has low-latency transmission capability; (2) if the data packet loss rate is higher than the third threshold or the network type is a low-speed mobile network (e.g., 2G or 3G), it is determined that the network does not support upgrades, and upgrading the terminal device to the cloud or edge side for processing is prohibited; (3) if the terminal's central processing unit utilization is lower than the fourth threshold or the graphics processor utilization is lower than the fifth threshold. If the local computing power of the terminal is saturated, an upgrade to the edge or cloud should be triggered; (4) If the task type is "open generation" or "multi-round inference" and the task length exceeds the sixth threshold (e.g., more than 256 tokens), the terminal device will be upgraded to the cloud or edge for processing; (5) If the historical output entropy of the task in the historical confidence record is higher than the seventh threshold (e.g., more than 2.8) or the maximum probability is lower than the eighth threshold (e.g., less than 0.65), the confidence of the preliminary output result (also known as "draft") quickly generated based on the intermediate hidden state at an early exit point is insufficient, and an upgrade to the edge or cloud will be triggered to improve the quality.

[0035] Optionally, in this embodiment, the aforementioned family model is a group of member models derived from the same base model, with different parameter scales but aligned intermediate feature spaces. This includes, but is not limited to, multiple small member models and large member models. These member models all share the same task output head and can be deployed on different nodes of the edge-cloud architecture to work collaboratively, breaking through the limitations of a single model. Large member models can relay inference based on the outputs of smaller member models to achieve optimal performance and efficiency. Some member models maintain a consistent local network architecture before a specific exit point (i.e., a key intermediate layer in the base model), differing only in the extended branches after the exit point (i.e., the sub-network between the exit point and the shared task head). It should be noted that all member models from the same origin are not required to have the same structure before the same exit point; rather, they can independently build aligned subfamilies according to the exit point, with all member models within each subfamilie sharing the front-end architecture of the same exit point.

[0036] Optionally, in the embodiments of this application, the above-mentioned task output head is a shared component of multiple member models, including but not limited to a language model head or classification head composed of layer normalization and linear projection, used to uniformly map the intermediate hidden states of member models of different sizes in the same semantic space to the vocabulary distribution or task label space, so as to ensure that the outputs of all member models are comparable and consistent.

[0037] Optionally, in this embodiment, the aforementioned small member model is a model with a small parameter size that is suitable for low-power, low-latency inference on the terminal side, and its model parameter size is small. Therefore, the aforementioned target small member model is a model within the family model that is selected and deployed on the terminal side according to the current environmental state, and has a smaller parameter size and faster, stronger inference capabilities.

[0038] Optionally, in this embodiment, the aforementioned large member model is a deep inference model with a large parameter scale that is suitable for high-computing environments at the edge or in the cloud, and its model parameter scale is large. Therefore, the aforementioned target large member model is a model within the family model that is deployed in the cloud or at the edge and has a higher parameter scale and stronger inference capabilities.

[0039] Optionally, in this embodiment, the aforementioned intermediate handover packet is structured inference state information generated by the target small member model at the exit point of the target key intermediate layer and used to transfer to the target large member model. It includes, but is not limited to, hidden state tensors, identification information of the exit point of the target key intermediate layer, drafts and their probabilities, task type identifiers, and available computing resource information. The intermediate hidden state is a sequence of high-dimensional semantic representation vectors generated by the preceding network of the target key intermediate layer at the exit point of the target key intermediate layer during the forward inference process of the base model.

[0040] It should be explained that the aforementioned cloud refers to a centralized computing service platform deployed in a remote data center, possessing large-scale computing resources and high-bandwidth network capabilities. This includes, but is not limited to, cloud server clusters consisting of high-performance graphics processing unit clusters, distributed inference engines, and low-latency network access. It is used to run large member models with significant parameter scale and high computational complexity, suitable for complex task scenarios requiring high response accuracy, semantic integrity, and contextual understanding. The aforementioned edge side, on the other hand, refers to distributed computing nodes deployed on the network access side, close to the terminal device. This includes, but is not limited to, lightweight computing units deployed in base stations, gateways, edge servers, or smart routers, possessing moderate computing power, low latency, and limited bandwidth. It is used for moderately complex tasks requiring a balance between response speed and computational cost, such as real-time voice interaction, video stream semantic analysis, and multimodal perception collaboration.

[0041] For example, S102 can be understood as follows: First, obtain the current available computing power resources of the terminal device, including but not limited to the CPU utilization, GPU utilization, idle video memory, memory usage, battery level and power consumption mode; at the same time, obtain network status information through network protocol stack monitoring, including network type (Wi-Fi, 5G, 4G, etc.), round-trip latency, packet loss rate, bandwidth estimate and connection stability score; then, normalize the above current available computing power resources information and network status information into a unified-dimensional environment state vector, and concatenate it with the task meta-information of the task to be processed (such as input length, task type, historical confidence requirements) to form joint environmental task perception information, which is used to determine whether relay-style reasoning is needed in the future.

[0042] For example, S104 above can be understood as follows: when the joint perception information of the environment and task meets the preset upgrade conditions, the system dynamically selects the optimal target small member model to perform front-end inference and generates an intermediate handover packet containing intermediate hidden states. Here, the family models, under the premise of sharing the base model and a unified task output head, achieve complete consistency in the local network structure of different scale models before the target key intermediate layer through decomposition, thereby ensuring that the intermediate hidden states can be seamlessly reused across models.

[0043] For example, S106 above can be understood as: using the target large member model that is the same as the target small member model to directly continue the subsequent layer and share the task output head at the same exit point to complete high-quality output and avoid the duplication of front-end calculations throughout the process.

[0044] In summary, this application's embodiments construct a distributed intelligent reasoning paradigm of "fast front-end output, deep back-end deduction, and lossless intermediate connection," which enables accurate reuse and collaborative reasoning of intermediate states across models of different scales. While ensuring reasoning quality, it significantly reduces edge-side energy consumption and cloud load, achieving a highly efficient, scalable, and low-latency reasoning system that integrates cloud, edge, and device.

[0045] As an optional approach, the training process of the aforementioned family model is as follows: Figure 2 As shown, it includes:

[0046] Step S1: Obtain multiple sets of training sample data from a preset corpus, wherein each set of training sample data includes: inference sample and standard answer of inference sample;

[0047] Step S2: Construct the basic model;

[0048] Step S3: Decompose multiple intermediate networks within the basic model to obtain multiple extended branch networks corresponding to each intermediate network, and determine multiple homologous models from the multiple extended branch networks corresponding to each intermediate network. The intermediate network is a sub-network between the exit point of the key intermediate layer of the basic model and the task output head.

[0049] Step S4: Determine the first predicted answer obtained by the base model from analyzing multiple inference samples and the second predicted answer obtained by multiple homologous models from analyzing multiple inference samples, and construct the target loss function based on the standard answer, the corresponding first predicted answer and the second predicted answer of each of the multiple inference samples.

[0050] Step S5: Adjust the parameters of the base model and multiple homologous models using the objective loss function, and use the trained homologous models and the trained base model as multiple member models to form a family model.

[0051] Optionally, in the embodiments of this application, the aforementioned corpus is a standardized data set used for model training and evaluation. It is structured and semantically annotated to support the joint optimization process of family models. It includes, but is not limited to, general text dialogue data, multi-turn question-answer pairs, instruction following tasks, long text generation samples, mathematical reasoning sequences, code generation fragments, and domain-specific corpora such as medical consultation, financial analysis, and legal text analysis.

[0052] Optionally, in this embodiment, the training sample data is extracted from a pre-defined corpus and organized into input-output pairs for training units. Each set of training sample data constitutes an independent inference sample and a corresponding standard answer. Specifically: the inference sample is raw text or sequence information used to stimulate the model's inference ability, including but not limited to user questions, instruction text, contextual paragraphs, text descriptions of multimodal inputs, task instructions, or dialogue history fragments; the standard answer is a high-quality reference output strictly corresponding to the sample, provided by manual annotation or an authoritative source, used to supervise model training and evaluate prediction accuracy, including but not limited to complete and coherent paragraph generation, accurate multi-hop inference conclusions, factual question-and-answer responses, structured output results, and generated text conforming to semantic logic and language norms.

[0053] Optionally, in the embodiments of this application, the above-mentioned basic model is the original deep neural network model (such as a large language generation model composed of multiple Transformers), which serves as the unified basic structure for all member models of the same origin. It includes, but is not limited to, encoder layers, decoder layers, embedding layers, positional encoding layers, and several linear projection layers and normalization layers.

[0054] Optionally, in the embodiments of this application, the aforementioned intermediate network is an intermediate layer in the backbone structure of the basic model, located between the deep feature extraction stage and the final task output head (such as the Q, K, V layers in an attention network or two linear layers in a feedforward network), whose function is to generate intermediate hidden states that can be used for early exit. For example, Figure 3 A schematic diagram of an optional base model is shown. It is easy to see that this base model contains four key intermediate layers (if the base model contains L key intermediate layers, then the value range of the key intermediate layer in the intermediate network can be 1 to L-1). The network between each key intermediate layer and the task output head is the intermediate network. Each key intermediate layer is connected to an exit point, which can be understood as a "terminating semantic node." Its output intermediate hidden state represents the semantic summary of the current context. This intermediate hidden state can be directly fed into the shared task output head, or it can be connected to the extended branch network obtained from subsequent decomposition for enhancement before output. It should be noted that the exit points of all key intermediate layers in the intermediate network share the same task output head to ensure the consistency of semantic mapping. The intermediate network provides structurally aligned and semantically coherent intermediate representations for these exit points.

[0055] Optionally, in the embodiments of this application, the above-mentioned homologous models are a set of derivative models with different parameter sizes but compatible structures, generated by performing low-rank matrix decomposition on the linear layers and extended branch networks in the basic model.

[0056] Optionally, in the embodiments of this application, the first predicted answer is the final prediction result generated by the task output head after the base model has completed the forward propagation, and serves as a benchmark reference for the model's capabilities.

[0057] Optionally, in the embodiments of this application, the second predicted response is the predicted output generated by the shared task output head after each homologous model has been processed by the pre-sequence network before the exit point of the key intermediate layer and the extended branch network after the exit point of the key intermediate layer, and is used to align with the output of the base model during the training phase.

[0058] Optionally, in the embodiments of this application, the above-mentioned objective loss function is a joint optimization objective that integrates the output of the base model and the output of each source model, used to simultaneously constrain the integrity of the base model and the prediction consistency and quality improvement capability of the source models at the exit points of each key intermediate layer.

[0059] For example, step S3 above can be understood as follows: by introducing a low-rank matrix factorization mechanism at multiple intermediate networks within the base model, the linear transformation weights contained in each intermediate network are structurally decomposed, and their initial weight matrix is ​​decomposed into the product of two low-rank weight matrices to obtain multiple extended branch networks corresponding to each intermediate network; subsequently, each extended branch network is bound at the exit point of the key intermediate layer of each intermediate network, and multiple homologous models are formed by the pre-order network of the exit point of the key intermediate layer of each intermediate network and multiple extended branch networks. These homologous models share the task output head as the output interface and share the same backbone structure (i.e., all homologous models obtained by decomposing the same intermediate network share the pre-order network of the exit point of the key intermediate layer of the base model in the intermediate network) and the alignment with the intermediate feature space, with only differences in the decomposition rank configuration, thereby forming a family model set with continuously adjustable parameter scale and compatible inference interface.

[0060] For example, step S4 above can be understood as follows: inputting the inference sample into the base model and obtaining the final output of its complete inference path as a global supervision signal (i.e., the first predicted answer), which is used to construct a global loss function that reflects the overall capability of the model; at the same time, inputting the same inference sample into each of the same source models respectively, and generating a final prediction (i.e., the second predicted answer) through the extended branch network connected by the exit point of the key intermediate layer of the intermediate network of each same source model, which serves as a local supervision signal; subsequently, based on the standard answer of the inference sample as the unified supervision target, constructing a target loss function based on the standard answer of each inference sample, the first predicted answer output by the base model, and the second predicted answer output by each same source model.

[0061] For example, step S5 above can be understood as: by using the target loss function as a unified optimization objective, end-to-end joint training is performed on the backbone parameters of the base model, the low-rank decomposition parameters of each key intermediate layer, and the scalable branch parameters of all source models. Specifically, during backpropagation, the gradient of the target loss function is simultaneously propagated back to the backbone network of the base model, the matrix factorization submatrices of each intermediate network, and the scalable branch network structure corresponding to each source model. This ensures that the backbone parameters maintain semantic consistency under global supervision, while the decomposition parameters and branch parameters adapt to the output quality at the corresponding scale under local supervision. During model training, gradient accumulation or grouped parameter update strategies can be used to reduce memory overhead and support efficient optimization of large-scale branch sets. After training, the parameters of the base model and all related models are fixed, forming a set of member models with the same structure, aligned features, and continuously adjustable parameter scale. Each member model has the same input and output interfaces and intermediate representation specifications, and can run independently or work together in the inference stage. This builds a unified, scalable, and low-maintenance family model, enabling seamless connection and smooth performance transition from lightweight deployment on the edge to deep inference on the edge or cloud.

[0062] Therefore, this application embodiment constructs a homogeneous model family generation mechanism with matrix decomposition as the core and multi-path joint supervision as the driving force, so as to uniformly realize the structural scalability, intermediate representation alignment and multi-scale output collaborative optimization of the basic model, thereby solving the core problem of "how to construct a family model system with continuously adjustable parameter scale, consistent inference performance and support for cloud-edge-device relay inference at low cost without relying on repeated training or independent parameter tuning of multiple versions".

[0063] As an alternative approach, for each intermediate network within the base model, the intermediate network corresponding to the key intermediate layer can be decomposed in the following way to obtain multiple extended branch networks corresponding to that intermediate network, including:

[0064] Step S21: Determine the input feature vector of the inference sample in each group of training sample data in the key intermediate layer, and form the input activation matrix of the intermediate network from each input feature vector.

[0065] Step S22: The input activation matrix is ​​centered and covariance whitened, and then linear regression is performed in combination with the initial weight matrix of the intermediate network to obtain the regression matrix corresponding to the intermediate network.

[0066] Step S23: Perform singular value decomposition on the regression matrix to obtain multiple singular values. Sort the multiple singular values ​​in descending order and determine the first r target singular values ​​in the sorting result. Multiple low-rank weight matrices are formed by the r target singular values ​​and the eigenvectors corresponding to each target singular value, where r is a positive integer greater than or equal to 1.

[0067] Step S24: Construct multiple extended branch networks corresponding to the intermediate network based on multiple low-rank weight matrices.

[0068] Optionally, in this embodiment, the input activation matrix is ​​a two-dimensional matrix formed by stacking the input feature vectors corresponding to multiple sets of training sample data in the intermediate network according to the sample dimension. Its rows correspond to the sequence number of each inference sample, and its columns correspond to the sample feature dimension of each inference sample. Therefore, the input activation matrix includes the input activation responses of all inference samples synchronously collected in the training batch at this layer.

[0069] Optionally, in the embodiments of this application, the above-mentioned centering process is to subtract the mean of each column (i.e. each feature dimension) of the input activation matrix from the mean of all inference samples, so that the mean of the feature distribution is zero, thereby eliminating bias drift, reducing the systematic error in the calculation of the covariance matrix, and improving the numerical stability of the subsequent whitening process.

[0070] Optionally, in the embodiments of this application, the above-mentioned covariance whitening process is to perform a linear transformation on the centered input activation matrix so that the covariance between its feature dimensions is close to the identity matrix, thereby achieving feature decorrelation and variance normalization, and generating a standardized feature space with zero mean, unit variance, and independent distribution.

[0071] Optionally, in the embodiments of this application, the aforementioned initial weight matrix is ​​the original linear transformation parameters of the intermediate network before decomposition. It can be generated by a standard initialization method (such as Xavier initialization), and the dimension of the initial weight matrix is ​​the number of output features × the number of input features, which is used to map the input features to the output space.

[0072] Optionally, in the embodiments of this application, the regression matrix is ​​a calibration transformation matrix obtained by modeling the mapping relationship between the whitened input activation matrix and the initial weight matrix through linear regression. It is obtained by minimizing the reconstruction error between the whitened input activation matrix and the initial weight matrix and is used to capture the optimal linear representation of the initial weight matrix in the whitened standardized feature space. Therefore, each column of the regression matrix corresponds to a projection direction of the original output dimension and each row corresponds to a principal component direction of the whitened input space.

[0073] Optionally, in the embodiments of this application, the above-mentioned singular value decomposition is to perform a matrix decomposition operation on the regression matrix to decompose it into a product of a left singular vector, singular values, and a right singular vector.

[0074] Optionally, in this embodiment, the aforementioned low-rank weight matrix is ​​an approximate weight matrix reconstructed from each set of target singular values ​​and their corresponding eigenvectors. Its form is the product of the left and right singular vectors, and its dimension is smaller than the initial weight matrix. This can be understood as follows: if the singular value decomposition result is expressed as: ,in, This indicates the output feature dimension of the intermediate network. Let U represent the input feature dimension of the intermediate network, U represent the left singular vector matrix, and V represent the right singular vector matrix. Let represent a singular value diagonal matrix, and For any given rank r, the following low-rank weight matrix can be constructed: ,in, This represents the first r columns of the left singular vector matrix U (corresponding to the first r target singular values ​​with the largest absolute values). This represents the first r columns of the right singular vector matrix V (corresponding to the first r target singular values ​​with the largest absolute values ​​of the right singular vectors). Let represent the diagonal matrix formed by the first r singular values ​​of the target. This matrix is ​​the low-rank weighted approximation matrix of rank r.

[0075] Optionally, in the embodiments of this application, the above-mentioned extended branch network is a low-rank substructure that replaces the original intermediate network after reconstruction based on the low-rank weight matrix. Each extended branch network corresponds to a low-rank weight matrix and has different parameter quantities and expressive capabilities, which are used to achieve continuously adjustable compression and expansion in the same family of models.

[0076] For example, step S21 above can be understood as: collecting the input activation responses of each intermediate network to each sample during the forward propagation process of the basic model, forming an input activation matrix that reflects the distribution of real data.

[0077] For example, step S22 can be understood as follows: performing centering and covariance whitening on the activation matrix obtained in step S21 to eliminate the correlation and scale difference between features, so that subsequent modeling is based on orthogonal, zero-mean standardized feature space, improving numerical stability and decomposition generalization; then, performing linear regression modeling on the whitened input activation matrix and the original weight matrix of the intermediate network to solve for a regression matrix, which represents the optimal linear approximation relationship of the original weight matrix in the whitened feature space.

[0078] For example, step S23 above can be understood as: performing singular value decomposition on the regression matrix obtained in step S22 above to obtain multiple singular values, sorting the multiple singular values ​​in descending order, and determining the first r (r is a positive integer greater than or equal to 1) target singular values ​​in the sorting result. A set of low-rank weight matrices with different ranks is constructed from the r target singular values ​​and the left and right singular vectors associated with each target singular value. Each low-rank weight matrix corresponds to a weight approximation of a parameter scale.

[0079] For example, step S24 above can be understood as: constructing a set of structurally equivalent but parameter-different extended branch networks based on multiple low-rank weight matrices. Each extended branch network consists of two cascaded low-rank projected sub-layers, in the form of... ,in, Represents the left projection matrix. This represents the right projection matrix, thus providing multiple configurable weight primitives for the intermediate network.

[0080] Through the above embodiments, by introducing a data-driven low-rank decomposition mechanism based on the statistical characteristics of sample activation, the initial weight matrices of each intermediate network in the basic model are structurally reconstructed. Multiple low-rank weight matrices of different ranks replace the initial weight matrices, thereby enabling continuous adjustment of the parameter scale of a single intermediate network without changing the input and output interfaces. This solves the technical problems of traditional multi-size models requiring independent training, inconsistent feature spaces, and difficulty in collaborative inference. It achieves unified construction and feature alignment of model families, allowing members of the same model to share intermediate activations, supporting relay-style inference, and providing a structural foundation and parameter configuration space for subsequent scalable branching and hierarchical continuation based on exit points. This significantly reduces the training and maintenance overhead of multi-level models, while laying a scalable and reusable model structure prerequisite for realizing a low-latency, high-precision cloud-edge-device collaborative inference system.

[0081] As an optional approach, a first predicted answer obtained by the base model analyzing multiple inference samples and a second predicted answer obtained by multiple homologous models analyzing multiple inference samples are determined. A target loss function is then constructed based on the multiple first predicted answers and the multiple second predicted answers, including:

[0082] Step S31: Construct a global loss function based on the standard answers, corresponding first and second predicted answers of each of the multiple inference samples;

[0083] Step S32: For each intermediate network, construct the branch loss function corresponding to the intermediate network based on the standard answers of multiple training samples and the second predicted answers output by multiple homologous models obtained by decomposing the same intermediate network.

[0084] Step S33: Perform a weighted summation of the global loss function and multiple branch loss functions to obtain the target loss function.

[0085] For example, step S31 above can be understood as follows: for each inference sample, with its standard answer as the unified supervision target, calculate the error between the first predicted answer output by the full path of the base model, the second predicted answer output by the full path of the homologous model, and the standard answer, and construct a global loss function to force all model members to learn semantic representations consistent with the complete base model, thereby ensuring the inference accuracy of the overall model.

[0086] For example, step S32 above can be understood as follows: for all homologous models with the same structure before the exit point of the same intermediate network (i.e., aligned sub-family, where all homologous models in the sub-family share the front-end architecture of the same exit point), calculate the error between the second predicted answer and the standard answer obtained after each inference sample is processed by the extended branch network connected to the corresponding exit point of these homologous models, construct a branch loss function to incentivize the extended branch network to accurately approximate the model output with low parameter overhead, significantly improve the quality of shallow inference, and avoid the performance collapse of traditional early exit methods.

[0087] For example, step S33 above can be understood as: linearly weighting the global loss function and all branch loss functions according to preset weights to form a unified target loss function. This weighting coefficient typically consists of static prior weights and a dynamic adjustment factor. Generally, a higher static prior weight is assigned to the global loss function, while for each branch loss function, decreasing weights can be set based on the depth of its corresponding key intermediate layer in the base model and the parameter scale of its parent model. The dynamic adjustment factor of the branch loss function is adjusted in real-time during training based on the gradient norm of each loss term or the performance fluctuations on the validation set, ensuring that the contribution of each loss term to the total gradient is balanced and avoiding optimization bias due to differences in loss magnitude. Therefore, this factor can be calculated by moving average of each loss term and variance, standardized using Z-score, and used as a multiplicative adjustment coefficient to achieve an adaptive mechanism of "automatic reinforcement of supervision for difficult-to-learn branches." Specifically, the expression for the target loss function can be written as: ,in, Represents the global loss function. This represents the static prior weights of the global loss function. This represents the e-th exit point. even The branch loss function of the b-th extended branch network. This represents the dynamic adjustment factor of the branch loss function for the b-th extended branch network connected to the e-th exit point. Let represent the static prior weights of the branch loss function of the b-th extended branch network connected to the e-th exit point. This represents the parameter size adjustment factor related to the b-th extended branch network (i.e., the smaller the parameter size, the higher the weight). denoted by depth decay rate, and L represents the total number of critical intermediate layers within the base model.

[0088] The multiple member models trained using the above technical solutions can be categorized into large member models and small member models based on their model parameter scale. Large member models are obtained at the exit point of a deeper critical intermediate layer (e.g., the sixteenth layer), using a high-rank configuration (e.g., r=256 or higher) for all critical networks. They typically retain more original weight information and have a large total number of model parameters. Small member models, on the other hand, are obtained at the exit point of a shallower critical intermediate layer (e.g., the fourth layer), using a low-rank configuration (e.g., r=64 or lower) for some or all critical networks. They typically retain less original weight information and have a smaller total number of model parameters. Therefore, the difference in model parameter scale between large and small member models stems from the decomposition rank configuration of the critical networks and the choice of branch rank after the exit point.

[0089] This application's embodiments can simultaneously optimize the global output of the base model and the local outputs of multiple homologous models after branch enhancement in the intermediate network within a unified training framework. It constructs a multi-path joint loss function with the standard answer as the common supervision target, thereby solving the technical problems of inconsistent representations, drastic drop in shallow output quality, and the need for repeated training of multiple model versions caused by differences in model size during the construction of family models. Therefore, the model training method provided in this application's embodiments is not independent modeling, but rather constructs a family-style model system with continuously adjustable parameter sizes under zero redundant training cost.

[0090] As an optional approach, the intermediate handover package also includes: identification information of the exit point of the target key intermediate layer (such as the index identifier of the exit point). Therefore, by analyzing the intermediate handover package using the target large member model within the family model, the first answer output by the task output head of the target large member model is obtained, including: determining the target sub-model within the target large member model based on the identification information, wherein the target sub-model consists of multiple extended branch networks and task output heads directly connected to the exit point of the target key intermediate layer within the target large member model; and inputting the intermediate hidden state in the intermediate handover package into the target sub-model to obtain the first answer output by the task output head of the target sub-model.

[0091] For example, the above technical solution can be understood as follows: In the relay inference process, firstly, based on the identification information of the exit point in the intermediate handover packet, the target sub-model from the exit point to the task output head is extracted from the network structure of the target large member model; then, the intermediate hidden state carried in the intermediate handover packet is directly used as input and injected into the target sub-model; subsequently, the target sub-model undergoes a series of processing operations (such as attention calculation, feedforward network transformation, layer normalization, etc.), and finally the first answer is generated by the shared task output head. The entire process relies on the alignment characteristics of the hidden state space of the member models in the family model in the intermediate network, ensuring that the intermediate hidden state from the target small member model can be received and processed by the target large member model without loss, thereby achieving zero redundancy, low latency, and high fidelity cross-device continuation computing, eliminating redundant inference overhead, and improving overall inference efficiency and resource utilization.

[0092] As an optional approach, the intermediate hidden states in the intermediate handover packet are input into the target sub-model to obtain the first answer output by the task output head of the target sub-model. This includes: randomly inputting the intermediate hidden states in the intermediate handover packet into any extended branch network directly connected to the exit point of the target key intermediate layer in the target sub-model, and inputting the intermediate hidden states output by the extended branch network into the task output head to obtain the first answer output by the task output head.

[0093] For example, the above scheme can be understood as follows: after receiving the intermediate handover packet, the system can proceed as follows Figure 4 The process described first involves randomly or adaptively selecting one of m extended branch networks (e.g., low-rank, medium-rank, or high-rank branches) from among the multiple extended branch networks connected to the target key intermediate layer, based on the current computing power budget, latency constraints, or confidence assessment results. Then, the system inputs the intermediate hidden state from the intermediate handover packet output at the exit point of the target small member model in the target key intermediate layer into the selected extended branch network 1. Feature enhancement and semantic refinement are then performed through extended branch network 1, and its output is the reconstructed intermediate representation. This representation is semantically aligned with the backbone model and its dimensions are compatible with the shared task output head. Finally, the refined intermediate hidden state is directly fed into the shared task output head, which completes the vocabulary mapping and outputs the final first answer. The above mechanism relies on the low-rank decomposition architecture design of all extension branches based on the base model, ensuring that different extension branch networks can be seamlessly connected in the same hidden space, realizing lightweight augmented inference with "on-demand activation and seamless replacement", significantly improving the output quality of shallow exit, while avoiding full model recalculation in the cloud or edge, achieving low-overhead and highly adaptable inference enhancement effect.

[0094] As an alternative approach, when the joint perception information of the environmental task does not meet the preset conditions, the target small member model within the family model is used to analyze the reasoning task, obtain the second answer output by the task output head of the target small member model, and output the second answer as the target answer of the reasoning task.

[0095] For example, the above scheme can be understood as follows: when the joint perception information of the environment and task (including device computing power, power consumption threshold, network bandwidth, latency budget and task semantic complexity, etc.) does not meet the triggering conditions for high-order inference, it is determined that the current environment only supports lightweight inference, and then the member model with the smallest parameter size and the lowest computational overhead in the family model is activated, and end-to-end inference is performed directly on its local machine. The specific process is as follows: First, the environmental task joint perception information is parsed. If it is lower than the preset relay upgrade conditions (such as insufficient computing power on the edge, too short task latency budget, or network unavailability), the intermediate handover and cross-device transmission links are skipped. Then, the target small member model that best matches the current environmental resources is selected from the family model. This model consists of the first few layers of the basic model and a low-rank extension branch. All its parameters have been fixed and deployed on the local device. Then, the input sequence of the task to be inferred is input into the target small member model, which is forward propagated to the exit point of the target key intermediate layer and semantically enhanced through its built-in lightweight extension branch network. Finally, the second answer is generated by the shared task output head. The second answer is returned directly as the final output result. No interaction with the edge or cloud is required throughout the process.

[0096] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0098] According to another aspect of the embodiments of this application, a task processing apparatus for implementing the task processing method provided in the above embodiments is also provided, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0099] Figure 5 This is a structural block diagram of an optional task processing device according to an embodiment of this application, such as... Figure 5 As shown, the task processing device includes:

[0100] The acquisition module 52 is used to acquire the task to be reasoned and the environment task joint perception information of the task to be reasoned, wherein the environment task joint perception information includes at least one of the following: network status information, local available computing resource information, and task meta information;

[0101] The first analysis module 54 is used to analyze the reasoning task by using the target small member model in the pre-trained family model when the joint perception information of the environmental task meets the preset conditions, and to obtain the intermediate handover packet of the target key intermediate layer output of the target small member model. The intermediate handover packet includes at least: the intermediate hidden state output by the target key intermediate layer. Multiple member models in the family model share the same task output head. The types of member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model.

[0102] The second analysis module 56 is used to analyze the intermediate handover packet using the target large member model within the family model, to obtain the first answer output by the task output head of the target large member model, and to output the first answer as the target answer of the task to be reasoned. The local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer.

[0103] In an exemplary embodiment, the above-described apparatus is further configured to train a family model by means of: constructing a base model; decomposing multiple intermediate networks within the base model to obtain multiple extended branch networks corresponding to each intermediate network, and determining multiple homologous models by the multiple extended branch networks corresponding to each intermediate network, wherein the intermediate network is a sub-network deployed between the exit point and the task output head of the key intermediate layer of the base model; determining the first predicted answer obtained by the base model from analyzing multiple inference samples and the second predicted answer obtained by the multiple homologous models from analyzing multiple inference samples, and constructing a target loss function based on the standard answer of each of the multiple inference samples, the corresponding first predicted answer and the second predicted answer; adjusting the parameters of the base model and the multiple homologous models using the target loss function, and using the trained multiple homologous models and the trained base model as multiple member models to form a family model.

[0104] In an exemplary embodiment, the above-described apparatus is further configured to decompose multiple intermediate networks within a base model to obtain multiple extended branch networks corresponding to each intermediate network by the following method: for each intermediate network within the base model, determining the input feature vector of the inference sample in each set of training sample data at the key intermediate layer, and forming the input activation matrix of the intermediate network by each input feature vector; performing centering and covariance whitening on the input activation matrix, and performing linear regression processing in combination with the initial weight matrix of the intermediate network to obtain the regression matrix corresponding to the intermediate network; performing singular value decomposition on the regression matrix to obtain multiple singular values, sorting the multiple singular values ​​in descending order, and determining the first r target singular values ​​in the sorting result, forming multiple low-rank weight matrices by the r target singular values ​​and the feature vector corresponding to each target singular value, where r is a positive integer greater than or equal to 1; and constructing multiple extended branch networks corresponding to the intermediate network based on the multiple low-rank weight matrices.

[0105] In an exemplary embodiment, the above-described apparatus is further configured to construct a target loss function based on the standard answers, first predicted answers, and second predicted answers of multiple inference samples by the following method: constructing a global loss function based on the standard answers of each of the multiple inference samples and the corresponding first predicted answers and second predicted answers; for each intermediate network, constructing a branch loss function corresponding to the intermediate network based on the standard answers of each of the multiple training samples and the second predicted answers output by each of the multiple homologous models obtained by decomposing the same intermediate network; and performing a weighted summation of the global loss function and the multiple branch loss functions to obtain the target loss function.

[0106] In an exemplary embodiment, the intermediate handover packet further includes: identification information of the exit point of the target key intermediate layer. The aforementioned apparatus is further configured to analyze the intermediate handover packet using the target large member model within the family model, and obtain a first response output by the task output head of the target large member model, including: determining a target sub-model within the target large member model based on the identification information, wherein the target sub-model consists of multiple extended branch networks and a task output head directly connected to the exit point of the target key intermediate layer within the target large member model; and inputting the intermediate hidden state from the intermediate handover packet into the target sub-model to obtain the first response output by the task output head of the target sub-model.

[0107] In an exemplary embodiment, the above-described apparatus is further configured to input the intermediate hidden state in the intermediate handover packet into the target sub-model to obtain the first answer output by the task output head of the target sub-model by means of: randomly inputting the intermediate hidden state in the intermediate handover packet into any extended branch network directly connected to the exit point of the target key intermediate layer in the target sub-model, and inputting the intermediate hidden state output by the extended branch network into the task output head to obtain the first answer output by the task output head.

[0108] In an exemplary embodiment, the above-described apparatus is further configured to, when the environmental task joint perception information does not meet preset conditions, analyze the reasoning task using the target small member model within the family model, obtain the second answer output by the task output head of the target small member model, and output the second answer as the target answer for the reasoning task.

[0109] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0110] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0111] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0112] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0113] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0114] According to another aspect of the embodiments of this application, a computer program product is also provided, the computer program product including a computer program / instructions containing program code for performing the method shown in the flowchart.

[0115] Figure 6 A structural block diagram of an electronic device for implementing embodiments of this application is shown. Figure 6 As shown, the electronic device 60 may include one or more processors 602 (shown as 602a, 602b, ..., 602n in the figure) (processor 602 may include, but is not limited to, a processing device such as a microprocessor or programmable logic device), a memory 604 for storing data, and a transmission device 606 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the computer system described above. For example, electronic device 60 may also include... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0116] It should be noted that the aforementioned one or more processors 602 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element of the electronic device 60. As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0117] The memory 604 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the task processing method in this embodiment. The processor 602 executes various functional applications and data processing by running the software programs and modules stored in the memory 604, thereby implementing the task processing method of the application described above. The memory 604 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 604 may further include memory remotely located relative to the processor 602, and these remote memories can be connected to the electronic device 60 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0118] The transmission device 606 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 60. In one example, the transmission device 606 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 606 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0119] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the electronic device 60.

[0120] It should be noted that, Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0121] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0122] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A task processing method, characterized in that, include: Obtain the joint perception information of the task to be reasoned and its environment, wherein the joint perception information of the environment includes at least one of the following: network status information, local available computing resource information, and task meta information; When the joint perception information of the environmental task meets the preset conditions, the target small member model in the pre-trained family model is used to analyze the task to be reasoned, and the intermediate handover packet of the target key intermediate layer output of the target small member model is obtained. The intermediate handover packet includes at least: the intermediate hidden state output by the target key intermediate layer. Multiple member models in the family model share the same task output head. The types of member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model. The intermediate handover packet is analyzed using the target large member model within the family model to obtain the first answer output by the task output head of the target large member model, and the first answer is output as the target answer of the task to be reasoned. The local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer. The training process of the family model includes: acquiring multiple sets of training sample data from a preset corpus, wherein each set of training sample data includes: inference samples and standard answers of the inference samples; constructing a base model; decomposing multiple intermediate networks within the base model to obtain multiple extended branch networks corresponding to each intermediate network, and determining multiple homologous models by the multiple extended branch networks corresponding to each intermediate network, wherein the intermediate network is a sub-network deployed between the exit point of the key intermediate layer of the base model and the task output head; determining the first predicted answer obtained by the base model from analyzing multiple inference samples and the second predicted answer obtained by the homologous models from analyzing multiple inference samples, and constructing a target loss function based on the standard answer, the corresponding first predicted answer and the second predicted answer of each of the multiple inference samples; adjusting the parameters of the base model and the multiple homologous models using the target loss function, and using the trained multiple homologous models and the trained base model as multiple member models to form the family model.

2. The method according to claim 1, characterized in that, The basic model is decomposed into multiple intermediate networks to obtain multiple extended branch networks corresponding to each intermediate network, including: For each intermediate network within the base model, the input feature vector of the inference sample in each group of training sample data is determined in the key intermediate layer, and the input activation matrix of the intermediate network is composed of each input feature vector. The input activation matrix is ​​centered and covariance whitened, and then linear regression is performed in combination with the initial weight matrix of the intermediate network to obtain the regression matrix corresponding to the intermediate network. The regression matrix is ​​subjected to singular value decomposition to obtain multiple singular values. The multiple singular values ​​are sorted in descending order, and the first r target singular values ​​in the sorting result are determined. Multiple low-rank weight matrices are formed by the r target singular values ​​and the eigenvector corresponding to each target singular value, where r is a positive integer greater than or equal to 1. Multiple extended branch networks corresponding to the intermediate network are constructed based on the multiple low-rank weight matrices.

3. The method according to claim 1, characterized in that, A target loss function is constructed based on the standard answers and corresponding first and second predicted answers of each of the multiple inference samples, including: A global loss function is constructed based on the standard answers, corresponding first and second predicted answers of each of the multiple inference samples. For each intermediate network, a branch loss function corresponding to the intermediate network is constructed based on the standard answers of the multiple training samples and the second predicted answers output by the multiple homologous models obtained by decomposing the same intermediate network. The target loss function is obtained by weighted summation of the global loss function and the multiple branch loss functions.

4. The method according to claim 1, characterized in that, The intermediate handover packet also includes: identification information of the exit point of the target key intermediate layer, wherein; the intermediate handover packet is analyzed using the target large member model within the family model to obtain the first response output by the task output header of the target large member model, including: Based on the identification information, a target sub-model within the target large member model is determined, wherein the target sub-model consists of multiple extended branch networks directly connected to the exit point of the target key intermediate layer within the target large member model and the task output head; The intermediate hidden state in the intermediate handover packet is input into the target sub-model to obtain the first answer output by the task output header of the target sub-model.

5. The method according to claim 4, characterized in that, The intermediate hidden state in the intermediate handover packet is input into the target sub-model to obtain the first answer output by the task output header of the target sub-model, including: The intermediate hidden state in the intermediate handover packet is randomly input into any extended branch network directly connected to the exit point of the target key intermediate layer in the target sub-model, and the intermediate hidden state output by the extended branch network is input into the task output head to obtain the first answer output by the task output head.

6. The method according to claim 1, characterized in that, The method further includes: If the joint perception information of the environmental task does not meet the preset conditions, the target small member model in the family model is used to analyze the task to be reasoned, and the second answer output by the task output head of the target small member model is obtained. The second answer is then output as the target answer of the task to be reasoned.

7. A task processing device, characterized in that, include: The acquisition module is used to acquire the task to be inferred and the environment task joint perception information of the task to be inferred, wherein the environment task joint perception information includes at least one of the following: network status information, local available computing resource information, and task meta information; The first analysis module is used to analyze the task to be reasoned using a target small member model within a pre-trained family model when the joint perception information of the environmental task meets preset conditions, and to obtain an intermediate handover packet output by the target key intermediate layer of the target small member model. The intermediate handover packet includes at least: the intermediate hidden state output by the target key intermediate layer. Multiple member models in the family model share the same task output head. The types of the member models include at least: large member models and small member models, and the model parameter scale of the large member model is larger than that of the small member model. The second analysis module is used to analyze the intermediate handover packet using the target large member model within the family model, to obtain the first answer output by the task output head of the target large member model, and to output the first answer as the target answer of the task to be reasoned. The local network of the target small member model before the target key intermediate layer is the same as the local network of the target large member model before the target key intermediate layer. The training process of the family model includes: acquiring multiple sets of training sample data from a preset corpus, wherein each set of training sample data includes: inference samples and standard answers of the inference samples; constructing a base model; decomposing multiple intermediate networks within the base model to obtain multiple extended branch networks corresponding to each intermediate network, and determining multiple homologous models by the multiple extended branch networks corresponding to each intermediate network, wherein the intermediate network is a sub-network deployed between the exit point of the key intermediate layer of the base model and the task output head; determining the first predicted answer obtained by the base model from analyzing multiple inference samples and the second predicted answer obtained by the homologous models from analyzing multiple inference samples, and constructing a target loss function based on the standard answer, the corresponding first predicted answer and the second predicted answer of each of the multiple inference samples; adjusting the parameters of the base model and the multiple homologous models using the target loss function, and using the trained multiple homologous models and the trained base model as multiple member models to form the family model.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the device containing the computer-readable storage medium executes the task processing method according to any one of claims 1 to 6 by running the computer program.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the task processing method according to any one of claims 1 to 6 through the computer program.

Images