Large model trusted collaborative inference method and device for wireless edge network

Abstract

The present application relates to a large model trusted collaborative inference method and device for a wireless edge network, and belongs to the technical field of trusted inference. The method comprises: dividing the multi-head attention module in each transformer layer of the large language model into multiple attention head blocks, and constructing a fine-grained computation graph with the attention aggregation unit and the feedforward network unit; setting a trusted domain according to the node trusted attribute; marking the relevant computation units in the graph as sensitive computation units according to the privacy protection depth, and marking the corresponding data edges as sensitive data flow; generating a deployment scheme under the constraint of the trusted domain; deploying the sensitive units in the trusted domain for execution according to the collaborative deployment scheme, and limiting the transmission of the sensitive data flow within the trusted domain; completing the collaborative inference and outputting the result in the order of the computation graph. The method can effectively limit the cross-domain flow of sensitive intermediate activation values while retaining the multi-node collaborative inference capability, thereby improving the security and trustworthiness of the collaborative inference of the large model in the wireless edge network.

Description

Technical Field

[0001] This invention relates to the field of trusted reasoning technology, and in particular to a trusted collaborative reasoning method and apparatus for large-scale models of wireless edge networks. Background Technology

[0002] With the widespread application of large language models in scenarios such as intelligent question answering, text generation, and semantic understanding, model inference places extremely high demands on computing resources, storage capacity, and communication bandwidth. Since large language models typically contain massive amounts of parameters and complex network structures, deploying them entirely on terminal devices or single edge nodes can easily lead to problems such as insufficient storage, high computational latency, and slow response speeds. Wireless edge networks consist of terminal devices, edge nodes, and cloud nodes. Terminal devices are close to the user, edge nodes have low access latency, and cloud nodes have strong computing capabilities. To alleviate the resource limitations of single nodes, existing technologies often employ collaborative inference, which involves splitting the large language model into multiple computational parts and deploying them on different nodes, allowing multiple nodes to collaboratively complete a single inference task.

[0003] Existing large language models mostly employ the Transformer architecture, which includes a multi-head attention module and a feedforward network module. The multi-head attention module extracts input features from different perspectives using multiple attention heads, then concatenates, aggregates, and linearly projects the outputs of each head to form intermediate activation values. In existing collaborative inference schemes, the model is typically divided into layers, inference stages, or large model segments. After the previous segment completes its computation, the intermediate results are transmitted to the node containing the next segment, thereby reducing the load on a single node. However, these schemes primarily focus on model splitting, computational task allocation, and inference latency optimization, neglecting the privacy risks associated with intermediate activation values ​​transmitted across nodes. Although intermediate activation values ​​are not the original input text, they are calculated jointly by user input and model parameters and may contain sensitive content such as user input semantics, contextual information, or task intent. Especially in the Transformer's multi-head attention module, the aggregated and projected outputs of multiple attention heads form a more concentrated semantic representation. If this representation is transmitted to an untrusted node during collaborative inference, it poses a privacy risk. Simply restricting the original input from being uploaded, or encrypting the transmission link, cannot completely prevent the cross-domain flow of sensitive intermediate activation values ​​from the perspective of model execution structure. Summary of the Invention

[0004] Therefore, it is necessary to provide a method and apparatus for large-scale trusted collaborative reasoning for wireless edge networks that can restrict the transmission and execution of sensitive intermediate activation values ​​only within the trusted domain from the perspective of model execution structure.

[0005] A trusted collaborative reasoning method for large-scale models in wireless edge networks, the method comprising: Step 1: Obtain the large language model to be executed, divide the multi-head attention module in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computing units. Construct a fine-grained computing graph based on the data dependencies between each fine-grained computing unit. Step 2: Determine the trusted attributes of each node in the wireless edge network, and set the trusted domain according to the trusted attributes; Step 3: According to the preset privacy protection depth, mark sensitive computing units in the fine-grained computing graph, and mark the data dependency edges between the sensitive computing units as sensitive data streams; Step 4: Obtain the available storage capacity and computing power of each node in the wireless edge network, and obtain the link bandwidth between each node. Generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraints. Step 5: According to the collaborative deployment scheme, deploy the sensitive computing unit to the node within the trusted domain for execution, and deploy the non-sensitive computing unit to the node outside the trusted domain for execution. During the execution process, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain. Step 6: Perform collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

[0006] On the other hand, a large-scale trusted collaborative reasoning device for wireless edge networks is also provided, comprising: The fine-grained computation graph construction module is used to obtain the large language model to be executed, divide the multi-head attention modules in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computation units, and construct a fine-grained computation graph based on the data dependencies between each fine-grained computation unit. The trusted domain configuration module is used to determine the trusted attributes of each node in the wireless edge network and set the trusted domain according to the trusted attributes. The sensitive labeling module is used to label sensitive computing units in the fine-grained computation graph according to a preset privacy protection depth, and to mark the data dependency edges between the sensitive computing units as sensitive data streams; The deployment scheme generation module is used to obtain the available storage capacity and computing power of each node in the wireless edge network, and to obtain the link bandwidth between each node, and to generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraints. The deployment and transmission control module is used to deploy the sensitive computing unit to the node within the trusted domain for execution, and to deploy the non-sensitive computing unit to the node outside the trusted domain for execution, according to the collaborative deployment scheme. During the execution process, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain. The collaborative reasoning module is used to complete collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

[0007] Compared with existing technologies, the large-model trusted collaborative reasoning method and apparatus for wireless edge networks provided by this invention have the following advantages: 1. The multi-head attention module of the large language model transformer layer is divided into multiple attention head blocks. A fine-grained computation graph is constructed by combining the attention aggregation unit and the feedforward network unit, which realizes fine-grained control over the model computation process. It can accurately locate the computation units and data flow that are prone to generating sensitive intermediate activation values, and solves the technical problem that the existing coarse-grained model splitting method cannot identify sensitive computation locations.

[0008] 2. By setting trusted domains and imposing trusted domain constraints, sensitive computing units are executed only on nodes within trusted domains, and sensitive data streams are transmitted only between nodes within trusted domains. This completely blocks the path of sensitive intermediate activation values ​​being leaked across untrusted nodes from the model execution structure level. Compared with existing technologies that only encrypt the transmission link or restrict the original input, privacy protection is more thorough and reliable.

[0009] 3. By adopting a hierarchical deployment of sensitive and non-sensitive computing units, non-sensitive computing units can be collaboratively executed by external nodes in the trusted domain. Under the premise of ensuring privacy and security, the heterogeneous node resources of the wireless edge network are fully utilized, which effectively alleviates the bottleneck of insufficient computing power and storage of single nodes and retains the high efficiency advantage of multi-node collaborative inference.

[0010] 4. Based on node resources, link bandwidth, and trusted domain constraints, a collaborative deployment scheme is generated to ensure that the inference process complies with privacy protection requirements throughout, thereby improving the security, controllability, and practicality of collaborative inference of large models in wireless edge networks. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention, and those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a large-model trusted collaborative reasoning method for wireless edge networks in one embodiment. Figure 2 This is a block diagram of a large-model trusted collaborative reasoning device for wireless edge networks in one embodiment. Figure 3 This is an internal structural diagram of a computer device in one embodiment.

[0013] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0015] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0016] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0017] Example 1 like Figure 1 As shown, a trusted collaborative reasoning method for large models in wireless edge networks is provided, including the following steps: Step 1: Obtain the large language model to be executed, divide the multi-head attention module in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computing units. Construct a fine-grained computing graph based on the data dependencies between each fine-grained computing unit.

[0018] Step 2: Determine the trusted attributes of each node in the wireless edge network and set the trusted domain based on the trusted attributes.

[0019] Step 3: Based on the preset privacy protection depth, mark sensitive computing units in the fine-grained computing graph and mark the data dependency edges between sensitive computing units as sensitive data streams.

[0020] Step 4: Obtain the available storage capacity and computing power of each node in the wireless edge network, and obtain the link bandwidth between each node. Generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraint.

[0021] Step 5: According to the collaborative deployment scheme, deploy sensitive computing units to nodes within the trusted domain for execution, and deploy non-sensitive computing units to nodes outside the trusted domain for execution. During execution, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain.

[0022] Step 6: Perform collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

[0023] The aforementioned trusted collaborative inference method for large models in wireless edge networks achieves precise control over the model computation process by performing fine-grained decomposition of the large language model at the HeadBlock level and constructing a fine-grained computation graph. This enables the identification of computation units and data flows that are prone to generating sensitive intermediate activation values. By setting trusted domains and imposing constraints, the path for sensitive intermediate activation values ​​to leak across untrusted nodes is blocked at the model execution structure level. Simultaneously, by adopting a hierarchical deployment approach for sensitive and non-sensitive computation units, the heterogeneous node resources of wireless edge networks are fully utilized while ensuring privacy and security. This effectively solves the technical problems in existing technologies, such as the inability of coarse-grained decomposition to identify sensitive computation locations, the inability of link encryption alone to fundamentally prevent the cross-domain flow of sensitive data, and the difficulty in balancing privacy protection and inference efficiency.

[0024] In the specific implementation of step 1, a large model inference request is first received. This inference request is a large model inference task instruction that contains the core information of the task, generated based on user-initiated tasks such as text generation, semantic understanding, code generation, intelligent question answering, and intelligent interaction. Based on the inference request, the large language model to be executed and the inference task parameters are determined. The inference task parameters include the original text content input by the user, the unique identifier of the large language model to be called, the task type identifier, the source node information of the inference task, the task security level requirements, and the inference result return address.

[0025] The structure of the large language model to be executed is analyzed. Taking the Transformer decoder structure (transformer network structure) as an example, each transformer layer contains a multi-head attention module and a feedforward network module. The multi-head attention module includes multiple attention heads, and different attention heads can extract input context features from different representation subspaces. Existing technologies typically deploy the multi-head attention module in whole layers or large model slices, while this invention further divides the multi-head attention module into multiple HeadBlocks, i.e., attention head blocks, making each attention head block an independently deployable and schedulable computational unit.

[0026] Suppose the large language model has L layers. For the Lth layer... A layer, assuming that the layer contains Each attention head contains [number] attention heads, and each attention head block contains [number] attention heads. The first attention point, then the second The number of attention head blocks that can be divided into layers is: (1) In the formula, This indicates the rounding up operation.

[0027] The first The layer converter is divided into layers. One attention head block, one attention aggregation unit, and one feedforward network unit form the set of fine-grained computational units for this layer: (2) in, Indicates the first Layer One attention-grabbing block; Indicates the first The layer's attention aggregation unit is used to concatenate and linearly project the outputs of multiple attention head blocks; Indicates the first The feedforward network units of the layer are used to perform subsequent nonlinear transformations.

[0028] Determine the first The data dependencies within the layer are as follows: (3) The above relationship indicates that multiple attention head blocks in the same layer can be executed in parallel, but the outputs of all attention head blocks need to be aggregated into the attention aggregation unit; after the attention aggregation unit completes the splicing and linear projection, the result is then input into the feedforward network unit.

[0029] By summarizing the fine-grained computational units and data dependencies of all converter layers, a fine-grained computational graph of the large model is constructed: (4) in, Represents the set of all computational units. This represents the set of data dependency edges between computing units.

[0030] This step breaks through the limitations of existing technologies that split entire layers or large segments by further breaking down the multi-head attention module into fine-grained computing units at the attention head block level. It can accurately locate the generation position, transmission direction, and receiving computing unit of intermediate activation values, providing a foundation for subsequent identification of sensitive computing units and deployment of trusted domain constraints.

[0031] In step 2, a trusted attribute refers to an attribute identifier indicating whether a node is allowed to process sensitive intermediate activation values; a trusted domain refers to a set of nodes that are allowed to process sensitive inputs and sensitive intermediate activation values; a trusted node refers to a node that is allowed to process sensitive inputs or sensitive intermediate activation values; and an untrusted node may participate in non-sensitive computations but may not receive or process intermediate activation values ​​marked as sensitive.

[0032] In the specific implementation of step 2, let the set of nodes in the wireless edge network that can participate in collaborative reasoning be . ,in Indicates the first There are 100 nodes that can participate in collaborative reasoning, including terminal devices, edge nodes, and cloud nodes.

[0033] Let the set of trusted domain nodes be: (5) For any node Define the trusted identifier of a node as: (6) In the formula, A trusted identifier for a node; This represents a set of nodes. Among them, Represents a node Belongs to the trusted domain. Represents a node It does not belong to the trusted domain.

[0034] The trusted domain setting includes two implementation methods. In the first implementation method, the trusted domain only includes the source node, that is, the terminal device or source edge node that directly receives user input, as the only trusted node. The node set of the trusted domain is set as follows: (7) in, Indicates the source node.

[0035] In the second implementation, the trusted domain includes the source node and several authorized trusted edge nodes. Therefore, the source node that directly receives user input will be... and A set of authorized trusted edge nodes collectively serve as trusted nodes, defining the trusted domain: (8) in, This indicates a trusted edge node that is authorized to participate in sensitive computations.

[0036] This step clarifies the range of nodes allowed to process sensitive intermediate activation values ​​by defining the trusted attributes of nodes and dividing trusted domains, providing a basis for subsequent restrictions on the deployment of sensitive computing units and the transmission of sensitive data streams.

[0037] In step 3, privacy protection depth refers to the number of front layers of the large language model that need to be restricted to execution within the trusted domain; sensitive computing unit refers to the computing unit that directly processes user input or generates highly sensitive intermediate activation values; sensitive data stream refers to the data dependency edges between sensitive computing units; intermediate activation values ​​refer to the intermediate computing results generated by the large model in the computing process of each layer, which may contain sensitive content such as user input semantics, contextual information or task intent.

[0038] In the specific implementation of step 3, for large language models with a Transformer structure, the first few layers are usually closer to the original input semantics, and the intermediate activation values ​​they generate may contain more information related to the user input. Therefore, this invention can set a privacy protection depth. Based on privacy protection depth Set a sensitive layer, and then... The relevant computational units in the layer are marked as sensitive computational units.

[0039] Privacy protection depth can be set using a fixed threshold or dynamically configured. When using a fixed threshold, the privacy protection depth is directly set according to the system's preset security policy. .

[0040] When using dynamic configuration, the source node performs a sensitivity assessment on the input content, task type, and user or business security level to obtain a comprehensive sensitivity score. The depth of privacy protection is dynamically determined based on a comprehensive sensitivity score. .

[0041] First, based on the sensitivity of the input content Task type sensitivity and user or business security level Calculate the overall sensitivity score: (9) In the formula, , , Let be the weighting coefficients, and satisfy: , , , .

[0042] The depth of privacy protection is determined based on a comprehensive sensitivity score: (10) In the formula, This is the preset minimum privacy protection depth; This represents the total number of layers in the large language model. This indicates the floor function (rounding up). This is understandable. The larger the value, the more front-end layers need to be restricted to execution within the trusted domain, and the stronger the privacy protection. The smaller the value, the more subsequent computing units can participate in cross-node collaborative execution, resulting in greater flexibility in inference deployment.

[0043] Determine the depth of privacy protection Then, let the set of sensitive layers be... .

[0044] Subsequently, all attention headers and attention aggregation units are marked as sensitive computation units, and these sensitive computation units are required to be deployed and executed only within a trusted domain, with their corresponding intermediate activation values ​​also only transmitted within the trusted domain. If a candidate deployment scheme violates these constraints, the scheme is deleted or readjusted; if no feasible scheme exists within the current trusted domain, the system can expand the trusted domain, reduce the scope of collaborative offloading, or reject insecure offloading. In this way, the privacy protection depth can be used as a fixed security policy or dynamically configured according to the sensitivity of the specific inference task.

[0045] For any layer, if All attention head blocks and attention aggregation units in this layer are marked as sensitive computational units, denoted as: In the formula, Indicates the first Layer One attention-grabbing block; Indicates the first Attention aggregation unit of the layer; This is a set of sensitive computing units. Specifically, this invention includes attention aggregation units. The sensitive computational unit set is included because the attention aggregation unit receives the outputs of multiple attention head blocks and concatenates and linearly projects these outputs. This operation concentrates the contextual information extracted by multiple attention heads into a new intermediate representation, which may contain more complete semantic information than the output of a single attention head. Therefore, its execution should be restricted to the trusted domain.

[0046] Furthermore, data dependency edges between sensitive computing units are marked as sensitive data flows, and the set of sensitive data flows is as follows: (11) In the formula, This represents the set of data dependency edges between all computational units in a fine-grained computation graph; Represents the source computation unit of a data-dependent edge; This represents the target computational unit for data-dependent edges. For sensitive data streams, their corresponding intermediate activation values ​​must not be transmitted from the trusted domain to the untrusted domain.

[0047] This step achieves differentiated protection for content with different sensitivity levels by setting up privacy protection to accurately identify sensitive computing units and sensitive data streams, thus ensuring privacy and security while maintaining sufficient deployment flexibility.

[0048] In step 4, the collaborative deployment scheme refers to the scheduling scheme that determines which node executes each fine-grained computing unit; the trusted domain constraint refers to the constraint that sensitive computing units can only be deployed on nodes within the trusted domain and sensitive data streams can only be transmitted within the trusted domain; the binary deployment variable is used to represent the binary identifier of the deployment location of the computing unit; and the deployment mapping refers to the function that establishes the correspondence between fine-grained computing units and execution nodes.

[0049] In the specific implementation of step 4, the available storage capacity, computing power, and link bandwidth information between nodes in the wireless edge network are first collected.

[0050] When performing collaborative reasoning, it is necessary to determine which node will execute each computational unit. Let the first node be... The deployment mapping within each inference cycle is as follows: (12) like: (13) This indicates a computing unit. In the Within each inference cycle, the nodes are... implement.

[0051] Subsequently, the trust region constraints are determined. Trust region constraints are expressed using binary deployment variables; specifically, for the first... Each inference cycle defines binary deployment variables: (14) The inference cycle can be a complete inference task or a scheduling phase within an inference task. Each computing unit can only be deployed to one node within the same inference cycle, thus satisfying the following constraints: (15) For sensitive computing units Its deployment nodes must belong to the trusted domain, thus satisfying the constraint: (16) For sensitive data streams Both the source computing unit and the target computing unit's deployment nodes must belong to the trusted domain, thus satisfying the constraint: (17) Equivalently, for binary deployment variables, we have: (18) when When it is a sensitive computing unit, if a certain node If it does not belong to the trusted domain, then As can be seen from the above constraints This means that the sensitive computing unit cannot be deployed on this node.

[0052] For sensitive data streams The deployment nodes of both the source computing unit and the target computing unit should belong to the trusted domain, thus satisfying the constraint condition: (19) In the formula, Indicates deployment variables; This represents the set of all computational units in a fine-grained computation graph; This represents the total number of nodes that can participate in collaborative reasoning; A trusted identifier for a node; A set of sensitive computing units; Represents a set of sensitive data streams; Represents the source computation unit of a data-dependent edge; Represents the target computational unit of the data-dependent edge; Indicates the first Deployment mapping within each inference cycle, Indicates the first There are 10 nodes that can participate in collaborative reasoning. The constraint in formula (17) is to prevent sensitive intermediate activation values ​​from being transmitted from the trusted domain to the untrusted domain. Even if an untrusted node has strong computing power, it cannot receive the intermediate activation values ​​corresponding to the sensitive data stream as long as it does not belong to the trusted domain.

[0053] In addition to meeting the trusted domain constraints, the deployment scheme must also meet the node resource constraints. (Design computing unit) The model parameter size is ,node The available storage capacity is Then deploy to the node The total number of computational unit parameters must not exceed the node storage capacity, satisfying the constraint: (20) We select all candidate deployment schemes that meet the trusted domain constraints and node resource constraints, and ensure that sensitive computing units are deployed only on nodes within the trusted domain.

[0054] Then, for each candidate deployment scheme, its corresponding total inference latency is calculated. The total inference latency is the sum of the computation latency of each computing unit and the transmission latency of intermediate activation values ​​across nodes. Specifically, the computing unit is designed... At the node The computational cost is ,node Its computing power is Then the computing unit At the node The computation delay is: (twenty one) If the calculation unit and If there is a data dependency between the two nodes and they are deployed on different nodes, then an intermediate activation value needs to be transferred between the two nodes.

[0055] set up Indicates from the computing unit Transmitted to computing unit The amount of intermediate activation value data, This indicates the distance between the two deployed nodes at the [number]th ...]. The available link bandwidth within each inference cycle, then the transmission delay of the intermediate activation value is: (twenty two) If the calculation unit and If deployed on the same node, no cross-node transmission occurs, and the corresponding transmission latency can be recorded as: (twenty three) Finally, the candidate deployment scheme with the minimum total inference latency is selected as the final collaborative deployment scheme. Specifically, first, all candidate schemes that violate the trusted domain constraint are eliminated, and then the deployment scheme with lower inference latency and more reasonable resource utilization is selected from the remaining candidate schemes. Let's assume the following deployment scheme is adopted. The reasoning delay is Privacy risk penalties are Then the deployment optimization objective can be expressed as: (twenty four) in, This represents the privacy risk weight. To structurally prevent sensitive computing units and sensitive data streams from crossing the trusted domain, this invention preferably employs a hard constraint approach: deployment schemes that satisfy the trusted domain constraint have a privacy risk penalty of 0, while deployment schemes that do not satisfy the trusted domain constraint are not selectable. (25) In practical implementation, the deployment decision module can generate a deployment scheme in the following way: First, select the set of deployable nodes for sensitive computing units based on the node trust identifier; second, select nodes that meet resource constraints based on storage capacity and computing power; third, estimate the transmission latency of intermediate activation values ​​across nodes based on link bandwidth; and finally, select the deployment scheme with lower overall latency among the schemes that meet the trust domain constraints and resource constraints.

[0056] If the trusted domain constraint of a certain inference task results in no feasible deployment scheme, the system can take one of the following actions: expand the trusted domain by adding authorized trusted edge nodes; reduce the scope of collaboration to allow more computing units to execute locally on the source node; or reject the candidate deployment scheme and regenerate a deployment scheme that satisfies the constraints.

[0057] This step constructs a trusted collaborative deployment optimization objective based on hard constraints by comprehensively considering trusted domain constraints, node resource constraints, and transmission latency. It generates a collaborative deployment scheme that can both ensure privacy and security and execute efficiently. Under the premise of strictly ensuring privacy and security, it maximizes the use of edge network resources and reduces inference latency.

[0058] In step 5, deployment execution refers to the process of distributing each computing unit to the corresponding node and starting computing according to the collaborative deployment scheme; transmission control refers to the process of checking and managing the cross-node transmission of intermediate activation values.

[0059] In the specific implementation of step 5, after determining the collaborative deployment scheme that satisfies the trusted domain constraints, collaborative inference is executed according to the fine-grained computation graph. For sensitive computation units, such as attention headers and attention aggregation units located within the privacy protection depth, they are only allowed to execute on trusted domain nodes. For sensitive data streams, such as data streams transmitted from sensitive attention headers to sensitive attention aggregation units, the system only allows them to be transmitted between trusted domain nodes. For non-sensitive computation units, such as some feedforward network units or subsequent network layer computation units beyond the privacy protection depth, the system can deploy them to untrusted edge nodes or cloud nodes for execution based on resource status.

[0060] Specifically, according to the collaborative deployment scheme, all sensitive computing units are distributed to corresponding nodes within the trusted domain for execution, and all non-sensitive computing units are distributed to corresponding nodes outside the trusted domain for execution. During inference execution, the transmission of all intermediate activation values ​​is monitored in real time. Before transmitting any intermediate activation value, the data stream corresponding to that intermediate activation value is determined. Is it a sensitive data stream? If the data stream is sensitive, check whether the deployment nodes of both the source and target computing units belong to the trusted domain. If they do, transmission is allowed; otherwise, transmission is prohibited, and the process returns to step 4 to regenerate the collaborative deployment scheme. If the data stream is not sensitive, transmission proceeds normally according to the collaborative deployment scheme. This check condition can be expressed as: (26) Through the above methods, a reliable collaborative reasoning mechanism is achieved, which restricts the transmission of sensitive intermediate activation values ​​and ensures the normal collaborative transmission of non-sensitive computation results.

[0061] This step further enhances privacy protection by performing real-time checks and controls on sensitive data streams during inference execution, ensuring that sensitive intermediate activation values ​​always flow within the trusted domain.

[0062] In step 6, collaborative reasoning execution refers to the process by which each node completes the computation task sequentially according to the dependencies of the fine-grained computation graph, and the reasoning result refers to the final result output by the large model after completing the reasoning task.

[0063] In the specific implementation of step 6, according to the deployment scheme determined in step 4 and the transmission control rules in step 5, each node executes its assigned computational tasks sequentially according to the execution order and data dependencies determined by the fine-grained computation graph. After the preceding computational unit completes its computation, it transmits the intermediate results to the node where the subsequent computational unit is located, until all computational tasks are completed and the final inference result is output. After the inference task is completed, the system records the deployment scheme of this inference task, the execution nodes of sensitive computational units, the transmission path of sensitive data streams, and the trusted domain check results. The above recorded information can be used for subsequent system auditing, deployment optimization, or security checks.

[0064] This step ensures the correctness and completeness of the reasoning process by executing the reasoning task in an orderly manner according to the fine-grained computation graph, while recording the reasoning process information to support the continuous optimization and security audit of the system.

[0065] It should be understood that, although this embodiment Figure 1 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0066] Example 2 Based on the large-model trusted collaborative reasoning method for wireless edge networks in Embodiment 1, this embodiment discloses a large-model trusted collaborative reasoning device for wireless edge networks, such as... Figure 2 As shown, the large-scale trusted collaborative reasoning device for wireless edge networks includes: a fine-grained computation graph construction module 701, a trusted domain configuration module 702, a sensitive labeling module 703, a deployment scheme generation module 704, a deployment and transmission control module 705, and a collaborative reasoning module 706, wherein: The fine-grained computation graph construction module 701 is used to obtain the large language model to be executed, divide the multi-head attention modules in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computation units, and construct a fine-grained computation graph according to the data dependency relationship between each fine-grained computation unit.

[0067] The Trusted Domain Configuration Module 702 is used to determine the trusted attributes of each node in the wireless edge network and set the trusted domain according to the trusted attributes.

[0068] The sensitive labeling module 703 is used to label sensitive computing units in the fine-grained computing graph according to the preset privacy protection depth, and to mark the data dependency edges between sensitive computing units as sensitive data streams.

[0069] The deployment scheme generation module 704 is used to obtain the available storage capacity and computing power of each node in the wireless edge network, and to obtain the link bandwidth between each node, and to generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraints.

[0070] The deployment and transmission control module 705 is used to deploy sensitive computing units to nodes within the trusted domain for execution according to the collaborative deployment scheme, and to deploy non-sensitive computing units to nodes outside the trusted domain for execution. During the execution process, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain.

[0071] The collaborative reasoning module 706 is used to complete collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

[0072] In this embodiment, the specific working processes and principles of the fine-grained computation graph construction module 701, trusted domain configuration module 702, sensitive labeling module 703, deployment scheme generation module 704, deployment and transmission control module 705, and collaborative reasoning module 706 are the same as those in Embodiment 1, and therefore will not be described again in this embodiment. Each unit module can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to the above unit modules.

[0073] Example 3 like Figure 3 The diagram illustrates a computer device disclosed in this embodiment, including a transmitter, a receiver, a memory, and a processor. The transmitter is used to send instructions and data, the receiver is used to receive instructions and data, the memory is used to store computer execution instructions, and the processor is used to execute the computer execution instructions stored in the memory to implement the method in Embodiment 1 above.

[0074] It is important to note that the aforementioned memory can be either standalone or integrated with the processor. When the memory is set up independently, the terminal device also includes a bus for connecting the memory and the processor.

[0075] Example 4 This embodiment discloses a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method in Embodiment 1 above.

[0076] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0077] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A trusted collaborative reasoning method for large-scale models in wireless edge networks, characterized in that, The method includes: Step 1: Obtain the large language model to be executed, divide the multi-head attention module in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computing units. Construct a fine-grained computing graph based on the data dependencies between each fine-grained computing unit. Step 2: Determine the trusted attributes of each node in the wireless edge network, and set the trusted domain according to the trusted attributes; Step 3: Based on the preset privacy protection depth, mark sensitive computing units in the fine-grained computation graph, and mark the data dependency edges between the sensitive computing units as sensitive data streams; wherein, the privacy protection depth adopts a fixed threshold method or a dynamic configuration method. When adopting the fixed threshold method, the privacy protection depth is directly set according to the system's preset security policy; when adopting the dynamic configuration method, a comprehensive sensitivity score is calculated by acquiring the sensitivity of input content, the sensitivity of task type, and the user or business security level, and the privacy protection depth is determined based on the comprehensive sensitivity score; When marking sensitive computing units, a set of sensitive layers is set according to the privacy protection depth. For any sensitive layer, all attention headers and attention aggregation units are marked as sensitive computing units. Step 4: Obtain the available storage capacity and computing power of each node in the wireless edge network, and obtain the link bandwidth between each node. Generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraints; wherein, the trusted domain constraints are expressed as binary deployment variables, including: For the Each inference cycle defines binary deployment variables: ； And it satisfies: ； Sensitive computing unit The deployment constraints are: ； For sensitive data streams Both the source computing unit and the target computing unit deployment nodes must belong to the trusted domain, that is: ； In the formula, Indicates deployment variables; This represents the set of all computational units in a fine-grained computation graph; This represents the total number of nodes that can participate in collaborative reasoning; A trusted identifier for a node; A set of sensitive computing units; Represents a set of sensitive data streams; Represents the source computation unit of a data-dependent edge; Represents the target computational unit of the data-dependent edge; Indicates the first Deployment mapping within each inference cycle; Indicates the first A node that can participate in collaborative reasoning; Subsequently, all candidate deployment schemes that meet the trusted domain constraints and node resource constraints are selected, ensuring that sensitive computing units are deployed only on nodes within the trusted domain. For each candidate deployment scheme, calculate the corresponding total inference latency; select the candidate deployment scheme with the minimum total inference latency as the final collaborative deployment scheme. Step 5: According to the collaborative deployment scheme, deploy the sensitive computing unit to the node within the trusted domain for execution, and deploy the non-sensitive computing unit to the node outside the trusted domain for execution. During the execution process, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain. Step 6: Perform collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

2. The large-model trusted collaborative reasoning method for wireless edge networks according to claim 1, characterized in that, Step 2 involves determining the trust attributes of each node in the wireless edge network, including: Let the set of nodes in the wireless edge network that can participate in collaborative reasoning be . ; Define a trusted identifier for each node: ； In the formula, A trusted identifier for a node; The set of nodes representing the trusted domain; where, Represents a node Belongs to the trusted domain. Represents a node It does not belong to the trusted domain.

3. The large-model trusted collaborative reasoning method for wireless edge networks according to claim 2, characterized in that, Step 2, setting the trusted domain based on the trusted attributes, includes: The source node will directly receive user input. The set of nodes that set up a trusted domain, serving as the only trusted node. .

4. The large-model trusted collaborative reasoning method for wireless edge networks according to claim 2, characterized in that, In step 2, setting the trusted domain based on the trusted attributes includes: including the source node that directly receives user input. and A set of authorized trusted edge nodes collectively serve as trusted nodes, defining the trusted domain. ; in, This indicates a trusted edge node that is authorized to participate in sensitive computations.

5. The large-model trusted collaborative reasoning method for wireless edge networks according to any one of claims 1 to 4, characterized in that, In step 3, when the preset privacy protection depth is configured dynamically, it includes: Get input content sensitivity Task type sensitivity and user or business security level ; Based on the sensitivity of the input content The sensitivity of the task type and the aforementioned user or business security level Calculate the overall sensitivity score: ； In the formula, , , These are the weighting coefficients; The depth of privacy protection is determined based on the comprehensive sensitivity score: ； In the formula, This is the preset minimum privacy protection depth; This represents the total number of layers in the large language model. This indicates the rounding up operation.

6. The large-model trusted collaborative reasoning method for wireless edge networks according to claim 5, characterized in that, In step 3, sensitive computational units are marked in the fine-grained computation graph according to the preset privacy protection depth, including: Let the set of sensitive layers be... ; For any sensitive layer All attention head blocks and attention aggregation units are marked as sensitive computational units, denoted as: In the formula, Indicates the first Layer One attention-grabbing block; Indicates the first Attention aggregation unit of the layer; A set of sensitive computing units; Data dependency edges between sensitive computing units are marked as sensitive data flows, and the set of sensitive data flows is as follows: ； In the formula, This represents the set of data dependency edges between all computational units in a fine-grained computation graph; Represents the source computation unit of a data-dependent edge; This represents the target computational unit for data-dependent edges.

7. The large-model trusted collaborative reasoning method for wireless edge networks according to any one of claims 1 to 4, characterized in that, In step 5, during execution, sensitive data streams are only allowed to be transmitted between nodes within the trusted domain, and transmission of sensitive data streams to nodes outside the trusted domain is prohibited, including: Before transmitting any intermediate activation value, determine whether the data stream corresponding to the intermediate activation value is a sensitive data stream; If it is a sensitive data stream, check whether the deployment nodes of the source computing unit and the target computing unit both belong to the trusted domain. If they both belong to the trusted domain, the transmission is allowed; otherwise, the transmission is prohibited and the collaborative deployment scheme is regenerated. If the data stream is not sensitive, it will be transmitted normally according to the collaborative deployment plan.

8. A large-scale trusted collaborative reasoning device for wireless edge networks, characterized in that, The device includes: The fine-grained computation graph construction module is used to obtain the large language model to be executed, divide the multi-head attention modules in each transformer layer of the large language model into multiple attention head blocks, and use the attention head blocks, attention aggregation units and feedforward network units in each transformer layer as fine-grained computation units, and construct a fine-grained computation graph based on the data dependencies between each fine-grained computation unit. The trusted domain configuration module is used to determine the trusted attributes of each node in the wireless edge network and set the trusted domain according to the trusted attributes. A sensitive labeling module is used to label sensitive computing units in the fine-grained computation graph according to a preset privacy protection depth, and to mark data dependency edges between the sensitive computing units as sensitive data streams. The privacy protection depth can be set using a fixed threshold or a dynamic configuration. When using a fixed threshold, the privacy protection depth is directly set according to the system's preset security policy. When using a dynamic configuration, a comprehensive sensitivity score is calculated based on the acquired input content sensitivity, task type sensitivity, and user or business security level, and the privacy protection depth is determined based on the comprehensive sensitivity score. When marking sensitive computing units, a set of sensitive layers is set according to the privacy protection depth. For any sensitive layer, all attention headers and attention aggregation units are marked as sensitive computing units. The deployment scheme generation module is used to obtain the available storage capacity and computing power of each node in the wireless edge network, and the link bandwidth between each node, and generate a collaborative deployment scheme under the condition of satisfying the trusted domain constraints; wherein, the trusted domain constraints are expressed in binary deployment variables, including: For the Each inference cycle defines binary deployment variables: ； And it satisfies: ； Sensitive computing unit The deployment constraints are: ； For sensitive data streams Both the source computing unit and the target computing unit deployment nodes must belong to the trusted domain, that is: ； In the formula, Indicates deployment variables; This represents the set of all computational units in a fine-grained computation graph; This represents the total number of nodes that can participate in collaborative reasoning; A trusted identifier for a node; A set of sensitive computing units; Represents a set of sensitive data streams; Represents the source computation unit of a data-dependent edge; Represents the target computational unit of the data-dependent edge; Indicates the first Deployment mapping within each inference cycle; Indicates the first A node that can participate in collaborative reasoning; Subsequently, all candidate deployment schemes that meet the trusted domain constraints and node resource constraints are selected, ensuring that sensitive computing units are deployed only on nodes within the trusted domain. For each candidate deployment scheme, calculate the corresponding total inference latency; select the candidate deployment scheme with the minimum total inference latency as the final collaborative deployment scheme. The deployment and transmission control module is used to deploy the sensitive computing unit to the node within the trusted domain for execution, and to deploy the non-sensitive computing unit to the node outside the trusted domain for execution, according to the collaborative deployment scheme. During the execution process, only sensitive data streams are allowed to be transmitted between nodes within the trusted domain, and sensitive data streams are prohibited from being transmitted to nodes outside the trusted domain. The collaborative reasoning module is used to complete collaborative reasoning according to the fine-grained computation graph and the execution order of each node, and output the reasoning results.

Images