Privacy computation and secure sharing method and device based on data identification

Abstract

The application relates to the technical field of data management, and particularly discloses a privacy calculation and security sharing method and device based on data identification. By acquiring a task logic description of a joint calculation task to be executed, a capability description of a first node and a second node, and a compensation operator directory, a scheduling node synchronously collects constraint data of each party, such as operator support types, bandwidth labels, maximum tolerable calculation overheads and maximum traffic thresholds, at the task initiation, and constructs a candidate atomic operator list according to the constraint data. By jointly analyzing the task logic and the node capability, a candidate sequence arranged in descending order of historical execution efficiency is generated, so that efficient operators can preferentially enter a verification process. Closed-loop adaptation based on the capability boundaries of each party is realized, and the scheduling success rate and overall execution efficiency of the joint calculation task between heterogeneous nodes are significantly improved under the premise of ensuring security.

Description

Technical Field

[0001] This invention belongs to the field of data management technology and relates to a method and apparatus for privacy computing and secure sharing based on data identification. Background Technology

[0002] Cross-platform privacy computation, as a key technology for breaking down data silos and enabling collaborative modeling and joint statistics among multiple parties, focuses on ensuring that all participants complete computational tasks without exposing their original data. Different privacy computation platforms deploy heterogeneous underlying security operators based on their respective security assumptions and compliance requirements, such as secure multi-party computation protocols under a semi-honest model and fully homomorphic encryption schemes based on lattice cryptography. Each operator represents a specific trade-off between computational efficiency, communication overhead, and security level, becoming the basic technical unit supporting the execution of privacy computation tasks.

[0003] In cross-platform collaborative scheduling scenarios, a single computational task often requires the cooperation of multiple participants, and the operators that each participant can invoke are pre-locked by their local hardware and software stacks and compliance policies. The scheduling center, as the unified coordinator of tasks, is responsible for mapping global computational logic onto operators that all parties can execute. Because the number of rounds and the amount of data transmitted per transaction differ significantly between different operators, the performance of the same task varies greatly under different operators. The scheduling process is essentially about finding feasible solutions in a discrete space defined by the capabilities of each participant. This makes scheduling efficiency highly dependent on the overall coordination of multi-dimensional constraints, rather than unilaterally pursuing the optimal performance of a single participant.

[0004] However, current mainstream scheduling methods exhibit significant shortcomings when facing heterogeneous constraint scenarios. On one hand, fixed operator selection mechanisms lack structured solutions to conflicts among multiple constraints, often rendering the entire task unexecutable due to bottlenecks in one party's capabilities, even though adaptation is still possible through local strategy adjustments. On the other hand, while exhaustive operator traversal theoretically has the potential to find a solution, its trial-and-error process lacks analysis and utilization of failure information, mechanically polling the candidate list and re-performing a complete feasibility check for each attempt, leading to a linear increase in scheduling latency with the number of operators and the dimension of constraints. When faced with complex constraints such as limited communication bandwidth among participants and a limited execution time window for a single task, existing mechanisms cannot specifically modify efficient but not fully constraint-satisfying operator schemes, and can only resort to choosing inefficient operators, resulting in a significant decrease in the overall throughput of privacy computing tasks. These shortcomings cause operator scheduling between heterogeneous platforms to face efficiency bottlenecks in practical applications, lacking a scheduling mechanism that optimizes the balance between feasibility and execution efficiency in a closed loop. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides a privacy computing and secure sharing method and apparatus based on data identification to solve the above-mentioned technical problems.

[0006] To achieve the above and other objectives, the technical solution adopted by the present invention is as follows: The first aspect of this invention provides a method for privacy-preserving computation and secure sharing based on data identification, the method comprising: Obtain the task logic description of the joint computation task to be executed; Obtain the capability description of the first node, which includes the operator types supported by the first node, the bandwidth label of the first node, and the maximum computational overhead that the first node can withstand. Obtain the capability description of the second node, which includes the operator types supported by the second node, the bandwidth label of the second node, and the maximum communication threshold corresponding to the bandwidth label of the second node. Obtain the compensation operator directory, which records the function type and performance overhead parameters of each compensation operator; Based on the task logic description, a candidate atomic operator list is generated from the operator types supported by the first node capability description and the second node capability description, and the candidate atomic operator list is sorted in descending order according to a preset execution efficiency rule. Extract the atomic operator that ranks first in the candidate atomic operator list, and calculate the first estimated total communication volume required for that atomic operator to perform the joint computation task; The first estimated total communication volume is compared with the maximum communication volume threshold. If the first estimated total communication volume does not exceed the maximum communication volume threshold, the atomic operator is directly selected as the task execution operator. If the first estimated total communication volume exceeds the maximum communication volume threshold, then a split compensation process is performed on the atomic operator; The splitting compensation process includes: splitting the atomic operator into a non-communication-intensive kernel and a communication-intensive periphery based on the protocol interaction logic of the atomic operator; selecting a compensation operator with a communication compression function from the compensation operator catalog, applying it to the communication-intensive periphery, and calculating the second estimated total communication volume after application and the additional local computing overhead generated for the first node; determining whether the second estimated total communication volume is less than or equal to the maximum communication volume threshold, and whether the additional local computing overhead is less than or equal to the maximum computing overhead that the first node can withstand; if both are satisfied, then encapsulating the atomic operator and the selected communication compression operator together into a combined execution operator; If any condition is not met during the split compensation process, the currently processed atomic operator is removed from the candidate atomic operator list, and the estimated total communication volume calculation, comparison, and split compensation process is repeated for the next atomic operator after descending order until a task execution operator or a combined execution operator is obtained. If all atomic operators in the candidate atomic operator list have been processed and no matching solution has been found, a task matching failure notification will be generated.

[0007] A second aspect of the present invention provides a privacy computing and secure sharing apparatus based on data identification, wherein the apparatus is configured with program instructions that, when executed, implement the privacy computing and secure sharing method based on data identification as described in any one of the present invention.

[0008] As described above, the privacy-preserving computation and secure sharing method and apparatus based on data identification provided by the present invention have at least the following beneficial effects: This invention generates a candidate atomic operator list by jointly parsing the capability identifiers of each node and the task logic. The list is then sorted in descending order based on historical execution efficiency. Starting with high-efficiency operators, the estimated communication volume is verified to ensure it meets the maximum communication volume threshold for low-bandwidth nodes. When the communication overhead of a high-efficiency operator exceeds the constraint, it is not discarded directly. Instead, a splitting and compensation process is initiated. This involves splitting the operator into a non-communication-intensive core and a communication-intensive periphery based on the operator's protocol interaction logic. A communication volume compression operator is then retrieved from the compensation operator directory and applied to the periphery. The compressed communication volume and additional computational overhead are re-estimated. When both the communication volume constraint and the computational overhead constraint are satisfied, the original operator and the compressed operator are encapsulated into a combined execution operator. This mechanism effectively avoids the rigidity of traditional scheduling, where high-efficiency operators are completely rejected due to single-dimensional constraint conflicts, forcing the selection of inefficient operators. It enables a closed-loop optimization process between feasibility and execution efficiency, resulting in scheduling results that better reflect the actual capability distribution in heterogeneous node coexistence scenarios.

[0009] This invention utilizes a progressive structure of split compensation and cyclic verification to transform previously mutually exclusive multi-party constraints into composable and adjustable matching conditions, significantly improving the success rate of cross-platform privacy computing task scheduling and overall execution efficiency. On one hand, the cyclic removal and replacement mechanism ensures that even when efficient operator split compensation fails, subsequent candidate operators can still be traversed in an orderly manner until the first task execution operator or combined execution operator that passes all verifications is locked. On the other hand, the generation of combined execution instruction packages clearly defines the local execution boundaries for each node, enabling the resolution of heterogeneous constraint conflicts through logical-level operator reconstruction even without introducing additional hardware. This scheduling method not only reduces scheduling latency caused by exhaustive traversal but also allows computing tasks that were originally rejected due to communication bottlenecks to be executed smoothly while considering the capacity of all parties, thereby improving the overall utilization efficiency of joint computing resources while ensuring security. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram showing the connections between the steps of the method of the present invention. Detailed Implementation

[0012] The present invention will be further described in detail below with reference to specific embodiments. The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the inventive concept or exceed the scope defined by the claims, all of which should fall within the protection scope of the present invention. Example

[0013] Please see Figure 1 As shown, the privacy-preserving computation and secure sharing method based on data identification includes: Obtain the task logic description of the joint computation task to be executed; Obtain the capability description of the first node, which includes the operator types supported by the first node, the bandwidth label of the first node, and the maximum computational overhead that the first node can withstand. Obtain the capability description of the second node, which includes the operator types supported by the second node, the bandwidth label of the second node, and the maximum communication threshold corresponding to the bandwidth label of the second node. Obtain the compensation operator directory, which records the function type and performance overhead parameters of each compensation operator.

[0014] This method operates on a scheduling node that connects multiple data participants. When a multi-party joint computation task is initiated, the scheduling node first gathers the capability constraints and available compensation methods of each party, providing a data foundation for subsequent operator loop selection and split compensation.

[0015] The scheduling node first obtains the task logic description of the joint computation task to be executed. This task logic description is a structured data submitted by the task initiator, the core of which is to declare what the computation will do. Specifically, the scheduling node parses this structured data, extracting the core operation type, the data pattern identifiers participating in the computation, and the security requirement tags for the computation results. The core operation type indicates whether the computation task performs addition, multiplication, comparison, or more complex compound operations; the data pattern identifiers participating in the computation indicate the format of the data fields that each participant needs to contribute; and the security requirement tags declare the minimum security level required by the task. This parsed information collectively forms the basis for subsequent operator selection.

[0016] Simultaneously with or after obtaining the task logic description, the scheduling node establishes communication with the first node, requesting and receiving the first node's capability description. The first node's capability description is generated by the first node based on its own hardware and software configuration and privacy compliance policies. It includes three data items with explicit values ​​or labels. The first item is the types of operators supported by the first node. This is an enumerated list of values, specifying the operators that the first node's local cryptography library can directly call and execute, such as whether it supports obfuscated circuit operators based on a semi-honest model, or arithmetic operators based on fully homomorphic encryption. The second item is the first node's bandwidth label, a qualitative label characterizing network capabilities. For example, based on the measured value of the first node's upload bandwidth, it is compared with several preset bandwidth thresholds and labeled as high bandwidth or low bandwidth. The third item is the first node's maximum tolerable computational overhead. This is a specific value, determined by converting the first node's maximum CPU computation time and memory overhead allocated for a single task into a dimensionless computational overhead value using the unit computational load conversion relationship obtained from hardware benchmark tests. This limit represents the maximum local computing cost that the first node is willing and able to pay for this joint computing task.

[0017] Similarly, the scheduling node also sends a capability request to the second node to obtain its capability description. This description has a similar data structure to the first node, but differs depending on the scenario's focus. Specifically, it includes the types of operators supported by the second node and its bandwidth label. Notably, because the second node's bandwidth label is set to a low-bandwidth label, a maximum communication threshold is forcibly associated with its capability description. This threshold is generated as follows: the second node reads the preset maximum number of bits transmitted per unit time corresponding to its low-bandwidth label, combines this with the maximum allowed execution time of the current joint computation task, and multiplies the two to obtain an upper limit value for the total data exchange that the second node can handle throughout the entire task cycle. This upper limit value is quantified as the maximum communication threshold. The introduction of this threshold transforms the vague description of the second node's low bandwidth into a hard constraint that can be directly compared with the estimated total communication volume in subsequent calculations.

[0018] Afterward, the scheduling node will access a local compensation operator repository to obtain a compensation operator directory. This directory is a pre-configured mapping table, where each record corresponds to an available compensation operator. Each record contains two core fields. The first field is the function type, which uses an enumeration value to indicate what problem the compensation operator is used to solve, such as traffic compression. The second field is the performance overhead parameter, a composite data structure that specifically records the compression ratio of the compensation operator and the local computational cost required per unit of data processed. The compression ratio directly determines the rate by which data transmission volume can be reduced after applying the compensation operator; while the local computational cost required per unit of data processed provides the initial computational factor for subsequent calculations to incur additional local computational overhead. The scheduling node loads the entire directory into memory for retrieval and invocation during subsequent loop splitting processes.

[0019] Based on the task logic description, a candidate atomic operator list is generated from the operator types supported by the first node capability description and the second node capability description, and the candidate atomic operator list is sorted in descending order according to a preset execution efficiency rule. Preferably, the step of generating a candidate atomic operator list from the operator types supported by the first node capability description and the second node capability description based on the task logic description, and sorting the candidate atomic operator list in descending order according to a preset execution efficiency rule, includes: Analyze the core operation types in the task logic description; Extract all atomic operators that support the execution of the core operation type from both the first node capability description and the second node capability description, and generate an initial set of atomic operators; Obtain the historical execution efficiency reference value of each atomic operator in the initial atomic operator set, and sort the initial atomic operator set in descending order according to the historical execution efficiency reference value to obtain the candidate atomic operator list.

[0020] After obtaining the task logic description, the capability description of each node, and the compensation operator catalog, the scheduling node begins to construct a list of candidate atomic operators for subsequent iterative filtering. The core problem this process needs to solve is: from the operators supported by both parties, find a sequence ranked from high to low execution efficiency, so as to prioritize trying the most efficient solution.

[0021] The scheduling node first parses the task logic description to extract the core operation type. The task logic description is a structured data body containing a field specifically for declaring operation requirements. The scheduling node reads the value of this field, which enumerates the essential operation category of the joint computation task. For example, when the task is to sum the values ​​held by two parties, the core operation type is marked as addition; when the task is to determine whether one value is greater than another, the core operation type is marked as comparison. The extracted core operation type is a clear category label, and subsequent operator matching will strictly revolve around this label.

[0022] After extracting the core operation types, the scheduling node begins comparing them with the capability descriptions of the first and second nodes to extract all atomic operators that both nodes support for executing that core operation type. Specifically, the scheduling node iterates through each operator type entry listed in the first node's capability description, checking if the set of executable operation types declared for each entry contains the extracted core operation type tag. If it does, the operator type is added to a temporary first support list. The same iteration and checking operation is performed on the second node's capability description, generating a temporary second support list. After constructing both lists, the scheduling node finds the intersection of these two lists, identifying operator types that appear in both the temporary first and temporary second support lists. Each operator type in this intersection represents an atomic operator that both nodes can execute locally and can complete the core operation. These atomic operators are then grouped together to form the initial atomic operator set.

[0023] Next, the scheduling node needs to assign a quantified efficiency evaluation metric to each atomic operator in the initial atomic operator set for sorting. The scheduling node maintains a local historical execution efficiency archive, indexed by operator type, recording the measured performance of each operator in past tasks of similar scale. For each atomic operator in the current initial atomic operator set, the scheduling node retrieves its corresponding historical execution efficiency reference value from this archive. The logic for generating this reference value is as follows: whenever an atomic operator is used to complete a joint computation task, the scheduling node records the data scale of the task and the total time taken by the operator from start to finish, calculates the time taken by the operator to process a unit of data, and appends this unit data processing time as a historical record to the operator's archive entry. When a historical execution efficiency reference value is needed, the scheduling node reads the unit data processing time of the most recent records under the operator's entry, takes their arithmetic mean, and uses this as the current historical execution efficiency reference value. The smaller this value, the higher the historical execution efficiency of the operator.

[0024] After obtaining the historical execution efficiency reference values ​​for each atomic operator in the initial atomic operator set, the scheduling node sorts these values ​​in descending order. The logic for descending order is as follows: comparing the historical execution efficiency reference values ​​of each atomic operator, since the smaller the reference value, the higher the efficiency, the atomic operator with the smallest reference value is assigned the highest ranking position and placed at the beginning of the list; the atomic operator with the largest reference value is assigned the lowest ranking position and placed at the end of the list. After sorting, this ordered list is officially recorded as the candidate atomic operator list. The first atomic operator in this list is the preferred solution with the most efficient historical performance under the current constraints of both parties' capabilities. At this point, the scheduling node has completed the construction of the candidate pool and can enter the subsequent stage of verification and splitting compensation.

[0025] Extract the atomic operator that ranks first in the candidate atomic operator list, and calculate the first estimated total communication volume required for that atomic operator to perform the joint computation task; Preferably, calculating the first estimated total communication volume required for the atomic operator to perform the joint computation task includes: Obtain the protocol interaction step sequence of this atomic operator; Identify each communication step in the protocol interaction sequence that requires data exchange between the first node and the second node; Based on the data transmission scale corresponding to each communication link, and combined with the network condition parameters mapped by the bandwidth labels of the first node and the second node, the first estimated total communication volume is calculated by summing them up.

[0026] Preferably, the first node bandwidth label is a high bandwidth label, the second node bandwidth label is a low bandwidth label, and the maximum communication threshold is a specific value preset based on the low bandwidth label.

[0027] After the scheduling node selects the top-ranked atomic operator from the candidate atomic operator list, it begins calculating the communication overhead between the two nodes if this operator is used to execute the current joint computation task. The result of this calculation is the first estimated total communication overhead, which serves as the direct basis for subsequent comparisons with the maximum communication overhead threshold.

[0028] The computation process begins with obtaining the protocol interaction step sequence for the atomic operator. Each atomic operator has a corresponding protocol description file locally on the scheduling node. This file records all the operations the operator undergoes from receiving input to producing output, in the form of a step sequence. The scheduling node reads this file and parses it into a linear sequence of steps. Each step in the sequence carries a type label to distinguish whether the step belongs to a local computation operation or a communication operation that requires data exchange with the peer node.

[0029] After parsing, the scheduling node traverses the sequence of steps, identifying all steps labeled as communication operations and marking them as communication links. For each identified link, the scheduling node extracts two key pieces of information from its description structure. The first is the scale of the data transmitted in that link, directly derived from the size declared for the data field in the task logic description, such as the preset number of bits or bytes. The second is the communication direction of that link, i.e., whether the data is sent from the first node to the second node or vice versa. This direction information determines which node's bandwidth should be used as a constraint in subsequent calculations.

[0030] After extracting all communication links and their transmitted data scales, the scheduling node introduces bandwidth conditions for conversion. The bandwidth labels of the first and second nodes each map to a network condition parameter. This parameter represents the upper limit of the data volume that the corresponding node can stably transmit per unit time, and its value comes from the results measured and reported by the bandwidth measurement tool during node registration. For scenarios where the first node has a high bandwidth label and the second node has a low bandwidth label, the scheduling node uses the network condition parameter mapped to the low bandwidth label as the constraint bandwidth for each communication link. The logic behind this approach is that the actual throughput of a bidirectional communication is limited by the lower bandwidth side; therefore, using the parameter of the lower bandwidth side to calculate the estimated cost yields a more conservative but realistic estimate.

[0031] For each communication link, the scheduling node divides the data transmission volume of that link by the constrained bandwidth to obtain the estimated transmission time for that link. The product of this estimated transmission time and the data transmission volume of that link is then used as the link's contribution to the total communication overhead. This approach, rather than simply summing the data volumes, is used because communication overhead is not only related to the data volume but also proportional to the length of time the channel is occupied. Incorporating the time factor into the calculation allows for a more consistent metric for subsequent comparisons with the maximum communication volume threshold. Alternatively, a more direct method can be used: directly summing the data transmission volumes of each communication link to obtain a total communication volume. While this method omits the time dimension, it is simpler to calculate and is also applicable in this scheme.

[0032] After summing up the communication overhead contributions of all communication links, the total sum is the first estimated total communication volume. This value represents the overall communication cost required to complete the joint computation task using the current atomic operator under ideal network conditions. The scheduling node then compares this first estimated total communication volume with the maximum communication volume threshold in the second node's capability description to determine whether the current atomic operator is within the communication capacity of the second node.

[0033] Here, it's necessary to further explain the origin of the maximum communication volume threshold. When the second node registers, based on its low-bandwidth label, it looks up the corresponding maximum transmission volume per unit time from a preset configuration table. This maximum transmission volume per unit time is a preset value that the second node sets and reports based on historical network conditions and service tolerance. When the scheduling node obtains the second node's capability description, it uses the maximum allowed execution time of the task declared in the task logic description as a time window. It multiplies the maximum transmission volume per unit time by this time window to calculate an upper limit on the total data exchange that the second node can accept throughout the entire task cycle. This upper limit is the maximum communication volume threshold. As a hard constraint, it is directly used by the scheduling node to compare with the first estimated total communication volume. When the first estimated total communication volume is less than or equal to this threshold, it indicates that the current atomic operator is feasible at the communication level and can directly enter the selection and confirmation phase; otherwise, the subsequent splitting and compensation process is triggered.

[0034] The first estimated total communication volume is compared with the maximum communication volume threshold. If the first estimated total communication volume does not exceed the maximum communication volume threshold, the atomic operator is directly selected as the task execution operator. If the first estimated total communication volume exceeds the maximum communication volume threshold, then a split compensation process is performed on the atomic operator; The splitting compensation process includes: splitting the atomic operator into a non-communication-intensive kernel and a communication-intensive periphery based on the protocol interaction logic of the atomic operator; selecting a compensation operator with a communication compression function from the compensation operator catalog, applying it to the communication-intensive periphery, and calculating the second estimated total communication volume after application and the additional local computing overhead generated for the first node; determining whether the second estimated total communication volume is less than or equal to the maximum communication volume threshold, and whether the additional local computing overhead is less than or equal to the maximum computing overhead that the first node can withstand; if both are satisfied, then encapsulating the atomic operator and the selected communication compression operator together into a combined execution operator; Preferably, the step of splitting the atomic operator into a non-communication-intensive kernel and a communication-intensive periphery based on the protocol interaction logic includes: Obtain the complete sequence of execution steps for this atomic operator; All operation steps in the complete execution sequence that are limited to being computed locally on a single node and do not require synchronous data exchange with another node are classified as non-communication-intensive kernels. The remaining steps in the complete execution sequence that require synchronous data exchange between the first node and the second node are classified as the communication-intensive periphery.

[0035] Preferably, the second estimated total communication volume after the calculation application and the additional local computing overhead incurred on the first node include: Based on the compression ratio recorded in the performance overhead parameters of the selected communication compression operator, the original communication data volume involved in the communication-intensive periphery is compressed and converted to obtain the second estimated total communication volume. Based on the computational cost required per unit of data compression recorded in the performance overhead parameters, and combined with the data scale involved in the communication-intensive peripheral, the additional local computational overhead is calculated.

[0036] Preferably, the step of encapsulating the atomic operator and the selected traffic compression operator together into a combined execution operator includes: A combined execution instruction package is generated, which includes instructions for instructing the first node to execute the non-communication-intensive kernel and instructions for instructing the second node to execute the communication-intensive periphery processed by the traffic compression operator, thereby serving as the combined execution operator.

[0037] Preferably, if during the split compensation process, the second estimated total communication volume is still greater than the maximum communication volume threshold, or the additional local computing overhead is greater than the maximum computing overhead that the first node can withstand, then the split compensation process further includes: Select at least one compensation operator with traffic compression as its function type from the compensation operator catalog; Based on preset combination rules, multiple selected traffic compression operators are logically combined to form composite compression logic; The composite compression logic is applied to the communication-intensive periphery to recalculate the third estimated total communication volume and the new additional local computing overhead. Based on the maximum communication volume threshold and the maximum tolerable computing overhead of the first node, a further judgment is made to obtain the combined execution operator.

[0038] After obtaining the initial estimated total traffic, the scheduling node enters a critical bidirectional verification branch. This branch directly determines whether the top-ranked atomic operator is directly adopted or requires a more complex splitting and compensation process.

[0039] The scheduling node first compares the estimated total communication volume with the maximum communication volume threshold carried in the second node's capability description. The logic of this comparison is straightforward: the scheduling node checks if the estimated total communication volume is less than or equal to the maximum communication volume threshold. If this condition is met, it means that using the current atomic operator, the total data exchange generated during the entire task execution will not reach the upper limit that the second node's low-bandwidth link can handle. In this case, no additional processing is necessary, and the scheduling node directly marks this atomic operator as a task execution operator. After marking, the subsequent step is to send the identifier and execution configuration of this task execution operator to both the first and second nodes, and the task enters the execution phase.

[0040] Conversely, if the first estimated total communication volume exceeds the maximum communication volume threshold, it means that the current atomic operator exceeds the second node's capacity at the communication level. However, directly discarding it and trying a less efficient candidate operator is often not cost-effective in terms of overall performance. In this case, the scheduling node does not discard the atomic operator directly, but instead initiates a splitting and compensation process for it.

[0041] The first step in the compensation process is to split the atomic operator into a non-communication-intensive kernel and a communication-intensive periphery, based on its protocol interaction logic. This is achieved by the scheduling node retrieving the complete execution step sequence from the atomic operator's protocol description file. Each step in this sequence carries a specific execution location label, which has only two values: local or remote. Local means the operation is completed entirely within the memory and processor of the first or second node, without involving the peer; remote means the step requires data transmission between the two nodes. The scheduling node iterates through this complete execution step sequence, selecting all steps with the local location label and combining them into a subsequence in their original execution order. This subsequence is named the non-communication-intensive kernel. Simultaneously, steps with the remote location label are also combined into another subsequence in their original order, named the communication-intensive periphery. In this way, the original atomic operator is logically split into two parts: a pure computational part and a communication interaction part.

[0042] After the splitting is complete, the scheduling node searches the previously loaded compensation operator directory for compensation operators whose function type is marked as traffic compression. The compensation operator directory is essentially a mapping table, with the function type as the index key. The scheduling node uses the traffic compression enumeration value to match and find the corresponding compensation operator entry, then reads its performance overhead parameters. The performance overhead parameters include two values: the compression ratio and the computational cost per unit of data compression. The compression ratio is defined as the amount of data before compression divided by the amount of data after compression; it is a real number greater than one. For example, a compression ratio of five means that one hundred units of data are compressed into twenty units. The computational cost per unit of data compression refers to the local CPU computational overhead required to compress one unit of data; this overhead uses the same unit of measurement as the maximum computational overhead that the first node can withstand.

[0043] The scheduling node applies the selected traffic compression operator to the communication-intensive periphery. This means that, assuming the compression operator is applied, the original traffic volume in each communication link involved in the communication-intensive periphery is re-estimated. For each communication link in the communication-intensive periphery, the scheduling node extracts its original traffic volume and divides it by the compression ratio; the quotient is the compressed traffic volume for that link. Summing up the compressed traffic volumes of all communication links yields the second estimated total traffic volume. Simultaneously, the scheduling node also needs to calculate the additional burden this compression operation places on the first node. It sums up the total original traffic volume involved in the communication-intensive periphery to obtain the total data size to be compressed, and then multiplies this total data size by the computational cost required to compress each unit of data; the product is the additional local computational overhead.

[0044] With the new values ​​of the second estimated total communication volume and the additional local computational overhead, the scheduling node enters another concurrent judgment process. It checks whether the second estimated total communication volume is less than or equal to the maximum communication volume threshold, and whether the additional local computational overhead is less than or equal to the maximum tolerable computational overhead of the first node. Both conditions must be met simultaneously to mean that the compression scheme, while addressing the communication bottleneck, has not placed demands on the first node's computational capabilities beyond its capacity. If both conditions are satisfied, the scheduling node encapsulates the current atomic operator and the selected communication volume compression operator together into a combined execution operator.

[0045] The specific implementation of encapsulating the combined execution operator involves generating a combined execution instruction package. This instruction package is a structured data block containing two execution instructions. The first instruction is received by the first node, and its content includes all the non-communication-intensive kernel operation steps that the first node needs to execute, along with a notification that the data at the other end of the communication has been compressed. The second instruction is received by the second node, and its content includes all the communication-intensive peripheral operation steps that the second node needs to execute after being processed by the traffic compression operator, and it specifies the identifier of the compression operator so that the second node can call the corresponding decompression logic when decompressing the communication data. This combined execution instruction package, as a whole, is regarded as the final selected combined execution operator by the scheduled node and can be issued to both nodes for execution.

[0046] In actual processing, another scenario exists: after the scheduling node selects a traffic compression operator and calculates the second estimated total traffic and additional local computational overhead, it determines that the second estimated total traffic is still greater than the maximum traffic threshold, or the additional local computational overhead exceeds the maximum computational overhead that the first node can withstand. In this case, the scheduling node will not immediately declare the current atomic operator unavailable, but will instead attempt another effort using composite compression.

[0047] The scheduling node selects at least one compensation operator with traffic compression as its function type from the compensation operator catalog. The compression ratio and computational cost per unit of data compression for the second selected compensation operator may be the same as or different from the first selection. After selection, the scheduling node logically combines the multiple selected traffic compression operators according to a preset combination rule to form a composite compression logic. The basic idea of ​​this combination rule is: when multiple compression operators operate serially, the data is first compressed by the first compression operator, and the output is then used as the input for the second compression operator for secondary compression. Therefore, the composite compression ratio is the product of the individual compression ratios, and the composite computational overhead is the sum of the computational overhead incurred by each compression operator in processing the original data size. Of course, parallel combination can also be used, but this usually introduces additional data fragmentation overhead; in implementation, serial combination is more direct.

[0048] The scheduling node applies this composite compression logic to the communication-intensive periphery, recalculating in a serial composite manner. When recalculating the third estimated total communication volume, it takes the original communication data volume of the communication-intensive periphery and divides it sequentially by the compression ratio of each compression operator, or directly by the composite compression ratio, to obtain the final compressed data volume. When recalculating the new additional local computational overhead, it multiplies the total data size to be compressed by the computational cost per unit of data compression required by each compression operator, and then adds the products to obtain the total additional computational overhead. After the calculation is completed, the scheduling node again judges based on the maximum communication volume threshold and the maximum computational overhead that the first node can withstand. If this judgment passes, it packages the atomic operator with the selected multiple communication volume compression operators to generate a combined execution operator; if it still fails, the scheduling node confirms that all attempts to split and compensate for the current atomic operator have failed. Subsequently, it removes the current atomic operator from the candidate atomic operator list, takes the next atomic operator from the list, and restarts the entire process from calculating the first estimated total communication volume to split and compensate, repeating this process until a feasible operator is found or the confirmation list is exhausted.

[0049] If any condition is not met during the split compensation process, the currently processed atomic operator is removed from the candidate atomic operator list, and the estimated total communication volume calculation, comparison, and split compensation process is repeated for the next atomic operator after descending order until a task execution operator or a combined execution operator is obtained. If all atomic operators in the candidate atomic operator list have been processed and no matching solution has been found, a task matching failure notification will be generated.

[0050] When the scheduling node's splitting and compensation process for the current atomic operator fails to simultaneously satisfy both the second estimated total communication volume being less than or equal to the maximum communication volume threshold and the additional local computational overhead being less than or equal to the maximum tolerable computational overhead of the first node (including cases where this is still not satisfied after attempting composite compression logic), the scheduling node will confirm that even after splitting and compensation, the current atomic operator cannot achieve a balance between the communication constraints of the second node and the computational constraints of the first node. At this point, the scheduling node performs a clear removal operation, deleting the currently processed atomic operator from the candidate atomic operator list. Essentially, this deletion action involves clearing the entire entry corresponding to the current index position in an ordered list data structure, and shifting the index of subsequent entries one position forward.

[0051] After removal, the scheduling node checks if there are any remaining entries in the list. If the list is not empty, the scheduling node extracts the new first-ranked atomic operator by following the descending order of the list; that is, the atomic operator that was originally second and is now moved to the first position. For this newly extracted atomic operator, the scheduling node no longer takes any shortcuts but starts from calculating its first estimated total communication volume and completely repeats the aforementioned process. In other words, the scheduling node re-acquires the protocol interaction step sequence of the atomic operator, identifies its communication links, calculates the new first estimated total communication volume, and compares it with the maximum communication volume threshold. If the new first estimated total communication volume does not exceed the threshold, the atomic operator is directly selected as the task execution operator, and the loop terminates. If the new first estimated total communication volume still exceeds the threshold, the scheduling node restarts the splitting and compensation process for this new atomic operator, attempting to find a balance point that satisfies both constraints through a combination of communication volume compression operators or composite compression logic. If the conditions are met after splitting and compensating, a combined execution operator is encapsulated, and the loop terminates; if the conditions are still not met after splitting and compensating, the removal operation is executed again, and the loop continues.

[0052] There are two termination conditions for this loop process. The first termination condition is that, in a certain loop iteration, the scheduling node successfully acquires either a task execution operator or a combined execution operator. Once acquired, the scheduling node immediately exits the loop and no longer processes the remaining atomic operators in the list. The second termination condition is that all atomic operators in the candidate atomic operator list have been removed, the list becomes empty, and no matching solution has been found throughout this process. At this point, the scheduling node confirms that no feasible execution solution can be found within the current node capability constraints and available compensation methods. Based on this state, the scheduling node generates a task matching failure notification. This notification is a structured message containing a failure reason code and a brief description, indicating that the failure type is "no feasible operator," and attaching the joint computation task identifier, the first node identifier, the second node identifier, and a summary of the capability constraints of both parties involved in this scheduling. After generation, the scheduling node returns this notification to the task initiator, and the entire scheduling process ends. Example

[0053] A privacy-preserving computation and secure sharing device based on data identification is provided, wherein the device is configured with program instructions, and when the program instructions are executed, the privacy-preserving computation and secure sharing method based on data identification as described in any one of the present invention is implemented.

[0054] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0055] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0056] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0057] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A privacy-preserving computation and secure sharing method based on data identifiers, characterized in that: include: Obtain the task logic description of the joint computation task to be executed; Obtain the capability description of the first node, which includes the operator types supported by the first node, the bandwidth label of the first node, and the maximum computational overhead that the first node can withstand. Obtain the capability description of the second node, which includes the operator types supported by the second node, the bandwidth label of the second node, and the maximum communication threshold corresponding to the bandwidth label of the second node. Obtain the compensation operator directory, which records the function type and performance overhead parameters of each compensation operator; Based on the task logic description, a candidate atomic operator list is generated from the operator types supported by the first node capability description and the second node capability description, and the candidate atomic operator list is sorted in descending order according to a preset execution efficiency rule. Extract the atomic operator that ranks first in the candidate atomic operator list, and calculate the first estimated total communication volume required for that atomic operator to perform the joint computation task; The first estimated total communication volume is compared with the maximum communication volume threshold. If the first estimated total communication volume does not exceed the maximum communication volume threshold, the atomic operator is directly selected as the task execution operator. If the first estimated total communication volume exceeds the maximum communication volume threshold, then a split compensation process is performed on the atomic operator; The splitting compensation process includes: splitting the atomic operator into a non-communication-intensive kernel and a communication-intensive periphery based on the protocol interaction logic of the atomic operator; selecting a compensation operator with a communication compression function from the compensation operator catalog, applying it to the communication-intensive periphery, and calculating the second estimated total communication volume after application and the additional local computing overhead generated for the first node; determining whether the second estimated total communication volume is less than or equal to the maximum communication volume threshold, and whether the additional local computing overhead is less than or equal to the maximum computing overhead that the first node can withstand; if both are satisfied, then encapsulating the atomic operator and the selected communication compression operator together into a combined execution operator; If any condition is not met during the split compensation process, the currently processed atomic operator is removed from the candidate atomic operator list, and the estimated total communication volume calculation, comparison, and split compensation process is repeated for the next atomic operator after descending order until a task execution operator or a combined execution operator is obtained. If all atomic operators in the candidate atomic operator list have been processed and no matching solution has been found, a task matching failure notification will be generated.

2. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, Based on the task logic description, a candidate atomic operator list is generated from the operator types supported by the first node capability description and the second node capability description, and the candidate atomic operator list is sorted in descending order according to a preset execution efficiency rule, including: Analyze the core operation types in the task logic description; Extract all atomic operators that support the execution of the core operation type from both the first node capability description and the second node capability description, and generate an initial set of atomic operators; Obtain the historical execution efficiency reference value of each atomic operator in the initial atomic operator set, and sort the initial atomic operator set in descending order according to the historical execution efficiency reference value to obtain the candidate atomic operator list.

3. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, The calculation of the first estimated total communication volume required for the atomic operator to perform the joint computation task includes: Obtain the protocol interaction step sequence of this atomic operator; Identify each communication step in the protocol interaction sequence that requires data exchange between the first node and the second node; Based on the data transmission scale corresponding to each communication link, and combined with the network condition parameters mapped by the bandwidth labels of the first node and the second node, the first estimated total communication volume is calculated by summing them up.

4. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, Based on the protocol interaction logic of this atomic operator, it is divided into a non-communication-intensive kernel and a communication-intensive periphery, including: Obtain the complete sequence of execution steps for this atomic operator; All operation steps in the complete execution sequence that are limited to being computed locally on a single node and do not require synchronous data exchange with another node are classified as non-communication-intensive kernels. The remaining steps in the complete execution sequence that require synchronous data exchange between the first node and the second node are classified as the communication-intensive periphery.

5. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, The second estimated total communication volume after the calculation application and the additional local computing overhead generated for the first node include: Based on the compression ratio recorded in the performance overhead parameters of the selected communication compression operator, the original communication data volume involved in the communication-intensive periphery is compressed and converted to obtain the second estimated total communication volume. Based on the computational cost required per unit of data compression recorded in the performance overhead parameters, and combined with the data scale involved in the communication-intensive peripheral, the additional local computational overhead is calculated.

6. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, The step of encapsulating the atomic operator and the selected traffic compression operator together into a combined execution operator includes: A combined execution instruction package is generated, which includes instructions for instructing the first node to execute the non-communication-intensive kernel and instructions for instructing the second node to execute the communication-intensive periphery processed by the traffic compression operator, thereby serving as the combined execution operator.

7. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, If, during the split compensation process, the second estimated total communication volume still exceeds the maximum communication volume threshold, or the additional local computing overhead exceeds the maximum computing overhead that the first node can withstand, then the split compensation process further includes: Select at least one compensation operator with traffic compression as its function type from the compensation operator catalog; Based on preset combination rules, multiple selected traffic compression operators are logically combined to form composite compression logic; The composite compression logic is applied to the communication-intensive periphery to recalculate the third estimated total communication volume and the new additional local computing overhead. Based on the maximum communication volume threshold and the maximum tolerable computing overhead of the first node, a further judgment is made to obtain the combined execution operator.

8. The privacy-preserving computation and secure sharing method based on data identification according to claim 1, characterized in that, The first node bandwidth label is a high bandwidth label, the second node bandwidth label is a low bandwidth label, and the maximum communication threshold is a specific value preset based on the low bandwidth label.

9. A privacy-preserving computation and secure sharing device based on data identification, characterized in that, The device is equipped with program instructions that, when executed, implement the privacy computing and secure sharing method based on data identification as described in any one of claims 1 to 8.

Images