A task scheduling method based on distributed privacy computing
By adopting distributed task scheduling methods in the privacy data computing platform, dynamically analyze task scheduling and multi-dimensional analysis task allocation, the privacy data leakage problem caused by simple task scheduling of the privacy data computing platform in the existing technology is solved, and the stability of computing is improved Sex and efficiency.
Patent Information
- Application Number
- CN202410961462.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-18
AI Technical Summary
When existing privacy data computing platforms handle large-scale privacy data, the task scheduling method is too simple, which can easily lead to privacy data leakage, which in turn affects the reliability and stability of the calculation.
A task scheduling method based on distributed privacy computing is adopted to obtain task scheduling values and privacy data complexity in real time, dynamically analyze whether task scheduling is needed, and rationalize task allocation through multi-dimensional analysis, using cosine similarity algorithm and knowledge graph Correlation analysis determines the final target node to ensure stable and reliable calculation of privacy data.
It improves the accuracy and rationality of privacy computing task scheduling, enhances the stability and computing efficiency of privacy data, avoids data leakage, and ensures the reliability and stability of the platform.
Smart Images

Figure CN118567815B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of privacy computing technology, and more specifically, to a task scheduling method based on distributed privacy computing. Background Art
[0002] Privacy computing is a technical system that aims to achieve the compliant and orderly release of data value under the premise of ensuring data security. The current privacy data computing platform needs to process various types of privacy data. In order to ensure the reliability and stability of privacy data processing, each node usually corresponds to one type of privacy data computing work. In order to ensure the computing efficiency of privacy data, before calculating the privacy data, the platform will calculate the size of the privacy data. When the amount of privacy data is large, the platform will split the privacy data and distribute it to the remaining idle privacy data computing nodes for privacy computing work. Although this task scheduling method can guarantee the processing efficiency of privacy data to a certain extent, the scheduling method is too simple and easily leads to the leakage of privacy data, which in turn leads to a decrease in the reliability and stability of the platform's privacy data processing. Summary of the invention
[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a task scheduling method based on distributed privacy computing.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] A task scheduling method based on distributed privacy computing includes the following steps:
[0006] Step 1: Accept the privacy computing task and upload the privacy data in the privacy computing task to the corresponding privacy computing node;
[0007] Step 2: When the privacy computing node performs privacy computing on the privacy data, it obtains the task scheduling value of the privacy data in real time, and determines whether the privacy computing task needs to be scheduled based on the comparison result between the task scheduling value and its threshold;
[0008] Step 3: When it is determined that privacy computing task scheduling is required, the final target node is obtained and the scheduling task is assigned to the final target node for privacy computing.
[0009] Furthermore, it accepts privacy computing tasks, obtains the privacy data type of each privacy data in the privacy computing task, uploads the privacy data of the same type to the privacy computing node responsible for the privacy data of that type, and the privacy computing node performs privacy computing on the privacy data of that type.
[0010] Furthermore, when the privacy computing node performs privacy computing on the private data, it obtains the task scheduling value of the private data in real time, specifically:
[0011] During the calculation process, based on the P period interval, a privacy calculation log is generated at each period node;
[0012] Obtain the privacy data complexity of the privacy computing log and set the privacy data complexity threshold;
[0013] When the privacy data complexity of the privacy computing log is less than or equal to the privacy data complexity threshold, maintain the P period interval and obtain the privacy computing log of the next period node;
[0014] When the privacy data complexity of the privacy computing log is greater than the privacy data complexity threshold, the privacy data complexity threshold is calculated by the ratio of the privacy data complexity to obtain the period adjustment index, which is marked as FC. Periodic interval, obtain the privacy computing log of the next period node;
[0015] Get all privacy computing logs after the calculation of this type of privacy data, sort all privacy computing logs in the order of node time, mark the privacy data complexity of the privacy computing log as PTi, i=1, 2, ..., n, i is the sequence number of the privacy computing log, calculate the time difference between the node times of two adjacent privacy computing logs after sorting, and get the computing node interval. Sum all computing node intervals and take the average to get the average computing node interval, and mark it as ZSW, using the formula:
[0016]
[0017] The task scheduling value PMB of this type of privacy data is obtained, where a1 is the complexity coefficient of the privacy data and a2 is the average computing node interval coefficient.
[0018] Furthermore, the privacy computing log includes the complexity of privacy data, node time,
[0019] Furthermore, based on the comparison result between the task scheduling value and its threshold, it is determined whether it is necessary to schedule the privacy computing task. Specifically, at each periodic node, the task scheduling value of this type of privacy data is obtained, and the task scheduling threshold is set. When the task scheduling value of this type of privacy data is greater than or equal to the task scheduling threshold, the remaining privacy data of this type that has not been privacy-calculated is marked as data to be scheduled, and the privacy computing node is marked as a node to be scheduled.
[0020] Furthermore, when it is determined that privacy computing task scheduling is required, the final target node is obtained, and the scheduling task is assigned to the final target node for privacy computing, specifically: the remaining privacy computing nodes associated with the node to be scheduled are marked as associated computing nodes, and the task scheduling value of the associated computing node is obtained. When the task scheduling value of the associated computing node is less than the task scheduling threshold, the associated computing node is marked as the initial target node;
[0021] Get the total number of stable tasks and scheduling evaluation of the node to be scheduled, and mark them as , A1 is the total number of stable tasks of the node to be scheduled, A2 is the scheduling evaluation of the node to be scheduled, obtain the total number of stable tasks and scheduling evaluation of the initial target node, and mark it as , B1 is the total number of stable tasks of the initial target node, B2 is the scheduling evaluation of the initial target node, and the scheduling correlation value between the node to be scheduled and the initial target node is calculated using the cosine similarity algorithm;
[0022] A scheduling association threshold is set. When the scheduling association value between the node to be scheduled and the initial target node is greater than or equal to the scheduling association threshold, the initial target node is marked as a secondary target node.
[0023] Obtain the relationship between the data type to be scheduled and other privacy data types, and mark the privacy data type associated with the data type to be scheduled as an associated data type through knowledge graph association analysis;
[0024] Obtain the type of privacy data that the secondary target node is responsible for. When the privacy data type that the secondary target node is responsible for is an associated data type, mark the secondary target node as the final target node, obtain the data volume of the data to be scheduled, and evenly distribute the data to be scheduled to each final target node according to the data volume for privacy data calculation. Synchronously generate the task acceptance log of the final target node, which includes the type of data to be scheduled and the task assignment time.
[0025] Furthermore, the total number of task stability times is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, and compare the scheduling data types of two adjacent task acceptance logs after sorting. When the two adjacent scheduling data types are the same, the task stability times are increased once. When the two adjacent scheduling data types are different, no corresponding processing is performed. The task stability times are summed to obtain the total number of task stability times.
[0026] Furthermore, the scheduling evaluation is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, calculate the time difference between the task assignment time of two adjacent task acceptance logs after sorting, and obtain the task acceptance interval, sum up all task acceptance intervals and take the average, and obtain the average task acceptance interval, and mark it as CDR, obtain the total number of task acceptance logs, and mark it as UBM, using the formula:
[0027]
[0028] The scheduling evaluation value FSH is obtained, where b1 is the average task acceptance interval coefficient and b2 is the task acceptance log quantity coefficient.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] Steps 1 and 2 of the method of the present invention can dynamically analyze whether privacy computing tasks need to be scheduled while the privacy computing nodes are performing privacy computing, thereby improving the accuracy of the privacy computing task scheduling judgment. Step 3 can perform a multi-dimensional analysis of the rationality and correlation of task scheduling between privacy computing nodes, and rationalize the allocation of privacy tasks and improve the efficiency of privacy computing on the basis of ensuring stable and reliable computing of privacy data. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flowchart of a task scheduling method based on distributed privacy computing;
[0032] Figure 2 A flowchart for determining whether privacy computing task scheduling is required;
[0033] Figure 3 A flowchart for performing privacy calculations to assign scheduling tasks to final target nodes. DETAILED DESCRIPTION
[0034] Reference Figure 1-Figure 3 , a task scheduling method based on distributed privacy computing, comprising the following steps:
[0035] Step 1: Accept the privacy computing task and upload the private data in the privacy computing task to the corresponding privacy computing node.
[0036] Step 2: When the privacy computing node performs privacy computing on private data, it obtains the task scheduling value of the private data in real time, and determines whether the privacy computing task needs to be scheduled based on the comparison result between the task scheduling value and its threshold.
[0037] Step 3: When it is determined that privacy computing task scheduling is required, the final target node is obtained and the scheduling task is assigned to the final target node for privacy computing.
[0038] Steps one and two of this method can dynamically analyze whether privacy computing tasks need to be scheduled while privacy computing is being performed on the privacy computing node, thereby improving the accuracy of the privacy computing task scheduling judgment. Step three can perform a multi-dimensional analysis of the rationality and correlation of task scheduling between privacy computing nodes, rationalize privacy task allocation, and improve privacy computing efficiency while ensuring stable and reliable computing of privacy data.
[0039] Accept the privacy computing task, obtain the privacy data type of each privacy data in the privacy computing task, upload the privacy data of the same type to the privacy computing node responsible for this type of privacy data, and the privacy computing node performs privacy computing on this type of privacy data. During the calculation process, based on the P cycle interval, a privacy computing log is generated at each cycle node. The privacy computing log includes the complexity of the privacy data (that is, the data complexity of the privacy computing node calculating this type of privacy data in the previous cycle) and the node time (that is, the Beijing time corresponding to the cycle node).
[0040] Different privacy data processing platforms will process different types of privacy data. For example, the energy privacy data processing platform will process electricity privacy data, water privacy data, gas privacy data, etc., which are different types of privacy data. For example, the electricity privacy data processing platform will process transformer data, generator data, etc., which are different types of privacy data.
[0041] In order to ensure the reliability and stability of the privacy data processing platform in processing privacy data, each privacy computing node in the privacy data processing platform is usually responsible for the same type of privacy data computing. For example, privacy computing node a is specifically responsible for the privacy computing of transformer data in the power privacy data processing platform, and privacy computing node b is specifically responsible for the privacy computing of generator data in the power privacy data processing platform.
[0042] Obtain the privacy data complexity of the privacy computing log and set the privacy data complexity threshold. The privacy data complexity threshold is a system preset threshold and can be modified according to actual needs.
[0043] When the privacy data complexity of the privacy computing log is less than or equal to the privacy data complexity threshold, maintain the P period interval and obtain the privacy computing log of the next period node.
[0044] When the privacy data complexity of the privacy computing log is greater than the privacy data complexity threshold, the privacy data complexity threshold is calculated by the ratio of the privacy data complexity to obtain the period adjustment index, which is marked as FC. Periodic interval, obtain the privacy computing log of the next period node.
[0045] At each periodic node, the task scheduling value of this type of privacy data is obtained, and the task scheduling threshold is set. The task scheduling threshold is a system preset threshold and can be modified according to actual needs. When the task scheduling value of this type of privacy data is greater than or equal to the task scheduling threshold, the remaining privacy data of this type that has not been privacy-calculated is marked as data to be scheduled, and the privacy computing node is marked as a node to be scheduled.
[0046] When the task scheduling value of this type of privacy data is less than the task scheduling threshold, no corresponding processing is performed.
[0047] The task scheduling value of this type of privacy data is obtained in the following way: obtain all privacy computing logs after the calculation of this type of privacy data, sort all privacy computing logs in the order of node time, mark the privacy data complexity of the privacy computing log as PTi, i=1, 2, ..., n, i is the sequence number of the privacy computing log, calculate the time difference between the node times of two adjacent privacy computing logs after sorting, and obtain the computing node interval, sum up all computing node intervals and take the average to obtain the average computing node interval, and mark it as ZSW, using the formula:
[0048]
[0049] The task scheduling value PMB of this type of privacy data is obtained, where a1 is the privacy data complexity coefficient, a2 is the average computing node interval coefficient, the value of a1 is 1.82, and the value of a2 is 0.97.
[0050] The remaining privacy computing nodes associated with the node to be scheduled are marked as associated computing nodes, and the task scheduling value of the associated computing node is obtained. When the task scheduling value of the associated computing node is less than the task scheduling threshold, the associated computing node is marked as the initial target node.
[0051] Get the total number of stable tasks and scheduling evaluation of the node to be scheduled, and mark them as , A1 is the total number of stable tasks of the node to be scheduled, A2 is the scheduling evaluation of the node to be scheduled, obtain the total number of stable tasks and scheduling evaluation of the initial target node, and mark it as , B1 is the total number of stable tasks of the initial target node, B2 is the scheduling evaluation value of the initial target node, and the cosine similarity algorithm is used to calculate the scheduling correlation value between the node to be scheduled and the initial target node.
[0052] For example: The total number of stable tasks of the node to be scheduled is 5, the scheduling evaluation of the node to be scheduled is 2.8, the total number of stable tasks of the initial target node is 6, and the scheduling evaluation of the initial target node is 3.1, then the scheduling correlation value between the node to be scheduled and the initial target node = .
[0053] Set the scheduling association threshold. The scheduling association threshold is a system preset threshold and can be modified according to actual needs.
[0054] When the scheduling association value between the node to be scheduled and the initial target node is greater than or equal to the scheduling association threshold, the initial target node is marked as a secondary target node.
[0055] When the scheduling correlation value between the node to be scheduled and the initial target node is less than the scheduling correlation threshold, no corresponding processing is performed.
[0056] The relationship between the data type to be scheduled and other privacy data types is obtained, and the privacy data type associated with the data type to be scheduled is marked as an associated data type through knowledge graph association analysis.
[0057] Each type of privacy data is represented in the form of a knowledge graph. For example, the relationships in the knowledge graph represent the associations between different types of privacy data, such as supply and demand relationships, affiliation relationships, etc., to ensure that the knowledge graph can accurately reflect the relationships between different types of privacy data.
[0058] For example, there is a supply and demand relationship between transformer data and generator data in electricity privacy data.
[0059] Obtain the type of privacy data that the secondary target node is responsible for. When the privacy data type that the secondary target node is responsible for is an associated data type, mark the secondary target node as the final target node, obtain the data volume of the data to be scheduled, evenly distribute the data to be scheduled to each final target node according to the data volume for privacy data calculation, and synchronously generate the task acceptance log of the final target node. The task acceptance log includes the type of data to be scheduled (that is, the type of data to be scheduled assigned to the final target node) and the task allocation time (that is, the Beijing time when the data to be scheduled is allocated to the final target node).
[0060] The total number of task stability times is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, and compare the scheduling data types of the two adjacent task acceptance logs after sorting. When the two adjacent scheduling data types are the same, the task stability times are increased once. When the two adjacent scheduling data types are different, no corresponding processing is performed. The task stability times are summed up to obtain the total number of task stability times.
[0061] The scheduling evaluation is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, calculate the time difference between the task assignment time of two adjacent task acceptance logs after sorting, and obtain the task acceptance interval. Sum and average all task acceptance intervals to obtain the average task acceptance interval, and mark it as CDR. Obtain the total number of task acceptance logs and mark it as UBM, using the formula:
[0062]
[0063] The scheduling evaluation value FSH is obtained, where b1 is the average task acceptance interval coefficient, b2 is the task acceptance log quantity coefficient, the value of b1 is 1.3, and the value of b2 is 1.5.
[0064] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0065] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0066] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0067] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0069] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0070] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.
[0071] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A task scheduling method based on distributed privacy computing, characterized in that: The steps include: Step 1: Accept the privacy computing task and upload the privacy data in the privacy computing task to the corresponding privacy computing node; Step 2: When the privacy computing node performs privacy computing on the privacy data, it obtains the task scheduling value of the privacy data in real time, and determines whether the privacy computing task needs to be scheduled based on the comparison result between the task scheduling value and its threshold; Step 3: When it is determined that privacy computing task scheduling is required, the final target node is obtained and the scheduling task is assigned to the final target node for privacy computing; Accept privacy computing tasks, obtain the privacy data type of each privacy data in the privacy computing tasks, upload the privacy data of the same type to the privacy computing node responsible for the privacy data of this type, and the privacy computing node performs privacy computing on the privacy data of this type; When the privacy computing node performs privacy computing on private data, it obtains the task scheduling value of the private data in real time, specifically: During the calculation process, based on the P period interval, a privacy calculation log is generated at each period node; Obtain the privacy data complexity of the privacy computing log and set the privacy data complexity threshold; When the privacy data complexity of the privacy computing log is less than or equal to the privacy data complexity threshold, maintain the P period interval and obtain the privacy computing log of the next period node; When the privacy data complexity of the privacy computing log is greater than the privacy data complexity threshold, the privacy data complexity threshold is calculated by the ratio of the privacy data complexity to obtain the period adjustment index, which is marked as FC. Periodic interval, obtain the privacy computing log of the next period node; Obtain all privacy computing logs after the calculation of this type of privacy data, sort all privacy computing logs in the order of node time, mark the privacy data complexity of the privacy computing log as PTi, i=1, 2, ..., n, i is the sequence number of the privacy computing log, calculate the time difference between the node times of two adjacent privacy computing logs after sorting, and get the computing node interval. Sum all computing node intervals and take the average to get the average computing node interval, which is marked as ZSW. Use the formula Get the task scheduling value PMB of this type of private data, where a1 is the complexity coefficient of the private data and a2 is the average computing node interval coefficient; Privacy computing logs include privacy data complexity and node time; According to the comparison result between the task scheduling value and its threshold, determine whether it is necessary to schedule the privacy computing task. Specifically, at each periodic node, obtain the task scheduling value of the type of privacy data, set the task scheduling threshold, and when the task scheduling value of the type of privacy data is greater than or equal to the task scheduling threshold, mark the remaining privacy data of this type that has not been privacy-calculated as data to be scheduled, and mark the privacy computing node as a node to be scheduled; When it is determined that privacy computing task scheduling is required, the final target node is obtained, and the scheduling task is assigned to the final target node for privacy computing. Specifically, the remaining privacy computing nodes associated with the node to be scheduled are marked as associated computing nodes, and the task scheduling value of the associated computing node is obtained. When the task scheduling value of the associated computing node is less than the task scheduling threshold, the associated computing node is marked as the initial target node; Get the total number of stable tasks and scheduling evaluation of the node to be scheduled, and mark them as , A1 is the total number of stable tasks of the node to be scheduled, A2 is the scheduling evaluation of the node to be scheduled, obtain the total number of stable tasks and scheduling evaluation of the initial target node, and mark it as , B1 is the total number of stable tasks of the initial target node, B2 is the scheduling evaluation of the initial target node, and the scheduling correlation value between the node to be scheduled and the initial target node is calculated using the cosine similarity algorithm; A scheduling association threshold is set. When the scheduling association value between the node to be scheduled and the initial target node is greater than or equal to the scheduling association threshold, the initial target node is marked as a secondary target node. Obtain the relationship between the data type to be scheduled and other privacy data types, and mark the privacy data type associated with the data type to be scheduled as an associated data type through knowledge graph association analysis; Obtain the type of privacy data that the secondary target node is responsible for. When the privacy data type that the secondary target node is responsible for is an associated data type, mark the secondary target node as the final target node, obtain the amount of data to be scheduled, evenly distribute the data to be scheduled to each final target node according to the data amount to calculate the privacy data, and synchronously generate the task acceptance log of the final target node, which includes the type of data to be scheduled and the task allocation time; The total number of task stability times is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, compare the scheduling data types of two adjacent task acceptance logs after sorting, and when the two adjacent scheduling data types are the same, increase the task stability times by one, and when the two adjacent scheduling data types are different, do not perform corresponding processing, sum the task stability times, and obtain the total number of task stability times; The scheduling evaluation is obtained in the following way: obtain all task acceptance logs of the privacy computing node before the current time of the system, sort all task acceptance logs in the order of task assignment time, calculate the time difference between the task assignment time of two adjacent task acceptance logs after sorting, and obtain the task acceptance interval, sum up all task acceptance intervals and take the average to obtain the average task acceptance interval, and mark it as CDR, obtain the total number of task acceptance logs, and mark it as UBM, and use the formula The scheduling evaluation value FSH is obtained, where b1 is the average task acceptance interval coefficient and b2 is the task acceptance log quantity coefficient.
