A large-scale data sharing method based on privacy computing

Abstract

The present invention discloses a large-scale data sharing method based on privacy computing, which belongs to the field of data processing technology. It is used to solve the technical problem that the overall effect of collaborative computing implemented by large-scale data sharing in the existing scheme is not good. The computing task saturation of different privacy computing nodes is monitored and analyzed to obtain the corresponding target demand computing nodes, and by performing data analysis on the collaborative whitelist associated with the target demand computing nodes, it is determined whether the target demand computing nodes in the collaborative whitelist that have historically collaborated with the target demand computing nodes meet the collaborative computing requirements of the current target demand computing nodes. The resource sharing data of all candidate privacy computing nodes that support collaborative computing are integrated and calculated to obtain a digital representation of the corresponding real-time resource sharing status. Further, data processing, analysis and control are performed on the collaborative reliability status between a number of previously screened privacy computing nodes and the target demand computing nodes.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a large-scale data sharing method based on privacy computing. Background Art

[0002] Privacy computing is a technology that processes and analyzes data while protecting data privacy and ensuring data security. It allows data to be calculated and analyzed without decrypting or directly exposing the original information, thereby maximizing the value of the data.

[0003] After searching, the prior art with application number 202310342479 and name "A data sharing platform and method based on privacy computing" was found, which disclosed that the platform includes a data application platform with multiple privacy computing nodes, and the data application platform is provided with a first target database and a computing resource library. The data sharing platform based on privacy computing of the invention can ensure that the data resources of the computing task will not be transferred to the user organization by setting up a data application platform with a high authority level and configuring the privacy computing nodes corresponding to the user organization, thereby ensuring the privacy and security of the data. In addition, by setting up the first target database on the data application platform, the transmission cost of the first target data between different organizations is reduced, and the privacy computing nodes with configurable computing resources also ensure that the computing requirements of the computing tasks can be met.

[0004] The existing large-scale data sharing technology solutions have certain defects in implementation. When the computing tasks of the privacy computing nodes are saturated, collaborative computing can be performed between different privacy computing nodes. However, it is not disclosed how to efficiently and reliably screen privacy computing nodes for collaborative computing, resulting in poor overall effect of collaborative computing in large-scale data sharing. Summary of the invention

[0005] The purpose of the present invention is to provide a large-scale data sharing method based on privacy computing, which is used to solve the technical problem that the overall effect of implementing collaborative computing in large-scale data sharing in existing solutions is poor.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A large-scale data sharing method based on privacy computing, comprising:

[0008] Monitor and analyze the computing task saturation of multiple privacy computing nodes set up in the data application platform, and mark the privacy computing nodes with saturated computing tasks as target demand computing nodes;

[0009] Obtain the collaborative whitelist associated with the target demand computing node and analyze whether the collaborative computing requirements are met; if so, control the target collaborative computing node in the collaborative whitelist to perform collaborative computing with the target demand computing node;

[0010] If it is not satisfied, the remaining privacy computing nodes in the data application platform are screened for collaborative computing effectiveness to obtain several screened privacy computing nodes. Data processing is performed on the collaborative reliability status between the several screened privacy computing nodes and the target demand computing nodes to obtain the corresponding resource sharing degree. The target collaborative computing node is determined according to the resource sharing degree and collaborative computing of data sharing with the target demand computing node is implemented.

[0011] Preferably, obtaining and analyzing the collaborative whitelist associated with the target demand computing node;

[0012] If the target collaborative computing node does not exist in the collaborative whitelist, a first collaborative screening instruction is generated;

[0013] If the target collaborative computing node exists in the collaborative whitelist, a collaborative verification instruction is generated, and the current collaborative feasibility of the target collaborative computing node is analyzed according to the collaborative verification instruction.

[0014] Preferably, if the collaborative computing between the target collaborative computing node and the target demand computing node is currently implementable, a collaborative implementation instruction is generated and the target collaborative computing node is controlled to perform collaborative computing with the target demand computing node;

[0015] If the collaborative computing between the target collaborative computing node and the target required computing node is not feasible at present, a second collaborative screening instruction is generated.

[0016] Preferably, when there are multiple target collaborative computing nodes in the collaborative whitelist, a collaborative evaluation instruction is generated, and the current collaborative feasibility of the target collaborative computing nodes is analyzed in turn according to the collaborative evaluation instruction, and the target collaborative computing nodes that can implement collaborative computing with the target demand computing node are marked as target collaborative computing nodes to be verified, and when performing target screening analysis on the target collaborative computing nodes to be verified, the collaborative support η corresponding to different target collaborative computing nodes to be verified is calculated in turn through the formula, and the target collaborative computing node to be verified that belongs to the first-ranked collaborative support is marked as the selected target collaborative computing node.

[0017] Preferably, the calculation formula of the collaborative support η is: Where ZLi is the available computing resources corresponding to the collaborative computing nodes of different targets to be verified, ZL0 is the standard available computing resources, Tyi is the delay time for the collaborative computing nodes of different targets to be verified to implement collaborative computing, and Ty0 is the standard delay time.

[0018] Preferably, perform collaborative computing effectiveness screening according to the first collaborative screening instruction or the second collaborative screening instruction, obtain all privacy computing nodes that support collaborative computing and mark them as alternative privacy computing nodes, obtain the numerical value of the available computing resources of the privacy computing nodes at present, and sort several alternative privacy computing nodes in descending order to obtain an alternative privacy computing node sorting set;

[0019] When performing collaborative computing effectiveness screening, through the formula calculate the resource sharing degree α corresponding to different alternative privacy computing nodes; in the formula, ZL′ is the real-time available computing resources corresponding to the alternative privacy computing node, ZX is the required computing resources required by the target demand computing node, and η is the available computing resource limit rate corresponding to the alternative privacy computing node.

[0020] Preferably, if the calculated resource sharing degree is greater than 0, generate a collaborative effective instruction and mark the corresponding alternative privacy computing node as a screened privacy computing node;

[0021] If the calculated resource sharing degree is less than or equal to 0, generate a collaborative invalid instruction;

[0022] Sort all the screened privacy computing nodes in descending order according to the numerical value of the resource sharing degree to obtain a screened privacy computing node sorting set.

[0023] Preferably, when processing the collaborative reliable state between several screened privacy computing nodes sorted in the screened privacy computing node sorting set and the target demand computing node, through the formula calculate the processing support degree β corresponding to the processing of the required computing resources by different screened privacy computing nodes; in the formula, a1 and a2 are both proportionality coefficients, and the value range is 0 < a1 ≤ 1, 0 < a2 ≤ 1; BC is the unit standard data processing volume when the screened privacy computing node performs data processing, and Ty′ is the standard delay duration when the screened privacy computing node performs collaborative computing.

[0024] Preferably, sort all the screened privacy computing nodes in descending order according to the processing support degree, mark the screened privacy computing node ranked first as the target collaborative computing node, supplement the target collaborative computing node to the collaborative whitelist associated with the target demand computing node, and at the same time control the collaborative computing for data sharing between the target required computing node and the target collaborative computing node.

[0025] Compared with the existing solutions, the beneficial effects achieved by the present invention:

[0026] The present invention obtains the corresponding target demand computing nodes by monitoring and analyzing the computing task saturation of different privacy computing nodes, and can provide reliable target data support for subsequent targeted data sharing collaborative computing analysis of target demand computing nodes; by performing data analysis on the collaborative whitelist associated with the target demand computing node, it is determined whether the target demand computing nodes in the collaborative whitelist that have historically collaborated with the target demand computing nodes meet the collaborative computing requirements of the current target demand computing nodes, thereby realizing the expanded utilization of historical collaborative computing data and effectively improving the overall effect of the current collaborative computing object screening and processing.

[0027] The present invention obtains the corresponding resource sharing degree by integrating the resource sharing data of all alternative privacy computing nodes that support collaborative computing. According to the resource sharing degree, the real-time resource sharing status of the candidate privacy computing nodes can be digitally represented, and reliable digital data support can be provided for the feasibility analysis of collaborative computing between the alternative privacy computing nodes and the target demand computing nodes, thereby improving the overall effect of processing and analyzing the effectiveness of collaborative computing by different privacy computing nodes.

[0028] The present invention further performs data processing on the collaborative reliability status between several previously screened privacy computing nodes and the target demand computing nodes to obtain corresponding processing support. According to the processing support, the collaborative reliability status between different screened privacy computing nodes and the target demand computing nodes can be digitally processed, and reliable data support can be provided for the subsequent screening of target collaborative computing nodes and the data update of the collaborative whitelist associated with the target demand computing nodes, as well as the collaborative computing control of data sharing with the target collaborative computing nodes, thereby improving the overall effect of data sharing in the whole process of collaborative computing implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention will be further described below in conjunction with the accompanying drawings.

[0030] Figure 1 This is a flowchart of a large-scale data sharing method based on privacy computing in the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary operation and maintenance personnel in this field without creative work are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, the present invention is a large-scale data sharing method based on privacy computing, comprising:

[0033] Monitor and analyze the computing task saturation of multiple privacy computing nodes set up in the data application platform, and mark the privacy computing nodes with saturated computing tasks as target demand computing nodes;

[0034] In the embodiment of the present invention, the data application platform is a supervised intermediate platform, and general user organizations cannot actively obtain data in the data application platform; the data application platform uses privacy computing technology to achieve data content non-circulation but data value sharing;

[0035] In addition, multiple privacy computing nodes are set up in the data application platform. Each privacy computing node communicates with different target organizations through its own computing interface, and different target organizations have their own independent databases for storing their own data.

[0036] In the embodiment of the present invention, by monitoring and analyzing the computing task saturation of different privacy computing nodes to obtain the corresponding target demand computing nodes, reliable target data support can be provided for subsequent target demand computing nodes to perform targeted data sharing collaborative computing analysis;

[0037] Obtain the collaborative whitelist associated with the target demand computing node and analyze whether the collaborative computing requirements are met; if so, control the target collaborative computing node in the collaborative whitelist to perform collaborative computing with the target demand computing node;

[0038] If not, the remaining privacy computing nodes in the data application platform are screened for collaborative computing effectiveness to obtain a number of screened privacy computing nodes. Data processing is performed on the collaborative reliability status between the screened privacy computing nodes and the target demand computing nodes to obtain the corresponding resource sharing degree. The target collaborative computing node is determined according to the resource sharing degree and the collaborative computing of data sharing with the target demand computing node is implemented. The specific steps include:

[0039] Obtain and analyze the collaborative whitelist associated with the target demand computing node;

[0040] If the target collaborative computing node does not exist in the collaborative whitelist, a first collaborative screening instruction is generated;

[0041] If the target collaborative computing node exists in the collaborative whitelist, a collaborative verification instruction is generated, and the current collaborative feasibility of the target collaborative computing node is analyzed according to the collaborative verification instruction. If the collaborative computing between the target collaborative computing node and the target demand computing node is feasible, a collaborative implementation instruction is generated and the target collaborative computing node is controlled to perform collaborative computing with the target demand computing node;

[0042] The determination of whether the collaborative computing between the target collaborative computing node and the target demand computing node is feasible may be: the currently available computing resources of the target collaborative computing node are greater than the demand computing resources required by the target demand computing node, and the computing resources are specifically CPU utilization resources and / or memory resources, which may be selected according to the actual application scenario and application requirements;

[0043] If the collaborative computing between the target collaborative computing node and the target required computing node is not feasible, that is, the currently available computing resources of the target collaborative computing node are not greater than the required computing resources required by the target required computing node, then a second collaborative screening instruction is generated;

[0044] Furthermore, when there are multiple target collaborative computing nodes in the collaborative whitelist, a collaborative evaluation instruction is generated, and the current collaborative feasibility of the target collaborative computing nodes is analyzed in turn according to the collaborative evaluation instruction, and the target collaborative computing nodes that can implement collaborative computing with the target demand computing node are marked as target collaborative computing nodes to be verified, and when the target collaborative computing nodes to be verified are subjected to target screening analysis, the collaborative support η corresponding to different target collaborative computing nodes to be verified is calculated in turn by the formula, and the target collaborative computing node to be verified that belongs to the top collaborative support is marked as the selected target collaborative computing node;

[0045] Among them, the calculation formula of collaborative support η is: Where ZLi is the available computing resources corresponding to the collaborative computing nodes of different targets to be verified, ZL0 is the standard available computing resources, Tyi is the delay time for the collaborative computing of different collaborative computing nodes to be verified, and Ty0 is the standard delay time; the standard available computing resources and standard delay time are determined according to the median of all available computing resources and the median of all delay times corresponding to all collaborative computing nodes of the targets to be verified;

[0046] It should be noted that the collaborative support is a numerical value used to digitally represent the collaborative computing status of the target collaborative computing node to be verified by integrating and calculating the available computing resource data of different aspects; the greater the collaborative support, the better the collaborative computing status;

[0047] In the embodiment of the present invention, by performing data analysis on the collaborative whitelist associated with the target demand computing node, it is determined whether the target demand computing node in the collaborative whitelist that has historically collaborated with the target demand computing node meets the collaborative computing requirements of the current target demand computing node, thereby realizing the expanded utilization of historical collaborative computing data and effectively improving the overall effect of the current collaborative computing object screening and processing;

[0048] It is worth noting that when the target collaborative computing node does not exist in the collaborative whitelist, or the collaborative computing between the target collaborative computing node and all the target required computing nodes is not feasible, it is necessary to re-screen and match the new target collaborative computing node;

[0049] Perform collaborative computing effectiveness screening according to the first collaborative screening instruction or the second collaborative screening instruction, obtain all privacy computing nodes that support collaborative computing and mark them as candidate privacy computing nodes. This does not include all target collaborative computing nodes in the collaborative whitelist associated with the target demand computing node. Whether to support collaborative computing is determined by the management agency of the privacy computing node. Obtain the value of the currently available computing resources of the privacy computing node to sort several candidate privacy computing nodes in descending order to obtain a sorted set of candidate privacy computing nodes;

[0050] When screening for collaborative computing effectiveness, the formula Calculate and obtain the resource sharing degree α corresponding to different candidate privacy computing nodes; where ZL′ is the real-time available computing resources corresponding to the candidate privacy computing node, ZX is the required computing resources required by the target demand computing node, and η is the available computing resource limit rate corresponding to the candidate privacy computing node. The available computing resource limit rate is used to limit the real-time available computing resources of collaborative computing to avoid the candidate privacy computing node being unable to meet its own data processing requests when implementing collaborative computing; the available computing resource limit rate corresponding to the candidate privacy computing node is determined according to the actual operation requirement parameters of the candidate privacy computing node;

[0051] If the resource sharing degree obtained by calculation is greater than 0, a collaborative effective instruction is generated and the corresponding candidate privacy computing node is marked as a screening privacy computing node;

[0052] If the resource sharing degree obtained by calculation is less than or equal to 0, a coordination invalidation instruction is generated;

[0053] Arrange all the filtered privacy computing nodes in descending order according to the value of the resource sharing degree to obtain a sorted set of filtered privacy computing nodes;

[0054] In the embodiment of the present invention, the resource sharing data of all candidate privacy computing nodes supporting collaborative computing are integrated and calculated to obtain the corresponding resource sharing degree. According to the resource sharing degree, the real-time resource sharing status of the candidate privacy computing nodes can be digitally represented, and reliable digital data support can be provided for the feasibility analysis of collaborative computing between the candidate privacy computing nodes and the target demand computing nodes, thereby improving the overall effect of processing and analyzing the effectiveness of collaborative computing by different privacy computing nodes.

[0055] When processing data on the collaborative reliable state between several screened privacy computing nodes sorted in the sorted set of screened privacy computing nodes and target demand computing nodes, through the formula Calculate the processing support degree β corresponding to the processing demand computing resources of different screened privacy computing nodes; in the formula, a1 and a2 are both proportionality coefficients, and the value range is 0 < a1 ≤ 1, 0 < a2 ≤ 1; BC is the unit standard data processing volume when the screened privacy computing node processes data, which is determined according to the operating design parameters of the screened privacy computing node or based on the historical big data processed, and Ty′ is the standard delay duration for the screened privacy computing node to perform collaborative computing, and the standard delay duration is determined according to the median value of all delay durations when the screened privacy computing node historically performs collaborative computing;

[0056] Arrange all the screened privacy computing nodes in descending order according to the processing support degree, mark the screened privacy computing node ranked first as the target collaborative computing node, supplement the target collaborative computing node to the collaborative white list associated with the target demand computing node, and at the same time control the collaborative computing for data sharing between the target demand computing node and the target collaborative computing node;

[0057] In the embodiments of the present invention, further process the data on the collaborative reliable state between several previously screened privacy computing nodes and target demand computing nodes to obtain the corresponding processing support degree. According to the processing support degree, not only can the collaborative reliable state between different screened privacy computing nodes and target demand computing nodes be digitally processed, but also reliable data support can be provided for the subsequent screening of target collaborative computing nodes, the data update of the collaborative white list associated with the target demand computing node, and the control of collaborative computing for data sharing with the target collaborative computing node, improving the overall effect of data sharing in the whole process of collaborative computing implementation.

[0058] In addition, the formulas involved above are all calculated by removing the dimension and taking their numerical values, and are obtained by a software through simulating a large amount of data to get a formula closest to the real situation.

[0059] In several embodiments provided by the present invention, it should be understood that the disclosed method can be implemented in other ways. For example, the above-described embodiments of the invention are merely illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0060] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules. They may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0062] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the basic features of the present invention.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, ordinary operation and maintenance personnel in the field should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A large-scale data sharing method based on privacy computing, characterized in that: include: Monitor and analyze the computing task saturation of multiple privacy computing nodes set up in the data application platform, and mark the privacy computing nodes with saturated computing tasks as target demand computing nodes; Obtain the collaborative whitelist associated with the target demand computing node and analyze whether the collaborative computing requirements are met; if so, control the target collaborative computing node in the collaborative whitelist to perform collaborative computing with the target demand computing node; If not, the remaining privacy computing nodes in the data application platform are screened for collaborative computing effectiveness to obtain a number of screened privacy computing nodes. Data processing is performed on the collaborative reliability status between the screened privacy computing nodes and the target demand computing nodes to obtain the corresponding processing support. The target collaborative computing node is determined based on the processing support and collaborative computing of data sharing with the target demand computing node is implemented. When screening for the effectiveness of collaborative computing, all privacy computing nodes that support collaborative computing are obtained and marked as candidate privacy computing nodes. The values ​​of the currently available computing resources of the privacy computing nodes are obtained to sort the candidate privacy computing nodes in descending order, and a sorted set of candidate privacy computing nodes is obtained. When screening for collaborative computing effectiveness, the formula Calculate and obtain the resource sharing degree α corresponding to different candidate privacy computing nodes; where ZL´ is the real-time available computing resources corresponding to the candidate privacy computing node, ZX is the required computing resources required by the target demand computing node, and η is the available computing resource limit rate corresponding to the candidate privacy computing node; When processing data on the collaborative reliable state between several screened privacy computing nodes sorted in the sorted set of screened privacy computing nodes and the target demand computing nodes, through the formula Calculate the processing support degree β corresponding to the processing demand computing resources of different screened privacy computing nodes; where a1 and a2 are both proportionality coefficients, and the value range is 0 < a1 ≤ 1, 0 < a2 ≤ 1; BC is the unit standard data processing volume when the screened privacy computing node performs data processing, and Ty´ is the standard delay duration for the screened privacy computing node to perform collaborative computing.

2. According to the large-scale data sharing method based on privacy computing according to claim 1, it is characterized in that: Obtain and analyze the collaborative whitelist associated with the target demand computing node; If the target collaborative computing node does not exist in the collaborative whitelist, a first collaborative screening instruction is generated; If the target collaborative computing node exists in the collaborative whitelist, a collaborative verification instruction is generated, and the current collaborative feasibility of the target collaborative computing node is analyzed according to the collaborative verification instruction.

3. According to claim 2, a large-scale data sharing method based on privacy computing is characterized in that: If the collaborative computing between the target collaborative computing node and the target demand computing node is currently implementable, a collaborative implementation instruction is generated and the target collaborative computing node is controlled to perform collaborative computing with the target demand computing node; If the collaborative computing between the target collaborative computing node and the target required computing node is not feasible at present, a second collaborative screening instruction is generated.

4. The method for large-scale data sharing based on privacy computing according to claim 3 is characterized in that: When there are multiple target collaborative computing nodes in the collaborative whitelist, a collaborative evaluation instruction is generated. According to the collaborative evaluation instruction, the current collaborative feasibility of the target collaborative computing nodes is analyzed in turn. The target collaborative computing nodes that can implement collaborative computing with the target demand computing node are marked as target collaborative computing nodes to be verified. When the target collaborative computing nodes to be verified are subjected to target screening analysis, the collaborative support η corresponding to different target collaborative computing nodes to be verified is calculated in turn by the formula. The calculation formula of the collaborative support η is: ; In the formula, ZLi is the available computing resources corresponding to the collaborative computing nodes of different targets to be verified, ZL0 is the standard available computing resources, Tyi is the delay time for the collaborative computing nodes of different targets to be verified to implement collaborative computing, and Ty0 is the standard delay time; The target collaborative computing node to be verified, to which the collaborative support degree ranks first, is marked as the selected target collaborative computing node.

5. The method for large-scale data sharing based on privacy computing according to claim 1, characterized in that: If the resource sharing degree obtained by calculation is greater than 0, a collaborative effective instruction is generated and the corresponding candidate privacy computing node is marked as a screening privacy computing node; If the resource sharing degree obtained by calculation is less than or equal to 0, a coordination invalidation instruction is generated; All the filtered privacy computing nodes are arranged in descending order according to the value of the resource sharing degree to obtain a sorted set of filtered privacy computing nodes.

6. The method for large-scale data sharing based on privacy computing according to claim 1, characterized in that: All the filtered privacy computing nodes are arranged in descending order according to the processing support, and the first-ranked filtered privacy computing node is marked as the target collaborative computing node. The target collaborative computing node is added to the collaborative whitelist associated with the target demand computing node. At the same time, the collaborative computing of data sharing between the target demand computing node and the target collaborative computing node is controlled.

Images