Data collection module and data collection method in privacy computing

Abstract

The embodiments of this specification provide a data collection module and a data collection method in privacy computing, and provide a data collection framework. A single computing node can configure a data collection module locally. The data collection module includes multiple collection units, and each collection unit is configured to collect network data, device data, application data, etc. Each collection unit obtains corresponding data by configuring a local data collection port, and then transmits the collected data to other computing nodes through a predetermined data port exposed to the outside. In this way, an effective and feasible technical solution is provided for data collection, aggregation, error analysis, etc. under the privacy computing framework.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of data security technology, and in particular, to a data collection module and a data collection method in privacy computing. Background Art

[0002] The “Scope and Development Trend of Privacy Computing Research” published in 2016 formally proposed the term “privacy computing” and defined privacy computing as: “Computational theories and methods for the protection of privacy information throughout its life cycle, which are computable models and axiomatic systems for privacy measurement, privacy leakage costs, privacy protection, and the complexity of privacy analysis when the ownership, management, and use rights of privacy information are separated”. Privacy computing is essentially about solving data service problems such as data circulation and data application for each data holder under the premise of protecting data privacy. Each data holder can be used as a computing node in privacy computing. Privacy computing is cross-domain computing. The data of the computing itself is invisible and has cross-domain operation and maintenance difficulties. In order to maintain system stability, or when encountering problems such as task failures, it is necessary to find the root cause of the problem, which increases the difficulty of troubleshooting. Therefore, how to monitor the relevant data of each computing node in real time is an important technical issue in privacy computing. Summary of the invention

[0003] One or more embodiments of this specification describe a data collection module and a data collection method in privacy computing to solve one or more problems mentioned in the background technology.

[0004] According to a first aspect, a data collection module in privacy computing is provided, which is used for a single computing node to collect local data under a privacy computing architecture performed by multiple parties; the data collection module includes multiple collection units, each of which is configured to collect network data, device data, and application data;

[0005] The first collection unit configured to collect application data is used to collect data of a predetermined application through a pre-set data port, and identify the port configured when a process is pulled up during business processing, thereby collecting application data in the business processing process in real time;

[0006] The data acquisition module also includes a predetermined data port exposed to the outside, which is used to transmit the collected data to other computing nodes when the data interaction conditions are met.

[0007] In one embodiment, the first acquisition unit synchronizes with the current service framework manager to obtain a data port of a process started by the manager.

[0008] In one embodiment, the data collection module further includes a first filtering unit configured to filter the collected data based on a preset first filtering rule so that the data collection module can summarize the filtered data.

[0009] In one embodiment, the data acquisition module also includes a second filtering unit configured to: receive a user's selection of the collected data according to the data item category, thereby forming a second filtering rule, so as to perform data filtering based on the second filtering rule when the collected data is transmitted to other computing nodes through the predetermined data port.

[0010] In one embodiment, the data interaction condition includes one of the following: new data is collected, a predetermined time interval arrives, or a data synchronization request from another computing node is received.

[0011] In one embodiment, the multiple collection units further include an external collection unit configured to obtain monitoring data collected by other computing nodes.

[0012] According to the second aspect, a data collection method in privacy computing is provided, which is used for a single computing node to collect local data under a privacy computing architecture performed by multiple parties; the method comprises: collecting network data, device data, and application data of the single computing node respectively through multiple collection units, wherein the first collection unit configured to collect application data among the multiple collection units is used to collect data of a predetermined application through a pre-set data port, and identify the port configured when the process is pulled up during business processing, thereby performing real-time collection of application data in the business processing process; when the data interaction conditions are met, the collected data is transmitted to other computing nodes through the predetermined data port.

[0013] In one embodiment, the identifying of the port configured when the process is started during the service processing includes: synchronizing with the current service framework manager to obtain the data port of the process started by the manager.

[0014] In one embodiment, the method further comprises: filtering the collected data based on a preset first filtering rule so that the data collection module can summarize the filtered data.

[0015] In one embodiment, transmitting the collected data to other computing nodes through a predetermined data port includes: after filtering the data based on the second filtering rule, transmitting the filtered data to other computing nodes through the predetermined data port, wherein the second filtering rule is formed by receiving a user's selection of the collected data according to the data item category.

[0016] In one embodiment, the data interaction condition includes one of the following: new data is collected, a predetermined time interval arrives, or a data synchronization request from another computing node is received.

[0017] In one embodiment, the multiple collection units further include an external collection unit, and the method further includes: requesting monitoring data collected by other computing nodes through the external collection unit.

[0018] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method of the second aspect.

[0019] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method of the second aspect is implemented.

[0020] Through the data collection module and method provided in the embodiments of this specification, under the privacy computing architecture, a single computing node can locally configure a data collection module, which includes multiple collection units, each of which is configured to collect network data, device data, application data, etc. Each collection unit obtains the corresponding data by configuring the local data collection port, and then transmits the collected data to other computing nodes through the predetermined data port exposed to the outside. In this way, a single computing node locally uniformly summarizes all machine indicators and engine indicators, shields the changes in the internal system from the external monitoring system, and protects the privacy of the data source (or running application), thereby providing an effective and feasible technical solution for data collection, aggregation, error analysis, etc. under the privacy computing framework. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 A schematic diagram of a privacy computing architecture is shown, wherein (a) shows a schematic diagram of a privacy computing architecture composed of a service provider and a terminal device, and (b) shows a schematic diagram of a privacy computing architecture of a multi-party secure computing;

[0023] Figure 2 A schematic diagram of the data acquisition architecture of a single computing node under the technical concept of this specification is shown;

[0024] Figure 3A schematic diagram of a collection architecture for a specific example of a single computing node collecting local data through an internal data port is shown;

[0025] Figure 4 A schematic diagram of a data interaction architecture between a current computing node and other computing nodes is shown in a specific example;

[0026] Figure 5 A flow chart of a data collection method in privacy computing according to one embodiment is shown. DETAILED DESCRIPTION

[0027] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0028] Protecting private data is one of the core purposes of privacy computing. Tasks performed through privacy computing can be called privacy computing tasks, that is, multiple participants jointly complete tasks such as data calculation and analysis while ensuring the security and privacy of the original data. Privacy computing tasks can be privacy-preserving data analysis, joint machine learning, etc.

[0029] Figure 1 A schematic diagram of the privacy computing architecture is shown.

[0030] One application of privacy computing is to protect the privacy information throughout its life cycle. For example, it can be applied to describe, measure, evaluate and integrate the privacy information involved when processing video, audio, images, graphics, text, numbers, ubiquitous network behavior information flow and other information, forming a set of symbolic, formalized privacy computing theories, algorithms and application technologies with quantitative evaluation standards, supporting the protection of privacy information in multiple systems. The application architecture of this scenario is as follows: Figure 1 As shown in (a) in FIG. 1 , a server and multiple terminal devices may be included. The terminal device may process the sampled private information through the running application and upload it to the server. A specific example is that the private information such as images, videos, and audio collected by the terminal device is processed into a vector or feature map and sent to the recognition server.

[0031] Another common application of privacy computing is Secure Multi-Party Computation (SMPC). SMPC solves the problem of privacy-preserving collaborative computing between a group of mutually distrusting parties, ensuring the independence of inputs and the correctness of calculations while not leaking input values ​​to other members participating in the calculation. Figure 1 (b) in FIG. 1 shows a schematic diagram of an implementation architecture of a multi-party secure computation. It can be understood that multi-party secure computation can be divided into a centralized architecture and a decentralized architecture. Figure 1(b) in the figure shows a decentralized architecture, where each participant (such as member 1, member 2, and member 3) can exchange encrypted data and complete related privacy computing tasks in a secret state through encrypted computing methods such as homomorphic encryption and secret sharing, such as federated learning tasks, joint online prediction or classification tasks, etc.

[0032] In the process of completing the privacy computing task, each participant is regarded as a computing node. In the multi-party secure computing scenario, you may encounter the situation where the task (such as the model training task) fails. In order to troubleshoot the cause of the failure, it is often necessary to combine the internal state of the machine and algorithm to determine the cause of the problem. In some online operation scenarios (such as joint prediction scenarios), it is necessary to pay attention to the stability of the overall system. In order to discover problems in advance, so that they can be discovered in time when they occur or are about to occur, it is necessary to collect machine and algorithm monitoring indicators in real time. Privacy computing is usually a joint calculation across institutions, and due to security reasons, the dynamic nature of task operation, and the different monitoring indicator collection ports of different algorithms, it is difficult to directly collect indicators from other institutions. Therefore, it is difficult to combine the monitoring indicators of multiple institutions to troubleshoot and warn problems.

[0033] In view of this, this specification provides a data collection solution under the privacy computing architecture, which can aggregate the current computing nodes (such as Figure 1 The data generated by the user terminals, member 1, member 2, member 3, etc. in the business processing process is collected, and the data port of the data acquisition module is exposed to other computing nodes, providing effective guarantee for data investigation and real-time detection based on privacy protection in privacy computing.

[0034] Figure 2 FIG. 1 shows a data collection architecture in privacy computing according to an embodiment of the present specification. Figure 2 As shown, under this data collection architecture, various network data, device data, application data, etc. can be collected through multiple collection units through a computing platform (which is itself or is provided with a data collection module). The computing platform can expose a data port (such as an https port) so that other devices or data parties can access the data port and obtain data from the computing platform. Among them, network data can include at least one of the network bandwidth, network delay, throughput, response time, retransmission rate, etc. of the current data party device; device data can include one or more of CPU usage, memory size, memory occupancy, disk IO, system average load, etc.; application data can also be called algorithm data, for example, it can include algorithm functions (such as addition, multiplication, etc.), processing object types (such as integers, floating point numbers, etc.), etc.

[0035] It can be understood that network data and device data are data collected from physical facilities such as devices and networks. They can be collected by monitoring the relevant collection units of the equipment or accessing the corresponding ports. Application data is data generated during the operation of the business processes of related applications.

[0036] The collection unit for collecting application data is, for example, recorded as the first collection unit, and the first collection unit is used to collect relevant data of each process launched during the business processing. It can be understood that a single process usually corresponds to a single data port. The first collection unit can automatically identify the corresponding port to obtain information such as the algorithm function, algorithm progress, process errors, etc. used in the business processing process. The first collection unit can identify the corresponding port, for example, by detecting the process. In a specific example, the process is launched by a manager Agent under a privacy computing framework (such as Kuscia, an open source privacy computing framework corresponding to the cryptanalysis), and the Agent can synchronize the port corresponding to the launched process to the first collection unit.

[0037] like Figure 3 As shown, the computing platform can collect the internal data of the current data party through various data ports, such as Secretflow (a unified framework for privacy-preserving data analysis and machine learning), data analysis engine SCQL, privacy intersection algorithm PSI or other applications (other APPs), etc. The computing platform can connect to each data port through each sub-collection unit of the first collection unit to collect corresponding data. Figure 3 The data ports shown in are all https ports. The operations of collecting data by the sub-collection units of the first collection unit through the corresponding data ports can be executed in parallel, and can be dynamically increased or decreased as the number of processes increases or decreases.

[0038] The network data collection can collect data related to the network connection of the current computing node based on the second collection unit. For example, the first subunit of the second collection unit collects data at the protocol level such as TP (multi-packet protocol) and TCP (Transmission Control Protocol / Internet Protocol), and the second subunit of the second collection unit collects L3 / L4 layer network data.

[0039] As a specific example, the first subunit may be a module formed by repackaging the open source SS (Socket Statistics) command. SS is also known as IPC (Inter-Process Communication) socket statistics, which is a powerful network management command mainly used to obtain statistical information of sockets in the system, so as to assist in diagnosing and troubleshooting network problems, including checking the current network connection and port status, searching for network problems, counting network conditions, filtering network data, etc.

[0040] As another specific example, the second subunit can be implemented by a distributed proxy Envoy. Envoy is used for inter-service communication in a service grid, supports multiple L3 / L4 filters, and can implement functions such as load balancing and health checks. The network data collected by the second collection unit can include, but is not limited to: QPS (Queries Per Second, search traffic received within 1 second), latency, traffic, timeout times, etc.

[0041] The device data may be collected, for example, by a third collection unit. The third collection unit may be recorded as NodeExporter, for example, for collecting data related to the current data party device, such as CPU usage, memory size, memory occupancy, disk IO, and system average load.

[0042] The data collected by the computing platform is aggregated and preliminarily screened according to the predetermined first screening rules. For example, irrelevant information and highly sensitive information are screened out. For example, the predetermined rules include screening out sensitive configuration information such as the process resource control group (selinux / cgroup) of the device, etc.

[0043] In one embodiment, the external monitoring system Prometheus can periodically pull the data collected by the computing platform through a public HTTP(S) data port exposed by the computing platform. Conversely, the multiple collection units in the computing platform of the current computing node can also include an external collection unit configured to obtain its monitoring data from other computing nodes.

[0044] According to a possible design, the data collected by the computing platform can be manually configured to form a second filtering rule. For example, the user can select the type of data that he wants to protect (privacy data, not to be transmitted to others) or the type of data that can be provided to an external device, thereby forming a corresponding second filtering rule.

[0045] The data obtained by filtering the second filtering rule can be provided to external data through the port exposed by the computing platform (such as the metrics port based on the http protocol). Figure 4In the example, after collecting its own indicators, institution B participating in privacy computing uses the filtering rules determined by user configuration to filter out the data that the user wants to protect, and pushes some insensitive monitoring items to the computing platform for data collection in institution A, such as MetricsExporter, so that institution A can collect the indicators of institution B. Since institution B only exposes a data port (such as http: / / 127.0.0.1:port / metrics) in the data transmission through the local computing platform, its internal applications and data ports do not need to be connected to the computing platform of institution A, thereby shielding the changes of the internal system from the external monitoring system, and the setting is simple. When collecting and pushing insensitive monitoring indicators to other cooperative institutions through the computing platform, local data privacy is effectively protected.

[0046] It can be understood that the current computing node can provide the collected data to other computing nodes if the data interaction conditions are met. According to business needs, in the case of real-time monitoring data, the data interaction condition can be the collection of new data, in the case of outputting monitoring data at a predetermined time interval, the data interaction condition can be the arrival of the predetermined time interval, in the case of data acquisition requests initiated by other computing nodes, the data interaction condition can be the reception of data synchronization requests from other computing nodes, and in other cases, the data interaction condition can also be other conditions, which will not be repeated here.

[0047] Based on the above technical concept, the embodiment of this specification also provides a data collection process in privacy computing. Figure 5 The data collection process in privacy computing according to an embodiment is shown, which is applicable to collecting computing nodes in the privacy computing process. The execution subject of the process can be a single computing node under the privacy computing framework, which can be implemented as any computer, device, server, etc. with certain computing capabilities. More specifically, Figure 2 The computing platform shown in FIG. 1 is used to collect local data so as to facilitate joint monitoring of the collected data during the joint computing process.

[0048] like Figure 5 As shown, the data collection process in the privacy computing may include the following steps: Step 501, collecting network data, device data, and application data of a single computing node through multiple collection units, wherein the first collection unit configured to collect application data among the multiple collection units is used to collect data of a predetermined application through a pre-set data port, and identify the configured port when the process is pulled up during business processing, thereby performing real-time collection of application data in the business processing process; Step 502, when the data interaction conditions are met, transmitting the collected data to other computing nodes through the predetermined data port.

[0049] First, in step 501, network data, device data, and application data of a single computing node are collected respectively by multiple collection units.

[0050] It can be understood that under the technical concept of this specification, local data (or local data, which may include one or more devices corresponding to the current computing node), such as network data, device data, application data, etc., can be collected via a data collection module.

[0051] Network data and device data are data collected from physical facilities such as devices and networks. They can be collected by monitoring the relevant collection units of the equipment or accessing the corresponding ports. Application data is data generated during the operation of the business processes of related applications.

[0052] The collection unit for collecting application data is, for example, recorded as the first collection unit, and the first collection unit is used to collect relevant data of each process launched during the business processing. It can be understood that a single process usually corresponds to a single data port. The first collection unit can automatically identify the corresponding port to obtain information such as the algorithm function, algorithm progress, process errors, etc. used in the business processing process. The first collection unit can identify the corresponding port, for example, by detecting the process. In a specific example, the process is launched by a manager Agent under a privacy computing framework (such as Kuscia, an open source privacy computing framework corresponding to the cryptanalysis), and the Agent can synchronize the port corresponding to the launched process to the first collection unit.

[0053] The network data collection can collect data related to the network connection of the current computing node based on the second collection unit. For example, the first subunit of the second collection unit collects data at the protocol level such as TP (multi-packet protocol) and TCP (Transmission Control Protocol / Internet Protocol), and the second subunit of the second collection unit collects L3 / L4 layer network data.

[0054] The device data may be collected, for example, by a third collection unit. The third collection unit may be recorded as NodeExporter, for example, for collecting data related to the current data party device, such as CPU usage, memory size, memory occupancy, disk IO, and system average load.

[0055] In an optional implementation, during the data collection process, the data may be summarized based on a preset first filtering rule, so that a preliminary screening may be performed according to the first screening rule, such as filtering out irrelevant information, highly sensitive information, etc. For example, the preset rule includes filtering out sensitive configuration information such as selinux / cgroup of the device, and so on.

[0056] In this way, local data or data from at least one device on the current computing node can be uniformly collected. Since the data collection process is completed internally, other computing nodes do not need to configure corresponding data ports and cannot perceive the source of the data.

[0057] Then, through step 502, when the data exchange conditions are met, the collected data is transmitted to other computing nodes through a predetermined data port.

[0058] Data interaction conditions, that is, the conditions that must be met to interact with other computing nodes. Data interaction conditions can be pre-set according to business needs. For example: in the case of real-time monitoring data, the data interaction condition can be the collection of new data; in the case of outputting monitoring data at a predetermined time interval, the data interaction condition can be the arrival of a predetermined time interval; in the case of data acquisition requests initiated by other computing nodes, the data interaction condition can be the receipt of data synchronization requests from other computing nodes; in other cases, the data interaction condition can also be other conditions, which will not be repeated here.

[0059] According to a possible design, before the collected data is interacted with other computing nodes, the data can also be filtered by receiving a second filtering rule formed by the user's selection of the collected data according to the data item category. The filtered data is transmitted to other computing nodes through a predetermined data port.

[0060] In one embodiment, the above multiple collection units also include an external collection unit, and the monitoring data collected by other computing nodes is obtained through the external collection unit. For example, the external collection unit sends data collection requests to other computing nodes, receives data fed back by other computing nodes, etc.

[0061] Looking back at the above process, under the privacy computing architecture, a data collection framework is provided. A single computing node can configure a data collection module locally. The data collection module includes multiple collection units, and each collection unit is configured to collect network data, device data, application data, etc. Each collection unit obtains the corresponding data by configuring the local data collection port, and then transmits the collected data to other computing nodes through the externally exposed predetermined data port. In this way, a single computing node locally uniformly summarizes all machine indicators and engine indicators, shields the changes in the internal system from the external monitoring system, and protects the privacy of the data source (or running application), thereby providing an effective and feasible technical solution for data collection, aggregation, error analysis, etc. under the privacy computing framework.

[0062] According to another embodiment, there is also provided a computer readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 5The method described by et al.

[0063] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the Figure 5 The method described by et al.

[0064] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0065] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the technical concept of this specification. It should be understood that the above description is only the specific implementation method of the technical concept of this specification, and is not used to limit the protection scope of the technical concept of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of this specification should be included in the protection scope of the technical concept of this specification.

Claims

1. A data collection module in privacy computing, used for a single computing node to collect local data under a privacy computing architecture performed by multiple parties; the data collection module includes multiple collection units, each of which is configured to collect network data, device data, and application data; in, The first collection unit configured to collect application data is used to collect data of a predetermined application through a preset data port, and identify the port configured when a process is pulled up during the business processing, so as to collect application data in the business processing process in real time; The data acquisition module also includes a predetermined data port exposed to the outside, which is used to transmit the collected data to other computing nodes when the data interaction conditions are met.

2. The data acquisition module according to claim 1, wherein: The first acquisition unit acquires the data port of the process started by the manager through synchronization with the current service framework manager.

3. The data acquisition module according to claim 1, wherein: The data collection module further includes a first filtering unit configured to filter the collected data based on a preset first filtering rule so that the data collection module can summarize the filtered data.

4. The data acquisition module according to claim 1, wherein: The data acquisition module also includes a second filtering unit, which is configured to receive a user's selection of the collected data according to the data item category, thereby forming a second filtering rule for data filtering based on the second filtering rule when the collected data is transmitted to other computing nodes through the predetermined data port.

5. The data acquisition module according to claim 1, wherein: The data interaction condition includes one of the following: new data is collected, a predetermined time interval arrives, and a data synchronization request from another computing node is received.

6. The data acquisition module according to claim 1, wherein: The multiple collection units also include an external collection unit configured to obtain monitoring data collected by other computing nodes.

7. A data collection method in privacy computing, used in a privacy computing architecture where multiple parties perform privacy computing, where a single computing node collects local data through the data collection module as claimed in claim 1; the method comprises: The network data, device data and application data of the single computing node are collected respectively by multiple collection units, wherein the first collection unit configured to collect application data among the multiple collection units is used to collect data of a predetermined application through a pre-set data port, and identify the port configured when the process is pulled up during the business processing, so as to collect the application data in the business processing process in real time; When the data exchange conditions are met, the collected data is transmitted to other computing nodes through the predetermined data port.

8. The method of claim 7, wherein: The identifying of the port configured when the process is started during the service processing includes: Acquire the data port of the process started by the current business framework manager through synchronization.

9. The method of claim 7, wherein: The method further comprises: Based on a preset first filtering rule, the collected data is filtered so that the data collection module can summarize the filtered data.

10. The method of claim 7, wherein: The collected data transmitted to other computing nodes through the predetermined data port includes: After filtering the data based on the second filtering rule, the filtered data is transmitted to other computing nodes through the predetermined data port, wherein the second filtering rule is formed by receiving a user's selection of the collected data according to the data item category.

11. The method of claim 7, wherein: The data interaction condition includes one of the following: new data is collected, a predetermined time interval arrives, and a data synchronization request from another computing node is received.

12. The method of claim 7, wherein: The plurality of acquisition units further include an external acquisition unit, and the method further includes: The monitoring data collected by other computing nodes is requested through the external collection unit.

13. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 7 to 12.

14. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 7 to 12 is implemented.

Images