Multi-device data transmission method based on differential privacy

Abstract

The invention discloses a multi-device data transmission method based on differential privacy, which is applied to a multi-device communication system, and comprises the following steps: the system acquires original data, resource information data and path information data of a plurality of devices contained in the system, preprocesses each original data, and then constructs a super object model; obtaining a resource state vector and a topological structure adjacency list of each device through the super object model, wherein the system performs differential privacy processing on the resource state vector, the topological structure adjacency list and parameters in the process of dynamically updating the super object model; carrying out shortest path selection on the data transmission path of each device; the system fragments a data packet to be sent and distributes fragmented data to each device according to a load balancing principle, each device transmits the fragmented data to a receiving device, and the receiving device recombines and verifies the fragmented data after receiving the fragmented data. According to the invention, the transmission efficiency is improved while the data transmission privacy is ensured.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, in particular to the fields of distributed computing, the Internet of Things and data privacy protection, and specifically relates to a multi-device data transmission method based on differential privacy. Background Art

[0002] With the rapid development of distributed computing and Internet of Things technologies, a large number of devices are exchanging and communicating data through the network. Traditional data transmission modes mostly use a single path, which is prone to data congestion and inefficiency. At the same time, data transmission between devices needs to protect privacy, especially when processing sensitive data. Existing privacy protection methods cannot cope well with the coordinated transmission of multiple devices in distributed nodes. Therefore, how to improve transmission efficiency while protecting data privacy has become an urgent problem to be solved.

[0003] In view of this, this invention is proposed. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a multi-device data transmission method based on differential privacy. The method constructs a super-object model, obtains the resource state vector and topological structure adjacency table of each device through the super-object model, and performs differential privacy processing on them, and then optimizes the transmission path of each device, and distributes shard data to each device according to the principle of balanced load. Each device sends shard data to the receiving device according to the selected shortest path; in addition, during the data transmission process, the shortest transmission path of each device is dynamically adjusted according to the status data, and the shard data is redistributed according to the new path. This method can significantly improve the performance and stability of data transmission.

[0005] The purpose of the present invention is to be solved by the following technical solutions:

[0006] The present invention provides a multi-device data transmission method based on differential privacy, which is applied to a multi-device communication system, such as a distributed system, an Internet of Things system, a mobile application system, etc., including:

[0007] The system collects raw data, resource information data and path information data of multiple devices it contains, and pre-processes each raw data to obtain a data packet to be sent, then uses the resource information data and path information data to build a super object model, and dynamically updates the super object model. The resource state vector and topological structure adjacency table of each device are obtained through the super object model. The system performs differential privacy processing on the resource state vector, the topological structure adjacency table and the parameters in the process of dynamically updating the super object model;

[0008] The shortest path is selected for the data transmission path of each device according to the path information data of each device and the data after differential privacy processing; the system fragments the data packet to be sent, and distributes the fragmented data to each device according to the load balancing principle, and then each device transmits the fragmented data to the receiving device, and the receiving device reassembles and verifies the fragmented data after receiving it;

[0009] During the transmission process, the receiving device monitors the status data of the shortest path in real time and feeds the status data back to the system, and then the system dynamically adjusts the shortest path and fragmentation data of each device according to the status data.

[0010] Furthermore, the process of preprocessing each raw data is as follows:

[0011] (1) All original data were processed for missing values;

[0012] (2) All original data after missing value processing are tested for outliers using the Z-score method, and the detected outliers are corrected or eliminated;

[0013] (3) All raw data after outlier detection and correction are denoised.

[0014] Furthermore, the super object model is

[0015] Where N represents the number of devices included in the system. represents the resource state vector of device i, T i Represents the topological adjacency table of device i;

[0016] The resource state vector is expressed as Among them, r ij Indicates the j-th resource status data of device i;

[0017] The topological structure adjacency table is represented by T i ={j 1 ,j 2 ,...,j k}; where j 1 ,j 2 ,...,jk represents the set of devices directly connected to device i.

[0018] Furthermore, the following formula is used to dynamically update the super object model:

[0019]

[0020] in, is a resource consumption vector of device i at time t, wherein the resource consumption vector includes the consumption of each type of resource status data;

[0021] The resource consumption vector is expressed as where c ij (t) is the consumption of the j-th resource status data at time t.

[0022] Furthermore, the devices include IoT devices, sensors and terminal devices; the resource status data include computing power, bandwidth and storage space.

[0023] Furthermore, the system processes the current resource state vector of the device. Topological structure adjacency table T i , resource consumption vector Differential privacy processing is performed.

[0024] Furthermore, each device includes multiple transmission paths when transmitting data to the receiving device, and the shortest path set P corresponding to each device is established by the following formula;

[0025]

[0026]

[0027] Among them, w p is the weight of path p, d p is the delay of path p, α, β, γ are weight coefficients, and their values ​​are 0.3, 0.3, 0.4 respectively, Bandwith p 、Delay p 、LossRate p They represent the bandwidth, delay and packet loss rate of path p respectively.

[0028] Furthermore, the receiving device reassembles the fragmented data into data of a data packet to be sent according to the timestamp and sequence number of the received fragmented data, and performs verification and recovery on the reassembled data.

[0029] Furthermore, during the transmission process, the rate on each shortest path is dynamically adjusted according to its corresponding packet loss rate.

[0030] Furthermore, the status data includes bandwidth, delay, and packet loss rate; during the transmission process, the system dynamically adjusts the shortest path of each device in accordance with the method of claim 7 based on the bandwidth, delay, and packet loss rate fed back by the receiving device, and redistributes the fragmented data to the new shortest path based on the principle of load balancing.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1) The present invention provides a multi-device data transmission method based on differential privacy. The method constructs a super-object model and performs dynamic updates. The resource state vector and topological structure adjacency table of each device are obtained through the super-object model, and differential privacy processing is performed on the resource state vector, topological structure adjacency table and resource consumption vector. Then, the transmission path of each device is optimized, and shard data is distributed to each device according to the principle of balanced load. Each device sends shard data to the receiving device according to the selected shortest path. At the same time, during the transmission process, the system uses a path optimization algorithm to reselect the optimal path according to the status data fed back by the receiving device, and maximizes bandwidth utilization by adjusting the sharding strategy, reducing transmission delay and packet loss rate. Moreover, the optimization of dynamic path selection and sharding strategy can significantly improve the performance and stability of data transmission, ensure reliable transmission of data in a complex network environment, and solve the problems of insufficient privacy protection and low transmission efficiency during multi-device data transmission in the prior art.

[0033] 2) The present invention provides a multi-device data transmission method based on differential privacy, which dynamically adjusts the transmission rate according to the packet loss rate of the transmission path during the transmission process, thereby ensuring the reliability of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are incorporated in and constitute a part of this specification and, together with the description, serve to explain the principles of the present invention.

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 It is a flow chart of the multi-device data transmission method based on differential privacy of the present invention. DETAILED DESCRIPTION

[0037] Here, exemplary embodiments will be described in detail. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are only examples consistent with some aspects of the present invention as detailed in the appended claims.

[0038] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a multi-device data transmission method based on differential privacy, which is applied to a distributed system, including:

[0040] The system collects raw data, resource information data and path information data of multiple devices it contains, and pre-processes each raw data to obtain a data packet to be sent, then uses the resource information data and path information data to build a super object model, and dynamically updates the super object model. The resource state vector and topological structure adjacency table of each device are obtained through the super object model. The system performs differential privacy processing on the resource state vector, the topological structure adjacency table and the parameters in the process of dynamically updating the super object model;

[0041] The shortest path is selected for the data transmission path of each device based on the path information data of each device and the data after differential privacy processing. The system fragments the data packets to be sent and distributes the fragmented data to each device according to the load balancing principle. Each device transmits the fragmented data to the receiving device, which reassembles and verifies the fragmented data after receiving it.

[0042] During the transmission process, the receiving device monitors the status data of the shortest path in real time and feeds the status data back to the system, and then the system dynamically adjusts the shortest path and fragmentation data of each device according to the status data.

[0043] Furthermore, the process of preprocessing each raw data is as follows:

[0044] (1) All original data are processed for missing values. Specifically, the mean interpolation method or K nearest neighbor interpolation algorithm is used to supplement the missing data;

[0045] (2) All original data after missing value processing are tested for outliers using the Z-score method, and the detected outliers are corrected or eliminated;

[0046] The calculation formula of Z-score is:

[0047]

[0048] Among them, x i represents the i-th data in the original data, μ represents the mean of the original data, and σ represents the standard deviation of the original data;

[0049] When | Z i When |>3, determine x i is an outlier;

[0050] (3) All original data after outlier detection and correction are subjected to denoising. Specifically, the denoising is performed using a Kalman filter or a wavelet transform.

[0051] Furthermore, a super object model is constructed using resource information data and path information data to represent the physical resources and logical topology of each device node.

[0052] Specifically, the super object model is expressed as

[0053] Where N represents the number of devices included in the system. represents the resource state vector of device i, T i Represents the topological adjacency table of device i;

[0054] Specifically, the resource state vector is expressed as Among them, r ij Indicates the j-th resource status data of device i;

[0055] The connection status of device i with other devices is represented by the topological structure adjacency table of the device. Specifically, the topological structure adjacency table is represented by T i ={j 1 ,j 2 ,...,j k}; where j 1 ,j 2 ,...,jk represents the set of devices directly connected to device i.

[0056] Furthermore, at each time step t, the resource state vector of device i is updated according to the current load and resource consumption. Specifically, the following formula (2) is used to dynamically update the super object model:

[0057]

[0058] in, is a resource consumption vector of device i at time t, wherein the resource consumption vector includes the consumption of each type of resource status data;

[0059] The resource consumption vector is expressed as where c ij (t) is the consumption of the j-th resource status data at time t.

[0060] In this embodiment, the devices include Internet of Things devices, sensors, and terminal devices; the resource status data include computing power, bandwidth, storage space, latency, bandwidth utilization, throughput, etc.; the resource information data includes computer resources, storage resources, network resources, service resources, security resources, configuration information, environmental information, and performance indicators, etc.

[0061] Differential privacy data processing: The system processes the resource state vector of the current device Topological structure adjacency table Ti and resource consumption vector Sensitive data such as images are differentially privately processed by adding Gaussian noise to prevent the overall statistical information of the system from being leaked, thereby ensuring the privacy security of data transmission.

[0062] Specifically, the process of differential privacy processing using Gaussian noise is as follows:

[0063] Assume that the data to be encrypted is f(x), and the encrypted data is M(x). The encryption algorithm is as follows:

[0064] M(x)=f(x)+N(0,σ 2 ) (3)

[0065] in In formula (3), the noise has a mean of 0 and a variance of σ 2 The calculation of σ depends on Δf (function sensitivity), ε (privacy budget) and δ (allowable probability failure).

[0066] In this embodiment, the transmission path selection includes: according to the path information data of each device and the data processed by differential privacy (the current resource state vector of the device Topological structure adjacency table T i and resource consumption vector The optimal multi-path selection scheme (also known as the shortest path selection) is determined by using differential privacy processing to select sensitive data. The goal is to find an optimal path for each device and then obtain a set of all optimal paths.

[0067] Specifically, each device includes multiple transmission paths when transmitting data to the receiving device, and the shortest path set P corresponding to each device is established by the following formula;

[0068]

[0069] Among them, w p is the weight of path p, d p is the delay of path p, α, β, and γ are weight coefficients, and their values ​​are 0.3, 0.3, and 0.4 respectively. The value can be adjusted according to the specific scenario. p 、Delay p 、LossRate p They represent the bandwidth, delay and packet loss rate of path p respectively.

[0070] Its specific implementation adopts heuristic algorithm or dynamic programming algorithm.

[0071] In this embodiment, the fragmentation and allocation of the data packets to be sent include: according to the above path selection results, the data packets to be sent are fragmented and allocated to the shortest path selected by each device. The allocation is based on the bandwidth and load capacity of the shortest path to ensure that each path is load balanced during the transmission process. The path fragment size allocation formula is as follows:

[0072]

[0073] Among them, D j represents the number of shard data allocated to path j, w j represents the weight of path j (calculated by formula 5), ​​D total Indicates the size of the data packet to be sent.

[0074] Furthermore, the receiving device reorganizes the fragmented data into data packets to be sent according to the timestamp and sequence number of the received fragmented data, and performs data verification and recovery using an error correction algorithm, wherein the error correction algorithm may use RS code or LDPC code.

[0075] Specifically, the process of data verification and recovery is as follows:

[0076] (1) Sequence number sorting: The receiving device first sorts the fragments in ascending order according to the sequence number of each fragment to ensure that the data packets are reassembled in the correct order;

[0077] (2) Timestamp alignment: The receiving device reassembles the fragmented data into the data packet to be sent based on the sequence number and timestamp. The formula for the data reassembly process is:

[0078]

[0079] Among them, d i Indicates the received i-th fragment data, D reconstructed Represents the complete data after final reorganization;

[0080] (3) Verification and error correction: Use an error correction algorithm to perform integrity verification and error correction on the final reorganized complete data to solve the data loss or error that may occur during network transmission. This embodiment uses RS (Reed-Solomon) code or LDPC code for data correction.

[0081] During the transmission process, the rate on each shortest path is dynamically adjusted according to its packet loss rate to ensure that the fragmented data on each path is transmitted independently. This can reduce data loss and congestion and ensure the stability and efficiency of transmission. The calculation formula for the data transmission rate of each path is:

[0082]

[0083] Among them, LossRate i represents the packet loss rate of path i.

[0084] Furthermore, the state data includes bandwidth, delay, and packet loss rate; during the transmission process, the receiving device feeds back bandwidth, delay, packet loss rate and other indicators to the system, and the system updates the data transmission path in real time according to the received indicators, and redistributes the fragmented data to each path according to the principle of load balancing, that is, for paths with reduced performance, the distribution of fragmented data is reduced or stopped, and for paths with improved performance, the distribution of fragmented data is increased. Specifically including:

[0085] Monitoring feedback: The receiving device continuously monitors the bandwidth, delay, packet loss rate and other status data of the transmission path.

[0086] Weight update: Based on the latest feedback status data, the system reselects the optimal multi-path selection scheme under the current system state according to the above-mentioned transmission path selection method, and dynamically adjusts the path selection.

[0087] Shard redistribution: Based on the new path selection results, the shard data is redistributed and the transmission tasks on each path are adjusted.

[0088] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention.

[0089] It should be understood that the present invention is not limited to what has been described above and that various modifications and changes may be made without departing from its scope. The scope of the present invention is limited only by the appended claims.

Claims

1. A multi-device data transmission method based on differential privacy, characterized in that: Applied to multi-device communication systems, including: The system collects raw data, resource information data and path information data of multiple devices it contains, and pre-processes each raw data to obtain a data packet to be sent, then uses the resource information data and path information data to build a super object model, and dynamically updates the super object model. The resource state vector and topological structure adjacency table of each device are obtained through the super object model. The system performs differential privacy processing on the resource state vector, the topological structure adjacency table and the parameters in the process of dynamically updating the super object model; The shortest path is selected for the data transmission path of each device according to the path information data of each device and the data after differential privacy processing; the system fragments the data packet to be sent, and distributes the fragmented data to each device according to the load balancing principle, and then each device transmits the fragmented data to the receiving device, and the receiving device reassembles and verifies the fragmented data after receiving it; During the transmission process, the receiving device monitors the status data of the shortest path in real time and feeds the status data back to the system, and then the system dynamically adjusts the shortest path and fragmentation data of each device according to the status data.

2. The multi-device data transmission method based on differential privacy according to claim 1, characterized in that: The preprocessing process for each raw data is as follows: (1) All original data were processed for missing values; (2) All original data after missing value processing are tested for outliers using the Z-score method, and the detected outliers are corrected or eliminated; (3) All raw data after outlier detection and correction are denoised.

3. The multi-device data transmission method based on differential privacy according to claim 1, characterized in that: The super object model is Where N represents the number of devices included in the system. represents the resource state vector of device i, T i Represents the topological adjacency table of device i; The resource state vector is expressed as Among them, r ij Indicates the j-th resource status data of device i; The topological structure adjacency table is represented by T i ={j1,j2,...,j k }; where j1, j2, ..., j k Represents the set of devices directly connected to device i.

4. The multi-device data transmission method based on differential privacy according to claim 3 is characterized in that: The following formula is used to dynamically update the super object model: in, is a resource consumption vector of device i at time t, wherein the resource consumption vector includes the consumption of each type of resource status data; The resource consumption vector is expressed as where c ij (t) is the consumption of the j-th resource status data at time t.

5. The multi-device data transmission method based on differential privacy according to claim 3, characterized in that: The devices include IoT devices, sensors and terminal devices; the resource status data include computing power, bandwidth and storage space.

6. The multi-device data transmission method based on differential privacy according to claim 3, characterized in that: The system has a current resource status vector for the device Topological structure adjacency table T i , resource consumption vector Differential privacy processing is performed.

7. The multi-device data transmission method based on differential privacy according to claim 1, characterized in that: When each device transmits data to the receiving device, multiple transmission paths are included. The shortest path set P corresponding to each device is established by the following formula; Among them, w p is the weight of path p, d p is the delay of path p, α, β, γ are weight coefficients, Bandwith p 、Delay p 、LossRate p They represent the bandwidth, delay and packet loss rate of path p respectively.

8. The multi-device data transmission method based on differential privacy according to claim 1, characterized in that: The receiving device reassembles the fragmented data into data of a data packet to be sent according to the timestamp and sequence number of the received fragmented data, and performs verification and recovery on the reassembled data.

9. The multi-device data transmission method based on differential privacy according to claim 1, characterized in that: During the transmission process, the rate on each shortest path is dynamically adjusted according to its corresponding packet loss rate.

10. The multi-device data transmission method based on differential privacy according to claim 7, characterized in that: The status data includes bandwidth, delay, and packet loss rate; during the transmission process, the system dynamically adjusts the shortest path of each device in accordance with the method of claim 7 based on the bandwidth, delay, and packet loss rate fed back by the receiving device, and redistributes the fragmented data to the new shortest path based on the principle of load balancing.

Images