Private data flow situation awareness method and device based on federal learning

Abstract

The invention relates to the technical field of data flow situation awareness, and provides a private data flow situation awareness method and device based on federal learning. The method comprises the following steps: collecting a data set of each data source terminal by each data source terminal, identifying sensitive data in the data set by using a preset strategy, and desensitizing the sensitive data in the data set by using a preset encryption algorithm to obtain a training data set; and the federated learning server and each data source terminal perform federated learning by using each training data set to obtain a data flow situation model for performing data flow situation analysis. According to the invention, each data source terminal automatically collects the data and desensitizes the data, so that the security in the data exchange process in the subsequent federated learning process is ensured, and each data source terminal and the federated learning server carry out federated learning together, so that the training process can be dispersed, and the training efficiency is improved. And the pressure of the federated learning server is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data flow dynamic trend perception, and particularly relates to a privacy data flow dynamic trend perception method and device based on federated learning. BACKGROUND

[0002] The existing data flow dynamic trend perception method is that a server centrally collects plaintext data in a data source and analyzes the data flow dynamic trend. Since the data is plaintext, centralized collection can easily cause data leakage, resulting in low data security. In addition, centralized analysis of a large amount of data can also cause the server to have increased running pressure, insufficient disk capacity and other problems.

[0003] Therefore, it is urgent to overcome the defects of the prior art in the technical field. SUMMARY

[0004] The present application solves the technical problem of providing a privacy data flow dynamic trend perception method and device based on federated learning to solve the problem of low data security in data flow dynamic trend perception in the prior art.

[0005] The present application adopts the following technical solutions: In a first aspect, the present application provides a privacy data flow dynamic trend perception method based on federated learning, comprising: Each data source terminal collects its own data set, uses a preset strategy to identify sensitive data in the data set, uses a preset encryption algorithm to desensitize the sensitive data in the data set, and obtains a training data set; The federated learning server and each data source terminal use each training data set to perform federated learning to obtain a data flow dynamic trend model for data flow dynamic trend analysis.

[0006] Preferably, the use of a preset strategy to identify sensitive data in the data set specifically includes: using a preset sensitive information list to identify first sensitive data in the data set; calculating the number of identical non-sensitive data derived from the first sensitive data in the data set; If the number of derived identical non-sensitive data is greater than a preset number, the non-sensitive data is taken as second sensitive data.

[0007] Preferably, the preset encryption algorithm is pre-issued by the federated learning server, specifically including: The federated learning server pre-generates a corresponding public key for each data source terminal and selects a corresponding encryption algorithm; The selected encryption algorithm and public key are sent to the data source terminal so that the data source terminal can use the encryption algorithm and public key as a preset encryption algorithm to de-identify sensitive data in the dataset.

[0008] Preferably, the federated learning server and each data source terminal use each training dataset to perform federated learning to obtain a data flow situation model for data flow situation analysis, specifically including: Each data source terminal performs interactive transmission of its training datasets, and uses all of its own training datasets after the exchange transmission to train the (i-1)th learning model, thus obtaining the i-th baseline model. The gradient information of the i-th baseline model is transmitted to the federated learning server, so that the federated learning server can aggregate the gradient information of each i-th baseline model to obtain the i-th learning model. The i-th learning model is then distributed to each data source terminal so that the data source terminal can train the i-th learning model until the K-th learning model is obtained. The K-th learning model is then used as the data flow situation model.

[0009] Preferably, after training the data flow situation model, the method further includes: The federated learning server receives subsets of de-identified data flow trends output by the data flow trend models of each data source terminal, integrates each subset of de-identified data flow trends into a set of de-identified data flow trends, and sends it to the analysis server. The analysis server decrypts each de-identified data in the de-identified data flow pattern set to obtain the sensitive data flow pattern set; Calculate the intersection between the nth subset of the sensitive data flow situation set and the nth subset of the preset data flow situation set. Determine the accuracy of the nth subset of the sensitive data flow situation set based on the proportion of data in the intersection to the data in the nth subset of the preset data flow situation set. Measure the accuracy of the data flow situation model based on the average accuracy of each subset in the sensitive data flow situation set.

[0010] Preferably, the analysis server decrypts each piece of de-identified data in the de-identified data flow trend set, specifically including: The federated learning server sends each decryption algorithm and its corresponding private key to the analysis server in advance, so that the analysis server can use the corresponding decryption algorithm and the corresponding private key to decrypt the anonymized data.

[0011] Preferably, the method further includes: The analysis server also extracts data nodes from the sensitive data flow pattern set and generates a directed graph based on the data flow relationships between the data nodes.

[0012] Preferably, the data set of each data source terminal comprises one or more of source data, a source data producer, a source data consumer, a data source to which the source data belongs, a data source to which the source data is directed, and derivative data of the source data.

[0013] In a second aspect, the present application further provides a privacy data flow dynamic trend perception device based on federated learning, which is used to implement the privacy data flow dynamic trend perception method based on federated learning in the first aspect, and the device comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the privacy data flow dynamic trend perception method based on federated learning in the first aspect.

[0014] In a third aspect, the present application further provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors to complete the method in the first aspect.

[0015] In a fourth aspect, a chip is provided, which comprises a processor and an interface, and is used to call and run a computer program stored in a memory to execute the method in any one of the first aspects.

[0016] In a fifth aspect, a computer program product comprising instructions which, when executed on a computer or processor, cause the computer or processor to carry out the method in any one of the first aspects.

[0017] The present application collects data by each data source terminal and desensitizes the data, thereby ensuring the security in the subsequent data exchange process in the federated learning process, and since each data source terminal and the federated learning server jointly perform federated learning, the training process can be dispersed, and the pressure on the federated learning server is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0019] Figure 1 is a flow diagram of a privacy data flow dynamic trend perception method of federated learning provided by the embodiments of the present application; Figure 2is a flowchart of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 3 is a flowchart of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 4 is a flowchart of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 5 is a flowchart of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 6 is a schematic diagram of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 7 is a schematic diagram of a privacy data flow dynamic trend perception method of federated learning provided by an embodiment of the present application; Figure 8 is an architecture schematic diagram of a privacy data flow dynamic trend perception device of federated learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0021] Unless otherwise required by context, the term "comprises" in the specification and claims is to be construed as open-ended, i.e. as "comprises but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example" or "some examples" are intended to mean that the specific feature, structure, material or characteristic associated with that embodiment or example is included in at least one embodiment or example of the present disclosure. The illustrative representation of the above terms does not necessarily mean that the same embodiment or example is referred to. In addition, the specific features, structures, materials or characteristics described can be included in any one or more embodiments or examples in any appropriate manner, i.e. although they are carried in the embodiments or examples of the above terms due to the order of appearance and location, they are not limited to being carried by one embodiment or example in a combined manner.

[0022] In the description of the present application, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "a plurality of" is two or more. In addition, for example, in the description, the same type of nouns will also be described as two independent individuals by adding "A", "B" at the end, in which case the features defined with "A", "B" are only used for the purpose of distinguishing the same type of individual description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0023] In the description of the present application, the expression "A and / or B" (where A and B represent specific feature content in the form) is used, and the corresponding expression includes the following three combinations: only A, only B, and the combination of A and B.

[0024] In the present application, "about", "approximately" or "approximately" includes the value stated and the average value within the acceptable deviation range of the specific value, wherein the acceptable deviation range is determined by considering the measurement being discussed and the error related to the measurement of the specific quantity (i.e. the limitation of the measurement system) by the person skilled in the art.

[0025] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0026] Embodiment 1: Embodiment 1 of the present application provides a privacy data flow dynamic trend perception method for federated learning, as shown in Figure 1 , comprising: In step 201, each data source terminal collects its own data set, uses a preset strategy to identify sensitive data in the data set, and uses a preset encryption algorithm to desensitize the sensitive data in the data set to obtain a training data set.

[0027] In step 202, the federated learning server and each data source terminal use each training data set to perform federated learning to obtain a data flow dynamic trend model for data flow dynamic trend analysis.

[0028] Wherein, the preset strategy and the preset encryption algorithm are obtained by demand analysis by those skilled in the art, and different data source terminals use different preset encryption algorithms for desensitization (i.e. encryption).

[0029] The data set of each data source terminal includes one or more of source data, source data producer, source data consumer, data source to which the source data belongs, destination data source of the source data, and derivative data of the source data.

[0030] As in an actual application scenario, five types of data source terminals are included, namely database terminal, file system terminal, web api http stream terminal, real-time data stream terminal, and message queue terminal. Each type of terminal collects the following data: 1) Data a1, a2, a3, …, a n , a1, a2, a3, …, a n are the data flowing in the five types of data sources (i.e., database, file system, web api http stream, real-time data stream, and message queue), i.e., source data, such as data in the database, documents and tables in the file system, interface data stream in the web api http stream, etc.

[0031] 2) Data a1, a2, a3, …, a n , p1, p2, p3, …, p n , i.e., data a n is collected in which data source p n , also known as the data source to which the source data belongs.

[0032] 3) Source data source (producer) s1, s2, s3, …, s n that produces data a1, a2, a3, …, a n , i.e., the data a n collected is flowing from which data source s n to the data source p n to which the collected data a n belongs, also known as the source data producer.

[0033] 4) Destination data source (consumer) d1, d2, d3, …, d n that uses data a1, a2, a3, …, a n , i.e., data a n will flow to which data source d n , also known as the destination data source of the source data.

[0034] 5) Data t1, t2, t3, …, t n produced by the consumer consuming data a1, a2, a3, …, a n , i.e., what data a n flows to the next data source d n , also known as the derivative data of the source data.

[0035] The embodiment ensures the security in the data exchange process in the subsequent federated learning process by collecting data by each data source terminal and desensitizing the data, and the training process can be dispersed and the pressure of the federated learning server can be reduced because the federated learning is performed by each data source terminal and the federated learning server together.

[0036] In a specific application scenario, the sensitive data in the data set is identified using a preset strategy, such as Figure 2 As shown, specifically includes: In step 301, the first sensitive data in the data set is identified using a preset sensitive information list; that is, a preset sensitive information list is set in advance, the preset sensitive information list stores the identification rules of each item of sensitive data, and the identification rules are used to identify the sensitive data first, and for the non-sensitive data identified as non-sensitive data, subsequent operations are performed.

[0037] In step 302, the number of the same non-sensitive data derived from the first sensitive data in the data set is calculated.

[0038] In step 303, if the number of the derived same non-sensitive data is greater than a preset number, the non-sensitive data is taken as second sensitive data, and the sensitive data in the data set includes the first sensitive data and the second sensitive data. The preset number is obtained by experience analysis by a person skilled in the art, and in an optional embodiment, the preset number can be 2. For example, first, the known sensitive data classification and grading list L M The data a n collected in each data source is desensitized. cn For example, in this step, the sensitive data classification and grading list L M (that is, the preset sensitive information list) identifies that a certain name a c1 and a certain mobile phone number a c2 stored in the database are sensitive data (that is, first sensitive data). Then, the web api http stream consumes the certain name a c1 and the certain mobile phone number a c2 (For example, a web api http stream can query the real-name ID card number corresponding to the name data or the mobile phone number data through the name data or the mobile phone number data, and the data flow in the consumption process is from the name data or the mobile phone number data to the ID card number data) generates ID card data (that is, derived data), and the two ID card data are the same. At this time, there are N L same data, so the ID card data can be considered as sensitive data (that is, second sensitive data).

[0039] The preset encryption algorithm is previously issued by the federated learning server, such as Figure 3 As shown in the figure, specifically comprising: In step 401, the federated learning server generates a corresponding public key for each data source terminal in advance, and selects a corresponding encryption algorithm; The selected encryption algorithm can be selected by generating a random number, that is, generating a random number in the total number of encryption algorithms, and the generated random number is the selected encryption algorithm.

[0040] In step 402, the selected encryption algorithm and public key are issued to the data source terminal, so that the data source terminal uses the encryption algorithm and the public key as the preset encryption algorithm to desensitize the sensitive data in the data set.

[0041] For the sensitive data a f {Z f1 {a f1 ,s f1 ,d f1 ,p f1},Z f2 {a f2 ,s f2 ,d f2 ,p f2},……,Z fn {a fn ,s fn ,d fn ,p fn}} in the data set Z f1 、a f2 、……、a fn and the sensitive data corresponding to the sensitive data belongs to the data source p f1 、p f2 、……、p fn , there are five kinds of encryption algorithms corresponding to the kind of data source to which the sensitive data belongs: database data source encryption algorithm f x1 , file system data source encryption algorithm f x2 , web api http stream data source encryption algorithm f x3 , real-time data stream data source encryption algorithm f x4 , message queue data source encryption algorithm f x5 , and the public key pub x1 、pub x2 、pub x3 、pub x4 、pub x5 .

[0042] There are also five kinds of decryption algorithms corresponding to the kind of data source to which the data belongs: database data source encrypted data decryption algorithm D x1, file system data source encrypted data decryption algorithm D x2 , web api http stream data source encrypted data decryption algorithm D x3 , real-time data stream data source encrypted data decryption algorithm D x4 , and message queue data source encrypted data decryption algorithm D x5 , five decryption algorithms D x1 , D x2 , D x3 , D x4 , and D x5 correspond to the respective private keys pri x1 , pri x2 , pri x3 , pri x4 , and pri x5 .

[0043] In actual use, five encryption algorithms f x1 , f x2 , f x3 , f x4 , f x5 and their encryption algorithm corresponding public keys pub x1 , pub x2 , pub x3 , pub x4 , pub x5 According to the data belonging to the data source p f1 , p f2 , ……, p fn , respectively, the data a f1 , a f2 , ……, a fn is encrypted; At the same time, the encryption algorithm f x1 , f x2 , f x3 , f x4 , f x5 will ensure that the same data in different data sources still has the same encryption result, that is, in the sensitive data collected by different data source terminals, if the sensitive data a1 collected by a certain database data source is equal to the sensitive data a1 collected by a certain file system, then the encryption result of the sensitive data a1 of the certain database data source and the sensitive data a1 of the certain file system is still the same, and the sensitive data a f1 , a f2 , ……, a fn is encrypted by the encryption algorithm f x1 , f x2 , f x3 , f x4 , f x5 encrypted sensitive data a fx1 , a fx2 , ……, afxn ; replace the encrypted sensitive data a fx1 , a fx2 , …, a fxn with the corresponding plaintext sensitive data in the data set Z f {Z f1 {a f1 , s f1 , d f1 , p f1}, Z f2 {a f2 , s f2 , d f2 , p f2}, …, Z fn {a fn , s fn , d fn , p fn}} to obtain the training data set Z f {Z f1 {a fx1 , s f1 , d f1 , p f1}, Z f2 {a fx2 , s f2 , d f2 , p f2}, …, Z fn {a fxn , s fn , d fn , p fn}}.

[0044] In an optional implementation, the federated learning server and each data source terminal perform federated learning using each training data set to obtain a data flow trend model for performing data flow trend analysis, as shown in FIG. 5, specifically comprising: Figure 4 In step 501, each data source terminal performs interactive transmission of each training data set, and uses all training data sets of itself after exchange transmission to train the i-1 learning model to obtain the i reference model.

[0045] ​In step 502, the gradient information of the i-th reference model is transmitted to the federated learning server, so that the federated learning server aggregates the gradient information of each i-th reference model to obtain an i-th learning model, and the i-th learning model is distributed to each data source terminal, so that the data source terminal trains the i-th learning model until a K-th learning model is obtained, and the K-th learning model is used as the data flow trend model; wherein the 0-th learning model is a preset initial learning model, which is pre-stored by the federated learning server and distributed to each data source terminal. In an optional embodiment, K = 10.

[0046] After training the data flow trend model, as shown in Figure 5 , the method further comprises: In step 601, the federated learning server receives the desensitized data flow trend subsets output by the data flow trend models of each data source terminal, and sends each desensitized data flow trend subset to the analysis server as a desensitized data flow trend set.

[0047] In step 602, the analysis server decrypts each desensitized data in the desensitized data flow trend set to obtain a sensitive data flow trend set.

[0048] In step 603, the intersection between the n-th subset in the sensitive data flow trend set and the n-th subset in the preset data flow trend set is calculated, the accuracy of the n-th subset in the sensitive data flow trend set is determined according to the proportion of the data in the intersection to the data in the n-th subset in the preset data flow trend set, and the accuracy of the data flow trend model is measured according to the average value of the accuracies of each subset in the sensitive data flow trend set.

[0049] The accuracy of the n-th subset in the sensitive data flow trend set can be determined according to the proportion of the data in the intersection to the data in the n-th subset in the preset data flow trend set, that is, when the proportion exceeds a preset proportion, the accuracy of the n-th subset in the sensitive data flow trend set is 1 (i.e. accurate), otherwise it is 0 (i.e. inaccurate).

[0050] The accuracy of the data flow trend model is measured according to the average value of the accuracies of each subset in the sensitive data flow trend set, which specifically includes: when the average value is greater than or equal to a preset threshold, the data flow trend model is considered to be accurate, otherwise, the data flow trend model is considered to be inaccurate. The preset threshold and the preset proportion are obtained by demand analysis by those skilled in the art. In an optional embodiment, the preset threshold can be 0.8.

[0051] For example, there is a known accurate data flow data set (i.e. the preset data flow trend set) DL true {DL true1 ,DLtrue2 DL true3 ,……,DL truen}, Analysis Server FX server The sensitive data flow dataset (i.e., the sensitive data flow situation set) DF{DF1,DF2,DF3,……,DF n The sensitive data flow subsets DF1, DF2, DF3, ..., DF in} n Compare with known accurate data flow datasets DL true DL of each subset true1 DL true2 DL true3 ...DL truen Find the intersection, i.e., calculate DF n ∩DL truen ,like , and DF n ∩DL truen Length percentage DL truen The total length is half or more, i.e., len(DF) n ∩DL truen ) / len(DL truen If )≥1 / 2, then the sensitive data flow data subset DF is considered to be... n If the accuracy is met, then len(DF) n ∩DL truen ) / len(DL truen If ) < 1 / 2, then the sensitive data flow data DF is considered. n Inaccuracy is not guaranteed; if sensitive data flows, data DF n If the accuracy is met, then the analysis server FX server It will transfer sensitive data flow data DF n Accuracy value T DFn Set to 1, if sensitive data flows, data DF n If it does not meet accuracy, then analyze the FX server. server It will transfer sensitive data flow data DF n Accuracy value T DFn Set to 0. Analyze server FX server Calculate DF1, DF2, DF3, ..., DF n Accuracy value T DF1 T DF2 T DF3 ... T DFn Then, for DF1, DF2, DF3, ..., DF n Accuracy value T DF1 T DF2 T DF3 ... T DFnThe average value of the calculation accuracy value is calculated, that is, ∑(T DF1 , T DF2 , T DF3 , …, T DFn ) / n, if the average value of the accuracy value ∑(T DF1 , T DF2 , T DF3 , …, T DFn ) / n is greater than or equal to T z (T z is 0.8 by default and can be set by the user, 0.5 < T z < 1), then the sensitive data flow data set DF{DF1, DF2, DF3, …, DF n} is the accurate result of data flow, and the data flow trend model is accurate; if the average value of the accuracy value ∑(T DF1 , T DF2 , T DF3 , …, T DFn ) / n is less than T z , then the data flow trend model is inaccurate.

[0052] In actual use, if the data flow trend model is found to be inaccurate by measurement, the analysis server sends the overall inaccurate result to the federated learning model server, and the federated learning model server adjusts the initial learning model LM c to obtain the learning model LM c1 , and repeats the federated learning and accuracy judgment. If the average value of the accuracy value ∑(T DF1 , T DF2 , T DF3 , …, T DFn ) / n obtained by calculation is still less than T z , the federated learning model server FL server adjusts the learning model LM c1 to obtain the learning model LM c2 , and repeats the federated learning algorithm and the accuracy judgment algorithm again, until after N rounds of circulation, the accuracy value average value ∑(T DF1 , T DF2 , T DF3 , …, T DFn ) / n of the encrypted sensitive data a fx1 , a fx2 , …, a fxn obtained by federated learning training of the learning model LM cn is greater than or equal to T z . At this time, the sensitive data flow data set DF{DF1, DF2, DF3, …, DF n} obtained after N rounds of circulation is the accurate result of data flow.

[0053] In actual use, the analysis server decrypts each desensitized data in the desensitized data flow trend set, specifically including: the federated learning server pre-sends each decryption algorithm and the corresponding private key to the analysis server, so that the analysis server uses the corresponding decryption algorithm and the corresponding private key to decrypt the desensitized data. That is, in the above step 402, when the federated learning server issues the selected encryption algorithm and the public key to the data source terminal, it also synchronously sends the private key and the decryption algorithm to the analysis server.

[0054] In actual application scenarios, the method further includes: the analysis server also extracts data nodes from the sensitive data flow trend set, generates a directed graph according to the data flow relationship of the data nodes, thereby realizing visualization of the data flow trend; wherein the data nodes are data nodes corresponding to each data, and when multiple data correspond to one data node, a de-duplication operation is also performed.

[0055] Embodiment 2: Based on the method described in embodiment 1, the present application is combined with specific application scenarios and described by related technical expressions in the scene to illustrate the implementation process in the specific scene of the present application.

[0056] The federated learning privacy data flow trend perception method provided in this embodiment, as shown in Figure 6 , includes the following steps: In step S1, data source data collection.

[0057] In step S2, data source sensitive data identification.

[0058] In step S3, sensitive data privacy protection.

[0059] In step S4, sensitive data exchange.

[0060] In step S5, federated learning algorithm training.

[0061] In step S6, federated learning result analysis.

[0062] In step S7, data flow trend visualization.

[0063] The data source data collection specifically includes: collecting by deploying a data collection device on the data source terminal, and the data source data collection includes the following contents: For a system X n , five types of data sources (database, file system, web api http stream, real-time data stream, and message queue) in system X n collect data a1, a2, a3, …, a n , production data a1, a2, a3, …, an Source data sources (producers) s1, s2, s3, …, s n , using data a1, a2, a3, …, a n Destination data sources (consumers) d1, d2, d3, …, d n , data a1, a2, a3, …, a n Belonging data sources p1, p2, p3, …, p n , consumers consuming data a1, a2, a3, …, a n Produced data t1, t2, t3, …, t n , a1, a2, a3, …, a n , s1, s2, s3, …, s n , d1, d2, d3, …, d n , p1, p2, p3, …, p n , t1, t2, t3, …, t n Integrated into data sets Z1{a1, s1, d1, p1, t1}, Z2{a2, s2, d2, p2, t2}, Z3{a3, s3, d3, p3, t3}, Z4{a4, s4, d4, p4, t4}, …, Z n {a n ,s n ,d n ,p n ,t n}, data sets Z1{a1, s1, d1, p1, t1}, Z2{a2, s2, d2, p2, t2}, Z3{a3, s3, d3, p3, t3}, Z4{a4, s4, d4, p4, t4}, …, Z n {a n ,s n ,d n ,p n ,t n} are integrated into data source terminal data set Z{Z1, Z2, Z3, …, Z n}.

[0064] The data source sensitive data identification specifically comprises: There is a known sensitive data classification and grading list L M and an algorithm A1 to identify sensitive data in the data source terminal data set Z{Z1, Z2, Z3, …, Z n}.

[0065] First, through the known sensitive data classification and grading list L MFor the data source terminal dataset Z{Z1{a1,s1,d1,p1,t1},Z2{a2,s2,d2,p2,t2},Z3{a3,s3,d3,p3,t3},Z4{a4,s4,d4,p4,t4},……,Z n {a n ,s n ,d n ,p n ,t n The data a1, a2, a3, ..., a n Initial sensitive data identification is performed, and the dataset Z is obtained after the initial sensitive data identification. c {Z c1 {a c1 ,s c1 ,d c1 ,p c1 ,t c1},Z c2 {a c2 ,s c2 ,d c2 ,p c2 ,t c2},Z c3 {a c3 ,s c3 ,d c3 ,p c3 ,t c3},Z c4 {a c4 ,s c4 ,d c4 ,p c4 ,t c4},……,Z cn {a cn ,s cn ,d cn ,p cn ,t cn Algorithm A1 is implemented as follows, taking consumer consumption-sensitive data a as an example. cn The generated data t cn Perform statistical analysis; if there exists N L (N L The default value is 2 (you can set it yourself) for 2 or more identical data t. cn And the data t cn Not known sensitive data a cn Then it is considered that the data t cn Also considered sensitive data, all sensitive data that meet the above conditions (t) cn and its sensitive data t cn The data generated from the corresponding source data source, destination data source, affiliated data source, and consumer-sensitive data are integrated into dataset Z.c {Z c1 {a c1 ,s c1 ,d c1 ,p c1 ,t c1},Z c2 {a c2 ,s c2 ,d c2 ,p c2 ,t c2},Z c3 {a c3 ,s c3 ,d c3 ,p c3 ,t c3},Z c4 {a c4 ,s c4 ,d c4 ,p c4 ,t c4},……,Z cn {a cn ,s cn ,d cn ,p cn ,t cn}} Get Z c1 {Z c11 {a c11 ,s c11 ,d c11 ,p c11 ,t c11},Z c12 {a c12 ,s c12 ,d c12 ,p c12 ,t c12},Z c13 {a c13 ,s c13 ,d c13 ,p c13 ,t c13},Z c14 {a c14 ,s c14 ,d c14 ,p c14 ,t c14},……,Z c1n {a c1n ,s c1n ,d c1n ,p c1n ,t c1n This process is repeated N times until no two or more consumers generate the same sensitive data. After N rounds, the final dataset Z is obtained.f {Z f1 {a f1 ,s f1 ,d f1 ,p f1 ,t f1},Z f2 {a f2 ,s f2 ,d f2 ,p f2 ,t f2},……,Z fn {a fn ,s fn ,d fn ,p fn ,t fn}}, will t f1 t f2 ... t fn From Z f Z in each subset of the dataset f1 Z f2 ... Z fn Extract and obtain dataset Z. f {Z f1 {a f1 ,s f1 ,d f1 ,p f1},Z f2 {a f2 ,s f2 ,d f2 ,p f2},……,Z fn {a fn ,s fn ,d fn ,p fn}}.

[0066] The aforementioned sensitive data privacy protection specifically includes: Dataset Z was encrypted using data encryption algorithm A2. f {Z f1 {a f1 ,s f1 ,d f1 ,p f1},Z f2 {a f2 ,s f2 ,d f2 ,p f2},……,Z fn {a fn ,s fn ,d fn ,p fn Plaintext sensitive data a in}} f1 af2 ,..., a fn Sensitive data is encrypted to protect the privacy of sensitive data.

[0067] Algorithm A2 is as follows, for data set Z f {Z f1 {a f1 ,s f1 ,d f1 ,p f1}, Z f2 {a f2 ,s f2 ,d f2 ,p f2},..., Z fn {a fn ,s fn ,d fn ,p fn}} The sensitive data a f1 , a f2 ,..., a fn and the sensitive data corresponding to the sensitive data of the data source p f1 , p f2 ,..., p fn There are five encryption algorithms corresponding to the type of data source to which the sensitive data belongs: database data source encryption algorithm f x1 , file system data source encryption algorithm f x2 , web api http stream data source encryption algorithm f x3 , real-time data stream data source encryption algorithm f x4 , message queue data source encryption algorithm f x5 , and the public key pub x1 , pub x2 , pub x3 , pub x4 , pub x5 , and the five decryption algorithms corresponding to the type of data source to which the data belongs: database data source encrypted data decryption algorithm D x1 , file system data source encrypted data decryption algorithm D x2 , web api http stream data source encrypted data decryption algorithm D x3 , real-time data stream data source encrypted data decryption algorithm D x4 , message queue data source encrypted data decryption algorithm D x5 , five decryption algorithms D x1 , D x2 , D x3 , D x4 , D x5 corresponding private key pri x1 , prix2 , pri x3 , pri x4 , pri x5 , by five encryption algorithms f x1 , f x2 , f x3 , f x4 , f x5 and the public key pub x1 , pub x2 , pub x3 , pub x4 , pub x5 According to the data source p f1 , p f2 , ……, p fn , the data a f1 , a f2 , ……, a fn is encrypted, and the encryption algorithm f x1 , f x2 , f x3 , f x4 , f x5 will ensure that the encryption results of the same data under different data sources are still the same, that is, in the sensitive data collected by different data source terminals, if the sensitive data a1 collected by a certain database data source is equal to the sensitive data a1 collected by a certain file system, then the encryption results of the sensitive data a1 of the database data source and the sensitive data a1 of the file system are still the same, and the sensitive data a f1 , a f2 , ……, a fn is encrypted by the encryption algorithm f x1 , f x2 , f x3 , f x4 , f x5 The encrypted sensitive data a fx1 , a fx2 , ……, a fxn , replace the data set Z fx1 , a fx2 , ……, a fxn , Z f {Z f1 {a f1 , s f1 , d f1 , p f1}, Z f2 {a f2 , s f2 , d f2 , p f2}, ……, Z fn {a fn , sfn ,d fn ,p fn}} corresponding to the plaintext sensitive data in the above equation (1) to obtain a data set Z f {Z f1 {a fx1 ,s f1 ,d f1 ,p f1},Z f2 {a fx2 ,s f2 ,d f2 ,p f2},……,Z fn {a fxn ,s fn ,d fn ,p fn}}.

[0068] The federated learning algorithm training, specifically comprising: In the system X n , each data source encrypts sensitive data exchange, and trains the model, which is implemented as follows: there is a federated learning model server FL server , the federated learning model server FL server issues an initial learning model LM n to each data source of the system X c , after issuing the initial learning model LM c , let the encrypted sensitive data a fx1 , a fx2 , …, a fxn in each data source exchange sensitive data between each data source, and start federated learning training, after training, each data source sends the encrypted result result encrypt obtained after training to the federated learning model server FL server , the federated learning model server FL server integrates the encrypted results of each data source into an encrypted sensitive data flow data set DF encrypt {DF encrypt1 , DF encrypt2 , DF encrypt3 , …, DF encryptn}, then the federated learning model server FL server sends the encrypted sensitive data flow data set DF encrypt {DF encrypt1 , DF encrypt2 , DF encrypt3 , …, DF encryptn} to the analysis server FX server , the analysis server FX server decrypts the algorithm Dx1 D x2 D x3 D x4 D x5 and the corresponding private key pri x1 pri x2 pri x3 pri x4 pri x5 DF, a dataset of encrypted sensitive data streams. encrypt {DF encrypt1 ,DF encrypt2 ,DF encrypt3 ,……,DF encryptn The encrypted data in} is decrypted to obtain the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF}. n}, after obtaining the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF n After that, the federated learning results are analyzed, specifically as follows: There exists a known accurate data flow dataset DL true {DL true1 DL true2 DL true3 ,……,DL truen}, Analysis Server FX server The sensitive data flow dataset DF{DF1,DF2,DF3,……,DF n The sensitive data flow data DF1, DF2, DF3, ..., DF in} n Compare with known accurate data flow datasets DL true DL of various data true1 DL true2 DL true3 ...DL truen Find the intersection, i.e., calculate DF n ∩DL truen ,like , and DF n ∩DL truen Length percentage DL truen The total length is half or more, i.e., len(DF) n ∩DL truen ) / len(DL truen If )≥1 / 2, then the sensitive data flow data DF is considered. n If the accuracy is met, then len(DF) n ∩DL truen ) / len(DL truen If ) < 1 / 2, then the sensitive data flow data DF is considered. nDoes not meet accuracy, if sensitive data flow data DF n Meets accuracy, then the analysis server FX server Will set the accuracy value T n of the sensitive data flow data DF DFn to 1, if the sensitive data flow data DF n Does not meet accuracy, then the analysis server FX server Will set the accuracy value T n of the sensitive data flow data DF DFn to 0. After the analysis server FX server Calculates the accuracy values T n , T DF1 , T DF2 , …, T DF3 of DF1, DF2, DF3, …, DF DFn , the analysis server FX n Calculates the average value of the accuracy values T DF1 , T DF2 , T DF3 , …, T DFn of DF1, DF2, DF3, …, DF DF1 , that is, calculates ∑(T DF2 , T DF3 , T DFn , …, T DF1 ) / n, if the average value of the accuracy values ∑(T DF2 , T DF3 , T DFn , …, T z ) / n is greater than or equal to T z (T z is 0.8 by default and can be set by the user, 0.5<T n <1), then the sensitive data flow data set DF {DF1, DF2, DF3, …, DF DF1} is the data flow accuracy result, if the average value of the accuracy values ∑(T DF2 , T DF3 , T DFn , …, T z ) / n is less than T server , the analysis server FX encrypt Will send the overall accuracy result of the encrypted sensitive data flow data set DF encrypt1 {DF encrypt2 , DF encrypt3 , DF encryptn , …, DF server} that does not meet accuracy to the federal learning model server FL server , the federal learning model server FL c Will retrain the initial learning model LMAdjustments are made to obtain the learning model LM c1 And repeat federated learning algorithm A3 and accuracy judgment algorithm A4, if the average of the calculated accuracy values ​​∑(T) DF1 T DF2 T DF3 , ..., T DFn ) / n is still less than T z Federated learning model server FL server The learning model LM will be tested again. c1 Obtain the learning model LM c2 The federated learning algorithm A3 and the accuracy judgment algorithm A4 are repeated again until N rounds are completed, and then the sensitive data a is encrypted. fx1 a fx2 ... a fxn Learning Model LM cn The average accuracy value obtained from federated learning training ∑(T) DF1 T DF2 T DF3 , ..., T DFn ) / n is greater than or equal to T z At this point, the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF} obtained after N rounds of iterations is... n} represents the accurate results of data flow.

[0069] The visualization of the data flow situation specifically includes: visualizing the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF n The situation visualization is performed as follows: Algorithm A5 first processes the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF}. n The sensitive data flow data DF1, DF2, DF3, ..., DF in} n Extract data nodes and deduplicate the extracted data nodes to form a data node dataset TD{TD1,TD2,TD3,……,TD}. n Then, algorithm A5 will use the data node dataset TD{TD1,TD2,TD3,……,TD} n The data nodes TD1, TD2, TD3, ..., TD in} n Generate the corresponding graph database node TD t1 TD t2 TD t3 , ..., TD tn Then, the graph database node TD t1 TD t2 TD t3 , ..., TD tnSensitive data flow data DF1, DF2, DF3, …, DF n The data flow relationship in the sensitive data flow data DF1, DF2, DF3, …, DF

[0070] Compared with the prior art, the privacy data flow situation awareness system based on federated learning provided by the application can collect and identify sensitive data at a data source terminal, encrypt the sensitive data, issue an initial federated learning model to the data source terminal and perform training, analyze the training result after training, adjust the federated learning model and reissue it for federated learning training if the training result does not meet the accuracy, and visualize the final training result obtained if the training result meets the accuracy to obtain a directed graph as shown in Figure 7 .

[0071] Taking a specific application scenario as an example, there are 2 database data source terminals Da1 and Da2, 2 file system data source terminals Fi1 and Fi2, 2 web api http stream data source terminals API1 and API2, 2 real-time data stream data source terminals Ds1 and Ds2, and 2 message queue data source terminals Qu1 and Qu2 in a system X1, and 10 data source terminals in the system X1 collect relevant data in real time. The database data source terminal Da1 obtains a data set DaZ1{Z1, Z2, Z3, …, Z 200} after collecting relevant data, the database data source terminal Da2 obtains a data set DaZ2{Z1, Z2, Z3, …, Z 300} after collecting relevant data, the file system data source terminal Fi1 obtains a data set FiZ1{Z1, Z2, Z3, …, Z 100} after collecting relevant data, the file system data source terminal Fi2 obtains a data set FiZ2{Z1, Z2, Z3, …, Z 100} after collecting relevant data, the web api http stream data source terminal API1 obtains a data set APIZ1{Z1, Z2, Z3, …, Z 50} after collecting relevant data, the web api http stream data source terminal API2 obtains a data set APIZ2{Z1, Z2, Z3, …, Z 50} after collecting relevant data, the real-time data stream data source terminal Ds1 obtains a data set DsZ1{Z1, Z2, Z3, …, Z 50} after collecting relevant data, the real-time data stream data source terminal Ds2 obtains a data set DsZ2{Z1, Z2, Z3, …, Z 50} after collecting relevant data, and the message queue data source terminal Qu1 obtains a data set QuZ1{Z1, Z2, Z3, …, Z50},message queue data source terminal Qu2 collects relevant data to obtain data set QuZ2{Z1, Z2, Z3, …, Z 50} after initial sensitive data identification.

[0072] In step S2, the known sensitive data classification and grading list L M is first issued to each data source terminal for initial sensitive data identification. In this embodiment, database data source terminal Dal data set DalZ1{Z1, Z2, Z3, …, Z 200} after initial sensitive data identification, obtains data set DalZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c100}, database data source terminal Da2 data set DaZ2{Z1, Z2, Z3, …, Z 300} after initial sensitive data identification, obtains data set DaZ c2 {Z c1 ,Z c2 ,Z c3 ,……,Z c150}, file system data source terminal Fi1 data set FiZ1{Z1, Z2, Z3, …, Z 100} after initial sensitive data identification, obtains data set FiZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c50}, file system data source terminal Fi2 data set FiZ2{Z1, Z2, Z3, …, Z 100} after initial sensitive data identification, obtains data set FiZ c2 {Z c1 ,Z c2 ,Z c3 ,……,Z c50}, web api http stream data source terminal API1 data set APIZ1{Z1, Z2, Z3, …, Z 50} after initial sensitive data identification, obtains data set APIZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c40}, web api http stream data source terminal API2 data set APIZ2{Z1, Z2, Z3, …, Z 50} after initial sensitive data identification, obtains data set APIZ c2 {Z c1 ,Z c2Z c3 ,……,Z c30}, Real-time data stream data source terminal Ds1 dataset DsZ1{Z1,Z2,Z3,……,Z 50 The dataset DsZ was obtained after initial sensitive data identification. c1 {Z c1 Z c2 Z c3 ,……,Z c30}, Real-time data stream data source terminal Ds2 dataset DsZ2{Z1,Z2,Z3,……,Z 50 The dataset DsZ was obtained after initial sensitive data identification. c2 {Z c1 Z c2 Z c3 ,……,Z c50}, message queue data source terminal Qu1 dataset QuZ1{Z1,Z2,Z3,……,Z 50 The dataset QuZ was obtained after initial sensitive data identification. c1 {Z c1 Z c2 Z c3 ,……,Z c30}, message queue data source terminal Qu2 dataset QuZ2{Z1,Z2,Z3,……,Z 50 The dataset QuZ was obtained after initial sensitive data identification. c2 {Z c1 Z c2 Z c3 ,……,Z c40 In this embodiment, the database data source terminal is Da1, and the dataset is DaZ. c1 {Z c1 Z c2 Z c3 ,……,Z c100 The dataset DaZ is obtained after calculation using algorithm A1. f1 {Z f1 Z f2 Z f3 ,……,Z f110}, Database data source terminal Da2 dataset DaZ f2 {Z f1 Z f2 Z f3 ,……,Z f150 The dataset DaZ is obtained after calculation using algorithm A1. f2 {Z f1 Z f2 Z f3 ,……,Z f155}, file system data source terminal Fi1 dataset FiZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c50} after algorithm A1 calculation dataset FiZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f50}, file system data source terminal Fi2 dataset FiZ c2 {Z c1 ,Z c2 ,Z c3 ,……,Z c50} after algorithm A1 calculation dataset FiZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f50}, web api http stream data source terminal API1 dataset APIZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c40} after algorithm A1 calculation dataset APIZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f50}, web api http stream data source terminal API2 dataset APIZ c2 {Z c1 ,Z c2 ,Z c3 ,……,Z c30} after algorithm A1 calculation dataset APIZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f50}, real-time data stream data source terminal Ds1 dataset DsZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c30} after algorithm A1 calculation dataset DsZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f30}, real-time data stream data source terminal Ds2 dataset DsZc2 {Z c1 ,Z c2 ,Z c3 ,……,Z c50} after algorithm A1 calculation to obtain data set DsZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f50} after algorithm A1 calculation to obtain data set QuZ c1 {Z c1 ,Z c2 ,Z c3 ,……,Z c30} after algorithm A1 calculation to obtain data set QuZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f30} after algorithm A1 calculation to obtain data set QuZ c2 {Z c1 ,Z c2 ,Z c3 ,……,Z c40} after algorithm A1 calculation to obtain data set QuZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f40}.

[0073] In step S3, the plaintext sensitive data in the data set in each data source terminal in step S2 is encrypted by the data encryption algorithm A2. In this embodiment, the database data source encryption algorithm f x1 and the corresponding public key pub x1 are issued to the database data source terminal Da1, Da2 for sensitive data encryption, the file system data source encryption algorithm f x2 and the corresponding public key pub x2 are issued to the file system data source terminal Fi1, Fi2 for sensitive data encryption, the web api http stream data source encryption algorithm f x3 and the corresponding public key pub x3 are issued to the web api http stream data source terminal API1, API2 for sensitive data encryption, the real-time data stream data source encryption algorithm f x4 and the corresponding public key pub x4 are issued to the real-time data stream data source terminal Ds1, Ds2 for sensitive data encryption, and the message queue data source encryption algorithm f x5 and the corresponding public key pubx5 The sensitive data is encrypted and sent to the message queue data source terminal Qu1, Qu2. Then the algorithm A2 replaces the plaintext sensitive data in the data set of each data source terminal with the encrypted ciphertext data. In this embodiment, DaZ f1 The data calculated by algorithm A2 to DaZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f110 {a fx110 ,s f110 ,d f110 ,p f110}},DaZ f2 The data calculated by algorithm A2 to DaZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f155 {a fx155 ,s f155 ,d f155 ,p f155}},FiZ f1 The data calculated by algorithm A2 to FiZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f50 {a fx50 ,s f50 ,d f50 ,p f50}},FiZ f2 The data calculated by algorithm A2 to FiZ f2 {Z f1 ,Z f2 ,Z f3 ,……,Z f50 {a fx50 ,s f50 ,d f50 ,p f50}},APIZ f1 The data calculated by algorithm A2 to APIZ f1 {Z f1 ,Z f2 ,Z f3 ,……,Z f50 {a fx50 ,s f50 ,d f50 ,p f50}},APIZ f2 The data calculated by algorithm A2 to APIZ f2 {Z f1 ,Z f2 ,Zf3 ,……,Z f50 {a fx50 ,s f50 ,d f50 ,p f50}},DsZ f1 DsZ is calculated using algorithm A2. f1 {Z f1 Z f2 Z f3 ,……,Z f30 {a fx30 ,s f30 ,d f30 ,p f30}},DsZ f2 DsZ is calculated using algorithm A2. f2 {Z f1 Z f2 Z f3 ,……,Z f50 {a fx50 ,s f50 ,d f50 ,p f50}}, QuZ f1 QuZ is calculated using algorithm A2. f1 {Z f1 Z f2 Z f3 ,……,Z f30 {a fx30 ,s f30 ,d f30 ,p f30}}, QuZ f2 QuZ is calculated using algorithm A2. f2 {Z f1 Z f2 Z f3 ,……,Z f40 {a fx40 ,s f40 ,d f40 ,p f40}}.

[0074] In step S4, the model is trained using the federated learning algorithm A3 during the encrypted exchange of sensitive data from various data sources in system X1. After the first model training, the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF} is obtained. 50}, for DF{DF1,DF2,DF3,……,DF 50 DF1, DF2, DF3, ..., DF in} 50 The accuracy values ​​were calculated as follows: DF2, DF3, DF5, DF8, DF 19 DF21 DF 33 DF 35 DF 37 DF 42 DF 46 DF 49 The accuracy value is 0, and all others are 1, DF1, DF2, DF3, ..., DF 50 The accuracy value is 0.76. In this embodiment, T z The value is 0.8. Since 0.76 < 0.8, the federated learning model server FL... server This will affect the initial learning model LM c Adjustments are made to obtain the learning model LM c1 And the learning model LM was distributed again. c1 Model training is performed at the data source terminal. After the tenth model training, the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF} is obtained. 50}, again for DF{DF1,DF2,DF3,……,DF 50 DF1, DF2, DF3, ..., DF in} 50 Calculation accuracy values: DF7, DF 36 DF 45 The accuracy value is 0, and all others are 1, DF1, DF2, DF3, ..., DF 50 The accuracy value is 0.94, since 0.94 > T. z Therefore, the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF 50} represents the accurate results of data flow.

[0075] In step S5, the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF} obtained in step S4 is visualized using the data flow situation visualization algorithm A5. 50 For visualization, algorithm A5 first visualizes the sensitive data flow dataset DF{DF1,DF2,DF3,……,DF}. 50 The sensitive data flow data DF1, DF2, DF3, ..., DF in} 50 Data nodes are extracted and deduplicated, and then integrated. In this embodiment, the extracted, deduplicated, and integrated data node dataset TD{TD1,TD2,TD3,……,TD} is obtained. 100 Algorithm A5 uses the data node dataset TD{TD1,TD2,TD3,……,TD}. 100 The data nodes TD1, TD2, TD3, ..., TD in} 50 The corresponding graph database node TD was generated. t1 TDt2 , TD t3 , …, TD t50 , and then the graph database node TD t1 , TD t2 , TD t3 , …, TD t50 The data flow relationship in the sensitive data flow data DF1, DF2, DF3, …, DF 50 is generated to form an ordered edge connection to form a data flow graph TX for situation visualization.

[0076] Embodiment 3: As Figure 8 shown, it is an architecture schematic diagram of the privacy data flow situation awareness device of federated learning of the embodiment of the application. The privacy data flow situation awareness device of federated learning of the embodiment includes one or more processors 21 and a memory 22. Wherein, Figure 8 In the embodiment, the processor 21 is taken as an example.

[0077] The processor 21 and the memory 22 can be connected through a bus or other ways, Figure 8 In the embodiment, the connection through the bus is taken as an example.

[0078] The memory 22 is a kind of nonvolatile computer readable storage medium, can be used to store nonvolatile software program and nonvolatile computer executable program, such as the privacy data flow situation awareness method of federated learning in embodiment 1. The processor 21 executes the privacy data flow situation awareness method of federated learning by running the nonvolatile software program and instruction stored in the memory 22.

[0079] The memory 22 can include high-speed random access memory, and can also include nonvolatile memory, for example at least one magnetic disk storage device, flash memory device or other nonvolatile solid-state storage device. In some embodiments, the memory 22 can optionally include a memory remotely arranged relative to the processor 21, which can be connected to the processor 21 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0080] The program instruction / module is stored in the memory 22, and when executed by the one or more processors 21, the privacy data flow situation awareness method of federated learning in the above-mentioned embodiment 1 is executed.

[0081] It is worth noting that the information interaction, execution process and the like between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiments of the application, and the specific content can be referred to the description in the method embodiments of the application, which will not be described here.

[0082] Those skilled in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing the related hardware with a program, and the program can be stored in a computer readable storage medium, which can include Read Only Memory (ROM), Random Access Memory (RAM), a magnetic disk or an optical disk, etc.

[0083] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A privacy data flow dynamic trend perception method based on federated learning, characterized in that, The method comprises the following steps: Each data source terminal collects its own data set, identifies sensitive data in the data set using a preset strategy, desensitizes the sensitive data in the data set using a preset encryption algorithm, and obtains a training data set; The federated learning server and each data source terminal perform federated learning using each training data set to obtain a data flow trend model for data flow trend analysis.

2. The federated learning based privacy data flow dynamic trend perception method according to claim 1, characterized in that, The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps:

3. The federated learning based privacy data flow dynamic trend perception method according to claim 1, characterized in that, The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps:

4. The federated learning based privacy data flow dynamic trend perception method according to claim 1, characterized in that, The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps:

5. The federated learning based privacy data flow dynamic trend perception method according to claim 4, characterized in that, The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps:

6. The federated learning based privacy data flow dynamic trend perception method according to claim 5, characterized in that, The method further comprises the following steps: The method further comprises the following steps:

7. The federated learning based privacy data flow dynamic trend perception method according to claim 5, characterized in that, The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further comprises the following steps: The method further The analysis server further extracts a data node from the sensitive data flow trend set, and generates a directed graph according to a data flow relationship of the data node.

8. The federated learning based privacy data flow dynamic trend perception method according to any one of claims 1-7, characterized in that, The data set of each data source terminal includes one or more of source data, a source data producer, a source data consumer, a data source to which the source data belongs, a destination data source of the source data, and derivative data of the source data.

9. A privacy data flow dynamic trend perception device based on federated learning, characterized in that, Comprise; At least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, the instructions executed by the processor for executing the privacy data flow trend perception method based on federated learning of any one of claims 1-8.

10. A non-transitory computer storage medium, comprising, The computer storage medium stores computer executable instructions executed by one or more processors for completing the privacy data flow trend perception method based on federated learning of any one of claims 1-8.

Images