A multi-source heterogeneous data fusion method based on privacy computing

Abstract

The present invention discloses a multi-source heterogeneous data fusion method based on privacy computing, which belongs to the technical field of multi-source heterogeneous data fusion. The method comprises: identifying each data source; determining the data collection scope corresponding to each data source; performing representative evaluation on each data source according to each data collection scope, and obtaining a representative value corresponding to each data source; obtaining simulated collection data of each data source, and marking a corresponding representative value for each simulated collection data; verifying and adjusting a preset data fusion scheme according to each simulated collection data; obtaining a target fusion scheme; setting a corresponding privacy protection module according to the target fusion scheme, the privacy protection module being established based on privacy computing, and being used for performing privacy protection in the process of fusing multi-source heterogeneous data; obtaining the collection data of each data source in real time, performing data fusion on each collection data according to the target fusion scheme, and performing privacy protection through the privacy protection module.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-source heterogeneous data fusion, and specifically is a multi-source heterogeneous data fusion method based on privacy computing. Background Art

[0002] With the rapid development of information technology and the widespread popularity of Internet applications, the collection, storage, analysis and utilization of data have become an indispensable part of modern society. Especially in the fields of medical health, finance and insurance, smart cities, etc., a large amount of multi-source heterogeneous data is constantly generated and accumulated. These data not only contain valuable information, but also involve personal privacy and sensitive information.

[0003] In the processing of multi-source heterogeneous data, data fusion technology is an important means, which can fuse data from different sources, formats or types to extract valuable information and knowledge. However, since the data involves personal privacy and sensitive information, how to achieve efficient fusion and utilization of data while protecting data privacy has become an urgent problem to be solved.

[0004] Based on this, the present invention provides a multi-source heterogeneous data fusion method based on privacy computing. Summary of the invention

[0005] In order to solve the problems existing in the above solutions, the present invention provides a multi-source heterogeneous data fusion method based on privacy computing.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A multi-source heterogeneous data fusion method based on privacy computing, the method comprising:

[0008] Step 1: Identify each data source; determine the data collection scope corresponding to each data source; perform representative evaluation on each data source according to each data collection scope to obtain a representative value corresponding to each data source;

[0009] Furthermore, the method of evaluating the representativeness of each data source according to each data collection scope includes:

[0010] Acquire material data corresponding to the corresponding data source according to the data collection scope, wherein the material data is composed of analysis data of each unit, and the unit analysis data includes historical collection data and integrity and authenticity judgment results corresponding to the historical collection data;

[0011] Conduct authenticity analysis on the material data to obtain the corresponding true value;

[0012] Identify the completeness corresponding to each unit analysis data in the material data, calculate the average value of each completeness, and mark the calculated average value of the completeness as a complete analysis value;

[0013] Calculate the corresponding representative value according to the formula DB = b1×ZH + b2×WZ;

[0014] In the formula: DB is the representative value; b1 and b2 are both proportionality coefficients, and the value range is 0 < b1 ≤ 1, 0 < b2 ≤ 1; ZH is the true value; WZ is the complete analysis value.

[0015] Furthermore, the method for analyzing the authenticity of material data includes:

[0016] Establish a unit evaluation model, and evaluate each unit analysis data in the material data through the unit evaluation model to obtain the unit value corresponding to each unit analysis data;

[0017] Set a unit time period, and merge each unit analysis data according to the unit time period to obtain each unit merge set;

[0018] Identify the unit value corresponding to each unit analysis data in the unit merge set;

[0019] According to the formula Calculate the unit time period value corresponding to each unit time period;

[0020] In the formula: DH is the unit time period value; i represents the unit analysis data in the unit merge set corresponding to the unit time period, i = 1, 2,..., n, and n is a positive integer;

[0021] Generate an initial analysis curve from the unit time period values corresponding to each unit time period; fit the initial analysis curve to obtain an initial curve function; mark the initial curve function as CL(t);

[0022] According to the formula Transform the initial curve function;

[0023] In the formula: DZt is the corresponding curve transformation value;

[0024] Generate a true analysis curve corresponding to the corresponding data source from each curve transformation value;

[0025] Identify the unit time period value corresponding to the current time in the initial analysis curve, and mark the obtained unit time period value as the reference value;

[0026] Determine the corresponding correction coefficient according to the reference value, the initial analysis curve, and the true analysis curve,

[0027] Calculate the corresponding true value according to the formula ZH = η×DH0; in the formula: ZH is the true value; η is the correction coefficient; DH0 is the reference value.

[0028] Furthermore, the expression of the unit evaluation model is:

[0029]

[0030] Where: s is the unit analysis data; the output data is the unit value 1 or 0.

[0031] Furthermore, the method for determining the correction coefficient includes:

[0032] According to the benchmark value, a corresponding benchmark curve segment is marked in the initial analysis curve; similarity matching of the benchmark curve segments is performed in the initial analysis curve to obtain each reference segment, and a first reference point corresponding to the benchmark value in each reference segment is identified; a second reference point corresponding to each first reference point in the real analysis curve is identified; a slope of each second reference point in the real analysis curve is identified and marked as a reference slope, an average value of each reference slope is calculated and marked as a representative slope, and a corresponding correction coefficient is set according to the representative slope.

[0033] Step 2: Perform data simulation collection from the corresponding data source according to each data collection range, obtain the simulated collection data corresponding to each data source, and mark the corresponding representative value for each simulated collection data; verify and adjust the preset data fusion scheme according to each simulated collection data; mark the verified data fusion scheme as the target fusion scheme;

[0034] Step 3: Set a corresponding privacy protection module according to the target fusion scheme. The privacy protection module is established based on privacy computing and is used to perform privacy protection in the process of fusion of multi-source heterogeneous data;

[0035] Furthermore, the setting method of the privacy protection module includes:

[0036] Establishing an encryption library, wherein the encryption library is used to store various encryption schemes and corresponding feature ranges;

[0037] Set each encryption feature according to the target fusion scheme, input each encryption feature into the encryption library for feature matching, and mark each matching encryption scheme as a candidate scheme;

[0038] Set up the test end; associate the test end with the corresponding encryption scheme;

[0039] Acquire each simulated collection data, perform desensitization adjustment on the simulated collection data, and obtain corresponding test analysis data;

[0040] Match the corresponding test end according to each candidate scheme, test the test analysis data through each test end to obtain the corresponding test effect data, determine the corresponding target encryption scheme according to each test effect data, and set the privacy protection module according to the target encryption scheme.

[0041] Furthermore, the test end setting method includes:

[0042] Determine each encryption channel, count the test duration of each encryption channel, and start timing from the sending of test analysis data for the exam; conduct a security assessment on each encryption channel to obtain the corresponding security value for each encryption channel;

[0043] Calculate the corresponding total test duration and total security value based on the test duration and security value of each encryption channel;

[0044] According to the formula Calculate the corresponding channel value for each encryption channel;

[0045] In the formula: PW is the channel value; b3 and b4 are both proportionality coefficients, and the value range is 0 < b3 ≤ 1, 0 < b4 ≤ 1, CW is the test duration of the corresponding encryption channel; CWZ is the total test duration; AQ is the security value of the corresponding encryption channel; AQZ is the total security value;

[0046] Determine the target channel corresponding to each encryption scheme according to the channel value of each encryption channel, and establish a corresponding test end based on the target channel.

[0047] Step Four: Real-time obtain the acquisition data of each data source, perform data fusion on each acquisition data according to the target fusion scheme, and perform privacy protection through the privacy protection module.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] By adopting advanced privacy computing technology, the present invention can ensure that no sensitive information is leaked during the data fusion process, effectively protecting personal privacy and data security; designing an efficient data fusion algorithm and data processing flow can significantly improve the efficiency and accuracy of data fusion on the premise of ensuring data privacy. This helps to extract more valuable information and knowledge from multi-source data; by having the platform party set the corresponding test end according to each encryption scheme in the encryption library; associating each test end with the corresponding encryption scheme; improving the accuracy of evaluating test analysis data, avoiding the situation of low accuracy and large errors caused by direct evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0051] Figure 1 It is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative work belong to the scope of protection of the present invention.

[0053] As Figure 1 shown, a multi-source heterogeneous data fusion method based on privacy computing includes:

[0054] Step 1: Identify each data source that needs to perform multi-source heterogeneous data fusion, and mark the corresponding data source as a data source; determine the data collection scope corresponding to each data source, that is, which types of data need to be collected from this data source, and integrate it into the data collection scope; perform a representative evaluation on each data source according to each data collection scope, and obtain the representative value corresponding to each data source;

[0055] The method for performing a representative evaluation on each data source according to each data collection scope includes:

[0056] Obtain the material data corresponding to the corresponding data source according to the data collection scope. The material data is composed of unit analysis data, and the unit analysis data includes historical collection data and the judgment results of the integrity and authenticity corresponding to the historical collection data; that is, analyze according to the historical collection data that has been generated and analyzed and applied, and determine the corresponding data integrity and the judgment result of whether the data is true; one unit analysis data corresponds to one historical collection data and the corresponding integrity and authenticity judgment results.

[0057] Perform a authenticity analysis on the material data to obtain the corresponding true value;

[0058] Identify the integrity corresponding to each unit analysis data in the material data, calculate the average value of each integrity, and mark the calculated average value of integrity as the integrity analysis value;

[0059] Calculate the corresponding representative value according to the formula DB = b1×ZH + b2×WZ;

[0060] In the formula: DB is the representative value; b1 and b2 are both proportionality coefficients, and the value range is 0 < b1 ≤ 1, 0 < b2 ≤ 1; ZH is the true value; WZ is the integrity analysis value.

[0061] The method for performing a authenticity analysis on the material data includes:

[0062] Establish a unit evaluation model for evaluating according to the authenticity judgment result in the unit analysis data. When the authenticity judgment result is that the data is not true, it is regarded that the unit analysis data does not meet the authenticity requirements. Otherwise, it is regarded as meeting the authenticity requirements. The expression is Where: s is the input data, i.e. the unit analysis data; the output data is the unit value 1 or 0;

[0063] Evaluate each unit analysis data in the material data through the unit evaluation model to obtain the unit value corresponding to each unit analysis data;

[0064] Set a unit period, that is, take a preset period as a data cycle of the data source, and set it according to the periodicity of the data source. Generally, one day is taken as a unit period, that is, 24 hours; merge each unit analysis data according to the unit period and the time corresponding to the unit analysis data to obtain each unit merged set, that is, integrate each unit analysis data belonging to the unit period into a unit merged set;

[0065] Identify the cell values ​​corresponding to the cell analysis data in the cell merge set;

[0066] According to the formula Calculate the unit time period value corresponding to each unit time period;

[0067] Wherein: DH is the unit period value; i represents the unit analysis data in the unit merge set corresponding to the unit period, i = 1, 2, ..., n, n is a positive integer, that is, n is the number of unit analysis data in the unit merge set;

[0068] Sort each unit period in chronological order to form a time series, set the corresponding coordinate horizontal axis according to the time series, such as 1, 2, 3, 4, etc. marked on the horizontal axis correspond to the unit periods corresponding to the corresponding sorting numbers in the time series; use the unit period value as the coordinate vertical axis to establish a corresponding coordinate system, input the unit period value corresponding to each unit period into the coordinate system, connect each coordinate point to form an initial analysis curve; fit the initial analysis curve to obtain the initial curve function; mark the initial curve function as CL(t), where t represents the corresponding horizontal axis variable;

[0069] According to the formula The initial curve function is transformed, where: DZt is the corresponding curve transformation value, that is, the ordinate value after the corresponding coordinate transformation; the real analysis curve corresponding to the corresponding data source is generated according to each curve transformation value;

[0070] Identify the unit period value corresponding to the current time in the initial analysis curve, and mark the obtained unit period value as a reference value;

[0071] Determine the corresponding correction coefficient based on the reference value, initial analysis curve and true analysis curve.

[0072] The corresponding true value is calculated according to the formula ZH=η×DH0; where: ZH is the true value; η is the correction coefficient; DH0 is the reference value.

[0073] Methods for determining the correction factor include:

[0074] First, determine the corresponding position in the initial analysis curve according to the benchmark value, that is, the position corresponding to the current time; identify the initial analysis curve segment within a period of time including the benchmark value, such as 5 days, 10 days, etc., and mark it as the benchmark curve segment;

[0075] Perform similarity matching of the benchmark curve segments in the initial analysis curve, mark the curve segments with similarity greater than the threshold value X1 as reference segments, identify the position of the corresponding benchmark value in the reference segment, and mark it as the first reference point, that is, the comparative correspondence of the curve segments; identify the position of each first reference point in the real analysis curve, and mark it as the second reference point; identify the slope of each second reference point in the real analysis curve, and mark it as the reference slope, calculate the average value of each reference slope, and mark it as the representative slope, and set the corresponding correction coefficient according to the representative slope.

[0076] Step 2: Perform data simulation collection from the corresponding data sources according to the data collection scope to obtain the corresponding simulated collection data, that is, collect data according to the subsequent real application process; identify the representative value corresponding to each data source, and mark the representative value in the corresponding simulated collection data; set up the data fusion scheme according to the representative value corresponding to each simulated collection data, that is, use the representative value corresponding to each simulated collection data as a reference benchmark to fuse the simulated collection data, so as to quickly determine the representativeness of each simulated collection data and improve the fusion efficiency; set up the data fusion scheme using the existing multi-source heterogeneous data fusion method; and use the corresponding simulated collection data for continuous verification and adjustment to meet the use requirements;

[0077] Step 3: Set a corresponding privacy protection module according to the target fusion scheme. The privacy protection module is established based on privacy computing and is used to perform privacy protection during the fusion process of multi-source heterogeneous data;

[0078] The setting methods of the privacy protection module include:

[0079] Set various encryption features according to the target fusion plan, including various data features that have an impact on the selection of encryption algorithms, such as data type, protection requirements, and device computing power. This can be set manually or based on existing artificial intelligence technology.

[0080] According to the existing privacy computing cases, various privacy computing methods are determined and marked as encryption schemes; and encryption features suitable for each encryption scheme are set, which can be summarized according to the corresponding historical cases; the feature range corresponding to each encryption scheme is formed; and the corresponding encryption library is established according to each encryption scheme and feature range;

[0081] Input each encryption feature into the encryption library for feature matching, and mark each encryption scheme that meets the feature requirements as a candidate scheme;

[0082] The platform sets up corresponding test terminals according to each encryption scheme in the encryption library; associates each test terminal with the corresponding encryption scheme; because in the actual application process, in order to analyze the application of each encryption scheme in actual situations, the corresponding encryption algorithm will not be specially built at the user's site for testing, so it can only be estimated according to the existing method, but the estimated results may have large errors due to the influence of various factors, and it is impossible to obtain the corresponding real evaluation results;

[0083] Acquire each simulated collection data, perform desensitization adjustment on the simulated collection data, and obtain corresponding test analysis data, that is, remove or replace the user's privacy data that may be included in the simulated collection data, so that the test analysis data can represent the corresponding simulated collection data without leaking user privacy;

[0084] Match the corresponding test end according to each candidate scheme, that is, the matched test end must be able to meet the test requirements, and fuse the test data according to the target fusion scheme. Perform privacy protection in this process and obtain the corresponding privacy protection test effect data; test the test analysis data through each test end to obtain the corresponding test effect data, sort the candidate schemes according to the test effect data, determine the corresponding target encryption scheme, and set the privacy protection module according to the target encryption scheme.

[0085] The test end setting methods include:

[0086] The platform builds the encryption channels by itself, negotiates with users who use the corresponding encryption scheme, and other existing encryption channels, etc., and determines the test duration of each encryption channel based on the historical test records of each encryption channel. The test duration starts from the start of sending the test analysis data until the corresponding duration when the test results are obtained, and the corresponding duration average is used as the test duration; conducts a security assessment on each encryption channel, that is, assesses the possibility of user information leakage based on the test analysis data sent by the user. The higher the possibility, the lower the assessed security value, with the minimum being 0 and the maximum being 100; sets the value based on the proportion of various similar historical leakage data corresponding to the corresponding encryption channel; in actual application, the platform staff will assess the corresponding encryption channel and obtain the corresponding security value;

[0087] According to the test duration and security value of each encrypted channel, the corresponding sum of the test duration and the sum of the security value are calculated to obtain the corresponding total test duration and total security value;

[0088] According to the formula Calculate the channel value corresponding to each encrypted channel;

[0089] Where: PW is the channel value; b3 and b4 are both proportionality coefficients, and the value ranges are 0 < b3 ≤ 1, 0 < b4 ≤ 1. CW is the test duration of the corresponding encrypted channel; CWZ is the total test duration; AQ is the security value of the corresponding encrypted channel; AQZ is the total security value.

[0090] Determine the target channel corresponding to each encryption scheme according to the channel value of each encrypted channel, that is, when there are multiple encrypted channels with the same function in the same encryption scheme, select the one with the highest channel value for association; establish a corresponding test end according to the target channel.

[0091] Step 4: Obtain the collected data of each data source in real time, perform data fusion on each collected data according to the target fusion scheme, and perform privacy protection through the privacy protection module.

[0092] The above formulas are all calculated by removing the dimension and taking their numerical values. The formulas are obtained by software simulation of a large amount of collected data to get a formula closest to the actual situation. The preset parameters and preset thresholds in the formulas are set by those skilled in the art according to the actual situation or obtained by simulation of a large amount of data.

[0093] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A multi-source heterogeneous data fusion method based on privacy computing, characterized in that: The method includes: Step 1: Identify each data source; Determine the data collection scope corresponding to each data source; Conduct a representative evaluation of each data source according to each data collection scope to obtain the representative value corresponding to each data source; Step 2: Conduct data simulation collection from the corresponding data sources according to each data collection scope to obtain the simulated collection data corresponding to each data source, and mark the corresponding representative value for each simulated collection data; Verify and adjust the preset data fusion scheme according to each simulated collection data; Mark the data fusion scheme that passes the verification as the target fusion scheme; Step 3: Set the corresponding privacy protection module according to the target fusion scheme. The privacy protection module is established based on privacy computing and is used for privacy protection during the process of fusing multi-source heterogeneous data; Step 4: Obtain the collection data of each data source in real time, perform data fusion on each collection data according to the target fusion scheme, and perform privacy protection through the privacy protection module.

2. According to the multi-source heterogeneous data fusion method based on privacy computing according to claim 1, it is characterized in that: The method for conducting a representative evaluation of each data source according to each data collection scope includes: Obtain the material data corresponding to the corresponding data source according to the data collection scope. The material data consists of unit analysis data, and the unit analysis data includes historical collection data and the judgment results of the integrity and authenticity corresponding to the historical collection data; Conduct authenticity analysis on the material data to obtain the corresponding true value; Identify the integrity corresponding to each unit analysis data in the material data, calculate the average value of each integrity, and mark the calculated average integrity value as the integrity analysis value; Calculate the corresponding representative value according to the formula DB = b1×ZH + b2×WZ; In the formula: DB is the representative value; b1 and b2 are both proportionality coefficients, and the value range is 0 < b1 ≤ 1, 0 < b2 ≤ 1; ZH is the true value; WZ is the integrity analysis value.

3. According to the multi-source heterogeneous data fusion method based on privacy computing according to claim 2, it is characterized in that: The method for conducting authenticity analysis on the material data includes: Establish a unit evaluation model, evaluate each unit analysis data in the material data through the unit evaluation model to obtain the unit value corresponding to each unit analysis data; Set the unit time period, and merge each unit analysis data according to the unit time period to obtain each unit merge set; Identify the unit value corresponding to each unit analysis data in the unit merge set; According to the formula Calculate the unit time period value corresponding to each unit time period; In the formula: DH is the unit time period value; i represents the unit analysis data in the unit merge set corresponding to the unit time period, i = 1, 2,..., n, and n is a positive integer; Generate an initial analysis curve from the unit time period values corresponding to each unit time period; fit the initial analysis curve to obtain the initial curve function; mark the initial curve function as CL(t); According to the formula Transform the initial curve function; In the formula: DZt is the corresponding curve conversion value; Generate the true analysis curve corresponding to the corresponding data source according to each curve conversion value; Identify the unit time period value corresponding to the current time in the initial analysis curve, and mark the obtained unit time period value as the reference value; Determine the corresponding correction coefficient according to the reference value, the initial analysis curve, and the true analysis curve, Calculate the corresponding true value according to the formula ZH = η×DH0; in the formula: ZH is the true value; η is the correction coefficient; DH0 is the reference value.

4. According to the multi-source heterogeneous data fusion method based on privacy computing according to claim 3, it is characterized in that: The expression of the unit evaluation model is: Where: s is the unit analysis data; the output data is the unit value 1 or 0.

5. According to the multi-source heterogeneous data fusion method based on privacy computing in claim 3, it is characterized in that: The method for determining the correction coefficient includes: Marking the corresponding reference curve segment on the initial analysis curve according to the reference value; performing similarity matching on the reference curve segment in the initial analysis curve to obtain each reference segment, and identifying the first reference point corresponding to the reference value in each reference segment; identifying the second reference point corresponding to each first reference point in the true analysis curve; identifying the slope of each second reference point in the true analysis curve, marking it as the reference slope, calculating the average value of each reference slope, marking it as the representative slope, and setting the corresponding correction coefficient according to the representative slope.

6. According to the privacy computing-based multi-source heterogeneous data fusion method of claim 1, it is characterized by: The method for setting the privacy protection module includes: Establishing an encryption library, which is used to store each encryption scheme and the corresponding feature range; Setting each encryption feature according to the target fusion scheme, inputting each encryption feature into the encryption library for feature matching, and marking each matched encryption scheme as a candidate scheme; Setting a test end; associating the test end with the corresponding encryption scheme; Obtaining each simulated acquisition data, performing desensitization adjustment on the simulated acquisition data to obtain the corresponding test analysis data; Matching the corresponding test end according to each candidate scheme, testing the test analysis data through each test end to obtain the corresponding test effect data, determining the corresponding target encryption scheme according to each test effect data, and setting the privacy protection module according to the target encryption scheme.

7. The multi-source heterogeneous data fusion method based on privacy computing according to claim 6 is characterized in that: The method for setting the test end includes: Determining each encryption channel, counting the test duration of each encryption channel, starting timing from the test analysis data sent in the examination; performing a security assessment on each encryption channel to obtain the security value corresponding to each encryption channel; Calculating the corresponding total test duration and total security value according to the test duration and security value of each encryption channel; According to the formula Calculate the channel value corresponding to each encrypted channel; Where: PW is the channel value; b3 and b4 are both proportionality coefficients, and the value range is 0 < b3 ≤ 1, 0 < b4 ≤ 1, CW is the test duration of the corresponding encryption channel; CWZ is the total test duration; AQ is the security value of the corresponding encryption channel; AQZ is the total security value; Determining the target channel corresponding to each encryption scheme according to the channel value of each encryption channel, and establishing the corresponding test end according to the target channel.

Images