Data fusion verification method, device and computer storage medium

Abstract

The present application provides a data fusion verification method, device and computer storage medium, including fusing at least one first data item in a first data table with at least one second data item in a second data table to obtain a fused data table; obtaining a blood relationship code of the fused data table according to at least one first code corresponding to at least one first data item and at least one second code corresponding to at least one second data item constituting the fused data table; obtaining a verification result of compliance or non-compliance of the fused data table according to the blood relationship code and a preset fusion rule. Therefore, the present application can complete cross-organizational data fusion processing under the premise of ensuring data privacy and security.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular to a data fusion verification method, device and computer storage medium. Background Art

[0002] At present, various departments and enterprises independently build independent information systems according to their own usage needs, resulting in the lack of intercommunication between the information systems of various departments and enterprises and serious data silo problems.

[0003] Furthermore, when it is necessary to perform cross-departmental data joint analysis and processing, the unclear ownership of the data will make it difficult to effectively protect the privacy and security of the data during the coordinated use process.

[0004] Specifically, inherent security issues of data will arise during the data exchange process. For example, multiple independent data that do not involve privacy security may cause privacy leakage problems after data fusion.

[0005] In view of this, there is an urgent need for a data fusion analysis technology that can be used to analyze whether the data fusion processing is compliant. Summary of the invention

[0006] In view of the above problems, the present application provides a data fusion verification method, device and computer storage medium, which can reduce the occurrence of data intrinsic security issues.

[0007] The first aspect of the present application provides a data fusion verification method, which includes: fusing at least one first data item in a first data table with at least one second data item in a second data table to obtain a fused data table; obtaining a blood relationship code of the fused data table based on at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item that constitute the fused data table; and obtaining a verification result of whether the fused data table is compliant or non-compliant based on the blood relationship code and a preset fusion rule.

[0008] A second aspect of the present application provides a computer storage medium, characterized in that the computer storage medium stores instructions for executing the steps in the method described in the first aspect.

[0009] The third aspect of the present application provides a data fusion verification device, which includes: a fusion module, used to fuse at least one first data item in a first data table with at least one second data item in a second data table to obtain a fused data table; an analysis module, used to obtain a blood relationship code of the fused data table based on at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table; a verification module, used to obtain a verification result of whether the fused data table is compliant or non-compliant based on the blood relationship code and a preset fusion rule.

[0010] To sum up, the data fusion verification method provided in the embodiment of the present application can generate data codes for each data item in the fused data table, and generate the blood relationship code of the fused data table based on the data codes, so as to verify whether there are risks in the fusion operation of each data item. Accordingly, the present application reduces the possibility of inherent data security occurring during the data fusion process to improve the privacy security of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1 It is a flowchart of the data fusion verification method of the first embodiment of the present application.

[0013] Figure 2 This is a schematic diagram of an exemplary cross-departmental data distribution architecture for this application.

[0014] Figure 3 This is a flow chart of a data fusion verification method according to the second embodiment of the present application.

[0015] Figure 4 It is a flowchart of the data fusion verification method of the third embodiment of the present application.

[0016] Figure 5 Schematic diagram of the flow of the data fusion verification method according to the fourth embodiment of the present application.

[0017] Figure 6 It is a schematic diagram of the architecture of a data fusion verification device according to the sixth embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0019] The specific embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0020] First embodiment

[0021] Figure 1 The processing flow of the data fusion verification method of the first embodiment of the present application is shown. As shown in the figure, this embodiment mainly includes the following steps:

[0022] Step S102: merge at least one first data item in the first data table with at least one second data item in the second data table to obtain a merged data table.

[0023] In this embodiment, the first data table and the second data table may belong to different departments or systems.

[0024] like Figure 2 As shown, for example, the first data table is data table A or data table B under department A, and the second data table is data table C under department B.

[0025] Among them, each data table may include at least one data item. For example, data table A (first data table) includes data item A and data item B (first data item); data table B (first data table) includes a data item C (first data item); and data table C (second data table) includes a data item D (second data item).

[0026] In this embodiment, each fused data item may be determined according to at least one first data item and at least one second data item, and a fused data table may be generated based on each fused data item.

[0027] For example, if data item A, data item B, and data item D need to be fused, it can be determined that the fused data item includes data item A, data item B, and data item D, and a fused data table D containing data item A, data item B, and data item D can be generated accordingly.

[0028] Optionally, individual or all data items (ie, first data items, second data items) in a data table (ie, first data table, second data table) may be selected for data fusion processing.

[0029] For example, all data items in data table A (ie, data item A and data item B) may be merged with data item D in data table C, or only data item A in data table A and data item D in data table C may be selected for merging.

[0030] Step S104, obtaining the blood relationship code of the fused data table according to at least one first code corresponding to at least one first data item and at least one second code corresponding to at least one second data item constituting the fused data table.

[0031] Optionally, the first encoding and the second encoding each include a shared identification code and a one-hot code.

[0032] Optionally, a shared identification code of each of the first code and the second code may be generated according to the shared attributes of each of the first data item and the second data item.

[0033] Optionally, the sharing attributes of the first encoding and the second encoding may include one of a non-shareable attribute, a conditional sharing attribute, and an unconditional sharing attribute.

[0034] In this embodiment, the sharing identification code may be composed of two digits, where 11 indicates a non-shareable attribute, 01 indicates conditional sharing, and 00 indicates unconditional sharing.

[0035] For example, Figure 2 The shared identification codes of data item A, data item B, and data item C are all 01, which means that these three data items are conditionally shared data. That is, after the data fusion processing is performed, these three data items may have inherent security issues, and fusion verification is required to verify whether the fusion processing is safe.

[0036] Another example: Figure 2 The sharing identification code of the data item D in is 11, which means that this data item is non-shareable data, that is, under any circumstances, the data item D cannot be used as a data fusion item to perform data fusion processing.

[0037] Optionally, a one-hot code of each of the first code and the second code is generated according to the field identification information of each of the first data item and the second data item.

[0038] Specifically, the total amount of field identification information can be counted based on the field identification information of each of the first data item and the second data item to determine the number of encoding bits of the one-hot code, and the one-hot codes of the first data item and the second data item can be generated based on the number of encoding bits of the one-hot code.

[0039] For example, in Figure 2In the schematic diagram shown, according to the field identification information of data items A to D, there are a total of 4 field identification information, thereby determining the number of encoding bits of the one-hot code to be 4 bits, and generating the one-hot codes of data items A to D accordingly, that is, the one-hot code of data item A is 1000, the one-hot code of data item B is 0100, the one-hot code of data item C is 0010, and the one-hot code of data item D is 0001.

[0040] In this embodiment, if different data items (the first data item or the second data item) have the same field identification information, their one-hot codes should also be equivalent.

[0041] For example, suppose Figure 2 If data item B and data item D in the data set both correspond to the "age" field, then their respective unique hot codes should be the same.

[0042] Optionally, the first code and the second code may each further include a department identification code for identifying the department to which the first data item and the second data item respectively belong.

[0043] In this embodiment, a bitwise union operation may be performed on the first code of the first data item and the second code of the second data item constituting each fused data item in the fused data table to obtain the blood relationship code.

[0044] by Figure 2 Taking the example shown in the figure, if the generated fused data table D contains data item A, data item B, and data item D, based on the respective codes of data item A, data item B, and data item D (i.e., the first code and the second code), the blood relationship code of the fused data table D is:

[0045] 011000|010100|010001=011101

[0046] Step S106, obtaining a verification result of compliance or non-compliance of the fused data table according to the blood relationship code and the preset fusion rules.

[0047] Optionally, the preset fusion rule may include a preset reference code and a rule code.

[0048] Optionally, a verification result of compliance or non-compliance of the fusion data table may be obtained based on blood relationship coding, preset reference coding, and rule coding.

[0049] In this embodiment, if the verification result of the fusion data table is compliant, it means that there is no privacy and security issue in the data fusion operation in the data fusion table. Conversely, if the verification result of the fusion data table is not compliant, it means that there is a privacy and security issue in the data fusion operation in the data fusion table.

[0050] To sum up, the data fusion verification method of this embodiment generates data codes for data items, obtains the blood relationship codes of the fused data table based on them, and analyzes whether there are security risks in the data fusion operation in the fused data table based on the blood relationship codes, thereby effectively avoiding the problem of privacy leakage.

[0051] Second embodiment

[0052] Figure 3 The processing flow of the data fusion verification method of the second embodiment of the present application is shown. This embodiment is a specific implementation plan of the above step S106. As shown in the figure, this embodiment mainly includes the following steps:

[0053] Step S302, obtaining any first data item and any second data item in the data fusion table, performing a bitwise AND operation on the first code of the extracted first data item and the second code of the second data item to obtain a regular code.

[0054] In this embodiment, any first data item and any second data item in the fusion data table may be paired and combined to calculate the rule codes (also referred to as intrinsic security rule codes) of the two data items.

[0055] For example, if a first data item and a second data item are extracted respectively Figure 2 For data item B and data item D in , a bitwise AND calculation is performed on the first encoding of data item B (010100) and the second encoding of data item D (010001). The generated regular encoding corresponding to data item B and data item D is:

[0056] 010100|010001=010101

[0057] It should be noted that although both the rule code and the blood relationship code are generated by performing a bitwise union calculation on the first code and the second code, the blood relationship code is generated based on all the first data items and the second data items that constitute the fusion data table (that is, the blood relationship code is generated based on at least two codes of at least two data items), while the rule code is generated based on a first data item and a second data item (that is, two codes of two data items).

[0058] Step S304, performing a bitwise AND operation on the preset reference code, the blood relationship code, and the rule code to obtain a first operation result.

[0059] Optionally, the preset reference code is also composed of a shared identification code and a one-hot code.

[0060] Among them, based on the unconditional sharing attribute, the shared identification code of the preset benchmark code can be set to 00, and each coding bit of the one-hot code of the preset benchmark code can be set to 1. For example, if the one-hot code of the preset benchmark code contains a 4-bit code, its one-hot code is 1111.

[0061] For example, Figure 2 Taking data item B and data item D in as an example, by performing a bitwise AND operation on the preset reference code (001111), the blood relationship code (for example, 011101), and the rule code (010101) corresponding to data item B and data item D, the first operation result generated is:

[0062] 001111&011101&010101=000101

[0063] Step S306, performing a bitwise AND operation on the preset reference code and the regular code to obtain a second operation result.

[0064] For example, Figure 2 Taking data item B and data item D in as an example, by performing a bitwise AND operation on the preset reference code (001111) and the rule code (010101), the second operation result generated is:

[0065] 001111&010101=000101

[0066] Step S308: Perform a check on the fused data table according to the first operation result and the second operation result.

[0067] In this embodiment, if the first operation result and the second operation result match, a verification result indicating that the fused data table is not compliant is obtained; if the first operation result and the second operation result do not match, a verification result indicating that the fused data table is compliant is obtained.

[0068] For example, Figure 2 Taking data item B and data item D in as an example, according to the first operation result obtained in step S304 (i.e. 000101) and the second operation result obtained in step S306 (i.e. 000101), it can be seen that the first operation result and the second operation result are consistent, which means that there is a risk in the fusion between data item B and data item D in the data fusion table.

[0069] Third embodiment

[0070] Figure 3 The flowchart of the data fusion verification method of the third embodiment of the present application is shown. This embodiment can be executed in succession to the above second embodiment. As shown in the figure, this embodiment includes the following steps:

[0071] Step S402, based on the verification result of the first verification, determine whether the fusion data table is compliant. If it is compliant, proceed to step S404; if it is not compliant, end this process.

[0072] Specifically, whether the fusion data table is compliant may be determined based on the verification result of the above step S308.

[0073] Step S404, performing a secondary check on the fused data table according to the first operation result and the rule code.

[0074] In this embodiment, it can be determined whether the first operation result obtained in step S304 and the rule code obtained in step S302 are consistent. If the first operation result and the rule code are consistent, a verification result indicating that the fused data table is compliant is obtained. If the first operation result and the rule code are inconsistent, a verification result indicating that the fused data table is not compliant is obtained.

[0075] Fourth embodiment

[0076] Figure 5 The flowchart of the data fusion verification method of the fourth embodiment of the present application is shown. This embodiment can be executed in succession to the second embodiment or the third embodiment mentioned above. As shown in the figure, this embodiment mainly includes the following steps:

[0077] Step S502, based on the verification result of the first verification or the second verification, determine whether the fusion data table is compliant. If it is compliant, proceed to step S504; if it is not compliant, end this process.

[0078] Specifically, whether the fusion data table is compliant may be determined based on the verification result of the first verification in step S308 or the verification result of the second verification in step S404.

[0079] Step S504, using the prediction model to perform compliance prediction on each fused data item in the fused data table, and obtaining each prediction result of compliance or non-compliance of each fused data item.

[0080] Optionally, the classification model may perform prediction based on the value domain of the fused data item to obtain a classification label of whether the fused data item is compliant or non-compliant.

[0081] In this embodiment, if the prediction results of all fused data items are compliant, the verification results of the compliance of the fused data table are output.

[0082] To sum up, the data fusion verification method provided in each embodiment of the present application can generate blood relationship coding and rule coding according to data coding, and based on the blood relationship coding, rule coding and preset benchmark coding, perform verification processing at different stages for the data fusion operation of the fused data table. It can not only analyze whether there is a privacy leakage problem in the fused data table from an overall perspective, but also further verify in detail which data items in the fused data table have risks in the fusion operation, so as to reduce the possibility of data intrinsic security.

[0083] Fifth embodiment

[0084] The fifth embodiment of the present application provides a computer storage medium, characterized in that the computer storage medium stores instructions for executing each step in the method described in each embodiment.

[0085] Sixth embodiment

[0086] Figure 6 The schematic diagram of the structure of the data fusion verification device of the sixth embodiment of the present application is shown. As shown in the figure, the data fusion verification device 600 of the present embodiment includes a fusion module 602, an analysis module 604, and a verification module 606.

[0087] The fusion module 602 is used to fuse at least one first data item in the first data table with at least one second data item in the second data table to obtain a fused data table.

[0088] The analysis module 604 is used to obtain the blood relationship code of the fused data table according to at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table.

[0089] The verification module 606 is used to obtain the verification result of whether the fusion data table is compliant or not according to the blood relationship code and the preset fusion rules.

[0090] Optionally, the fusion module 602 is further used to: determine each fused data item according to the at least one first data item and the at least one second data item; and generate the fused data table according to each fused data item.

[0091] Optionally, the data fusion verification device 600 also includes an encoding module, which is used to: generate the shared identification code of the first encoding and the second encoding according to the shared attributes of the first data item and the second data item; generate the unique hot code of the first encoding and the second encoding according to the field identification information of the first data item and the second data item; wherein the shared attribute includes one of a non-shareable attribute, a conditional shared attribute, and an unconditional shared attribute.

[0092] Optionally, the analysis module 604 is further used to: perform a bitwise union operation on at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fusion data table to obtain the blood relationship code.

[0093] Optionally, the preset fusion rule includes a preset benchmark code and a rule code, and the verification module 606 is also used to: obtain any one of the first data items and any one of the second data items in the data fusion table, and perform a bitwise union operation on the first code of the extracted first data item and the second code of the second data item to obtain the rule code; perform a bitwise AND operation on the preset benchmark code, the blood relationship code, and the rule code to obtain a first operation result, and perform a bitwise AND operation on the preset benchmark code and the rule code to obtain a second operation result; perform a verification on the fusion data table according to the first operation result and the second operation result, if the first operation result and the second operation result are consistent, obtain a verification result that the fusion data table is non-compliant, if the first operation result and the second operation result are inconsistent, obtain a verification result that the fusion data table is compliant.

[0094] Optionally, a sharing identification code of the preset reference code may be set based on the unconditional sharing attribute.

[0095] Optionally, the verification module 606 is also used to: respond to the verification result of the compliance of the fused data table, perform a secondary verification on the fused data table according to the first operation result and the rule code, if the first operation result is consistent with the rule code, obtain a verification result of the compliance of the fused data table; if the first operation result is inconsistent with the rule code, obtain a verification result of the non-compliance of the fused data table.

[0096] Optionally, the verification module 606 is also used to: respond to the verification result of the compliance of the fused data table, use the prediction model to perform compliance prediction for each fused data item in the fused data table, and obtain each prediction result of compliance or non-compliance of each fused data item; if the prediction result that each fused data item is compliant is obtained, output the verification result of the compliance of the fused data table.

[0097] Optionally, the classification model may perform prediction according to the value range of the fused data item to obtain a classification label of compliance or non-compliance of the fused data item.

[0098] To summarize, the data fusion verification scheme provided in each embodiment of the present application generates a blood relationship code for the fused data table to verify whether there are risks in the data fusion processing of the fused data table, thereby ensuring that the data fusion processing is completed under the premise of data privacy and security.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data fusion verification method, It is characterized in that include: Merging at least one first data item in the first data table with at least one second data item in the second data table to obtain a merged data table; Obtaining the blood relationship code of the fused data table according to at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table; as well as According to the blood relationship code and the preset fusion rule, the verification result of compliance or non-compliance of the fusion data table is obtained; the preset fusion rule includes a preset reference code and a rule code; wherein, The obtaining of the verification result of compliance or non-compliance of the fusion data table according to the blood relationship code and the preset fusion rule includes: Obtain any one of the first data items and any one of the second data items in the fused data table, and perform a bitwise AND operation on the first code of the extracted first data item and the second code of the second data item to obtain the regular code; Performing a bitwise AND operation on the preset reference code, the blood relationship code, and the rule code to obtain a first operation result, and performing a bitwise AND operation on the preset reference code and the rule code to obtain a second operation result; According to the first operation result and the second operation result, a check is performed on the fused data table. If the first operation result and the second operation result are consistent, a check result indicating that the fused data table is not compliant is obtained. If the first operation result and the second operation result are inconsistent, a check result indicating that the fused data table is compliant is obtained. The method of obtaining the blood relationship code of the fused data table based on at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table comprises: performing a bitwise union operation on at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table to obtain the blood relationship code.

2. The data fusion verification method according to claim 1, It is characterized in that The fusing at least one first data item in the first data table with at least one second data item in the second data table to obtain the fused data table comprises: Determine each fused data item according to the at least one first data item and the at least one second data item; The fused data table is generated according to each of the fused data items.

3. The data fusion verification method according to claim 2, It is characterized in that The first code and the second code each include a shared identification code and a one-hot code; wherein the method further includes: Generate the shared identification code of the first code and the second code respectively according to the shared attributes of the first data item and the second data item; Generate the one-hot code of the first encoding and the second encoding respectively according to the field identification information of the first data item and the second data item; The sharing attribute includes one of a non-sharing attribute, a conditional sharing attribute, and an unconditional sharing attribute.

4. The data fusion verification method according to claim 3, It is characterized in that The method further comprises: Based on the unconditional sharing attribute, a sharing identification code of the preset reference code is set.

5. The data fusion verification method according to claim 1, It is characterized in that The method further comprises: In response to the verification result that the fused data table is in compliance, a secondary verification is performed on the fused data table based on the first operation result and the rule code. If the first operation result is consistent with the rule code, a verification result that the fused data table is in compliance is obtained; if the first operation result is inconsistent with the rule code, a verification result that the fused data table is not in compliance is obtained.

6. The data fusion verification method according to claim 1, It is characterized in that The method further comprises: In response to the compliance verification result of the fused data table, using the prediction model to perform compliance prediction on each fused data item in the fused data table, and obtaining each prediction result of compliance or non-compliance of each fused data item; If the prediction results obtained are all in compliance with the regulations, the verification results of compliance of the fused data table are output.

7. The data fusion verification method according to claim 6, It is characterized in that The prediction model includes a classification model; wherein, The classification model can perform prediction according to the value range of the fused data item to obtain a classification label of compliance or non-compliance of the fused data item.

8. A computer storage medium, It is characterized in that The computer storage medium stores instructions for executing the steps in the method according to any one of claims 1 to 7.

9. A data fusion verification device, It is characterized in that include: A fusion module, used to fuse at least one first data item in the first data table with at least one second data item in the second data table to obtain a fused data table; An analysis module, configured to obtain a blood relationship code of the fused data table according to at least one first code corresponding to the at least one first data item and at least one second code corresponding to the at least one second data item constituting the fused data table; The obtaining of the blood relationship code of the fused data table according to the at least one first code corresponding to the at least one first data item and the at least one second code corresponding to the at least one second data item constituting the fused data table comprises: performing a bitwise AND operation on the at least one first code corresponding to the at least one first data item and the at least one second code corresponding to the at least one second data item constituting the fused data table to obtain the blood relationship code; A verification module is used to obtain a verification result of compliance or non-compliance of the fusion data table according to the blood relationship code and the preset fusion rule; the preset fusion rule includes a preset reference code and a rule code; wherein, The obtaining of the verification result of compliance or non-compliance of the fusion data table according to the blood relationship code and the preset fusion rule includes: Obtain any one of the first data items and any one of the second data items in the fused data table, and perform a bitwise AND operation on the first code of the extracted first data item and the second code of the second data item to obtain the regular code; Performing a bitwise AND operation on the preset reference code, the blood relationship code, and the rule code to obtain a first operation result, and performing a bitwise AND operation on the preset reference code and the rule code to obtain a second operation result; According to the first operation result and the second operation result, a check is performed on the fused data table. If the first operation result and the second operation result match, a check result indicating that the fused data table is non-compliant is obtained. If the first operation result and the second operation result do not match, a check result indicating that the fused data table is compliant is obtained.

Images