Methods, equipment, storage media and devices for detecting document leakage risks

Abstract

This invention belongs to the field of data processing technology and discloses a method, device, storage medium, and apparatus for detecting file leakage risks. The invention determines a storage primary key based on the first metadata information corresponding to the current data file, and then queries second metadata information in a preset database using the storage primary key. A preset distributed bitwise XOR algorithm is used to perform metadata consistency verification on the first and second metadata information. Based on the consistency verification result, the invention determines whether the current data file has a leakage risk. The invention generates and saves metadata information when the data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and early warnings triggered in a timely manner when data security risks occur, thus improving data security.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, device, storage medium and apparatus for detecting the risk of document leakage. Background Technology

[0002] Today, big data systems such as Apache Spark and Apache Hadoop have become popular for using machine clusters to store, process, and analyze massive datasets. With the widespread adoption of cloud computing services, big data systems can easily access vast amounts of computing resources (e.g., CPU, memory, storage) to perform data-intensive tasks. When running big data programs to solve problems on clustered machines, if intermediate files storing the internal state are tampered with, the output may be incorrect, and there are also risks to data security.

[0003] Current big data technologies such as Apache Spark and Apache Hadoop are based on data lineage, which is determined by the MD5 hash of the recorded data. The uniqueness of the MD5 hash of the file content ensures that the data in temporary files cannot be tampered with. If the data is tampered with, the current computing application will stop, ensuring the security of subsequent computing data and the correctness of results.

[0004] Currently, existing technologies have the following serious drawbacks: Although Apache Spark and Apache Hadoop can ensure the correctness of computation results by checking the uniqueness of temporary files to detect whether the data has been tampered with.

[0005] However, in recent years, cloud computing services and data centers have become increasingly vulnerable to attacks / malware. Malware that infiltrates cloud data centers is more likely to steal information, but current technologies cannot guarantee that data will not be spied on, stolen, or tampered with in public clouds or untrusted environments, nor can they guarantee information security in a timely manner when data security risks arise.

[0006] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0007] The main objective of this invention is to provide a method, device, storage medium, and apparatus for detecting the risk of file leakage, aiming to solve the technical problems in the prior art that cannot ensure that data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and also cannot guarantee information security in a timely manner when data security risks occur.

[0008] To achieve the above objectives, the present invention provides a method for detecting document leakage risks, the method comprising the following steps:

[0009] The storage primary key is determined based on the first metadata information corresponding to the current data file, and the second metadata information is queried in the preset database using the storage primary key.

[0010] The first metadata information and the second metadata information are subjected to metadata consistency verification according to a preset distributed group bit XOR algorithm to obtain the consistency verification result.

[0011] The consistency verification result is used to determine whether the current data file has a risk of leakage.

[0012] Optionally, the step of performing metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed block bit XOR algorithm to obtain a consistency verification result includes:

[0013] The first metadata information is hashed according to preset fields to generate a first metadata group;

[0014] The second metadata information is hashed according to the preset fields to generate a second metadata group;

[0015] The first metadata group and the second metadata group are subjected to metadata consistency verification according to a preset distributed group bit XOR algorithm to obtain the consistency verification result.

[0016] Optionally, the step of performing metadata consistency verification on the first metadata group and the second metadata group according to a preset distributed group bit XOR algorithm to obtain a consistency verification result further includes:

[0017] Each metadata element in the first metadata group is hashed according to its position value to obtain the first hash value;

[0018] Each metadata element in the second metadata group is hashed according to its position value to obtain the second hash value;

[0019] The first hash value and the second hash value are subjected to a bitwise XOR operation according to a preset distributed group bitwise XOR algorithm to obtain the operation result;

[0020] Based on the calculation results, perform metadata consistency verification to obtain the consistency verification result.

[0021] Optionally, the calculation result includes a first calculation result and a second calculation result; the step of performing a bitwise XOR operation on the first hash value and the second hash value according to a preset distributed group bitwise XOR algorithm to obtain the calculation result includes:

[0022] According to the preset distributed group bit XOR algorithm, the first hash value and the second hash value are XORed bitwise to obtain the first operation result corresponding to each sequence number;

[0023] The integer values ​​in the first data group and the integer values ​​in the second data group are subjected to a bitwise XOR operation according to the preset distributed group bitwise XOR algorithm to obtain the second operation result.

[0024] Optionally, the step of performing metadata consistency verification based on the calculation result to obtain a consistency verification result includes:

[0025] Determine the data difference information based on the first calculation result and the second calculation result;

[0026] Based on the data difference information, a metadata consistency check is performed to obtain the consistency check result.

[0027] Optionally, the step of determining whether the current data file has a risk of leakage based on the consistency verification result includes:

[0028] If the consistency check result is consistent, then it is determined that the current data file does not pose a risk of data leakage.

[0029] If the consistency check result is inconsistent, it is determined that the current data file has a risk of leakage.

[0030] Optionally, after the step of determining whether the current data file has a risk of leakage based on the consistency verification result, the method further includes:

[0031] If it is determined that the current data file does not pose a risk of leakage, the application will return to continue with subsequent tasks.

[0032] If it is determined that the current data file poses a risk of data leakage, the file will be deleted, an alert will be issued, and the application will be stopped.

[0033] Furthermore, to achieve the above objectives, the present invention also proposes a file leakage risk detection device, which includes a memory, a processor, and a file leakage risk detection program stored in the memory and executable on the processor. The file leakage risk detection program is configured to implement the file leakage risk detection steps described above.

[0034] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a file leakage risk detection program, wherein when the file leakage risk detection program is executed by a processor, it implements the steps of the file leakage risk detection method described above.

[0035] Furthermore, to achieve the above objectives, the present invention also proposes a document leakage risk detection device, which includes:

[0036] The data query module is used to determine the storage primary key based on the first metadata information corresponding to the current data file, and to query the second metadata information in the preset database using the storage primary key.

[0037] The data verification module is used to perform metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed group bit XOR algorithm, and obtain the consistency verification result;

[0038] The risk assessment module is used to determine whether the current data file has a risk of leakage based on the consistency verification result.

[0039] This invention determines a storage primary key based on the first metadata information corresponding to the current data file, and then queries the second metadata information in a preset database using the storage primary key. A metadata consistency check is performed on the first and second metadata information using a preset distributed group bitwise XOR algorithm to obtain a consistency check result. Based on the consistency check result, it is determined whether the current data file is at risk of leakage. Compared to existing technologies that cannot ensure data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and cannot guarantee information security in a timely manner when data security risks arise, this invention generates and saves metadata information each time a data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and early warnings triggered in a timely manner when data security risks occur, thus improving data security. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the structure of a file leakage risk detection device for the hardware operating environment involved in the embodiments of the present invention;

[0041] Figure 2 This is a flowchart illustrating the first embodiment of the document leakage risk detection method of the present invention;

[0042] Figure 3 This is a flowchart illustrating the second embodiment of the document leakage risk detection method of the present invention;

[0043] Figure 4 This is a flowchart illustrating the third embodiment of the document leakage risk detection method of the present invention;

[0044] Figure 5This is a schematic diagram of hash value comparison in the third embodiment of the document leakage risk detection method of the present invention;

[0045] Figure 6 This is a schematic diagram of the implementation process of the third embodiment of the document leakage risk detection method of the present invention;

[0046] Figure 7 This is a schematic diagram of the consistency verification process of the third embodiment of the document leakage risk detection method of the present invention;

[0047] Figure 8 This is a structural block diagram of the first embodiment of the document leakage risk detection device of the present invention.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0050] Reference Figure 1 , Figure 1 This is a schematic diagram of the file leakage risk detection device in the hardware operating environment involved in the embodiments of the present invention.

[0051] like Figure 1 As shown, the document leakage risk detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1The structure shown does not constitute a limitation on the document leakage risk detection device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a file leakage risk detection program.

[0054] exist Figure 1 In the file leakage risk detection device shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to the user device; the file leakage risk detection device calls the file leakage risk detection program stored in the memory 1005 through the processor 1001 and executes the file leakage risk detection method provided in the embodiment of the present invention.

[0055] Based on the above hardware structure, an embodiment of the document leakage risk detection method of the present invention is proposed.

[0056] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the document leakage risk detection method of the present invention, which presents the first embodiment of the document leakage risk detection method of the present invention.

[0057] In this embodiment, the file leakage risk detection method includes the following steps:

[0058] Step S10: Determine the storage primary key based on the first metadata information corresponding to the current data file, and query the second metadata information in the preset database using the storage primary key.

[0059] It should be noted that the executing entity in this embodiment can be a device containing a file leakage risk detection system, such as a computer, tablet, mobile phone, or laptop, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, a computer is used as an example to describe the file leakage risk detection method of the present invention. The application corresponding to the file leakage risk detection method of the present invention has a built-in file metadata collection device, a decision-making device, and a processing device. The collection device can be used to collect file-specific metadata information; the decision-making device can determine whether there is a risk of file leakage by comparing the consistency of metadata information during file reading and writing; the processing device can be used to perform operations when data security risks occur; the storage medium can be used to efficiently read and write metadata information. When comparing the consistency of two sets of file metadata, the decision-making device proposes an original distributed group bit XOR algorithm. This algorithm uses distributed technology to perform XOR operations on each group of data bits in parallel, which has the characteristics of being fast and efficient.

[0060] It should be understood that when performing calculations on big data computing engines such as Spark or Hadoop, whether it is the initial original file or a temporary file created during the calculation process, after a new file appears, the file metadata detection process in this solution can be invoked to collect metadata information using a file metadata collection device. The metadata information includes information such as file name, file path, file size (number of bytes), file access permissions, file last access time, file content last modification time, file last modification time, total number of data blocks in the file, file owner's user ID, file group ID, number of file connections, inode number, device number, server IP address, and current application ID.

[0061] It is understandable that the first metadata information corresponding to the current data file can refer to the latest metadata information of the current file when the application and data file are submitted to the computer cluster in a big data computing environment. When the application starts running, it reads the data file and collects the latest metadata information of the current file through a preset metadata collection device. Therefore, the first metadata information refers to the latest metadata information of the current file. It should also be understood that the storage primary key can refer to each row of data in the metadata table generated based on the first metadata information. Each table can only have one primary key, which can be a single column or a combination of multiple columns. These values ​​must be unique. The calculation method for the storage primary key is as follows:

[0062] key=hash(ip+appid+path+name).

[0063] The preset database can refer to a pre-set database used to store historical metadata information. When historical metadata information is saved, it is identified by the storage primary key. Therefore, the database can be queried to see if historical metadata information exists based on the storage primary key corresponding to the first metadata information. If metadata information matching the storage primary key is found, it is extracted as the historical metadata information corresponding to the current file, i.e., the second metadata information, so as to facilitate consistency verification later.

[0064] Step S20: Perform metadata consistency verification on the first metadata information and the second metadata information according to the preset distributed group bit XOR algorithm to obtain the consistency verification result.

[0065] It should be noted that the preset distributed grouped bit XOR algorithm is a pre-set grouped XOR algorithm based on the preset storage method in this scheme. Here, XOR operation generally refers to XOR. XOR (XOR) is a mathematical operator applied to logical operations. The grouped XOR algorithm is used to group integer arrays according to the number of bits and perform XOR operation on the grouped arrays. If the two numbers are different, the result is true (1). Simply put, its function is 0^1 = 1, 0^0 = 0, 1^1 = 0. If the two binary bits participating in the operation have the same sign, the result is 0; if they have different signs, the result is 1.

[0066] It is understandable that a pre-defined distributed group bit XOR algorithm is used to perform pairwise XOR operations on each metadata field contained in the first metadata information and the second metadata information to obtain the operation result. Based on the operation result, metadata consistency verification is performed to obtain the consistency verification result.

[0067] It should be understood that both the first metadata information and the second metadata information include the following fields:

[0068]

[0069]

[0070] In its specific implementation, this scheme performs a one-to-one XOR operation on the 15 type fields included in the first metadata information and the 15 type fields included in the second metadata information to obtain the XOR operation results among the 15 data groups. Based on the XOR operation results, metadata consistency verification is performed to obtain a consistency verification result. The consistency verification result includes both consistent and inconsistent results.

[0071] Step S30: Determine whether there is a risk of data leakage in the current data file based on the consistency verification result.

[0072] It should be noted that the consistency verification results include both consistent and inconsistent results to determine whether the current data file is at risk of leakage.

[0073] Understandably, if the consistency check result shows consistency, it is determined that the current data file does not pose a risk of data leakage; conversely, if the consistency check result shows inconsistency, it is determined that the current data file poses a risk of data leakage. This determines whether the data is at risk of being stolen. By checking the consistency results of metadata, this device can promptly destroy temporary data and trigger alerts when data security risks arise, thereby improving data security.

[0074] This embodiment determines the storage primary key based on the first metadata information corresponding to the current data file, and then queries the second metadata information in a preset database using the storage primary key. A metadata consistency check is performed on the first and second metadata information using a preset distributed group bitwise XOR algorithm to obtain a consistency check result. Based on the consistency check result, it is determined whether the current data file is at risk of leakage. Compared to existing technologies that cannot ensure data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and cannot guarantee information security in a timely manner when data security risks arise, this embodiment generates and saves metadata information each time a data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and warnings triggered in a timely manner when data security risks occur, thus improving data security.

[0075] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the document leakage risk detection method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the document leakage risk detection method of the present invention.

[0076] In this embodiment, step S20 includes:

[0077] Step S201: Hash the first metadata information according to preset fields to generate a first metadata group.

[0078] It should be noted that the default fields are pre-set fields that store metadata information bit by bit according to file name, file path, file size (number of bytes), file access permissions, file last access time, file content last modification time, file last modification time, total number of data blocks in the file, file owner's user ID, file group ID, number of file connections, inode number, device number, server IP address, and current application ID.

[0079] It should be understood that collecting current file metadata information involves calling the Linux command `stat` to collect file metadata information, including 15 fields: filename, file path, file size (in bytes), file access permissions, last access time, last modification time, last change time, total number of data blocks, file owner's user ID, file group ID, number of file links, inode number, device number, server IP address, and current application ID. When the big data computing engine performs subsequent calculations, if a new file operation is added, the metadata is stored bit-wise, totaling 15 bits. Therefore, the first metadata group contains the aforementioned 15-bit field data.

[0080] In the specific implementation, the first metadata information is hashed to form (key, value) data pairs, and a complete set of metadata information is denoted as:

[0081] key_value_pairs=[kv1,kv2,...,kvn].

[0082] Step S202: Hash the second metadata information according to the preset fields to generate a second metadata group.

[0083] It should be noted that the second metadata information can be the metadata information of an existing file or a previously created temporary file when the big data computing engine performs subsequent calculations. Specifically, the extraction method involves first collecting the metadata information of the current state of the target file, calculating the storage primary key, and then querying the preset database for matching metadata information based on the key. This yields the second metadata information.

[0084] Understandably, since the metadata in this scheme is stored bit by bit, the second metadata group is also a data group generated based on the number of bits corresponding to the preset field and the field data.

[0085] In practice, the second metadata information is hashed to form (key, value) data pairs. A complete set of metadata information is denoted as:

[0086] key_value_pairs=[kv1,kv2,...,kvn].

[0087] Step S203: Perform metadata consistency verification on the first metadata group and the second metadata group according to the preset distributed group bit XOR algorithm to obtain the consistency verification result.

[0088] It should be noted that the field data contained in the first metadata group and the second metadata group are checked for metadata consistency by a pre-set distributed group bit XOR algorithm to obtain the consistency check result. The consistency check method can be to compare the field data contained in the first metadata group and the field data contained in the second metadata group one by one according to the position value, and then determine whether the metadata is consistent based on the comparison result to obtain the consistency check result.

[0089] In the specific implementation, the 15 sets of data corresponding to the first metadata information are compared one by one with the 15 sets of data corresponding to the second metadata information, so as to determine whether the metadata is consistent based on the comparison results and obtain the consistency verification result.

[0090] Furthermore, step S203 further includes: performing a hash value conversion on each metadata element in the first metadata group according to its position value to obtain a first hash value; performing a hash value conversion on each metadata element in the second metadata group according to its position value to obtain a second hash value; performing a bitwise XOR operation on the first hash value and the second hash value according to a preset distributed group bitwise XOR algorithm to obtain a calculation result; and performing metadata consistency verification based on the calculation result to obtain a consistency verification result.

[0091] It should be noted that, to improve consistency verification efficiency, this scheme converts all metadata data except for integer values ​​into hash values ​​for consistency verification. Specifically, the hash value conversion process for the first metadata information involves hashing each metadata element in the first metadata group according to its position value to obtain the first hash value. The position value refers to the tag value corresponding to each metadata element, for example: 1-15, meaning the key_value_pairs has 15 data pairs. Each metadata element is labeled from 1 to 15, and each metadata element is converted into a binary hash value based on its value.

[0092] Understandably, the hash value conversion process for the second metadata information involves hashing each metadata element in the second metadata group according to its position value to obtain the second hash value. The position value refers to the tag number corresponding to each metadata element, for example: 1-15, meaning the key_value_pairs has 15 data pairs. Each metadata element is labeled 1 to 15, and each metadata element is converted into a binary hash value based on its value.

[0093] In the specific implementation, after conversion using the above method, the two sets of binary hash values ​​at the same sequence number bits of the old and new metadata are subjected to a bitwise XOR operation. If both bits are 0 or both are 1, the result is 0; otherwise, it is 1, as shown in the following formula:

[0094] 10010xor 10010 = 0;

[0095] 10001xor 10101 = 1.

[0096] The above bitwise XOR operation is used to perform metadata consistency verification based on the operation result, and the consistency verification result is obtained.

[0097] Further, the calculation result includes a first calculation result and a second calculation result; the step of performing a bitwise XOR operation on the first hash value and the second hash value according to a preset distributed block bitwise XOR algorithm to obtain the calculation result includes: performing a bitwise XOR operation on the first hash value and the second hash value according to the preset distributed block bitwise XOR algorithm to obtain a first calculation result corresponding to each sequence number; and performing a bitwise XOR operation on the integer values ​​in the first data group and the integer values ​​in the second data group according to the preset distributed block bitwise XOR algorithm to obtain a second calculation result.

[0098] It should be noted that the first operation result includes a comparison between the hash value of the first metadata group after bitwise conversion and the hash value of the second metadata group after bitwise conversion, and the second operation result includes a comparison between the integer value of the first metadata group and the integer value of the second metadata group.

[0099] Understandably, in the specific calculation process, the metadata corresponding to the preset field is converted into a hash value to obtain the converted hash value, and the hash value or integer value corresponding to each sequence number is compared one by one to obtain the calculation result.

[0100] Furthermore, the step of performing metadata consistency verification based on the calculation result to obtain a consistency verification result includes: determining data difference information based on the first calculation result and the second calculation result; performing metadata consistency verification based on the data difference information to obtain a consistency verification result.

[0101] It should be noted that the data difference information is determined by comparing the hash value difference information in the first operation result with the integer difference information in the second operation structure, and the metadata consistency is verified through the data difference information to obtain the consistency verification result.

[0102] Understandably, the consistency check result is determined based on whether there are numerical differences in the data difference information. If there are numerical differences, the consistency check result is determined to be inconsistent; if there are no numerical differences, the consistency check result is determined to be consistent.

[0103] This embodiment determines the storage primary key based on the first metadata information corresponding to the current data file, and queries the second metadata information in a preset database using the storage primary key. The first metadata information is hashed according to preset fields to generate a first metadata group; the second metadata information is hashed according to the preset fields to generate a second metadata group; a metadata consistency check is performed on the first and second metadata groups using a preset distributed group bitwise XOR algorithm to obtain a consistency check result; based on the consistency check result, it is determined whether the current data file has a risk of leakage. Compared to existing technologies that cannot ensure data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and cannot guarantee information security in a timely manner when data security risks arise, this embodiment generates and saves metadata information each time a data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency between the metadata information, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and warnings triggered in a timely manner when data security risks arise, thus improving data security.

[0104] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the document leakage risk detection method of the present invention, based on the above. Figure 2 The first embodiment shown presents a third embodiment of the document leakage risk detection method of the present invention.

[0105] In this embodiment, step S30 includes:

[0106] Step S301: If the consistency verification result is consistent, then it is determined that the current data file does not pose a risk of leakage.

[0107] It should be noted that the consistency verification result is determined by the data difference information contained in the calculation result. Specifically, the consistency verification result is determined based on whether there is a numerical difference in the data difference information. If there is a numerical difference, the consistency verification result is determined to be inconsistent; if there is no numerical difference, the consistency verification result is determined to be consistent.

[0108] Understandably, if the consistency check result is consistent, it is determined that there is no risk of data leakage in the current data file.

[0109] Step S302: If the consistency check result is inconsistent, it is determined that the current data file has a risk of leakage.

[0110] It should be noted that if the consistency check result is inconsistent, the current data file is deemed to have a risk of leakage.

[0111] In specific implementation, Figure 5 The hash value comparison diagram shown illustrates this. Taking a change in the file's atime (the time the file was last accessed) as an example, the key_value pair of the aTime field shows a difference of two bits. This means the consistency check result is inconsistent, therefore, the current data file is deemed to pose a risk of data leakage.

[0112] Furthermore, after step S30, the method further includes: if it is determined that the current data file does not pose a risk of leakage, then return to the application to continue completing subsequent work; if it is determined that the current data file poses a risk of leakage, then delete the file, issue an alarm, and stop the application.

[0113] It should be noted that, for further explanation of the metadata leakage prevention detection process in this solution, please refer to [link / reference]. Figure 6 The schematic diagram of the implementation process of the scheme provides a file leakage prevention and inspection method based on metadata in a big data computing environment. It can be applied to a file leakage prevention and inspection device based on metadata in a big data computing environment, including: metadata storage medium, file metadata acquisition device, decision-making device and processing device.

[0114] Understandably, the metadata storage medium stores file metadata information, totaling 15 bits. Input: the key value calculated in step 102; Output: 15-bit metadata information. The file metadata collection device, by improving the coding of big data computing engines such as Spark and Hadoop and embedding it within their code, collects the metadata information of temporary files generated during computation and stores them in the metadata storage medium after storing the temporary files. Before loading temporary files, it collects the metadata information of the temporary files and submits it to the decision-making device. Input: (full file path, operation command); Output: (current file metadata, metadata information in the database). The decision-making device has a built-in algorithm to check whether the metadata of temporary files has changed, to determine whether the temporary files have been stolen by other users or applications, and submits the decision result to the disposal device. Input: (current file metadata, metadata information in the database); Output: metadata consistency verification result, True or False. The disposal device, by improving the coding of big data computing engines such as Spark and Hadoop and embedding it within their code, if the decision result indicates a data security risk, triggers the application to delete the temporary file, triggers an alarm message to the application publisher, and stops the application from running. Input: Metadata consistency verification result, True or False; Output: Metadata consistency verification result returned to the main process, True or False.

[0115] It should be understood that, reference Figure 6The schematic diagram illustrates the implementation process of this scheme. The specific steps in the implementation are as follows: Step 101: Collect the current file metadata information. Call the Linux command `stat` to collect file metadata information, which consists of 15 fields. Step 102: Calculate the metadata storage primary key. Step 103: When the big data computing engine performs subsequent calculations, if it's a new file operation, store the metadata. The metadata is stored bit-by-bit, totaling 15 bits. Step 104: When the big data computing engine performs subsequent calculations, if it's reading an existing file or a previously created temporary file, first call step 101 to collect the current state metadata information of the target file, then call step 102 to calculate the storage primary key. Step 105: Query the matching metadata information in the storage medium based on the key. Step 106: Submit both sets of metadata to the decision-making device. At this point, the decision-making device possesses two sets of metadata information. Step 107: The decision-making device uses a distributed bitwise XOR algorithm to perform metadata consistency verification. First, the metadata is hashed to form (key, value) data pairs. A complete set of metadata information is denoted as: key_value_pairs = [kv1, kv2, ..., kvn]. Step 108: Referring to the above metadata having 15 fields, key_value_pairs has 15 sets of data. These are labeled 1 to 15, and each metadata bit is converted into a binary hash value. The two sets of binary hash values ​​with the same sequence number bits in the old and new metadata are XORed. If both bits are 0 or both are 1, the result is 0; otherwise, it is 1. Step 109: Distributed parallel technology is used to perform parallel computation on the 15 sets of data, which can further improve efficiency. The consistency verification process is illustrated in the diagram below. Figure 7 As shown. Step 110: If the number of bit differences in a certain set of results is not 0, it can be considered that there is a discrepancy between the two sets of metadata. This means that during the time period between when the application finishes writing data in the pre-processing stage and before reading data in the post-processing stage, the current file was accessed by users, applications, or malware outside the current application. At this time, there is a risk of data leakage. Otherwise, it is determined that the data is currently in a safe state. Step 111: If the decision result is that the data is safe, the application returns to continue to complete the subsequent work; if the decision result is that there is a risk of data leakage, the disposal device is submitted to delete the file, issue an alarm, and stop the application.

[0116] This embodiment determines the storage primary key based on the first metadata information corresponding to the current data file, and queries the second metadata information in a preset database using the storage primary key. It then performs a metadata consistency check on the first and second metadata information using a preset distributed group bitwise XOR algorithm to obtain a consistency check result. If the consistency check result is consistent, it is determined that the current data file has no risk of leakage; if the consistency check result is inconsistent, it is determined that the current data file has a risk of leakage. Compared to existing technologies that cannot ensure data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and cannot guarantee information security in a timely manner when data security risks arise, this embodiment generates and saves metadata information each time a data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency between the metadata information, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and warnings triggered in a timely manner when data security risks arise, thus improving data security.

[0117] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a file leakage risk detection program, wherein when the file leakage risk detection program is executed by a processor, it implements the steps of the file leakage risk detection method described above.

[0118] Reference Figure 8 , Figure 8 This is a structural block diagram of the first embodiment of the document leakage risk detection device of the present invention.

[0119] like Figure 8 As shown, the document leakage risk detection device proposed in this embodiment of the invention includes:

[0120] The data query module 10 is used to determine the storage primary key based on the first metadata information corresponding to the current data file, and to query the second metadata information in the preset database using the storage primary key.

[0121] Data verification module 20 is used to perform metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed group bit XOR algorithm, and obtain a consistency verification result;

[0122] The risk assessment module 30 is used to determine whether there is a risk of data leakage in the current data file based on the consistency verification result.

[0123] This embodiment determines the storage primary key based on the first metadata information corresponding to the current data file, and then queries the second metadata information in a preset database using the storage primary key. A metadata consistency check is performed on the first and second metadata information using a preset distributed group bitwise XOR algorithm to obtain a consistency check result. Based on the consistency check result, it is determined whether the current data file is at risk of leakage. Compared to existing technologies that cannot ensure data is not spied on, stolen, or tampered with in public clouds or untrusted environments, and cannot guarantee information security in a timely manner when data security risks arise, this embodiment generates and saves metadata information each time a data file is created, reads the metadata information of the current data file, and compares it with the previous metadata information to ensure consistency, thereby determining whether the data is at risk of being stolen. Through the consistency result of the metadata, temporary data can be destroyed and warnings triggered in a timely manner when data security risks occur, thus improving data security.

[0124] Furthermore, the data verification module 20 is also used to hash the first metadata information according to a preset field to generate a first metadata group; hash the second metadata information according to the preset field to generate a second metadata group; and perform metadata consistency verification on the first metadata group and the second metadata group according to a preset distributed group bit XOR algorithm to obtain a consistency verification result.

[0125] Furthermore, the data verification module 20 is also used to perform a hash value conversion on each metadata bit in the first metadata group according to its position value to obtain a first hash value; perform a hash value conversion on each metadata bit in the second metadata group according to its position value to obtain a second hash value; perform a bitwise XOR operation on the first hash value and the second hash value according to a preset distributed group bit XOR algorithm to obtain a calculation result; and perform metadata consistency verification based on the calculation result to obtain a consistency verification result.

[0126] Furthermore, the calculation result includes a first calculation result and a second calculation result; the data verification module 20 is also used to perform a bitwise XOR operation on the first hash value and the second hash value according to a preset distributed group bitwise XOR algorithm to obtain a first calculation result corresponding to each sequence number; and to perform a bitwise XOR operation on the integer values ​​in the first data group and the integer values ​​in the second data group according to the preset distributed group bitwise XOR algorithm to obtain a second calculation result.

[0127] Furthermore, the data verification module 20 is also used to determine data difference information based on the first calculation result and the second calculation result; perform metadata consistency verification based on the data difference information, and obtain a consistency verification result.

[0128] Furthermore, the risk assessment module 30 is also used to determine that the current data file does not have a risk of leakage if the consistency verification result is consistent; and to determine that the current data file has a risk of leakage if the consistency verification result is inconsistent.

[0129] Furthermore, the risk assessment module 30 is also used to return to the application to continue the subsequent work if it is determined that the current data file does not pose a risk of leakage; and to delete the file, issue an alarm, and stop the application if it is determined that the current data file poses a risk of leakage.

[0130] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0131] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0132] In addition, for technical details not described in detail in this embodiment, please refer to the document leakage risk detection method provided in any embodiment of the present invention, which will not be repeated here.

[0133] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0134] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0136] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for detecting document leakage risk, characterized in that, The document leakage risk detection method includes the following steps: The storage primary key is determined based on the first metadata information corresponding to the current data file, and the second metadata information is queried in the preset database using the storage primary key. The first metadata information refers to the latest metadata information of the current file, and the second metadata information refers to the historical metadata information corresponding to the current file. The first metadata information and the second metadata information are subjected to metadata consistency verification according to a preset distributed group bit XOR algorithm to obtain the consistency verification result. Based on the consistency verification results, determine whether the current data file has a risk of leakage. The step of performing metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed block bit XOR algorithm to obtain a consistency verification result includes: The hash value of the first metadata group corresponding to the first metadata information is converted to obtain the first hash value; The second metadata group corresponding to the second metadata information is hashed to obtain the second hash value; According to the preset distributed group bit XOR algorithm, the first hash value and the second hash value are XORed bitwise to obtain the first operation result corresponding to each sequence number; The integer values ​​in the first metadata group and the integer values ​​in the second metadata group are subjected to a bitwise XOR operation according to the preset distributed group bitwise XOR algorithm to obtain the second operation result; Based on the first and second calculation results, a metadata consistency check is performed to obtain the consistency check result.

2. The document leakage risk detection method as described in claim 1, characterized in that, The step of performing metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed block bit XOR algorithm to obtain a consistency verification result includes: The first metadata information is hashed according to preset fields to generate a first metadata group; The second metadata information is hashed according to the preset fields to generate a second metadata group; The first metadata group and the second metadata group are subjected to metadata consistency verification according to a preset distributed group bit XOR algorithm to obtain the consistency verification result.

3. The document leakage risk detection method as described in claim 2, characterized in that, The step of performing metadata consistency verification on the first metadata group and the second metadata group according to a preset distributed block bit XOR algorithm to obtain a consistency verification result further includes: Each metadata element in the first metadata group is hashed according to its position value to obtain the first hash value; Each metadata element in the second metadata group is hashed according to its position value to obtain the second hash value; The first hash value and the second hash value are subjected to a bitwise XOR operation according to a preset distributed group bitwise XOR algorithm to obtain the operation result; Based on the calculation results, perform metadata consistency verification to obtain the consistency verification result.

4. The document leakage risk detection method as described in claim 1, characterized in that, The step of performing metadata consistency verification based on the calculation result to obtain the consistency verification result includes: Determine the data difference information based on the first calculation result and the second calculation result; Based on the data difference information, a metadata consistency check is performed to obtain the consistency check result.

5. The document leakage risk detection method as described in any one of claims 1-4, characterized in that, The step of determining whether the current data file has a risk of leakage based on the consistency verification result includes: If the consistency check result is consistent, then it is determined that the current data file does not pose a risk of data leakage. If the consistency check result is inconsistent, it is determined that the current data file has a risk of leakage.

6. The document leakage risk detection method as described in claim 5, characterized in that, After the step of determining whether the current data file has a risk of leakage based on the consistency verification result, the method further includes: If it is determined that the current data file does not pose a risk of leakage, the application will return to continue with subsequent tasks. If it is determined that the current data file poses a risk of data leakage, the file will be deleted, an alert will be issued, and the application will be stopped.

7. A document leakage risk detection device, characterized in that, The file leakage risk detection device includes: a memory, a processor, and a file leakage risk detection program stored in the memory and executable on the processor. When the file leakage risk detection program is executed by the processor, it implements the file leakage risk detection method as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium stores a file leakage risk detection program, which, when executed by a processor, implements the file leakage risk detection method as described in any one of claims 1 to 6.

9. A document leakage risk detection device, characterized in that, The document leakage risk detection device includes: The data query module is used to determine the storage primary key based on the first metadata information corresponding to the current data file, and query the second metadata information in the preset database through the storage primary key. The first metadata information refers to the latest metadata information of the current file, and the second metadata information refers to the historical metadata information corresponding to the current file. The data verification module is used to perform metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed group bit XOR algorithm, and obtain the consistency verification result; The risk assessment module is used to determine whether the current data file has a risk of leakage based on the consistency verification result; The step of performing metadata consistency verification on the first metadata information and the second metadata information according to a preset distributed block bit XOR algorithm to obtain a consistency verification result includes: The hash value of the first metadata group corresponding to the first metadata information is converted to obtain the first hash value; The second metadata group corresponding to the second metadata information is hashed to obtain the second hash value; According to the preset distributed group bit XOR algorithm, the first hash value and the second hash value are XORed bitwise to obtain the first operation result corresponding to each sequence number; The integer values ​​in the first metadata group and the integer values ​​in the second metadata group are subjected to a bitwise XOR operation according to the preset distributed group bitwise XOR algorithm to obtain the second operation result; Based on the first and second calculation results, a metadata consistency check is performed to obtain the consistency check result.

Images