Hierarchical data fingerprint auditing and tracing method and device in data exchange

By using the hierarchical data fingerprint calculation method in the data exchange system, different levels of data fingerprints are extracted and audited and traced, the problem of inability to effectively trace data exchange behavior in the existing technology is solved, an efficient and accurate traceability process is achieved, and computing and storage resources are saved.

CN119675940AInactive Publication Date: 2025-03-21NO 30 INST OF CHINA ELECTRONIC TECH GRP CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411788938.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-06
Publication Date
2025-03-21
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

During the data fingerprint extraction and audit traceability process, existing data exchange systems cannot effectively discover changes in file content, resulting in the inability to accurately trace the exchange behavior.

Method used

The hierarchical data fingerprint calculation method is adopted, and different levels of processing methods are adopted according to different hierarchies of data files. Data fingerprints of level 1, level 2, level 3, level 4 and level 5 are extracted and stored in the log audit library to achieve efficient and accurate traceability of data exchange behavior.

Benefits of technology

While ensuring the accuracy of audit traceability of exchange behavior audits, the system's computing resources and storage resources are greatly saved, and the efficiency and security of the data exchange system are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119675940A_ABST
    Figure CN119675940A_ABST
Patent Text Reader

Abstract

The invention discloses a hierarchical data fingerprint auditing and tracing method and device in data exchange, and belongs to the technical field of data exchange, and the method comprises the following steps: based on different grades of data files, adopting processing modes of different grades to calculate hierarchical data fingerprints; extracting hierarchical data fingerprints; and performing auditing backtracking based on behaviors of the data content. According to the method, the data fingerprint extraction and auditing traceability process is optimized, and the computing resources and the storage resources of the system are greatly saved under the condition that the auditing traceability accuracy of the exchange behavior is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data exchange, and more specifically, to a hierarchical data fingerprint audit tracing method and device in data exchange. Background Art

[0002] With the rapid development of informatization, more and more data need to be shared and exchanged securely and controllably between different networks, and data exchange systems have emerged.

[0003] The data exchange system can provide data exchange services for various business systems. However, on the one hand, the data exchange system provides services to the outside world and should not store log records that can be restored to original data to avoid information leakage. On the other hand, when the exchanged data contains malicious code that spreads in the receiving network domain or files containing sensitive information are illegally transmitted across the network, it is necessary to trace the exchange behavior based on audit records so that managers can understand the situation or hold them accountable. Therefore, it is necessary to adopt a data fingerprint method to record the characteristics of the exchanged data but cannot restore the original information through the data fingerprint. The traditional data fingerprint extraction method in data exchange is to calculate the fingerprint of the entire file; if the file content is modified, the exchange behavior cannot be discovered. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a hierarchical data fingerprint audit tracing method and device in data exchange, which optimizes the data fingerprint extraction and audit tracing process, and greatly saves the system's computing resources and storage resources while ensuring the accuracy of the audit tracing of the exchange behavior.

[0005] The object of the present invention is achieved through the following solutions:

[0006] A hierarchical data fingerprint audit tracing method in data exchange includes the following steps:

[0007] S1, based on the different classifications of data files, different levels of processing methods are used to calculate the hierarchical data fingerprints;

[0008] S2, extracting hierarchical data fingerprints;

[0009] S3, audit backtracking based on data content behavior.

[0010] Furthermore, in step S1, based on different classifications of the data files, different levels of processing methods are used to calculate the hierarchical data fingerprints, which specifically includes the following sub-steps:

[0011] The data fingerprint is divided into five levels according to the different granularities of extraction, and the data fingerprint is calculated in units of files:

[0012] Level 1 fingerprint: This level of data fingerprint extracts the data fingerprint of the entire file, and the calculation result is the data fingerprint of the entire file; each file corresponds to a level 1 data fingerprint;

[0013] Level 2 fingerprint: This level of data fingerprint first extracts the file content, then performs data denoising, and finally extracts data fingerprints from the denoised data content; each file corresponds to a level 2 data fingerprint;

[0014] Level 3 fingerprint: This level of data fingerprint extracts the file content and divides it into segments according to natural segments, then performs denoising operation on each natural segment, and finally extracts data fingerprint from the data content after denoising operation; a set of level 3 data fingerprints is extracted from each file, and the number of level 3 data fingerprints is the number of natural segments;

[0015] Level 4 fingerprint: This level of data fingerprint extracts the file content and divides it according to sentences, then performs denoising and deduplication operations on each sentence, and finally extracts data fingerprints from the denoised data content; each file will extract a set of level 4 data fingerprints;

[0016] Level 5 fingerprint: This level of data fingerprint requires the extraction of file content, which is then divided into natural segments. Each segment uses natural language semantic analysis to segment the data file content. After segmentation, unimportant modal particles are removed, repeated segmentations are merged and weighted. Finally, data fingerprints are extracted from the remaining segmentations, and the weights of the data fingerprints are recorded. Multiple sets of level 5 data fingerprints with data weights are extracted from each file.

[0017] Furthermore, the denoising operation specifically includes three contents: ① removing spaces from Chinese characters and separating English words with spaces; ② removing punctuation marks; ③ converting traditional Chinese characters into simplified Chinese characters.

[0018] Furthermore, the merging of repeated participles and assigning weights specifically includes: merging repeated participles and assigning weights according to the number of occurrences.

[0019] Furthermore, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0020] S21, performing first-level data fingerprint calculation on the exchange file;

[0021] S22, judging the exchange data type, whether the exchange data is an office file; if it is not an office file, jumping to step S26;

[0022] S23, for office documents, extracting level 2 data fingerprints;

[0023] S24, judging the data fingerprint level to be extracted according to the file classification and importance; if further extraction is not required, skipping to step S26;

[0024] S25, extracting data fingerprints of level 3, 4 or 5 according to requirements;

[0025] S26, storing the extracted data fingerprint into the log audit library.

[0026] Furthermore, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0027] Classify the confidentiality of documents into four levels: non-confidential, internal, secret, confidential and above;

[0028] The importance of files is judged manually or by keywords, and is divided into two types: general and important. ① Manual judgment is performed by the reviewer who sends the file for review and manually marks it. ② Keyword judgment is performed by the management personnel who design keywords based on the characteristics of the application system where the data exchange system is located. If the file name contains keywords, it is considered to be an important file.

[0029] Furthermore, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0030] Design the principles for extracting hierarchical data fingerprints, as shown in Table 1 below:

[0031] Table 1 Principles of hierarchical data fingerprint extraction

[0032] Data fingerprint level extraction File classification and importance Level 1 fingerprint All files Level 2 fingerprint Non-confidential office documents Level 3 fingerprint General internal office documents Level 4 fingerprint Important internal office documents, general confidential office documents 5-level fingerprint Important secret office documents, confidential and above office documents .

[0033] Further, in step S3, the audit backtracking of the behavior based on the data content specifically includes the following sub-steps: if it is found that malicious code is spreading in the receiving network domain or files containing sensitive information are leaked, it is necessary to restore and trace the exchange behavior based on the recorded data fingerprint, and the restoration and tracing process specifically includes the following sub-steps:

[0034] S31, calculate the level 1 fingerprint of the file and compare it with the level 1 fingerprint of the fingerprint library; if a match is found, the process ends; otherwise, continue to execute the subsequent steps;

[0035] S32, file type determination, if it is an office file, calculate the second-level fingerprint of the file, compare it with the second-level fingerprint of the fingerprint library, and end if a match is found; otherwise, continue to execute subsequent steps;

[0036] S33, judge the importance of the file. If it is a non-confidential office file, end; if it is a general internal office file, calculate the level 3 fingerprint of the file and compare it with the level 3 fingerprint in the fingerprint library; if it is an important internal office file and a general secret office file, calculate and compare the level 3 and level 4 fingerprints of the file in turn; for important secret office files and confidential and above office files, calculate and compare the level 3, level 4 and level 5 fingerprints of the file in turn.

[0037] Furthermore, in step S3, the audit backtracking of the behavior based on the data content specifically includes the following sub-steps:

[0038] During fingerprint matching, if a single fingerprint is equal to the fingerprint value in the fingerprints to be matched, the fingerprint is matched; if it is not equal to all the fingerprints to be matched, the fingerprint is not matched; if the 1st and 2nd level fingerprint matching hits are 100% or the 3rd, 4th and 5th level fingerprint matching hits exceed the set threshold, an alarm is triggered;

[0039] For level 3 and 4 fingerprints, the percentage of the number of matching fingerprints to the total number of fingerprints is calculated. If it exceeds the threshold, the file is a hit and is a file that has been transferred before;

[0040] For level 5 fingerprints, the comparison is performed according to the groups. During the matching process of each group, if the threshold is exceeded, the group is considered to have hit; the hit rate of all groups is then calculated, and if the threshold is exceeded, the level 5 fingerprint is considered to have hit;

[0041] When comparing data fingerprints at levels 3, 4, and 5, a threshold for the match hit rate is set. When the threshold is exceeded, the file contents are considered to be basically similar. The higher the match hit rate, the more closely the content matches. Users can set the threshold based on actual needs.

[0042] A hierarchical data fingerprint audit and tracing device in data exchange includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, any of the methods described above is executed.

[0043] The beneficial effects of the present invention include:

[0044] Aiming at the demand for auditing, analyzing and tracing the exchange behavior of exchanged data in the field of data exchange, the present invention proposes an audit tracing method based on multi-level data fingerprints. The method optimizes the data fingerprint extraction and audit tracing process, and greatly saves the system's computing and storage resources while ensuring the accuracy of the audit tracing of exchange behavior. It has important practical significance for the actual construction and operation of the data exchange system. The design of the method can also be further extended to other similar demand scenarios.

[0045] The method of the present invention can judge the type of exchanged files, classify them based on the confidentiality and importance of file data, use different data fingerprint calculation methods to calculate the fingerprint of exchanged data, and provide an efficient and accurate content-based exchange behavior audit and tracing process. The method reduces the system's demand for computing resources and storage resources, and has important practical significance for improving the construction of data exchange systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A hierarchical data fingerprint calculation method according to an embodiment of the present invention;

[0048] Figure 2 The data fingerprint extraction process of the data exchange file in the embodiment of the present invention;

[0049] Figure 3 The present invention is a flowchart of a behavior audit and tracing process based on data content according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0051] In view of the problems in the background, the present invention aims at the need to realize content-based exchange behavior audit analysis and behavior backtracking in the field of data exchange without saving the original data, and proposes an audit tracing method based on multi-level data fingerprints. Further, the specific implementation process of the present invention is as follows:

[0052] 1. Hierarchical data fingerprint calculation

[0053] The data fingerprint calculation method of the present invention uses a hash algorithm and adopts different levels of processing methods based on the different classifications of data files.

[0054] The hierarchical data fingerprint method is divided into five levels of data fingerprints according to the different granularities of extraction. The higher the level, the finer the granularity, the higher the accuracy of content-based detection, but the greater the consumption of computing resources and storage resources; the lower the level, the lower the accuracy of content-based detection, but the lower the consumption of computing and storage resources. Data exchange data calculates data fingerprints in units of files (compressed files need to be decompressed into original files). The hierarchical data fingerprint calculation method is as follows: Figure 1 The specific description is as follows:

[0055] Level 1 fingerprint: This level of data fingerprint extracts the data fingerprint of the entire file, and the calculation result is the data fingerprint of the entire file; each file corresponds to a level 1 data fingerprint;

[0056] Level 2 fingerprint: This level of data fingerprint first needs to extract the file content, then perform data denoising, and finally extract the data fingerprint from the denoised data content; each file corresponds to a level 2 data fingerprint;

[0057] Level 3 fingerprint: This level of data fingerprint needs to extract the file content and divide it into segments according to natural segments, then perform denoising on each natural segment, and finally extract data fingerprints from the denoised data content; each file will extract a set of level 3 data fingerprints, and the number of level 3 data fingerprints is the number of natural segments;

[0058] Level 4 fingerprint: This level of data fingerprint needs to extract the file content and divide it according to sentences (not word segmentation, but can be divided according to punctuation marks such as periods, exclamation marks, question marks, ellipsis marks, etc.), then perform denoising and deduplication operations on each sentence, and finally extract data fingerprints from the denoised data content; each file will extract a set of level 4 data fingerprints;

[0059] Level 5 fingerprint: This level of data fingerprint needs to extract the file content and then divide it into natural segments. Each natural segment uses intelligent natural language semantic analysis to segment the data file content. After segmentation, unimportant modal particles are removed, repeated segmented words are merged and weighted (based on the number of occurrences). Finally, data fingerprints are extracted from the remaining segmented words, and the weights of the data fingerprints are recorded. Multiple sets of level 5 data fingerprints with data weights will be extracted from each file.

[0060] The denoising process mainly includes three aspects: ① Remove spaces from Chinese characters and separate English words with spaces; ② Remove special characters such as punctuation marks; ③ Convert traditional Chinese characters into simplified Chinese characters.

[0061] 2. Fingerprint extraction of data exchange data

[0062] During data exchange, the hierarchical data fingerprint extraction process is as follows: Figure 2 The specific steps are described as follows:

[0063] (1) Perform the first-level data fingerprint calculation on the exchange file;

[0064] (2) Determine the exchange data type to determine whether the exchanged data is an office file (word, pdf, txt, etc., files whose data content can be extracted); if not, jump to (6);

[0065] (3) For office documents, extract level 2 data fingerprints;

[0066] (4) Determine the data fingerprint level to be extracted based on the confidentiality and importance of the file; if further extraction is not required, jump to (6);

[0067] (5) Extract data fingerprints at level 3, 4 or 5 according to requirements;

[0068] (6) Store the extracted data fingerprint into the log audit library.

[0069] Among them, the confidentiality level of files is divided into four levels: non-confidential, internal, secret, confidential and above; the importance of files can be judged manually or by keywords, and is divided into two types: general and important; ① Manual judgment is done by the reviewer who sends the file for manual marking; ② Keyword judgment is done by the management personnel who design keywords (such as technical solutions) based on the characteristics of the application system where the data exchange system is located. If the file name contains keywords, it is considered an important file.

[0070] The principles of data fingerprint extraction are as shown in Table 1 (managers can adjust according to actual needs):

[0071] Table 1 Principles of hierarchical data fingerprint extraction

[0072] Data fingerprint level extraction File classification and importance Level 1 fingerprint All files Level 2 fingerprint Non-confidential office documents Level 3 fingerprint General internal office documents Level 4 fingerprint Important internal office documents, general confidential office documents 5-level fingerprint Important secret office documents, confidential and above office documents

[0073] 3. Behavioral audit backtracking based on data content

[0074] If malicious code is found to be spreading in the receiving network domain or files containing sensitive information are leaked, it is necessary to restore and trace the exchange behavior based on the recorded data fingerprint. Figure 3 , the specific steps are as follows.

[0075] (1) Calculate the level 1 fingerprint of the file and compare it with the level 1 fingerprint in the fingerprint library; if a match is found, the process ends;

[0076] (2) File type determination;

[0077] (3) If it is an office document, calculate the second-level fingerprint of the document and compare it with the second-level fingerprint in the fingerprint database; if a match is found, the process ends;

[0078] (4) Determine the importance of the file; if it is a non-confidential office file, then end the process;

[0079] (5) For general internal office documents, it is also necessary to calculate the level 3 fingerprint of the document and compare it with the level 3 fingerprint in the fingerprint database;

[0080] (6) For important internal office documents and general confidential office documents, the Level 3 and Level 4 fingerprints of the documents need to be calculated and compared in sequence;

[0081] (7) For important secret office documents and confidential or above office documents, it is necessary to calculate and compare the level 3 fingerprint, level 4 fingerprint and level 5 fingerprint of the document in sequence.

[0082] When comparing fingerprints, as an implementation method, if a single fingerprint is equal to the fingerprint value in the fingerprint (group) to be compared, the fingerprint is matched; if it is not equal to all the fingerprints (groups) to be compared, the fingerprint is not matched. Therefore, if the 1st and 2nd level fingerprint matching hits are 100% or the 3rd, 4th, and 5th level fingerprint matching hits exceed the set threshold, an alarm is triggered.

[0083] For level 3 and 4 fingerprints, as an implementation method, it is necessary to calculate the percentage of the number of matching fingerprints to the total number of fingerprints. When the percentage exceeds the threshold, the file is a hit and is a previously transferred file. For level 5 fingerprints, the comparison is performed in groups. During each group matching process, if the percentage exceeds the threshold, the group is considered to have hit; all groups are then hit-rate calculated. If the percentage exceeds the threshold, the level 5 fingerprint is considered to have hit.

[0084] When comparing data fingerprints at levels 3, 4, and 5, as an implementation method, a threshold for the matching hit rate needs to be set. When the threshold is exceeded, the file contents are considered to be basically similar. The higher the matching hit rate, the more the content matches. Users can set the threshold according to actual needs; generally speaking, the threshold for level 3 fingerprints can be set to 20%; the threshold for level 4 fingerprints can be set to 35%; and the threshold for level 5 fingerprints can be set to 50%.

[0085] It should be noted that within the scope of protection defined in the claims of the present invention, the following embodiments can be combined and / or expanded or replaced in any logical way from the above specific implementation methods, such as disclosed technical principles, disclosed technical features or implicitly disclosed technical features.

[0086] Example 1

[0087] A hierarchical data fingerprint audit tracing method in data exchange includes the following steps:

[0088] S1, based on the different classifications of data files, different levels of processing methods are used to calculate the hierarchical data fingerprints;

[0089] S2, extracting hierarchical data fingerprints;

[0090] S3, audit backtracking based on data content behavior.

[0091] Example 2

[0092] On the basis of Example 1, in step S1, the hierarchical data fingerprint is calculated by using different levels of processing methods based on different levels of data files, which specifically includes the following sub-steps:

[0093] The data fingerprint is divided into five levels according to the different granularities of extraction, and the data fingerprint is calculated in units of files:

[0094] Level 1 fingerprint: This level of data fingerprint extracts the data fingerprint of the entire file, and the calculation result is the data fingerprint of the entire file; each file corresponds to a level 1 data fingerprint;

[0095] Level 2 fingerprint: This level of data fingerprint first extracts the file content, then performs data denoising, and finally extracts data fingerprints from the denoised data content; each file corresponds to a level 2 data fingerprint;

[0096] Level 3 fingerprint: This level of data fingerprint extracts the file content and divides it into segments according to natural segments, then performs denoising operation on each natural segment, and finally extracts data fingerprint from the data content after denoising operation; a set of level 3 data fingerprints is extracted from each file, and the number of level 3 data fingerprints is the number of natural segments;

[0097] Level 4 fingerprint: This level of data fingerprint extracts the file content and divides it according to sentences, then performs denoising and deduplication operations on each sentence, and finally extracts data fingerprints from the denoised data content; each file will extract a set of level 4 data fingerprints;

[0098] Level 5 fingerprint: This level of data fingerprint requires the extraction of file content, which is then divided into natural segments. Each segment uses natural language semantic analysis to segment the data file content. After segmentation, unimportant modal particles are removed, repeated segmentations are merged and weighted. Finally, data fingerprints are extracted from the remaining segmentations, and the weights of the data fingerprints are recorded. Multiple sets of level 5 data fingerprints with data weights are extracted from each file.

[0099] Example 3

[0100] Based on Example 1, the denoising operation specifically includes three contents: ① removing spaces from Chinese characters and separating English words with spaces; ② removing punctuation marks; ③ converting traditional Chinese characters into simplified Chinese characters.

[0101] Example 4

[0102] On the basis of Example 2, the step of merging repeated participles and assigning weights specifically includes: merging repeated participles and assigning weights according to the number of occurrences.

[0103] Example 5

[0104] On the basis of Example 2, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0105] S21, performing first-level data fingerprint calculation on the exchange file;

[0106] S22, judging the exchange data type, whether the exchange data is an office file; if it is not an office file, jumping to step S26;

[0107] S23, for office documents, extracting level 2 data fingerprints;

[0108] S24, judging the data fingerprint level to be extracted according to the file classification and importance; if further extraction is not required, skipping to step S26;

[0109] S25, extracting data fingerprints of level 3, 4 or 5 according to requirements;

[0110] S26, storing the extracted data fingerprint into the log audit library.

[0111] Example 6

[0112] On the basis of Example 2, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0113] Classify the confidentiality of documents into four levels: non-confidential, internal, secret, confidential and above;

[0114] The importance of files is judged manually or by keywords, and is divided into two types: general and important. ① Manual judgment is performed by the reviewer who sends the file for review and manually marks it. ② Keyword judgment is performed by the management personnel who design keywords based on the characteristics of the application system where the data exchange system is located. If the file name contains keywords, it is considered to be an important file.

[0115] Example 7

[0116] On the basis of Example 6, in step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps:

[0117] Design the principles for extracting hierarchical data fingerprints, as shown in Table 1 below:

[0118] Table 1 Principles of hierarchical data fingerprint extraction

[0119] Data fingerprint level extraction File classification and importance Level 1 fingerprint All files Level 2 fingerprint Non-confidential office documents Level 3 fingerprint General internal office documents Level 4 fingerprint Important internal office documents, general confidential office documents 5-level fingerprint Important secret office documents, confidential and above office documents .

[0120] Example 8

[0121] On the basis of Example 7, in step S3, the audit backtracking of the behavior based on the data content specifically includes the following sub-steps: if it is found that malicious code is spreading in the receiving network domain or files containing sensitive information are leaked, it is necessary to restore and trace the exchange behavior based on the recorded data fingerprint, and the restoration and tracing process specifically includes the following sub-steps:

[0122] S31, calculate the level 1 fingerprint of the file and compare it with the level 1 fingerprint of the fingerprint library; if a match is found, the process ends; otherwise, continue to execute the subsequent steps;

[0123] S32, file type determination, if it is an office file, calculate the second-level fingerprint of the file, compare it with the second-level fingerprint of the fingerprint library, and end if a match is found; otherwise, continue to execute subsequent steps;

[0124] S33, judge the importance of the file. If it is a non-confidential office file, end; if it is a general internal office file, calculate the level 3 fingerprint of the file and compare it with the level 3 fingerprint in the fingerprint library; if it is an important internal office file and a general secret office file, calculate and compare the level 3 and level 4 fingerprints of the file in turn; for important secret office files and confidential and above office files, calculate and compare the level 3, level 4 and level 5 fingerprints of the file in turn.

[0125] Example 9

[0126] Based on Example 8, in step S3, the audit backtracking based on the behavior of the data content specifically includes the following sub-steps:

[0127] During fingerprint matching, if a single fingerprint is equal to the fingerprint value in the fingerprints to be matched, the fingerprint is matched; if it is not equal to all the fingerprints to be matched, the fingerprint is not matched; if the 1st and 2nd level fingerprint matching hits are 100% or the 3rd, 4th and 5th level fingerprint matching hits exceed the set threshold, an alarm is triggered;

[0128] For level 3 and 4 fingerprints, the percentage of the number of matching fingerprints to the total number of fingerprints is calculated. If it exceeds the threshold, the file is a hit and is a file that has been transferred before;

[0129] For level 5 fingerprints, the comparison is performed according to the groups. During the matching process of each group, if the threshold is exceeded, the group is considered to have hit; the hit rate of all groups is then calculated, and if the threshold is exceeded, the level 5 fingerprint is considered to have hit;

[0130] When comparing data fingerprints at levels 3, 4, and 5, a threshold for the match hit rate is set. When the threshold is exceeded, the file contents are considered to be basically similar. The higher the match hit rate, the more closely the content matches. Users can set the threshold based on actual needs.

[0131] Example 10

[0132] A hierarchical data fingerprint audit tracing device in data exchange includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method described in any one of Examples 1 to 9 is executed.

[0133] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0134] According to one aspect of an embodiment of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations.

[0135] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

Claims

1. A hierarchical data fingerprint audit tracing method in data exchange, characterized in that: The following steps are involved: S1, based on the different classifications of data files, different levels of processing methods are used to calculate the hierarchical data fingerprints; S2, extracting hierarchical data fingerprints; S3, audit backtracking based on data content behavior.

2. The hierarchical data fingerprint audit tracing method in data exchange according to claim 1 is characterized in that: In step S1, based on different classifications of data files, different levels of processing methods are used to calculate the hierarchical data fingerprint, which specifically includes the following sub-steps: The data fingerprint is divided into five levels according to the different granularities of extraction, and the data fingerprint is calculated in units of files: Level 1 fingerprint: This level of data fingerprint extracts the data fingerprint of the entire file, and the calculation result is the data fingerprint of the entire file; each file corresponds to a level 1 data fingerprint; Level 2 fingerprint: This level of data fingerprint first extracts the file content, then performs data denoising, and finally extracts data fingerprints from the denoised data content; each file corresponds to a level 2 data fingerprint; Level 3 fingerprint: This level of data fingerprint extracts the file content and divides it into segments according to natural segments, then performs denoising operation on each natural segment, and finally extracts data fingerprint from the data content after denoising operation; a set of level 3 data fingerprints is extracted from each file, and the number of level 3 data fingerprints is the number of natural segments; Level 4 fingerprint: This level of data fingerprint extracts the file content and divides it according to sentences, then performs denoising and deduplication operations on each sentence, and finally extracts data fingerprints from the denoised data content; each file will extract a set of level 4 data fingerprints; Level 5 fingerprint: This level of data fingerprint requires extracting the file content and then dividing it into natural segments. Each natural segment uses natural language semantic analysis to perform word segmentation on the data file content. After word segmentation, unimportant modal particles are removed, repeated word segmentations are merged and assigned weights; finally, data fingerprints are extracted from the remaining word segmentations, and the weights of the data fingerprints are recorded; multiple groups of 5-level data fingerprints with data weights are extracted from each file.

3. The hierarchical data fingerprint audit tracing method in data exchange according to claim 1 is characterized in that: The denoising operation specifically includes three contents: ① removing spaces from Chinese characters and separating English words with spaces; ② removing punctuation marks; ③ converting traditional Chinese characters into simplified Chinese characters.

4. The hierarchical data fingerprint audit tracing method in data exchange according to claim 2 is characterized in that: The step of merging repeated participles and assigning weights specifically includes: merging repeated participles and assigning weights according to the number of occurrences.

5. The hierarchical data fingerprint audit tracing method in data exchange according to claim 2 is characterized in that: In step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps: S21, performing first-level data fingerprint calculation on the exchange file; S22, judging the exchange data type, whether the exchange data is an office file; if it is not an office file, jumping to step S26; S23, for office documents, extracting level 2 data fingerprints; S24, judging the data fingerprint level to be extracted according to the file classification and importance; if further extraction is not required, skipping to step S26; S25, extracting data fingerprints of level 3, 4 or 5 according to requirements; S26, storing the extracted data fingerprint into the log audit library.

6. The hierarchical data fingerprint audit tracing method in data exchange according to claim 2 is characterized in that: In step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps: Classify the confidentiality of documents into four levels: non-confidential, internal, secret, confidential and above; The importance of files is judged manually or by keywords, and is divided into two types: general and important. ① Manual judgment is performed by the reviewer who sends the file for review and manually marks it. ② Keyword judgment is performed by the management personnel who design keywords based on the characteristics of the application system where the data exchange system is located. If the file name contains keywords, it is considered to be an important file.

7. The hierarchical data fingerprint audit tracing method in data exchange according to claim 6 is characterized in that: In step S2, the hierarchical data fingerprint extraction specifically includes the following sub-steps: Design the principles for extracting hierarchical data fingerprints, as shown in Table 1 below: Table 1 Principles of hierarchical data fingerprint extraction 。 8. The hierarchical data fingerprint audit tracing method in data exchange according to claim 7 is characterized in that: In step S3, the audit backtracking of the behavior based on the data content specifically includes the following sub-steps: if it is found that malicious code is spreading in the receiving network domain or files containing sensitive information are leaked, it is necessary to restore and trace the exchange behavior based on the recorded data fingerprint, and the restoration and tracing process specifically includes the following sub-steps: S31, calculate the level 1 fingerprint of the file and compare it with the level 1 fingerprint of the fingerprint library; if a match is found, the process ends; otherwise, continue to execute the subsequent steps; S32, file type determination, if it is an office file, calculate the second-level fingerprint of the file, compare it with the second-level fingerprint of the fingerprint library, and end if a match is found; otherwise, continue to execute subsequent steps; S33, judge the importance of the file. If it is a non-confidential office file, end; if it is a general internal office file, calculate the level 3 fingerprint of the file and compare it with the level 3 fingerprint in the fingerprint library; if it is an important internal office file and a general secret office file, calculate and compare the level 3 and level 4 fingerprints of the file in turn; for important secret office files and confidential and above office files, calculate and compare the level 3, level 4 and level 5 fingerprints of the file in turn.

9. The hierarchical data fingerprint audit tracing method in data exchange according to claim 8 is characterized in that: In step S3, the audit backtracking of the behavior based on the data content specifically includes the following sub-steps: During fingerprint matching, if a single fingerprint is equal to the fingerprint value in the fingerprints to be matched, the fingerprint is matched; if it is not equal to all the fingerprints to be matched, the fingerprint is not matched; if the 1st and 2nd level fingerprint matching hits are 100% or the 3rd, 4th and 5th level fingerprint matching hits exceed the set threshold, an alarm is triggered; For level 3 and 4 fingerprints, the percentage of the number of matching fingerprints to the total number of fingerprints is calculated. If it exceeds the threshold, the file is a hit and is a file that has been transferred before; For level 5 fingerprints, the comparison is performed according to the groups. During the matching process of each group, if the threshold is exceeded, the group is considered to have hit; the hit rate of all groups is then calculated, and if the threshold is exceeded, the level 5 fingerprint is considered to have hit; When comparing data fingerprints at levels 3, 4, and 5, a threshold for the match hit rate is set. When the threshold is exceeded, the file contents are considered to be basically similar. The higher the match hit rate, the more closely the content matches. Users can set the threshold based on actual needs.

10. A hierarchical data fingerprint audit and tracing device in data exchange, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is loaded by the processor, the method according to any one of claims 1 to 9 is executed.

Citation Information

Patent Citations

  • Data tracing method and system for data exchange

    CN115766020A

  • Data cross-domain exchange behavior auditing and tracking method and data cross-domain exchange system

    CN116418587A

  • Hierarchical data fingerprint auditing and tracing method in cross-network exchange

    CN116723049A

  • Text duplicate checking method, electronic device and computer-readable storage medium

    US20230122661A1