Method, system and device for deep mining of hidden information in electronic documents

Abstract

The present application discloses a method, system and device for deep mining of hidden information in electronic documents. First, the binary format of electronic documents is deeply analyzed to parse the formats of various types of files; then, binary structural features are extracted based on the format analysis, and various data hiding methods are analyzed in detail to build a knowledge base of structural features of hidden information in electronic documents; finally, by calling the knowledge base, a deep mining engine for hidden information in electronic documents is designed, and the mining results are displayed based on the display template. Automatic batch inspection of electronic documents with sensitive information and setting protection measures for them can effectively block the leakage loopholes of electronic documents. And through the technical solution of building a knowledge base, any type of electronic document can be quickly expanded into the knowledge base, the supported file types and future data hiding methods can be expanded, and the detection of electronic documents in extended formats can be realized, so that the system has good scalability.

Description

Technical Field

[0001] The present application belongs to the field of network security technology, and specifically, relates to a method, system and device for deep mining of hidden information in electronic documents. Background Art

[0002] After the comprehensive informatization of enterprise business, each terminal stores a large number of documents, and various important and ordinary documents are mixed together, making it difficult for administrators to determine which documents contain sensitive information, and it is impossible to cover all documents containing sensitive information with control measures. On the other hand, the operation applications of terminals are becoming more and more abundant, and many network channels provide convenience for document transmission. It is also difficult to detect that documents containing sensitive information are uploaded, sent out, and modified. The binary formats of various types of electronic documents vary greatly, and it is extremely easy to hide information in them. Hiding and transmitting sensitive information in electronic document data has become one of the main means of internal data leakage.

[0003] Data leakage protection uses technical means to prevent data assets from leaking out of the enterprise in the form of violating security policies. Current data leakage protection methods mainly include internal data encryption, identity authentication, data flow control, storage hardware management, outbound control, etc. Based on the urgent need for information security, it is of great significance to develop electronic document information hiding mining methods and systems.

[0004] In the related technologies, the proposed information security solutions still rely on firewalls, intrusion detection, network antivirus and other methods. These methods are relatively general and rough, lacking the judgment and detection of information content, and unable to refine the management and control methods and means. Summary of the invention

[0005] Therefore, the embodiments of the present application provide a method, system and device for deep mining of hidden information in electronic documents, aiming to solve at least one problem existing in the above-mentioned prior art.

[0006] To achieve the above objectives, in a first aspect, the present application provides a method for deep mining of hidden information in electronic documents, comprising:

[0007] In response to a user uploading a target file to be detected and submitting a detection task, the target file is saved in a preset file system and a task entry is created in a task database, and task attributes are written into the task entry and then added to a task queue, wherein the task attributes include a save path of the target file in the file system;

[0008] After the preset mining engine polls the task entry in the task queue, it searches for the target file according to the save path of the target file, parses the binary structure of the target file and extracts the binary structure comprehensive features of the target file, wherein the binary structure comprehensive features are used to identify the file type of the target file;

[0009] The mining engine matches the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scans the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with knowledge representations of file types corresponding to multiple types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and data hiding method analysis methods, and is configured with the hidden information mining rule, which is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method;

[0010] The hidden information detection result is used to generate a detection result display file based on a preset result display template and is saved to a specified path.

[0011] Optionally, the comprehensive characteristics of the binary structure include: a file header in binary format, a file data block description and a file tail feature, the file header in binary format includes a file type feature description, file length check information, a file content description, and a data block starting address, and the file tail feature includes a file end identifier and a file tail feature description information.

[0012] Optionally, the data hiding methods include: file type tampering, multiple compression of compressed packages, encryption hiding, Office redundant data hiding, Office document embedding hiding, data stream hiding, merged file hiding, metadata hiding, and image content hiding. Correspondingly, the data hiding method analysis method includes: file type inspection, compressed package scanning, file encryption scanning, Office document structure inspection, Office hidden data extraction detection, data stream detection, merged file scanning, fast scanning of keywords in files, and fast search of image content. When scanning the target file, the corresponding data hiding method is determined according to the file type of the target file, and the corresponding data hiding method analysis method is determined according to the data hiding method. Then, according to the determined data hiding method analysis method, the target file is scanned and parsed based on the mining process to obtain the hidden information corresponding to the target file, and the data hiding method and hidden information are determined as the hidden information detection result.

[0013] Optionally, the data hiding method analysis method determines whether the data hiding method is used for analysis or not, and scans the target file based on the mining process to obtain hidden information corresponding to the target file, including: analyzing the binary structure of the target file based on the mining process according to a preset knowledge base matching inference algorithm corresponding to the data hiding method analysis method to extract the hidden information.

[0014] Optionally, the method for analyzing the data hiding method determines the data hiding method, and scans the target file based on the mining process to obtain hidden information corresponding to the target file, including: performing one or more scans of the following mining processes on the binary format of the target file according to the method for analyzing the data hiding method:

[0015] File type check, identifying whether the content of the target file conforms to the format declared by its suffix name. If not, it is reported as a suspicious file;

[0016] File encryption scanning to check whether the file is encrypted. If it is encrypted, it will be reported as a suspicious file;

[0017] Data stream detection, checking whether the target file has additional NTFS data streams. If so, it is reported as a suspicious file;

[0018] Merge file scanning to check whether there is tail splicing data at the tail of the target file. If so, report it as a suspicious file;

[0019] Office document structure check: Checks whether there is redundant data in the file structure of the Office document. If so, it is reported as a suspicious file.

[0020] Office hidden data extraction detection, checking whether there are OLE objects embedded in Office documents. If so, all embedded OLE objects are extracted and reported as suspicious files;

[0021] Quickly scan keywords in files to see if each byte in the target file is in the preset keyword table. If so, report it as a suspicious file;

[0022] Image content quick search, optical character recognition of image content, converting image content into text content and then performing a quick keyword scan in the file. If the bytes in the text content are in the preset keyword table, it will be reported as a suspicious file;

[0023] Compressed package scanning: after decompressing the compressed package layer by layer, return to the file type verification step and re-execute it to determine whether the compressed package is a suspicious file;

[0024] The results of all data hiding analysis methods performed on the target file are determined as the hidden information.

[0025] Optionally, it also includes: the preset result display template is an html file template or an xml file template, and after the hidden information and the hiding method are correspondingly associated, the detection result display file is generated according to the preset result display template.

[0026] Optionally, after matching the file type corresponding to the binary structure comprehensive feature in a preset knowledge base and before scanning the target file, it also includes: determining whether the file type is in a preset whitelist, if so, scanning the target file, if not, directly generating the hidden information detection result.

[0027] Optionally, it also includes: in response to a user's request for the task status of the detection task, querying the task status of the task entry in the task database, and if the task status is completed, reading the detection result display file from the specified path and returning it to the user's browser.

[0028] In a second aspect, the present application also provides a system for deep mining of hidden information in electronic documents, including:

[0029] A request identification module, for responding to a user uploading a target file to be detected and submitting a detection task, saving the target file to a preset file system and creating a task entry in a task database, writing task attributes to the task entry and adding it to a task queue, wherein the task attributes include a saving path of the target file in the file system;

[0030] A parsing module, which is used for searching the target file according to the save path of the target file after the preset mining engine polls the task entry in the task queue, parsing the binary structure of the target file and extracting the binary structure comprehensive features of the target file, wherein the binary structure comprehensive features are used to identify the file type of the target file;

[0031] A mining engine module, used for matching the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scanning the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with knowledge representations of file types corresponding to multiple types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and data hiding method analysis methods, and is configured with the hidden information mining rule, the hidden information mining rule is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method;

[0032] The result generation module is used to input the hidden information detection result into a preset result display template to generate a detection result display file and save it to a specified path.

[0033] In a third aspect, the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the electronic document information hidden deep mining method.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the steps of the electronic document information hidden deep mining method.

[0035] The embodiment of the present application provides a method, system and device for deep mining of hidden information in electronic documents. First, the binary format of the electronic document is deeply analyzed to parse the formats of various types of files; then, the binary structure features are extracted based on the format analysis, and various data hiding methods are analyzed in detail to build a knowledge base of hidden information structure features of electronic documents; finally, by calling the knowledge base, a deep mining engine for hidden information in electronic documents is designed, and the mining results are displayed based on the display template. Automatic batch inspection of electronic documents with sensitive information and setting protection measures for them can effectively block the leakage loopholes of electronic documents. And through the technical solution of building a knowledge base, any type of electronic document can be quickly expanded into the knowledge base, the supported file types and future data hiding methods can be expanded, and the detection of electronic documents in extended formats can be realized, which makes the system have good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of this application are used to provide a further understanding of this application, so that other features, purposes and advantages of this application become more obvious. The schematic embodiment drawings and their descriptions of this application are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0037] Figure 1 A framework diagram of knowledge representation of the electronic document information hidden deep mining method provided in the embodiment of the present application;

[0038] Figure 2 A diagram for constructing a knowledge base of the electronic document information hiding deep mining method provided in an embodiment of the present application;

[0039] Figure 3 A flowchart of the hidden information mining process of the electronic document information hidden deep mining method provided in the embodiment of the present application;

[0040] Figure 4A flowchart of a hidden information search process of a method for deep mining of hidden electronic document information provided in an embodiment of the present application;

[0041] Figure 5 A system architecture diagram of the electronic document information hidden deep mining method provided in the embodiment of the present application;

[0042] Figure 6 A processing flow chart of the electronic document information hidden deep mining method provided in the embodiment of the present application;

[0043] Figure 7 A schematic diagram of the main modules of the electronic document information hiding deep mining system provided in an embodiment of the present application;

[0044] Figure 8 An exemplary system architecture diagram that can be applied to the embodiments of the present application;

[0045] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0047] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] In addition, some of the above terms may be used to express other meanings in addition to indicating orientation or positional relationship. For example, the term "on" may also be used to express a certain dependency or connection relationship in some cases. For those of ordinary skill in the art, the specific meanings of these terms in this application can be understood according to specific circumstances.

[0049] In addition, the term "plurality" shall mean two or more.

[0050] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. For ease of description, only the parts related to the embodiments of the present application are shown, which are described in detail as follows:

[0051] A method for deep mining of electronic document information hiding comprises the following steps:

[0052] Step 1: In response to a user uploading a target file to be detected and submitting a detection task, the target file is saved in a preset file system and a task entry is created in a task database. Task attributes are written into the task entry and then added to a task queue. The task attributes include a save path of the target file in the file system.

[0053] Step 2: After the preset mining engine polls the task entry in the task queue, it searches for the target file according to the save path of the target file, parses the binary structure of the target file and extracts the binary structure comprehensive features of the target file, where the binary structure comprehensive features are used to identify the file type of the target file;

[0054] Step 3: The mining engine matches the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scans the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with knowledge representations of file types corresponding to various types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and data hiding method analysis methods, and is configured with the hidden information mining rule, which is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method;

[0055] Step 4: Generate a detection result display file based on the preset result display template and save it to a specified path.

[0056] In step 1, the user can upload the target file to be detected at the detection end of the application system of the method, and click to submit the task after uploading the target file. Based on the response to the user's submission of the task, the target file uploaded by the user is saved in the preset file system and a task entry is created in the task database. Then, the task entry is written into the task attribute and added to the task queue so that the task is polled and processed. Here, a file system for storing the target file uploaded by the user to be detected and a task database for user cache tasks can be preset. The task attribute is various information of the task entry, which is used to identify and process the task, such as including the save path of the target file so that the target file can be obtained and scanned when processing the task.

[0057] In step 2, after the preset mining engine polls the task entry in the task queue, it searches for the target file according to the save path of the target file, parses the binary structure of the target file and extracts the binary structure comprehensive features of the target file, and the binary structure comprehensive features are used to identify the file type of the target file. In this embodiment, a mining engine is preset, and the mining engine is used to analyze, process and calculate various tasks in the detection and obtain results. It should be noted that the mining engine can be a module of a system, or it can be composed of multiple components or modules.

[0058] Here, the preset mining engine continuously polls the pending task items in the task queue according to the set polling time, and processes the polled task items. After polling the task item, the mining engine finds the corresponding target file according to the save path in the task item, and then parses the target file to obtain the detailed binary structure of the target file, and at the same time extracts the binary structure comprehensive features of the target file from the detailed binary structure obtained by parsing, and the binary structure comprehensive features are used to identify the file type of the target file. It should be noted that different types of electronic documents have different binary structure comprehensive features, and the file type of the corresponding electronic document can be identified by the binary file comprehensive features.

[0059] In one embodiment, the binary structure comprehensive features include: a binary file header, a file data block description, and a file tail feature, wherein the binary file header includes a file type feature description, file length check information, a file content description, and a data block start address, and the file tail feature includes a file end identifier and a file tail feature description information. Thus, the file type of the target file can be accurately analyzed through the binary structure comprehensive features, so as to facilitate the subsequent hidden information mining based on the knowledge base.

[0060] In step 3, the mining engine matches the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scans the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with file types corresponding to various types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and knowledge representations between data hiding method analysis methods, and is configured with the hidden information mining rule, which is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method.

[0061] Here, by presetting the knowledge base, the mining engine is guided so that the mining engine can perform hidden information mining scanning on the target file based on the file types of different types of electronic documents in the knowledge base, the corresponding binary structure comprehensive characteristics, data hiding methods, knowledge representation between data hiding method analysis methods, and the configured hidden information mining strategy.

[0062] In one embodiment, after matching the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, before scanning the target file, it also includes: determining whether the file type is in a preset whitelist, if so, scanning the target file, if not, directly generating the hidden information detection result.

[0063] It should be noted that when constructing the knowledge base, it is possible to collect various commonly used types of electronic documents and analyze different types of electronic documents to obtain various commonly known information hiding methods, and then design corresponding data hiding method analysis methods, establish knowledge representations between file types, corresponding binary structure comprehensive characteristics, data hiding methods, and data hiding method analysis methods, and then construct the knowledge base.

[0064] For example, in this embodiment, the common electronic document detailed formats are analyzed mainly based on file types such as Office, images, compressed files, and PDF, and security analysis is performed on the file structure. By analyzing the binary structures of various commonly used document files, corresponding data hiding method analysis methods are designed for some possible data (information) hiding methods. For example, common possible hiding methods include file type tampering, multiple compression of compressed packages, encryption hiding, Office redundant data hiding, Office document embedding hiding, data stream hiding, merged file hiding, metadata hiding, and image content hiding; correspondingly, the data hiding method analysis method can be designed as: file type inspection, compressed package scanning, file encryption scanning, Office document structure inspection, Office hidden data extraction detection, data stream detection, merged file scanning, fast scanning of keywords in files, and fast search of image content.

[0065] in:

[0066] (1) File type verification: It can check the original type of the file and whether the actual type of the file content is consistent with the suffix name. If the file suffix name has been artificially changed, the file may contain hidden information, and thus the data hiding method of file type tampering can be identified;

[0067] (2) Compressed package scanning: For ZIP and RAR compressed packages, all files can be decompressed and scanned. If the files being decompressed still contain compressed packages, they will continue to be decompressed layer by layer until all sub-files in all compressed packages are scanned to ensure that no files are missed. After decompression, the decompressed files are analyzed by other data hiding methods to identify the data hiding methods of multiple compressions of compressed packages;

[0068] (3) File encryption scanning: Office documents, RAR and ZIP compressed files, and PDF documents all support document encryption features. This may be a deliberate attempt to hide content. The file encryption scanning method can identify in batches whether documents are encrypted and further identify the encryption and hiding methods.

[0069] (4) Office document structure check: Check the contents of Office docx, xlsx, pptx and other types of files, find Office documents with redundant files, extract the redundant files, and then detect the hiding methods of Office redundant data;

[0070] (5) Office hidden data extraction detection: Various types of files can be embedded in Office documents as OLE objects. When the OLE object becomes extremely small or hides behind an image in the document, the normal process review cannot find the hidden OLE object. This method analyzes the binary structure of various versions of Office document types to extract all embedded OLE objects and restore their contents, thereby realizing the detection of Office document embedding and hiding methods;

[0071] (6) Data stream detection: NTFS data stream is a feature of the NTFS disk format, but it is often used to hide data, and it is a very covert method of hiding data. There will be no abnormality in reading, writing, copying, or even viewing file attributes of files carrying NTFS data streams. This method identifies and extracts NTFS data streams into separate files, thereby realizing the detection of data stream hiding methods;

[0072] (7) Merged file scanning: Use the Windows command line command copy file1+file2+…file3 to connect multiple files into one. When opening the connected files, only the first one will be opened, and the subsequent ones will be ignored. This method identifies such files that are merged from multiple files and cuts out the redundant data at the end of the file, thereby realizing the detection of the hidden method of the merged file;

[0073] (8) Quick scanning of keywords in files: Find out whether the file contains sensitive keywords. This method searches for keywords in the text content of Office, PDF, RAR, ZIP and other files, supports hierarchical management of keywords, provides keywords of multiple risk levels, and gives a report on the danger level of the file based on the search results, thereby realizing the detection of metadata hiding methods for keywords;

[0074] (9) Rapid search of image content: Images may also contain some text content. This method can quickly and accurately identify Chinese and English characters in images and quickly search for keywords, thereby realizing the detection of hidden methods of image content.

[0075] It should be noted that the various data hiding analysis methods designed above can be implemented using various existing known algorithms, such as OCR recognition algorithms, data parsing algorithms, etc., or various reasoning algorithms based on knowledge bases, which will not be repeated here.

[0076] The electronic documents of the collected samples are analyzed in binary format by the above-designed data hiding analysis method. Since there are many types of files and the binary formats of various types of files vary greatly, it is necessary to analyze the binary formats of the files one by one to extract the comprehensive features that best represent the files of this type. Therefore, based on the analysis results, the binary structure comprehensive features are extracted. Here, the electronic document binary structure comprehensive features include: binary format file header (the information of the file header includes file type feature description, file length check information, file content description, data block start address), file data block description, file tail feature (file tail feature includes file end identifier and file tail feature description information).

[0077] Furthermore, the extracted binary structure comprehensive features are combined with the file type, data hiding method and the corresponding data hiding method analysis method to establish the knowledge representation of data hiding of the collected samples, such as Figure 1 As shown. Furthermore, Figure 2 As shown, the knowledge base of this embodiment is constructed to facilitate subsequent identification and scanning of electronic documents of known file types in the knowledge base directly based on the knowledge base.

[0078] In this embodiment, a mining engine is constructed based on a knowledge base, and various detection tasks are processed by the mining engine. The mining engine can be developed based on the above knowledge base, and the mining engine can be designed as an electronic document hidden information mining expert system, including a knowledge base, a task engine, a configuration module and a result display template module.

[0079] In one embodiment, the data hiding methods include: file type tampering, multiple compression of compressed packages, encryption hiding, Office redundant data hiding, Office document embedding hiding, data stream hiding, merged file hiding, metadata hiding, and image content hiding. Correspondingly, the data hiding method analysis method includes: file type inspection, compressed package scanning, file encryption scanning, Office document structure inspection, Office hidden data extraction detection, data stream detection, merged file scanning, fast scanning of keywords in files, and fast search of image content; when scanning the target file, the corresponding data hiding method is determined according to the file type of the target file, and the corresponding data hiding method analysis method is determined according to the data hiding method, and then the target file is scanned based on the mining process according to the determined data hiding method analysis method to obtain the hidden information corresponding to the target file, and the data hiding method and hidden information are determined as the hidden information detection result.

[0080] In one embodiment, the data hiding method analysis method determines whether the data hiding method is used for analysis. The method scans the target file based on the mining process to obtain hidden information corresponding to the target file, including: analyzing the binary structure of the target file based on the mining process according to a preset knowledge base matching inference algorithm corresponding to the data hiding method analysis method to extract the hidden information.

[0081] In another embodiment, the method for analyzing the determined data hiding method scans the target file based on the mining process to obtain hidden information corresponding to the target file, including: performing one or more scans of the following mining processes on the binary format of the target file according to the method for analyzing the data hiding method:

[0082] File type check, identifying whether the content of the target file conforms to the format declared by its suffix name. If not, it is reported as a suspicious file;

[0083] File encryption scanning to check whether the file is encrypted. If it is encrypted, it will be reported as a suspicious file;

[0084] Data stream detection, checking whether the target file has additional NTFS data streams. If so, it is reported as a suspicious file;

[0085] Merge file scanning to check whether there is tail splicing data at the tail of the target file. If so, report it as a suspicious file;

[0086] Office document structure check: Checks whether there is redundant data in the file structure of the Office document. If so, it is reported as a suspicious file.

[0087] Office hidden data extraction detection, checking whether there are OLE objects embedded in Office documents. If so, all embedded OLE objects are extracted and reported as suspicious files;

[0088] Quickly scan keywords in files to see if each byte in the target file is in the preset keyword table. If so, report it as a suspicious file;

[0089] Image content quick search, optical character recognition of image content, converting image content into text content and then performing a quick keyword scan in the file. If the bytes in the text content are in the preset keyword table, it will be reported as a suspicious file;

[0090] Compressed package scanning: after decompressing the compressed package layer by layer, return to the file type verification step and re-execute it to determine whether the compressed package is a suspicious file;

[0091] The results of all data hiding analysis methods performed on the target file are determined as the hidden information.

[0092] It should be noted that, in this embodiment, each scanning step in the mining process can be implemented based on various knowledge base reasoning algorithms or data processing algorithms in the prior art, which will not be described in detail here.

[0093] In step 4, the hidden information detection result is used to generate a detection result display file based on a preset result display template and saved to a specified path. Then, the user can directly call and view the detection result to achieve safe management of electronic documents.

[0094] In one embodiment, the preset result display template is an html file template or an xml file template, and the detection result display file is generated according to the preset result display template after the hidden information and the hiding method are correspondingly associated.

[0095] In one embodiment, it also includes: in response to a user's request for the task status of the detection task, querying the task status of the task entry in the task database, and if the task status is completed, reading the detection result display file from the specified path and returning it to the user's browser.

[0096] Here, in order to express the compatibility of this method, two result expressions are adopted: HTML display of hidden information deep mining results and XML display of hidden information deep mining results, and HTML and XML result display templates are designed. The elements in the HTML format report of the scanning results include: inspection time, number of checked folders / files, number of suspicious files, path and file name of each file, and whether the inspection result is suspicious. If the file is suspicious, the danger level and suspicious type are displayed; the meanings of the keywords in the XML format report of the scanning results are shown in the following table.

[0097] Keywords meaning scan_report Inspection Report information Aggregate Information scan_type Check if it is completed generate_time Inspection report completion time file_count Number of files folder_count Number of folders suspicious_count Number of suspicious files results Specific file report file type File Type filepath File path / file name scan_result Scan Results result_description Result Description attachments appendix

[0098] In this embodiment, if Figure 5 As shown, when this method is applied, the system can be designed as a browser / server (B / S) mode. The Web server program is deployed on the server side to provide Web services to the network. In the Web service, it is responsible for interacting with the user's browser and accepting and processing the scanning tasks submitted by the user. In addition, in order to realize the characteristics of multiple users accessing and submitting tasks at the same time, the system supports the task queue function. The specific steps are as follows:

[0099] (1) The user uploads the target file that needs to be security checked from the client browser and submits the task;

[0100] (2) After receiving the task request and the attached file to be detected, the Web server saves the file to be detected to the file system, creates a task entry in the task database, writes various information of the new task (including the path where the file to be scanned is saved), and then waits for the task to be completed;

[0101] (3) The mining engine polls the database regularly. If a new task is found, it reads the task attributes, obtains the path of the file to be detected, and scans the file according to the path. The system supports automatic scanning and manual scanning. The automatic method automatically scans the specified files according to the configuration file, and the manual method manually scans the specified files according to the user's operation. After the scan is completed, the html or xml result report will be checked. The file is saved back to the file system, and the status of the current task entry in the task database is updated to completed;

[0102] (4) When the user requests the task status next time, the Web server program queries the task entry in the database. If the task status is completed, the scan result is read from the file system according to the specified path and returned to the user's browser. The user will see the hidden information mining report of the task in the browser.

[0103] In one embodiment, the mining engine may mainly include two parts.

[0104] The first part is hidden information mining, such as Figure 3 As shown, a mining process is set up. When detecting the target file, the mining engine can process according to the mining process, and the mining process corresponds to the data hiding analysis method. Specifically, when the electronic document mining object (i.e., the target file) arrives, the binary structure is first analyzed to extract the structural features; then, the matching file type is searched in the knowledge base for the extracted features, and the hidden information mining strategy is designed according to the specific framework of the knowledge base; then, the binary logical position of the possible hidden information of the mining object is located according to the mining strategy, and it is determined whether there is hidden information, and if so, it is extracted; finally, the hidden information is mined and a mining report is generated according to the designed template.

[0105] The second part is the keyword mining of hidden information. During the process of mining hidden information in electronic documents, a quick search is also performed on the keyword information. Generally speaking, there are three formats for storing text information in electronic documents: Unicode, Utf8 and ANSI codes. For example, the binary storage format of docx is compressed Utf8 code. Transcoding is required during the search process. First, analyze the binary storage format of the electronic document to be searched and match the encoding type of the keyword. If it is in image format, OCR image content recognition and conversion are required to extract its text content; then customize the content search strategy and report template; finally, based on the search strategy and encoding, quickly search the search object and generate a report based on the search results. The specific implementation process is as follows: Figure 4 shown.

[0106] In an optional embodiment, if Figure 6 The scanning and execution process of a task entry of a target file is as follows:

[0107] (1) First, obtain the target file to be detected;

[0108] (2) Filter the target files by file type, and only the file types listed in the whitelist can pass; verify the file type, identify whether the content of the specified file conforms to the format declared by its suffix name, and report unrecognizable file types as suspicious; check file encryption, scan the file types that support encryption to check whether they are encrypted, and report encrypted files as suspicious; check NTFS data stream, check whether there are additional NTFS data streams in the file, and report files with additional NTFS data streams as suspicious; check tail data, prevent sensitive data from being written to the tail of a normal file, and report files with tail data as suspicious; if the file is not reported as a suspicious file, proceed to the next step of detection;

[0109] (3) Perform the following steps on the Office document: (1) Check the file structure of the Office document, check the image data hiding method, and report the files containing hidden data; (2) Scan the text keywords, and scan each byte of the file in binary mode to compare and search for keywords. The keyword table is defined by the user. If the file contains hidden data or matches the user-defined keywords, the file is reported as a suspicious file; if the detection result is normal, proceed to the next step of detection;

[0110] (4) For image files, perform image optical character recognition, read images of various formats and recognize multiple languages ​​and characters in them, then convert the characters into text and perform text keyword scanning;

[0111] (5) For the compressed file, decompress the files in it, return to the file type filtering step, and scan all the files in the compressed file;

[0112] (6) When the file scan is completed, the report generator collects all the detection results. It organizes the results into mining result display files according to the HTML and XML result display templates and saves them to the specified path. At this point, the task execution is completed.

[0113] Therefore, the electronic document information hidden deep mining method provided by the embodiment of the present application firstly conducts an in-depth analysis of the electronic document binary format and parses the formats of various types of files; then, based on the format analysis, the binary structure features are extracted, and various data hiding methods are analyzed in detail to construct a knowledge base of the structure features of the electronic document hidden information; finally, by calling the knowledge base, a deep mining engine for the electronic document hidden information is designed, and the mining results are displayed based on the display template. Automatic batch inspection of electronic documents with sensitive information and setting protection measures for them can effectively block the leakage loopholes of electronic documents. And through the technical solution of constructing a knowledge base, any type of electronic document can be quickly expanded into the knowledge base, the supported file types and future data hiding methods can be expanded, and the detection of electronic documents in extended formats can be realized, which makes the system have good scalability.

[0114] Figure 7 The main module schematic diagram of the electronic document information hidden deep mining system provided by the embodiment of the present application is shown. For the convenience of explanation, only the part related to the embodiment of the present application is shown, which is described in detail as follows:

[0115] An electronic document information hidden deep mining system 200, comprising:

[0116] The request identification module 201 is used for, in response to a user uploading a target file to be detected and submitting a detection task, saving the target file to a preset file system and creating a task entry in a task database, writing task attributes to the task entry and adding it to a task queue, wherein the task attributes include a saving path of the target file in the file system;

[0117] A parsing module 202, which is used for, after a preset mining engine polls the task entry in the task queue, searching for the target file according to the save path of the target file, parsing the binary structure of the target file and extracting the binary structure comprehensive features of the target file, wherein the binary structure comprehensive features are used to identify the file type of the target file;

[0118] The mining engine module 203 is used to match the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scan the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with the file types corresponding to various types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and knowledge representations between data hiding method analysis methods, and is configured with the hidden information mining rule, which is used to characterize the mining process of determining the hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method;

[0119] The result generation module 204 is used to input the hidden information detection result into a preset result display template to generate a detection result display file and save it to a specified path.

[0120] It should be noted that other embodiments and beneficial effects of the electronic document information hidden deep mining system provided in the embodiment of the present application correspond to the electronic document information hidden deep mining method provided in the embodiment, and will not be repeated here.

[0121] Therefore, the electronic document information hidden deep mining system provided by the embodiment of the present application first conducts an in-depth analysis of the electronic document binary format and parses the formats of various types of files; then, based on the format analysis, the binary structure features are extracted, and various data hiding methods are analyzed in detail to build a knowledge base of the hidden information structure features of electronic documents; finally, by calling the knowledge base, an electronic document hidden information deep mining engine is designed, and the mining results are displayed based on the display template. Automatic batch inspection of electronic documents with sensitive information and setting protection measures for them can effectively block the leakage loopholes of electronic documents. And through the technical solution of building a knowledge base, any type of electronic document can be quickly expanded into the knowledge base, the supported file types and future data hiding methods can be expanded, and the detection of electronic documents in extended formats can be realized, which makes the system have good scalability.

[0122] It should be noted that other embodiments of the electronic document information hidden deep mining system provided in the embodiment of the present application correspond to the embodiments of the electronic document information hidden deep mining method provided in the embodiment of the present application, and will not be repeated here.

[0123] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the electronic document information hidden deep mining method.

[0124] The embodiment of the present application also provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the electronic document information hidden deep mining method of the embodiment of the present application is implemented.

[0125] Figure 8 An exemplary system architecture 300 is shown to which the electronic document information hidden deep mining method or system according to the embodiment of the present application can be applied.

[0126] like Figure 8As shown, the system architecture 300 may include terminal devices 301, 302, 303, a network 304 and a server 305. The network 304 is used to provide a medium for communication links between the terminal devices 301, 302, 303 and the server 305. The network 304 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0127] Users can use terminal devices 301, 302, 303 to interact with server 305 through network 304 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 301, 302, 303, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0128] The terminal devices 301 , 302 , and 303 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0129] The server 305 may be a server providing various services, such as a background management server providing support for the messages sent by the users using the terminal devices 301, 302, 303. The background management server may perform analysis and other processing after receiving the request of the terminal device, and feed back the processing result to the terminal device.

[0130] It should be noted that the electronic document information hidden deep mining method provided in the embodiment of the present application is generally executed by the terminal device 301, 302, 303 or the server 305. Accordingly, the electronic document information hidden deep mining system is generally set in the terminal device 301, 302, 303 or the server 305.

[0131] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0132] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 400 suitable for implementing an electronic device of an embodiment of the present application. Fig. 9 The computer system shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0133] like Fig. 9As shown, the computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the system 400 are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0134] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0135] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above-mentioned functions defined in the system of the present application are executed.

[0136] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The modules involved in the embodiments described in the present application may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a determination module, an extraction module, a training module and a screening module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the determination module may also be described as a "module for determining a candidate user set".

[0139] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0140] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for deep mining of hidden information in electronic documents, characterized in that: include: In response to a user uploading a target file to be detected and submitting a detection task, the target file is saved in a preset file system and a task entry is created in a task database, and task attributes are written into the task entry and then added to a task queue, wherein the task attributes include a save path of the target file in the file system; After the preset mining engine polls the task entry in the task queue, it searches for the target file according to the save path of the target file, parses the binary structure of the target file and extracts the binary structure comprehensive features of the target file, wherein the binary structure comprehensive features are used to identify the file type of the target file; The mining engine matches the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scans the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with knowledge representations of file types corresponding to multiple types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and data hiding method analysis methods, and is configured with the hidden information mining rule, which is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method; Generate a detection result display file based on the preset result display template and save it to a specified path; The binary structure comprehensive features include: a binary file header, a file data block description and a file tail feature, wherein the binary file header includes a file type feature description, file length check information, a file content description and a data block start address, and the file tail feature includes a file end identifier and a file tail feature description information; The data hiding methods include: file type tampering, multiple compression of compressed packages, encryption hiding, Office redundant data hiding, Office document embedding hiding, data stream hiding, merged file hiding, metadata hiding, and image content hiding. Correspondingly, the data hiding method analysis method includes: file type inspection, compressed package scanning, file encryption scanning, Office document structure inspection, Office hidden data extraction detection, data stream detection, merged file scanning, fast scanning of keywords in files, and fast search of image content. When scanning the target file, the corresponding data hiding method is determined by the file type of the target file, and the corresponding data hiding method analysis method is determined by the data hiding method. Then, the target file is scanned based on the mining process according to the determined data hiding method analysis method to obtain the hidden information corresponding to the target file, and the data hiding method and hidden information are determined as the hidden information detection result. The method for analyzing the data hiding method determines the data hiding method, and scans the target file based on the mining process to obtain the hidden information corresponding to the target file, including: analyzing the binary structure of the target file based on the mining process according to the preset knowledge base matching inference algorithm corresponding to the method for analyzing the data hiding method, and extracting the hidden information; The method for analyzing the data hiding method determines the data hiding method, and scans the target file based on the mining process to obtain the hidden information corresponding to the target file, including: performing one or more of the following mining processes on the binary format of the target file according to the method for analyzing the data hiding method: File type check, identifying whether the content of the target file conforms to the format declared by its suffix name. If not, it is reported as a suspicious file; File encryption scanning to check whether the file is encrypted. If it is encrypted, it will be reported as a suspicious file; Data stream detection, checking whether the target file has additional NTFS data streams. If so, it is reported as a suspicious file; Merge file scanning to check whether there is tail splicing data at the tail of the target file. If so, report it as a suspicious file; Office document structure check: Checks whether there is redundant data in the file structure of the Office document. If so, it is reported as a suspicious file. Office hidden data extraction detection, checking whether there are OLE objects embedded in Office documents. If so, all embedded OLE objects are extracted and reported as suspicious files; Quickly scan keywords in files to see if each byte in the target file is in the preset keyword table. If so, report it as a suspicious file; Image content quick search, optical character recognition of image content, converting image content into text content and then performing a quick keyword scan in the file. If the bytes in the text content are in the preset keyword table, it will be reported as a suspicious file; Compressed package scanning: after decompressing the compressed package layer by layer, return to the file type verification step and re-execute it to determine whether the compressed package is a suspicious file; Determine the results of all data hiding analysis methods performed on the target file as the hidden information; The preset result display template is an html file template or an xml file template, and the detection result display file is generated according to the preset result display template after the hidden information and the hiding method are correspondingly associated; After matching the file type corresponding to the binary structure comprehensive feature in the preset knowledge base, before scanning the target file, it also includes: judging whether the file type is in a preset white list, if so, scanning the target file, if not, directly generating the hidden information detection result; It also includes: in response to a user's request for the task status of the detection task, querying the task status of the task entry in the task database, and if the task status is completed, reading the detection result display file from the specified path and returning it to the user's browser.

2. A system for deep mining of hidden information in electronic documents, characterized in that: include: A request identification module, for responding to a user uploading a target file to be detected and submitting a detection task, saving the target file to a preset file system and creating a task entry in a task database, writing task attributes to the task entry and adding it to a task queue, wherein the task attributes include a saving path of the target file in the file system; A parsing module, which is used for searching the target file according to the save path of the target file after the preset mining engine polls the task entry in the task queue, parsing the binary structure of the target file and extracting the binary structure comprehensive features of the target file, wherein the binary structure comprehensive features are used to identify the file type of the target file; A mining engine module, used for matching the file type corresponding to the binary structure comprehensive feature in a preset knowledge base, and scanning the target file according to the hidden information mining rule corresponding to the file type to obtain the hidden information detection result of the target file, wherein the knowledge base is constructed with knowledge representations of file types corresponding to multiple types of electronic documents and their corresponding binary structure comprehensive features, data hiding methods, and data hiding method analysis methods, and is configured with the hidden information mining rule, the hidden information mining rule is used to characterize the mining process of determining hidden information after determining the file type of the target file, and the mining process includes the data hiding method analysis method; A result generation module, used for inputting the hidden information detection result into a preset result display template to generate a detection result display file and save it to a specified path; The binary structure comprehensive features include: a binary file header, a file data block description and a file tail feature, wherein the binary file header includes a file type feature description, file length check information, a file content description and a data block start address, and the file tail feature includes a file end identifier and a file tail feature description information; The data hiding methods include: file type tampering, multiple compression of compressed packages, encryption hiding, Office redundant data hiding, Office document embedding hiding, data stream hiding, merged file hiding, metadata hiding, and image content hiding. Correspondingly, the data hiding method analysis method includes: file type inspection, compressed package scanning, file encryption scanning, Office document structure inspection, Office hidden data extraction detection, data stream detection, merged file scanning, fast scanning of keywords in files, and fast search of image content. When scanning the target file, the corresponding data hiding method is determined by the file type of the target file, and the corresponding data hiding method analysis method is determined by the data hiding method. Then, the target file is scanned based on the mining process according to the determined data hiding method analysis method to obtain the hidden information corresponding to the target file, and the data hiding method and hidden information are determined as the hidden information detection result. The method for analyzing the data hiding method determines the data hiding method, and scans the target file based on the mining process to obtain the hidden information corresponding to the target file, including: analyzing the binary structure of the target file based on the mining process according to the preset knowledge base matching inference algorithm corresponding to the method for analyzing the data hiding method, and extracting the hidden information; The method for analyzing the data hiding method determines the data hiding method, and scans the target file based on the mining process to obtain the hidden information corresponding to the target file, including: performing one or more of the following mining processes on the binary format of the target file according to the method for analyzing the data hiding method: File type check, identifying whether the content of the target file conforms to the format declared by its suffix name. If not, it is reported as a suspicious file; File encryption scanning to check whether the file is encrypted. If it is encrypted, it will be reported as a suspicious file; Data stream detection, checking whether the target file has additional NTFS data streams. If so, it is reported as a suspicious file; Merge file scanning to check whether there is tail splicing data at the tail of the target file. If so, report it as a suspicious file; Office document structure check: Checks whether there is redundant data in the file structure of the Office document. If so, it is reported as a suspicious file. Office hidden data extraction detection, checking whether there are OLE objects embedded in Office documents. If so, all embedded OLE objects are extracted and reported as suspicious files; Quickly scan keywords in files to see if each byte in the target file is in the preset keyword table. If so, report it as a suspicious file; Image content quick search, optical character recognition of image content, converting image content into text content and then performing a quick keyword scan in the file. If the bytes in the text content are in the preset keyword table, it will be reported as a suspicious file; Compressed package scanning: after decompressing the compressed package layer by layer, return to the file type verification step and re-execute it to determine whether the compressed package is a suspicious file; Determine the results of all data hiding analysis methods performed on the target file as the hidden information; The preset result display template is an html file template or an xml file template, and the detection result display file is generated according to the preset result display template after the hidden information and the hiding method are correspondingly associated; After matching the file type corresponding to the binary structure comprehensive feature in the preset knowledge base, before scanning the target file, it also includes: judging whether the file type is in a preset white list, if so, scanning the target file, if not, directly generating the hidden information detection result; It also includes: in response to a user's request for the task status of the detection task, querying the task status of the task entry in the task database, and if the task status is completed, reading the detection result display file from the specified path and returning it to the user's browser.

3. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the electronic document information hidden deep mining method according to claim 1.

Images