Safe mail analysis method and device and readable medium

Abstract

The invention discloses a safe mail analysis method and device and a readable medium. The method comprises the steps that a to-be-analyzed mail is acquired and preprocessed, and an intermediate file is obtained through analysis; screening the intermediate files through a preset filtering condition, if hit files are obtained through screening, determining the type of each hit file, and if the hit files are simple files, extracting necessary information and performing encryption processing; if the file is the complex file, filling the file heads of the encrypted simple file and the complex file with a preset file head structure to obtain a processed intermediate file; the method comprises the following steps: performing encryption compression processing on a to-be-analyzed mail, performing serialization to disk operation on a processed intermediate file and a processed mail to obtain an intermediate file path and a mail path, and associatively storing necessary information, the intermediate file path or the mail path in a database. According to the method and the device, the mail can be quickly analyzed, and the hit intermediate file and the mail can be safely stored.

Description

Technical Field

[0001] The present invention relates to the field of mail parsing, and in particular to a safe mail parsing method, device and readable medium. Background Art

[0002] In today's information age, email has become an important tool for internal communication in enterprises. It not only improves communication efficiency, but also greatly promotes the development of business activities. However, a large amount of business secrets and sensitive information are transmitted through emails, which undoubtedly poses a severe challenge to the information security of enterprises. In order to ensure the security of this information, enterprises often need to regularly parse and filter internal emails to generate relevant security reports. However, the current development status of related technologies shows that there are still significant deficiencies in security protection during email parsing.

[0003] Under the existing technical framework, parsing and filtering of internal corporate emails has become a routine operation. This operation involves in-depth analysis of email content, attachments, and metadata, but during the parsing process, the protection of intermediate files and parsing results is often overlooked. If not handled properly, these intermediate files and parsing results may become a risk point for serious leaks.

[0004] With the surge in email usage, how to effectively protect data security during the parsing process has become an urgent problem to be solved. First, traditional email parsing tools and methods lack effective security protection mechanisms when processing large amounts of emails. Second, although some companies have begun to pay attention to security protection during the parsing process, existing protection measures often focus on email transmission and storage, and do not adequately protect intermediate parsing files and parsing results.

[0005] In traditional email parsing technologies, some solutions store these temporary files, attachments, and metadata in a temporary directory after parsing the email, and destroy them centrally after filtering or even do not process them; other solutions throw the intermediate results and intermediate files into an encrypted database. These technologies have the following problems:

[0006] 1. Process data may remain due to untimely destruction or power outage;

[0007] 2. All data is stored in the database or the intermediate process data is fully encrypted, which is inefficient.

[0008] In view of this situation, developing a secure email parsing and processing technology that can effectively protect the parsing of intermediate files and parsing has become an urgent need and development trend in the current information security field. Summary of the invention

[0009] The purpose of this application is to propose a secure email parsing method, device and readable medium to address the above-mentioned technical problems.

[0010] In a first aspect, the present invention provides a secure email parsing method, comprising the following steps:

[0011] Obtain the email to be parsed and perform preprocessing to obtain metadata of the email to be parsed, extract basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type;

[0012] If one or more attachments are extracted from the metadata, each attachment is parsed according to different file types to obtain one or more attachment documents of readable document types, and the basic information document or the basic information document and one or more attachment documents constitute an intermediate file corresponding to the email to be parsed;

[0013] The intermediate files are screened by the preset filtering conditions. In response to determining that a hit file is obtained through the screening, the type of each hit file is determined. In response to determining that the type of the hit file is a simple file, necessary information is extracted and the hit file is encrypted. The file header of the encrypted simple file is filled with a preset file header structure to obtain a processed intermediate file. In response to determining that the type of the hit file is a complex file, the file header of the complex file is filled with a preset file header structure to obtain a processed intermediate file.

[0014] The email to be parsed is encrypted and compressed to obtain the processed email, a unique identifier of the intermediate file and a unique identifier of the email are generated for the processed intermediate file and the processed email respectively, the processed intermediate file and the processed email are serialized to disk to obtain the intermediate file path and the email path, the unique identifier of the intermediate file and the unique identifier of the email are used as the storage names of the processed intermediate file and the processed email respectively, and are associated with necessary information, the intermediate file path or the email path and stored in a database.

[0015] Preferably, the basic information of the email includes the recipient, sender, time, subject and body; the basic information of the email and attachments are obtained after processing, and the processing method includes noise removal and / or unified encoding; the hit files with txt, csv, log, tsv, dat, ini, conf, c, cpp, java, py, cs, markdown or md file suffixes are simple files, and the hit files with other file suffixes are complex files.

[0016] Preferably, an association relationship between the MD5 value of the parsed email and the parsing result is established. During the preprocessing of the email to be parsed, the MD5 value of the email to be parsed is calculated. If the MD5 value of the email to be parsed is matched in the MD5 list of the parsed email, the parsing process of the email to be parsed is skipped, and the corresponding parsing result is obtained according to the MD5 value and the association relationship of the email to be parsed. If the MD5 value of the email to be parsed is not matched in the MD5 list of the parsed email, the parsing process of the email to be parsed is performed to obtain the metadata of the email to be parsed.

[0017] Preferably, the intermediate files are screened by preset filtering conditions, specifically including:

[0018] Constructing several preset filtering conditions corresponding to different filtering condition IDs, wherein the preset filtering conditions include preset keyword matching, keyword number threshold filtering, or multiple keyword hit percentage threshold filtering;

[0019] Perform word segmentation on each basic information document or attachment document in the intermediate file to obtain corresponding word segmentation results;

[0020] The word segmentation result is matched with each preset filtering condition. If there are words matching the preset keywords in the word segmentation result, the number of words matching the preset keywords in the word segmentation result exceeds the keyword number threshold, or the percentage of words matching the preset keywords in the word segmentation result exceeds the percentage threshold, the basic information document or attachment document is determined to be a hit file; otherwise, the basic information document or attachment document is determined to be a non-hit file, and the non-hit files or intermediate files in the intermediate files and the emails to be parsed are destroyed.

[0021] Preferably, the preset file header structure includes a base64 string obtained after base64 processing of a label, a structure size and a field containing at least an encryption type or a file type in the necessary information, the fields in the necessary information include the encryption type, the MD5 value of the intermediate file, the filter condition ID, the file type, the file size, the original name of the intermediate file and a reserved field, the database includes email entries and intermediate file entries, the email entry includes the email sequence number, the unique identifier of the email, the email path, the original name of the email and the intermediate file sequence number corresponding to the included hit file, the intermediate file entry includes the intermediate file sequence number, the unique identifier of the intermediate file, the intermediate file path, the original name of the intermediate file, the filter condition ID, the hit value, the file type, the file size, the MD5 value of the intermediate file and whether it is an attachment.

[0022] As a preference, it also includes:

[0023] Obtain the mail path and unique identifier of the mail to be restored, and query whether there is a corresponding mail entry in the database according to the mail path and unique identifier of the mail to be restored. If there is, obtain the processed mail according to the mail path, decrypt and decompress it, and rename it to the original name of the mail in the mail entry to obtain the restored mail;

[0024] Locate the intermediate file entry corresponding to the intermediate file to be restored in the database according to the sequence number of the intermediate file in the email entry, and determine whether the intermediate file to be restored is a simple file or a complex file and the corresponding intermediate file path according to the file type in the intermediate file entry;

[0025] In response to determining that the intermediate file to be restored is a simple file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and then decrypted and renamed to the original name of the intermediate file in the intermediate file entry, so as to obtain the restored intermediate file;

[0026] In response to determining that the intermediate file to be restored is a complex file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and the file is renamed to the original name of the intermediate file in the intermediate file entry to obtain the restored intermediate file.

[0027] In a second aspect, the present invention provides a secure email parsing device, comprising:

[0028] The data extraction module is configured to obtain the email to be parsed and perform preprocessing to obtain metadata of the email to be parsed, extract basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type;

[0029] The parsing module is configured to parse each attachment according to different file types if one or more attachments are extracted from the metadata, obtain one or more attachment documents of readable document types, and form an intermediate file corresponding to the email to be parsed with the basic information document or the basic information document and the one or more attachment documents;

[0030] The filtering module is configured to filter the intermediate files according to the preset filtering conditions, determine the type of each hit file in response to determining that the hit file is a simple file, extract necessary information and encrypt the hit file in response to determining that the type of the hit file is a simple file, fill the preset file header structure in the file header of the encrypted simple file to obtain the processed intermediate file; in response to determining that the type of the hit file is a complex file, fill the preset file header structure in the file header of the complex file to obtain the processed intermediate file;

[0031] The storage module is configured to perform encryption and compression processing on the email to be parsed to obtain the processed email, generate a unique identifier of the intermediate file and a unique identifier of the email for the processed intermediate file and the processed email respectively, perform serialization operations on the processed intermediate file and the processed email to the disk to obtain the intermediate file path and the email path, use the unique identifier of the intermediate file and the unique identifier of the email as the storage name of the processed intermediate file and the processed email respectively, and store them in a database in association with necessary information, the intermediate file path or the email path.

[0032] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0034] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) The secure email parsing method proposed in the present invention first pre-processes the email to be parsed to obtain an intermediate file, matches and filters the intermediate file through preset filtering conditions, and classifies the hit intermediate files into simple files and complex files according to their types. After corresponding processing, the preset file header structure is filled without distinction, and then serialized to the disk, thereby realizing rapid parsing of the email content and ensuring the security and integrity of the data during the processing.

[0037] (2) The secure email parsing method proposed by the present invention encrypts the hit file and modifies the file header structure, so there is no intermediate file available for direct viewing, and records the hit basic information document or attachment document and the hit email to be parsed into an encrypted database. The unhit emails to be parsed and the intermediate files are destroyed, so the intermediate files and hit emails can be stored quickly and more securely. Even if there is a downtime or abnormality in the middle, the intermediate files cannot be directly viewed after recovery.

[0038] (3) The secure email parsing method proposed in the present invention has a relatively high parsing efficiency and can be widely used in the fields of enterprise email management, review, compliance inspection, data mining and analysis, and can provide strong technical support for the enterprise's informatization construction and data-driven decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a secure email parsing method according to an embodiment of the present application;

[0041] Figure 2 A schematic diagram of a framework of a secure email parsing method according to an embodiment of the present application;

[0042] Figure 3 A flowchart of a secure email parsing method according to an embodiment of the present application;

[0043] Figure 4 A schematic diagram of a preset file header structure of a secure email parsing method according to an embodiment of the present application;

[0044] Figure 5 A schematic diagram of an email entry of a secure email parsing method according to an embodiment of the present application;

[0045] Figure 6 A schematic diagram of an intermediate file entry of a secure email parsing method according to an embodiment of the present application;

[0046] Figure 7 A schematic diagram of a secure email parsing device according to an embodiment of the present application;

[0047] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Figure 1A secure email parsing method provided by an embodiment of the present application is shown, comprising the following steps:

[0050] S1, obtain the email to be parsed and perform preprocessing to obtain the metadata of the email to be parsed, extract the basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type.

[0051] In a specific embodiment, the basic information of the email includes the recipient, sender, time, subject and body, and the basic information of the email and the attachments are obtained after processing, and the processing method includes noise removal and / or unified coding.

[0052] Specifically, refer to Figure 2 The embodiment of the present application mainly consists of three parts. The first part is to pre-process the email to be parsed; the second part is to filter the intermediate files obtained according to the preset filtering conditions; the third part is to quickly encrypt the hit intermediate files, encrypt and compress the email to be parsed, and store the association between the hit files and the files to be parsed in the data.

[0053] refer to Figure 3 , distribute and perform parsing operations on different emails or email collections according to the characteristics according to the established rules, and obtain the metadata of the email to be parsed, the metadata includes the recipient, sender, time, subject, text or attachments, where the attachments include documents (office files, pdf files, txt files, rtf files, etc.), pictures, audio, video, compressed files, etc. The embodiment of the present application stores the basic information of the email except the attachments in the metadata in a txt file, that is, obtains the basic information document.

[0054] In a specific embodiment, an association relationship between the MD5 value of a parsed email and the parsing result is established. During the preprocessing of the email to be parsed, the MD5 value of the email to be parsed is calculated. If the MD5 value of the email to be parsed is matched in the MD5 list of parsed emails, the parsing process of the email to be parsed is skipped, and the corresponding parsing result is obtained based on the MD5 value of the email to be parsed and the association relationship. If the MD5 value of the email to be parsed is not matched in the MD5 list of parsed emails, the parsing process of the email to be parsed is performed to obtain the metadata of the email to be parsed.

[0055] Specifically, in order to speed up processing efficiency, de-duplication filtering is added in the preprocessing process of the emails to be parsed, and a fast MD5 calculation is performed on the emails to be parsed and the emails that have been processed, and the calculations are stored in a cache file. If an email with the same MD5 value is processed, the parsing process is skipped directly. When restarting after an exception or a crash, the MD5 value of the email to be parsed is matched according to the MD5 list of the remaining cache file. If a match is found, it means that the email to be parsed has been processed and can be quickly restored to its original location.

[0056] S2, if one or more attachments are extracted from the metadata, each attachment is parsed according to different file types to obtain one or more attachment documents of readable document types, and the basic information document or the basic information document and one or more attachment documents constitute the intermediate file corresponding to the email to be parsed.

[0057] Specifically, the file types of existing attachments are diverse, including office documents, pdf, txt, pictures, audio, video, compressed files, etc. If there are attachments in the metadata, the attachments will be distributed to the corresponding processing modules for multi-threaded processing according to different file types. One file type corresponds to one processing module. For example, an office file corresponds to an office parsing module. The office parsing module contains sub-modules such as docx sub-module, xlsx sub-module, doc sub-module, and xls sub-module. The main function is to convert the unique document content into a unified readable document type. It also involves type nesting processing. For example, an office file contains compressed attachments, picture attachments, etc., so it is also necessary to process different types of files nested in an attachment separately in the corresponding processing modules. Finally, the attachment is parsed to obtain the corresponding attachment document, which is also a readable document type, such as a txt file type. Basic information documents and attachment documents are collectively referred to as intermediate files.

[0058] S3, filter the intermediate files by preset filtering conditions, and in response to determining that a hit file is obtained through the screening, determine the type of each hit file, and in response to determining that the type of the hit file is a simple file, extract necessary information and encrypt the hit file, fill the preset file header structure in the file header of the encrypted simple file to obtain the processed intermediate file; in response to determining that the type of the hit file is a complex file, fill the preset file header structure in the file header of the complex file to obtain the processed intermediate file.

[0059] In a specific embodiment, hit files containing the file suffixes of txt, csv, log, tsv, dat, ini, conf, c, cpp, java, py, cs, markdown or md are simple files, and hit files with the other file suffixes are complex files.

[0060] In a specific embodiment, the intermediate files are screened by preset filtering conditions, specifically including:

[0061] Constructing several preset filtering conditions corresponding to different filtering condition IDs, wherein the preset filtering conditions include preset keyword matching, keyword number threshold filtering, or multiple keyword hit percentage threshold filtering;

[0062] Perform word segmentation on each basic information document or attachment document in the intermediate file to obtain corresponding word segmentation results;

[0063] The word segmentation result is matched with each preset filtering condition. If there are words matching the preset keywords in the word segmentation result, the number of words matching the preset keywords in the word segmentation result exceeds the keyword number threshold, or the percentage of words matching the preset keywords in the word segmentation result exceeds the percentage threshold, the basic information document or attachment document is determined to be a hit file; otherwise, the basic information document or attachment document is determined to be a non-hit file, and the non-hit files or intermediate files in the intermediate files and the emails to be parsed are destroyed.

[0064] In a specific embodiment, the preset file header structure includes a tag, a structure size, and a base64 string obtained after base64 processing of a field containing at least an encryption type or a file type in the necessary information. The fields in the necessary information include the encryption type, the MD5 value of the intermediate file, the filter condition ID, the file type, the file size, the original name of the intermediate file, and a reserved field.

[0065] Specifically, the screening process is as follows:

[0066] After the basic information document or attachment document in the intermediate file is segmented, it is screened based on preset filtering conditions. The preset filtering conditions can be keyword matching, keyword number threshold filtering or multiple keyword hit percentage threshold filtering (matching degree). Keyword matching is to search whether there are preset keywords in the content of the basic information document or attachment document in the intermediate file. If so, the intermediate file is a hit file, otherwise it is a non-hit file; keyword number threshold filtering is to determine whether the number of preset keywords existing in the content of the basic information document or attachment document in the intermediate file reaches the number threshold. If so, the intermediate file is a hit file, otherwise it is a non-hit file; multiple keyword hit percentage threshold filtering is to determine whether the number of preset keywords existing in the content of the basic information document or attachment document in the intermediate file and the total number of preset keywords reach a percentage threshold (taking 60% as an example). If so, the intermediate file is a hit file, otherwise it is a non-hit file.

[0067] If all basic information documents or attachment documents in the intermediate file are non-hit files, the intermediate file and the email to be parsed are directly destroyed. If some basic information documents or attachment documents in the intermediate file are non-hit files, the non-hit files are destroyed, and it is determined whether the hit file is a simple file, that is, whether the file suffix of the hit file is txt, csv, log, tsv, dat, ini, conf, c, cpp, java, py, cs, markdown, md, etc. If so, the necessary information is extracted from the simple file. The original name of the intermediate file in the necessary information is the file name including the suffix, and the file types include png, etc. And fast encryption processing is performed on the content part of the simple file. The encryption processing method includes XOR encryption, SM4, third-party software encryption (such as zip, rar) or other custom encryption rules. The encrypted simple file and complex file are subjected to the same processing, that is, the file header of the encrypted simple file and complex file is filled with the preset file header structure to obtain the processed intermediate file. In one embodiment, the preset file header structure is as follows Figure 4 In another embodiment, the information required for the preset file header structure is as follows:

[0068] Tag|Structure size|Encryption type|MD5 value of intermediate file|File type|File size|Original name of intermediate file|Reserved field|;

[0069] Among them, the tag is: MYEANT, which occupies 6 bytes;

[0070] The structure size is: 88, occupying 8 bytes;

[0071] The encryption type is: 1, occupies 2 bytes;

[0072] The MD5 value of the intermediate file is: FEB768788FC43062EF71F5B4332809B6, occupying 16 bytes;

[0073] The file type is: png, occupies 8 bytes;

[0074] File size: 10MB, 8 bytes;

[0075] The original name of the intermediate file is: test, occupying 100 bytes;

[0076] The reserved fields are: NULL, occupying 16 bytes;

[0077] The characters after the label and structure size are processed with base64 to obtain a string of length 64 and filled after the structure size. These file types, file sizes, original names of intermediate files, etc. correspond to those in the database.

[0078] The default file header structure is (example):

[0079] MYEANT|88|MDAwMDAxRkVCNzY4Nzg4RkM0MzA2MkVGNzFGNUI0MzMyODA5QjZwb mdcMFwwXDBcMFwwXDAxMDI0MDAwMHRlc3Q=.

[0080] The final effect of the added preset file header structure is label + structure size + base64 string, and a fixed 64 bits is added to the header of the file, and this size fills the length of the base64 string. The preset file header structure is a structure defined in the embodiment of the present application to distinguish it from other file structures. It is worth noting that each file has a header structure (except simple files). Once the header structure of the file is changed, it cannot be recognized. For example, the header structure of a png format file is usually 0x89 0x50 0x4e 0x47. If it is changed, it cannot be recognized and opened. Therefore, the embodiment of the present application proposes to add a preset file header structure. In this preset file header structure, the label is filled, the base64 field of the file size is filled with the original file size, and the base64 field of the file type is filled with the original file type. The original intermediate file can be restored by reading this field later.

[0081] S4, encrypt and compress the email to be parsed to obtain the processed email, generate a unique identifier of the intermediate file and a unique identifier of the email for the processed intermediate file and the processed email respectively, serialize the processed intermediate file and the processed email to the disk to obtain the intermediate file path and the email path, use the unique identifier of the intermediate file and the unique identifier of the email as the storage name of the processed intermediate file and the processed email respectively, and store them in the database in association with necessary information, the intermediate file path or the email path.

[0082] In a specific embodiment, the database includes email entries and intermediate file entries. The email entries include the email sequence number, the unique identifier of the email, the email path, the original name of the email, and the intermediate file sequence number corresponding to the hit file contained therein. The intermediate file entries include the intermediate file sequence number, the unique identifier of the intermediate file, the intermediate file path, the original name of the intermediate file, the filter condition ID, the hit value, the file type, the file size, the MD5 value of the intermediate file, and whether it is an attachment.

[0083] Specifically, a corresponding unique identifier is generated for the processed intermediate file, and the intermediate file is associated with it, and the processed intermediate file is serialized to disk, which is generally set in an email subdirectory. If the serialized file is a simple file, it is encrypted. If it is a complex file, the preset file header structure is added and the file name is a unique identifier (GUID). The corresponding APP cannot associate the file, that is, it cannot access the corresponding data to achieve the purpose of obfuscated encryption. For example, an intermediate file called ABC.docx is hit, and when it is stored here, it will become 219DEF9D-B7FA-4983-A495-886340C9BC68, without the original name of the intermediate file and no suffix. Then the file size, MD5 value of the intermediate file, original name of the intermediate file and other information are encrypted and placed in the file header and database.

[0084] Furthermore, since the corresponding email to be parsed has the need to trace its source, it is encrypted and compressed, and the information including the original name of the email including the suffix, the email size, the MD5 value of the email, the intermediate file sequence number, etc. is extracted, and a corresponding unique identifier is assigned to the email to be parsed, and the email to be parsed is associated. The above information is saved in the database, and the email to be parsed is serialized to the disk according to the preset path. At this time, the serialized email is encrypted content and the file name is the assigned unique identifier (GUID). The email to be parsed is saved in the database along with the hit result and related information. The stored information includes the original name of the email, the email path and the hit attachment information, and the original email to be parsed and the hit attachment information can be restored based on this information. The related information in the database is as follows: Figure 5 and Figure 6 In one embodiment, the original name of the email to be parsed is TEST.eml. After calculating the MD5 value and size of the email, the email to be parsed is encrypted and compressed, and then saved as 6C5C672E-7E2F-4AAD-B54D-66A4A0057AE2 (GUID). Then, the original name of the email, the MD5 value of the email, the size of the email, and other information are stored in the database. In another embodiment, the original name of the email to be parsed is TTT.outlook. The email to be parsed is encrypted and compressed, and saved as

[0085] F5E418E7-B374-42AF-9746-4623B1048F69, stores the original name of the email to be parsed, the MD5 value of the email, and the size of the email in the database. The above database and the content stored on the disk are the parsing results.

[0086] In a specific embodiment, it also includes:

[0087] Obtain the mail path and unique identifier of the mail to be restored, and query whether there is a corresponding mail entry in the database according to the mail path and unique identifier of the mail to be restored. If there is, obtain the processed mail according to the mail path, decrypt and decompress it, and rename it to the original name of the mail in the mail entry to obtain the restored mail;

[0088] Locate the intermediate file entry corresponding to the intermediate file to be restored in the database according to the sequence number of the intermediate file in the email entry, and determine whether the intermediate file to be restored is a simple file or a complex file and the corresponding intermediate file path according to the file type in the intermediate file entry;

[0089] In response to determining that the intermediate file to be restored is a simple file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and then decrypted and renamed to the original name of the intermediate file in the intermediate file entry, so as to obtain the restored intermediate file;

[0090] In response to determining that the intermediate file to be restored is a complex file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and the file is renamed to the original name of the intermediate file in the intermediate file entry to obtain the restored intermediate file.

[0091] Specifically, first obtain the mail path and unique identifier of the mail to be restored, and then determine whether there is a corresponding mail record on the database. If the mail exists, the encrypted mail file can be found according to the "mail path" field of the mail entry stored in the database. Since encryption and compression processing has been performed before, decryption and decompression processing is required, and the name is changed according to the "original mail name" field in the mail entry to obtain the restored mail. Then find the corresponding intermediate file entry according to the "intermediate file sequence number" field of the mail entry, and find the processed intermediate file according to the "intermediate file path" field in the intermediate file entry. According to the "file type" field in the intermediate file entry, determine whether the processed intermediate file is a simple file or a complex file.

[0092] If it is a simple file, remove the data before the base64 string and then decrypt it to get the restored intermediate file;

[0093] If it is a complex file, simply remove the data before the base64 string to get the restored intermediate file. A safer approach is to encrypt and decrypt it like a simple file.

[0094] Further references Figure 7As an implementation of the methods shown in the above figures, the present application provides an embodiment of a secure email parsing device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0095] The present application provides a secure email parsing device, including:

[0096] The data extraction module 1 is configured to obtain the email to be parsed and perform preprocessing to obtain the metadata of the email to be parsed, extract the basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type;

[0097] The parsing module 2 is configured to parse each attachment according to different file types if one or more attachments are extracted from the metadata, obtain one or more attachment documents of readable document types, and form an intermediate file corresponding to the email to be parsed with the basic information document or the basic information document and the one or more attachment documents;

[0098] The filtering module 3 is configured to filter the intermediate files according to the preset filtering conditions, and in response to determining that a hit file is obtained by the screening, determine the type of each hit file, and in response to determining that the type of the hit file is a simple file, extract necessary information and encrypt the hit file, fill the preset file header structure in the file header of the encrypted simple file to obtain the processed intermediate file; in response to determining that the type of the hit file is a complex file, fill the preset file header structure in the file header of the complex file to obtain the processed intermediate file;

[0099] The storage module 4 is configured to perform encryption and compression processing on the email to be parsed to obtain the processed email, generate a unique identifier of the intermediate file and a unique identifier of the email for the processed intermediate file and the processed email respectively, perform serialization operations on the processed intermediate file and the processed email to the disk to obtain the intermediate file path and the email path, use the unique identifier of the intermediate file and the unique identifier of the email as the storage name of the processed intermediate file and the processed email respectively, and store them in a database in association with necessary information, the intermediate file path or the email path.

[0100] Figure 8 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 8 As shown, the electronic device of this embodiment includes: a processor 801 and a memory 802; wherein the memory 802 is used to store computer-executable instructions; the processor 801 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0101] Optionally, the memory 802 may be independent or integrated with the processor 801 .

[0102] When the memory 802 is independently provided, the electronic device further includes a bus 803 for connecting the memory 802 and the processor 801 .

[0103] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 801 executes the computer execution instructions, the above method is implemented.

[0104] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 801, the above method is implemented.

[0105] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0106] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0107] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0108] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 801 to perform some steps of the methods of various embodiments of the present application.

[0109] It should be understood that the processor 801 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 801 may be any conventional processor 801, etc. The steps of the method disclosed in the invention may be directly embodied in the hardware processor 801 for execution, or may be executed by a combination of hardware and software modules in the processor 801.

[0110] The memory 802 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0111] The bus 803 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 803 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 803 in the drawings of the present application is not limited to only one bus 803 or one type of bus 803.

[0112] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0113] An exemplary storage medium is coupled to the processor 801, so that the processor 801 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 801. The processor 801 and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor 801 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0114] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A secure email parsing method, characterized in that: The following steps are involved: Acquire the email to be parsed and pre-process it to obtain metadata of the email to be parsed, extract basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type; If one or more attachments are extracted from the metadata, each attachment is parsed according to different file types to obtain one or more attachment documents of readable document types, and the basic information document or the basic information document and one or more attachment documents constitute an intermediate file corresponding to the email to be parsed; The intermediate files are screened by a preset filtering condition, and in response to determining that a hit file is obtained by the screening, the type of each hit file is determined, and in response to determining that the type of the hit file is a simple file, necessary information is extracted and the hit file is encrypted, and a preset file header structure is filled in the file header of the encrypted simple file to obtain a processed intermediate file; In response to determining that the type of the hit file is a complex file, filling a preset file header structure in the file header of the complex file to obtain a processed intermediate file; The email to be parsed is encrypted and compressed to obtain a processed email, a unique identifier of the intermediate file and a unique identifier of the email are generated for the processed intermediate file and the processed email respectively, the processed intermediate file and the processed email are serialized to a disk to obtain an intermediate file path and an email path, the unique identifier of the intermediate file and the unique identifier of the email are used as storage names of the processed intermediate file and the processed email respectively, and are associated with the necessary information, the intermediate file path or the email path and stored in a database.

2. The secure email parsing method according to claim 1, characterized in that: The basic information of the email includes the recipient, sender, time, subject and body. The basic information and attachments of the email are obtained after processing, and the processing method includes removing noise and / or unified encoding. The hit files with txt, csv, log, tsv, dat, ini, conf, c, cpp, java, py, cs, markdown or md file suffixes are the simple files, and the hit files with other file suffixes are the complex files.

3. The secure email parsing method according to claim 1, characterized in that: Establishing an association relationship between the MD5 value of the parsed email and the parsing result, calculating the MD5 value of the email to be parsed during the preprocessing of the email to be parsed, and if the MD5 value of the email to be parsed is matched in the MD5 list of the parsed emails, skipping the parsing process of the email to be parsed, and obtaining the corresponding parsing result according to the MD5 value of the email to be parsed and the association relationship; If the MD5 value of the email to be parsed is not matched in the MD5 list of the parsed emails, the parsing process of the email to be parsed is performed to obtain the metadata of the email to be parsed.

4. The secure email parsing method according to claim 1, characterized in that: The intermediate files are filtered by preset filtering conditions, specifically including: Constructing a plurality of preset filtering conditions corresponding to different filtering condition IDs, wherein the preset filtering conditions include preset keyword matching, keyword number threshold filtering, or multiple keyword hit percentage threshold filtering; Perform word segmentation processing on each basic information document or attachment document in the intermediate file to obtain corresponding word segmentation results; The word segmentation result is matched with each preset filtering condition. If there is a word segmentation matching the preset keyword in the word segmentation result, the number of word segmentations matching the preset keyword in the word segmentation result exceeds the keyword number threshold, or the percentage of word segmentations matching the preset keyword in the word segmentation result exceeds the percentage threshold, the basic information document or the attachment document is determined to be a hit file; otherwise, the basic information document or the attachment document is determined to be a non-hit file, and the non-hit file in the intermediate file or the intermediate file and the email to be parsed are destroyed.

5. The secure email parsing method according to claim 4, characterized in that: The preset file header structure includes a label, a structure size, and a base64 string obtained after base64 processing of a field in the necessary information that at least includes an encryption type or a file type. The fields in the necessary information include an encryption type, an MD5 value of an intermediate file, a filtering condition ID, a file type, a file size, an original name of an intermediate file, and a reserved field. The database includes email entries and intermediate file entries. The email entries include an email sequence number, a unique identifier of the email, an email path, an original name of the email, and an intermediate file sequence number corresponding to the included hit file. The intermediate file entries include an intermediate file sequence number, a unique identifier of the intermediate file, an intermediate file path, an original name of the intermediate file, a filtering condition ID, a hit value, a file type, a file size, an MD5 value of the intermediate file, and whether it is an attachment.

6. The secure email parsing method according to claim 5, characterized in that: Also includes: Obtaining the mail path and unique identifier of the mail to be restored, querying whether there is a corresponding mail entry in the database according to the mail path and unique identifier of the mail to be restored, and if so, obtaining the processed mail according to the mail path, decrypting and decompressing it, and renaming it to the original name of the mail in the mail entry, to obtain the restored mail; Locating the intermediate file entry corresponding to the intermediate file to be restored in the database according to the sequence number of the intermediate file in the email entry, and determining whether the intermediate file to be restored is a simple file or a complex file and the corresponding intermediate file path according to the file type in the intermediate file entry; In response to determining that the intermediate file to be restored is a simple file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and then decrypted and renamed to the original name of the intermediate file in the intermediate file entry, so as to obtain the restored intermediate file; In response to determining that the intermediate file to be restored is a complex file, the processed intermediate file is found according to the intermediate file path, the data before the base64 string in the file header structure of the processed intermediate file is removed, and the file is renamed to the original name of the intermediate file in the intermediate file entry to obtain the restored intermediate file.

7. A secure email parsing device, characterized in that: include: A data extraction module is configured to obtain and pre-process the email to be parsed, obtain metadata of the email to be parsed, extract basic information of the email from the metadata, and store the basic information of the email as a basic information document of a readable document type; A parsing module is configured to parse each attachment according to different file types if one or more attachments are extracted from the metadata, obtain one or more attachment documents of readable document types, and form an intermediate file corresponding to the email to be parsed with the basic information document or the basic information document and the one or more attachment documents; A filtering module is configured to filter the intermediate files according to a preset filtering condition, determine the type of each hit file in response to determining that the hit file is a simple file, extract necessary information and encrypt the hit file in response to determining that the type of the hit file is a simple file, fill the preset file header structure in the file header of the encrypted simple file, and obtain a processed intermediate file; In response to determining that the type of the hit file is a complex file, filling a preset file header structure in the file header of the complex file to obtain a processed intermediate file; The storage module is configured to encrypt and compress the email to be parsed to obtain the processed email, generate a unique identifier of the intermediate file and a unique identifier of the email for the processed intermediate file and the processed email respectively, serialize the processed intermediate file and the processed email to the disk to obtain the intermediate file path and the email path, use the unique identifier of the intermediate file and the unique identifier of the email as the storage name of the processed intermediate file and the processed email respectively, and store them in a database in association with the necessary information, the intermediate file path or the email path.

8. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images