File type identification method and device, electronic equipment and storage medium
By acquiring and parsing data packets to reconstruct file data, and using the Libmagic database and feature matching algorithm to determine file types, the problem of low accuracy in file type identification is solved, achieving more efficient file type identification and secure file restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN SIPU TECH CO LTD
- Filing Date
- 2023-04-14
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies have low accuracy in identifying file types and are easily compromised by viruses that disguise their file types, causing security issues.
By acquiring multiple data packets of the file to be identified, parsing and reconstructing the file data, extracting feature data using the Libmagic database, and performing feature matching using the Aho-Corasick or Hyperscan algorithm, the file type is determined.
It improves the accuracy of file type identification, avoids security problems caused by virus files that disguise file types, and ensures the targeted and secure restoration and saving of files.
Smart Images
Figure CN116431575B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of computer technology, specifically to a file type identification method, apparatus, electronic device, and storage medium. Background Technology
[0002] The Internet is the main way for people to transmit information, and the types of protocols that carry file transfers have become increasingly diverse, such as Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Simple Mail Transfer Protocol (SMTP), Post Office Protocol (POP), and SMB (Server Message Block, a protocol developed by Microsoft and Intel). Different protocols have different rules for processing files, which makes it difficult to identify file types.
[0003] Meanwhile, when identifying file types, the file type is mainly determined based on the file extension. Although this method is relatively simple, the file extension is easily modified, so this simple method of identifying file extensions can easily lead to incorrect identification of file types. Summary of the Invention
[0004] In view of the above problems, embodiments of the present invention provide a file type identification method, apparatus, electronic device and storage medium to solve the problem of low accuracy in file type identification in the prior art.
[0005] According to one aspect of the present invention, a file type identification method is provided, the method comprising: acquiring a plurality of data packets corresponding to a file to be identified; generating file data of the file to be identified based on the plurality of data packets; extracting feature data of the file data; and performing feature matching on the feature data to determine the file type of the file to be identified.
[0006] In some examples, generating file data of the file to be identified based on the plurality of data packets includes: parsing each of the data packets to determine the packet order; and reassembling the data packets according to the packet order to obtain the file data.
[0007] In some examples, extracting feature data from the file data includes: extracting feature data from the file data using a feature extraction database, which may include the Libmagic database.
[0008] In some examples, the step of extracting feature data of the file data using a feature extraction database includes: storing multiple sub-file data included in the file data into multiple cache areas respectively; sequentially reading the sub-file data in at least one cache area; using file type detection features in the Libmagic database to match at least one of the sub-file data to obtain feature data of at least one of the sub-file data; and determining the feature data of the file data based on the feature data of at least one of the sub-file data.
[0009] In some examples, the sub-file data in the at least one cache region is the first N bytes of the file data, where N is a positive integer.
[0010] In some examples, feature matching is performed on the feature data to determine the file type of the file to be identified, including: using a matching algorithm to perform feature matching on the feature data to determine the file type of the file to be identified; wherein the matching algorithm includes the Aho-Corasick algorithm or the Hyperscan algorithm.
[0011] In some examples, after determining the file type of the file to be identified, the method further includes: generating an information digest of the file to be identified when the file type of the file to be identified matches a preset file type; and determining the security detection result of the file to be identified based on the information digest.
[0012] According to another aspect of the present invention, a file type identification device is provided. The device includes: an acquisition module for acquiring multiple data packets corresponding to a file to be identified; a generation module for generating file data of the file to be identified based on the multiple data packets; an extraction module for extracting feature data of the file data; and a determination module for performing feature matching on the feature data to determine the file type of the file to be identified.
[0013] In some examples, the generation module is used to parse each of the data packets, determine the packet order of each of the data packets, and reassemble each of the data packets according to the packet order to obtain the file data.
[0014] In some examples, the extraction module is used to extract feature data from the file data using a feature extraction database, including the Libmagic database.
[0015] In some examples, the extraction module is used to store multiple sub-file data including the file data into multiple cache areas respectively, sequentially read the sub-file data in at least one cache area, use the file type detection features in the Libmagic database to match at least one of the sub-file data to obtain feature data of at least one of the sub-file data, and determine the feature data of the file data based on the feature data of at least one of the sub-file data.
[0016] In some examples, the sub-file data in the at least one cache region is the first N bytes of the file data, where N is a positive integer.
[0017] In some examples, the determining module is used to perform feature matching on the feature data using a matching algorithm to determine the file type of the file to be identified; wherein the matching algorithm includes the Aho-Corasick algorithm or the Hyperscan algorithm.
[0018] In some examples, after determining the file type of the file to be identified, the determining module is further configured to generate an information digest of the file to be identified when the file type of the file to be identified is consistent with a preset file type; and determine the security detection result of the file to be identified based on the information digest.
[0019] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing at least one executable instruction; the executable instruction causing the processor to perform operations as described in any of the preceding embodiments of the file type identification method.
[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the operation of the file type identification method as described in any of the preceding claims.
[0021] In summary, the file type identification method, apparatus, electronic device, and storage medium provided in the embodiments of the present invention can acquire multiple data packets corresponding to the file to be identified, generate file data of the file to be identified based on the multiple data packets, extract feature data of the file data, perform feature matching on the feature data, and determine the file type of the file to be identified. Compared with the method of identifying file type only by file extension, it can improve the accuracy of file type identification, avoid security problems caused by virus files disguised as file types, and facilitate the restoration of secure files. Operators can also choose to save files of the file types they need, making file restoration and saving more targeted.
[0022] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0023] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0024] Figure 1 A flowchart of a file type identification method provided by the present invention is shown;
[0025] Figure 2 A sub-flowchart of a file type identification method provided by the present invention is shown;
[0026] Figure 3 A sub-flowchart of another file type identification method provided by the present invention is shown;
[0027] Figure 4 A sub-flowchart of another file type identification method provided by the present invention is shown;
[0028] Figure 5 A schematic diagram of a file type application interface provided by the present invention is shown;
[0029] Figure 6 A flowchart of another file type identification method provided by the present invention is shown;
[0030] Figure 7 A schematic diagram of the structure of a document type identification device provided by the present invention is shown;
[0031] Figure 8 A schematic diagram of the structure of an electronic device provided by the present invention is shown. Detailed Implementation
[0032] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.
[0033] Figure 1 The flowchart illustrates a file type identification method provided by an embodiment of the present invention. This method can be executed by an electronic device, which can be any device with file data processing capabilities, such as a computer, a smartphone, or a server or server cluster.
[0034] By applying the file type identification method provided in this embodiment, multiple data packets corresponding to the file to be identified can be obtained. Based on the multiple data packets, file data of the file to be identified can be generated. Then, feature data of the file data can be extracted, and feature matching can be performed on the feature data to determine the file type of the file to be identified. This can improve the accuracy of file type identification, avoid security problems caused by virus files that disguise file types, and also facilitate the restoration of secure files.
[0035] refer to Figure 1 As shown, the method may include the following steps:
[0036] Step 110: Obtain multiple data packets corresponding to the file to be identified.
[0037] The file to be identified refers to the file whose file type needs to be determined. This can include any file type transmitted over the network, such as Word, Portable Document Format (PDF), and PowerPoint files. Depending on the file type and usage scenario, the file to be identified can be transmitted over the network using different transmission protocols. A data packet is a unit of data exchanged and transmitted over the network; that is, a block of data that a station can transmit at one time.
[0038] Depending on the file transfer protocol, data packets can include various types, such as Transmission Control Protocol (TCP) packets, User Datagram Protocol (UDP) packets, and Internet Protocol (IP) packets. Among these, TCP and UDP packets are two different application layer packet types. TCP packets deliver data completely to the application, while UDP packets deliver data packets to the application. IP packets deliver data packets to the destination host.
[0039] When transmitting data over a network, electronic devices can capture the transmitted data packets through appropriate acquisition interfaces or tools. Since the amount of data transmitted in a single network transmission is limited, while the file to be identified typically has a large amount of data, it is transmitted through multiple data packets. During transmission, each data packet includes not only the file data of the file to be identified but also message information such as message length, message order, and other information.
[0040] To improve the processing efficiency of data packets, in some examples, after acquiring the data packet, the electronic device can also determine whether the data packet contains file data of the file to be identified based on the data packet's transmission protocol. For example, it can determine whether the data packet's transmission protocol is a file transfer protocol. If so, the file type identification method provided in this embodiment is executed; if not, it can be ignored and no processing is performed.
[0041] In this way, data packets containing file data can be filtered out from the data packets, improving the processing efficiency of data packets and avoiding the resource consumption problem caused by processing data packets that do not contain file data.
[0042] Step 120: Generate file data for the file to be identified based on multiple data packets.
[0043] After acquiring multiple data packets, in order to reconstruct the file to be identified, each data packet can be parsed to obtain the file data of the file to be identified in each data packet. Then, the file data of the file to be identified in each data packet can be integrated to obtain the complete file data of the file to be identified.
[0044] As mentioned earlier, file transfer protocols can carry file data transmission. However, since the amount of data that a network can transmit is limited, file data is usually split and encapsulated into multiple data packets for transmission. Furthermore, during transmission, the order of these data packets can become disordered due to network jitter or insufficient bandwidth. Therefore, in order to generate the file data for the file to be identified, in some examples, such as... Figure 2 As shown, the following methods can be executed:
[0045] Step 210: Parse each data packet to determine the packet order.
[0046] There are usually multiple data packets corresponding to the file to be identified. The packet order refers to the order of a data packet among the multiple data packets corresponding to the file to be identified.
[0047] Step 220: Reassemble the data packets according to their order to obtain the file data.
[0048] Taking TCP packets as an example, the TCP packets can be parsed to obtain the TCP packet header, and the packet order can be read from the packet header to obtain the packet order of each TCP packet.
[0049] Since the packet encapsulation rules for data packets corresponding to different file transfer protocols may differ, in some examples, different parsing rules can be used to obtain the packet order of each data packet based on the file transfer protocol type corresponding to the data packet. For example, in the TCP packet mentioned above, the packet order can be obtained by reading the TCP packet header. In some cases, the packet order can also be obtained by reading specific bytes of data in the data packet. This embodiment does not specifically limit this.
[0050] After obtaining the packet order of each TCP packet, the data packets can be reassembled according to their order to obtain the file data of the file to be identified. For example, the TCP packets can be arranged and reassembled in ascending order to obtain the complete file data of the file to be identified.
[0051] It should be noted that the method of arranging and reorganizing data packets in ascending order as described above is merely an illustrative example. In reality, since different file transfer protocols have different rules for processing data packets, after obtaining the packet order, the data packets can be arranged and reorganized according to the order rules of their respective file transfer protocols to obtain the complete file data of the file to be identified.
[0052] Using the above method, the message order of each data message can be obtained by parsing the data messages, and the data messages can be reassembled according to the message order to obtain complete file data, thereby realizing the file restoration of the file to be identified.
[0053] Furthermore, the file data of the file to be identified can be transmitted and stored in electronic devices in the form of a binary stream file.
[0054] Step 130: Extract feature data from the file data.
[0055] Feature data of file data refers to the key data that can represent the file data. To determine the file type of a file to be identified, feature data can be extracted from the file data. For example, appropriate feature extraction algorithms can be used to process the file data and obtain its feature data.
[0056] In some examples, step 130 can be achieved by the following method:
[0057] Use a feature extraction database to extract feature data from file data.
[0058] The feature extraction database may include the Libmagic database, which is a development library that identifies file types based on file headers.
[0059] After obtaining the file data of the file to be identified, feature extraction databases, such as the Libmagic database, can be used to extract the feature data of the file data. Using the Libmagic database, the header file of the file data can be examined according to a predefined list of file types to extract the feature data. For example, the feature data of the file data can be a string containing file type characteristic information, such as CSV text.
[0060] The process of feature extraction using the Libmagic database can include memory allocation, feature file loading, feature matching, and memory reclamation. Memory allocation and reclamation is a dynamic adjustment process; that is, memory is requested when feature extraction is needed and released when it is no longer needed. This enables dynamic memory management and avoids the problem of continuously increasing memory usage leading to a continuous decrease in available memory space.
[0061] Specifically, in some examples, refer to Figure 3 As shown, the feature data of file data can be extracted using the following methods:
[0062] Step 310: Store the data of multiple sub-files, including the file data, into multiple cache areas respectively.
[0063] The cache area refers to the temporary storage area for file data.
[0064] To improve the efficiency of feature data extraction, the file data can be divided into multiple sub-files, each with a different data size. The data from each sub-file is then stored in a cache area for later reading and processing.
[0065] Step 320: Read the sub-file data in at least one cache area in sequence.
[0066] When extracting feature data from file data, sub-file data in at least one cache region can be read sequentially. For example, sub-file data in some cache regions can be read in sequence, or sub-file data in all cache regions can be read.
[0067] The researchers disclosed in this publication discovered that the header data of a file, i.e., the first few bytes, contains file type characteristics. Therefore, to improve the efficiency of file data analysis, in some examples, the sub-file data in at least one of the aforementioned cache areas consists of the first N bytes of the file data, where N is a positive integer. N can be customized according to requirements, such as being set to 1024, 2048, etc.
[0068] For example, when reading subfile data in a cached area, the first 2048 bytes of data in the first cached area can be read sequentially.
[0069] This method eliminates the need to analyze all the file data of the file to be identified; instead, it only requires analyzing a portion of the file data, thus improving the efficiency of file analysis.
[0070] Step 330: Use the file type detection features in the Libmagic database to match at least one sub-file data to obtain the feature data of at least one sub-file data.
[0071] The Libmagic database includes many file type detection features. Each sub-file data can be matched with these file type detection features. When it is determined that any sub-file data matches any file type detection feature, the matching data between the sub-file data and the file type detection feature can be determined as the feature data of the sub-file data.
[0072] The matching data can be data from any of the aforementioned sub-file data that includes any of the aforementioned file type detection features. The length of this data can be the same as or greater than the length of any of the aforementioned file type detection features. When the length of the matching data is greater than the length of any of the aforementioned file type detection features, it indicates that the matching data includes not only any of the aforementioned file type detection features but also other data from any of the aforementioned sub-file data.
[0073] Step 340: Determine the characteristic data of the file data based on the characteristic data of at least one sub-file data.
[0074] After obtaining the feature data of each sub-file, the feature data of the entire file to be identified can be determined based on the feature data of each sub-file. For example, the feature data of all sub-files can be used to determine the feature data of the entire file. Alternatively, statistical analysis can be performed on the feature data of all sub-files to determine the feature data of the file.
[0075] Through the above steps 310-340, the file data of the file to be identified can be divided into multiple sub-file data, and then each sub-file data is analyzed to obtain the feature data of the file data. The feature data of the file to be identified can be extracted, and since it is not necessary to process all the file data of all the files to be identified, the efficiency of file data analysis can be improved.
[0076] Furthermore, in the above method, in order to ensure that the read sub-file data is the header data of the file data to be identified, in some examples, a queue can be created to store the file data of the file to be identified in at least one cache area corresponding to the queue, and each cache area can store the sub-file data of the file to be identified.
[0077] Since a queue is a "first-in, first-out" data storage structure, when performing feature extraction, the file type detection thread can sequentially retrieve the sub-file data stored in each cache area from the queue, thereby performing feature extraction on the sub-file data in each cache area, obtaining the feature data of each sub-file data, and then obtaining the feature data of the file data of the file to be identified.
[0078] This method ensures that the file data of the file to be identified is stored and read in sequence, making it easier to obtain the header data of the file data to be identified.
[0079] Step 140: Perform feature matching on the feature data to determine the file type of the file to be identified.
[0080] After obtaining the feature data of the file, feature matching can be performed on the feature data, such as using a corresponding feature recognition database to match the feature data to determine the file type of the file to be identified. For example, it can be determined whether the feature data matches a file type string in the feature library. If a match is found, the file type indicated by that file type string can be identified as the file type of the file to be identified.
[0081] To improve the efficiency and accuracy of feature matching, in some examples, step 140 can be implemented using the following method:
[0082] A matching algorithm is used to perform feature matching on the feature data to determine the file type of the file to be identified.
[0083] The matching algorithms mentioned above may include the Aho-Corasick algorithm or the Hyperscan algorithm. The Aho-Corasick algorithm is a classic algorithm in multi-pattern matching; Hyperscan is an automaton-based algorithm engine. The Hyperscan algorithm is based on automaton theory, and its workflow can be divided into two parts: compilation and runtime.
[0084] During compilation, operators can set regular expressions and use the Hyperscan engine's compiler to compile the regular expressions, generating a feature recognition database. Then, during runtime, the feature data of the file data to be recognized is matched with the features in the feature recognition database to determine the file type of the file to be recognized.
[0085] By using the Aho-Corasick or Hyperscan algorithms to quickly scan and match feature data, the file type of the file to be identified can be determined rapidly. For example, for the string feature "CSV text", when the Aho-Corasick or Hyperscan algorithm scans for CSV or csv in the feature library, the file type of the file to be identified can be determined to be a CSV file.
[0086] Compared to the Aho-Corasick algorithm, the Hyperscan algorithm has a faster matching speed, is not affected by the case of feature data, has higher scalability, and supports multi-threaded fast matching.
[0087] In some examples, reference Figure 4 As shown, after determining the file type of the file to be identified, the following methods can also be executed:
[0088] Step 410: When the file type of the file to be identified matches the preset file type, generate an information digest of the file to be identified.
[0089] The preset file types can be pre-set by the operator; these can be file types that the user prefers, such as... Figure 5 As shown, operators can select which file transfer protocols were used to restore the file data through the settings interface, and select the required file types, such as... Figure 5 The PDF and DOC files shown are displayed.
[0090] A message digest is a fixed-length one-way hash value generated from a file to be identified using a specific algorithm. The key characteristic of a message digest is that even a small change in a single character within the file's data can cause a significant change in the message digest. Therefore, a message digest can be used to determine whether a file has been tampered with.
[0091] Therefore, when the file type of the file to be identified is a PDF or DOC file, it can be determined that the file type of the file to be identified is consistent with the preset file type. At this time, a message digest of the file to be identified can be generated. For example, the MD5 message-digest algorithm or the SHA algorithm (Security Hash Algorithm) can be used to convert the file to be identified in order to generate a message digest of the file to be identified.
[0092] Step 420: Determine the security detection result of the document to be identified based on the message digest.
[0093] The security detection result of the file to be identified refers to whether the file to be identified is a virus file or whether there are any security risks.
[0094] For example, the information digest of the file to be identified can be compared with the information digest of a previously detected virus file. If they match, it means that the file to be identified is a virus file; otherwise, if they do not match, it means that the file to be identified is not a virus file.
[0095] The above methods can be used to detect risks in files of specific file types, ensuring network security.
[0096] Figure 6 A flowchart of another file type identification method provided in this embodiment is shown, such as... Figure 6 As shown, the following steps may be included:
[0097] Step 610: Obtain multiple data packets corresponding to the file to be identified.
[0098] Step 620: Determine whether the data transmission protocol of each data packet is the file transfer protocol. If not, proceed to step 630; if yes, proceed to step 640.
[0099] File transfer protocols can include HTTP, FTP, SMTP, POP, and SMB protocols.
[0100] For example, it can be determined whether the data packet transmission protocol is one of the file transfer protocols mentioned above. If it is, then step 640 is executed; if not, then step 630 is executed to end the process.
[0101] Step 630: End the process.
[0102] Step 640: Parse each data packet, determine the packet order, and reassemble the data packets according to the packet order to obtain the file data.
[0103] Taking TCP packets as an example, the TCP packets can be parsed to obtain the TCP packet header, and the packet order can be read from the packet header to obtain the packet order of each TCP packet.
[0104] Then, the TCP packets can be rearranged and reassembled in ascending order according to their packet sequence to obtain the complete file data of the file to be identified.
[0105] Step 650: Store the file data of the file to be identified into the corresponding cache areas of the queue.
[0106] The file data is divided into multiple sub-files, each with a different amount of data. The data from each sub-file is then stored sequentially in its corresponding cache area within a queue for later reading and processing.
[0107] Step 660: Using the file type detection features in the Libmagic database, extract features from the sub-file data in each cache area to obtain the feature data of the file data to be identified.
[0108] Each sub-file data is matched with the file type detection features in the Libmagic database. When any sub-file data is determined to match any file type detection feature, the matching data between the sub-file data and the file type detection feature can be determined as the feature data of the sub-file data.
[0109] After obtaining the feature data of each sub-file, the feature data of the file data to be identified can be determined based on the feature data of all sub-files. For example, the feature data of all sub-files can be used as the feature data of the file data to be identified.
[0110] Step 670: Use the Hyperscan algorithm to perform feature matching on the feature data to determine the file type of the file to be identified.
[0111] After obtaining the feature data, the feature recognition database generated by the Hyperscan algorithm can be used to quickly scan and match the feature data to determine the file type indicated by the feature data, thereby obtaining the file type of the file to be identified.
[0112] In summary, the file type identification method provided by the embodiments of the present invention can acquire multiple data packets corresponding to the file to be identified, generate file data of the file to be identified based on the multiple data packets, extract feature data of the file data, perform feature matching on the feature data, and determine the file type of the file to be identified. Compared with the method of identifying file type only by file extension, it can improve the accuracy of file type identification, avoid security problems caused by virus files disguised as file types, and is also conducive to the restoration of secure files. Operators can also choose to save files of the file types they need, making file restoration and saving more targeted.
[0113] In addition, this solution can use an asynchronous detection method, that is, the file data of the file to be identified is transferred to the detection thread for detection, which will not affect the operation of the main business.
[0114] Figure 7 A schematic diagram of a file type identification device provided in an embodiment of the present invention is shown. Figure 7As shown, the file type identification device 700 may include: an acquisition module 710, which can be used to acquire multiple data packets corresponding to the file to be identified; a generation module 720, which can be used to generate file data of the file to be identified based on the multiple data packets; an extraction module 730, which can be used to extract feature data of the file data; and a determination module 740, which can be used to perform feature matching on the feature data to determine the file type of the file to be identified.
[0115] In some examples, the generation module 720 can be used to parse each data packet, determine the packet order, and reassemble the data packets according to the packet order to obtain file data.
[0116] In some examples, the extraction module 730 can be used to extract feature data from file data using a feature extraction database, including the Libmagic database.
[0117] In some examples, the extraction module 730 can be used to store multiple sub-file data included in the file data into multiple cache areas respectively, read the sub-file data in at least one cache area in sequence, use the file type detection features in the Libmagic database to match at least one sub-file data to obtain the feature data of at least one sub-file data, and determine the feature data of the file data based on the feature data of at least one sub-file data.
[0118] In some examples, the subfile data in at least one cache region consists of the first N bytes of the file data, where N is a positive integer.
[0119] In some examples, the determination module 740 can be used to perform feature matching on feature data using a matching algorithm to determine the file type of the file to be identified; wherein the matching algorithm includes the Aho-Corasick algorithm or the Hyperscan algorithm.
[0120] In some examples, after determining the file type of the file to be identified, the determination module 740 can also be used to generate a message digest of the file to be identified when the file type of the file to be identified is consistent with the preset file type; and determine the security detection result of the file to be identified based on the message digest.
[0121] The specific details of each module in the above-mentioned device have been described in detail in the method section of the implementation plan. For details of the undisclosed scheme, please refer to the implementation plan of the method section, and therefore will not be repeated here.
[0122] Figure 8 The diagram shows a structural schematic of an electronic device provided by an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.
[0123] like Figure 8 As shown, the electronic device may include: a processor 802, a communications interface 804, a memory 806, and a communications bus 808.
[0124] The processor 802, communication interface 804, and memory 806 communicate with each other via communication bus 808. Communication interface 804 is used to communicate with other network elements such as clients or other servers. Processor 802 executes program 810, specifically performing the relevant steps described in the embodiment of the file type identification method.
[0125] Specifically, program 810 may include program code, which includes computer-executable instructions.
[0126] Processor 802 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.
[0127] Memory 806 is used to store program 810. Memory 806 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.
[0128] Specifically, program 810 can be called by processor 802 to enable the electronic device to perform the operation steps of the above-mentioned file type identification method.
[0129] This invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform the file type identification method described in any of the above method embodiments.
[0130] The executable instructions can be used to cause the electronic device to perform the operation steps of the above-mentioned file type identification method.
[0131] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments of this invention are not directed to any particular programming language.
[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. Similarly, for the sake of brevity and to aid in understanding one or more aspects of the invention, in the description of exemplary embodiments of the invention above, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.
[0133] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.
[0134] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.
Claims
1. A method for identifying file types, characterized in that, The method includes: Obtain multiple data packets corresponding to the file to be identified transmitted over the network; Determine whether the transmission protocol of the multiple data packets is a file transfer protocol. If so, generate the file data of the file to be identified based on the multiple data packets. Create a queue and store the multiple sub-file data included in the file data into multiple cache areas corresponding to the queue; sequentially read the sub-file data from at least one cache area in the queue, wherein the sub-file data in the at least one cache area is the first N bytes of data in the file data, where N is a positive integer, and the first N bytes of data are used to characterize the file type features; By using file type detection features in the Libmagic database, at least one of the sub-file data is matched to obtain feature data of at least one of the sub-file data; The characteristic data of the file data is determined based on the characteristic data of at least one of the sub-file data; A matching algorithm is used to perform feature matching on the feature data to determine the file type of the file to be identified; wherein, the matching algorithm includes the Aho-Corasick algorithm or the Hyperscan algorithm; When the file type of the file to be identified matches the preset file type, an information digest of the file to be identified is generated; Based on the information digest, the security detection result of the file to be identified is determined, and the security detection result includes at least whether the file to be identified is a virus file and whether there is a security risk. The operations of extracting the feature data and determining the file type are performed asynchronously by a detection thread that is different from the main business operations.
2. The method according to claim 1, characterized in that, The step of generating file data for the file to be identified based on the plurality of data packets includes: Each of the data packets is parsed to determine the packet order. The data packets are reassembled according to their order to obtain the file data.
3. A document type identification device, characterized in that, The device includes: The acquisition module is used to acquire multiple data packets corresponding to the file to be identified transmitted over the network; The generation module is used to determine whether the transmission protocol of the multiple data packets is a file transfer protocol. If so, it generates the file data of the file to be identified based on the multiple data packets. An extraction module is used to create a queue and store multiple sub-file data, including the file data, into multiple cache areas corresponding to the queue; sequentially read sub-file data from at least one cache area in the queue, wherein the sub-file data in the at least one cache area is the first N bytes of data in the file data, where N is a positive integer, and the first N bytes of data are used to characterize file type features; match at least one sub-file data using file type detection features in the Libmagic database to obtain feature data of at least one sub-file data; and determine the feature data of the file data based on the feature data of at least one sub-file data. The determination module is used to perform feature matching on the feature data using a matching algorithm to determine the file type of the file to be identified, wherein the matching algorithm includes the Aho-Corasick algorithm or the Hyperscan algorithm; when the file type of the file to be identified matches a preset file type, an information digest of the file to be identified is generated; based on the information digest, a security detection result of the file to be identified is determined, wherein the security detection result includes at least whether the file to be identified is a virus file and whether there is a security risk; The operations of extracting the feature data and determining the file type are performed asynchronously by a detection thread that is different from the main business operations.
4. An electronic device, characterized in that, include: processor; Memory, used to store at least one executable instruction; The executable instructions cause the processor to perform the operation of the file type identification method as described in claim 1 or 2.
5. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the operation of the file type identification method as described in claim 1 or 2.