Virus file detection method and device based on flow management mechanism

Abstract

The invention relates to a virus file detection method and device based on a flow management mechanism. The method comprises the following steps: performing flow distribution processing on a network data packet based on a flow management mechanism so as to divide the network data packet into file flows; performing stream order-preserving processing on the file stream based on a stream management mechanism so as to distribute the file stream to a target CPU processor in a plurality of CPU processors; in each CPU processor, when a preset intelligent triggering condition is met, performing batch parallel abstract calculation on the file stream based on the SIMD instruction to obtain an abstract value; comparing the abstract value with a preset virus feature library to perform virus file detection; and executing corresponding security processing according to a comparison result. According to the virus file detection method and device based on the flow management mechanism, the throughput of abstract calculation can be remarkably improved, the detection delay is reduced, and the virus file detection capability of network equipment can be improved under the condition that the memory is not increased.

Description

Technical Field

[0001] This application relates to the field of computer information processing, and more specifically, to a method and apparatus for detecting virus files based on a flow management mechanism. Background Technology

[0002] In network firewall devices, virus detection based on file digest values ​​is a common security method. This technology requires identifying and extracting file transfer traffic passing through the device, calculating the digest value of the file data (such as MD5, SHA-256, etc.), and comparing it with a database of known virus sample digests. If the comparison is successful, the file is considered a virus. Calculating the digest value requires performing calculations on the entire file content, so the computational workload is significant and it consumes a lot of CPU resources.

[0003] Typical network security devices perform SHA / MD5 hash calculations on each data packet of the file data stream. After calculating the hash of the last data packet, a final hash value is obtained, which is then compared with virus sample data. Another approach is to cache the file data and then perform hash calculations on the cached file data. Hardware encryption cards can also be used to accelerate the process, offloading the computational task to hardware devices, which can significantly improve throughput performance, but increases equipment costs.

[0004] In summary, traditional methods of summarizing data have the following limitations: Serial computation is inefficient: The digest algorithm itself is a serial computation process. The computation of a single data stream cannot fully utilize the multi-core and vectorized processing capabilities of modern CPUs, especially the computation of MD5 values, which is strictly serialized.

[0005] Single-stream computation bottleneck: Even with SIMD instruction set optimization, existing implementations are mainly optimized for computation within a single data stream and cannot achieve multi-stream parallel processing.

[0006] Insufficient high-concurrency processing capability: When faced with a large number of concurrent file transfers, the serial computing method becomes a performance bottleneck, affecting the overall throughput of the firewall.

[0007] Therefore, a new method and device for detecting virus files based on a flow management mechanism is needed.

[0008] The information disclosed in the background section is only intended to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0009] In view of this, this application provides a virus file detection method and apparatus based on a flow management mechanism, which can significantly improve the throughput of digest calculation, reduce detection latency, and improve the virus file detection capability of network devices without increasing memory.

[0010] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0011] According to one aspect of this application, a virus file detection method based on a flow management mechanism is proposed. The method includes: performing flow allocation processing on network data packets based on the flow management mechanism to divide the network data packets into file streams; performing flow order preservation processing on the file streams based on the flow management mechanism to distribute the file streams to target CPU processors among multiple CPU processors; in each CPU processor, when a preset intelligent triggering condition is met, performing batch parallel digest calculation on the file streams based on SIMD instructions to obtain a digest value; comparing the digest value with a preset virus signature database to detect virus files; and performing corresponding security processing based on the comparison result.

[0012] In one exemplary embodiment of this application, network data packets are processed for flow allocation based on a flow management mechanism to divide the network data packets into file streams, including: parsing session identification information in the network data packets; and associating network data packets belonging to the same file transfer session as the same file stream according to the session identification information.

[0013] In one exemplary embodiment of this application, network data packets are processed for stream allocation based on a stream management mechanism to divide the network data packets into file streams, and the method further includes: extracting file data for digest calculation from the file streams.

[0014] In one exemplary embodiment of this application, the file stream is processed to preserve its order based on a stream management mechanism in order to distribute the file stream to a target CPU processor among multiple CPU processors. This includes: performing order correction processing on the network data packets corresponding to the file stream; and allocating the network data packets of the file stream to the target CPU processor based on a preset processor allocation rule.

[0015] In one exemplary embodiment of this application, in each CPU processor, when a preset smart trigger condition is met, batch parallel digest calculation is performed on the file stream based on SIMD instructions to obtain a digest value, including: maintaining a file stream queue in each CPU processor; for each file stream in the file stream queue, caching its corresponding data in the form of a data block linked list; and when the preset smart trigger condition is met, batch parallel digest calculation is performed on the file stream based on SIMD instructions.

[0016] In one exemplary embodiment of this application, when a preset smart trigger condition is met, batch parallel digest calculation is performed on the file streams based on SIMD instructions, including: when the preset smart trigger condition is met, selecting multiple file streams to be processed from the file stream queue; performing data block alignment processing on the file data in the multiple file streams; and performing digest calculation on the multiple file streams in parallel based on a single SIMD instruction.

[0017] In one exemplary embodiment of this application, the preset smart triggering conditions include: the number of file streams to be processed in the file stream queue of the CPU processor reaches a first threshold; the amount of unprocessed file data accumulated in the file stream queue of the CPU processor reaches a second threshold; and the last message block of any file stream is received.

[0018] In one exemplary embodiment of this application, performing digest calculations in parallel on the multiple file streams includes: storing the intermediate state of the digest calculation in the stream management structure corresponding to the stream management mechanism during the parallel execution of the digest calculation, so as to support performing digest calculations on the file streams in batches.

[0019] In one exemplary embodiment of this application, in each CPU processor, when a preset smart trigger condition is met, batch parallel digest calculation is performed on the file stream based on SIMD instructions. The method further includes: after completing one parallel digest calculation, for file streams that have not yet completed file transmission, writing back the corresponding intermediate state of digest calculation to the stream management mechanism for subsequent digest calculation; for file streams that have completed transmission and calculation, clearing their context in the stream queue and releasing the queue slots.

[0020] According to one aspect of this application, a virus file detection device based on a flow management mechanism is proposed. The device includes: an allocation module for performing flow allocation processing on network data packets based on the flow management mechanism to divide the network data packets into file streams; an order preservation module for performing flow order preservation processing on the file streams based on the flow management mechanism to distribute the file streams to target CPU processors among multiple CPU processors; a parallel module for performing batch parallel digest calculations on the file streams based on SIMD instructions in each CPU processor when a preset intelligent triggering condition is met, to obtain a digest value; a detection module for comparing the digest value with a preset virus signature database for virus file detection; and a processing module for performing corresponding security processing based on the comparison result.

[0021] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.

[0022] According to one aspect of this application, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described above.

[0023] According to the virus file detection method and apparatus based on the flow management mechanism of this application, network data packets are divided into file streams by performing flow allocation processing based on the flow management mechanism; the file streams are then distributed to target CPU processors among multiple CPU processors by performing flow order preservation processing based on the flow management mechanism; in each CPU processor, when a preset intelligent triggering condition is met, batch parallel digest calculation is performed on the file streams based on SIMD instructions to obtain a digest value; the digest value is compared with a preset virus feature database for virus file detection; and corresponding security processing is performed according to the comparison result. This method can significantly improve the throughput of digest calculation, reduce detection latency, and improve the virus file detection capability of network devices without increasing memory.

[0024] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0025] The above and other objects, features, and advantages of this application will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. The drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 This is a flowchart illustrating a virus file detection method based on a flow management mechanism according to an exemplary embodiment.

[0027] Figure 2 This is a schematic diagram illustrating a virus file detection method based on a flow management mechanism according to another exemplary embodiment.

[0028] Figure 3 This is a schematic diagram illustrating a virus file detection method based on a flow management mechanism according to another exemplary embodiment.

[0029] Figure 4 This is a schematic diagram illustrating a virus file detection method based on a flow management mechanism according to another exemplary embodiment.

[0030] Figure 5 This is a block diagram illustrating a virus file detection device based on a flow management mechanism according to an exemplary embodiment.

[0031] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0032] Figure 7 This is a block diagram illustrating a computer-readable medium according to an exemplary embodiment. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0034] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0038] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing this application, and therefore cannot be used to limit the scope of protection of this application.

[0039] The technical abbreviations used in this application are explained as follows: Digest algorithm: A one-way hash function that maps data of arbitrary length to a digest value of fixed length. It is used for data integrity verification and identity authentication, including algorithms such as MD5, SHA-1, SHA-256, and SHA-512.

[0040] SIMD: Single Instruction Multiple Data is a technique that uses a single controller to control multiple processors, which simultaneously perform the same operation on each of a set of data, thereby achieving spatial parallelism.

[0041] SSE / AVX: Streaming SIMD Extensions / Advanced Vector Extensions, are SIMD instruction set extensions for the x86 architecture used to accelerate multimedia and data processing applications.

[0042] CPU affinity: A technique that binds a process or thread to a specific CPU core for execution in order to reduce cache misses and improve performance.

[0043] Figure 1 This is a flowchart illustrating a virus file detection method based on a flow management mechanism according to an exemplary embodiment. The virus file detection method 10 based on the flow management mechanism includes at least steps S102 to S110.

[0044] like Figure 1 As shown, in S102, network data packets are processed for flow allocation based on a flow management mechanism to divide the network data packets into file streams. For example, session identification information in the network data packets can be parsed; based on the session identification information, network data packets belonging to the same file transfer session can be associated as the same file stream.

[0045] During this process, the system creates and maintains a digest computation context for each file stream. This context includes state variables (e.g., A, B, C, D for MD5), internal buffers, byte counters, etc., for the selected digest algorithm (such as MD5 or SHA-256). Crucially, this context is not allocated independently but is directly embedded or associated with the network device's inherent flow management structure, thereby avoiding additional memory allocation and data copying overhead, achieving lightweight integration.

[0046] It may also include, for example, extracting file data from the file stream for digest calculation.

[0047] In S104, the file stream is processed to preserve its order based on a flow management mechanism, thereby distributing the file stream to a target CPU processor among multiple CPU processors. For example, the network data packets corresponding to the file stream may be processed to correct their order; based on preset processor allocation rules, the network data packets of the file stream are allocated to the target CPU processor.

[0048] More specifically, the network data packets corresponding to the file stream are processed for order correction to handle TCP out-of-order issues and ensure data consistency; based on preset processor allocation rules and CPU affinity settings, all network data packets of the same file stream are allocated to the same target CPU processor to maintain cache locality and ensure that all subsequent digest calculations of the file stream are executed on the same CPU core.

[0049] In S106, in each CPU processor, when a preset smart trigger condition is met, batch parallel digest calculation is performed on the file stream based on SIMD instructions to obtain a digest value. For example, a file stream queue is maintained in each CPU processor; for each file stream in the file stream queue, its corresponding data is cached in the form of a data block linked list; when the preset smart trigger condition is met, batch parallel digest calculation is performed on the file stream based on SIMD instructions.

[0050] For example, a separate file stream queue can be maintained for each CPU core. For each file stream in the queue, the file data fragments extracted from network packets (along with length and offset information) are organized into data block linked list nodes for caching. The system implements memory control strategies, such as limiting a single file stream to caching a maximum of 5 data packets, to achieve a balance between performance and memory usage. When preset intelligent triggering conditions are met, batch parallel digest calculations are performed on the file streams based on SIMD instructions.

[0051] In one specific embodiment, each CPU queue manages multiple active file streams, selecting an appropriate number based on the instruction set used. SSE instructions typically support a maximum of 4 data sets, AVX typically supports 8, AVX2 supports 16, and there are other instruction sets as well. Each stream corresponds to a linked list of data blocks, with each node storing the file data fragment extracted from the message, its length, offset, and end marker.

[0052] In another specific embodiment, the memory control strategy ensures that a single file stream caches only a limited number of data packets (e.g., a maximum of 5), improving performance while keeping memory overhead negligible, making it perfectly suitable for high-performance network devices.

[0053] In one embodiment, when a preset smart trigger condition is met, batch parallel digest computation is performed on the file streams based on SIMD instructions, including: when the preset smart trigger condition is met, selecting multiple file streams to be processed from the file stream queue; performing data block alignment processing on the file data in the multiple file streams; and performing digest computation on the multiple file streams in parallel based on a single SIMD instruction.

[0054] More specifically, the preset intelligent triggering conditions include: the number of file streams to be processed in the file stream queue of the CPU processor reaches a first threshold; the amount of unprocessed file data accumulated in the file stream queue of the CPU processor reaches a second threshold; and the last message block of any file stream is received.

[0055] In practical applications, multi-condition intelligent triggers can be used to determine when to initiate batch computation, balancing computational throughput and detection real-time performance. Triggering conditions may include: Queue saturation trigger: When the number of active streams in a CPU queue reaches the upper limit (e.g., 8), batch computation is immediately triggered.

[0056] Data volume threshold trigger: When the accumulated data volume of a single file stream reaches a preset threshold (such as 2KB), calculation is triggered to prevent excessive data in a single stream from causing calculation delay.

[0057] Stream completion trigger: When the last message block of a file stream is received, the calculation is triggered immediately to ensure the real-time nature of virus detection.

[0058] In practical applications, for example, during parallel digest computation, the intermediate state of the digest computation can be stored in the stream management structure corresponding to the stream management mechanism to support batch digest computation of file streams. This ensures that the computation state is bound to the lifecycle of the network stream, resulting in extremely low context switching overhead.

[0059] In one embodiment, the method further includes: after completing a parallel digest calculation, for file streams that have not yet completed file transmission, writing back the corresponding intermediate state of the digest calculation to the stream management mechanism for subsequent digest calculation; for file streams that have completed transmission and calculation, clearing their context in the stream queue and releasing the queue slots.

[0060] Figure 2 This is a schematic diagram illustrating a virus file detection method based on a flow management mechanism, according to another exemplary embodiment. For example... Figure 2As shown, the data block alignment process refers to the parallel computing engine extracting aligned data blocks of 64 bytes (one standard digest calculation block) from the data block linked lists of each file stream. For any remaining data in each stream that cannot be aligned due to its current length being less than 64 bytes, it is retained in the buffer of its respective stream context, awaiting combination with data from subsequent messages to form a complete calculation block. Subsequently, using SIMD instructions such as SSE, AVX, or AVX2, these aligned 64-byte data blocks from different streams are simultaneously loaded into 128-bit, 256-bit, or 512-bit wide registers. Multiple streams are then executed in parallel using a single instruction (e.g., SSE supports 4, AVX2 supports 8 or 16), performing the same calculation steps (e.g., one round of MD5 transformation). For file streams that have received the end-of-flight (EOF) marker, the last block of data is calculated serially, regardless of alignment, to ensure the correctness of the result.

[0061] Figure 2 It is a schematic diagram of parallel computing, which intuitively shows the process of extracting aligned data blocks from a linked list of data blocks in multiple file streams, processing them in parallel through the SIMD engine, and processing the remaining unaligned data and updating the context.

[0062] More specifically, the engine extracts aligned data blocks, each 64 bytes long (one digest block), from the data block list of each file stream. Any remaining data in each stream that cannot be aligned due to insufficient length is retained in its respective stream context buffer, awaiting combination with subsequent message data.

[0063] Using SIMD instructions such as SSE and AVX2, aligned data blocks from multiple streams (e.g., 4 or 8) can be loaded into a wide-bit register simultaneously, and the same computation steps (e.g., one round of MD5 transformation) from multiple streams can be executed in parallel with a single instruction.

[0064] It supports multiple digest algorithms, and the system automatically selects the optimal computation strategy based on algorithm characteristics and hardware capabilities. It dynamically selects algorithm implementations such as SSE, AVX, and AVX2 based on different hardware, prioritizing implementations with stronger parallel computing capabilities.

[0065] For the last block of data in a completed stream (EOF received), regardless of alignment, the remaining portion is calculated serially to ensure the accuracy and real-time nature of the digest result. The end-of-file marker can be obtained through fields such as file length in the transmission protocol; for example, when downloading files via HTTP, a `content-length:` field typically indicates the file length. (How to determine the end of a file is not the focus of this invention.) In step S108, the digest value is compared with a preset virus signature database to detect virus files. After batch calculation, the system quickly compares the generated digest value with the virus signature database and executes the corresponding security policy based on the result.

[0066] In S110, corresponding security actions are performed based on the comparison results. Security actions may include, but are not limited to: recording security logs, generating alarms, blocking network transmission of the file stream, or discarding identified virus file data packets.

[0067] According to the virus file detection method based on the flow management mechanism of this application, network data packets are divided into file streams by flow allocation processing based on the flow management mechanism; the file streams are then distributed to target CPU processors among multiple CPU processors by flow order preservation processing based on the flow management mechanism; in each CPU processor, when a preset intelligent trigger condition is met, batch parallel digest calculation is performed on the file streams based on SIMD instructions to obtain a digest value; the digest value is compared with a preset virus feature database for virus file detection; and corresponding security processing is performed according to the comparison result. This method can significantly improve the throughput of digest calculation, reduce detection latency, and improve the virus file detection capability of network devices without increasing memory.

[0068] It should be clearly understood that this application describes how specific examples are formed and used, but the principles of this application are not limited to any details of these examples. Rather, based on the teachings of the disclosure of this application, these principles can be applied to many other embodiments.

[0069] Figure 3 This is a schematic diagram illustrating a virus file detection method based on a stream management mechanism according to another exemplary embodiment. This application provides a multi-stream digest batch calculation method based on SIMD instructions, employing a multi-stream digest batch calculation scheme deeply integrated with a stream management mechanism. Through per-CPU queue management and an intelligent triggering mechanism, it achieves parallel digest calculation of multiple independent data streams. Figure 3 The system architecture of this application is described exemplarily. The core of this application lies in seamlessly embedding the digest calculation process into the existing flow management framework of network devices: State integration: The intermediate states of digest computation for each network stream (Session) (such as MD5 A, B, C, D state variables, buffer data, byte counters, etc.) are directly stored in the device’s inherent stream management structure, without the need for additional memory allocation, thus avoiding data copying overhead.

[0070] Data association: By using the Session ID field in the flow management structure, network packets are precisely associated with their corresponding digest calculation contexts to ensure computational continuity.

[0071] Stream allocation and order preservation: The stream order preservation module ensures that all packets in the same data stream are sent to the same CPU core for processing, while also handling TCP out-of-order issues and ensuring data consistency.

[0072] Figure 4 This is a schematic diagram illustrating a virus file detection method based on a flow management mechanism according to another exemplary embodiment. Figure 4 The basic processing procedure of this application is described exemplarily. For example... Figure 4 As shown, the method may include the following processing stages and logic modules: Stream classification processing: The system receives network data packets and performs preliminary analysis to identify and extract data packets belonging to file transfer sessions, classifying them into different file streams. This stage mainly relies on network protocol characteristics (such as HTTP headers, FTP commands, etc.) and session identification information for stream classification.

[0073] Stream order preservation and CPU affinity binding: To ensure that packets from the same file stream are processed in the correct order and executed on the same CPU core, the system performs stream order preservation processing on the classified file streams (e.g., reassembling out-of-order TCP packets) and binds each file stream to a specific target CPU processor based on a CPU affinity strategy. Simultaneously, a digest computation context (including algorithm state, buffers, etc.) corresponding to that file stream is created and stored in the system's inherent stream management structure, achieving state-stream binding.

[0074] Per-CPU Stream Queue: Each CPU core maintains an independent stream queue to manage all file streams allocated to that core. Each file stream corresponds to a linked list of data blocks in the queue, used to cache file data fragments extracted from the stream's data packets that have not yet undergone digest calculation.

[0075] Intelligent triggering processing: The system continuously monitors the stream queue status of each CPU core. When preset triggering conditions are met (e.g., the number of active streams in the queue reaches a threshold, the accumulated data volume of a file stream reaches a threshold, or the transmission of a file stream is detected to have ended), subsequent batch calculations are triggered.

[0076] SIMD Parallel Computing Processing: Once the triggering condition is met, the system selects multiple file streams from the current CPU's stream queue and starts the SIMD parallel computing engine. This engine extracts length-aligned data blocks (e.g., 64 bytes) from the data block linked lists of each file stream and uses SIMD instructions such as SSE and AVX to execute the summary computation steps of multiple streams in parallel within a single instruction. During the computation process, intermediate states stored in the stream management structure are read / updated.

[0077] Virus signature database comparison and security policy execution: After calculation, the obtained file digest value (such as MD5, SHA-256) will be compared with the virus signature database. If a match is found, the file is identified as a virus. The system then executes the security policy, which may include generating a policy log and performing corresponding network response actions, such as blocking connections, discarding files, or issuing alerts.

[0078] The entire process uses a flow management structure as the core of state storage, enabling flow-aware, parallel, and efficient virus file detection, which is especially suitable for high-concurrency network environments.

[0079] Those skilled in the art will understand that all or part of the steps of the above embodiments are implemented as a computer program executed by a CPU. When the computer program is executed by the CPU, it performs the functions defined by the method provided in this application. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0080] Furthermore, it should be noted that the above figures are merely illustrative representations of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0081] The following are embodiments of the apparatus of this application, which can be used to execute the embodiments of the method of this application. For details not disclosed in the embodiments of the apparatus of this application, please refer to the embodiments of the method of this application.

[0082] Figure 5 This is a block diagram illustrating a virus file detection device based on a flow management mechanism according to an exemplary embodiment. Figure 5 As shown, the virus file detection device 50 based on the flow management mechanism includes: an allocation module 502, an order preservation module 504, a parallel module 506, a detection module 508, and a processing module 510.

[0083] The allocation module 502 is used to perform flow allocation processing on network data packets based on the flow management mechanism, so as to divide the network data packets into file streams; the allocation module 502 is also used to parse the session identification information in the network data packets; and associate network data packets belonging to the same file transfer session as the same file stream according to the session identification information.

[0084] The allocation module 502 is also used to extract file data from the file stream for digest calculation.

[0085] The order preservation module 504 is used to perform flow order preservation processing on the file stream based on the flow management mechanism, so as to distribute the file stream to the target CPU processor among multiple CPU processors; the order preservation module 504 is also used to perform order correction processing on the network data packets corresponding to the file stream; and allocate the network data packets of the file stream to the target CPU processor based on the preset processor allocation rules.

[0086] The parallel module 506 is used in each CPU processor to perform batch parallel digest calculations on file streams based on SIMD instructions to obtain digest values ​​when preset smart trigger conditions are met. The parallel module 506 is also used to maintain a file stream queue in each CPU processor; for each file stream in the file stream queue, its corresponding data is cached in the form of a data block linked list; when preset smart trigger conditions are met, batch parallel digest calculations are performed on the file streams based on SIMD instructions. The parallel module 506 is also used to, after completing a parallel digest calculation, write back the corresponding intermediate digest calculation state to the stream management mechanism for file streams that have not yet completed file transmission for subsequent digest calculations; for file streams that have completed transmission and calculation, clean up their context in the stream queue and release queue slots.

[0087] The detection module 508 is used to compare the summary value with a preset virus feature library to detect virus files; The processing module 510 is used to perform corresponding security processing based on the comparison results.

[0088] The virus file detection device based on the flow management mechanism of this application divides network data packets into file streams by performing flow allocation processing based on the flow management mechanism; performs flow order preservation processing on the file streams based on the flow management mechanism to distribute the file streams to target CPU processors among multiple CPU processors; in each CPU processor, when a preset intelligent trigger condition is met, batch parallel digest calculation is performed on the file streams based on SIMD instructions to obtain a digest value; the digest value is compared with a preset virus feature database to detect virus files; and corresponding security processing is performed according to the comparison result. This method can significantly improve the throughput of digest calculation, reduce detection latency, and improve the virus file detection capability of network devices without increasing memory.

[0089] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0090] The following reference Figure 6 To describe an electronic device 600 according to this embodiment of the present application. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0091] like Figure 6 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0092] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in this specification according to various exemplary embodiments of this application. For example, the processing unit 610 can perform actions such as... Figure 1 , Figure 2 , Figure 3 , Figure 4 The steps are shown in the figure.

[0093] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only memory unit (ROM) 6203.

[0094] The storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0095] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0096] Electronic device 600 can also communicate with one or more external devices 600' (e.g., keyboard, pointing device, Bluetooth device, etc.), enabling users to communicate with devices that interact with electronic device 600, and / or any device that allows electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0097] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 7 As shown, the technical solution according to the embodiments of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of this application.

[0098] In summary, this disclosure addresses the problem of low efficiency in file data digest computation in network security devices, particularly the performance bottleneck in high-concurrency file transfer scenarios. It provides a multi-stream digest batch computation method that fully utilizes modern CPU multi-core and SIMD instruction sets, applicable to various digest algorithms (MD5, SHA series, etc.). Specifically, this disclosure offers a SIMD instruction-based multi-stream digest batch computation method, employing a multi-stream digest batch computation scheme deeply integrated with the flow management mechanism. Through per-CPU queue management and intelligent triggering mechanisms, parallel digest computation of multiple independent data streams is achieved. This disclosure seamlessly embeds the digest computation process into the existing flow management framework of network devices. During state integration, the intermediate states of digest computation for each network flow (Session) (such as MD5 A, B, C, D state variables, buffer data, byte counters, etc.) are directly stored in the device's inherent flow management structure, eliminating the need for additional memory allocation and avoiding data copying overhead. During data association, the Session ID field in the flow management structure precisely associates network packets with their corresponding digest computation contexts, ensuring computational continuity. During stream allocation and order preservation, the stream order preservation module ensures that all packets from the same data stream are sent to the same CPU core for processing, while also handling TCP out-of-order issues and guaranteeing data consistency. In the management of file stream queues and data block linked lists per CPU, each CPU core maintains an independent management queue for organizing and scheduling file stream data to be processed. Specifically, in the queue structure, each CPU queue manages multiple active file streams, selecting an appropriate number based on the instruction set used. SSE instructions typically support a maximum of 4 data sets, AVX typically supports 8, AVX2 supports 16, and there are other instruction sets as well. Each stream corresponds to a data block linked list, with linked list nodes storing the file data fragments extracted from the packets, their length, offset, and end marker. For memory optimization, memory control strategies ensure that a single file stream caches a maximum of a limited number of data packets (e.g., a maximum of 5), improving performance while keeping memory overhead negligible, making it perfectly suitable for high-performance network devices. In the flexible triggering and dynamic batch processing mechanism, multi-condition intelligent triggers determine when to initiate batch computation to balance computational throughput and real-time detection. Specifically, the computation is triggered under the following conditions: **Queue saturation:** When the number of active streams in a CPU queue reaches its limit (e.g., 8), batch computation is immediately triggered. **Data volume threshold trigger:** When the accumulated data volume of a single file stream reaches a preset threshold (e.g., 2KB), computation is triggered to prevent computational delays caused by excessive data in a single stream. **Stream completion trigger:** When the last packet block of a file stream is received, computation is immediately triggered to ensure real-time virus detection. After computation, the current CPU's cache queue is cleared in real time, awaiting the arrival of new database entries. The SIMD parallel computing engine for network data is the core of performance acceleration.First, data preparation and alignment: The engine extracts aligned data blocks of 64 bytes each (one digest calculation block) from the data block linked list of each file stream. For any remaining data in each stream that cannot be aligned due to insufficient length, it is retained in its respective stream context buffer, awaiting combination with subsequent message data. Next, parallel computation is performed: Using SIMD instructions such as SSE and AVX2, aligned data blocks from multiple streams (e.g., 4 or 8) are simultaneously loaded into wide-bit registers, and the same computation steps (e.g., one round of MD5 transformation) are executed in parallel using a single instruction. Third, dynamic algorithm adaptation: Supporting multiple digest algorithms, the system automatically selects the optimal computation strategy based on algorithm characteristics and hardware capabilities. It dynamically selects algorithm implementations such as SSE, AVX, and AVX2 based on different hardware, prioritizing implementations with stronger parallel computing capabilities. Finally, tail processing is performed: For the last block of data in a completed stream (received EOF), regardless of alignment, its remaining portion is calculated serially to ensure the correctness and real-time performance of the digest result. The end-of-file marker can be obtained through fields such as file length in the transmission protocol. For example, when downloading a file via HTTP, there is usually a `content-length:` field indicating the file length. This disclosure also processes and feeds back the calculation results. Specifically, after batch calculations are completed, the system quickly compares the generated digest value with a virus signature database and executes corresponding security policies based on the results. For file streams that have not received the last piece of data, it is only necessary to write the intermediate state back to the stream management structure and clean up the stream context that has completed the calculations, releasing queue slots to receive new file streams.

[0099] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0100] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0101] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0102] The aforementioned computer-readable medium carries one or more programs. When these programs are executed by a device, the computer-readable medium performs the following functions: it performs stream allocation processing on network data packets based on a stream management mechanism to divide the network data packets into file streams; it performs stream ordering processing on the file streams based on the stream management mechanism to distribute the file streams to target CPU processors among multiple CPU processors; in each CPU processor, when a preset intelligent trigger condition is met, it performs batch parallel digest calculation on the file streams based on SIMD instructions to obtain a digest value; it compares the digest value with a preset virus signature database to detect virus files; and it performs corresponding security processing based on the comparison result.

[0103] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and located in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0104] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0105] Exemplary embodiments of this application have been specifically shown and described above. It should be understood that this application is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, this application is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A virus file detection method based on a flow management mechanism, characterized in that, include: The network data packets are divided into file streams based on the flow management mechanism. The file stream is processed to preserve its order based on a stream management mechanism, so as to distribute the file stream to the target CPU processor among multiple CPU processors; In each CPU processor, when the preset smart triggering conditions are met, batch parallel digest calculations are performed on the file stream based on SIMD instructions to obtain the digest value; The summary value is compared with a preset virus signature database to detect virus files; Perform the corresponding security procedures based on the comparison results.

2. The method as described in claim 1, characterized in that, The network data packets are allocated and processed based on a flow management mechanism to divide the network data packets into file streams, including: Parse the session identification information in the network data packets; Based on the session identification information, network data packets belonging to the same file transfer session are associated as the same file stream.

3. The method as described in claim 2, characterized in that, Based on the flow management mechanism, network data packets are allocated and processed to divide the network data packets into file streams, and the process further includes: Extract file data for digest calculation from the file stream.

4. The method as described in claim 1, characterized in that, The file stream is processed using a stream management mechanism to preserve its order, thereby distributing the file stream to target CPU processors across multiple CPU processors, including: Perform sequence correction processing on the network data packets corresponding to the file stream; Based on preset processor allocation rules, the network data packets of the file stream are allocated to the target CPU processor.

5. The method as described in claim 1, characterized in that, In each CPU processor, when preset smart triggering conditions are met, batch parallel digest calculations are performed on the file stream based on SIMD instructions to obtain digest values, including: Maintain a file stream queue in each CPU processor; For each file stream in the file stream queue, its corresponding data is cached in the form of a linked list of data blocks; When the preset intelligent triggering conditions are met, batch parallel digest calculations are performed on the file stream based on SIMD instructions.

6. The method as described in claim 5, characterized in that, When preset smart triggering conditions are met, batch parallel digest computation is performed on the file stream based on SIMD instructions, including: When the preset intelligent triggering conditions are met, multiple file streams to be processed are selected from the file stream queue; Perform data block alignment processing on the file data in the multiple file streams; Digest computation is performed in parallel on the multiple file streams based on a single SIMD instruction.

7. The method as described in claim 6, characterized in that, The preset smart trigger conditions include: The number of file streams to be processed in the file stream queue of the CPU processor has reached the first threshold. The amount of unprocessed file data accumulated in the file stream queue of the CPU processor reaches the second threshold; The last message block of any file stream has been received.

8. The method as described in claim 6, characterized in that, Performing digest computation in parallel on the multiple file streams includes: During the parallel execution of digest computation, the intermediate state of digest computation is stored in the stream management structure corresponding to the stream management mechanism to support digest computation of file streams in batches.

9. The method as described in claim 1, characterized in that, In each CPU processor, when preset smart triggering conditions are met, batch parallel digest computation of the file stream based on SIMD instructions is performed, which also includes: After completing a parallel digest calculation, for file streams that have not yet completed file transfer, the corresponding intermediate state of the digest calculation is written back to the stream management mechanism for use in subsequent digest calculations. For file streams that have finished transmitting and completing computation, clean up their context in the stream queue and release the queue slots.

10. A virus file detection device based on a flow management mechanism, characterized in that, include: The allocation module is used to perform flow allocation processing on network data packets based on the flow management mechanism, so as to divide the network data packets into file streams; The order preservation module is used to perform stream order preservation processing on the file stream based on the stream management mechanism, so as to distribute the file stream to the target CPU processor among multiple CPU processors; The parallel module is used in each CPU processor to perform batch parallel digest calculations on the file stream based on SIMD instructions to obtain digest values ​​when preset smart trigger conditions are met. The detection module is used to compare the summary value with a preset virus feature database to detect virus files; The processing module is used to perform corresponding security processing based on the comparison results.

Images