High-speed keyword filtering method based on FPGA and CPU collaboration
Through the collaboration of FPGA and CPU, the parallel processing capability of FPGA and the management capability of CPU are used to realize high-speed keyword filtering, solving the problems of slow CPU processing rate and insufficient multi-session filtering capabilities in the existing technology, achieving 5.6Gbps filtering Quick filtering effect of rate and multi-session and multi-keywords.
Patent Information
- Application Number
- CN202210509758.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-11
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-05-11
AI Technical Summary
The existing keyword filtering technology mainly relies on CPU implementation, resulting in a slow processing rate, which cannot meet the needs of high-speed data transmission. In the case of multi-data packet connection, the CPU capability is insufficient, making it difficult to achieve efficient filtering for multiple sessions.
FPGA and CPU work together, use the high-speed computing power of FPGA to filter keywords, the CPU is responsible for transaction management and data format conversion, and realizes keyword issuance and filtering results through the PCIE interface, making full use of FPGA parallel processing and RAM resources realize rapid filtering of multiple sessions and multiple keywords.
It realizes a high-speed keyword filtering rate, reaching 5.6Gbps, supports rapid filtering of multiple sessions and multiple keywords, improves filtering performance, and has the advantages of accurate and efficient filtering.
Smart Images

Figure CN114968872B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of data information security application, adopts FPGA and CPU to work in coordination, and relates to a method for implementing high-speed data stream keyword filtering. Background Art
[0002] Currently, keyword filtering is mostly implemented by high-performance CPUs, and data packets are searched through software implementation methods. The CPU implementation rate is slow and cannot meet the high-speed data transmission performance; the memory occupancy rate is high, which affects the overall performance and power consumption of the device. Existing search technologies are mostly limited to filtering the current data packet message, and it is difficult to achieve keyword filtering for multiple data packet connections. When there are many filtering threads, the CPU capacity is obviously insufficient.
[0003] With the increasing number of data communication services, information security issues are becoming increasingly prominent. During the communication process, some sensitive information and words may be mixed in the text for transmission, causing security information leakage. In order to prevent such incidents from happening, it is necessary to filter the content of the data message during the communication process to see if there are keywords. The current keyword filtering technology is mainly implemented by the CPU, which filters the keywords during the communication process. However, the CPU is slow in processing such events, which seriously affects the data communication rate. Summary of the invention
[0004] In view of the above-mentioned prior art, the present invention proposes a method for implementing high-speed data stream keyword filtering, which adopts CPU and FPGA to jointly implement keyword filtering, wherein the characteristic of FPGA that can be used for keyword filtering in situations with higher processing rates is fully utilized, and the CPU is used to handle some transaction-type management.
[0005] In order to solve the above technical problems, the present invention proposes a high-speed keyword filtering method based on the collaborative work of FPGA and CPU. In the CPU, the data sending thread and the data receiving thread are two threads, and the data sending thread includes the following steps:
[0006] 1) Keyword setting: Each session sets 1-8 groups of keywords, each group of keywords is 1-32 bytes long, the keyword configuration space for each session in the FPGA internal cache is 256 bytes, each group of keywords is configured with a corresponding session number, and the FPGA internal cache supports up to 64 sessions; receive the keyword configured by the computer through the serial port, the length of the keyword is recorded as L; determine the size of L,
[0007] If L<32byte, use KEY_MASK to indicate the effective length of the keyword, generate the keyword KEY and its keyword mask KEY_MASK, and execute step 2);
[0008] If 32byte<L≤256byte, the keyword space is cascaded to generate the keyword KEY and its keyword mask KEY_MASK, and DATA_MASK is used to indicate the cascade relationship between the keywords, and step 2) is executed;
[0009] If L>256byte, a message is sent back to the computer indicating that the keyword length is too long;
[0010] 2) Keyword distribution: The generated keyword session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship represented by DATA_MASK are distributed to the FPGA internal cache through PCIE;
[0011] At the same time, the CPU determines whether the file format of the data packet of the current session is a plain text data format. If it is a plain text data format, the data packet of the current session is sent to the internal cache of the FPGA through PCIE; if it is not a plain text data format, the data packet of the current session is cached until the data packet of the current session is received, and the file data in the non-text data format is converted into data in the plain text data format and sent to the internal cache of the FPGA through PCIE;
[0012] 3) Keyword filtering: The FPGA performs keyword filtering on the data packets received from the CPU according to the keyword information corresponding to the session number in the data packets;
[0013] 4) Determine whether the filtered keywords have a cascade relationship DATA_MASK. If so, enter the cascade result processing. If not, execute step 5);
[0014] Cascade result processing: first determine whether there is a cascade relationship between keyword 1 and keyword 2. If there is no cascade between keyword 1 and keyword 2, output the result of keyword 1 filtering and enter the cascade judgment state of keyword 2 and keyword 3. If there is a cascade relationship between keyword 1 and keyword 2, directly enter the cascade judgment state of keyword 2 and keyword 3. In the cascade judgment state of keyword 2 and keyword 3, determine the cascade state of keyword 2 and keyword 3. If there is no cascade relationship between keyword 2 and keyword 3, it is necessary to output the cascade relationship between keyword 1 and keyword 2 and the filtering result and then enter the cascade judgment state of keyword 3 and keyword 4. If there is no cascade relationship, directly enter the cascade judgment state of keyword 3 and keyword 4, and so on, until the relationship between all keyword filtering results and cascades is determined, the cascade and filtering results are output, and step 5 is executed;
[0015] Step 5) Report the keyword filtering result to the CPU, the reported result includes the filtering hit result, session number, keyword hit position, and if the keywords that have completed the filtering have a cascade relationship DATA_MASK, the cascade result is also included;
[0016] The data receiving thread is always waiting for a data packet with keyword filtering results reported by the FPGA end; if so, the filtering results are sent to the data source end through the external interface.
[0017] Furthermore, the high-speed keyword filtering method based on the collaborative work of FPGA and CPU described in the present invention comprises:
[0018] In step 2), the generated keyword information and session number are sent to the internal cache of FPGA through PCIE, and the keyword information is read according to the session number when a data packet arrives. The user configures keyword filtering information according to the session number, and the keyword information is stored in FPGA according to the session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship DATA_MASK; when a data packet arrives, the keyword information of the current session is taken out in FPGA according to the session number of the keyword in the data packet for update.
[0019] The process of step 3) is as follows:
[0020] 3-1) Update keywords according to session number: according to the session number in the data packet, take out the keyword information of the current session number in the keyword cache;
[0021] 3-2) Determine the packet sequence number of the received data packet: After extracting the keyword information corresponding to the session number, determine whether the received data packet is the first packet. If it is the first packet, determine the length of the data packet.
[0022] If it is not the first packet, it is popped out of the stack according to the session number of the data packet. The stack includes the filtered positions and the unfinished filtering results;
[0023] 3-3) Determine the length of the data packet to determine whether the length of the data packet is greater than 32 bytes. If it is ≥32 bytes, execute step 3-4) to perform keyword filtering. If it is <32 bytes, cache the current data packet, update the cache space according to the size of the current data packet, and execute step 3-4) to perform keyword filtering;
[0024] 3-4) Keyword filtering: Filter keywords in the data packet according to the keyword information extracted according to the session number. First, copy the received data packet 8 times, one for each keyword group. Copy each copy into 32 paths according to the maximum length of the keyword. The offset address of each path is 0B, 1B, 2B, 3B, ..., 30B, 31B. Perform parallel keyword filtering according to different address offsets, so as to traverse all keyword combinations in the data packet.
[0025] 3-5) Determine whether filtering is completed: If the current data packet filtering is completed, it enters the filtering state and is pushed into the stack. If it is not completed, it needs to wait for keyword filtering until it is completed;
[0026] 3-6) Push filter status into stack: put the filtered session number, filtered position, and unfinished filtering results into the stack, and wait for the next data packet of the same session to arrive before popping the filter status out of the stack.
[0027] In step 3-2), the data packet is popped according to the session number: for the unfinished keyword filtering results, the filtered session number, the filtered position, and the unfinished filtering results are placed in the stack. After the next packet of data of the current session arrives, the information pushed into the stack is popped and the data is filtered together with the current data to solve the situation where the keyword appears in adjacent data packets.
[0028] In the present invention, the clock used by the FPGA is 100MHz, the data stream bit width is 256bit, and the theoretical rate is 100MHZ*256bit≈25Gbps; 64 sessions and 8 groups of keywords are used, and filtering is performed with each group of keywords having a length of 0 to 32B. The filtering rate is 5.6Gbps under 10G network communication.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] The invention discloses a high-speed keyword filtering method based on the collaborative work of FPGA and CPU. The method realizes the high-speed keyword filtering implementation method by dividing the functions in the keyword retrieval process, utilizing the efficient business processing capability of CPU and the high-speed computing capability of FPGA. The file type conversion is realized on the CPU side, and the data packets such as word, excel, PPT, xml, etc. are cached and converted, and finally a data packet in a plain text format is obtained. The parallel working capability of FPGA is fully utilized to realize a high-speed keyword filtering engine inside FPGA. The filtering engine includes keyword filtering, data packet splicing filtering, data packet cache filtering, and keyword splicing filtering. The rich RAM resources inside FPGA are fully utilized, and the filtering scenes of different sessions need to be maintained for simultaneous filtering of multiple sessions, so that fast switching of different session scenes can be achieved. Fast filtering of multiple sessions and multiple keywords has been achieved. Multiple sessions and multiple keywords are supported to be searched simultaneously during the filtering process. The CPU and FPGA are used to work together and their respective characteristics are fully utilized to achieve filtering performance of 5.6 Gbps. It has the advantages of fast filtering speed and accurate filtering of multiple sessions and multiple keywords. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is the keyword filtering full flow chart of the present invention and the keyword filtering implementation block diagram;
[0032] Figure 2 It is the storage format of keywords in the FPGA in the present invention;
[0033] Figure 3 It is a flowchart of keyword filtering inside FPGA in the present invention;
[0034] Figure 4 It is a schematic diagram of the small data packet buffering principle of the present invention;
[0035] Figure 5 It is a block diagram of FPGA implementation of multi-keyword filtering in the present invention;
[0036] Figure 6 It is a schematic diagram of multi-address offset parallel filtering in the present invention;
[0037] Figure 7 This is the keyword cascade result processing in the present invention. DETAILED DESCRIPTION
[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.
[0039] The design idea of a high-speed keyword filtering method based on the collaborative work of FPGA and CPU of the present invention is: based on PCIE communication, the CPU is used as the master device and the FPGA is used as the slave device to work together, so as to realize high-speed data stream keyword filtering. This filtering method can perform keyword filtering on multiple sessions, multiple keywords, and multiple data formats. Among them, the CPU performs data format conversion on data packets that are not in plain text data format, converts them into plain text data packet format, and the FPGA performs high-speed keyword filtering on the data stream sent by the CPU. The data interface between the CPU and FPGA is PCIE, and there are two threads inside the CPU: one data sending thread, which is used for the CPU to send data to the FPGA through PCIE; the other is a data receiving thread, which is used for the CPU to read the keyword filtering results in the FPGA through PCIE. See Figure 1 The filtering method of the present invention comprises the following steps:
[0040] 1. The data sending thread comprises the following steps:
[0041] 1. Keyword setting: In the present invention, each session supports 8 groups of keywords, each group of keywords is 1-32 bytes in size, and the maximum keyword configuration space supported in the FPGA internal cache is 32 bytes*8=256 bytes. To set a keyword, you need to specify the session number of the keyword. In the present invention, a maximum of 64 sessions can be supported.
[0042] The CPU receives the keywords configured by the computer through the serial port. The minimum space occupied by each keyword is 32 bytes, and the total space occupied by all keywords cannot exceed 256 bytes.
[0043] If the length of the keyword is less than 32 bytes, KEY_MASK is used to indicate the effective length of the keyword, and the keyword KEY and its keyword mask KEY_MASK are generated. The function of KEY_MASK is to remove the part of the keyword that does not need to be filtered, so that the filtering range of the keyword is 0B to 32B.
[0044] If the length of the keyword is between 32 bytes and 256 bytes, the keyword space needs to be cascaded to generate the keyword KEY and its keyword mask KEY_MASK, and DATA_MASK is used to indicate the cascade relationship between keywords. In actual use, DATA_MASK is introduced to indicate the cascade relationship between keywords, so that the maximum filtering range of keywords can reach 32B*8=256B. This application sacrifices the number of filtered keywords in exchange for longer keywords to filter the text.
[0045] If the length of the keyword is greater than 256 bytes, a prompt indicating that the keyword length is too long will be sent back to the computer.
[0046] 2. Keyword delivery and caching: The session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship represented by DATA_MASK of the specified keyword are delivered to the FPGA internal cache through PCIE, and stored in the FPGA internal cache in a table format. The storage format is as follows: Figure 2 As shown, a maximum of 64 sessions are supported internally.
[0047] When a data packet arrives, the keyword information is read according to the session number. The user configures the keyword filtering information according to the session number. The keyword information is stored in the FPGA according to the session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship DATA_MASK; when a data packet arrives, the keyword information of the current session is taken out from the FPGA according to the session number of the keyword in the data packet for update.
[0048] At the same time, the CPU receives the data stream sent by the upstream data, and determines from the packet header information whether the received data is plain text data or data in other formats, because the data source sent by the upstream data may be word, excel, PPT, xml, etc., and the data in these formats cannot be correctly recognized by the FPGA when sent, and they need to be format converted before they can be recognized. If it is a plain text data format, the data packet of the current session is sent to the internal cache of the FPGA through PCIE; if the data format conversion program finds that the data format of a session is not plain text data, the data packet of the current session needs to be cached until the file data packet of the current session is received, and the file data in non-text data format is converted into data in plain text data format and sent to the internal cache of the FPGA through PCIE.
[0049] 3. Keyword filtering: The keyword filtering program performs keyword filtering. After receiving the data packet, the FPGA extracts the session number from the data packet, extracts the keyword information from the keyword cache according to the session number, and uses the extracted keyword information to perform keyword filtering on the data packet received from the CPU. Figure 3 As shown, the specific process is as follows:
[0050] Update keywords according to the session number: According to the session number in the received data packet, the keyword information of the current session number is taken out from the keyword cache and the keyword is updated.
[0051] Determination of whether the data packet is the first packet: The keyword filtering program inside the FPGA receives the keyword to be filtered, determines the packet sequence number of the received data packet, and after taking out the keyword information corresponding to the session number, determines whether the received data packet is the first packet. If it is the first packet, the data packet length is determined. If it is not the first packet, it is popped out of the stack according to the session number of the data packet. That is, for the unfinished keyword filtering results, the filtered session number, the filtered position, and the unfinished filtering results are placed in the stack. After the next packet of data of the current session arrives, the information pushed into the stack is popped out of the stack and the data is filtered together with the current data to solve the situation where the keyword appears in adjacent data packets.
[0052] For unfinished keyword filtering, the present invention introduces the concept of stack in computer. For the keyword processing program of unfinished filtering, the filtered session number, the filtered position, the unfinished filtering result, etc. are put into the stack. When the next packet of data of the current session comes, the information put into the stack is popped out, and the data is filtered together with the current data packet. This can solve the situation where the keyword appears in adjacent data packets. For example, if the keyword to be filtered is 0x123456, the end data of a data packet is 0x123, and the next packet of data of the same session is 0x456, the first packet filtering is not completed, half of the keywords are matched, the data flow is terminated, and the next data packet may be a data packet of another session. Therefore, for the unfinished filtering, it is necessary to record which session's data packet it is, half of the keywords have been matched, the filtering result is incomplete, and the unfinished filtering result is put into the stack. After the 0x456 data packet arrives, the unfinished filtering result is taken out of the stack, and the previous filtering state is restored, so that the keyword 0x123456 can be correctly hit;
[0053] Determine the length of the data packet: Determine whether the length of the data packet is greater than 32 bytes. If it is ≥32 bytes, perform keyword filtering. If it is <32 bytes, cache the current data packet, update the cache space by shifting left according to the size of the current data packet, and then perform keyword filtering.
[0054] During data transmission, it cannot be guaranteed that the size of each data packet can reach more than 32 bytes. Therefore, a 32-byte cache space is established inside the FPGA for each session (such as Figure 4 As shown in the figure, it is used to cache small data packets during data transmission. The small data packet cache adopts the left shift splicing cache technology to ensure seamless connection between data packets.
[0055] Before keyword filtering begins, a 32-byte cache space of all zeros is established for each session. For the received data packet, it is determined whether it is a small data packet. A small data packet is a data packet with a data length of less than 32 bytes. Taking the case of continuously received small data packets, when the first small data packet is received, the current data packet is placed at the end of the cache, and then the data in the 32-byte cache space is keyword filtered. When the second data packet is received as a small data packet, the data in the cache space is shifted left by the length of the current received data packet, which is 8 bytes in this example, and the received 8 bytes are placed at the end of the 256 bytes after the left shift. Then keyword filtering is performed. Although the above process involves repeated filtering, it relies on the high-speed computing capability of the FPGA and can meet the requirements for small packets. It will not affect the overall link speed.
[0056] For example: the keyword to be searched is 0x0102030405060708; however, when the data source sends data, it divides the data packet into eight data packets of 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, and 0x08 and sends them downward. Moreover, the data packet is small and it is more complicated to use the stack. Therefore, the cache repeated filtering technology is used for this type of data packet. When the length of the first data packet 0x01 is less than 32, the data packet is placed in the cache space for filtering, and the filtered cache data is 0x000000000000001; there is no keyword, and the second packet is received and placed in the cache space, and the filtered data is 0x0000000000000102; and so on, when all 0x0102030405060708 are received, the keyword can be hit.
[0057] Keyword filtering: Figure 5 and Figure 6 As shown, according to the keyword information taken out according to the session number, the keyword is filtered in the data packet. There are eight groups of keywords in the present invention. First, the received data packet is copied into 8 copies, one copy for each group of keywords, and distributed to 8 groups of keyword filtering programs. Then, the received data stream is copied into 8 copies, as shown in FIG. Figure 5 As shown in the figure. If each keyword filter program needs to support a keyword length of 32B, keyword filtering needs to be performed according to different data offsets. Figure 6 As shown, the received data stream is replicated in 32 ways for keyword parallel filtering. The filtering start offset address of each way is 0B, 1B, 2B, 3, ..., 30B, 31B, and keyword filtering is performed in parallel according to different address offsets, so that all keyword combinations in the data stream can be traversed to ensure that no keyword is missed.
[0058] Determine whether filtering is completed: In the FPGA process, the FPGA filtering program outside needs to wait for the completion of keyword filtering. If the current data packet filtering is completed, it enters the filtering state and pushes it into the stack. If it is not completed, it needs to wait for keyword filtering until it is completed.
[0059] Filter status stacking: During the data filtering process, there are 32 programs that perform keyword filtering at different address offsets. When the data packet of this session ends, some filtering programs have not completed keyword filtering. For example, if the keyword is 32B long, but the remaining keyword length of some address offsets is less than 32B, the filtered data is incomplete. Therefore, the filtered session number, the filtered position, and the unfinished filtering results are put into the stack, and the filtering status is popped out of the stack after the next data packet of the same session arrives. This ensures that the keyword can also be retrieved and hit in adjacent data packets.
[0060] 4. Keyword cascade judgment: such as Figure 7 As shown, it is determined whether the keywords that have completed filtering have a cascade relationship DATA_MASK. If so, the cascade result processing program is entered. If not, the filtering result reporting program is entered.
[0061] Cascade result processing program: When the set keyword length is greater than 32B, keyword cascading is required to obtain a longer keyword length. The main function of keyword cascading is to implement the cascade result processing program. First, determine whether there is a cascade relationship between keyword 1 and keyword 2. If there is no cascade between keyword 1 and keyword 2, the result of keyword 1 filtering is output to enter the keyword 2 and keyword 3 cascade judgment state. If there is a cascade relationship between keyword 1 and keyword 2, directly enter the keyword 2 and keyword 3 cascade judgment state. In the keyword 2 and keyword 3 cascade judgment state, the cascade state of keyword 2 and keyword 3 is judged. If there is no cascade relationship between keyword 2 and keyword 3, it is necessary to output the cascade relationship between keyword 1 and keyword 2 and the filtering result, and then enter the keyword 3 and keyword 4 cascade judgment state. If there is no cascade relationship, directly enter the keyword 3 and keyword 4 cascade judgment state. And so on, in the current state, it is necessary to judge the cascade relationship with the next keyword. If there is a cascade, enter the next state judgment. If there is no cascade, output the keyword filtering result cascaded with the current state keyword until the relationship between all keyword filtering results and the cascade is determined, and the cascade and filtering results are output.
[0062] 5. Filter result reporting program: Report the results of keyword filtering to the CPU, including the filtering hit results, session number, keyword hit position, etc. If the keywords that have completed filtering have a cascade relationship DATA_MASK, the cascade results are also included.
[0063] 2. The data receiving thread: it is always waiting for whether there is a data packet of keyword filtering result reported by the FPGA end; if there is, the filtering result is sent to the data source end through the external interface.
[0064] In summary, the filtering method proposed in the present invention uses CPU and FPGA to work together, and different processors complete different processing tasks, specializing in specific tasks, and improving the performance of the equipment. Make full use of the transaction processing capability of the CPU and the parallel processing capability of the FPGA. In the FPGA implementation process, the area is used in exchange for speed, and multiple programs are run in parallel to achieve a high-speed filtering rate, realizing multi-session multi-keyword filtering. According to the keyword setting, the location where the keyword of the data packet appears can be accurately filtered out, and the filtering results can be uploaded. In the present invention, the clock used inside the FPGA is 100MHz, the data stream bit width is 256bit, and the theoretical rate is 100MHZ*256bit≈25Gbps, which can adapt to most application scenarios. In the actual measurement process, 64 sessions, 8 groups of keywords, and each group of keywords 0~32B length are used for filtering, which can correctly filter the keyword information, and the filtering rate is 5.6Gbps under 10G network communication.
[0065] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many modifications without departing from the purpose of the present invention, all of which are within the protection of the present invention.
Claims
1. A high-speed keyword filtering method based on the collaborative work of FPGA and CPU, in which the data sending thread and the data receiving thread are two threads, characterized in that: The data sending thread includes the following steps: 1) Keyword setting: Each session is set with 1-8 groups of keywords, each group of keywords is 1-32 bytes long, and the configuration space of each session keyword in the internal cache of FPGA is 256 bytes. Each group of keywords is configured with a corresponding session number, and the internal cache of FPGA supports up to 64 sessions; Receive the keyword configured by the computer through the serial port. The length of the keyword is recorded as L. Determine the size of L. If L<32byte, use KEY_MASK to indicate the effective length of the keyword, generate the keyword KEY and its keyword mask KEY_MASK, and execute step 2); If 32byte<L≤256byte, the keyword space is cascaded to generate the keyword KEY and its keyword mask KEY_MASK, and DATA_MASK is used to indicate the cascade relationship between the keywords, and step 2 is executed; If L>256byte, a message is sent back to the computer indicating that the keyword length is too long; 2) Keyword distribution: The generated keyword session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship represented by DATA_MASK are sent to the internal cache of FPGA through PCIE; At the same time, the CPU determines whether the file format of the data packet of the current session is a plain text data format. If it is a plain text data format, the data packet of the current session is sent to the internal cache of the FPGA through PCIE; if it is not a plain text data format, the data packet of the current session is cached until the data packet of the current session is received, and the file data in the non-text data format is converted into data in the plain text data format and sent to the internal cache of the FPGA through PCIE; 3) Keyword filtering: The FPGA performs keyword filtering on the data packets received from the CPU according to the keyword information corresponding to the session number in the data packets; 4) Determine whether the filtered keywords have a cascade relationship DATA_MASK. If so, enter the cascade result processing. If not, execute step 5); Cascade result processing: first determine whether there is a cascade relationship between keyword 1 and keyword 2. If there is no cascade between keyword 1 and keyword 2, output the result of keyword 1 filtering and enter the keyword 2 and keyword 3 cascade judgment state. If there is a cascade relationship between keyword 1 and keyword 2, directly enter the keyword 2 and keyword 3 cascade judgment state, and determine the cascade state of keyword 2 and keyword 3 in the keyword 2 and keyword 3 cascade judgment state. If there is no cascade relationship between keyword 2 and keyword 3, it is necessary to output the cascade relationship between keyword 1 and keyword 2 and the filtering result and then enter the keyword 3 and keyword 4 cascade judgment state; if there is no cascade relationship, directly enter the keyword 3 and keyword 4 cascade judgment state and so on, until the relationship between all keyword filtering results and cascades is determined, the cascade and filtering results are output, and step 5 is executed; Step 5) Report the keyword filtering result to the CPU, the reported result includes the filtering hit result, session number, keyword hit position, and if the keywords that have completed the filtering have a cascade relationship DATA_MASK, the cascade result is also included; The data receiving thread is always waiting for a data packet with keyword filtering results reported by the FPGA end; if so, the filtering results are sent to the data source end through the external interface.
2. The high-speed keyword filtering method based on the collaborative work of FPGA and CPU according to claim 1 is characterized in that: In step 2), for different sessions, the user configures keyword filtering information according to the session number, and stores it in the FPGA according to the keyword's session number, keyword KEY, keyword mask KEY_MASK and keyword cascade relationship DATA_MASK; When a data packet arrives, the keyword information of the current session is taken out from the FPGA for updating according to the session number of the keyword in the data packet.
3. The high-speed keyword filtering method based on the collaborative work of FPGA and CPU according to claim 1 is characterized in that: The process of step 3) is as follows: 3-1) Update keywords according to session number: according to the session number in the data packet, take out the keyword information of the current session number in the keyword cache; 3-2) Determine the packet sequence number of the received data packet: After extracting the keyword information corresponding to the session number, determine whether the received data packet is the first packet. If it is the first packet, determine the length of the data packet. If it is not the first packet, it is popped out of the stack according to the session number of the data packet. The stack includes the filtered positions and the unfinished filtering results; 3-3) Determine the length of the data packet to determine whether the length of the data packet is greater than 32 bytes. If it is ≥32 bytes, execute step 3-4) to perform keyword filtering. If it is <32 bytes, cache the current data packet, update the cache space according to the size of the current data packet, and execute step 3-4) to perform keyword filtering; 3-4) Keyword filtering: Keywords are filtered in the data packet according to the keyword information retrieved according to the session number. First, the received data packet is copied into 8 copies, one copy for each group of keywords; each copy is copied into 32 paths according to the maximum length of the keyword, and the offset address of each path is 0B, 1B, 2B, 3B, ..., 30B, 31B, and parallel keyword filtering is performed according to different address offsets, thereby traversing all keyword combinations in the data packet; 3-5) Determine whether filtering is completed: If the current data packet filtering is completed, it enters the filtering state and is pushed into the stack. If it is not completed, it needs to wait for keyword filtering until it is completed; 3-6) Push filter status into stack: put the filtered session number, filtered position, and unfinished filtering results into the stack, and wait for the next data packet of the same session to arrive before popping the filter status out of the stack.
4. The high-speed keyword filtering method based on the collaborative work of FPGA and CPU according to claim 3 is characterized in that: In step 3-2), the data packet is popped according to the session number: for the unfinished keyword filtering results, the filtered session number, the filtered position, and the unfinished filtering results are placed in the stack. After the next packet of data of the current session arrives, the information pushed into the stack is popped and the data is filtered together with the current data to solve the situation where the keyword appears in adjacent data packets.
5. The high-speed keyword filtering method based on FPGA and CPU collaboration according to claim 1, characterized in that: The clock used in the FPGA is 100MHz, the data stream bit width is 256bit, and the theoretical rate is 100MHZ*256bit≈25Gbps; 64 sessions and 8 groups of keywords are used, and filtering is performed with each group of keywords having a length of 0 to 32B. The filtering rate is 5.6Gbps under 10G network communication.
