Method and system for resisting network storm by updating white list in real time
By updating the whitelist and FPGA in parallel processing in real time, dynamically classifying and processing network storm messages is solved, and the problem of excessive CPU burden and inability to adapt to variable environments is achieved, and efficient and flexible network storm resistance is achieved.
Patent Information
- Application Number
- CN202411276134.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-31
AI Technical Summary
When the existing network storm resistance algorithm processes a large number of traffic packets, the CPU is overloaded, resulting in poor real-time performance, and the FPGA-based solution cannot dynamically adapt to variable environments.
By updating the whitelist in real time, using FPGA to process the packet feature values in parallel, generating feature value statistics tables and change models, dynamically classifying storm messages, reducing CPU burden, and realizing filtering of storm messages and retaining subscription messages.
It realizes rapid and effective resistance to network storms in a dynamic environment, reduces CPU burden, improves processing efficiency and flexibility, and ensures that the delivery of subscription packets is not affected.
Smart Images
Figure CN119232661A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electrical communication technology, and in particular to a method and system for resisting network storms by updating a white list in real time. Background Art
[0002] Most traditional algorithms for resisting network storms are based on the CPU receiving and processing messages, which have poor real-time performance. Their ability to resist storms depends on the CPU's processing power. A large number of traffic messages will greatly increase the CPU's burden, causing the CPU to freeze or even crash. Later, a solution based on FPGA to resist network storms appeared. Usually, FPGA is used to parse and obtain the characteristics of the message. The CPU sends a filtering whitelist based on the known characteristics, and the FPGA then filters according to the whitelist. This solution is suitable for situations where the filtering characteristics are known and cannot dynamically adapt to variable environments. Therefore, it is urgent to design a method to resist network storms by updating the whitelist in real time. Summary of the invention
[0003] The object of the present invention is to provide a method and system for updating a whitelist in a dynamic message environment and classifying traffic messages to realize real-time updating of the whitelist and resisting network storms.
[0004] The technical solution adopted by the present invention is: a method for resisting network storms by updating the whitelist in real time, and the specific steps are: S1. Perform traffic statistics on input messages, and determine whether the message is in the subscription message stage or the storm message stage according to the preset traffic range, where the messages in the storm message stage include subscription messages and storm messages; S2. Classify the messages in the storm message stage according to the generated whitelist, filter out the storm messages, and retain the subscription messages; The generation process of the whitelist is as follows: FPGA parallel processing is used to parse the input message, obtain its characteristic value and perform statistics, and establish a characteristic value statistical table; according to the characteristic value statistical table, the characteristic value change model is obtained, and the characteristic value and its change model are written into the whitelist.
[0005] Furthermore, the method for traffic statistics is specifically as follows: counting the number of message bytes received within a unit time, wherein the size of the unit time depends on the performance of the CPU.
[0006] Furthermore, the step of establishing a characteristic value statistical table is specifically as follows: establishing RAM tables for different types of messages respectively, dividing each RAM table into different intervals according to the message content, and counting the characteristic values and the number of characteristic values in each interval.
[0007] Furthermore, the characteristic values of the input message are divided into single characteristic values and variable characteristic values, wherein a single characteristic value means that there is only one characteristic value in one interval, and a variable characteristic value means that there are multiple characteristic values in one interval.
[0008] Furthermore, the analysis of the change model of the eigenvalue is specifically to input the current eigenvalue into a function designed in advance according to a known change model, and compare the calculation result with the actual next eigenvalue to determine whether to update the change model.
[0009] Furthermore, the whitelist is obtained specifically through the following steps: writing the whitelist RAM1 at any time into the next level whitelist RAM2 after a unit time, comparing the CRC fields of RAM1 and RAM2, and if they are the same, taking RAM2 as the final whitelist.
[0010] Furthermore, the messages in the storm message stage include duplicate messages and non-duplicate messages, wherein the non-duplicate messages include messages whose characteristic values are not in the whitelist, messages whose characteristic values are in the whitelist but have abnormal contents, and messages whose characteristic values are in the whitelist and have normal contents.
[0011] Furthermore, duplicate messages are filtered by limiting the flow; among non-duplicate messages, messages whose characteristic values are not in the whitelist and messages whose characteristic values are in the whitelist but have abnormal content are filtered by discarding them, and messages whose characteristic values are in the whitelist and have normal content are filtered by transparent transmission, limiting the flow or dropping packets. The choice of which method depends on the agreement between the receiver and the sender.
[0012] Furthermore, the current limiting method is specifically to upload duplicate messages within a preset number and discard duplicate messages exceeding the preset number.
[0013] The present invention provides a system for resisting network storms by updating a whitelist in real time, and specifically comprises: a flow statistics module, which is used for performing flow statistics on input messages, and judging whether the message is in a subscription message stage or a storm message stage according to a preset flow range, wherein the message in the storm message stage comprises a subscription message and a storm message; a characteristic value statistics module, which is used for adopting FPGA parallel processing to parse the input message, obtain its characteristic value and perform statistics, and establish a characteristic value statistics table; a whitelist construction module, which is used for obtaining a change model of the characteristic value according to the characteristic value statistics table, writing the characteristic value and its change model into a whitelist, and classifying the messages in the storm message stage according to the generated whitelist; and a message processing module, which is used for filtering out the storm message according to the whitelist and retaining the subscription message.
[0014] The beneficial effects of the present invention are as follows: the present invention determines the stage of the real-time received message according to the flow rate, and uses FPGA to parse the real-time received message in parallel, obtains its characteristic value and performs statistics, analyzes the change model of the obtained characteristic value to establish a white list, and classifies the messages in the storm message stage according to the white list, and then adopts different methods to complete the filtering of the storm message and retain the subscription message. The present invention makes full use of the parallel operation characteristics of FPGA and integrates the "pipeline" processing concept, so that the speed of parsing characteristic values is faster and more efficient; and the present invention does not affect the transmission of subscription messages while filtering storm messages and resisting network storms.
[0015] Furthermore, the generation process of the whitelist of the present invention integrates AI algorithms, does not require a large number of samples, self-learns messages and trains to automatically obtain key field characteristics, and makes real-time adjustments to establish a stable whitelist.
[0016] Furthermore, the present invention can accurately find the boundary between storm messages and subscription messages through the whitelist. The process of real-time updating of the whitelist does not require CPU participation in parsing, which greatly reduces the burden on the CPU. The message processing strategy is adaptive and can flexibly and reliably adapt to dynamic environments and accurately resist network storms.
[0017] Furthermore, in the process of analyzing the message characteristic values, the prediction method can be used to efficiently and accurately analyze the change model of the characteristic values.
[0018] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1A This is a flow chart of a method for resisting network storms by updating a whitelist in real time according to an embodiment of the present invention; Figure 1B This is a flow chart of a method for resisting network storms by updating a whitelist in real time according to another embodiment of the present invention; Figure 2 This is a structural design diagram of a system for resisting network storms by updating a whitelist in real time according to the present invention; Figure 3 A storage structure diagram of a characteristic value statistical table in one embodiment of the present invention; Figure 4A storage structure diagram of a whitelist in one embodiment of the present invention; Figure 5 A schematic diagram of the process of constructing a stable whitelist for the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0022] It should be noted that the illustrations provided in the embodiments of the present invention are only used to illustrate the basic concept of the present invention in a schematic manner. Therefore, the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout type may also be more complicated.
[0023] In the present invention, it is also necessary to explain that, if the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, if the terms "first" and "second" appear, they are only used for description and distinction purposes, and cannot be understood as indicating or implying relative importance.
[0024] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the common meanings understood by persons having ordinary skills in the field to which the present disclosure belongs.
[0025] In order to facilitate the understanding of this application document, the following keywords need to be explained as follows: FPGA: The full name is Field-Programmable Gate Array, which is field programmable gate array; RAM: The full name is Random-Access Memory, which is random access memory; OFFSET: A function used to return a new cell or range of cells given a starting cell, row number, and column number; CRC field: The full name is Cyclic Redundancy Check, which is a data transmission error detection technology commonly used in the data communication field.
[0026] The method of the present invention realizes the protection against network storms by updating the whitelist in real time, parsing the input messages and establishing the whitelist, and processing in different ways according to the flow and type of the messages, thereby filtering the storm messages and retaining the subscribed messages, without the need for CPU participation and a large number of known samples, and can realize real-time protection against network storms.
[0027] Example 1 like Figure 1A As shown, the embodiment of the present invention implements a method for resisting network storms by updating the whitelist in real time, and the specific steps are: S1. Perform traffic statistics on input messages, and determine whether the message is in the subscription message stage or the storm message stage according to the preset traffic range, where the messages in the storm message stage include subscription messages and storm messages; S2. Classify the messages in the storm message stage according to the generated whitelist, filter out the storm messages, and retain the subscription messages; The generation process of the whitelist is as follows: FPGA parallel processing is used to parse the input message, obtain its characteristic value and perform statistics, and establish a characteristic value statistical table; according to the characteristic value statistical table, the characteristic value change model is obtained, and the characteristic value and its change model are written into the whitelist.
[0028] It can be seen that this embodiment determines the stage of the real-time received message according to the traffic, and uses FPGA to parse the real-time received message in parallel, builds a whitelist, and classifies the messages in the storm message stage according to the whitelist, filters out the storm messages and retains the subscription messages. This method does not require CPU to participate in the analysis, has a fast learning speed, high real-time performance, and a flexible and reliable processing strategy. It can be applied to fields that require fast and reliable resistance to various network storms.
[0029] Example 2 This embodiment provides a method for resisting network storms by updating the whitelist in real time. Figure 1B As shown, including: Input messages and count the traffic size. According to the preset traffic range, the functions of the input messages are divided into a subscription message stage and a storm message stage. The messages in the storm message stage include subscription messages and storm messages.
[0030] Specifically, the number of bytes of the received message per unit time of the input message is counted. The unit time can be adjusted according to the CPU performance: the stronger the CPU performance, the larger the unit time needs to be set; the weaker the CPU performance, the smaller the unit time needs to be set. The statistical unit time should be designed reasonably to avoid the lag in the start of filtering due to the setting of too long statistical unit time, and a large number of traffic messages are uploaded to the CPU before filtering is completed, causing the CPU to freeze or abnormal.
[0031] While counting traffic, FPGA parallel processing is used to parse input messages, obtain their characteristic values and perform statistics to establish a characteristic value statistics table.
[0032] Specifically, the characteristic value of the message is parsed according to common network message protocols such as ARP, ICMP, UDP, TCP, PTP, GOOSE protocol, etc.
[0033] Specifically, the characteristic values include: message type, message length, destination MAC address, source MAC address, destination IP address, IP source address, APPID, source port number, and destination port number.
[0034] Specifically, the characteristic value statistics table is a RAM table. Each table is divided into different intervals according to the message content. Each interval covers all fields of the message. The storage structure is as follows: Figure 3 As shown, interval 1 of the RAM table, address 0 to address N-1, stores N data with characteristic value A and its count, and interval 2, address N to address 2*N-1, stores N data with characteristic value B and its count, until all characteristic value statistics of this type of message are completed.
[0035] The change model of the obtained characteristic value is analyzed, the characteristic value and its change model are written into the white list, and the classification of the messages in the storm message stage is completed according to the white list.
[0036] Specifically, if only one eigenvalue in the same interval has a count and the other counts are 0, it means that the eigenvalue of the interval is unique; if there are multiple eigenvalue counts in an interval that are not 0, it means that the value is variable.
[0037] Specifically, common change models of variable eigenvalues include data increasing, data decreasing, data multiple relationship, etc.
[0038] Specifically, the method for analyzing the change model can be prediction, that is, based on the parallel processing capability of FPGA, the current eigenvalue is input into each function designed by the conventional change model, and the predicted value of the next eigenvalue in each function is obtained. These predicted values are compared with the actual next eigenvalue, and the predicted value consistent with the actual next eigenvalue is selected, and the change model to which this predicted value belongs is used as the change model of the eigenvalue. This embodiment integrates the AI algorithm, does not require a large number of samples, self-learns messages and trains to automatically obtain key field characteristics, and adjusts in real time to establish a stable whitelist.
[0039] Specifically, the whitelist RAM1 is written according to the characteristic value and its change model, and the storage structure is as follows: Figure 4As shown, the same field may have one or more eigenvalues, or the eigenvalue may not be a valid eigenvalue. VLD in the whitelist table can indicate whether the current eigenvalue is valid. Model indicates the change model, which can be numbered in sequence according to common rules. The next eigenvalue OFFSET indicates the position of the next eigenvalue in the table, and can also indicate the number of current eigenvalues.
[0040] Specifically, the whitelist RAM1 is written into the whitelist RAM2 after a unit time. In this embodiment, the unit time is 1s. Compare the CRC fields of the two-level RAM whitelists. If the CRC fields remain unchanged, the latter-level RAM whitelist is used as a stable whitelist. If the CRC fields of the two-level RAM tables are different, it means that the learning and training stage is still in progress. Continue to build the whitelist. The process of implementing this step is as follows: Figure 5 shown.
[0041] Specifically, the messages in the storm message stage include duplicate messages and non-duplicate messages, wherein the non-duplicate messages include messages whose characteristic values are not in the whitelist, messages whose characteristic values are in the whitelist but have abnormal contents, and messages whose characteristic values are in the whitelist and have normal contents.
[0042] Filter storm messages and retain subscription messages.
[0043] Specifically, the subscription messages include messages in the subscription message stage and subscription messages in the storm message stage.
[0044] Specifically, if the input message is in the subscription message stage, all the input messages are uploaded to the CPU without processing; if the input message belongs to the storm message stage, different methods are used to filter the storm message, and then the remaining subscription messages are uploaded to the CPU.
[0045] Specifically, duplicate messages are filtered by limiting the flow, which specifically allows a certain number of identical messages to be uploaded, and messages exceeding this number are discarded; among non-duplicate messages, messages whose characteristic values are not in the whitelist and messages whose characteristic values are in the whitelist but have abnormal content are filtered by discarding, and messages whose characteristic values are in the whitelist and have normal content are filtered according to the method agreed upon by the receiver and the sender.
[0046] Specifically, after the storm message filtering is completed, statistical processing information is fed back, and the message before filtering and the message after filtering are compared to determine whether the processing result is within the expected range.
[0047] Example 3 This embodiment provides a system for resisting network storms by updating the whitelist in real time. Figure 2 As shown, it includes a traffic statistics module, a feature value statistics module, a whitelist construction module and a message processing module.
[0048] The traffic statistics module receives and counts the number of bytes of input messages per unit time, and divides the messages into the subscription message stage and the storm message stage.
[0049] The characteristic value statistics module uses FPAG to parse the input message, obtain its characteristic value, perform statistics, and establish a characteristic value statistics table.
[0050] Specifically, the characteristic value of the message is parsed according to common network message protocols such as ARP, ICMP, UDP, TCP, PTP, GOOSE protocol, etc.
[0051] Specifically, the characteristic value statistics table is a RAM table. Each table is divided into different intervals according to the message content. Each interval covers all fields of the message. The storage structure is as follows: Figure 3 As shown, interval 1 address 0 to address N-1 of the RAM table stores N data with characteristic value A and count of their occurrences, interval 2 address N to address 2*N-1 stores N data with characteristic value B and count of their occurrences, and so on, until all characteristic value statistics of this type of message are completed.
[0052] The whitelist building module is used to analyze the change model of the characteristic value, establish a whitelist to store the characteristic value and its change model, and complete the type classification of the message in the storm message stage according to the whitelist.
[0053] Specifically, if only one eigenvalue in the same interval has a count and the other counts are 0, it means that the eigenvalue of the interval is unique; if there are multiple eigenvalue counts in an interval that are not 0, it means that the eigenvalue is variable.
[0054] Specifically, the changing characteristics of the variable eigenvalues include data increasing, data decreasing, data multiple relationship, etc.
[0055] Specifically, the method for analyzing the change model can be prediction. Based on the parallel processing capability of FPGA, the current eigenvalue is input into each function designed by the conventional change model respectively, and the predicted value of the next eigenvalue in each function is obtained. These predicted values are compared with the actual next eigenvalue, and the predicted value consistent with the actual next eigenvalue is selected, and the change model to which this predicted value belongs is used as the change model of the eigenvalue.
[0056] Specifically, the characteristic value and its change model are written into the whitelist RAM1, and the storage structure is as follows: Figure 4 As shown, the same field may have one or more eigenvalues, or the eigenvalue may not be a valid eigenvalue. VLD in the whitelist table can indicate whether the current eigenvalue is valid. The model represents the change rule and can be numbered in sequence according to common rules. The next eigenvalue OFFSET indicates the position of the next eigenvalue in the table, and can also indicate the number of current eigenvalues.
[0057] Specifically, the whitelist RAM1 is written into the whitelist RAM2 after a unit time. In this embodiment, the unit time is 1s. Compare the CRC fields of the two-level RAM whitelists. If the CRC fields remain unchanged, the latter-level RAM whitelist is used as a stable whitelist. If the CRC fields in the two-level RAM tables are different, it means that it is still in the learning and training stage, and continue to build the whitelist. The implementation process is as follows: Figure 5 shown.
[0058] Specifically, the messages in the storm message stage are compared with the whitelist, and the messages are divided into duplicate messages and non-duplicate messages, where non-duplicate messages include messages whose characteristic values are not in the whitelist, messages whose characteristic values are in the whitelist but have abnormal content, and messages whose characteristic values are in the whitelist and have normal content.
[0059] Specifically, the whitelist construction module receives the message processing information returned by the message processing module, compares the messages before and after filtering, and determines whether the processing result is within the expected range. In this embodiment, the number of filtered messages is compared to see if it is consistent with the number of subscribed messages. If not, the whitelist is further adjusted.
[0060] The message processing module is used to filter storm messages and retain subscribed messages.
[0061] Specifically, duplicate messages are filtered by limiting the flow, which allows a certain number of identical messages to be uploaded, and messages exceeding this number are discarded; among non-duplicate messages, messages whose characteristic values are not in the whitelist and messages whose characteristic values are in the whitelist but have abnormal content are filtered by discarding, and messages whose characteristic values are in the whitelist and have normal content are either transmitted through, limited, or dropped. The choice of which method depends on the agreement between the receiver and the sender.
[0062] The various modules of the system are mainly used to implement the various steps of the above method embodiment, which will not be described one by one here. Elaborate.
[0063] In summary, the present invention provides a method for resisting network storms by updating the whitelist in real time. First, the flow of input messages is counted to determine the stage of the message. Then, the characteristic values of the messages are analyzed in parallel using FPGA, and the change model of the characteristic values is analyzed to obtain the whitelist. According to the whitelist, the messages in the storm message stage are classified, and finally the storm messages are filtered out and the subscribed messages are retained. The CPU does not need to be involved in the analysis throughout the process, which greatly reduces the burden on the CPU, and achieves flexible and reliable adaptation to the dynamic message environment and accurate resistance to network storms.
[0064] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.
[0065] The order of execution of each step in the above embodiment does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
[0066] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A method for resisting network storms by updating a whitelist in real time, characterized in that: The specific steps are: S1. Perform traffic statistics on input messages, and determine whether the message is in the subscription message stage or the storm message stage according to the preset traffic range, where the messages in the storm message stage include subscription messages and storm messages; S2. Classify the messages in the storm message stage according to the generated whitelist, filter out the storm messages, and retain the subscription messages; The generation process of the whitelist is as follows: FPGA parallel processing is used to parse the input message, obtain its characteristic value and perform statistics, and establish a characteristic value statistical table; according to the characteristic value statistical table, the characteristic value change model is obtained, and the characteristic value and its change model are written into the whitelist.
2. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The method for traffic statistics is specifically as follows: counting the number of message bytes received within a unit time, wherein the size of the unit time depends on the performance of the CPU.
3. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The step of establishing the characteristic value statistical table is specifically as follows: establishing RAM tables for different messages respectively, dividing each RAM table into different intervals according to the message content, and counting the characteristic values and the number of characteristic values in each interval.
4. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The characteristic values of the input message are divided into single characteristic values and variable characteristic values, wherein a single characteristic value has only one characteristic value in one interval, and a variable characteristic value has multiple characteristic values in one interval.
5. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The analysis of the change model of the characteristic value specifically includes inputting the current characteristic value into a function designed in advance according to a known change model, and comparing the calculation result with the actual next characteristic value to determine whether to update the change model.
6. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The whitelist is obtained specifically by the following steps: writing the whitelist RAM1 at any time into the next level whitelist RAM2 after a unit time, comparing the CRC fields of RAM1 and RAM2, and if they are the same, taking RAM2 as the final whitelist.
7. The method for resisting network storms by updating the whitelist in real time according to claim 1, characterized in that: The messages in the storm message stage include duplicate messages and non-duplicate messages, wherein the non-duplicate messages include messages whose characteristic values are not in the whitelist, messages whose characteristic values are in the whitelist but whose contents are abnormal, and messages whose characteristic values are in the whitelist and whose contents are normal.
8. The method for resisting network storms by updating the whitelist in real time according to claim 7, characterized in that: Duplicate messages are filtered by limiting the flow. Among non-duplicate messages, messages whose characteristic values are not in the whitelist and messages whose characteristic values are in the whitelist but have abnormal contents are filtered by discarding them. Messages whose characteristic values are in the whitelist and have normal contents are filtered by transparent transmission, limiting the flow or dropping packets. The selection of the method depends on the agreement between the receiver and the sender.
9. The method for resisting network storms by updating the whitelist in real time according to claim 8, characterized in that: The current limiting method is specifically to upload duplicate messages within a preset number and discard duplicate messages exceeding the preset number.
10. A system for resisting network storms by updating a whitelist in real time, characterized in that: include: A traffic statistics module is used to perform traffic statistics on input messages and determine whether the message is in the subscription message stage or the storm message stage according to a preset traffic range, wherein the messages in the storm message stage include subscription messages and storm messages; The characteristic value statistics module is used to parse the input message by FPGA parallel processing, obtain its characteristic value and perform statistics, and establish a characteristic value statistics table; A whitelist construction module is used to obtain a change model of the characteristic value according to the characteristic value statistical table analysis, write the characteristic value and its change model into the whitelist, and classify the messages in the storm message stage according to the generated whitelist; The message processing module is used to filter out storm messages according to the whitelist and retain the subscription messages.