A log processing method and device

Abstract

The embodiment of the present application provides a log processing method and device, which relates to the field of communication technology. The method is applied to a service board, and the method includes: for each system log of a target service type generated, judging whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type; if the generation rate is greater than the sending rate threshold, sending the system log of the target service type to the main board according to the sending rate threshold; if the generation rate is not greater than the sending rate threshold, sending the system log of the target service type to the main board according to the generation rate. In this way, the situation that the main board is abnormal due to forwarding the system log can be avoided to a certain extent.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a log processing method and device. Background Art

[0002] With the rapid development of communication technology, current network security equipment can be distributed equipment including business boards and main boards. The business board is responsible for processing network security services and will generate corresponding system logs. The business board can send the generated system logs to the main board, and the main board will upload the system logs to the log management device (e.g., log server) used to manage the system logs.

[0003] However, if the service board generates a large number of system logs, the main board will consume more system resources when forwarding the system logs sent by the service board, resulting in a decrease in the performance of the main board or even an abnormality of the main board. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a log processing method and device, which can avoid the situation where the mainboard is abnormal due to forwarding system logs to a certain extent. The specific technical solution is as follows:

[0005] In a first aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application discloses a log processing method, which is applied to a service board and includes:

[0006] For each generated system log of the target service type, determining whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type;

[0007] If the generation rate is greater than the sending rate threshold, sending the system log of the target service type to the mainboard according to the sending rate threshold;

[0008] If the generation rate is not greater than the sending rate threshold, the system log of the target service type is sent to the mainboard according to the generation rate.

[0009] Optionally, before determining, for each generated system log of the target service type, whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type, the method further includes:

[0010] According to the maximum sending rate of the service board, a sending rate threshold corresponding to the system log of each target service type generated by the service board is determined.

[0011] Optionally, determining, according to the maximum sending rate of the service board, a sending rate threshold corresponding to the system log of each target service type generated by the service board includes:

[0012] According to the maximum sending rate of the service board and the weight of the system log of each target service type generated by the service board, a sending rate threshold corresponding to the system log of each target service type is calculated.

[0013] Optionally, the method further includes:

[0014] If the generation rate is greater than the sending rate threshold, discarding the system logs of the target service type that are not sent to the mainboard;

[0015] The number of discarded system logs of the target service type is sent to the log management device.

[0016] Optionally, before determining, for each generated system log of the target service type, whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type, the method further includes:

[0017] Determine whether the log rate limit function is currently enabled;

[0018] If the log rate limit function is currently enabled, the step of determining whether the generation rate of the system log of each target business type is greater than the sending rate threshold corresponding to the system log of the target business type is executed for each system log generated for the target business type;

[0019] If the log rate limit function is not currently enabled, for each generated system log of the target service type, the system log of the target service type is sent to the mainboard according to the generation rate of the system log of the target service type.

[0020] Optionally, before determining, for each generated system log of the target service type, whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type, the method further includes:

[0021] Determine whether the total generation rate of the system logs of each target business type is greater than a preset generation rate threshold;

[0022] If the total generation rate is greater than the preset generation rate threshold, executing the step of determining, for each generated system log of the target business type, whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type;

[0023] If the total generation rate is not greater than the preset generation rate threshold, then for each generated system log of the target business type, the system log of the target business type is sent to the mainboard according to the generation rate of the system log of the target business type.

[0024] In a second aspect, in order to achieve the above-mentioned purpose, an embodiment of the present application discloses a log processing device, which is applied to a service board, and the device includes:

[0025] A first judgment module is used to judge, for each generated system log of a target business type, whether a generation rate of the system log of the target business type is greater than a sending rate threshold corresponding to the system log of the target business type;

[0026] A first sending module, configured to send the system log of the target service type to the mainboard according to the sending rate threshold if the generation rate is greater than the sending rate threshold;

[0027] The second sending module is used to send the system log of the target business type to the mainboard according to the generation rate if the generation rate is not greater than the sending rate threshold.

[0028] Optionally, the device further comprises:

[0029] A sending rate threshold determination module is used to determine the sending rate threshold corresponding to the system log of each target business type generated by the business board according to the maximum sending rate of the business board before judging whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type.

[0030] Optionally, the sending rate threshold determination module is specifically used to calculate the sending rate threshold corresponding to the system log of each target business type based on the maximum sending rate of the business board and the weight of the system log of each target business type generated by the business board.

[0031] Optionally, the device further comprises:

[0032] A discarding module, configured to discard the system logs of the target service type that are not sent to the main board if the generation rate is greater than the sending rate threshold;

[0033] The third sending module is used to send the number of discarded system logs of the target service type to the log management device.

[0034] Optionally, the device further comprises:

[0035] A second judgment module is used to judge whether the log speed limit function is currently enabled before judging whether the generation rate of the system log of each target business type generated is greater than the sending rate threshold corresponding to the system log of the target business type; if so, trigger the first judgment module; if not, trigger the fourth sending module;

[0036] The fourth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

[0037] Optionally, the device further comprises:

[0038] a third judgment module, configured to judge whether the total generation rate of the system logs of each target business type generated is greater than a preset generation rate threshold before judging whether the generation rate of the system logs of the target business type is greater than the sending rate threshold corresponding to the system logs of the target business type for each target business type generated; if so, trigger the first judgment module, and if not, trigger the fifth sending module;

[0039] The fifth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

[0040] In another aspect of the implementation of the present application, in order to achieve the above-mentioned purpose, the embodiment of the present application further discloses a business board, the business board includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0041] The memory is used to store computer programs;

[0042] The processor is used to implement the log processing method described in the first aspect when executing the program stored in the memory.

[0043] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the log processing method described in the first aspect is implemented.

[0044] In another aspect of the implementation of the present application, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the log processing method described in the first aspect above.

[0045] Beneficial effects of the embodiments of the present application:

[0046] The log processing method provided in the embodiment of the present application determines, for each system log generated for a target business type, whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type; if the generation rate is greater than the sending rate threshold, the system log of the target business type is sent to the main board according to the sending rate threshold; if the generation rate is not greater than the sending rate threshold, the system log of the target business type is sent to the main board according to the generation rate.

[0047] Based on the above processing, if the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system log of the target business type can be sent to the main board according to the sending rate threshold. The system log sent to the main board can be controlled within a certain number to avoid the main board receiving too many system logs, so as to reduce the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, it can avoid the situation where the main board abnormalities are caused by forwarding the system logs to a certain extent.

[0048] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0050] Figure 1 A flow chart of a log processing method provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of another log processing method provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of another log processing method provided in an embodiment of the present application;

[0053] Figure 4 A flowchart of another log processing method provided in an embodiment of the present application;

[0054] Figure 5 A flowchart of another log processing method provided in an embodiment of the present application;

[0055] Figure 6 A flowchart of another log processing method provided in an embodiment of the present application;

[0056] Figure 7 A flowchart of another log processing method provided in an embodiment of the present application;

[0057] Figure 8 A structural diagram of a log processing device provided in an embodiment of the present application;

[0058] Fig. 9 A structural diagram of a service board provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0060] In the prior art, if the service board generates a large number of system logs, the main board will consume a large number of system resources when forwarding the system logs sent by the service board, resulting in a decrease in the performance of the main board and even an abnormality of the main board.

[0061] In order to solve the above technical problems, the present application embodiment provides a log processing method, which can be applied to the business board, see Figure 1 , Figure 1 A flow chart of a log processing method provided in an embodiment of the present application, the method may include the following steps:

[0062] S101: for each generated system log of the target service type, determining whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type.

[0063] S102: If the generation rate is greater than the sending rate threshold, the system log of the target service type is sent to the mainboard according to the sending rate threshold.

[0064] S103: If the generation rate is not greater than the sending rate threshold, the system log of the target service type is sent to the mainboard according to the generation rate.

[0065] The log processing method provided in the embodiment of the present application, if the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system logs of the target business type can be sent to the main board according to the sending rate threshold, and the system logs sent to the main board can be controlled within a certain number to avoid the main board receiving too many system logs, so as to reduce the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, it can avoid the situation where the main board is abnormal due to forwarding the system logs to a certain extent.

[0066] With respect to step S101 , the service board may be responsible for processing network security services of multiple service types. When processing the network security services of each service type, the service board may generate a system log corresponding to the network security services of the service type.

[0067] Exemplarily, the service type of each network security service may include: threat type, audit statistics type, policy type, ATK (attack) type, and FW-OTHER (other) type, etc.

[0068] The threat-type network security services may include at least one of IPS (Intrusion Prevention System), AV (Anti-Virus), and WAF (Web Application Firewall).

[0069] The network security services of the audit statistics type may include at least one of AUDIT (application audit), UFLT (URL filtering), DFILTER (data filtering), and FFILTER (file filtering).

[0070] The policy-type network security service may include at least one of AVC (bandwidth policy), APPPROXY (application proxy), and NETSHARE (network sharing).

[0071] The ATK type network security service may include at least one of flooding and blacklist network security services.

[0072] The FW-OTHER type network security services may include: at least one network security service of ASPF (Application Specific Packet Filter, application layer message filtering), NAT (Network Address Translation, network address translation), and Session (session control).

[0073] The target service type can be a service type that matches the current traffic data among the service types. For example, the network security services corresponding to the current traffic data include: IPS, WAF, NETSHARE, where the service types of IPS and WAF are threat types, and the service type of NETSHARE is policy type. Accordingly, the target service types include: threat type and policy type.

[0074] For step S102, for each target business type, if the generation rate of the system log of the target business type is greater than the sending rate threshold, it indicates that the business board is currently generating the system log of the target business type at a faster rate. If the business board sends the system log of the target business type to the main board according to the generation rate of the system log of the target business type, the main board will receive more system logs.

[0075] Therefore, the business board can send the system log of the target business type to the main board according to the sending rate threshold, and can control the system logs sent to the main board within a certain number to prevent the main board from receiving too many system logs, so as to reduce the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, it can avoid the situation where the main board abnormalities are caused by forwarding the system logs to a certain extent.

[0076] For step S103, for each target business type, if the generation rate of the system log of the target business type is not greater than the sending rate threshold, it indicates that the business board currently generates the system log of the target business type at a slow rate, and if the system log of the target business type is sent to the main board at the generation rate of the system log of the target business type, the main board will not receive more system logs. Therefore, the business board can send the system log of the target business type to the main board at the generation rate of the system log of the target business type.

[0077] In one embodiment of the present application, Figure 1 Based on Figure 2 Before step S101, the method may further include the following steps:

[0078] S104: Determine a sending rate threshold corresponding to the system log of each target service type generated by the service board according to the maximum sending rate of the service board.

[0079] Among them, the maximum sending rate of the business board can be determined according to the performance of the main board. That is to say, if the business board sends system logs to the main board at a sending rate not higher than the maximum sending rate, it will not cause the main board to receive too many system logs; if the business board sends system logs to the main board at a sending rate higher than the maximum sending rate, it may cause the main board to receive too many system logs.

[0080] In one implementation, the service board may determine the number of each target service type, and then calculate the ratio of the maximum sending rate of the service board to the number of each target service type, and then use the obtained ratio as the sending rate threshold corresponding to the system log of each target service type.

[0081] Based on the above processing, corresponding system logs of multiple target business types can be sent to the main board, avoiding the situation where only one business type's system log is sent to the main board due to log speed limit, so that the main board can obtain the situation of the business board processing traffic data.

[0082] In another implementation, see Figure 3 ,exist Figure 2 On the basis of, the above step S104 may include the following steps:

[0083] S1041: Calculate a sending rate threshold corresponding to the system log of each target service type according to the maximum sending rate of the service board and the weight of the system log of each target service type generated by the service board.

[0084] In the embodiment of the present application, different service types may correspond to different priorities. Accordingly, a larger weight may be assigned to a service type with a higher priority, and a smaller weight may be assigned to a service type with a lower priority. For example, the priority may be determined by a technician based on the importance of the network security service.

[0085] The service board can determine the corresponding transmission rate threshold of each target service type based on the maximum transmission rate and the weight of each target service type. The target service type with a larger weight has a larger corresponding transmission rate threshold, and the sum of the transmission rate thresholds is the maximum transmission rate.

[0086] In one embodiment of the present application, Figure 1 Based on Figure 4 , the method may further include the following steps:

[0087] S105: If the generation rate is greater than the sending rate threshold, the system logs of the target service type that are not sent to the mainboard are discarded.

[0088] S106: Send the number of discarded system logs of the target service type to the log management device.

[0089] For each target service type, when the generation rate of the system log of the target service type is greater than the sending rate threshold, the service board sends the system log of the target service type to the main board according to the sending rate threshold. Since the sending rate threshold is less than the generation rate of the system log of the target service type, the service board only sends a part of the system log of the target service type to the main board.

[0090] Accordingly, the service board can also discard the system logs of the target service type that are not sent to the main board. Furthermore, in order to enable the log management device to obtain the situation of the service board processing traffic data, the service board can send the number of system logs of the target service type that are discarded to the log management device.

[0091] In one implementation, the service board can send the number of discarded system logs of the target service type to the log management device in a preset format (eg, service type_service module name_number of discarded logs). The service module name indicates the network security service to which the discarded system log belongs.

[0092] In one embodiment of the present application, Figure 1 Based on Figure 5 Before step S101, the method may further include the following steps:

[0093] S107: Determine whether the log rate limit function is currently enabled. If yes, execute step S101; if not, execute step S108.

[0094] S108: For each generated system log of the target service type, the system log of the target service type is sent to the mainboard according to the generation rate of the system log of the target service type.

[0095] In one implementation, the business board can turn on / off the log speed limit function. For example, the log speed limit function can be turned on during a period of high traffic. The specific way to turn on the log speed limit function is to turn on the log speed limit function when receiving a turn-on instruction from a technician. Alternatively, the business board can automatically turn on the log speed limit function when the corresponding time period is reached.

[0096] If the log rate limit function is currently enabled, the business board can determine whether the generation rate of the system log of each target business type is greater than the sending rate threshold corresponding to the system log of the target business type, and send the system log of the target business type to the main board based on the judgment result.

[0097] If the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system logs of the target business type can be sent to the main board according to the sending rate threshold. The system logs sent to the main board can be controlled within a certain number to prevent the main board from receiving too many system logs, thereby reducing the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, the situation where the main board is abnormal due to forwarding the system logs can be avoided to a certain extent.

[0098] If the log rate limit function is not currently enabled, the business board does not need to determine whether the generation rate of the system log of each target business type is greater than the sending rate threshold corresponding to the system log of the target business type. In other words, for each system log of the target business type generated, the business board can directly send the system log of the target business type to the main board according to the generation rate of the system log of the target business type.

[0099] In one embodiment of the present application, Figure 1 Based on Figure 6 Before step S101, the method may further include the following steps:

[0100] S109: Determine whether the total generation rate of the system logs of each target business type is greater than a preset generation rate threshold, if yes, execute step S101, if no, execute step S1010.

[0101] S1010: For each generated system log of the target service type, the system log of the target service type is sent to the mainboard according to the generation rate of the system log of the target service type.

[0102] The preset generation rate threshold may be the maximum sending rate of the service board.

[0103] In one implementation, the business board can determine whether the total generation rate of the system logs generated for each target business type is greater than the preset generation rate threshold. If the total generation rate of the system logs generated for each target business type is greater than the preset generation rate threshold, it indicates that the business board currently generates a large number of system logs of each target business type. The business board can further determine whether the generation rate of the system log of each target business type is greater than the sending rate threshold corresponding to the system log of the target business type, and send the system log of the target business type to the main board based on the determination result.

[0104] If the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system logs of the target business type can be sent to the main board according to the sending rate threshold. The system logs sent to the main board can be controlled within a certain number to prevent the main board from receiving too many system logs, thereby reducing the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, the situation where the main board is abnormal due to forwarding the system logs can be avoided to a certain extent.

[0105] If the total generation rate of the system logs of each target business type is not greater than the preset generation rate threshold, it indicates that the business board currently generates fewer system logs of each target business type, and the business board does not need to determine whether the generation rate of the system log of each target business type is greater than the sending rate threshold corresponding to the system log of the target business type. In other words, for each system log of the target business type generated, the business board can directly send the system log of the target business type to the main board according to the generation rate of the system log of the target business type.

[0106] See also Figure 7 , Figure 7 A flowchart of another log processing method provided in an embodiment of the present application, the method may include the following steps:

[0107] S701: Traffic matching service generates logs.

[0108] When receiving the traffic data, the service board can determine the network security service that the current traffic data matches, and the service type to which the network security service that the current traffic data matches belongs is the target service type. When processing the network security service that the current traffic data matches, the service board generates a system log corresponding to the target service type.

[0109] S702: Determine whether to enable log rate limit. If yes, execute step S704; if no, execute step S703.

[0110] S703: Send the actually generated logs to the main board.

[0111] The service board can determine whether the log speed limit function is currently enabled. If the log speed limit function is not currently enabled, the service board does not need to determine whether the generation rate of the system log of each target service type is greater than the corresponding sending rate threshold of the system log of the target service type. In other words, the service board can send the actual generated logs to the main board, that is, for each generated system log of the target service type, the service board can directly send the system log of the target service type to the main board according to the generation rate of the system log of the target service type.

[0112] S704: Determine whether the log speed limit threshold is reached. If yes, execute step S705; if no, execute step S703.

[0113] If the log speed limit function is currently enabled, the business board can determine whether the log speed limit threshold is reached, that is, the business board determines whether the total generation rate of the system logs generated for each target business type is greater than the preset generation rate threshold. The preset generation rate threshold is the maximum sending rate of the business board. If the total generation rate of the system logs generated for each target business type is not greater than the preset generation rate threshold, the business board can send the logs actually generated to the main board, that is, for each system log generated for each target business type, the business board can directly send the system log of the target business type to the main board according to the generation rate of the system log of the target business type.

[0114] S705: Obtain information on the number of enabled target service types and divide the threshold into two equal parts.

[0115] If the total generation rate of system logs of each target service type is greater than the preset generation rate threshold, it indicates that the service board currently generates a large number of system logs of each target service type. The service board can obtain the number of enabled target service types and divide the threshold by half. That is, the service board can obtain the number of target service types, calculate the ratio of the maximum sending rate of the service board to the number of target service types, and use the obtained ratio as the sending rate threshold corresponding to each target service type.

[0116] S706: Determine whether the target business type reaches the threshold after equal division. If yes, execute step S707; if no, execute step S709.

[0117] If the total generation rate of the system logs of each target business type is greater than the preset generation rate threshold, the business board can further determine whether the target business type reaches the threshold after equalization, that is, the business board determines whether the generation rate of the system logs of each target business type is greater than the sending rate threshold corresponding to the system logs of the target business type, and sends the system logs of the target business type to the main board based on the judgment result.

[0118] S707: Send logs to the main board according to the threshold, and discard those exceeding the threshold.

[0119] For each target business type, if the generation rate of the system log of the target business type is greater than the sending rate threshold, the business board can send the system log of the target business type to the main board according to the sending rate threshold, and discard the system log of the target business type that is not sent to the main board.

[0120] S708: Count the number of discarded logs and send the logs to the log management device in the form of logs.

[0121] In order to enable the log management device to obtain the traffic data processing status of the service board, for each target service type, the service board can count the number of system logs of the target service type discarded, and send the counted number to the log management device.

[0122] S709: Send the actually generated logs to the main board.

[0123] For each target business type, if the generation rate of the system log of the target business type is not greater than the sending rate threshold, the business board can send the logs actually generated to the main board, that is, the business board can directly send the system log of the target business type to the main board according to the generation rate of the system log of the target business type.

[0124] Based on the above processing, if the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system log of the target business type can be sent to the main board according to the sending rate threshold. The system log sent to the main board can be controlled within a certain number to avoid the main board receiving too many system logs, so as to reduce the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, it can avoid the situation where the main board abnormalities are caused by forwarding the system logs to a certain extent.

[0125] Based on the same inventive concept, the present application embodiment also provides a log processing device, see Figure 8 , Figure 8 A structural diagram of a log processing device provided in an embodiment of the present application, the device is applied to a service board, and the device includes:

[0126] The first judgment module 801 is used to judge, for each generated system log of the target service type, whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type;

[0127] A first sending module 802, configured to send the system log of the target service type to the mainboard according to the sending rate threshold if the generation rate is greater than the sending rate threshold;

[0128] The second sending module 803 is configured to send the system log of the target service type to the mainboard according to the generation rate if the generation rate is not greater than the sending rate threshold.

[0129] Optionally, the device further comprises:

[0130] A sending rate threshold determination module is used to determine the sending rate threshold corresponding to the system log of each target business type generated by the business board according to the maximum sending rate of the business board before judging whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type.

[0131] Optionally, the sending rate threshold determination module is specifically used to calculate the sending rate threshold corresponding to the system log of each target business type based on the maximum sending rate of the business board and the weight of the system log of each target business type generated by the business board.

[0132] Optionally, the device further comprises:

[0133] A discarding module, configured to discard the system logs of the target service type that are not sent to the main board if the generation rate is greater than the sending rate threshold;

[0134] The third sending module is used to send the number of discarded system logs of the target service type to the log management device.

[0135] Optionally, the device further comprises:

[0136] A second judgment module is used to judge whether the log speed limit function is currently enabled before judging whether the generation rate of the system log of each target business type generated is greater than the sending rate threshold corresponding to the system log of the target business type; if so, trigger the first judgment module; if not, trigger the fourth sending module;

[0137] The fourth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

[0138] Optionally, the device further comprises:

[0139] a third judgment module, configured to judge whether the total generation rate of the system logs of each target business type generated is greater than a preset generation rate threshold before judging whether the generation rate of the system logs of the target business type is greater than the sending rate threshold corresponding to the system logs of the target business type for each target business type generated; if so, trigger the first judgment module, and if not, trigger the fifth sending module;

[0140] The fifth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

[0141] Based on the log processing device provided in the embodiment of the present application, if the system log generation rate of the target business type is greater than the sending rate threshold, it indicates that the current business board generates a large number of system logs. Therefore, the system logs of the target business type can be sent to the main board according to the sending rate threshold, and the system logs sent to the main board can be controlled within a certain number to avoid the main board receiving too many system logs, so as to reduce the system resources consumed when the main board forwards the system logs sent by the business board. Furthermore, it can avoid the situation where the main board is abnormal due to forwarding the system logs to a certain extent.

[0142] The present application also provides a service board, such as Fig. 9 As shown, it includes a processor 901, a communication interface 902, a memory 903 and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0143] Memory 903, used for storing computer programs;

[0144] The processor 901 is used to execute the program stored in the memory 903 to implement the following steps:

[0145] For each generated system log of the target service type, determining whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type;

[0146] If the generation rate is greater than the sending rate threshold, sending the system log of the target service type to the mainboard according to the sending rate threshold;

[0147] If the generation rate is not greater than the sending rate threshold, the system log of the target service type is sent to the mainboard according to the generation rate.

[0148] The communication bus mentioned in the above business board can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0149] The communication interface is used for communication between the above service boards and other devices.

[0150] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0151] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0152] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned log processing methods are implemented.

[0153] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any log processing method in the above embodiments.

[0154] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0155] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0156] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, service board, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A log processing method, characterized in that: The method is applied to a service board that processes network security services of various service types, and the method comprises: For each generated system log of the target service type, determining whether a generation rate of the system log of the target service type is greater than a sending rate threshold corresponding to the system log of the target service type; If the generation rate is greater than the sending rate threshold, sending the system log of the target service type to the mainboard according to the sending rate threshold; If the generation rate is not greater than the sending rate threshold, sending the system log of the target service type to the mainboard according to the generation rate; Before determining, for each generated system log of the target service type, whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type, the method further includes: Determine, according to the maximum sending rate of the service board, a sending rate threshold corresponding to the system log of each target service type generated by the service board; The determining, according to the maximum sending rate of the service board, a sending rate threshold corresponding to the system log of each target service type generated by the service board includes: According to the maximum sending rate of the business board and the weight of the system log of each target business type generated by the business board, the sending rate threshold corresponding to the system log of each target business type is calculated; different business types correspond to different priorities, and business types with high priorities are assigned large weights, and business types with low priorities are assigned small weights; the target business type with a larger weight has a larger sending rate threshold corresponding to it, and the sum of the sending rate thresholds corresponding to the system logs of each target business type is the maximum sending rate.

2. The method according to claim 1, characterized in that The method further comprises: If the generation rate is greater than the sending rate threshold, discarding the system logs of the target service type that are not sent to the mainboard; The number of discarded system logs of the target service type is sent to the log management device.

3. The method according to claim 1, characterized in that Before determining, for each generated system log of the target service type, whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type, the method further includes: Determine whether the log rate limit function is currently enabled; If the log rate limit function is currently enabled, the step of determining, for each generated system log of the target service type, whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type is executed; If the log rate limit function is not currently enabled, for each generated system log of the target service type, the system log of the target service type is sent to the mainboard according to the generation rate of the system log of the target service type.

4. The method according to claim 1, characterized in that: Before determining, for each generated system log of the target service type, whether the generation rate of the system log of the target service type is greater than the sending rate threshold corresponding to the system log of the target service type, the method further includes: Determine whether the total generation rate of the system logs of each target business type is greater than a preset generation rate threshold; If the total generation rate is greater than the preset generation rate threshold, executing the step of determining, for each generated system log of the target business type, whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type; If the total generation rate is not greater than the preset generation rate threshold, then for each generated system log of the target business type, the system log of the target business type is sent to the mainboard according to the generation rate of the system log of the target business type.

5. A log processing device, characterized in that: The device is applied to a service board for processing network security services of various service types, and the device comprises: A first judgment module is used to judge, for each generated system log of a target business type, whether a generation rate of the system log of the target business type is greater than a sending rate threshold corresponding to the system log of the target business type; A first sending module, configured to send the system log of the target service type to the mainboard according to the sending rate threshold if the generation rate is greater than the sending rate threshold; A second sending module, configured to send the system log of the target service type to the mainboard according to the generation rate if the generation rate is not greater than the sending rate threshold; The device also includes: A sending rate threshold determination module, used to determine the sending rate threshold corresponding to the system log of each target business type generated by the business board according to the maximum sending rate of the business board before judging whether the generation rate of the system log of the target business type is greater than the sending rate threshold corresponding to the system log of the target business type; The sending rate threshold determination module is specifically used to calculate the sending rate threshold corresponding to the system log of each target business type according to the maximum sending rate of the business board and the weight of the system log of each target business type generated by the business board; different business types correspond to different priorities, high-priority business types are assigned large weights, and low-priority business types are assigned small weights; the target business type with a larger weight corresponds to a larger sending rate threshold, and the sum of the sending rate thresholds corresponding to the system logs of each target business type is the maximum sending rate.

6. The device according to claim 5, characterized in that The device also includes: A discarding module, configured to discard the system logs of the target service type that are not sent to the main board if the generation rate is greater than the sending rate threshold; The third sending module is used to send the number of discarded system logs of the target service type to the log management device.

7. The device according to claim 5, characterized in that The device also includes: A second judgment module is used to judge whether the log speed limit function is currently enabled before judging whether the generation rate of the system log of each target business type generated is greater than the sending rate threshold corresponding to the system log of the target business type; if so, trigger the first judgment module; if not, trigger the fourth sending module; The fourth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

8. The device according to claim 5, characterized in that The device also includes: a third judgment module, configured to judge whether the total generation rate of the system logs of each target business type generated is greater than a preset generation rate threshold before judging whether the generation rate of the system logs of the target business type is greater than the sending rate threshold corresponding to the system logs of the target business type for each target business type generated; if so, trigger the first judgment module, and if not, trigger the fifth sending module; The fifth sending module is used to send the system log of each target business type generated according to the generation rate of the system log of the target business type to the main board.

9. A service board, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 4 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 4 are implemented.

Images