A big data analysis and processing method and system for intelligent network security

Abstract

The present invention discloses a big data analysis and processing method and system for intelligent network security, the method comprising: obtaining network logs and user logs based on the interaction process of network data; extracting valid logs from network logs based on network protocol addresses and network interaction dates to construct a security detection data set; designing a security evaluation algorithm and calculating a security evaluation coefficient; establishing a security analysis model to analyze the network security status of user logs; updating the security detection data set and security evaluation parameters, and publishing the user's current network security status. The present invention analyzes the network security status by using big data technology in a combination of designing a security evaluation algorithm and establishing a security analysis model, removes redundant information when constructing a security detection data set, and regularly updates the security detection data set and security evaluation parameters to improve the accuracy of network security analysis.

Description

Technical Field

[0001] The present invention relates to the field of network security analysis, and in particular to an intelligent network security big data analysis and processing method and system. Background Art

[0002] With the popularization of the Internet in today's society, human economic production and life are increasingly closely integrated through the network. With the increase in the complexity of processing tasks, the network structure has not only become more complex, but the data scale has also become larger and larger, which has led to network security threats and risks faced by users in the process of using the Internet. Network attacks have the characteristics and development trends of being distributed, large-scale and complex, and network security issues have become the focus of attention.

[0003] Traditional network security detection methods only detect and identify network protocols or single network protocol addresses. Not only do they fail to utilize rich network resource information, but they also have inaccurate identification. In the detection process, fixed thresholds have detection defects, high computational complexity, and low computational efficiency. The increasingly mature big data technology can make full use of the existing rich information and intelligently analyze the user's network security status. Therefore, it is urgently necessary to propose a new network security analysis and processing method and system in combination with big data technology. Summary of the invention

[0004] The present invention provides an intelligent network security big data analysis and processing method and system to solve the current technical problems of insufficient network security analysis information richness, low computing efficiency, inaccurate network attack identification, and detection defects of fixed thresholds.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] The present invention provides an intelligent network security big data analysis and processing method, comprising:

[0007] S1: Based on the interactive process of network data, network logs and user logs are obtained;

[0008] S2: Extract valid logs from network logs based on network protocol addresses and network interaction dates to build a security detection dataset;

[0009] S3: Design a safety evaluation algorithm and initialize safety evaluation parameters, including the upper health threshold. , Risk lower limit threshold and capacity thresholds ;

[0010] Via user log Total traffic and number of downloaded files , calculate the average file size of user logs ,in ; Malicious traffic percentage in user logs Total traffic volume of user logs from 0:00 to 6:00 and the total traffic volume of user logs from 6:00 to 24:00 Calculate, where ;

[0011] Calculate the safety evaluation factor F if Less than or equal to at the same time Less than or equal to the capacity threshold , F is 0; if Greater than or equal to , F is 2; in other cases, F is 1;

[0012] If F is equal to 0, it is healthy; if F is equal to 1, it is a potential risk; if F is equal to 2, it is a high risk; if the safety evaluation coefficient is healthy and high risk, directly output and warn that the user log status is high risk; if the safety evaluation coefficient is a potential risk, execute step S4;

[0013] S4: Establish a safety analysis model and define the initial value range of the health center and the initial value range of the risk center;

[0014] Computational safety detection dataset The average capacity of each valid log file The reciprocal of and malicious traffic share ; and based on and Select the initial health cluster center and the initial risk cluster center;

[0015] Use the security analysis model to determine the security status of valid logs in the security detection data set;

[0016] After iterative solution, the health center and risk center are obtained and calculated. Distance to health centres and The distance to the risk center is used to determine the security of user logs;

[0017] S5: Update the security detection data set and security evaluation parameters, and publish the user's current network security status.

[0018] Furthermore, the step S1 comprises:

[0019] The interactive process based on network data refers to the process in which the user terminal device performs network data interaction to form data file transmission, and obtains network logs and user logs in this process;

[0020] The network log Contains user's previous logs and other logs , the expression is , Represents the network, Indicates the user's previous information. Indicates other;

[0021] The network log extracts the total traffic and the number of downloaded files, among which the user's previous log Contains the total traffic volume of the user's previous logs and the number of downloaded files in the user's previous logs. Contains the total traffic of other logs and the number of downloaded files of other logs, and the user's previous logs and other logs The expression is as follows:

[0022] ;

[0023] ;

[0024] in, Indicates The total traffic volume of the previous logs of users, Indicates The number of downloaded files of a user's previous logs, n is the number of previous logs of the user, , is an integer; Indicates b Total traffic of other logs, Indicates b The number of other logs downloaded, z is the number of other logs, , z is an integer;

[0025] User past logs It also carries timestamp, network protocol address and network interaction date information. Other logs It also carries timestamp, network protocol address and network interaction date information;

[0026] The user log is , whose expression is ,in, Indicates the total traffic volume of user logs. Indicates the number of downloaded files for the user log.

[0027] Furthermore, the step S2 comprises:

[0028] The total number of valid logs is set to 720;

[0029] The effective log is screened and extracted based on the network protocol address and the network interaction date. The effective log has a cycle of 24 hours. The specific steps of screening and extraction are as follows:

[0030] (1) Filter and extract network logs whose network interaction dates are within 180 cycles before the user log date;

[0031] (2) Based on the network logs screened and extracted in step (1), 180 previous user logs and 540 other logs are extracted according to the network protocol address. The number of other logs extracted from the same network protocol address shall not exceed 5;

[0032] The 180 previous logs of users and 540 other logs constitute 720 valid logs, and a security detection data set is constructed. The security detection data set includes the 720 valid logs.

[0033] It should be noted that the security detection data set not only includes network logs, but also previous user logs, which makes the judgment of the user's current network security status more targeted, and the selection of other network logs is random. However, excluding extreme cases, the security detection data set contains both healthy logs and high-risk logs by default.

[0034] Furthermore, the step S3 comprises:

[0035] Design a safety evaluation algorithm and initialize safety evaluation parameters, including the upper health threshold. , Risk lower limit threshold and capacity thresholds ;

[0036] Via user log Total traffic and number of downloaded files , calculate the average file size of user logs , the specific expression is as follows:

[0037] ;

[0038] Via user log Total traffic Calculate the malicious traffic ratio in user logs and the malicious traffic ratio in user logs Total traffic volume of user logs from 0:00 to 6:00 and the total traffic volume of user logs from 6:00 to 24:00 The specific expression is shown in the following formula: ;

[0039] Mean capacity by file The reciprocal of and malicious traffic share Calculate the safety assessment factor;

[0040] If the safety evaluation coefficient is healthy and high risk, it is directly output and a warning is issued that the user log status is high risk. If the safety evaluation coefficient is potential risk, step S4 is executed.

[0041] Furthermore, the average capacity of the file The reciprocal of and malicious traffic share Calculation of safety assessment factors includes:

[0042] Mean capacity by file The reciprocal of and malicious traffic share Calculate the safety evaluation factor, if the average file capacity The reciprocal of Less than or equal to Malicious traffic ratio Less than or equal to the capacity threshold , the safety evaluation factor F is 0; if the average file capacity The reciprocal of Greater than or equal to , the safety evaluation coefficient F is 2; in other cases, the safety evaluation coefficient F is 1. The specific expression is shown in the following formula:

[0043] ;

[0044] ;

[0045] ;

[0046] in, Indicates other situations;

[0047] If the safety evaluation coefficient F is equal to 0, it is healthy; if the safety evaluation coefficient F is equal to 1, it is a potential risk; if the safety evaluation coefficient F is equal to 2, it is a high risk.

[0048] Furthermore, the step S4 comprises:

[0049] Establish a safety analysis model and define the initial value range of the health center and the initial value range of the risk center ;

[0050] Security detection dataset Process the valid logs in , Indicates i Valid logs, , i is an integer; i The total number of valid logs is , No. i The number of downloaded files of valid logs is ;

[0051] Calculate the average file size of each valid log The reciprocal of and malicious traffic share ;

[0052] ;

[0053] ;

[0054] in, Indicates i The average file size of valid logs, Indicates i Malicious traffic accounts for 100% of valid logs; Indicates i The total traffic volume from 0:00 to 6:00 in the valid logs. Indicates i Total traffic volume of valid logs from 6:00 to 24:00;

[0055] Select exist The valid logs in the range are taken as the initial healthy cluster centers; exist The valid log in the range is taken as the initial risk cluster center;

[0056] It should be noted that the initial value range of the health center and the initial value range of the risk center are empirical value ranges;

[0057] The security status of 720 valid logs in the security detection dataset is judged.

[0058] Furthermore, the determination of the security status of the 720 valid logs in the security detection data set includes:

[0059] The security status of 720 valid logs in the security detection data set is judged. With 2-dimensional attribute data, , For effective log The first dimension feature attribute data corresponds to the effective log The average file size in The reciprocal of , For effective log The second dimension feature attribute data corresponds to the proportion of malicious traffic ;

[0060] There are two state clusters, one of which is healthy and the other is risky. The one state cluster is healthy, which means that all valid logs belonging to the cluster are in a healthy state, and the other state cluster is risky, which means that all valid logs belonging to the cluster are in a high-risk state. The number of valid logs under the state cluster is 720. The expression of the security analysis model is shown in the following formula:

[0061] ;

[0062] The following constraints are met:

[0063] ;

[0064] The U represents the membership degree of the valid log , represents the membership factor of the i-th valid log corresponding to the j-th cluster center, Represents the cluster centers corresponding to the two state clusters, where represents the initial healthy cluster center, represents the initial risk cluster center, for and The Euclidean distance between them, m represents the fuzzy index, and the range of m is (1.5, 2.5);

[0065] The operation process of the safety analysis model is to continuously iterate the objective function The minimization process uses the Lagrange multiplier method in the iterative solution to obtain , the specific expression is as follows:

[0066] ;

[0067] ;

[0068] After iterative solution, we can get the health center and Risk Center , calculate the average file size of user logs The reciprocal of and malicious traffic share Composition coordinates , To the health center Distance and To the Risk Center Distance ,if , then the security status of the user log is healthy if , then the network security status of the user log is high risk.

[0069] It should be noted that rational values ​​can be obtained by setting the number of iterations to the security analysis model. For example, the number of iterations is set to 10, where the center that minimizes the objective function is the health center and the risk center. The above user logs are healthy or in a healthy state, and the meaning is the same; the user logs are high risk or in a high risk state, and the meaning is the same;

[0070] This solution constructs a coordinate system based on the inverse of the average file capacity and the proportion of malicious traffic extracted from user logs and valid logs, and constructs data coordinates based on the inverse of the average file capacity and the proportion of malicious traffic extracted from user logs and valid logs. It uses coordinate clustering to perform screening and judgment, and then judges the security status of user logs. Compared with existing solutions, this solution is more intuitive, easy to operate and highly accurate.

[0071] Furthermore, the step S5 comprises:

[0072] For the same user, the security detection data set and security evaluation parameters are updated every 90 cycles;

[0073] The security detection data set is updated every 90 cycles. The network logs in the security detection data set are valid logs. For the previous logs of users, the network interaction date is within 180 cycles before the user log date. For other logs, the random principle is adopted for screening, and the network protocol address shall not be extracted by different security monitoring data sets;

[0074] The safety evaluation parameters are updated every 90 cycles, and the updated safety evaluation parameters include the upper health threshold and the lower risk threshold , calculate the average capacity of all valid log files in the updated security detection dataset The reciprocal of The average , calculate the percentage of high risk in the past 90 cycles , the upper health threshold and the lower risk threshold The update expression is as follows:

[0075] ;

[0076] ;

[0077] in, Indicates the updated upper health threshold, Indicates the updated lower risk threshold;

[0078] Publish the user's current network security status.

[0079] By regularly updating the upper health threshold and the lower risk threshold The limitation of fixed threshold can be avoided, thereby improving the intelligence of the scheme.

[0080] On the other hand, the present invention also provides an intelligent network security big data analysis and processing system, comprising:

[0081] The data acquisition and preprocessing module obtains network logs and user logs based on the interaction process of network data. The network logs contain the user's previous logs and other logs. Both network logs and user logs contain the total traffic volume and the number of downloaded files. Based on the network protocol address and network interaction date, valid logs are extracted from the network logs to build a security detection data set.

[0082] The security evaluation module designs a security evaluation algorithm, initializes security evaluation parameters, and calculates the security evaluation coefficient by the average file capacity of the user log and the proportion of malicious traffic. If the security evaluation coefficient belongs to healthy and high risk, it directly outputs and warns that the user log status belongs to high risk. If the security evaluation coefficient belongs to potential risk, execute step S4;

[0083] The security analysis module builds a security analysis model and uses security detection data sets to analyze the network security status of user logs;

[0084] The data set update and status release module updates the security detection data set and security evaluation parameters, and releases the user's current network security status.

[0085] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0086] 1. The intelligent network security big data analysis method proposed in the present invention is divided into two aspects. On the one hand, in order to ensure the rapidity of the method, a security evaluation algorithm is designed to calculate the security evaluation coefficient through the average file capacity of the user log and the proportion of malicious traffic. If the user log status is healthy or high risk, it is output in time, and further analysis is performed on the potential risks. On the other hand, a security analysis model is established to further discriminate and analyze the user log status that belongs to potential risks. Combined with the security detection data set, the status of the user log can be accurately analyzed. Compared with other network security analysis methods, this scheme adopts a combination of coarse analysis and fine analysis while ensuring the detection accuracy, which can further improve the computing efficiency and improve the utilization efficiency of computing resources.

[0087] 2. When analyzing big data, this solution establishes a dedicated security detection data set for user logs. Combined with the user's previous log information, it can further explore the user's network risk characteristics and remove redundant information, reducing the interference of redundant information on network security analysis and improving the accuracy of the analysis. The security detection data set will be updated regularly to avoid detection defects caused by fixed data sets.

[0088] 3. The present invention establishes a security analysis model to further analyze and judge the potential risks in the security evaluation algorithm. In order to improve the iteration efficiency of the health center and the risk center, the initial value range of the health center and the initial value range of the risk center are defined in combination with the security detection data set, so as to quickly obtain accurate health centers and risk centers, and regularly update security evaluation parameters to avoid fixed security evaluation parameters from causing misanalysis of network security, improve accuracy, and further improve the intelligence level of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0090] Figure 1 A big data analysis and processing method for intelligent network security and a system step implementation diagram provided by an embodiment of the present invention;

[0091] Figure 2 A schematic diagram of a big data analysis and processing system module for intelligent network security provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0093] Embodiment 1:

[0094] This embodiment will provide a more detailed description of an intelligent network security big data analysis and processing method of the present invention in conjunction with the corresponding drawings. Figure 1 As shown, the following steps are included:

[0095] Step S1: Based on the interactive process of network data, network logs and user logs are obtained, including:

[0096] The interactive process based on network data refers to the process in which the user terminal device performs network data interaction to form data file transmission, and obtains network logs and user logs in this process;

[0097] The network log Contains user's previous logs and other logs , the expression is , Represents the network, Indicates the user's previous information. Indicates other;

[0098] The network log extracts the total traffic and the number of downloaded files, among which the user's previous log Contains the total traffic volume of the user's previous logs and the number of downloaded files in the user's previous logs. Contains the total traffic of other logs and the number of downloaded files of other logs, and the user's previous logs and other logs The expression is as follows:

[0099] ;

[0100] ;

[0101] in, Indicates The total traffic volume of the previous logs of users, Indicates The number of downloaded files of a user's previous logs, n is the number of previous logs of the user, , is an integer; Indicates b Total traffic of other logs, Indicates b The number of other logs downloaded, z is the number of other logs, , z is an integer;

[0102] User past logs It also carries timestamp, network protocol address and network interaction date information. Other logs It also carries timestamp, network protocol address and network interaction date information;

[0103] The user log is , whose expression is ,in, Indicates the total traffic volume of user logs. Indicates the number of downloaded files for the user log.

[0104] In order to realize the security analysis of user logs, this solution combines network logs for big data analysis and processing. In the specific analysis process, the total traffic volume of network logs and user logs, the number of downloaded files, timestamps, network protocol addresses, and network interaction dates are required to realize multi-dimensional analysis and improve analysis accuracy.

[0105] Step S2: Extract valid logs from network logs based on network protocol addresses and network interaction dates to construct a security detection data set, including:

[0106] The total number of valid logs is set to 720;

[0107] Valid logs are filtered and extracted based on the network protocol address and network interaction date. The valid logs are filtered and extracted in a cycle of 24 hours. The specific steps of filtering and extraction are as follows:

[0108] (1) Filter and extract network logs whose network interaction dates are within 180 cycles before the user log date;

[0109] (2) Based on the network logs screened and extracted in step (1), 180 previous user logs and 540 other logs are extracted according to the network protocol address. The number of other logs extracted from the same network protocol address shall not exceed 5;

[0110] The 180 previous logs of users and 540 other logs constitute 720 valid logs, and a security detection data set is constructed. The security detection data set includes the 720 valid logs.

[0111] In order to avoid the impact of redundant information on this solution, this solution screens the network logs and extracts valid logs. Compared with the solution that processes all network logs, this solution can reduce unnecessary computing power loss and thus reduce the hardware cost of implementing this solution.

[0112] Step S3: Design a security evaluation algorithm, initialize security evaluation parameters, calculate the security evaluation coefficient by the average file capacity of the user log and the proportion of malicious traffic, if the security evaluation coefficient is healthy and high risk, directly output and warn that the user log status is high risk, if the security evaluation coefficient is a potential risk, execute step S4, including:

[0113] Design a safety evaluation algorithm and initialize safety evaluation parameters, including the upper health threshold. , Risk lower limit threshold and capacity thresholds ;

[0114] Via user log Total traffic and number of downloaded files , calculate the average file size of user logs , the specific expression is as follows:

[0115] ;

[0116] Via user log Total traffic Calculate the malicious traffic ratio in user logs and the malicious traffic ratio in user logs Total traffic volume of user logs from 0:00 to 6:00 and the total traffic volume of user logs from 6:00 to 24:00 The specific expression is shown in the following formula: ;

[0117] Mean capacity by file The reciprocal of and malicious traffic share Calculate the safety assessment factor;

[0118] If the safety evaluation coefficient is healthy and high risk, it is directly output and a warning is issued that the user log status is high risk. If the safety evaluation coefficient is potential risk, step S4 is executed.

[0119] Mean capacity by file The reciprocal of and malicious traffic share Calculate the safety evaluation factor, if the average file capacity The reciprocal of Less than or equal to Malicious traffic ratio Less than or equal to the capacity threshold , the safety evaluation factor F is 0; if the average file capacity The reciprocal of Greater than or equal to , the safety evaluation coefficient F is 2; in other cases, the safety evaluation coefficient F is 1. The specific expression is shown in the following formula:

[0120] ;

[0121] ;

[0122] ;

[0123] in, Indicates other situations;

[0124] If the safety evaluation coefficient F is equal to 0, it is healthy; if the safety evaluation coefficient F is equal to 1, it is a potential risk; if the safety evaluation coefficient F is equal to 2, it is a high risk.

[0125] In this embodiment, the main focus is on the risk of network attacks that launch attacks in the early morning and install a large number of junk files. This type of network risk has a large number of files, a single file has a very small capacity, and the traffic is time-sharing, which is not easy to detect and difficult to distinguish through traditional methods.

[0126] Step S4: Establish a security analysis model and use the security detection data set to analyze the network security status of the user log, including:

[0127] Establish a safety analysis model and define the initial value range of the health center and the initial value range of the risk center ;

[0128] Security detection dataset Process the valid logs in , Indicates i Valid logs, , i is an integer; i The total number of valid logs is , No. i The number of downloaded files of valid logs is ;

[0129] Calculate the average file size of each valid log The reciprocal of and malicious traffic share ;

[0130] ;

[0131] ;

[0132] in, Indicates i The average file size of valid logs, Indicates i Malicious traffic accounts for 100% of valid logs; Indicates i The total traffic volume from 0:00 to 6:00 in the valid logs. Indicates i Total traffic volume of valid logs from 6:00 to 24:00;

[0133] Select exist The valid logs in the range are used as the initial healthy cluster centers, where the selected valid logs correspond to ;

[0134] Select exist The valid logs in the range are taken as the initial risk clustering centers, where the selected valid logs correspond to , ;

[0135] It should be noted that the above process is carried out in two steps. i Take 10 o'clock exist range, you can As the initial health cluster center, it can be used as i Take 120 hours exist range, you can As the initial value of health cluster center;

[0136] The security status of 720 valid logs in the security detection data set is judged. With 2-dimensional attribute data, , For effective log The first dimension feature attribute data corresponds to the effective log The average file size in The reciprocal of , For effective log The second dimension feature attribute data corresponds to the proportion of malicious traffic ;

[0137] Suppose there are two state clusters, the number of valid logs under the state clusters is 720, and the expression of the security analysis model is as follows:

[0138] ;

[0139] The following constraints are met:

[0140] ;

[0141] The U represents the membership degree of the valid log, , represents the membership factor of the i-th valid log corresponding to the j-th cluster center, Represents the cluster centers corresponding to the two state clusters, where represents the initial healthy cluster center, represents the initial risk cluster center, for and The Euclidean distance between them, m represents the fuzzy index, and the range of m is (1.5, 2.5). In this scheme, ;

[0142] The operation process of the safety analysis model is to continuously iterate the objective function The minimization process uses the Lagrange multiplier method in the iterative solution to obtain , the specific expression is as follows:

[0143] ;

[0144] ;

[0145] After iterative solution, we get the health center and Risk Center , calculate the average file size of user logs The reciprocal of and malicious traffic share Composition coordinates , To the health center Distance and To the Risk Center Distance ,if , then the security status of the user log is healthy if , then the network security status of the user log is high risk.

[0146] This solution adopts the idea of ​​clustering to cluster the healthy intervals and risk intervals of user logs, thereby improving the accuracy of user log security analysis. Through clustering, it is more vivid and clear, and has a higher level of intelligence. Compared with traditional methods, this solution has more convenient operability and higher accuracy.

[0147] Step S5: Update the security detection data set and security evaluation parameters, and publish the user's current network security status, including:

[0148] For the same user, the security detection data set and security evaluation parameters are updated every 90 cycles;

[0149] The security detection data set is updated every 90 cycles. The network logs in the security detection data set are valid logs. For the previous logs of users, the network interaction date is within 180 cycles before the user log date. For other logs, the random principle is adopted for screening, and the network protocol address shall not be extracted by different security monitoring data sets;

[0150] The safety evaluation parameters are updated every 90 cycles, and the updated safety evaluation parameters include the upper health threshold and the lower risk threshold , calculate the average capacity of all valid log files in the updated security detection dataset The reciprocal of The average , calculate the percentage of high risk in the past 90 cycles , the upper health threshold and the lower risk threshold The update expression is as follows:

[0151] ;

[0152] ;

[0153] in, Indicates the updated upper health threshold, Indicates the updated lower risk threshold;

[0154] Publish the user's current network security status.

[0155] Embodiment 2:

[0156] The present invention also provides an intelligent network security big data analysis and processing system, the main modules of which are as follows: Figure 2 As shown:

[0157] The data acquisition and preprocessing module obtains network logs and user logs based on the interaction process of network data. The network logs contain the user's previous logs and other logs. Both network logs and user logs contain the total traffic volume and the number of downloaded files. Based on the network protocol address and network interaction date, valid logs are extracted from the network logs to build a security detection data set.

[0158] The security evaluation module designs a security evaluation algorithm, initializes security evaluation parameters, and calculates the security evaluation coefficient by the average file capacity of the user log and the proportion of malicious traffic. If the security evaluation coefficient belongs to healthy and high risk, it directly outputs and warns that the user log status belongs to high risk. If the security evaluation coefficient belongs to potential risk, execute step S4;

[0159] The security analysis module builds a security analysis model and uses security detection data sets to analyze the network security status of user logs;

[0160] The data set update and status release module updates the security detection data set and security evaluation parameters, and releases the user's current network security status.

[0161] The intelligent network security big data analysis and processing system of the present embodiment corresponds to the multi-information source aviation deception detection method of the above-mentioned embodiment; wherein, the functions implemented by each functional module in the intelligent network security big data analysis and processing system of the present embodiment correspond one-to-one to each process step in the multi-information source aviation deception detection method of the above-mentioned embodiment; therefore, they will not be repeated here.

[0162] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0164] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A big data analysis and processing method for intelligent network security, characterized in that: include: S1: Based on the interactive process of network data, network logs and user logs are obtained; S2: Extract valid logs from network logs based on network protocol addresses and network interaction dates to build a security detection dataset; S3: Design a safety evaluation algorithm and initialize safety evaluation parameters, including the upper health threshold. , Risk lower limit threshold and capacity thresholds ; Via user log Total traffic and number of downloaded files , calculate the average file size of user logs ,in ; Malicious traffic percentage in user logs Total traffic volume of user logs from 0:00 to 6:00 and the total traffic volume of user logs from 6:00 to 24:00 Calculate, where ; Calculate the safety evaluation factor F if Less than or equal to at the same time Less than or equal to the capacity threshold , F is 0; if Greater than or equal to , F is 2; in other cases, F is 1; If F is equal to 0, it is healthy; if F is equal to 1, it is a potential risk; if F is equal to 2, it is a high risk; if the safety evaluation coefficient is healthy and high risk, directly output and warn that the user log status is high risk; if the safety evaluation coefficient is a potential risk, execute step S4; S4: Establish a safety analysis model and define the initial value range of the health center and the initial value range of the risk center; Computational safety detection dataset The average file size of each valid log file The reciprocal of and malicious traffic share ; and based on and Select the initial health cluster center and the initial risk cluster center; Use the security analysis model to determine the security status of valid logs in the security detection data set; After iterative solution, the health center and risk center are obtained and calculated. Distance to health centres and The distance to the risk center is used to determine the security of user logs; The step S4 comprises: Establish a safety analysis model and define the initial value range of the health center and the initial value range of the risk center ; Security detection dataset Process the valid logs in , Indicates i Valid logs, , i is an integer; i The total number of valid logs is , No. i The number of downloaded files of valid logs is ; Calculate the average file size of each valid log The reciprocal of and malicious traffic share ; ； ； in, Indicates i The average file size of valid logs, Indicates i Malicious traffic accounts for 100% of valid logs; Indicates i The total traffic volume from 0:00 to 6:00 in the valid logs. Indicates i Total traffic volume of valid logs from 6:00 to 24:00; Select exist The valid logs in the range are taken as the initial healthy cluster centers; exist The valid log in the range is taken as the initial risk cluster center; Determine the security status of 720 valid logs in the security detection data set; The determination of the security status of the 720 valid logs in the security detection data set includes: The security status of 720 valid logs in the security detection data set is judged. With 2-dimensional attribute data, , For effective log The first dimension feature attribute data corresponds to the effective log The average file size in The reciprocal of , For effective log The second dimension feature attribute data corresponds to the proportion of malicious traffic ; Suppose there are two state clusters, the number of valid logs under the state clusters is 720, and the expression of the security analysis model is as follows: ； The following constraints are met: ； The U represents the membership degree of the valid log, , represents the membership factor of the i-th valid log corresponding to the j-th cluster center, Represents the cluster centers corresponding to the two state clusters, where represents the initial healthy cluster center, represents the initial risk cluster center, for and The Euclidean distance between them, m represents the fuzzy index, and the range of m is (1.5, 2.5); The operation process of the safety analysis model is to continuously iterate the objective function The minimization process uses the Lagrange multiplier method in the iterative solution to obtain ; After iterative solution, we can get the health center and Risk Center , the average file size of user logs The reciprocal of and malicious traffic share Composition coordinates ,calculate To the health center Distance and To the Risk Center Distance ,if , then the security status of the user log is healthy if , then the network security status of the user log is high risk; S5: Update the security detection data set and security evaluation parameters, and publish the user's current network security status.

2. The big data analysis and processing method for intelligent network security according to claim 1, characterized in that: The step S1 comprises: The interactive process based on network data refers to the process in which the user terminal device performs network data interaction to form data file transmission, and obtains network logs and user logs in this process; The network log Contains user's previous logs and other logs , the expression is , Represents the network, Indicates the user's previous information. Indicates other; The network log extracts the total traffic and the number of downloaded files, among which the user's previous log Contains the total traffic volume of the user's previous logs and the number of downloaded files in the user's previous logs. Contains the total number of traffic and the number of downloaded files of other logs; The user's previous logs It also carries timestamps, network protocol addresses, and network interaction date information, as well as other logs. It also carries timestamp, network protocol address, and network interaction date information; The user log is , whose expression is ,in, Indicates the total traffic volume of user logs. Indicates the number of downloaded files for the user log.

3. The big data analysis and processing method for intelligent network security according to claim 1, characterized in that: The step S2 comprises: The total number of valid logs is set to 720; The effective log is screened and extracted based on the network protocol address and the network interaction date. The effective log has a cycle of 24 hours. The specific steps of screening and extraction are as follows: (1) Filter and extract network logs whose network interaction dates are within 180 cycles before the user log date; (2) Based on the network logs screened and extracted in step (1), 180 previous user logs and 540 other logs are extracted according to the network protocol address. The number of other logs extracted from the same network protocol address shall not exceed 5; The 180 previous logs of users and 540 other logs constitute 720 valid logs, and a security detection data set is constructed. The security detection data set includes the 720 valid logs.

4. The intelligent network security big data analysis and processing method according to claim 1, characterized in that: The step S5 comprises: For the same user, the security detection data set and security evaluation parameters are updated every 90 cycles; The security detection data set is updated every 90 cycles. The network logs in the security detection data set are valid logs. For the previous logs of users, the network interaction date is within 180 cycles before the user log date. For other logs, the random principle is adopted for screening, and the network protocol address shall not be extracted by different security monitoring data sets; The safety evaluation parameters are updated every 90 cycles, and the updated safety evaluation parameters include the upper limit threshold of health and the lower risk threshold , calculate the average capacity of all valid log files in the updated security detection dataset The reciprocal of The average , calculate the percentage of high risk in the past 90 cycles , the upper health threshold and the lower risk threshold The update expression is as follows: ； ； in, Indicates the updated upper health threshold, Indicates the updated lower risk threshold; Publish the user's current network security status.

5. An intelligent network security big data analysis and processing system, characterized in that: include: The data acquisition and preprocessing module obtains network logs and user logs based on the interaction process of network data. The network logs contain the user's previous logs and other logs. Both network logs and user logs contain the total traffic volume and the number of downloaded files. Based on the network protocol address and network interaction date, valid logs are extracted from the network logs to build a security detection data set. The security evaluation module designs a security evaluation algorithm, initializes security evaluation parameters, and calculates the security evaluation coefficient by the average file capacity of the user log and the proportion of malicious traffic. If the security evaluation coefficient belongs to healthy and high risk, it directly outputs and warns that the user log status belongs to high risk. If the security evaluation coefficient belongs to potential risk, execute step S4; The security analysis module builds a security analysis model and uses security detection data sets to analyze the network security status of user logs; The data set update and status release module updates the security detection data set and security evaluation parameters, and releases the user's current network security status; To realize an intelligent network security big data analysis and processing method as described in any one of claims 1-4.

Images