A big data management platform for intelligent analysis of computer networks

Abstract

The present invention discloses a big data management platform for computer network intelligent analysis, belonging to the technical field of computer network intelligent analysis. The present invention includes a data acquisition and processing module, a computer network status acquisition module, a computer network diagnosis module, an intelligent analysis module, a computer network repair module, and a big data management module; the data acquisition and processing module is used to collect and process the data input into the big data management platform, and transmit the collected and processed data to the intelligent analysis module; the computer network status acquisition module is used to collect the load information, network traffic information, and network status information during the operation of the computer network, and transmit the collected information to the computer network diagnosis module; the computer network diagnosis module is used to receive the collection information transmitted by the computer network status acquisition module, diagnose the computer network according to the received information, and transmit the diagnosis result to the intelligent analysis module.

Description

Technical Field

[0001] The present invention relates to the technical field of computer network intelligent analysis, and specifically to a big data management platform for computer network intelligent analysis. Background Technique

[0002] Existing big data has irregularity and uncertainty. Simply relying on early computer logical analysis cannot accurately judge the authenticity of big data. Therefore, higher requirements are put forward for computer network information security. Computer network intelligent analysis is mainly reflected in the intelligent analysis of firewall systems and intrusion systems, thereby ensuring computer network information security.

[0003] When the existing big data management platform manages big data, it cannot quickly perform intelligent identification and processing on big data, so it cannot diagnose the faults existing in the operation of the computer network in a timely manner, resulting in the long-term failure state of the big data management platform, reducing the ability of the big data management platform to prevent network viruses, and when using the existing firewall system to perform intelligent identification on the computer network, the ability to intercept harmful data is weak, thereby reducing the management effect of the big data management platform. Also, when detecting and intercepting intrusion information existing in the computer network, usually through operations such as data information collection and collation, intelligent identification and detection of harmful big data are performed. Due to the irregularity and uncertainty of big data, the processing efficiency of big data is low, reducing the management efficiency of the big data management platform. Summary of the Invention

[0004] The purpose of the present invention is to provide a big data management platform for computer network intelligent analysis to solve the problems raised in the above background technique.

[0005] To solve the above technical problems, the present invention provides the following technical solution: A big data management platform for computer network intelligent analysis, the big data management platform includes a data acquisition and processing module, a computer network status acquisition module, a computer network diagnosis module, an intelligent analysis module, a computer network repair module, and a big data management module;

[0006] The data acquisition and processing module is used to collect and process the data input into the big data management platform, and transmit the collected and processed data to the intelligent analysis module;

[0007] The computer network status acquisition module is used to collect the load information, network traffic information, and network status information during the operation of the computer network, and transmit the collected information to the computer network diagnosis module;

[0008] The computer network diagnosis module is used to receive the acquisition information transmitted by the computer network status acquisition module, diagnose the computer network according to the received information, and transmit the diagnosis result to the intelligent analysis module;

[0009] The intelligent analysis module is used to receive the data information transmitted by the data acquisition and processing module and the diagnosis result transmitted by the computer network diagnosis module, perform intelligent analysis on the computer network based on the received content, and transmit the analysis result to the computer network repair module and the big data management module respectively;

[0010] The computer network repair module is used to receive the analysis result transmitted by the intelligent analysis module and repair the computer network based on the analysis result;

[0011] The big data management module is used to receive the analysis result transmitted by the intelligent analysis module and effectively manage the big data according to the analysis result.

[0012] Furthermore, the data acquisition and processing module includes a data acquisition unit, a data screening unit, and a data processing unit;

[0013] The data acquisition unit acquires the data input into the big data management platform, the data input time, and the expected data propagation path, and transmits the acquired information to the data processing unit;

[0014] The data screening unit receives the acquisition information transmitted by the data acquisition unit, judges the type of the acquired data, predicts the propagation path of the acquired data of the corresponding type according to the judgment result, matches the predicted propagation path with the expected propagation path of the corresponding data, marks the acquired data that fails to match, and transmits the marked acquired data to the data processing unit;

[0015] The data processing unit receives the marked data transmitted by the data screening unit, predicts the triggering probability of the acquired data of the corresponding type according to the received data, and predicts the mutation probability of the acquired data, and transmits the prediction result to the intelligent analysis module.

[0016] Furthermore, the specific method for the data screening unit to predict the propagation path of the acquired data of the corresponding type is as follows:

[0017] 1) Obtain the keywords in the acquired data, calculate the proportion of various keywords using the probability formula, and judge whether the acquired data belongs to enterprise data, machine and sensor data, or social data based on the calculation result;

[0018] 2) For single-path propagation, determine the propagation path of the corresponding type of data according to the judgment result in 1). For multi-path propagation, predict the data propagation path according to the data transmission volume. Single-path propagation means that the data can reach the target position through only one channel during the propagation process, and multi-path propagation means that the data can reach the target position through multiple channels during the propagation process.

[0019] Furthermore, the specific method for the data processing unit to predict the triggering probability and mutation probability of the marked data is as follows:

[0020] (1) Construct a mathematical model W i , and predict the triggering probability of the marked data. The specific mathematical model W is:

[0021]

[0022] where i = 1, 2, …, represents the number corresponding to the marked data, k i represents the number of programs or storage media corresponding to the triggering of the i-th marked data, x represents the total number of programs corresponding to the triggering of the marked data, and y represents the total number of storage media corresponding to the triggering of the marked data;

[0023] (2) Based on the prediction result in (1), combine the propagation path of the marked data to predict the mutation probability of the marked data. The specific prediction formula D i is:

[0024]

[0025] where g ≤ i, j = 1, 2, …, n represents the transmission channel number corresponding to the corresponding marked data, and l gj represents the mutation probability generated when the j-th transmission channel of the i-th marked data intersects with the transmission channel of the g-th marked data.

[0026] Furthermore, the computer network diagnosis module includes a network operation condition description unit and a network diagnosis unit;

[0027] The network operation condition description unit describes the network operation condition according to the network load information and network traffic information, and transmits the described network operation condition to the network diagnosis unit;

[0028] The network diagnosis unit receives the network operation condition transmitted by the network operation condition description unit, compares the state corresponding to the described network operation anomaly with the actual network operation state, diagnoses the network state, and transmits the diagnosis result to the intelligent analysis module.

[0029] Further, the network operation condition description unit describes the network operation condition according to network load information and network traffic information. The specific method is as follows:

[0030] Predict the network overload position according to the network load information. The specific prediction formula Q is:

[0031]

[0032] where q j represents the network load value corresponding to the j-th transmission channel of the i-th marked data, represents the network load threshold corresponding to the j-th transmission channel of the i-th marked data, represents determining whether the network load corresponding to the j-th transmission channel of the i-th marked data exceeds the threshold. When , it means that the corresponding network load exceeds the threshold. When , it means that the corresponding network load does not exceed the threshold. When Q ij =e, it means that the corresponding network load does not exceed the threshold. When Q ij <e, it means that the corresponding network load exceeds the threshold;

[0033] Mark the position where the network load exceeds the threshold;

[0034] Based on the marked position where the network load exceeds the threshold, obtain the network traffic of the corresponding position data. If the network traffic consumed by the data is greater than q j *δ, it is determined that the network operation at this position is abnormal. If the network traffic consumed by the data is equal to or less than q j *δ, it is initially determined that the network operation at this position is normal, where δ represents the network traffic consumed by a network load value.

[0035] Further, the intelligent analysis module includes a network anomaly cause analysis unit and a marked data anomaly cause analysis unit;

[0036] The network anomaly cause analysis unit receives the diagnosis result transmitted by the network diagnosis unit, analyzes the cause of network anomaly based on the diagnosis result, and transmits the analysis result to the computer network repair module;

[0037] The marked data anomaly cause analysis unit receives the processed data transmitted by the data processing unit, analyzes the cause of marked data anomaly based on the received content, and transmits the analysis result to the big data management module.

[0038] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0039] 1. The present invention marks the data whose actual propagation path does not match the expected propagation path, and determines the propagation path of the marked data for multipath transmission based on the data increment, which facilitates determining the specific fault point when the later big data management platform or computer network fails. By managing the data or network before and after the fault point, the normal operation of the big data management platform can be ensured, and the data management effect of the platform is further improved.

[0040] 2. The present invention calculates the trigger probability and mutation probability of the marked data, predicts the probability of the marked data being infected with a virus based on the calculation results, and intelligently identifies the location of the virus based on the prediction results, ensuring that the marked data is processed before the virus infects other data, thereby improving the virus interception ability.

[0041] 3. The present invention describes the network operation situation according to the network load information and network traffic information, determines the network abnormal location in combination with the actual network operation situation, ensures that the network operation state can be understood in time, avoids the change of the data storage location caused by abnormal network operation, is conducive to the effective management of big data, and the network load information and network traffic information are easy to obtain, further improving the processing efficiency of big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0043] Figure 1 is a schematic diagram of the working principle structure of a big data management platform for intelligent analysis of a computer network according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0044] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] Please refer to Figure 1 , the present invention provides a technical solution: a big data management platform for intelligent analysis of a computer network, the big data management platform includes a data acquisition and processing module, a computer network status acquisition module, a computer network diagnosis module, an intelligent analysis module, a computer network repair module, and a big data management module;

[0046] The data acquisition and processing module is used to collect and process the data input into the big data management platform, and transmit the collected and processed data to the intelligent analysis module;

[0047] The data acquisition and processing module includes a data acquisition unit, a data screening unit, and a data processing unit;

[0048] The data acquisition unit collects the data input into the big data management platform, the data input time, and the expected data propagation path, and transmits the collected information to the data processing unit;

[0049] The data screening unit receives the collected information transmitted by the data acquisition unit, judges the type to which the collected data belongs, predicts the propagation path of the collected data of the corresponding type according to the judgment result, matches the predicted propagation path with the expected propagation path of the corresponding data, marks the collected data that fails to match, and transmits the marked collected data to the data processing unit;

[0050] The specific method for the data screening unit to predict the propagation path of the collected data of the corresponding type is as follows:

[0051] 1) Obtain the keywords in the collected data, calculate the proportion of various keywords using the probability formula, and judge whether the collected data belongs to enterprise data, machine and sensor data, or social data based on the calculation results;

[0052] 2) For single-path propagation, determine the propagation path of the corresponding type of data according to the judgment result in 1). For multi-path propagation, predict the data propagation path according to the data transmission volume. Single-path propagation means that the data can reach the target location through only one channel during the propagation process, and multi-path propagation means that the data can reach the target location through multiple channels during the propagation process;

[0053] The data processing unit receives the marked data transmitted by the data screening unit, predicts the trigger probability of the collected data of the corresponding type according to the received data, and predicts the mutation probability of the collected data, and transmits the prediction results to the intelligent analysis module;

[0054] The specific method for the data processing unit to predict the trigger probability and mutation probability of the marked data is as follows:

[0055] (1) Construct a mathematical model W i , and predict the trigger probability of the marked data. The specific mathematical model W is:

[0056]

[0057] where i = 1, 2,..., represents the number corresponding to the marked data, k iIndicates the number of programs or storage media corresponding to the triggering of the i-th marked data, x represents the total number of programs corresponding to the triggering of the marked data, and y represents the total number of storage media corresponding to the triggering of the marked data;

[0058] (2) Based on the prediction result in (1), combined with the propagation path of the marked data, predict the mutation probability of the marked data. The specific prediction formula D i is:

[0059]

[0060] where g ≤ i, j = 1, 2, …, n represents the transmission channel number corresponding to the corresponding marked data, and l gj represents the mutation probability generated when the j-th transmission channel of the i-th marked data intersects with the transmission channel of the g-th marked data;

[0061] The computer network status acquisition module is used to acquire the load information, network traffic information, and network status information during the operation of the computer network, and transmit the acquired information to the computer network diagnosis module;

[0062] The computer network diagnosis module is used to receive the acquisition information transmitted by the computer network status acquisition module, diagnose the computer network according to the received information, and transmit the diagnosis result to the intelligent analysis module; The computer network diagnosis module includes a network operation situation description unit and a network diagnosis unit;

[0063] The network operation situation description unit describes the network operation situation according to the network load information and network traffic information, and transmits the described network operation situation to the network diagnosis unit;

[0064] The network operation situation description unit describes the network operation situation according to the network load information and network traffic information. The specific method is:

[0065] Predict the network overload location according to the network load information. The specific prediction formula Q is:

[0066]

[0067] where q j represents the network load value corresponding to the j-th transmission channel of the i-th marked data, represents the network load threshold corresponding to the j-th transmission channel of the i-th marked data, represents determining whether the network load corresponding to the j-th transmission channel of the i-th marked data exceeds the threshold. When it means that the corresponding network load exceeds the threshold. When 0, it means that the corresponding network load does not exceed the threshold. When Qij When it is equal to e, it indicates that the corresponding network load does not exceed the threshold. When Q ij When it is less than e, it indicates that the corresponding network load exceeds the threshold;

[0068] Mark the positions where the network load exceeds the threshold;

[0069] Based on the marked positions where the network load exceeds the threshold, obtain the network traffic of the corresponding position data. If the network traffic consumed by the data is greater than q j *δ, it is determined that the network operation at this position is abnormal. If the network traffic consumed by the data is equal to or less than q j *δ, it is initially determined that the network operation at this position is normal, where δ represents the network traffic consumed by a network load value;

[0070] The network diagnosis unit receives the network operation conditions transmitted by the network operation condition description unit, compares the corresponding state when the described network operation is abnormal with the actual network operation state, diagnoses the network state, and transmits the diagnosis result to the intelligent analysis module;

[0071] The intelligent analysis module is used to receive the data information transmitted by the data acquisition and processing module and the diagnosis result transmitted by the computer network diagnosis module, perform intelligent analysis on the computer network based on the received content, and transmit the analysis results to the computer network repair module and the big data management module respectively; The intelligent analysis module includes a network anomaly cause analysis unit and a marked data anomaly cause analysis unit;

[0072] The network anomaly cause analysis unit receives the diagnosis result transmitted by the network diagnosis unit, analyzes the cause of the network anomaly based on the diagnosis result, and transmits the analysis result to the computer network repair module;

[0073] The marked data anomaly cause analysis unit receives the processed data transmitted by the data processing unit, analyzes the cause of the marked data anomaly based on the received content, and transmits the analysis result to the big data management module;

[0074] The computer network repair module is used to receive the analysis result transmitted by the intelligent analysis module and repair the computer network based on the analysis result;

[0075] The big data management module is used to receive the analysis result transmitted by the intelligent analysis module and effectively manage the big data according to the analysis result.

[0076] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0077] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A big data management platform for intelligent analysis of computer networks, characterized in that: the big data management platform includes a data acquisition and processing module, a computer network status acquisition module, a computer network diagnosis module, an intelligent analysis module, a computer network repair module, and a big data management module; the data acquisition and processing module is used to acquire and process the data input into the big data management platform, and transmit the acquired and processed data to the intelligent analysis module; the data acquisition and processing module includes a data acquisition unit, a data screening unit, and a data processing unit; the data acquisition unit acquires the data input into the big data management platform, the data input time, and the expected data propagation path, and transmits the acquired information to the data processing unit; the data screening unit receives the acquisition information transmitted by the data acquisition unit, judges the type of the acquired data, predicts the propagation path of the acquired data of the corresponding type according to the judgment result, matches the predicted propagation path with the expected propagation path of the corresponding data, marks the acquired data with unsuccessful matching, and transmits the marked acquired data to the data processing unit; the specific method for the data screening unit to predict the propagation path of the acquired data of the corresponding type is: 1) Obtain the keywords in the acquired data, calculate the proportion of various keywords using the probability formula, and judge whether the acquired data belongs to enterprise data, machine and sensor data, or social data based on the calculation result; 2) For single-path propagation, determine the propagation path of the corresponding type of data according to the judgment result in 1), and for multi-path propagation, predict the data propagation path according to the data transmission volume. Single-path propagation means that the data can reach the target position through only one channel during the propagation process, and multi-path propagation means that the data can reach the target position through multiple channels during the propagation process; the data processing unit receives the marked data transmitted by the data screening unit, predicts the trigger probability of the acquired data of the corresponding type according to the received data, and predicts the mutation probability of the acquired data, and transmits the prediction results to the intelligent analysis module; the specific method for the data processing unit to predict the trigger probability and mutation probability of the marked data is: (1)Construct a mathematical model W i , and predict the triggering probability of the labeled data. The specific mathematical model W is as follows: where \(i = 1, 2,\cdots\) represents the number corresponding to the marked data, \(k\) i represents the number of programs or storage media corresponding to the triggering of the \(i\)-th marked data, \(x\) represents the total number of programs corresponding to the triggering of the marked data, and \(y\) represents the total number of storage media corresponding to the triggering of the marked data; (2) Based on the prediction result in (1), the mutation probability of the labeled data is predicted by combining the propagation path of the labeled data. The specific prediction formula D i is as follows: where g ≤ i, j = 1, 2, …, n represents the transmission channel number corresponding to the corresponding marked data, I gj represents the mutation probability generated when the j-th transmission channel of the i-th marked data intersects with the transmission channel of the g-th marked data; the computer network status acquisition module is used to acquire the load information, network traffic information, and network status information during the operation of the computer network, and transmit the acquired information to the computer network diagnosis module; the computer network diagnosis module is used to receive the acquisition information transmitted by the computer network status acquisition module, diagnose the computer network according to the received information, and transmit the diagnosis result to the intelligent analysis module; the intelligent analysis module is used to receive the data information transmitted by the data acquisition and processing module and the diagnosis result transmitted by the computer network diagnosis module, perform intelligent analysis on the computer network based on the received content, and transmit the analysis results to the computer network repair module and the big data management module respectively; the computer network repair module is used to receive the analysis result transmitted by the intelligent analysis module and repair the computer network based on the analysis result; The big data management module is used to receive the analysis results transmitted by the intelligent analysis module and effectively manage the big data according to the analysis results.

Images