Network management method based on 5G embedded network management software

Abstract

The invention discloses a network management method based on 5G embedded network management software, and solves the technical problem that part of existing 5G embedded network management software is insufficient in intelligent level. Comprising the following steps: acquiring data by using a Scrapy framework and a Selenium tool; storing data by using an HDFS system; a Spark SQL module is used for preprocessing the data; using a reinforcement learning algorithm to analyze and process the data, and using a supervised learning algorithm to learn and predict the network state according to the data; the network state is predicted based on a supervised learning algorithm, and a fault point is verified and detected; and based on the verification detection result of the fault point, sending out an alarm notification and carrying out fault recovery. In the aspects of network fault prediction and network performance optimization management, a supervised learning algorithm in a machine learning algorithm is introduced to predict network faults, a reinforcement learning algorithm is introduced to adjust network configuration so as to optimize network performance, and compared with the prior art, the intelligent level of network management is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of network management, and specifically relates to a network management method based on 5G embedded network management software. Background Art

[0002] Existing 5G embedded network management software usually has functions such as real-time monitoring, fault management, configuration management, security management, and performance management to support the stable operation and efficient management of 5G networks. The technical defects of some existing 5G embedded network management software are: insufficient intelligence level, that is, although some existing 5G embedded network management software has introduced AI and big data technologies, in actual applications, the integration and application of these technologies are not deep enough, and embedded network management software still has certain limitations in network fault prediction and performance optimization. Summary of the invention

[0003] The technical problem to be solved by the present invention is: to provide a network management method based on 5G embedded network management software to solve the technical problem that some existing 5G embedded network management software has insufficient intelligence level.

[0004] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] A network management method based on 5G embedded network management software comprises the following steps:

[0006] Step 1: Use the Scrapy framework to crawl structured data from static or dynamically generated web pages, and use the Selenium tool to crawl data that requires user interaction from pages that require user login or interaction to access;

[0007] Step 2: Store the structured data captured by the Scrapy framework and the data captured by the Selenium tool that requires user interaction in the HDFS system;

[0008] Step 3: Use the Spark SQL module in the Spark framework to read the data stored in the HDFS system. The SparkSQL module preprocesses the read data.

[0009] Step 4: Implement the reinforcement learning algorithm on the Eclipse RCP framework and implement the supervised learning algorithm using the Mahout tool on the Hadoop framework;

[0010] Step 5: Input the preprocessed data in step 3 into the reinforcement learning algorithm and the supervised learning algorithm. The reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network operation status based on the received data.

[0011] Step 6: The reinforcement learning algorithm generates network performance optimization suggestions based on the analysis and processing results of the data, and automatically adjusts the network configuration;

[0012] Step 7: Based on the prediction of the network operation status by the supervised learning algorithm, the fault point is verified and detected; Step 8: Based on the verification and detection results of the fault point, an alarm notification is issued and fault recovery is performed.

[0013] Furthermore, in step 1, the Scrapy framework crawls data from the management interface, network status dashboard or API interface of the network device to obtain the operation status data, traffic statistics data, report and event log data, and network topology data of the network device.

[0014] Furthermore, in step 1, the Selenium tool simulates the interaction between the user and the web page to obtain data that requires user interaction to access.

[0015] Furthermore, in step 3, Spark SQL cleans, transforms, and aggregates the read data, and saves the processed data.

[0016] Furthermore, in step 5, the MapReduce programming model on the Hadoop framework is used to implement parallelization of the reinforcement learning algorithm and the supervised learning algorithm.

[0017] Furthermore, in step 7, an anomaly detection algorithm is used to verify and detect the fault point.

[0018] Furthermore, in step 8, a fault recovery strategy is used to perform fault recovery.

[0019] Furthermore, in step 5, a dynamic module system on the OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention has a simple structure, a scientific and reasonable design, and is easy to use. In terms of network fault prediction and network performance optimization management, the present invention introduces a supervised learning algorithm in a machine learning algorithm to predict network faults, and introduces a reinforcement learning algorithm in a machine learning algorithm to adjust the network configuration to optimize network performance. The use of a machine learning algorithm can more accurately predict network faults and optimize network performance. Compared with the prior art, the present invention improves the level of intelligence in network management. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore they cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; of course, it can also be a mechanical connection or an electrical connection; in addition, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] like Figure 1 As shown, the present invention provides a network management method based on 5G embedded network management software to solve the technical problem that some existing 5G embedded network management software has insufficient intelligence level.

[0027] The present invention has a simple structure, a scientific and reasonable design, and is easy to use. In terms of network fault prediction and network performance optimization management, the present invention introduces a supervised learning algorithm in a machine learning algorithm to predict network faults, and introduces a reinforcement learning algorithm in a machine learning algorithm to adjust the network configuration to optimize network performance. The use of a machine learning algorithm can more accurately predict network faults and optimize network performance. Compared with the prior art, the present invention improves the level of intelligence in network management.

[0028] The present invention comprises the following steps:

[0029] Step 1: Use the Scrapy framework to crawl structured data from static or dynamically generated web pages, and use the Selenium tool to crawl data that requires user interaction from pages that require user login or interaction to access;

[0030] Step 2: Store the structured data captured by the Scrapy framework and the data that requires user interaction and is captured by the Selenium tool in the HDFS system;

[0031] Step 3: Use the Spark SQL module in the Spark framework to read the data stored in the HDFS system. The SparkSQL module preprocesses the read data.

[0032] Step 4: Implement the reinforcement learning algorithm on the Eclipse RCP framework and implement the supervised learning algorithm using the Mahout tool on the Hadoop framework;

[0033] Step 5: Input the preprocessed data in step 3 into the reinforcement learning algorithm and the supervised learning algorithm. The reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network operation status based on the received data.

[0034] Step 6: The reinforcement learning algorithm generates network performance optimization suggestions based on the analysis and processing results of the data, and automatically adjusts the network configuration;

[0035] Step 7: Based on the prediction of the network operation status by the supervised learning algorithm, verify and detect the fault point;

[0036] Step 8: Based on the verification and detection results of the fault point, an alarm notification is issued and fault recovery is performed.

[0037] In some embodiments, in step 1, the Scrapy framework crawls data from the management interface, network status dashboard or API interface of the network device to obtain the operating status data, traffic statistics data, report and event log data, and network topology data of the network device.

[0038] The operating status data of network devices refers to CPU usage, memory usage, interface status, etc. Traffic statistics refers to inbound / outbound traffic, number of packets, etc. Report and event log data refers to device failures, security events, etc. Network topology data refers to the connection relationship between network devices.

[0039] In some embodiments, in step 1, the Selenium tool simulates the interaction between the user and the web page to obtain data that requires user interaction to access.

[0040] In some embodiments, in step 3, Spark SQL cleans, transforms, and aggregates the read data, and saves the processed data.

[0041] In some embodiments, in step 5, the parallelization of the reinforcement learning algorithm and the supervised learning algorithm is implemented using the MapReduce programming model on the Hadoop framework. The simple principle of MapReduce to implement the parallelization of the reinforcement learning algorithm and the supervised learning algorithm is as follows: MapReduce divides the preprocessed data in step 3 and assigns the divided data to multiple map tasks; MapReduce writes a map function for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the map function generates a series of intermediate key-value pairs from the input key-value pairs; MapReduce implements a custom Partitioner for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the Partitioner is used to ensure that the intermediate key-value pairs from the same algorithm will be sent to the same reducer; MapReduce writes a reduce function for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the reduce function will merge the intermediate key-value pairs from the map stage and generate the final output. MapReduce sets the corresponding class and task number in the job configuration to start a sufficient number of map and reduce tasks for the reinforcement learning algorithm and the supervised learning algorithm.

[0042] In some embodiments, in step 6, the reinforcement learning algorithm automatically adjusts the network configuration based on the generated network performance optimization suggestions to improve network performance. The preprocessed data in step 3 is input into the reinforcement learning algorithm, and the reinforcement learning algorithm obtains network status information based on the input data, takes a tentative action on the network based on the obtained network status information, and obtains the network feedback's evaluation of this action and the new network status; if an action leads to a positive reward for the network, then the trend of generating this action in the future will be strengthened; conversely, the trend of generating this action will be weakened. In the process of repeated interaction with the network, the mapping strategy from network status to tentative action is continuously modified in a learning manner to achieve the purpose of optimizing system performance.

[0043] In some embodiments, in step 7, an anomaly detection algorithm is used to verify the fault point. In supervised learning, the algorithm receives the preprocessed data set from step 3, which contains a series of input examples and their corresponding correct output labels. By using this data set, the algorithm can learn a function to predict the output label corresponding to the new example. That is, the supervised learning algorithm can predict the corresponding state of the network at this moment based on the learned function and the real-time collected network data.

[0044] In some embodiments, in step 8, a failure recovery strategy is used to perform failure recovery.

[0045] In some embodiments, in step 5, a dynamic module system on an OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

[0046] Finally, it should be noted that the above embodiments are only preferred embodiments of the present invention to illustrate the technical solutions of the present invention, rather than limiting them, and certainly not limiting the patent scope of the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention. In other words, any changes or modifications made to the main design concept and spirit of the present invention that have no substantive significance, and the technical problems they solve are still consistent with the present invention, should be included in the protection scope of the present invention. In addition, the direct or indirect application of the technical solutions of the present invention in other related technical fields is also included in the patent protection scope of the present invention.

Claims

1. A network management method based on 5G embedded network management software, characterized in that: The steps include: Step 1: Use the Scrapy framework to crawl structured data from static or dynamically generated web pages, and use the Selenium tool to crawl data that requires user interaction from pages that require user login or interaction to access; Step 2: Store the structured data captured by the Scrapy framework and the data captured by the Selenium tool that requires user interaction in the HDFS system; Step 3: Use the Spark SQL module in the Spark framework to read the data stored in the HDFS system. The Spark SQL module preprocesses the read data. Step 4: Implement the reinforcement learning algorithm on the Eclipse RCP framework and implement the supervised learning algorithm using the Mahout tool on the Hadoop framework; Step 5: Input the preprocessed data in step 3 into the reinforcement learning algorithm and the supervised learning algorithm. The reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network operation status based on the received data. Step 6: The reinforcement learning algorithm generates network performance optimization suggestions based on the analysis and processing results of the data, and automatically adjusts the network configuration; Step 7: Based on the prediction of the network operation status by the supervised learning algorithm, verify and detect the fault point; Step 8: Based on the verification and detection results of the fault point, an alarm notification is issued and fault recovery is performed.

2. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 1, the Scrapy framework crawls data from the management interface, network status dashboard or API interface of the network device to obtain the operation status data, traffic statistics data, report and event log data, and network topology data of the network device.

3. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 1, the Selenium tool simulates user interaction with the web page to obtain data that requires user interaction to access.

4. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 3, Spark SQL cleans, transforms, and aggregates the read data, and saves the processed data.

5. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 5, the MapReduce programming model on the Hadoop framework is used to parallelize the reinforcement learning algorithm and the supervised learning algorithm.

6. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 7, an anomaly detection algorithm is used to verify the fault point.

7. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 8, the failover strategy is used to perform failover.

8. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 5, the dynamic module system on the OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

Images