A network management method based on 5G embedded network management software

Abstract

The application discloses a network management method based on 5G embedded network management software, and solves the technical problem of insufficient intelligent level of some existing 5G embedded network management software. The method comprises the following steps: collecting data by using a Scrapy framework and a Selenium tool; storing data by using an HDFS system; pre-processing data by using a Spark SQL module; analyzing and processing data by using a reinforcement learning algorithm; learning and predicting network states according to data by using a supervised learning algorithm; verifying and detecting fault points based on the prediction of network states by using the supervised learning algorithm; and issuing an alarm notification and recovering from a fault based on the verification and detection result of the fault points. In the aspects of network fault prediction and network performance optimization management, the application introduces a supervised learning algorithm in a machine learning algorithm to predict network faults, and introduces a reinforcement learning algorithm to adjust network configuration, so as to optimize network performance. Compared with the prior art, the application improves the intelligent level of network management.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network management, and particularly relates to a network management method based on 5G embedded network management software. BACKGROUND

[0002] The existing 5G embedded network management software usually has functions of real-time monitoring, fault management, configuration management, security management, performance management and the like to support stable operation and efficient management of the 5G network. The technical defects of the existing part of 5G embedded network management software are as follows: the intelligent level is insufficient, that is, although the AI and big data technology are introduced into the existing part of 5G embedded network management software, the fusion and application of these technologies are not deep enough in actual application, and the embedded network management software still has certain limitations in network fault prediction and performance optimization. SUMMARY

[0003] The application aims to solve the technical problem of providing a network management method based on 5G embedded network management software to solve the technical problem of insufficient intelligent level of the existing part of 5G embedded network management software.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the application is as follows:

[0005] A network management method based on 5G embedded network management software comprises the following steps:

[0006] Step 1: using the Scrapy framework to grab structured data from static or dynamically generated web pages, and using the Selenium tool to grab data that needs user interaction to access from pages that need user login or interaction to access;

[0007] Step 2: storing the structured data grabbed by the Scrapy framework and the data that needs user interaction to access grabbed by the Selenium tool in the HDFS system;

[0008] Step 3: using the Spark SQL module in the Spark framework to read the data stored in the HDFS system, and the Spark SQL module pre-processes the read data;

[0009] Step 4: implementing the reinforcement learning algorithm on the Eclipse RCP framework, and implementing the supervised learning algorithm on the Hadoop framework using the Mahout tool;

[0010] Step 5: inputting the pre-processed data in step 3 into the reinforcement learning algorithm and the supervised learning algorithm, the reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network operation state according to the received data;

[0011] Step 6: The reinforcement learning algorithm generates network performance optimization suggestions based on the analysis results of the data and automatically configures and adjusts the network;

[0012] Step 7: Based on the prediction of the network operation state by the supervised learning algorithm, the fault point is verified and detected.

[0013] Further, in step 1, the Scrapy framework crawls data from the management interface of the network device, the network status dashboard or the API interface to obtain the running state data, traffic statistics data, log data, and network topology data of the network device.

[0014] Further, in step 1, the Selenium tool simulates user interaction with the webpage to obtain data that requires user interaction to access.

[0015] Further, in step 3, the Spark SQL cleans, transforms and aggregates the read data, and saves the processed data.

[0016] Further, in step 5, the reinforcement learning algorithm and the supervised learning algorithm are parallelized using the MapReduce programming model on the Hadoop framework.

[0017] Further, in step 7, the anomaly detection algorithm is used to verify and detect the fault point.

[0018] Further, in step 8, the fault recovery strategy is used for fault recovery.

[0019] Further, in step 5, the dynamic module system on the OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

[0020] Compared with the prior art, the present application has the following beneficial effects:

[0021] The present application has the following beneficial effects: BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 The present application has the following beneficial effects: DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have a specific orientation or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0025] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; of course, they may also refer to mechanical connections or electrical connections; in addition, they may refer to direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0026] like Figure 1 As shown, the present invention provides a network management method based on 5G embedded network management software to solve the technical problem that some existing 5G embedded network management software has insufficient intelligence level.

[0027] The present invention has a simple structure, a scientific and reasonable design, and is easy to use. In terms of network fault prediction and network performance optimization management, the present invention introduces a supervised learning algorithm in a machine learning algorithm to predict network faults, and introduces a reinforcement learning algorithm in a machine learning algorithm to adjust the network configuration to optimize network performance. The use of a machine learning algorithm can more accurately predict network faults and optimize network performance. Compared with the existing technology, the present invention improves the intelligent level of network management.

[0028] The present invention comprises the steps of:

[0029] Step 1: Use the Scrapy framework to scrape structured data from static or dynamically generated web pages, and use the Selenium tool to scrape data that requires user interaction to access from pages that require user login or interaction to access;

[0030] Step 2: Store the structured data scraped by the Scrapy framework and the data that requires user interaction to access scraped by the Selenium tool in the HDFS system;

[0031] Step 3: Use the Spark SQL module in the Spark framework to read the data stored in the HDFS system, and the SparkSQL module to preprocess the read data;

[0032] Step 4: Implement reinforcement learning algorithms on the Eclipse RCP framework and supervised learning algorithms on the Hadoop framework using the Mahout tool;

[0033] Step 5: Input the preprocessed data in step 3 into the reinforcement learning algorithm and the supervised learning algorithm, and the reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network running state based on the received data;

[0034] Step 6: The reinforcement learning algorithm generates network performance optimization suggestions based on the analysis and processing results of the data, and automatically configures and adjusts the network;

[0035] Step 7: Based on the prediction of the network running state by the supervised learning algorithm, verify and detect the fault point;

[0036] Step 8: Based on the verification and detection results of the fault point, issue an alarm notification and perform fault recovery.

[0037] In some embodiments, in step 1, the Scrapy framework scrapes data from the management interface of the network device, the network status dashboard or the API interface, obtains the running state data of the network device, traffic statistics data, log and event data, and network topology data.

[0038] The running state data of the network device refers to CPU usage, memory usage, interface status, etc. The traffic statistics data refers to inbound / outbound traffic, packet quantity, etc. The log and event data refers to device faults, security events, etc. The network topology data refers to the connection relationship between network devices.

[0039] In some embodiments, in step 1, the Selenium tool simulates user interaction with web pages to obtain data that requires user interaction to access.

[0040] In some embodiments, in step 3, Spark SQL cleans, transforms and aggregates the read data and saves the processed data.

[0041] In some embodiments, in step 5, the reinforcement learning algorithm and the supervised learning algorithm are parallelized using the MapReduce programming model on the Hadoop framework. The simple principle of parallelizing the reinforcement learning algorithm and the supervised learning algorithm by MapReduce is that MapReduce divides the preprocessed data in step 3 and distributes the divided data to multiple map tasks; MapReduce writes a map function for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the map function generates a series of intermediate key-value pairs from the input key-value pairs; MapReduce implements a custom Partitioner for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the Partitioner is used to ensure that the intermediate key-value pairs from the same algorithm are sent to the same reducer; MapReduce writes a reduce function for the reinforcement learning algorithm and the supervised learning algorithm respectively, and the reduce function combines the intermediate key-value pairs from the map phase and generates the final output. MapReduce sets the corresponding classes and the number of tasks in the job configuration, so as to start a sufficient number of map and reduce tasks for the reinforcement learning algorithm and the supervised learning algorithm.

[0042] In some embodiments, in step 6, the reinforcement learning algorithm automatically configures and adjusts the network based on the generated network performance optimization suggestions to improve the network performance. The preprocessed data in step 3 is input into the reinforcement learning algorithm, the reinforcement learning algorithm obtains network state information according to the input data, takes an exploratory behavior according to the obtained network state information, obtains the evaluation of the action from the network feedback and the new network state; if a certain action leads to a positive reward, the trend of producing this action will be strengthened; otherwise, the trend of producing this action will be weakened. In the repeated interaction with the network, the mapping strategy from the network state to the exploratory action is continuously modified in a learning manner to achieve the purpose of optimizing system performance.

[0043] In some embodiments, in step 7, the anomaly detection algorithm is used to verify and detect the fault point. In supervised learning, the algorithm receives a data set preprocessed in step 3, which contains a series of input examples and their corresponding correct output labels. By using this data set, the algorithm can learn a function to predict the output label corresponding to a new example. That is, the supervised learning algorithm can predict the corresponding state of the network according to the real-time collected network data according to the learned function.

[0044] In some embodiments, in step 8, failure recovery is performed using a failure recovery policy.

[0045] In some embodiments, in step 5, a dynamic module system on the OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

[0046] Finally, it should be noted that: the above embodiments are only the preferred embodiments of the present application for describing the technical solutions of the present application, not limiting them, of course, nor limit the patent scope of the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application; that is to say, any modification or embellishment without substantial meaning in the main design idea and spirit of the present application, the technical problems solved are still consistent with the present application, and should be included in the protection scope of the present application; in addition, the technical solutions of the present application are directly or indirectly applied to other related technical fields, which are also included in the patent protection scope of the present application.

Claims

1. A network management method based on 5G embedded network management software, characterized in that: The steps include: Step 1: Use the Scrapy framework to crawl structured data from static or dynamically generated web pages, and use the Selenium tool to crawl data that requires user interaction from pages that require user login or interaction. Step 2: Store the structured data captured by the Scrapy framework and the data that requires user interaction and is captured by the Selenium tool in the HDFS system; Step 3: Use the Spark SQL module within the Spark framework to read the data stored in the HDFS system. The Spark SQL module preprocesses the read data. Step 4: Implement the reinforcement learning algorithm on the Eclipse RCP framework and the supervised learning algorithm using the Mahout tool on the Hadoop framework; Step 5: Use the reinforcement learning algorithm and the supervised learning algorithm to process the data pre-processed in step 3. The reinforcement learning algorithm analyzes and processes the received data, and the supervised learning algorithm learns and predicts the network operation status based on the received data. Step 6: The reinforcement learning algorithm generates network performance optimization recommendations based on the data analysis and processing results, and automatically adjusts the network configuration; Step 7: Based on the supervised learning algorithm's prediction of the network's operating status, verify and detect the fault point; Step 8: Based on the verification and detection results of the fault point, an alarm notification is issued and fault recovery is performed; In step 1, the Scrapy framework crawls data from the management interface, network status dashboard or API interface of the network device to obtain the network device's operating status data, traffic statistics, report and event log data and network topology data; report and event log data refers to device failures and security events.

2. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 1, the Selenium tool simulates user interaction with the web page to obtain data that requires user interaction to access.

3. The network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 3, Spark SQL cleans, transforms, and aggregates the read data and saves the processed data.

4. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 5, the MapReduce programming model on the Hadoop framework is used to parallelize the reinforcement learning algorithm and the supervised learning algorithm.

5. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 7, an anomaly detection algorithm is used to verify the fault point.

6. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 8, the failure recovery strategy is used to recover from the failure.

7. A network management method based on 5G embedded network management software according to claim 1, characterized in that: In step 5, the dynamic module system on the OSGi framework is used to manage and update the reinforcement learning algorithm and the supervised learning algorithm.

Images