Analysis method, system and equipment based on 5GC network element risk operation and medium

Abstract

The invention discloses an analysis method, system and device based on 5GC network element risk operation and a medium, and the method comprises the following steps: collecting work orders of successful cutover, and carrying out the data classification and induction; the collected work orders are screened, and unreasonable cutover work orders are filtered out; key words are extracted according to business rules, data analysis and exploration are carried out on the filtered cutover work orders, and preliminary business classification is formed; performing model training according to a business classification result of data analysis and exploration; performing service diagnosis again according to a model training result, and analyzing the accuracy of model output; carrying out persistent storage on the model which is judged to be qualified through business diagnosis; and continuously collecting new cutover work orders of the user, collecting the satisfaction degree of the user to a model output scheme, and continuously optimizing the model. According to the analysis method based on the 5GC network element risk operation, the cutover scheme is intelligently generated for the user to autonomously select or carry out personalized editing, and the cutover scheme analysis is carried out on the final scheme.

Description

Technical Field

[0001] The present invention relates to the technical field of 5G core network element risk operation control, and specifically to an analysis method, system, equipment and medium based on 5G core network element risk operation. Background Art

[0002] Since the maintenance of 5G core network equipment, such as version upgrades, disaster recovery switching, hidden danger repair, capacity expansion and other operations, involves a large number of command executions, complex cutover plans need to be formulated before cutover. The accuracy of command review and the reliability of cutover plan formulation usually affect the final 5G network element cutover results due to factors such as personal experience and focus.

[0003] For example, users will write instruction codes and create cutover steps based on their personal experience. However, the impact of unreasonable cutover steps and the execution of certain high-risk instructions on 5G performance indicators may be difficult for users to predict. Users tend to pay more attention to the thresholds of key performance indicators and ignore the abnormal performance of some secondary indicators, which may result in some secondary functions of the equipment being restricted or unavailable, causing small-scale production accidents or unpredictable failures. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an analysis method, system, device and medium based on 5GC network element risk operation, which intelligently generates a cutover plan for users to select or edit individually, and analyzes the final plan.

[0005] The present invention is achieved through the following technical solutions:

[0006] The first aspect of the present application provides a method for analyzing risk operations of 5GC network elements, comprising the following steps:

[0007] Collect the work orders of successful cutovers in the past and classify and summarize the data;

[0008] Screen the collected work orders and filter out unreasonable cutover work orders;

[0009] Extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification;

[0010] Conduct model training based on the business classification results of data analysis and exploration;

[0011] Conduct business diagnosis again based on the model training results and analyze the accuracy of the model output;

[0012] For models that are qualified after business diagnosis, they are persistently stored;

[0013] Continue to collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model.

[0014] To optimize the above technical solutions, the specific measures taken also include:

[0015] The work orders of successful cutover in the past are collected and data is classified and summarized, including four input feature classifications: user province, cutover 5G network element type, cutover scenario and cutover content.

[0016] Furthermore, the extracting of keywords according to business rules and preliminary business classification include:

[0017] The following basic information of the cutover plan is generated through the combination of the cutover scenario, cutover content, and user province input features: operator, reviewer, applicant unit approver, cutover impact scope, and cutover estimated duration;

[0018] The following steps of the cutover solution are generated by combining the cutover scenario, cutover content, and cutover 5G network element type input features:

[0019] Generate relevant steps for pre-operation preparation, operation implementation, operation reversal, post-operation inspection, and next-day on-duty inspection.

[0020] Furthermore, the business classification results explored through data analysis are used to perform model training, so as to automatically plan the optimal path and timing according to preset cutover goals and constraints; wherein the model training is performed using a model combining deep learning and decision tree + random forest algorithm.

[0021] Furthermore, a cutover plan is intelligently generated by the model that is qualified after business diagnosis, and the cutover plan is used to predict the result of the user's current cutover, and the cutover time and prediction result are displayed to the user.

[0022] Furthermore, the cutover plan is provided for users to select or edit individually, and the performance trends of various indicators are predicted, so that users can evaluate the impact of the cutover accordingly.

[0023] Furthermore, we analyze the cutover plan for the customer's final plan, and use a dialogue assistant to prompt the performance indicators that may be affected by the cutover and the types of alarms that may be generated during the cutover process.

[0024] A second aspect of the present application provides a 5GC network element risk operation analysis system, including:

[0025] The data collection module is used to collect the work orders of successful cutovers in the past and classify and summarize the data;

[0026] The data cleaning module is used to screen the collected work orders and filter out unreasonable cutover work orders;

[0027] The data analysis and exploration module is used to extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification;

[0028] Model training module, used to perform model training based on the business classification results of data analysis and exploration;

[0029] The model tuning module is used to conduct business diagnosis again based on the model training results and analyze the accuracy of the model output;

[0030] The model persistence storage module is used to persistently store models that have been determined to be qualified through business diagnosis;

[0031] The model dynamic update module is used to continuously collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model.

[0032] The 5GC network element risk operation analysis system is used to execute the 5GC network element risk operation analysis method provided in the first aspect of the present application.

[0033] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for analyzing risk operations of 5GC network elements as provided in the first aspect of the present application.

[0034] A fourth aspect of the present application provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the analysis method provided in the first aspect of the present application.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention automatically generates a cutover plan through intelligent means, which significantly improves work efficiency. After adopting the model constructed by the method of the present invention, the time for formulating the cutover plan is greatly shortened, and the plan defects caused by human errors are reduced. The method of the present invention can analyze the historical cutover status and resource distribution of the network to ensure that the plan produced by the algorithm has the minimum impact on network stability and service continuity during the cutover process.

[0037] This application models the existing network and then automatically plans the optimal path and timing according to preset cutover goals and constraints, such as cutover scenarios, network element types, etc. This process not only improves the accuracy of cutover, but also reduces the dependence on professional technicians, allowing non-professionals to quickly master cutover operations.

[0038] In order to solve the reliability problem of complex cutover plan formulation, the present invention constructs a reliable and comprehensive 5G network element cutover plan generation algorithm model based on historical excellent cutover orders by using deep learning and decision tree + random forest algorithm according to the characteristics of different provinces, 5G network element types, cutover scenarios, cutover contents, etc., and intelligently generates cutover plans for users to choose or edit them individually. The cutover plan analysis is performed on the customer's final plan, and the performance indicators that may be affected by this cutover and the types of alarms that may be generated during the cutover process are prompted in the form of a dialogue assistant.

[0039] The present invention utilizes deep learning and decision tree + random forest algorithm based on the analysis algorithm of 5GC network element risk operation, combined with the characteristics of 5G network, to achieve efficient processing of complex network element cutover; specifically, it adopts heuristic search strategy and multi-objective optimization method, and can produce a cutover plan that minimizes cutover cost and time consumption while ensuring service quality.

[0040] In summary, the analysis algorithm based on 5GC network element risk operation shows significant advantages in improving cutover efficiency, ensuring the safe and stable operation of the network, and reducing labor costs. Its application not only provides strong technical support for the maintenance and management of 5G networks, but also lays a solid foundation for the development of future communication technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 : A structural diagram of the analysis method of 5GC network element risk operations of the present invention.

[0042] Figure 2 : Multi-classification ROC curve diagram in the random forest algorithm adopted by the present invention.

[0043] Figure 3 : The random forest algorithm adopted by the present invention generates a cutover interruption duration graph. DETAILED DESCRIPTION

[0044] The above contents of the present invention are further described in detail below in the form of embodiments, but this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the above contents of the present invention belong to the scope of the present invention.

[0045] Some of the terms in the present invention are explained as follows:

[0046] 5GC (5th Generation Core Network) refers to the core network of the fifth generation mobile communication system. It is a new mobile network architecture that can provide faster, safer and more reliable mobile communication services.

[0047] Decision Tree is a decision analysis method that uses a decision tree to construct a decision tree based on the known probabilities of various situations to obtain the probability that the expected value of the net present value is greater than or equal to zero, evaluate project risks, and determine its feasibility. It is a graphical method that intuitively uses probability analysis.

[0048] The random forest algorithm is an ensemble learning algorithm and belongs to the bagging method in ensemble learning. It predicts the target variable by building multiple decision trees and combining their prediction results.

[0049] In one embodiment of the present invention, a method for analyzing risk operations of 5GC network elements is proposed, and the flow chart is as follows: Figure 1 As shown, the following steps are included:

[0050] Collect the work orders of successful cutovers in the past and classify and summarize the data;

[0051] Screen the collected work orders and filter out unreasonable cutover work orders;

[0052] Extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification;

[0053] Conduct model training based on the business classification results of data analysis and exploration;

[0054] Conduct business diagnosis again based on the model training results. For example, analyze the business elements such as the network element work order cutover time, cutover scenario, cutover content, and risk level output by the model to see whether they match the actual business rules. This will analyze the accuracy of the model output and dynamically update the model library.

[0055] For models that are qualified after business diagnosis, they are persistently stored;

[0056] Continue to collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model.

[0057] The specific measures taken include:

[0058] Collect work orders for successful cutovers in the past and classify and summarize the data, including four input feature classifications: user province, cutover 5G network element type, cutover scenario, and cutover content.

[0059] In some implementations, keywords are extracted according to business rules to perform preliminary business classification, including:

[0060] The following basic information of the cutover plan is generated through the combination of the cutover scenario, cutover content, and user province input features: operator, reviewer, applicant unit approver, cutover impact scope, and cutover estimated duration;

[0061] The following steps of the cutover solution are generated by combining the cutover scenario, cutover content, and cutover 5G network element type input features:

[0062] Generate relevant steps for pre-operation preparation, operation implementation, operation reversal, post-operation inspection, and next-day on-duty inspection.

[0063] In some embodiments, model training is performed based on the business classification results of data analysis exploration to automatically plan the optimal path and timing based on preset cutover goals and constraints; wherein the model training is performed using a model combining deep learning and decision tree + random forest algorithm, historical work order data is extracted for interruption duration distribution statistics, 5 time distribution intervals are set according to the distribution law, and vector features are extracted for text features using a deep learning model, feature encoding and feature engineering are performed on the data, and a balancing algorithm is used to balance the interval samples, and then a decision tree + random forest algorithm is used for model fitting, model parameters are tuned, and model evaluation is performed using accuracy, precision, recall rate, and confusion matrix, and ROC curves and PR curves are used for presentation.

[0064] Figure 2 This is a multi-classification ROC curve diagram of the random forest algorithm used in a specific embodiment of the present invention. A specific example is: using a one-vs-all (OVA) strategy to generate ROC curves for multiple binary classification problems, the micro-average ROC curve in the image (area is 0.99) represents the overall performance of the model in all categories, and the macro-average ROC curve in the image (area is 0.99) represents the average performance of the model in each category. At the same time, a time interval corresponding to each category (such as 0-30 minutes, 30-90 minutes, etc.) is intuitively presented, and the area reaches more than 98%.

[0065] Figure 3 A cutover interruption duration graph is generated for the random forest algorithm used in a specific embodiment of the present invention. The graph shows the comparison between the model prediction results and the actual results. By analyzing the matrix, it is possible to identify which categories the model performs well in and which categories need improvement. For example, the model performs better in the "0-30 minutes" and "more than 210 minutes" categories. In addition, other performance indicators such as precision, recall, and F1 score can also be calculated to more comprehensively evaluate the performance of the model. Based on deep learning and decision tree + random forest algorithm, the present invention combines the characteristics of 5G networks to achieve efficient processing of complex network element cutover.

[0066] In some implementations, a cutover plan is intelligently generated by a model that is qualified after business diagnosis, and the cutover plan is used to predict the result of the user's current cutover, and the cutover time and prediction result are displayed to the user.

[0067] In some implementations, the cutover plan is provided for the user to select or edit on his own, and the performance trends of various indicators are predicted, so that the user can evaluate the impact of the cutover accordingly.

[0068] In some implementations, a cutover solution analysis is performed on the customer's finalized solution, and the performance indicators that may be affected by the cutover and the types of alarms that may be generated during the cutover are prompted in the form of a dialogue assistant.

[0069] In another embodiment, the present invention proposes a 5GC network element risk operation analysis system, including:

[0070] The data collection module is used to collect the work orders of successful cutovers in the past and classify and summarize the data;

[0071] The data cleaning module is used to screen the collected work orders and filter out unreasonable cutover work orders;

[0072] The data analysis and exploration module is used to extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification;

[0073] Model training module, used to perform model training based on the business classification results of data analysis and exploration;

[0074] The model tuning module is used to conduct business diagnosis again based on the model training results and analyze the accuracy of the model output;

[0075] The model persistence storage module is used to persistently store models that have been determined to be qualified through business diagnosis;

[0076] The model dynamic update module is used to continuously collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model.

[0077] The 5GC network element risk operation analysis system is used to execute the 5GC network element risk operation analysis method provided in the first aspect of the present application.

[0078] In another embodiment, the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for analyzing risk operations of 5GC network elements as provided in the first aspect of the present application.

[0079] In another embodiment, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the analysis method provided in the first aspect of the present application.

[0080] In the current rapidly developing communications industry, the application of 5G technology is becoming more and more extensive. With the expansion of network scale and the increase in user demand, how to efficiently and accurately complete network element cutover has become a challenge. The present invention automatically generates a cutover plan through intelligent means, which significantly improves work efficiency.

[0081] The data show that after adopting the model constructed by the method of the present invention, the time for formulating the cutover plan is shortened by about 40%, while the plan defects caused by human errors are reduced.

[0082] In addition, the method of the present invention can analyze the historical cutover status and resource distribution of the network, and ensure that the scheme produced by the algorithm reduces the impact on network stability and service continuity to a minimum during the cutover process.

[0083] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0084] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any technician familiar with the profession, without departing from the scope of the technical solution of the present invention, according to the technical essence of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for analyzing risky operations of 5GC network elements, characterized in that: The following steps are involved: Collect the work orders of successful cutovers in the past and classify and summarize the data; Screen the collected work orders and filter out unreasonable cutover work orders; Extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification; Conduct model training based on the business classification results of data analysis and exploration; Conduct business diagnosis again based on the model training results and analyze the accuracy of the model output; For models that are qualified after business diagnosis, they are persistently stored; Continue to collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model.

2. The method for analyzing risky operations of 5GC network elements according to claim 1, characterized in that: The work orders of successful cutover in the past are collected and data is classified and summarized, including four input feature classifications: user province, cutover 5G network element type, cutover scenario and cutover content.

3. The method for analyzing risky operations of 5GC network elements according to claim 2, characterized in that: The aforementioned extraction of keywords based on business rules and preliminary business classification includes: The following basic information of the cutover plan is generated by combining the cutover scenario, cutover content, and user province input features: Operator, reviewer, applicant unit approver, cutover impact scope, and cutover estimated duration; The following steps of the cutover solution are generated by combining the cutover scenario, cutover content, and cutover 5G network element type input features: Generate relevant steps for pre-operation preparation, operation implementation, operation reversal, post-operation inspection, and next-day on-duty inspection.

4. The method for analyzing risky operations of 5GC network elements according to claim 1, characterized in that: The model training is carried out according to the business classification results explored by data analysis, so as to automatically plan the optimal path and timing according to the preset cutover goals and constraints; wherein the model training is carried out using a model combining deep learning and decision tree + random forest algorithm.

5. The method for analyzing risky operations of 5GC network elements according to claim 1, characterized in that: The cutover plan is intelligently generated by the model that is qualified after business diagnosis. The cutover plan is used to predict the result of the user's current cutover, and the cutover time and prediction result are displayed to the user.

6. The method for analyzing risky operations of 5GC network elements according to claim 5, characterized in that: The cutover plan is provided for users to select or edit individually, and the performance trends of various indicators are predicted, so that users can evaluate the impact of cutover accordingly.

7. The method for analyzing risky operations of 5GC network elements according to claim 5, characterized in that: The cutover solution is analyzed for the customer's final solution, and the user is prompted with a dialogue assistant to identify the performance indicators that may be affected by the cutover and the types of alarms that may be generated during the cutover process.

8. A 5GC network element risk operation analysis system, characterized in that: include: The data collection module is used to collect the work orders of successful cutovers in the past and classify and summarize the data; The data cleaning module is used to screen the collected work orders and filter out unreasonable cutover work orders; The data analysis and exploration module is used to extract keywords based on business rules, conduct data analysis and exploration on the filtered cutover work orders, and form a preliminary business classification; Model training module, used to perform model training based on the business classification results of data analysis and exploration; The model tuning module is used to conduct business diagnosis again based on the model training results and analyze the accuracy of the model output; The model persistence storage module is used to persistently store models that have been determined to be qualified through business diagnosis; The model dynamic update module is used to continuously collect new cutover work orders from users, collect user satisfaction with model output solutions, and continuously optimize the model. The 5GC network element risk operation analysis system is used to execute the 5GC network element risk operation analysis method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for analyzing risk operations of 5GC network elements as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the 5GC network element risk operation analysis method as described in any one of claims 1 to 7.

Images