Platform operation method, electronic device, medium and product

Abstract

The application discloses a platform operation and maintenance method, an electronic device, a medium and a product, relates to the technical field of artificial intelligence, and the method comprises the following steps: acquiring current state information of the service platform; extracting a current state feature vector from the current state information by a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained by preset fine-tuning operation and maintenance tasks; inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform. That is, the application realizes the automation and intelligentization of operation decision, and the decision network model can automatically learn and optimize from a large amount of network data, thereby continuously improving the accuracy and effect of the operation and maintenance strategy, reducing the demand for and dependence on manual intervention, improving operation efficiency, and reducing labor cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a platform operation and maintenance method, an electronic device, a medium and a product. BACKGROUND

[0002] Current operation and maintenance solutions for a capability middle platform (a service platform integrating multiple advanced technologies) can mainly include a traditional operation and maintenance solution, an automated operation and maintenance solution, a DevOps operation and maintenance solution, and a containerized operation and maintenance solution. However, in the above existing operation and maintenance methods, human intervention is mostly relied on in the application process, such as manual operation and management, or maintenance and updating of automated tools. Therefore, the current existing operation and maintenance methods have a high dependence on human labor.

[0003] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a platform operation and maintenance method, an electronic device, a medium and a product, which aims to solve the technical problem of high dependence on human labor in the current existing operation and maintenance methods.

[0005] To achieve the above purpose, the present application provides a platform operation and maintenance method applied to a service platform, which comprises the following steps:

[0006] obtaining current state information of the service platform;

[0007] extracting a current state feature vector from the current state information by a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained by preset fine-tuning of a preset fine-tuning operation and maintenance task;

[0008] inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform.

[0009] Optionally, before the step of extracting the current state feature vector from the current state information by the preset BERT pre-training model, the method comprises:

[0010] obtaining historical network traffic data of the service platform, wherein the source of the historical network traffic data includes at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log;

[0011] After the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, feature extraction is performed on the labeled historical network traffic data to obtain a training feature sample set;

[0012] Fine-tune the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model.

[0013] Optionally, the preset fine-tuning operation and maintenance task includes a traffic pattern recognition task, a network fault identification task, and a resource occupation prediction task, the label type of the labeling includes a mode label corresponding to the traffic pattern recognition task, a fault label corresponding to the network fault identification task, and a resource occupation label corresponding to the resource occupation prediction task, and the step of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model includes:

[0014] Building a traffic pattern recognition model based on the initial BERT pre-training model;

[0015] Inputting the training feature sample in the training feature sample set into the traffic pattern recognition model to obtain a traffic pattern recognition result;

[0016] Iteratively training the traffic pattern recognition model based on the difference between the traffic pattern recognition result and the mode label corresponding to the training feature sample, and obtaining a first fine-tuned BERT pre-training model after the training of the traffic pattern recognition model is completed, which is fine-tuned on the initial BERT pre-training model;

[0017] Building a network fault identification model based on the first fine-tuned BERT pre-training model;

[0018] Inputting the training feature sample in the training feature sample set into the network fault identification model to obtain a network fault identification result;

[0019] Iteratively training the network fault identification model based on the difference between the network fault identification result and the fault label corresponding to the training feature sample, and obtaining a second fine-tuned BERT pre-training model after the training of the network fault identification model is completed, which is fine-tuned on the first fine-tuned BERT pre-training model;

[0020] Building a resource occupation prediction model based on the second fine-tuned BERT pre-training model;

[0021] Inputting the training feature sample in the training feature sample set into the resource occupation prediction model to obtain a resource occupation prediction result;

[0022] Iteratively training the resource occupation prediction model based on the difference between the resource occupation prediction result and the resource occupation label corresponding to the training feature sample, and obtaining a preset BERT pre-training model after the training of the resource occupation prediction model is completed, which is fine-tuned on the second fine-tuned BERT pre-training model.

[0023] Optionally, the current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting a current state feature vector from the current state information by using a preset BERT pre-training model includes:

[0024] extracting a traffic feature vector from the current network traffic data by using the preset BERT pre-training model;

[0025] reconstructing the system state data based on a dimension feature of the traffic feature vector to obtain an array conforming to a last dimension feature of the traffic feature vector;

[0026] combining the array with the last dimension feature of the traffic feature vector to obtain the current state feature vector.

[0027] Optionally, before the step of inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform, the method includes:

[0028] constructing a priority replay buffer based on an initial neural network decision model;

[0029] iteratively training the initial neural network decision model based on experience samples stored in the priority replay buffer to obtain the preset neural network decision model.

[0030] Optionally, the step of constructing the priority replay buffer based on the initial neural network decision model includes:

[0031] obtaining historical real-time state information of the service platform in a historical period;

[0032] extracting a historical real-time state feature vector from the historical real-time state information by using the preset BERT pre-training model;

[0033] inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy;

[0034] obtaining an execution reward of the historical real-time operation and maintenance strategy and a new historical real-time state feature vector after the service platform executes the historical real-time operation and maintenance strategy, wherein the execution reward is determined by at least one of a change amount of throughput, a change amount of delay, a change amount of resource utilization, and a change amount of error rate of the service platform after the historical real-time operation and maintenance strategy is executed;

[0035] generate an experience sample based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward, and the new historical real-time state feature vector, and construct a priority replay buffer based on the generated experience sample.

[0036] Optionally, the step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy comprises:

[0037] inputting the historical real-time state feature vector into the initial neural network decision model, and predicting, by the initial neural network decision model, a predicted execution reward of each candidate action in a preset action space under an environment corresponding to the historical real-time state feature vector;

[0038] taking, as the historical real-time operation and maintenance strategy, a candidate action with a maximum predicted execution reward among the candidate actions.

[0039] Optionally, the initial neural network decision model comprises a main Q network and a target Q network, model parameters in the main Q network are to be updated to the target Q network according to a preset period, and the step of iteratively training the initial neural network decision model based on the experience samples stored in the priority replay buffer to obtain the preset neural network decision model comprises:

[0040] collecting part of the experience samples from the priority replay buffer to obtain each candidate experience sample;

[0041] for any one of the candidate experience samples, calculating a target Q value of the candidate experience sample by the target Q network;

[0042] calculating a predicted Q value of the candidate experience sample by the main Q network;

[0043] updating model parameters in the main Q network based on a difference between the target Q value and the predicted Q value, and putting the candidate experience sample with the updated priority back into the priority replay buffer after updating the priority of the candidate experience sample based on the difference between the target Q value and the predicted Q value;

[0044] after the main Q network in the initial neural network decision model meets a preset model training condition, taking the trained initial neural network decision model as the preset neural network decision model.

[0045] In addition, to achieve the above object, the present application also proposes an electronic device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the platform operation and maintenance method as described above.

[0046] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program realizes the steps of the platform operation and maintenance method when executed by a processor.

[0047] In addition, to achieve the above object, the present application also provides a computer program product, which comprises a computer program, and the computer program realizes the steps of the platform operation and maintenance method when executed by a processor.

[0048] The one or more technical solutions provided by the present application have at least the following technical effects:

[0049] In the embodiment of the present application, the current state information of the service platform is acquired, a current state feature vector is extracted from the current state information by a preset BERT pre-training model, the preset BERT pre-training model is obtained by preset fine-tuning operation and maintenance tasks, the current state feature vector is input into a preset neural network decision model, and a target operation and maintenance strategy of the service platform is obtained. That is, the preset BERT pre-training model obtained by preset fine-tuning operation and maintenance tasks can extract the current state feature vector related to operation and maintenance from the current state information of the service platform, so as to ensure the pertinence of the data input into the preset neural network decision model, so that the preset neural network decision model makes a decision based on the current state feature vector, and the target operation and maintenance strategy obtained can effectively improve the performance of the service platform. The present application uses the pre-training and fine-tuning mode in combination with the decision network model to realize the automation and intelligentization of operation and maintenance decision. The decision network model can automatically learn and optimize from a large amount of network data, thereby continuously improving the accuracy and effect of the operation and maintenance strategy, reducing the need for and dependence on manual intervention, improving the operation efficiency, and reducing the labor cost. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor.

[0052] Figure 1 Flowchart of the first embodiment of the platform operation and maintenance method of the present application;

[0053] Figure 2A flowchart of a second embodiment of the platform operation and maintenance method of the present application;

[0054] Figure 3 A flowchart of a third embodiment of the platform operation and maintenance method of the present application;

[0055] Figure 4 A model framework involved in the platform operation and maintenance method of the present application;

[0056] Figure 5 A device structure of a hardware operating environment involved in the platform operation and maintenance method of the present application.

[0057] The object, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0059] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings and specific embodiments.

[0060] The current operation and maintenance scheme for the capability middle platform (a service platform integrating multiple advanced technologies) can mainly include: traditional operation and maintenance scheme, automatic operation and maintenance scheme, DevOps operation and maintenance scheme, and containerized operation and maintenance scheme. The traditional operation and maintenance scheme is based on manual operation and management, usually uses scripts (such as Shell, Python) and basic system management tools (such as cron, rsync), and relies on the experience and skills of system administrators to perform various management and maintenance tasks; the automatic operation and maintenance scheme completes most of the operation and maintenance tasks through automatic tools and scripts, reduces the dependence on manual operation, reduces the risk of human error, and ensures that the same operation gets consistent results in different environments; the DevOps operation and maintenance scheme is a concept and practice that combines software development (Development) and IT operation (Operations), aiming to improve the efficiency and quality of software development and deployment through improved collaboration, automation and continuous delivery; the containerized operation and maintenance scheme is a modern operation and maintenance method that uses container technology to realize the packaging, deployment and management of applications. Containerized operation and maintenance provides higher flexibility, portability and resource utilization.

[0061] It is worth noting that in the traditional operation and maintenance scheme, the operation and maintenance work of the platform depends on a large amount of manual operation; in the automatic operation and maintenance scheme, although the manual operation can be reduced, the manual operation is still dependent on the maintenance and update of the automatic tool; in the DevOps operation and maintenance scheme, manual intervention is still required, and the configuration and integration of the tool chain required for the implementation of the scheme need a higher initial investment and learning cost of the operation and maintenance personnel; in the containerized operation and maintenance scheme, the learning curve of the containerized technology and tool is steep, and is also not friendly to the operation and maintenance personnel who are beginners. Therefore, as described above, the existing operation and maintenance mode has a high dependence on manual operation, and has a high requirement on the technical level of the operation and maintenance personnel.

[0062] The main solution of the embodiment of the application is: obtaining current state information of the service platform; extracting a current state feature vector from the current state information through a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained through preset fine-tuning of a preset fine-tuning operation and maintenance task; inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform.

[0063] That is, the preset BERT pre-training model obtained through the preset fine-tuning of the preset fine-tuning operation and maintenance task can extract the current state feature vector related to operation and maintenance from the current state information of the service platform, thereby ensuring the pertinence of the data input into the preset neural network decision model, so that the preset neural network decision model makes a decision based on the current state feature vector, and the target operation and maintenance strategy obtained can effectively improve the performance of the service platform. The application utilizes the pre-training and fine-tuning mode in combination with the decision network model to realize the automation and intelligentization of operation decision. The decision network model can automatically learn and optimize from a large amount of network data, thereby continuously improving the accuracy and effect of the operation and maintenance strategy, reducing the need for and dependence on manual intervention, improving operation efficiency, and reducing labor cost.

[0064] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a cloud platform, a computer, a mobile phone, etc., or an electronic device capable of realizing the above functions.

[0065] Based on this, the embodiment of the application provides a platform operation and maintenance method, which is described with reference to Figure 1 The flowchart of the first embodiment of the platform operation and maintenance method of the application is shown in FIG. 1.

[0066] In this embodiment, the platform operation and maintenance method applied to a service platform includes steps S10-S30:

[0067] Step S10, obtaining current state information of the service platform;

[0068] It should be noted that the service platform can be a capability platform, which is a smart platform integrating multiple advanced technologies such as Internet of Things, cloud computing, big data, artificial intelligence, 5G and blockchain, and is used to provide different service capabilities. In the embodiment, the platform operation and maintenance scheme is mainly to provide a scheme for maintaining the service platform to ensure the performance of the service platform.

[0069] For example, in the embodiment, the current state information of the service platform can be obtained in various ways. The current state information can include current network traffic data of the service platform and current system state data of the service platform. For example, the current network traffic data can include bandwidth utilization, delay, packet loss rate, etc. of the service platform, and the current system state data can include server load, CPU utilization, memory usage, etc. of the service platform. The current network traffic data can be obtained by network packet capture tools, traffic monitoring tools, and logs of firewalls and routers. The current system state data can be obtained by the built-in state monitoring tools of the service platform.

[0070] In step S20, a current state feature vector is extracted from the current state information by a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained by fine-tuning a preset fine-tuning operation and maintenance task.

[0071] It should be noted that the preset BERT pre-training model is obtained by fine-tuning a preset fine-tuning operation and maintenance task, that is, the BERT pre-training model is retrained for a specific task to obtain the preset BERT pre-training model. The preset fine-tuning operation and maintenance task can be set by a technician according to the operation and maintenance requirements of the platform, for example, the preset fine-tuning operation and maintenance task can be a network fault identification task, a network traffic pattern identification task, etc. It can be understood that since the preset BERT pre-training model is fine-tuned based on the preset fine-tuning operation and maintenance task, the preset BERT pre-training model can sensitively capture features related to platform operation and maintenance in the state information. For example, in the case where the preset fine-tuning operation and maintenance task includes a network fault identification task, the preset BERT pre-training model can extract features related to network faults in the state information, and in the case where the preset fine-tuning operation and maintenance task includes a network traffic pattern identification task, the preset BERT pre-training model can extract features related to network traffic patterns in the state information. Therefore, the current state feature vector is extracted from the current state information of the service platform by the preset BERT pre-training model, which is used for the operation and maintenance decision of the preset neural network decision model in the subsequent steps.

[0072] In step S30, the current state feature vector is input into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform.

[0073] It should be noted that the preset neural network decision model can be a PrioritizedDQN (Prioritized Experience Replay Deep Q-Network) reinforced neural network model, and the preset neural network decision model is trained in advance using experience samples, so as to have an operation and maintenance decision capability.

[0074] Exemplarily, the current state feature vector output by the preset BERT pre-training model is input into the preset neural network decision model, the preset neural network decision model can predict the revenue (i.e., system performance improvement of the service platform) that can be obtained by the service platform when different operation and maintenance strategies are executed in the environment corresponding to the current state feature vector, and the operation and maintenance strategy with the highest revenue is taken as the target operation and maintenance strategy. In actual application, the target operation and maintenance strategy can be used as a basis for selection of the operation and maintenance strategy of the service platform, or the service platform can be directly controlled to execute the target operation and maintenance strategy.

[0075] In the embodiment, the current state information of the service platform is acquired, a current state feature vector is extracted from the current state information by a preset BERT pre-training model, the preset BERT pre-training model is obtained by fine-tuning a preset fine-tuning operation and maintenance task, the current state feature vector is input into a preset neural network decision model, and a target operation and maintenance strategy of the service platform is obtained. That is, the preset BERT pre-training model obtained by fine-tuning the preset fine-tuning operation and maintenance task can extract an operation and maintenance related current state feature vector from the current state information of the service platform, so as to ensure the pertinence of the data input into the preset neural network decision model, the target operation and maintenance strategy obtained by the preset neural network decision model based on the current state feature vector can effectively improve the performance of the service platform. The pre-training and fine-tuning method is combined with the decision network model, so as to realize automation and intelligentization of operation and maintenance decision. The decision network model can automatically learn and optimize from a large amount of network data, so as to continuously improve the accuracy and effect of the operation and maintenance strategy, reduce the need for and dependence on manual intervention, improve operation efficiency, and reduce labor cost.

[0076] In a feasible embodiment, the current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting a current state feature vector from the current state information by the preset BERT pre-training model includes steps S21-S23.

[0077] In step S21, a traffic feature vector is extracted from the current network traffic data by the preset BERT pre-training model.

[0078] Step S22, reconstructing the system state data based on the dimension feature of the traffic feature vector, to obtain an array conforming to the last dimension feature of the traffic feature vector in form;

[0079] Step S23, combining the array with the last dimension feature of the traffic feature vector to obtain the current state feature vector.

[0080] It should be noted that the above-mentioned current state information includes the current network traffic data of the service platform and the current system state data of the service platform, and accordingly can include the bandwidth utilization rate, delay, packet loss rate, etc. of the service platform, and the current system state data can include the server load, CPU utilization rate, memory usage, etc. of the service platform. It is worth noting that since the system state data itself is quantitative data, it can not pass through the preset BERT pre-training model.

[0081] For example, the current network traffic data can be input into the preset BERT pre-training model, and the preset BERT pre-training model can output the traffic feature vector. Then, according to the dimension feature of the traffic feature vector, the system state data is reconstructed to form an array (such as an array with the same data structure) conforming to the last dimension feature of the traffic feature vector in form. Then, the array is combined with the last dimension feature of the traffic feature vector to obtain the current state feature vector. The combination can be adding the last dimension feature of the traffic feature vector to the array, or multiplying the array by a preset coefficient, and then adding the multiplied result to the last dimension feature of the traffic feature vector.

[0082] In addition, it should also be noted that in actual application, the current network traffic data can be divided into network traffic packet data and network traffic index data, wherein the network traffic index data itself also belongs to quantitative data, and therefore can be combined with the traffic feature vector in a manner similar to the above-mentioned system state data to obtain the current state feature vector, which will not be described here.

[0083] Referring to Figure 2 For the first embodiment of the platform operation method based on the present application, the flowchart of the second embodiment is proposed. In this embodiment, the same or similar contents as the above-mentioned embodiments can be referred to the above introduction, and will not be described hereinafter. Before the step of extracting the current state feature vector from the current state information by the preset BERT pre-training model, the method comprises steps S210-S230:

[0084] Step S210, obtaining the historical network traffic data of the service platform, wherein the source of obtaining the historical network traffic data includes at least one of network packet capturing tool, traffic monitoring tool, network security log or network routing log;

[0085] Step S220, after the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, feature extraction is performed on the labeled historical network traffic data to obtain a training feature sample set;

[0086] Step S230, fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model.

[0087] It should be noted that before applying the above preset BERT pre-training model and the preset neural network decision model to make operation and maintenance decisions, the preset BERT pre-training model needs to be obtained through fine-tuning.

[0088] For example, the historical network traffic data of the service platform is obtained, wherein the obtained historical network traffic data can be saved as a.pcap file, and the source of the obtained historical network traffic data includes at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log. In actual application, it is preferred to obtain historical network traffic data using each of the above sources to ensure the richness of the historical network traffic data. In addition, the technician can also select based on the actual situation of the service platform.

[0089] The obtained historical network traffic data needs to be preprocessed. The preprocessing can include deleting duplicate packets and irrelevant packets to reduce noise, such as ARP (Address Resolution Protocol), ICMP (Internet Control Message Protocol), etc.; filtering specific types of traffic according to specific fine-tuning task scenarios: for system traffic data in network traffic data, retaining specific types of traffic related to business scenarios, such as HTTP (HyperText Transfer Protocol) requests, database queries, etc., while using business scenario labels to label the traffic data to form labels related to business scenarios; for network failure traffic in network traffic data, retaining traffic data containing network failure records and related early signs (such as abnormal traffic, delay, packet loss, etc.), while labeling failure and non-failure data; for user behavior data in network traffic data, retaining traffic data related to user online behavior (such as browsing web pages, downloading files, video streaming, etc.) and network resource allocation data (such as bandwidth, connection number, etc.). Use traffic analysis tools to analyze.pcap files (i.e. the above historical network traffic data), extract application layer protocol data, and extract features such as source IP, destination IP, source port, destination port, protocol type, packet size, traffic direction, etc. based on the extracted traffic data. Convert packet features into text representation, represent each packet as a string containing the values of each feature, separated by spaces. Use the Tokenizer of the bert model to convert text data into an input format that the bert model can process, including token IDs (a numerical representation of text data), attention masks (a matrix similar to Token IDs), etc. Tokenizer is a step of feature extraction, and feature vector refers to a complete data representation. After Tokenizer, each part of the packet, after feature extraction, is a dimension of the feature vector, i.e. the feature vector obtained after Tokenizer of each part of a network traffic. The complete feature vector of the network traffic, combined with the labels labeled in the above process, can obtain the training feature samples in the training feature sample set. Then use the training feature sample set to fine-tune the initial bert pre-training model (i.e. the bert pre-training model that has not been fine-tuned), and the above preset bert pre-training model can be obtained.

[0090] In an implementable embodiment, the preset fine-tuning operation and maintenance task includes a traffic pattern identification task, a network fault identification task, and a resource occupation prediction task, the label type of the labeling includes a pattern label corresponding to the traffic pattern identification task, a fault label corresponding to the network fault identification task, and a resource occupation label corresponding to the resource occupation prediction task, and the step of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model includes steps S231-S239.

[0091] Step S231, a traffic pattern identification model is built based on the initial BERT pre-training model.

[0092] Step S232, the training feature samples in the training feature sample set are input into the traffic pattern identification model to obtain traffic pattern identification results.

[0093] Step S233, the traffic pattern identification model is iteratively trained based on the difference between the traffic pattern identification results and the pattern labels corresponding to the training feature samples, and after the traffic pattern identification model is trained, a first fine-tuned BERT pre-training model is obtained by fine-tuning the initial BERT pre-training model.

[0094] Step S234, a network fault identification model is built based on the first fine-tuned BERT pre-training model.

[0095] Step S235, the training feature samples in the training feature sample set are input into the network fault identification model to obtain network fault identification results.

[0096] Step S236, the network fault identification model is iteratively trained based on the difference between the network fault identification results and the fault labels corresponding to the training feature samples, and after the network fault identification model is trained, a second fine-tuned BERT pre-training model is obtained by fine-tuning the first fine-tuned BERT pre-training model.

[0097] Step S237, a resource occupation prediction model is built based on the second fine-tuned BERT pre-training model.

[0098] Step S238, the training feature samples in the training feature sample set are input into the resource occupation prediction model to obtain resource occupation prediction results.

[0099] Step S239, based on the difference between the resource occupation prediction result and the resource occupation label corresponding to the training feature sample, the resource occupation prediction model is iteratively trained, and after the resource occupation prediction model is trained, a preset bert pre-training model after the second fine-tuning of the fine-tuned bert pre-training model is obtained.

[0100] It should be noted that the preset fine-tuning operation and maintenance task includes a traffic pattern recognition task, a network fault identification task, and a resource occupation prediction task. Correspondingly, different labels are labeled for different preset fine-tuning operation and maintenance tasks, that is, the types of labels include pattern labels corresponding to the traffic pattern recognition task, fault labels corresponding to the network fault identification task, and resource occupation labels corresponding to the resource occupation prediction task. The preset bert pre-training model is obtained by fine-tuning for different preset fine-tuning operation and maintenance tasks.

[0101] For example, the initial bert pre-training model refers to a bert pre-training model that has not been fine-tuned. In this embodiment, the initial bert pre-training model can be directly used as a traffic pattern recognition model, and an Adam (Adaptive Moment Estimation, an optimization algorithm for deep learning) optimizer, a cross-entropy loss (Cross-Entropy Loss) as a loss function, and a network traffic pattern recognition task are configured to construct a network traffic pattern recognition task to identify the current network traffic pattern. The traffic pattern recognition model has network traffic feature extraction capability. In the fine-tuning process, the training feature samples in the training feature sample set are input into the traffic pattern recognition model, and the traffic pattern recognition model outputs a traffic pattern recognition result. Based on the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample, the traffic pattern recognition model is iteratively trained, such as including the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample in the above loss function. Until the traffic pattern recognition model is trained, that is, the first fine-tuned bert pre-training model after fine-tuning of the initial bert pre-training model is obtained, that is, the trained traffic pattern recognition model or the trained initial bert pre-training model. Training is usually referred to as reaching a preset model training condition, such as reaching a preset model training times or model convergence.

[0102] After obtaining the first fine-tuned BERT pre-training model, a network fault identification model can be built based on the first fine-tuned BERT pre-training model, and the network fault identification model corresponds to the network fault identification task described above. The network fault identification model as a whole can adopt a Bi-LSTM (Bidirectional Long Short-Term Memory Network) model structure, and the first fine-tuned BERT pre-training model described above can serve as an input layer in the network fault identification model for feature extraction. Similarly, an Adam optimizer can be configured, and a binary cross-entropy loss can be used as a loss function to train the model, so that the fine-tuned BERT model has the ability to capture the association between network traffic and network faults. During the fine-tuning process, the training feature samples in the training feature sample set can be input to the network fault identification model, and the network fault identification model can output network fault identification results. Then, according to the difference between the network fault identification results and the corresponding fault labels of the training feature samples, the network fault identification model is iteratively trained. Similarly, the difference between the network fault identification results and the corresponding fault labels of the training feature samples can be included in the loss function of the binary cross-entropy loss. After the network fault identification model is trained, a second fine-tuned BERT pre-training model that is fine-tuned based on the first fine-tuned BERT pre-training model can be obtained, that is, the first fine-tuned BERT pre-training model part in the trained network fault identification model.

[0103] After obtaining the second fine-tuned BERT pre-training model, a resource occupation prediction model is built by using the second fine-tuned BERT pre-training model. The resource occupation prediction model can be a deep learning model. Similarly, in the resource occupation prediction model, the second fine-tuned BERT pre-training model can serve as an input layer, and the Stochastic Gradient Descent (SGD) is configured as an optimization algorithm, and the Mean Squared Error (MSE) is configured as a loss function to train the model. After the BERT model is fine-tuned for the network resource occupation prediction task, the BERT model will have the ability to capture the correlation between network traffic and resource occupation, so that the BERT model can capture more rich network traffic features. During the fine-tuning process, the training feature samples in the training feature sample set can be input into the resource occupation prediction model, and the resource occupation prediction model will output a resource occupation prediction result. Based on the difference between the resource occupation prediction result and the resource occupation label corresponding to the training feature sample, the resource occupation prediction model is iteratively trained. It is worth noting that since the historical network traffic data can be obtained synchronously with the system resource usage of the service platform, that is, the resource occupation, the relationship between the historical network traffic data and the resource occupation can be automatically established, that is, the resource occupation label is labeled. In addition, after the resource occupation prediction model is trained, a preset BERT pre-training model fine-tuned based on the second fine-tuned BERT pre-training model can be obtained, that is, the part of the original second fine-tuned BERT pre-training model in the trained resource occupation prediction model.

[0104] It can be understood that in the present embodiment, the BERT pre-training model is fine-tuned by using three pre-training tasks, i.e., the network traffic pattern recognition task, the network fault prediction task, and the network resource occupation prediction task, to obtain a customized fine-tuned preset BERT pre-training model. After fine-tuning, the BERT is used as a network data feature extractor, so that the extracted feature vector contains rich network data features, which can provide rich network information for model training and subsequent operation and maintenance decision-making.

[0105] Reference Figure 3 For the third embodiment of the platform operation and maintenance method based on the first embodiment and the second embodiment, the flowchart of the third embodiment is the same as or similar to the above embodiments. In the step of inputting the current state feature vector into the preset neural network decision model to obtain the target operation and maintenance strategy of the service platform, the method comprises steps A10-A20:

[0106] Step A10, constructing a priority replay buffer based on an initial neural network decision model;

[0107] Step A20, based on the experience samples stored in the priority replay buffer, iteratively training the initial neural network decision model to obtain the preset neural network decision model.

[0108] It should be noted that the initial neural network decision model described above can be a PrioritizedDQN network model that has not been trained. Before training, the decision results of the initial neural network decision model in the actual working environment of the service platform can be used to generate experience samples, and a priority replay buffer can be constructed based on the generated experience samples. It should be noted that when the priority replay buffer is initially constructed, the initial priority of each experience sample can be the highest priority, so that each experience sample can be obtained once. And in the subsequent training process, the priority of each experience sample will be updated gradually. Then a batch of experience samples are sampled from the priority replay buffer (which can be sampled in a weighted random manner, and the basis for weighting is the priority of the experience sample, and the higher the priority of the experience sample, the easier it is to be sampled and used for training), which is used for iteratively training the initial neural network decision model, until the preset training condition is met, the trained initial neural network decision model is used as the preset neural network decision model.

[0109] In a feasible implementation, the step of constructing a priority replay buffer based on the initial neural network decision model includes steps A11-A15.

[0110] Step A11, obtaining historical real-time state information of the service platform in a historical period;

[0111] Step A12, extracting a historical real-time state feature vector from the historical real-time state information by using the preset BERT pre-training model;

[0112] Step A13, inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy;

[0113] Step A14, after executing the historical real-time operation and maintenance strategy on the service platform, obtaining an execution reward of the historical real-time operation and maintenance strategy and a new historical real-time state feature vector, wherein the execution reward is determined by at least one of a change in throughput, a change in delay, a change in resource utilization, and a change in error rate of the service platform after executing the historical real-time operation and maintenance strategy;

[0114] Step A15, generating experience samples based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward, and the new historical real-time state feature vector, and constructing a priority replay buffer based on the generated experience samples.

[0115] It should be noted that the above historical period refers to a period before the preset BERT pre-training model and the preset neural network decision model are actually used to make service platform operation and maintenance decisions. In this historical period, the main purpose is to generate experience samples for training the neural network decision model. And the front part of the experience sample generation is similar to the actual process of making service platform operation and maintenance decisions.

[0116] For example, in the historical period, the historical real-time state information of the service platform can be obtained, wherein the data in the historical real-time state information is similar to the current state information of the service platform, except that the data acquisition period is different. The preset BERT pre-training model is used to extract the historical real-time state feature vector from the historical real-time state information, and the historical real-time state feature vector is input into the initial neural network decision model to obtain the historical real-time operation and maintenance strategy. It is worth noting that the process of obtaining the historical real-time operation and maintenance strategy is similar to the process of obtaining the target operation and maintenance strategy, except that the historical real-time state feature vector and the initial neural network decision model are used to obtain the historical real-time operation and maintenance strategy, and the current state information and the preset neural network decision model are used to obtain the target operation and maintenance strategy. Therefore, this place will not be repeated.

[0117] After obtaining the historical real-time operation strategy, the service platform executes the historical real-time operation strategy, monitors the changes of various performance indicators of the service platform, generates an execution reward of the historical real-time operation strategy, and forms a new environment after the service platform executes the historical real-time operation strategy. The state information collected in the new environment is extracted by using the preset BERT pre-training model to obtain a new historical real-time state feature vector. The performance indicators can be at least one of a throughput change, a delay change, a resource utilization change, and an error rate change of the service platform. Preferably, the execution reward is calculated by the throughput change, the delay change, the resource utilization change, and the error rate change. For example, the execution reward function R(s, a) = w1Throughput(s, a) - w2Delay(s, a) - w3ResourceUsage(s, a) - w4ErrorRate(s, a). Wherein, R(s, a) is the reward (such as the historical real-time operation strategy) of executing action a in state s (such as the historical real-time state feature vector), w1, w2, w3, w4 are the weight coefficients of each factor. Throughput(s, a) is the reward of the throughput change of executing action a in state s, if the throughput increases, a positive reward is given, and if the throughput decreases, a negative reward is given; Delay(s, a) is the reward of the delay change of executing action a in state s, if the delay increases, a negative reward is given, and if the delay decreases, a positive reward is given; ResourceUsage(s, a) is the reward of the resource utilization change of executing action a in state s, if the utilization rate is too high or too low, a negative reward is given, and if the utilization rate is normal, a positive reward is given; ErrorRate(s, a) is the reward of the error rate change of executing action a in state s, if the error rate decreases, a positive reward is given, and if the error rate increases, a negative reward is given.

[0118] The historical real-time state feature vector, the historical real-time operation strategy, the execution reward, and the new historical real-time state feature vector are used to form an experience sample, and the buffer area for storing the experience sample with priority is the priority replay buffer.

[0119] For example, the network structure of the initial neural network decision model includes three parts, namely, an input layer, a hidden layer, and an output layer. The vector representation S of the historical real-time state information is input into the input layer. t As input, after nonlinear transformation of multiple fully connected layers in the hidden layer, the inference obtains the optimal behavior strategy A (i.e., the historical real-time operation strategy) for the historical real-time state information, and then executes the strategy to obtain the feature vector S of the new state t+1 (i.e., the new historical real-time state feature vector) and the reward R (i.e., the execution reward), thereby forming an experience sample (S t ,A,R,S t+1), and stores it in the priority replay buffer. In addition, if the experience sample is used for training in the model training stage, the experience sample can be updated according to the result of model training, i.e., whether the model convergence condition is_end is met, to obtain (S t ,A,R,S t+1 ,is_end), and is also stored in the priority replay buffer. For the experience sample that meets the model convergence condition, it can not be sampled for training in the training process.

[0120] In an embodiment, the step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy includes steps A131-A132.

[0121] Step A131: inputting the historical real-time state feature vector into the initial neural network decision model, and predicting, by the initial neural network decision model, a predicted execution reward of each candidate action in a preset action space under an environment corresponding to the historical real-time state feature vector.

[0122] Step A132: selecting, as the historical real-time operation and maintenance strategy, a candidate action with a maximum predicted execution reward from the candidate actions.

[0123] It should be noted that in the embodiment, an action space corresponding to the operation and maintenance strategy can be constructed first, and different actions in the action space represent different operation and maintenance strategies. For example, the operation and maintenance strategy can adjust the network bandwidth, increase the server resources, and optimize the routing, etc. In actual applications, a technical person can set different operation and maintenance strategies according to actual needs, which will not be described herein.

[0124] For example, the historical real-time state feature vector can be input into the initial neural network decision model, and the initial neural network decision model can predict an execution reward of each candidate action in a preset action space under an environment corresponding to the historical real-time state feature vector, i.e., obtain the predicted execution reward. Then, a candidate action with a maximum predicted execution reward can be selected from the candidate actions as the historical real-time operation and maintenance strategy. Similarly, the process of obtaining the target operation and maintenance strategy of the service platform in actual applications is basically similar to the above content.

[0125] In an embodiment, the initial neural network decision model includes a main Q network and a target Q network, model parameters in the main Q network are updated to the target Q network according to a preset period, and the step of iteratively training the initial neural network decision model based on the experience sample stored in the priority replay buffer to obtain the preset neural network decision model includes steps A21-A25.

[0126] Step A21, collecting part of experience samples from the priority replay buffer to obtain each candidate experience sample;

[0127] Step A22, for any one of the candidate experience samples, calculating a target Q value of the candidate experience sample by the target Q network;

[0128] Step A23, calculating a predicted Q value of the candidate experience sample by the main Q network;

[0129] Step A24, updating the model parameters in the main Q network based on the difference between the target Q value and the predicted Q value, and after updating the priority of the candidate experience sample according to the difference between the target Q value and the predicted Q value, putting the candidate experience sample with updated priority back to the priority replay buffer;

[0130] Step A25, after the main Q network in the initial neural network decision model meets a preset model training condition, taking the trained initial neural network decision model as the preset neural network decision model.

[0131] It should be noted that in the embodiment, the initial neural network decision model includes the main Q network and the target Q network, wherein the target Q network can be a copy of the main Q network, and the model parameters in the main Q network will be updated to the target Q network according to a preset period, that is, after reaching the update period, the model parameters in the main Q network can be copied to the target Q network to complete the update of the target Q network.

[0132] For example, part of the experience samples are collected from the priority replay buffer to obtain each candidate experience sample for the training of the current round. Among them, the higher the priority of the experience sample, the higher the probability of being collected as a candidate experience sample, and for the initial priority replay buffer, the priority of the existing experience samples is the same and is the highest. Since the training process using each candidate experience sample is roughly the same, in the embodiment, one candidate experience sample will be taken as an example for description. The target Q value of the candidate experience sample is calculated by the target Q network.

[0133] The target Q value calculation formula is as follows:

[0134]

[0135] In the formula, R j is the reward (i.e. the execution reward in the candidate experience sample j), γ is the decay factor, Q' is the target Q network, Q is the main Q network, is the state feature vector (such as the historical real-time state feature vector) in the candidate experience sample j, The new state feature vector obtained by performing action A (i.e., the new historical real-time state feature vector in the candidate experience sample j), A is the performed action (i.e., the historical real-time operation and maintenance strategy in the candidate experience sample j), w' is the model parameter of the target Q network, w is the model parameter of the main Q network, y j is the target Q value, the representation model converges to true, the representation model converges to false.

[0136] In the process of calculating the predicted Q value of the candidate experience sample by the main Q network, the predicted Q value is the reward of the decision made by the main Q network for the environment corresponding to the historical real-time state feature vector in the candidate experience sample j, that is, the predicted reward of each candidate action in the environment corresponding to the historical real-time state feature vector in the candidate experience sample j using the main Q network. The highest execution reward is the predicted Q value. The model parameters in the main Q network are updated based on the difference between the target Q value and the predicted Q value. The difference between the target Q value and the predicted Q value can be represented by a loss function, for example, the loss function can be a least square error loss function, and the loss function formula can be as follows:

[0137] In the formula, m is the number of samples of batch gradient descent, w j is the loss function weight, y j is the target Q value of the jth candidate experience sample, Q is the main Q network, is the state feature vector (such as the historical real-time state feature vector) in the candidate experience sample j, A j is the operation and maintenance strategy taken by the main Q network for the environment corresponding to the historical real-time state feature vector in the candidate experience sample j, w is the model parameter of the main Q network. is the predicted Q value.

[0138] The gradient can be determined by the above loss function, and all model parameters w of the main Q network are updated based on the gradient backpropagation, thereby completing one update. It is worth noting that after obtaining the difference between the target Q value and the predicted Q value, the priority of the candidate experience sample can be updated based on the difference, wherein the priority of the candidate experience sample is proportional to the difference between the target Q value and the predicted Q value.

[0139] The above process is repeated until the main Q network in the initial neural network decision model meets the preset model training condition, which can be a preset training number and model convergence. The initial neural network decision model that meets the preset model training condition can be used as the preset neural network decision model.

[0140] In addition, with reference to Figure 4This is a schematic diagram of the model framework involved in the platform operation and maintenance method of this application. Figure 4 As shown, the initial steps involve data collection, data denoising, and data preprocessing. The preprocessed results will be used for fine-tuning of BERT (i.e., the BERT pre-trained model). Fine-tuning tasks may include network traffic pattern recognition, network fault prediction, and network occupancy prediction. After fine-tuning, the preset BERT pre-trained model is obtained. The preset BERT pre-trained model will extract the state (i.e., current state information or historical real-time state information) from the service platform environment and input the state into the deep neural network (i.e., the aforementioned preset neural network decision model or initial neural network decision model). This allows the deep neural network to select actions (i.e., operation and maintenance strategies) from the action space for execution, such as action 3, and generate impacts and rewards on the environment. This data will be used to form experience samples and stored in a priority replay buffer.

[0141] Understandably, in this embodiment, by fine-tuning BERT and combining it with PrioritizedDQN to strengthen the neural network, multiple factors and features can be comprehensively considered. BERT, as a pre-trained model, can extract semantic and contextual features from network data by learning language and contextual information. PrioritizedDQN, on the other hand, can comprehensively consider factors such as network traffic patterns, fault signs, and network resource consumption based on these features and real-time network conditions, thereby deriving the optimal operation and maintenance strategy. Compared to traditional operation solutions, it can respond more promptly to network changes and fault conditions, and make intelligent decisions based on pre-trained and fine-tuned knowledge, thus improving the efficiency and quality of network operation.

[0142] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the platform operation and maintenance method in Embodiment 1 above.

[0143] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as computers. Figure 5The electronic device shown is merely an example and should not impose any limitation on the functions and usage range of the embodiments of the present application.

[0144] As shown in Figure 5 The electronic device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for operation of the electronic device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the electronic device having various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0145] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are performed.

[0146] The electronic device provided in the present application adopts the platform operation and maintenance method in the above-mentioned embodiments, and can solve the technical problem that the existing operation and maintenance method is highly dependent on manual operation. Compared with the prior art, the electronic device provided in the present application has the same beneficial effects as the platform operation and maintenance method provided in the above-mentioned embodiments, and other technical features in the electronic device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0147] It should be understood that portions of the application disclosed can be implemented in hardware, software, firmware, or combinations thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0148] The above description is merely illustrative of the application and the application should not be limited thereto. Any variations and modifications of the application falling within the scope of the application disclosed herein will readily occur to those skilled in the art. Accordingly, to protect the novelty of the application, the sphere of protection of the application should extend to the broadest scope of protection of the principles and novel features disclosed herein.

[0149] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the platform operation method in the above embodiments.

[0150] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0151] The above computer readable storage medium can be included in an electronic device; or can exist separately and not be assembled into an electronic device.

[0152] The above computer readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0153] obtain current state information of the service platform;

[0154] extract a current state feature vector from the current state information through a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained through preset fine-tuning of a maintenance task;

[0155] input the current state feature vector into a preset neural network decision model to obtain a target maintenance strategy of the service platform.

[0156] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0157] The flow and block diagrams in the drawings show the architectural, functional and operational views of possible implementations of systems, methods and computer program products according to the various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code that comprises one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may also be executed in reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0158] The modules involved in the embodiments of the present application can be implemented in a software manner or in a hardware manner. In some cases, the name of the module does not constitute a limitation on the module itself.

[0159] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the platform operation and maintenance method described above, and can solve the technical problem that the existing operation and maintenance method has high dependence on manual operation. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the platform operation and maintenance method provided by the above-mentioned embodiments, and will not be described here.

[0160] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the platform operation and maintenance method as described above.

[0161] The computer program product provided by the present application can solve the technical problem of platform operation and maintenance. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the platform operation and maintenance method provided by the above-mentioned embodiments, and will not be described here.

[0162] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A platform operation method, characterized by, The application is applied to a service platform, and the platform operation and maintenance method comprises the following steps: Obtain the current state information of the service platform; Extract the current state feature vector from the current state information through a preset BERT pre-training model, wherein the preset BERT pre-training model is obtained through preset fine-tuning of a preset fine-tuning operation and maintenance task; Input the current state feature vector into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform; Before the step of extracting the current state feature vector from the current state information through the preset BERT pre-training model, the method comprises: Obtain the historical network traffic data of the service platform, wherein the sources of the historical network traffic data include at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log; After the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, perform feature extraction on the labeled historical network traffic data to obtain a training feature sample set; Fine-tune an initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model; The preset fine-tuning operation and maintenance task includes a traffic pattern recognition task, a network fault recognition task, and a resource occupation prediction task, the label types of the labeling include a pattern label corresponding to the traffic pattern recognition task, a fault label corresponding to the network fault recognition task, and a resource occupation label corresponding to the resource occupation prediction task, and the step of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model comprises: Build a traffic pattern recognition model based on the initial BERT pre-training model; Input the training feature samples in the training feature sample set into the traffic pattern recognition model to obtain traffic pattern recognition results; Iteratively train the traffic pattern recognition model based on the difference between the traffic pattern recognition results and the corresponding pattern labels of the training feature samples, and obtain a first fine-tuned BERT pre-training model after the traffic pattern recognition model is trained; Build a network fault recognition model based on the first fine-tuned BERT pre-training model; Input the training feature samples in the training feature sample set into the network fault recognition model to obtain network fault recognition results; Iteratively train the network fault recognition model based on the difference between the network fault recognition results and the corresponding fault labels of the training feature samples, and obtain a second fine-tuned BERT pre-training model after the network fault recognition model is trained; Build a resource occupation prediction model based on the second fine-tuned BERT pre-training model; Input the training feature samples in the training feature sample set into the resource occupation prediction model to obtain resource occupation prediction results; The resource occupation prediction model is iteratively trained based on a difference between the resource occupation prediction result and a resource occupation label corresponding to a training feature sample, and after the resource occupation prediction model is trained, a preset BERT pre-training model is obtained after the second fine-tuning BERT pre-training model is fine-tuned.

2. The platform operation method of claim 1, wherein, The current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting a current state feature vector from the current state information by the preset BERT pre-training model includes: extracting a traffic feature vector from the current network traffic data by the preset BERT pre-training model; reconstructing the system state data based on a dimension feature of the traffic feature vector to obtain an array conforming to a last dimension feature of the traffic feature vector; combining the array with the last dimension feature of the traffic feature vector to obtain the current state feature vector. 3.The platform operation method of claim 1, wherein, Before the step of inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform, the method includes: constructing a priority replay buffer based on an initial neural network decision model; iteratively training the initial neural network decision model based on experience samples stored in the priority replay buffer to obtain the preset neural network decision model. 4.The platform operation method of claim 3, wherein, The step of constructing the priority replay buffer based on the initial neural network decision model includes: obtaining historical real-time state information of the service platform in a historical period; extracting a historical real-time state feature vector from the historical real-time state information by the preset BERT pre-training model; inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy; obtaining an execution reward of the historical real-time operation and maintenance strategy and a new historical real-time state feature vector after the service platform executes the historical real-time operation and maintenance strategy, wherein the execution reward is determined by at least one of a change amount of throughput, a change amount of delay, a change amount of resource utilization, and a change amount of error rate of the service platform after executing the historical real-time operation and maintenance strategy; generating experience samples based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward, and the new historical real-time state feature vector, and constructing a priority replay buffer based on the generated experience samples. 5.The platform operation method of claim 4, wherein, The step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy includes: inputting the historical real-time state feature vector into the initial neural network decision model to predict a predicted execution reward of each candidate action in a preset action space under the historical real-time state feature vector corresponding environment by the initial neural network decision model; taking a candidate action with the maximum predicted execution reward as the historical real-time operation and maintenance strategy. 6.The platform operation method of claim 4, wherein, The initial neural network decision model comprises a main Q network and a target Q network, model parameters in the main Q network are to be updated to the target Q network according to a preset period, and the step of iteratively training the initial neural network decision model based on the experience samples stored in the priority replay buffer to obtain the preset neural network decision model comprises: collecting part of the experience samples from the priority replay buffer to obtain each candidate experience sample; for any one of the candidate experience samples, calculating a target Q value of the candidate experience sample through the target Q network; calculating a predicted Q value of the candidate experience sample through the main Q network; updating model parameters in the main Q network based on the difference between the target Q value and the predicted Q value, and putting the candidate experience sample with the updated priority back into the priority replay buffer after updating the priority of the candidate experience sample based on the difference between the target Q value and the predicted Q value; after the main Q network in the initial neural network decision model meets a preset model training condition, taking the trained initial neural network decision model as the preset neural network decision model.

7. An electronic device, comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the platform operation and maintenance according to any one of claims 1 to 6.

8. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the platform operation and maintenance according to any one of claims 1 to 6.

9. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the steps of the platform operation and maintenance according to any one of claims 1 to 6.

Images