Platform operation and maintenance method, electronic equipment, medium and product

Abstract

The invention discloses a platform operation and maintenance method, electronic equipment, a medium and a product, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining the current state information of a service platform; a current state feature vector is extracted from the current state information through a preset bert pre-training model, and the preset bert pre-training model is obtained through fine adjustment of a preset fine adjustment operation and maintenance task; and inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy of the service platform. According to the method, automation and intelligentization of operation decision making are realized, and the decision making network model can automatically learn and optimize from a large amount of network data, so that the accuracy and effect of the operation and maintenance strategy are continuously improved, the manual intervention demand and dependence are reduced, the operation efficiency is improved, and the labor cost is reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a platform operation and maintenance method, electronic equipment, medium and product. Background Art

[0002] The current operation and maintenance solutions for the capability platform (a service platform that integrates multiple advanced technologies) mainly include: traditional operation and maintenance solutions, automated operation and maintenance solutions, DevOps operation and maintenance solutions, and containerized operation and maintenance solutions. However, most of the above existing operation and maintenance methods rely on human participation during the application process, such as manual operation and management, or manual maintenance and updating of automated tools. Therefore, the current existing operation and maintenance methods are highly dependent on human labor.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a platform operation and maintenance method, electronic equipment, medium and product, aiming to solve the technical problem that the current operation and maintenance methods are highly dependent on manual labor.

[0005] To achieve the above objectives, the present application proposes a platform operation and maintenance method, which is applied to a service platform. The platform operation and maintenance method comprises the following steps:

[0006] Obtaining current status information of the service platform;

[0007] Extracting a current state feature vector from the current state information by using a preset BERT pre-trained model, wherein the preset BERT pre-trained model is fine-tuned by a preset fine-tuning operation and maintenance task;

[0008] The current state feature vector is input into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform.

[0009] Optionally, before the step of extracting the current state feature vector from the current state information by using a preset BERT pre-training model, the method includes:

[0010] Acquire historical network traffic data of the service platform, wherein the historical network traffic data is acquired from at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log;

[0011] After the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, feature extraction is performed on the labeled historical network traffic data to obtain a training feature sample set;

[0012] The initial BERT pre-training model is fine-tuned based on the training feature sample set to obtain the preset BERT pre-training model.

[0013] Optionally, the preset fine-tuning operation and maintenance task includes a traffic pattern recognition task, a network fault recognition task, and a resource occupancy prediction task, and the label type for marking includes a pattern label corresponding to the traffic pattern recognition task, a fault label corresponding to the network fault recognition task, and a resource occupancy label corresponding to the resource occupancy prediction task. The step of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model includes:

[0014] Building a traffic pattern recognition model based on the initial BERT pre-training model;

[0015] Inputting the training feature samples in the training feature sample set into the traffic pattern recognition model to obtain a traffic pattern recognition result;

[0016] Based on the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample, the traffic pattern recognition model is iteratively trained, and after the traffic pattern recognition model training is completed, a first fine-tuned BERT pre-trained model is obtained by fine-tuning the initial BERT pre-trained model;

[0017] Building a network fault identification model based on the first fine-tuned BERT pre-trained model;

[0018] Inputting the training feature samples in the training feature sample set into the network fault identification model to obtain a network fault identification result;

[0019] Based on the difference between the network fault identification result and the fault label corresponding to the training feature sample, the network fault identification model is iteratively trained, and after the network fault identification model training is completed, a second fine-tuned BERT pre-trained model is obtained by fine-tuning the first fine-tuned BERT pre-trained model;

[0020] Building a resource occupancy prediction model based on the second fine-tuned BERT pre-trained model;

[0021] Inputting the training feature samples in the training feature sample set into the resource occupancy prediction model to obtain a resource occupancy prediction result;

[0022] Based on the difference between the resource occupancy prediction result and the resource occupancy label corresponding to the training feature sample, the resource occupancy prediction model is iteratively trained, and after the resource occupancy prediction model training is completed, a preset BERT pre-trained model is obtained after fine-tuning the second fine-tuning BERT pre-trained model.

[0023] Optionally, the current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting a current state feature vector from the current state information by using a preset BERT pre-trained model includes:

[0024] Extracting a traffic feature vector from the current network traffic data through the preset BERT pre-training model;

[0025] Based on the dimensional characteristics of the traffic feature vector, the system state data is reconstructed to obtain an array in a form that conforms to the last dimensional characteristics of the traffic feature vector;

[0026] The array is combined with the last dimension feature in the traffic feature vector to obtain a current state feature vector.

[0027] Optionally, before the step of inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy for the service platform, the method includes:

[0028] Based on the initial neural network decision model, a priority playback buffer is constructed;

[0029] Based on the experience samples stored in the priority playback buffer, the initial neural network decision model is iteratively trained to obtain the preset neural network decision model.

[0030] Optionally, the step of constructing a priority playback buffer based on the initial neural network decision model includes:

[0031] Under the historical time period, obtain the historical real-time status information of the service platform;

[0032] Extracting a historical real-time state feature vector from the historical real-time state information by using the preset BERT pre-training model;

[0033] Inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy;

[0034] After the service platform executes the historical real-time operation and maintenance strategy, an execution reward of the historical real-time operation and maintenance strategy and a new historical real-time state feature vector are obtained, wherein the execution reward is determined by at least one of a throughput change, a delay change, a resource utilization change, and an error rate change of the service platform after executing the historical real-time operation and maintenance strategy;

[0035] An experience sample is generated based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward, and the new historical real-time state feature vector, and a priority playback buffer is constructed based on the generated experience sample.

[0036] Optionally, the step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain the historical real-time operation and maintenance strategy includes:

[0037] Inputting the historical real-time state feature vector into the initial neural network decision model, and predicting the predicted execution reward of each candidate action in the preset action space under the environment corresponding to the historical real-time state feature vector through the initial neural network decision model;

[0038] The candidate action with the largest predicted execution reward among the candidate actions is used as the historical real-time operation and maintenance strategy.

[0039] Optionally, the initial neural network decision model includes a main Q network and a target Q network, the model parameters in the main Q network will be updated to the target Q network according to a preset period, and the step of iteratively training the initial neural network decision model based on the experience samples stored in the priority playback buffer to obtain the preset neural network decision model includes:

[0040] Collecting some experience samples from the priority playback buffer to obtain candidate experience samples;

[0041] For any one of the candidate experience samples, calculating a target Q value of the candidate experience sample by using the target Q network;

[0042] Calculate the predicted Q value of the candidate experience sample through the main Q network;

[0043] The model parameters in the main Q network are updated based on the difference between the target Q value and the predicted Q value, and the priority of the candidate experience sample is updated according to the difference between the target Q value and the predicted Q value, and then the candidate experience sample with the updated priority is placed back into the priority playback buffer;

[0044] After the main Q network in the initial neural network decision model meets the preset model training conditions, the trained initial neural network decision model is used as the preset neural network decision model.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the platform operation and maintenance method described above.

[0046] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the platform operation and maintenance method described above are implemented.

[0047] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the platform operation and maintenance method described above.

[0048] One or more technical solutions proposed in this application have at least the following technical effects:

[0049] In an embodiment of the present application, the current state information of the service platform is obtained; the current state feature vector is extracted from the current state information by a preset BERT pre-trained model, wherein the preset BERT pre-trained model is obtained by fine-tuning the preset fine-tuning operation and maintenance task; the current state feature vector is input into the preset neural network decision model to obtain the target operation and maintenance strategy of the service platform. That is, the preset BERT pre-trained model obtained by fine-tuning the preset fine-tuning operation and maintenance task can extract the current state feature vector related to operation and maintenance from the current state information of the service platform, thereby ensuring the pertinence of the data input into the preset neural network decision model, so that the preset neural network decision model makes a decision based on the current state feature vector, and the obtained target operation and maintenance strategy can effectively improve the performance of the service platform. The present application uses the method of pre-training and fine-tuning, combined with the decision network model, to realize the automation and intelligence of operation and maintenance decision-making. And the decision network model can automatically learn and optimize from a large amount of network data, thereby continuously improving the accuracy and effect of the operation and maintenance strategy, reducing the need and dependence of manual intervention, improving operational efficiency, and reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 This is a flow chart of the first embodiment of the platform operation and maintenance method of this application;

[0053] Figure 2This is a flow chart of the second embodiment of the platform operation and maintenance method of this application;

[0054] Figure 3 This is a flow chart of the third embodiment of the platform operation and maintenance method of this application;

[0055] Figure 4 This is a schematic diagram of the model framework involved in the operation and maintenance method of this application platform;

[0056] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the platform operation and maintenance method in the embodiment of the present application.

[0057] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0059] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0060] The current operation and maintenance solutions for the capability platform (a service platform that integrates multiple advanced technologies) can mainly include: traditional operation and maintenance solutions, automated operation and maintenance solutions, DevOps operation and maintenance solutions, and containerized operation and maintenance solutions. Among them, the traditional operation and maintenance solution is based on manual operation and management, usually using scripts (such as Shell, Python) and basic system management tools (such as cron, rsync), relying on the experience and skills of system administrators to perform various management and maintenance tasks; in the automated operation and maintenance solution, most operation and maintenance tasks are completed through automated tools and scripts, reducing the reliance on manual operations, reducing the risk of human errors, and ensuring that the same operation obtains consistent results in different environments; the DevOps operation and maintenance solution is a concept and practice that integrates software development (Development) and IT operation and maintenance (Operations), aiming to improve the efficiency and quality of software development and deployment by improving collaboration, automation and continuous delivery; the containerized operation and maintenance solution is a modern operation and maintenance method that uses container technology to achieve application packaging, deployment and management. Containerized operation and maintenance provides higher flexibility, portability and resource utilization.

[0061] It is worth noting that in traditional operation and maintenance solutions, the operation and maintenance of the platform relies heavily on manual operations; in automated operation and maintenance solutions, although manual operations can be reduced, it still relies on manual maintenance and updates of automated tools; in DevOps operation and maintenance solutions, manual intervention is still required, and the configuration and integration of the tool chain required for solution implementation requires operation and maintenance personnel to have a high initial investment and learning cost; in containerized operation and maintenance solutions, the learning curve of containerized technology and tools is steep, which is also not friendly to novice operation and maintenance personnel. Therefore, as mentioned above, the current operation and maintenance methods are highly dependent on manual labor and have high requirements on the technical level of operation and maintenance personnel.

[0062] The main solution of the embodiment of the present application is: obtaining the current state information of the service platform; extracting the current state feature vector from the current state information through a preset BERT pre-trained model, wherein the preset BERT pre-trained model is obtained by fine-tuning the preset fine-tuning operation and maintenance task; inputting the current state feature vector into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform.

[0063] That is, the preset BERT pre-trained model obtained through fine-tuning of the preset fine-tuning operation and maintenance tasks can extract the current state feature vector related to operation and maintenance from the current state information of the service platform, thereby ensuring the pertinence of the data input into the preset neural network decision model, so that the preset neural network decision model makes decisions based on the current state feature vector, and the obtained target operation and maintenance strategy can effectively improve the performance of the service platform. This application utilizes pre-training and fine-tuning methods, combined with a decision network model, to achieve automation and intelligence of operational decisions. Moreover, the decision network model can automatically learn and optimize from a large amount of network data, thereby continuously improving the accuracy and effectiveness of the operation and maintenance strategy, reducing the need and dependence on manual intervention, improving operational efficiency, and reducing labor costs.

[0064] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a cloud platform, a computer, a mobile phone, etc., or an electronic device that can realize the above functions.

[0065] Based on this, the present application embodiment provides a platform operation and maintenance method, referring to Figure 1 , which is a flow chart of the first embodiment of the platform operation and maintenance method of this application.

[0066] In this embodiment, the platform operation and maintenance method is applied to a service platform and includes steps S10 to S30:

[0067] Step S10, obtaining current status information of the service platform;

[0068] It should be noted that the above service platform can be a capability platform, which is an intelligent platform that integrates multiple advanced technologies such as the Internet of Things, cloud computing, big data, artificial intelligence, 5G and blockchain to provide different service capabilities. In this embodiment, the above platform operation and maintenance plan mainly provides a plan for maintaining the service platform to ensure the performance of the service platform.

[0069] Exemplarily, in this embodiment, the current status information of the service platform can be obtained through a variety of ways. The current status information may include the current network traffic data of the service platform and the current system status data of the service platform. For example, the current network traffic data may include the bandwidth utilization, delay, packet loss rate, etc. of the service platform, and the current system status data may include system indicators such as the server load, CPU utilization, and memory usage of the service platform. The current network traffic data may be obtained through network packet capture tools, traffic monitoring tools, and logs of firewalls and routers. The current system status data may be obtained through the built-in status monitoring tool of the service platform.

[0070] Step S20, extracting a current state feature vector from the current state information by using a preset BERT pre-trained model, wherein the preset BERT pre-trained model is obtained by fine-tuning a preset fine-tuning operation and maintenance task;

[0071] It should be noted that the above-mentioned preset BERT pre-trained model is obtained by fine-tuning the preset fine-tuning operation and maintenance task, that is, the preset BERT pre-trained model is obtained by training the BERT pre-trained model twice for a specific task. Among them, the preset fine-tuning operation and maintenance task can be set by the technical personnel according to the operation and maintenance requirements of the platform. For example, the preset fine-tuning operation and maintenance task can be a network fault identification task, a network traffic pattern identification task, etc. It can be understood that since the preset BERT pre-trained model is obtained by fine-tuning the preset fine-tuning operation and maintenance task, the preset BERT pre-trained model can keenly capture the features related to the platform operation and maintenance in the state information. For example, when the preset fine-tuning operation and maintenance task includes a network fault identification task, the preset BERT pre-trained model can extract the features related to the network fault in the state information, and when it includes a network traffic pattern identification task, the preset BERT pre-trained model can extract the features related to the network traffic pattern in the state information. Therefore, the current state feature vector is extracted from the current state information of the service platform through the preset BERT pre-trained model, which is used for the operation and maintenance decision of the preset neural network decision model in the subsequent steps.

[0072] Step S30, inputting the current state feature vector into a preset neural network decision model to obtain a target operation and maintenance strategy for the service platform.

[0073] It should be noted that the above-mentioned preset neural network decision model can be a PrioritizedDQN (PrioritizedExperienceReplayDeepQ-Network) enhanced neural network model, and is pre-trained using experience samples to enable it to have operation and maintenance decision-making capabilities.

[0074] Exemplarily, the current state feature vector output by the preset BERT pre-training model is input into the preset neural network decision model. The preset neural network decision model can predict the benefits that the service platform can obtain (i.e., the system performance improvement of the service platform) by executing different operation and maintenance strategies under the environment corresponding to the current state feature vector, and then use the operation and maintenance strategy with the highest benefit as the target operation and maintenance strategy. In practical applications, the target operation and maintenance strategy can be used as the basis for selecting the operation and maintenance strategy of the service platform, and can also control the service platform to directly execute the target operation and maintenance strategy.

[0075] In this embodiment, the current state information of the service platform is obtained; the current state feature vector is extracted from the current state information by a preset BERT pre-trained model, wherein the preset BERT pre-trained model is obtained by fine-tuning the preset fine-tuning operation and maintenance task; the current state feature vector is input into the preset neural network decision model to obtain the target operation and maintenance strategy of the service platform. That is, the preset BERT pre-trained model obtained by fine-tuning the preset fine-tuning operation and maintenance task can extract the current state feature vector related to operation and maintenance from the current state information of the service platform, thereby ensuring the pertinence of the data input into the preset neural network decision model, so that the preset neural network decision model makes a decision based on the current state feature vector, and the obtained target operation and maintenance strategy can effectively improve the performance of the service platform. This application uses the method of pre-training and fine-tuning, combined with the decision network model, to realize the automation and intelligence of operation decision-making. And the decision network model can automatically learn and optimize from a large amount of network data, so as to continuously improve the accuracy and effect of the operation and maintenance strategy, reduce the need and dependence of manual intervention, improve operational efficiency, and reduce labor costs.

[0076] In a feasible implementation manner, the current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting the current state feature vector from the current state information by using the preset BERT pre-training model includes steps S21 to S23:

[0077] Step S21, extracting a traffic feature vector from the current network traffic data through the preset BERT pre-training model;

[0078] Step S22, based on the dimensional characteristics of the flow feature vector, reconstructing the system state data to obtain an array in a form that conforms to the last dimensional characteristics of the flow feature vector;

[0079] Step S23, combining the array with the last dimension feature in the traffic feature vector to obtain a current state feature vector.

[0080] It should be noted that the above current status information includes the current network traffic data of the service platform and the current system status data of the service platform, and accordingly, it may include the bandwidth utilization, delay, packet loss rate, etc. of the service platform. The current system status data may include system indicators such as the server load, CPU utilization, and memory usage of the service platform. It is worth noting that since the system status data itself is quantified data, it does not need to be pre-trained by the preset BERT model.

[0081] Exemplarily, the current network traffic data can be input into a preset BERT pre-trained model, and the preset BERT pre-trained model can output a traffic feature vector. Then, according to the dimensional characteristics of the traffic feature vector, the system state data is reconstructed to form an array that conforms to the last dimensional characteristics of the traffic feature vector in form (such as an array with the same data structure). Then, the array is combined with the last dimensional characteristics in the traffic feature vector to obtain the current state feature vector. The combination can be to add the last dimensional characteristics in the traffic feature vector to the array, or to multiply the array by a preset coefficient and then add the multiplied result to the last dimensional characteristics in the traffic feature vector.

[0082] In addition, it should be noted that in actual applications, the current network traffic data can be divided into network traffic packet data and network traffic index data. Among them, the network traffic index data itself is also quantified data, so it can be processed in a similar way to the above-mentioned system status data and combined with the traffic feature vector to obtain the current state feature vector, which will not be repeated here.

[0083] Reference Figure 2 , is a flowchart of a second embodiment proposed based on the first embodiment of the platform operation and maintenance method of the present application. In this embodiment, the same or similar contents as the above embodiment can be referred to the above introduction, and will not be repeated later. Before the step of extracting the current state feature vector from the current state information by using the preset BERT pre-training model, the method includes steps S210 to S230:

[0084] Step S210, obtaining historical network traffic data of the service platform, wherein the historical network traffic data is obtained from at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log;

[0085] Step S220, after the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, feature extraction is performed on the labeled historical network traffic data to obtain a training feature sample set;

[0086] Step S230, fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model.

[0087] It should be noted that before applying the above preset BERT pre-trained model and the preset neural network decision model for operation and maintenance decision-making, it is necessary to obtain the preset BERT pre-trained model by fine-tuning.

[0088] Exemplarily, the historical network traffic data of the service platform is obtained, wherein the obtained historical network traffic data can be saved as a .pcap file, and the historical network traffic data is obtained from at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log. In practical applications, it is preferred to use each of the above sources to obtain historical network traffic data to ensure the richness of historical network traffic data. In addition, the technicians can also select based on the actual situation of the service platform.

[0089] The acquired historical network traffic data needs to be preprocessed first. Preprocessing can include deleting duplicate and irrelevant data packets to reduce noise, such as ARP (Address Resolution Protocol), ICMP (Internet Control Message Protocol), etc.; filtering specific types of traffic according to specific fine-tuning task scenarios: for system traffic data in network traffic data, retain specific types of traffic related to business scenarios, such as HTTP (HyperText Transfer Protocol) requests, database queries, etc., and use business scenario labels to annotate traffic data to form labels related to business scenarios; for network fault traffic in network traffic data, retain traffic data containing network fault records and related early signs (such as abnormal traffic, delay, packet loss, etc.), and annotate fault and non-fault data; for user behavior data in network traffic data, retain traffic data related to user online behavior (such as browsing web pages, downloading files, video streaming, etc.) and network resource allocation data (such as bandwidth, number of connections, etc.). Use the traffic parsing tool to parse the .pcap file (i.e. the above historical network traffic data) to extract the application layer protocol data, and extract its features based on the extracted traffic data, such as source IP, destination IP, source port, destination port, protocol type, packet size, traffic direction, etc. Convert the packet features into a text representation, and represent each packet as a string containing the values ​​of each feature, which can be separated by spaces. Use the Tokenizer of the Bert model to convert the text data into an input format that the Bert model can process, including token IDs (a digital expression of text data), attention masks (a matrix similar to Token IDs), etc. Tokenizer is a step in feature extraction. The feature vector refers to a complete data representation. The various parts of the data packet after passing through the Tokenizer are the dimensions of the feature vector after feature extraction. That is, the feature vectors obtained from the various parts of a network flow after passing through the Tokenizer constitute the complete feature vector of this network flow. The complete feature vector combined with the labels marked in the above process can obtain the training feature samples in the training feature sample set. Then, the initial BERT pre-training model (referring to the BERT pre-training model that has not been fine-tuned) is fine-tuned using the training feature sample set to obtain the above-mentioned preset BERT pre-training model.

[0090] In a feasible implementation manner, the preset fine-tuning operation and maintenance tasks include a traffic pattern recognition task, a network fault recognition task, and a resource occupancy prediction task, the label types for annotation include a pattern label corresponding to the traffic pattern recognition task, a fault label corresponding to the network fault recognition task, and a resource occupancy label corresponding to the resource occupancy prediction task, and the step of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model includes steps S231 to S239:

[0091] Step S231, building a traffic pattern recognition model based on the initial BERT pre-training model;

[0092] Step S232, inputting the training feature samples in the training feature sample set into the traffic pattern recognition model to obtain a traffic pattern recognition result;

[0093] Step S233, based on the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample, iteratively train the traffic pattern recognition model, and after the training of the traffic pattern recognition model is completed, obtain a first fine-tuned BERT pre-trained model after fine-tuning the initial BERT pre-trained model;

[0094] Step S234, building a network fault identification model based on the first fine-tuned BERT pre-training model;

[0095] Step S235, inputting the training feature samples in the training feature sample set into the network fault identification model to obtain a network fault identification result;

[0096] Step S236, based on the difference between the network fault identification result and the fault label corresponding to the training feature sample, iteratively train the network fault identification model, and after the network fault identification model training is completed, obtain a second fine-tuned BERT pre-trained model that is fine-tuned by fine-tuning the first fine-tuned BERT pre-trained model;

[0097] Step S237, building a resource occupancy prediction model based on the second fine-tuned BERT pre-training model;

[0098] Step S238, inputting the training feature samples in the training feature sample set into the resource occupancy prediction model to obtain a resource occupancy prediction result;

[0099] Step S239, based on the difference between the resource occupancy prediction result and the resource occupancy label corresponding to the training feature sample, iteratively train the resource occupancy prediction model, and after the resource occupancy prediction model training is completed, obtain the preset BERT pre-trained model after fine-tuning the second fine-tuning BERT pre-trained model.

[0100] It should be noted that the preset fine-tuning operation and maintenance tasks include traffic pattern recognition tasks, network fault recognition tasks, and resource occupancy prediction tasks. Different labels will be marked for different preset fine-tuning operation and maintenance tasks, that is, the label types marked include pattern labels corresponding to traffic pattern recognition tasks, fault labels corresponding to network fault recognition tasks, and resource occupancy labels corresponding to resource occupancy prediction tasks. Then fine-tune the above preset BERT pre-trained model for different preset fine-tuning operation and maintenance tasks.

[0101] Exemplarily, the above-mentioned initial BERT pre-trained model refers to a BERT pre-trained model that has not been fine-tuned. In this embodiment, the initial BERT pre-trained model can be directly used as a traffic pattern recognition model, and the corresponding Adam (Adaptive Moment Estimation, an optimization algorithm for deep learning) optimizer is configured, and the cross-entropy loss (Cross-Entropy Loss) is used as the loss function to construct a network traffic pattern recognition task to identify the current network traffic pattern, so that the traffic pattern recognition model has the ability to extract network traffic features. During the fine-tuning process, the training feature samples in the training feature sample set can be input into the traffic pattern recognition model, and the traffic pattern recognition model will output the traffic pattern recognition result. Based on the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample, the traffic pattern recognition model is iteratively trained, such as the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample will be included in the above-mentioned loss function. Until the training of the traffic pattern recognition model is completed, the first fine-tuned BERT pre-trained model after fine-tuning the initial BERT pre-trained model can be obtained, that is, the traffic pattern recognition model after training is completed or the initial BERT pre-trained model after training is completed, and training completion usually means reaching the preset model training conditions, such as reaching the preset number of model training times or model convergence.

[0102] After obtaining the first fine-tuned BERT pre-trained model, a network fault identification model can be built based on the first fine-tuned BERT pre-trained model, and the network fault identification model corresponds to the above-mentioned network fault identification task. The network fault identification model as a whole can adopt the Bi-LSTM (bidirectional long short-term memory network) model structure, and in the network fault identification model, the above-mentioned first fine-tuned BERT pre-trained model can serve as an input layer for feature extraction. Similarly, an Adam optimizer can be configured, and the Binary Cross-Entropy Loss is used as the loss function training model, so that the fine-tuned BERT model has the ability to capture the correlation between network traffic and network faults. In the process of fine-tuning, the training feature samples in the training feature sample set can be input into the network fault identification model, and the network fault identification model then outputs the network fault identification result, and then according to the difference between the network fault identification result and the fault label corresponding to the training feature sample, the network fault identification model is iteratively trained. Similarly, the difference between the network fault identification result and the fault label corresponding to the training feature sample can be included in the loss function of the second hospital cross entropy loss. After the network fault identification model training is completed, the second fine-tuned BERT pre-trained model can be obtained by fine-tuning the first fine-tuned BERT pre-trained model, that is, the original first fine-tuned BERT pre-trained model part in the trained network fault identification model.

[0103] After obtaining the second fine-tuned BERT pre-trained model, a resource occupancy prediction model is built through the second fine-tuned BERT pre-trained model. The resource occupancy prediction model can be a deep learning model. Similarly, in the resource occupancy prediction model, the second fine-tuned BERT pre-trained model can serve as an input layer, and the machine gradient descent (SGD) is configured as the optimization algorithm, and the mean squared error (MSE) is used as the loss function training model. After the BERT model is fine-tuned for the network resource occupancy prediction task, it will have the ability to capture the correlation between network traffic and resource occupancy, so that the BERT model can capture richer network traffic features. In the process of fine-tuning, the training feature samples in the training feature sample set can be input into the resource occupancy prediction model, and the resource occupancy prediction model will output the resource occupancy prediction result. Based on the difference between the resource occupancy prediction result and the resource occupancy label corresponding to the training feature sample, the resource occupancy prediction model is iteratively trained. It is worth noting that since the system resource usage of the service platform, i.e., resource occupancy, can be obtained synchronously when the historical network traffic data is obtained, the connection between the historical network traffic data and the resource occupancy can be automatically established, i.e., the resource occupancy label is marked. In addition, after the resource occupancy prediction model training is completed, the preset BERT pre-trained model after fine-tuning the second fine-tuning BERT pre-trained model can be obtained, that is, the original second fine-tuning BERT pre-trained model part in the resource occupancy prediction model after training is completed.

[0104] It can be understood that in this embodiment, three pre-training tasks, namely, network traffic pattern recognition task, network fault prediction task, and network resource occupancy prediction task, are used to fine-tune the BERT pre-training model to obtain a customized fine-tuned preset BERT pre-training model. After fine-tuning, BERT is used as a network data feature extractor so that the extracted feature vector contains rich network data features, which can provide rich network information for model training and subsequent operation and maintenance decisions.

[0105] Reference Figure 3 , is a flowchart of the third embodiment proposed based on the first embodiment and the second embodiment of the platform operation and maintenance method of the present application. In this embodiment, the same or similar contents as the above embodiments can be referred to the above introduction, and no further description will be given later. Before the step of inputting the current state feature vector into the preset neural network decision model to obtain the target operation and maintenance strategy of the service platform, the method includes steps A10 to A20:

[0106] Step A10, constructing a priority playback buffer based on the initial neural network decision model;

[0107] Step A20, iteratively training the initial neural network decision model based on the experience samples stored in the priority playback buffer to obtain the preset neural network decision model.

[0108] It should be noted that the above-mentioned initial neural network decision model can be an untrained PrioritizedDQN network model. Before training, the decision results of the initial neural network decision model in the actual working environment of the service platform can be used to generate experience samples, and a priority replay buffer can be constructed based on the generated experience samples. It should be noted that when the priority replay buffer is initially constructed, the initial priority of each experience sample can be the highest priority, so that each experience sample can be obtained once. And in the subsequent training process, the priority of each experience sample will be gradually updated. Then sample a batch of experience samples from the priority replay buffer (weighted random sampling can be used, and the basis for weighting is the priority of the experience sample, and the higher the priority, the easier it is to sample and use for training), and use the initial neural network decision model for iterative training until the preset training conditions are met, and the initial neural network decision model that has completed the training is used as the preset neural network decision model.

[0109] In a feasible implementation manner, the step of constructing a priority playback buffer based on the initial neural network decision model includes steps A11 to A15:

[0110] Step A11, obtaining historical real-time status information of the service platform in a historical period;

[0111] Step A12, extracting a historical real-time state feature vector from the historical real-time state information through the preset BERT pre-training model;

[0112] Step A13, inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy;

[0113] Step A14, after the service platform executes the historical real-time operation and maintenance strategy, obtaining an execution reward for the historical real-time operation and maintenance strategy and a new historical real-time state feature vector, wherein the execution reward is determined by at least one of a throughput change, a delay change, a resource utilization change, and an error rate change of the service platform after executing the historical real-time operation and maintenance strategy;

[0114] Step A15, generating experience samples based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward and the new historical real-time state feature vector, and constructing a priority playback buffer based on the generated experience samples.

[0115] It should be noted that the above historical period refers to the period before the preset BERT pre-training model and the preset neural network decision model are used to actually make service platform operation and maintenance decisions. In this historical period, the main purpose is to generate experience samples for training the neural network decision model. And the content before generating the experience samples is roughly similar to the process of actually making service platform operation and maintenance decisions.

[0116] Exemplarily, in the historical time period, the historical real-time status information of the service platform can be obtained, wherein the data in the historical real-time status information is similar to the current status information of the above-mentioned service platform, except that the data acquisition period is different. The historical real-time status feature vector is extracted from the historical real-time status information through the above-mentioned preset BERT pre-training model, and the historical real-time status feature vector is input into the initial neural network decision model to obtain the historical real-time operation and maintenance strategy. It is worth noting that the process of obtaining the historical real-time operation and maintenance strategy is similar to the process of obtaining the target operation and maintenance strategy, except that the historical real-time status feature vector and the initial neural network decision model are used to obtain the historical real-time operation and maintenance strategy, and the current status information and the preset neural network decision model are used to obtain the target operation and maintenance strategy, so it will not be repeated here.

[0117] After obtaining the historical real-time operation and maintenance strategy, the service platform executes the historical real-time operation and maintenance strategy, monitors the changes in the performance indicators of the service platform, generates the execution reward of the historical real-time operation and maintenance strategy, and the new environment formed after the service platform executes the historical real-time operation and maintenance strategy. Similarly, the preset BERT pre-trained model is used to extract features of the state information collected in the new environment to obtain a new historical real-time state feature vector. The above-mentioned performance indicators can be at least one of the throughput change, delay change, resource utilization change and error rate change of the service platform. Preferably, the execution reward is calculated by the throughput change, delay change, resource utilization change and error rate change. Exemplarily, the execution reward function R(s,a)=w1Throughput(s,a)-w2Delay(s,a)-w3ResourceUsage(s,a)-w4ErrorRate(s,a). Among them, R(s,a) is the reward for executing action a in state s (such as the historical real-time state feature vector) (such as the historical real-time operation and maintenance strategy), and w1, w2, w3, and w4 are the weight coefficients of each factor. Throughput(s,a) is the reward for the change in throughput of action a in state s. If the throughput increases, a positive reward is given, and if the throughput decreases, a negative reward is given; Delay(s,a) is the reward for the change in delay of action a in state s. If the delay increases, a negative reward is given, and if the delay decreases, a positive reward is given; ResourceUsage(s,a) is the reward for the change in resource utilization of action a in state s. If the utilization is too high or too low, a negative reward is given, and if it is not, a positive reward is given; ErrorRate(s,a) is the reward for the change in error rate of action a in state s. If the error rate decreases, a positive reward is given, and if the error rate increases, a negative reward is given.

[0118] The historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward and the new historical real-time state feature vector are then used to form an experience sample, and the buffer for storing the experience samples with priority is the priority playback buffer.

[0119] For example, the network structure of the initial neural network decision model includes three parts, namely, the input layer, the hidden layer, and the output layer. t As input, after nonlinear transformation of multiple layers of fully connected layers in the hidden layer, the optimal behavior strategy A (i.e., historical real-time operation and maintenance strategy) for the historical real-time state information is inferred, and then the strategy is executed to obtain the feature vector S of the new state. t+1 (i.e., the new historical real-time state feature vector) and reward R (i.e., execution reward), thus forming an experience sample (S t ,A,R,S t+1), and store it in the priority playback buffer. In addition, if the experience sample is used for training in the model training phase, the experience sample can be updated according to the result of the model training, that is, whether the model convergence condition is_end is met, to obtain (S t ,A,R,S t+1 ,is_end), and also store them in the priority playback buffer. For the test samples that meet the model convergence conditions, they may not be sampled for training during the training process.

[0120] In a feasible implementation manner, the step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain the historical real-time operation and maintenance strategy includes steps A131 to A132:

[0121] Step A131, inputting the historical real-time state feature vector into the initial neural network decision model, and predicting the predicted execution reward of each candidate action in the preset action space under the environment corresponding to the historical real-time state feature vector through the initial neural network decision model;

[0122] Step A132, taking the candidate action with the largest predicted execution reward among the candidate actions as the historical real-time operation and maintenance strategy.

[0123] It should be noted that, in this embodiment, an action space corresponding to the operation and maintenance strategy can be constructed first. Different actions in the action space represent different operation and maintenance strategies. For example, the type of operation and maintenance strategy can adjust network bandwidth, increase server resources, optimize routing, etc. In actual applications, technicians can set different operation and maintenance strategies according to actual needs, which will not be repeated here.

[0124] Exemplarily, the historical real-time state feature vector can be input into the initial neural network decision model, and the initial neural network decision model predicts the execution reward of each candidate action in the preset action space under the environment corresponding to the historical real-time state feature vector, that is, the above-mentioned predicted execution reward is obtained, and then the candidate action with the largest predicted execution reward is selected from each candidate action as the above-mentioned historical real-time operation and maintenance strategy. Similarly, the process of obtaining the target operation and maintenance strategy of the service platform in actual application is basically similar to the above content.

[0125] In a feasible implementation manner, the initial neural network decision model includes a main Q network and a target Q network, the model parameters in the main Q network will be updated to the target Q network according to a preset period, and the initial neural network decision model is iteratively trained based on the experience samples stored in the priority playback buffer to obtain the preset neural network decision model. The steps include steps A21 to A25:

[0126] Step A21, collecting some experience samples from the priority playback buffer to obtain candidate experience samples;

[0127] Step A22, for any one of the candidate experience samples, calculating the target Q value of the candidate experience sample through the target Q network;

[0128] Step A23, calculating the predicted Q value of the candidate experience sample through the main Q network;

[0129] Step A24, updating the model parameters in the main Q network based on the difference between the target Q value and the predicted Q value, and updating the priority of the candidate experience sample according to the difference between the target Q value and the predicted Q value, and then putting the candidate experience sample with the updated priority back into the priority playback buffer;

[0130] Step A25, after the main Q network in the initial neural network decision model meets the preset model training conditions, the trained initial neural network decision model is used as the preset neural network decision model.

[0131] It should be noted that, in this embodiment, the initial neural network decision model includes a main Q network and a target Q network, wherein the target Q network can be a copy of the main Q network, and the model parameters in the main Q network will be updated to the target Q network according to a preset period, that is, after the update period is reached, the model parameters in the main Q network can be copied to the target Q network to complete the update of the target Q network.

[0132] Exemplarily, some experience samples are collected from the priority playback buffer to obtain candidate experience samples for the current round of training. The higher the priority of the experience sample, the higher the probability of being collected as a candidate experience sample, and for the initial priority playback buffer, the priorities of the experience samples existing therein are all the same and are all the highest. Since the process of using each candidate experience sample for training is roughly the same, in this embodiment, a candidate experience sample will be used as an example for explanation. The target Q value of the candidate experience sample is calculated by the target Q network.

[0133] Wherein, the target Q value calculation formula is as follows:

[0134]

[0135] In the formula, R j is the reward (i.e., the execution reward in candidate experience sample j), γ is the decay factor, Q′ is the target Q network, Q is the main Q network, is the state feature vector in the candidate experience sample j (such as the historical real-time state feature vector), The feature vector of the new state obtained by executing action A (i.e., the new historical real-time state feature vector in candidate experience sample j), A is the executed action (i.e., the historical real-time operation and maintenance strategy in candidate experience sample j), w′ is the model parameter of the target Q network, w is the model parameter of the main Q network, y j is the target Q value, The characterization model converges to true, Characterizes model convergence as False.

[0136] The predicted Q value of the candidate experience sample is calculated by the main Q network. The predicted Q value is the benefit of the decision made by the main Q network in the environment corresponding to the historical real-time state feature vector in the candidate experience sample j, that is, the execution reward of each candidate action in the environment corresponding to the historical real-time state feature vector in the candidate experience sample j is predicted by the main Q network, and the highest execution reward is the above-mentioned predicted Q value. The model parameters in the main Q network are updated based on the difference between the target Q value and the predicted Q value. Among them, the difference between the target Q value and the predicted Q value can be characterized by a loss function. For example, the loss function can be a minimum mean square error loss function, and the loss function formula can be as follows:

[0137] Where m is the number of samples for batch gradient descent, w j is the loss function weight, y j is the target Q value of the jth candidate experience sample, Q is the main Q network, is the state feature vector in the candidate experience sample j (such as the historical real-time state feature vector), A j The operation and maintenance strategy adopted by the main Q network for the environment corresponding to the historical real-time state feature vector in the candidate experience sample j, and w is the model parameter of the main Q network. To predict the Q value.

[0138] The gradient can be determined through the above loss function, and then all model parameters w of the main Q network are updated based on the gradient back propagation, thereby completing an update. It is worth noting that after obtaining the difference between the above target Q value and the above predicted Q value, the priority of the above candidate experience samples can be updated based on the difference, wherein the priority of the candidate experience sample is proportional to the difference between the above target Q value and the above predicted Q value.

[0139] The above process is repeated cyclically until the main Q network in the initial neural network decision model meets the preset model training conditions, and the preset model training may be reaching the preset training times and model convergence, etc. The initial neural network decision model that meets the preset model training conditions can be used as the above preset neural network decision model.

[0140] In addition, refer to Figure 4 , is a schematic diagram of the model framework involved in the platform operation and maintenance method of this application. Figure 4 As shown, data collection, data denoising and data preprocessing steps are involved in the early stage. The preprocessed results will be used for fine-tuning of Bert (i.e., Bert pre-trained model), and the fine-tuning tasks may include network traffic pattern recognition tasks, network fault prediction tasks and network occupancy prediction tasks. After fine-tuning, the preset Bert pre-trained model can be obtained. The preset Bert pre-trained model will extract the state (i.e., current state information or historical real-time state information) from the environment of the service platform, and then input the state into the deep neural network (i.e., the above-mentioned preset neural network decision model or the initial neural network decision model), so that the deep neural network selects actions (i.e., operation and maintenance strategies) from the action space to execute, such as action 3, and generates impacts and rewards on the environment, and these contents will be used to form experience samples and stored in the priority playback buffer.

[0141] It can be understood that in this embodiment, by fine-tuning BERT and combining it with PrioritizedDQN to strengthen the neural network, multiple factors and features can be comprehensively considered. As a pre-trained model, BERT can extract semantic and contextual features in network data by learning language and context information. PrioritizedDQN can comprehensively consider factors such as network traffic patterns, fault signs, and network resource occupancy based on these features and real-time network conditions to derive the optimal operation and maintenance strategy. Compared with traditional operation solutions, it can respond to network changes and faults more promptly, and make intelligent decisions based on pre-trained and fine-tuned knowledge, thereby improving the efficiency and quality of network operations.

[0142] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the platform operation and maintenance method in the above-mentioned embodiment one.

[0143] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as computers. Figure 5The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0144] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0145] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0146] The electronic device provided by the present application adopts the platform operation and maintenance method in the above embodiment, which can solve the technical problem that the existing operation and maintenance methods are highly dependent on manual labor. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the platform operation and maintenance method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0147] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0148] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0149] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the platform operation and maintenance method in the above-mentioned embodiment.

[0150] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0151] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0152] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device:

[0153] Obtaining current status information of the service platform;

[0154] Extracting a current state feature vector from the current state information by using a preset BERT pre-trained model, wherein the preset BERT pre-trained model is fine-tuned by a preset fine-tuning operation and maintenance task;

[0155] The current state feature vector is input into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform.

[0156] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0158] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0159] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned platform operation and maintenance method, and can solve the technical problem that the existing operation and maintenance methods are highly dependent on manual labor. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the platform operation and maintenance method provided in the above-mentioned embodiment, and will not be repeated here.

[0160] The present application also provides a computer program product, including a computer program, which implements the steps of the platform operation and maintenance method as described above when executed by a processor.

[0161] The computer program product provided in this application can solve the technical problems of platform operation and maintenance. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the platform operation and maintenance method provided in the above embodiment, which will not be repeated here.

[0162] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A platform operation and maintenance method, characterized in that: Applied to the service platform, the platform operation and maintenance method comprises the following steps: Obtaining current status information of the service platform; Extracting a current state feature vector from the current state information by using a preset BERT pre-trained model, wherein the preset BERT pre-trained model is fine-tuned by a preset fine-tuning operation and maintenance task; The current state feature vector is input into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform.

2. The platform operation and maintenance method according to claim 1, characterized in that: Before the step of extracting the current state feature vector from the current state information by using the preset BERT pre-training model, the method includes: Acquire historical network traffic data of the service platform, wherein the historical network traffic data is acquired from at least one of a network packet capture tool, a traffic monitoring tool, a network security log, or a network routing log; After the historical network traffic data is labeled based on the preset fine-tuning operation and maintenance task, feature extraction is performed on the labeled historical network traffic data to obtain a training feature sample set; The initial BERT pre-training model is fine-tuned based on the training feature sample set to obtain the preset BERT pre-training model.

3. The platform operation and maintenance method according to claim 2, characterized in that: The preset fine-tuning operation and maintenance tasks include a traffic pattern recognition task, a network fault recognition task, and a resource occupancy prediction task. The label types for annotation include a pattern label corresponding to the traffic pattern recognition task, a fault label corresponding to the network fault recognition task, and a resource occupancy label corresponding to the resource occupancy prediction task. The steps of fine-tuning the initial BERT pre-training model based on the training feature sample set to obtain the preset BERT pre-training model include: Building a traffic pattern recognition model based on the initial BERT pre-training model; Inputting the training feature samples in the training feature sample set into the traffic pattern recognition model to obtain a traffic pattern recognition result; Based on the difference between the traffic pattern recognition result and the pattern label corresponding to the training feature sample, the traffic pattern recognition model is iteratively trained, and after the traffic pattern recognition model training is completed, a first fine-tuned BERT pre-trained model is obtained by fine-tuning the initial BERT pre-trained model; Building a network fault identification model based on the first fine-tuned BERT pre-trained model; Inputting the training feature samples in the training feature sample set into the network fault identification model to obtain a network fault identification result; Based on the difference between the network fault identification result and the fault label corresponding to the training feature sample, the network fault identification model is iteratively trained, and after the network fault identification model training is completed, a second fine-tuned BERT pre-trained model is obtained by fine-tuning the first fine-tuned BERT pre-trained model; Building a resource occupancy prediction model based on the second fine-tuned BERT pre-trained model; Inputting the training feature samples in the training feature sample set into the resource occupancy prediction model to obtain a resource occupancy prediction result; Based on the difference between the resource occupancy prediction result and the resource occupancy label corresponding to the training feature sample, the resource occupancy prediction model is iteratively trained, and after the resource occupancy prediction model training is completed, a preset BERT pre-trained model is obtained after fine-tuning the second fine-tuning BERT pre-trained model.

4. The platform operation and maintenance method according to claim 1, characterized in that: The current state information includes current network traffic data of the service platform and current system state data of the service platform, and the step of extracting the current state feature vector from the current state information by using the preset BERT pre-training model includes: Extracting a traffic feature vector from the current network traffic data through the preset BERT pre-training model; Based on the dimensional characteristics of the traffic feature vector, the system state data is reconstructed to obtain an array in a form that conforms to the last dimensional characteristics of the traffic feature vector; The array is combined with the last dimension feature in the traffic feature vector to obtain a current state feature vector.

5. The platform operation and maintenance method according to claim 1, characterized in that: Before the step of inputting the current state feature vector into a preset neural network decision model to obtain the target operation and maintenance strategy of the service platform, the method includes: Based on the initial neural network decision model, a priority playback buffer is constructed; Based on the experience samples stored in the priority playback buffer, the initial neural network decision model is iteratively trained to obtain the preset neural network decision model.

6. The platform operation and maintenance method according to claim 5, characterized in that: The step of constructing a priority playback buffer based on the initial neural network decision model includes: Under the historical time period, obtain the historical real-time status information of the service platform; Extracting a historical real-time state feature vector from the historical real-time state information by using the preset BERT pre-training model; Inputting the historical real-time state feature vector into the initial neural network decision model to obtain a historical real-time operation and maintenance strategy; After the service platform executes the historical real-time operation and maintenance strategy, an execution reward of the historical real-time operation and maintenance strategy and a new historical real-time state feature vector are obtained, wherein the execution reward is determined by at least one of a throughput change, a delay change, a resource utilization change, and an error rate change of the service platform after executing the historical real-time operation and maintenance strategy; An experience sample is generated based on the historical real-time state feature vector, the historical real-time operation and maintenance strategy, the execution reward, and the new historical real-time state feature vector, and a priority playback buffer is constructed based on the generated experience sample.

7. The platform operation and maintenance method according to claim 6, characterized in that: The step of inputting the historical real-time state feature vector into the initial neural network decision model to obtain the historical real-time operation and maintenance strategy includes: Inputting the historical real-time state feature vector into the initial neural network decision model, and predicting the predicted execution reward of each candidate action in the preset action space under the environment corresponding to the historical real-time state feature vector through the initial neural network decision model; The candidate action with the largest predicted execution reward among the candidate actions is used as the historical real-time operation and maintenance strategy.

8. The platform operation and maintenance method according to claim 6, characterized in that: The initial neural network decision model includes a main Q network and a target Q network, the model parameters in the main Q network will be updated to the target Q network according to a preset period, and the steps of iteratively training the initial neural network decision model based on the experience samples stored in the priority playback buffer to obtain the preset neural network decision model include: Collecting some experience samples from the priority playback buffer to obtain candidate experience samples; For any one of the candidate experience samples, calculating a target Q value of the candidate experience sample by using the target Q network; Calculate the predicted Q value of the candidate experience sample through the main Q network; The model parameters in the main Q network are updated based on the difference between the target Q value and the predicted Q value, and the priority of the candidate experience sample is updated according to the difference between the target Q value and the predicted Q value, and then the candidate experience sample with the updated priority is placed back into the priority playback buffer; After the main Q network in the initial neural network decision model meets the preset model training conditions, the trained initial neural network decision model is used as the preset neural network decision model.

9. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of platform operation and maintenance as claimed in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of platform operation and maintenance as described in any one of claims 1 to 8 are implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of platform operation and maintenance as claimed in any one of claims 1 to 8 are implemented.

Images