Operation and maintenance learning piece model determination method, system and equipment, medium and product

Abstract

The invention relates to the technical field of business support, and provides an operation and maintenance learning piece model determination method, system and device, a medium and a product, and the method comprises the steps: obtaining a scene description of a target operation and maintenance scene, and determining a scene feature value of the scene description; determining a target learning piece model corresponding to the target operation and maintenance scene according to the scene feature value and a label corresponding to at least one pre-trained learning piece model; wherein the learning piece comprises a pre-training model and a protocol used for representing a model intention and / or model characteristics of the pre-training model, and the learning piece model is obtained by training the learning piece through an index feature training set corresponding to at least one operation and maintenance scene and at least one training algorithm. According to the method, the target learning item model corresponding to the target operation and maintenance scene is selected from the pre-trained learning item models, so that the matching degree of the operation and maintenance scene and the learning item model is improved, the workload of operation and maintenance personnel is reduced, and the operation and maintenance efficiency is improved while the operation and maintenance accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of business support technology, and in particular to a method, system, device, medium and product for determining an operation and maintenance learning model. Background Art

[0002] Most of the current intelligent operation and maintenance systems are dedicated to solving a certain type of operation and maintenance technical problems or solving specific operation and maintenance technical problems in a certain scenario. For example, an intelligent operation and maintenance system for massive log data includes a log collection module, a log processing module (intelligent model processing), and a log storage module. It can manage massive logs and avoid the occurrence of alarm storms. Another example is an intelligent operation and maintenance monitoring method for an IT system and its intelligent operation and maintenance monitoring system, which can match business information in a specific scenario with the intelligent model, and use the matched intelligent model to perform operation and maintenance tasks.

[0003] Most of the existing operation and maintenance systems are built according to specific scenarios and are based on the scenario information at a certain moment. As the data information in the scenario continues to change, the originally built operation and maintenance system may no longer be applicable. At this time, the original system needs to be repaired, which puts great pressure on the operation and maintenance personnel. Alternatively, an operation and maintenance system suitable for the existing scenario needs to be rebuilt, which takes a long time and does not meet the actual situation. Summary of the invention

[0004] The present application provides a method, system, device, medium and product for determining an operation and maintenance learning model, which is used to solve the defects in the prior art that it takes a long time to re-establish a new operation and maintenance system or to improve the original operation and maintenance system, which does not meet the actual needs and causes great pressure on the operation and maintenance personnel.

[0005] In a first aspect, the present application provides a method for determining an operation and maintenance learning piece model, comprising: obtaining a scenario description of a target operation and maintenance scenario, and determining a scenario feature value of the scenario description; determining a target learning piece model corresponding to the target operation and maintenance scenario based on the scenario feature value and a label corresponding to at least one pre-trained learning piece model; wherein the learning piece comprises a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning piece model is obtained by training the learning piece using an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0006] As an embodiment, the training step of the learning model includes: obtaining data information of a preset data source, selecting corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario, and obtaining the indicator feature training set according to the indicator features; determining at least one training algorithm of the pre-trained model and the hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; training each of the intermediate models based on the indicator feature training set, and taking the optimal model as the learning model, or uploading the indicator feature training set to a preset platform so that the preset platform trains each of the intermediate models according to the indicator feature training set and takes the optimal model as the learning model, and obtaining the learning model returned by the preset platform; wherein the label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

[0007] As an embodiment, obtaining the indicator feature training set based on the indicator feature includes: performing full data feature statistics on the indicator feature, clustering and grouping the indicator feature according to the statistical results to obtain at least one feature grouping; and selecting data that meets preset conditions in each of the feature groups to obtain the indicator feature training set.

[0008] As an embodiment, before selecting the corresponding indicator feature from the data information according to the indicator information corresponding to the operation and maintenance scenario, it also includes: preliminarily screening the data source according to preset reliable data source information to obtain a first intermediate data source; performing secondary screening on the first intermediate data source based on preset screening rules or manual screening methods to obtain a second intermediate data source; performing regular data monitoring on the second intermediate data source based on a preset monitoring time interval to obtain a target data source; correspondingly, selecting the corresponding indicator feature from the data information according to the indicator information corresponding to the operation and maintenance scenario includes: selecting the corresponding indicator feature from the target data source according to the indicator information corresponding to the operation and maintenance scenario.

[0009] As an embodiment, after determining the target learning model corresponding to the target operation and maintenance scenario, it also includes: executing at least one operation of publishing, starting, pausing, restarting or going offline in the target operation and maintenance scenario based on the target learning model to obtain an execution result corresponding to the target operation and maintenance scenario.

[0010] In the second aspect, the present application also provides an operation and maintenance learning material model determination system, including a scenario management module and a learning material model management module; the scenario management module is used to obtain a scenario description of a target operation and maintenance scenario and determine a scenario feature value of the scenario description; according to the scenario feature value and a label corresponding to at least one pre-trained learning material model, a target learning material model corresponding to the target operation and maintenance scenario is determined; the learning material model management module is used to pre-train the learning material model, the learning material includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning material model is obtained by training the learning material with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0011] As an embodiment, the learning model management module and the scenario management module are arranged in a visual interface, and the learning model management module includes a data source management module, an indicator set management module, a training set management module and a learning model generation module; the data source management module is used to manage the data source; the indicator set management module is used to select corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario; the training set management module is used to obtain the indicator feature training set according to the indicator feature; the learning model generation module is used to determine at least one training algorithm of the pre-trained model and the hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; each of the intermediate models is trained based on the indicator feature training set, and the optimal model is used as the learning model, or the indicator feature training set is uploaded to a preset platform so that the preset platform trains each of the intermediate models according to the indicator feature training set and uses the optimal model as the learning model, and obtains the learning model returned by the preset platform; wherein the label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

[0012] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any one of the above-described methods for determining an operation and maintenance learning model.

[0013] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining an operation and maintenance learning model as described in any one of the above.

[0014] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for determining an operation and maintenance learning model.

[0015] The operation and maintenance learning piece model determination method, system, device, medium and product provided in the present application obtain the scenario description of the target operation and maintenance scenario, determine the scenario feature value of the scenario description; determine the target learning piece model corresponding to the target operation and maintenance scenario according to the scenario feature value and the label corresponding to at least one pre-trained learning piece model; wherein the learning piece includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning piece model is obtained by training the learning piece with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm. The present application selects the target learning piece model corresponding to the target operation and maintenance scenario from the pre-trained learning piece models, improves the matching degree between the operation and maintenance scenario and the learning piece model, reduces the workload of the operation and maintenance personnel, and improves the efficiency of the operation and maintenance while improving the accuracy of the operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is one of the flow charts of the method for determining the operation and maintenance learning model provided in this application.

[0018] Figure 2 It is a schematic diagram of the matching between the operation and maintenance scenario provided by this application and the learning model.

[0019] Figure 3 This is the second flow chart of the method for determining the operation and maintenance learning model provided in this application.

[0020] Figure 4 It is a structural diagram of the operation and maintenance learning model determination system provided by this application.

[0021] Figure 5 This is a page diagram of the data source management module provided by this application.

[0022] Figure 6a This is one of the page diagrams of the indicator set management module provided by this application. Figure 6b This is the second page diagram of the indicator set management module provided by this application.

[0023] Figure 7 This is a page diagram of the data exploration provided by this application.

[0024] Figure 8 This is a page diagram of the training set management module provided by this application.

[0025] Figure 9a~Figure 9f These are page diagrams one to six generated by the learning model provided in this application.

[0026] Fig.10a This is one of the page diagrams of the scene management module provided by this application. Fig.10b This is the second page diagram of the scene management module provided by this application.

[0027] Fig.11 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0030] Most of the existing operation and maintenance systems are built according to specific scenarios, and have the following disadvantages: the production process is cumbersome and time-consuming, and the operation and maintenance model cannot be intelligently produced through a visual page; the data collection quality is not high, resulting in poor operation and maintenance effects of the final operation and maintenance model, causing resource waste and affecting the accuracy of operation and maintenance; it is necessary to produce a specific operation and maintenance system that meets the operation and maintenance needs for each scenario, which results in the original operation and maintenance system being unable to meet the corresponding needs of the existing scenario when the scene is switched, and the time spent on re-formulating a new operation and maintenance system or improving the original operation and maintenance system is relatively long, which does not meet the actual needs; the model needs to be trained through a training set, however, the data set in the resource library is too large. If all the data is used for training, the training time will be long, and the data is not targeted enough to train the scene data well, which ultimately leads to poor model effects.

[0031] In order to solve at least one of the above-mentioned defects, the present application provides a method, system, device, medium and product for determining an operation and maintenance learning model. Figure 1-Figure 11 The operation and maintenance learning model determination method, system, equipment, medium and product provided in this application are described in detail.

[0032] Figure 1This is one of the flow charts of the method for determining the operation and maintenance learning model provided by this application, such as Figure 1 As shown, the present application provides a method for determining an operation and maintenance learning model, comprising the following steps S100 to S200.

[0033] Step S100, obtaining a scenario description of a target operation and maintenance scenario, and determining a scenario feature value of the scenario description; the scenario description of the target operation and maintenance scenario is provided by an operation and maintenance personnel, and optionally, the scenario description of the target operation and maintenance scenario input by the operation and maintenance personnel can be obtained through a human-computer interaction module (such as a visual interface). The scenario feature value is a keyword extracted from the scenario description for characterizing the operation and maintenance scenario.

[0034] Step S200, determining a target learning model corresponding to the target operation and maintenance scenario based on the scenario feature value and the label corresponding to at least one pre-trained learning model; specifically, matching the scenario feature value with the label of the learning model can be performed, and the learning model with the highest matching degree can be used as the target learning model corresponding to the target operation and maintenance scenario.

[0035] The learning component includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model. The learning component model is obtained by training the learning component with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0036] Learnware is a pre-trained machine learning model with good performance, which has a set of specifications that explain the model's intent and / or characteristics. That is, learnware = model + specification. The model needs to meet three characteristics: reusable, evolvable and understandable. The specification can give a suitable characterization of the model, which can be a logic-based description, and / or reveal the statistical data of the target targeted by the model, and / or even some simplified training samples that can describe the model's application scenario. The owner of the learnware can put it on the market with almost no risk of data privacy leakage, and it can play an indispensable and important role in operation and maintenance.

[0037] It can be understood that the present application produces learning models suitable for the needs of various operation and maintenance systems through pre-training, selects the model that best suits the current operation and maintenance scenario from multiple learning models based on the scenario description of the target operation and maintenance scenario, and uses this model in the scenario to perform operation and maintenance tasks, thereby improving the matching degree between the model and the scenario, and thereby improving the operation and maintenance efficiency in specific scenarios.

[0038] Based on the above embodiment, as an optional embodiment, the training step of the learning model includes steps S010 to S030.

[0039] Step S010, obtain data information of a preset data source, select corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario, and obtain the indicator feature training set according to the indicator features; data source types include message middleware, relational database, FTP (file transfer protocol), text, etc., and data from all data sources will be saved in clickhouse, and the data of the indicator feature training set comes from the data in the table in clickhouse. Select the indicator information required for pre-training the learning model from the selected data source, and the indicator information used in different learnings for the same data source may be different.

[0040] Step S020, determine at least one training algorithm of the pre-trained model and the hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; the training algorithm may include LSTM, ISOLATIONFOREST, DBSCAN, SARIMA, etc.

[0041] Step S030, train each of the intermediate models based on the indicator feature training set, and use the optimal model as the learning model, or upload the indicator feature training set to a preset platform so that the preset platform can train each of the intermediate models according to the indicator feature training set and use the optimal model as the learning model, and obtain the learning model returned by the preset platform. Specifically, the training can be performed on the execution subject of the present application, and can also be performed in a dedicated model training platform, and the selection can be made based on the computing power and resources of the execution subject of the present application.

[0042] The label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

[0043] It is understandable that in the prior art, the optimal model generated by the learning piece under each algorithm usually requires operation and maintenance personnel to select a suitable model based on specific scenario requirements. However, such a selection method relies too much on the past experience of the operation and maintenance personnel and is prone to making wrong judgments. The present application can intelligently recommend suitable models based on the operation and maintenance scenarios, greatly improving the matching degree between operation and maintenance and scenarios, so that the learning piece model can be better invested in scenario operation and maintenance.

[0044] Based on the above embodiment, as an optional embodiment, before selecting corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario, it also includes steps S011 to S013.

[0045] Step S011, preliminarily screening the data source according to preset reliable data source information to obtain a first intermediate data source; specifically, preliminarily screening the data source according to logs, monitoring indicators, performance data, etc. to obtain a first intermediate data source.

[0046] Step S012, based on the preset screening rules or manual screening methods, the first intermediate data source is screened for a second time to obtain the second intermediate data source; specifically, the screening rules can be pre-set, and the massive data can be intelligently screened by combining artificial intelligence and according to the pre-set rules; or, a visual manual screening method can be used to output a visual regression function according to the artificial expected value, and the acquired data can be presented in the form of a scatter plot, and the operation and maintenance personnel can perform a secondary screening according to the visual interface. In other embodiments, the first intermediate data source can also be screened for integrity screening (screening out incomplete data), accuracy screening (screening out inaccurate and missing data), etc. to obtain the second intermediate data source.

[0047] Step S013, based on a preset monitoring time interval, periodic data monitoring is performed on the second intermediate data source to obtain a target data source; the monitoring time interval can be preset, and periodic data monitoring is performed on the screened second intermediate data source at regular intervals, including monitoring whether there is data failure / expiration, whether there is data abnormality, and whether the data volume of the second intermediate data source is sufficient (there may be insufficient data volume after eliminating abnormal data, and new data needs to be added at this time), and finally the target data source is obtained. By periodically monitoring the data, the high quality of the data is further ensured.

[0048] Correspondingly, selecting corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario includes: selecting corresponding indicator features from the target data source according to the indicator information corresponding to the operation and maintenance scenario.

[0049] It can be understood that the present application selects high-quality data from the data source used to train the learning model through layer-by-layer screening and timed screening methods, obtains the target data source, and improves the accuracy of learning model training.

[0050] Based on the above embodiment, as an optional embodiment, obtaining the indicator feature training set according to the indicator feature includes steps S014 and S015.

[0051] Step S014, performing full data feature statistics on the indicator features, and clustering and grouping the indicator features according to the statistical results to obtain at least one feature grouping.

[0052] Step S015, select data that meets the preset conditions in each of the feature groups to obtain the indicator feature training set. The preset condition is that the difference between the mean and the average mean of the group is the smallest. The obtained training set can be more targeted, and the accuracy of the learning model finally generated is higher.

[0053] It can be understood that the present application performs statistics on all data features of indicator features and clusters them, selects data that meets preset conditions to construct a training set, so that the training set better meets the training requirements and further improves the accuracy of learning model training.

[0054] Based on the above embodiment, as an optional embodiment, after determining the target learning model corresponding to the target operation and maintenance scenario, step S300 is also included.

[0055] Step S300, based on the target learning model, at least one operation of publishing, starting, pausing, restarting or going offline is executed in the target operation and maintenance scenario to obtain an execution result corresponding to the target operation and maintenance scenario.

[0056] The learning tool provides available learning tool models. After configuring the learning tool model in the scenario, the scenario needs to be published. The learning tool model configured in the scenario is loaded on the algorithm platform, and the operation and maintenance scenario is started. After the flink platform starts a task to acquire, clean, and transform the test data, the learning tool model is called to perform data detection, and the test results are obtained to display the effect in the scenario. During this period, the execution can be suspended and offline. By canceling the flink task, the call detection of the learning tool model can be stopped. To perform the offline operation, you need to pause the scene first, and then offline the learning tool model.

[0057] It should be noted that one learningware can generate multiple learningware models. All learningware models are generated by the same data source, indicator information, and training set. The difference lies in the training algorithm used and the hyperparameters corresponding to the training algorithm. After the learningware model training is completed, a better model can be selected from the learningware model for use in the operation and maintenance scenario. The result parameters of the learningware models generated by the same training algorithm with different hyperparameters in the learningware are the same, so the quality of the learningware model can be judged by the result parameters. The result parameters generated by different training algorithms may be different. The learningware can provide the optimal model generated by each training algorithm for use in the operation and maintenance scenario.

[0058] The multiple learning models in one learning item are shown in the following table.

[0059]

[0060] Figure 2 This is a schematic diagram of the matching of the operation and maintenance scenario provided by this application with the learning model, such as Figure 2As shown in the figure, there are two learning pieces, each of which uses multiple training algorithms, each algorithm generates multiple learning piece models, and selects the optimal learning piece model for each algorithm for the operation and maintenance scenario. The optimal model of each algorithm can be provided in one learning piece, and only one model of one learning piece can be selected in each operation and maintenance scenario for release and use in the operation and maintenance scenario. Learning pieces can be applied to multiple operation and maintenance scenarios, and one learning piece contains multiple learning piece models, so each learning piece can be used in anomaly detection scenarios and capacity prediction scenarios.

[0061] Different training algorithms can be applied to different scenarios. For example, the ISOLATIONFOREST algorithm is used for anomaly detection, and the DBSCAN algorithm is used for capacity prediction. Algorithm labels can be set for the algorithms in each learning component. The training set of each learning component has a corresponding scenario label. The suitable model for a specific scenario can be selected from multiple learning components based on the scenario label and algorithm label. This not only improves the accuracy of model selection, but also reduces the operation and maintenance burden for operation and maintenance personnel and improves operation and maintenance efficiency.

[0062] Exemplarily, training set 1 is used in learning piece 1, and training set 1 contains scene feature values ​​that identify scenes, and each algorithm used in learning piece 1 corresponds to a different algorithm label. After the learning piece model is generated, the operation and maintenance personnel input the scene description according to the current scene, and the scene feature values ​​can be matched with the scene feature values ​​and algorithm labels, and finally a model matching the current scene is obtained. For example, the scene feature values ​​contained in training set 1 are network, transaction, virtual, and currency, and the algorithm label of the ISOLATIONFOREST algorithm in learning piece 1 is abnormal. The scene description entered by the operation and maintenance personnel is "anomaly detection in network transactions". By extracting keywords from the scene description entered by the operation and maintenance personnel, scene features such as "network", "transaction" and "anomaly detection" can be obtained, and by intelligently matching the scene feature values ​​and algorithm labels of multiple learning piece models, it is finally determined that the model under this scene is the optimal model under the ISOLATIONFOREST algorithm in learning piece 1.

[0063] It can be understood that the present application reduces the model selection made by operation and maintenance personnel based on empiricism through the intelligent selection of learning material models, and avoids the disadvantage that operation and maintenance personnel are unable to correctly select learning material models in a short period of time when a large number of learning materials are generated. While intelligently generating learning material models, it improves the matching degree between learning material models and specific scenarios, thereby improving the operation and maintenance work efficiency of operation and maintenance personnel.

[0064] Figure 3 This is the second flow chart of the method for determining the operation and maintenance learning model provided by this application, such as Figure 3 As shown, the steps of the preferred embodiment of the present application for determining the operation and maintenance learning model are as follows.

[0065] (1) Get the data information of the preset data source. The data source types include message middleware, relational database, FTP, text, etc. The data of all configured data sources will be saved in ClickHouse. Get the data information of the preset data source means to get the data information of the preset data source from ClickHouse.

[0066] (2) Select the required indicator information from the data source. The indicator information used in different learning materials for the same data source may be different.

[0067] (3) Perform full data feature statistics on the selected indicator information, perform clustering based on the statistical features, and obtain multiple feature groups.

[0068] (4) Based on the data source, indicator information, ETL and other information, select the indicator with the smallest difference between the mean and the average mean of the group in each feature group to construct a training set. The training set can be constructed by using data of different types and sources, so that the obtained training set can be more targeted and the final generated model can have a higher accuracy.

[0069] (5) Select training algorithms and hyperparameters for training. Specifically, the flink platform filters and transforms the data of the data source according to the configuration of the data source, indicators, and ETL information to obtain a training set, and trains the pre-trained model of the learning component according to the training set, or uploads the training set to the model training platform, and the model training platform trains the pre-trained model of the learning component. The training set can be used by multiple learning components, and multiple training algorithms can be selected before training, and multiple sets of hyperparameters can also be selected for the same training algorithm. In other embodiments, the relevant configuration information, indicator information, and ETL information of the data source can also be directly uploaded to the model training platform, and the model training platform processes the data of the data source, constructs a training set, and trains the pre-trained model that determines the training algorithm and hyperparameters according to the training set.

[0070] (6) After the training is completed, the optimal model is selected based on the result parameters. Specifically, the result parameters of the learning models corresponding to different training algorithms can be viewed in the learning software, and the optimal model can be obtained based on the predetermined result parameter evaluation criteria.

[0071] (7) Obtain the scenario description input by the operation and maintenance personnel, and intelligently select a model based on the scenario feature values, algorithm labels, and scenario labels. Specifically, extract keywords from the scenario description input by the operation and maintenance personnel, and compare them with the scenario feature values, algorithm labels, and scenario labels in each model. Select the one with the highest match as the target learning model for the target operation and maintenance scenario.

[0072] The above steps can be run in a visual interface, which solves the inefficiency and inconvenience caused by invisible operations in the construction of operation and maintenance systems using various algorithms and frameworks in traditional technologies. By pre-training the learning model, the convenience of operation is further enhanced and the operation and maintenance efficiency is improved. Intelligent selection can be made based on the scenario description of the target operation and maintenance scenario entered by the operation and maintenance personnel, so that the final learning model used is more in line with the scenario needs, further improving the operation and maintenance efficiency. By screening the indicator features, it is avoided that all training data in the database is used for model training, which improves the training efficiency of the model.

[0073] The operation and maintenance learning material model determination system provided in the present application is described below. The operation and maintenance learning material model determination system described below and the operation and maintenance learning material model determination method described above can be referenced to each other.

[0074] Figure 4 is a structural diagram of the operation and maintenance learning model determination system provided by this application, such as Figure 4 As shown, the present application also provides an operation and maintenance learning model determination system, including a scenario management module 410 and a learning model management module 420.

[0075] The scenario management module 410 is used to obtain a scenario description of a target operation and maintenance scenario and determine a scenario feature value of the scenario description; and determine a target learning model corresponding to the target operation and maintenance scenario based on the scenario feature value and a label corresponding to at least one pre-trained learning model.

[0076] The learningware model management module 420 is used to pre-train the learningware model. The learningware includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model. The learningware model is obtained by training the learningware with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0077] As an embodiment, the learning model management module 420 and the scenario management module 410 are arranged in a visual interface, and the learning model management module includes a data source management module 421 , an indicator set management module 422 , a training set management module 423 and a learning model generation module 424 .

[0078] The data source management module 421 is used to manage data sources; managing data sources specifically refers to configuring and managing the indicator data source to be connected. The data source may include a kafka data source, a text data source, an ftp data source, other databases and other data sources.

[0079] Figure 5 This is a page diagram of the data source management module provided by this application, such as Figure 5As shown, the data source management module can configure the basic information of the data source and the specific connected data source as needed. The role of the relevant data of the training set is very important, which determines the quality level of the learning model obtained by subsequent training. Obtaining high-quality data can improve the accuracy of the learning model. High-quality data is the basis for making high-quality learning models. It is necessary to collect high-quality data as the data required for model training and prediction.

[0080] The following factors need to be considered in the selection of high-quality data: the source of the data should be reliable; data integrity: the data should be complete, without missing important fields or information; data accuracy: the data should be accurate, without problems such as manual entry errors or system errors; data timeliness: the data should be updated in a timely manner, without a large lag time; the data volume should be large enough: the data volume needs to be large enough to ensure that the model has sufficient training samples and can cover various operation and maintenance scenarios; data quality monitoring: regular monitoring of the collected data to ensure that the data quality meets the requirements, such as whether there are outliers, whether the data is outdated, etc.

[0081] According to the above factors, the data source management module can also set a data screening function. Specifically, according to reliable data source information, such as logs, monitoring indicators, performance data, etc., high-quality data is preliminarily screened to obtain the first intermediate data source; the first intermediate data source is screened for information, and the available methods include intelligent screening, specifically, pre-setting relevant screening rules, and intelligently screening the massive data by combining artificial intelligence and pre-set rules; or, a visual manual screening method can be used to output a visual regression function according to the artificial expectation value, and the acquired data is presented in the form of a scatter plot, and the operation and maintenance personnel can screen the first intermediate data source according to the visual interface. Through including but not limited to the above two methods, the data is further screened, including integrity screening (screening out incomplete data), accuracy screening (screening out inaccurate and missing data) and other data screening of the first intermediate data source, so as to further obtain the screened second intermediate data source; pre-setting the monitoring time interval, and performing regular data monitoring on the screened second intermediate data source at intervals, including monitoring whether there is data failure / expiration, whether there is data abnormality, and whether the data volume of the second intermediate data source is sufficient (there may be insufficient data volume after removing abnormal data, and new data is needed at this time), and finally obtaining a high-quality target data source. This application further ensures the high quality of data by regularly monitoring the data source.

[0082] In the Data Source Basic Information of the Data Source Management Module page, select the data source type. After selecting the data source, relevant configuration information will appear below the data source basic information, such as Figure 5If the selected data source is Kafka, the relevant configuration information of Kafka is displayed at the bottom of the page. Then, continue to fill in the basic information in the basic information of the data source, which determines the method of obtaining the data information of the preset data source; continue to configure the relevant information of the data source at the bottom of the data source management module page, and select Save and Execute to complete the configuration of the data source, and then you can obtain data according to the configured information. This data acquisition method only requires the input of relevant data requirements, and the system can obtain data according to the settings, which greatly improves the convenience of data acquisition.

[0083] The indicator set management module 422 is used to select corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario.

[0084] Figure 6a This is one of the page diagrams of the indicator set management module provided by this application. Figure 6b This is the second page diagram of the indicator set management module provided by this application, such as Figure 6a and Figure 6b As shown, different operation and maintenance scenarios correspond to different indicators. After the data source management module configures the data source, the indicator set management module 422 can read all data information from the configured data source and obtain indicator metadata after data exploration of the data information. The indicator metadata can be displayed in the indicator list, which is convenient for configuring and managing the read indicator metadata, thereby providing support for data processing and management of indicator features.

[0085] Among them, the indicator metadata includes key tags, and after the data exploration is completed, the key tags of the data with higher frequency can be assigned the key tag results of key data. Therefore, in the indicator list, the indicator metadata whose key tag results are key data will be displayed better than the indicator metadata whose key tag results are not key data. That is, the key data are divided by key tags so that the indicator metadata can be displayed in a hierarchical manner. In this way, the data display can be clearer, and the operation and maintenance personnel can pay more attention to the key indicators, which improves the visualization and convenience of the generation of operation and maintenance learning materials.

[0086] In addition, in the indicator list, you can not only view the original data, but also search for indicator metadata. By entering indicator metadata features in the search bar, or filtering indicator metadata feature values, you can obtain filtered indicator metadata, which is convenient for operation and maintenance personnel to view indicator metadata in a targeted manner.

[0087] Next, the detailed indicator information after data exploration is completed can be displayed in the data source indicator information box, including indicator number, indicator hierarchy directory, indicator classification, data type, and key information such as maximum value, minimum value, average value, missing value ratio, variance and standard deviation obtained after data exploration. This information can be used to guide the generation of training sets in the next step. Specifically, the training set can be selected based on the key information obtained after data exploration, rather than using all data information in the resource library for training, which improves the accuracy of the training set and further improves the accuracy of the model.

[0088] Figure 7 This is a schematic diagram of the data exploration page provided by this application, such as Figure 7 As shown in the figure, data exploration refers to the statistical feature analysis of all historical data of indicators for a certain indicator set. The statistical features include: maximum value, minimum value, quartile, tertile, quartile, mean, median, mode, variance, standard deviation, skewness, kurtosis, difference threshold, etc. Kmeans algorithm is used for clustering, and the clustering is divided into K feature groups. The obtained K feature groups can facilitate further training set selection later.

[0089] Figure 8 This is a page diagram of the training set management module provided by this application, such as Figure 8 As shown, the training set management module 423 is used to obtain the indicator feature training set according to the indicator feature. Specifically, the original indicator feature to be processed can be determined by selecting the data source, the indicator set, and the range information of the training data of an indicator with the smallest difference between the mean value in each feature group and the average mean value of the group, and the original indicator feature can be filtered, supplemented, and transformed into a data set using flink through the ETL information configured on the page, and then uploaded to the model training platform for model training.

[0090] The learning model generation module 424 is used to determine at least one training algorithm of the pre-trained model and the hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; train each of the intermediate models based on the indicator feature training set, and use the optimal model as the learning model, or upload the indicator feature training set to a preset platform so that the preset platform trains each of the intermediate models according to the indicator feature training set and uses the optimal model as the learning model, and obtains the learning model returned by the preset platform; wherein the label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

[0091] Figure 9a~Figure 9f This is one of the six page diagrams generated by the learning model provided by this application, such as Figure 9a~Figure 9fOptionally, the learning component model generation module 424 includes a data source setting module, a training set setting module, an algorithm selection module, an algorithm parameter setting module, a model training module, and a model result viewing module. Among them, a learning component can be configured with a data source, an indicator set, a set of training sets, and multiple algorithms, and each algorithm can be configured with multiple sets of hyperparameters. Finally, a learning component can generate multiple models under multiple algorithms. The models generated by the learning component can be viewed in the model result viewing module, and the optimal model can be selected under each algorithm. The optimal model can be selected by comparing the result parameters (such as root mean square error, etc.).

[0092] Fig.10a This is one of the page diagrams of the scene management module provided by this application. Fig.10b This is the second page diagram of the scene management module provided by this application, such as Fig.10a and Fig.10b As shown in the figure, the operation and maintenance scenarios include offline scenarios and real-time scenarios. Different data sources and different models of learning components are selected in different operation and maintenance scenarios for publishing and execution. Scenario operations include publishing, starting, pausing, restarting, and offline operations. Scenario execution filters and transforms data source data through Flink, and then calls the model of the algorithm platform for data detection.

[0093] The operation and maintenance learning material model determination system provided in the present application is used to implement the operation and maintenance learning material model determination method provided in any of the above-mentioned embodiments, and has the technical effect corresponding to the operation and maintenance learning material model determination method, which will not be repeated here.

[0094] Fig.11 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.11 As shown, the electronic device may include: a processor 1110, a communication interface 1120, a memory 1130 and a communication bus 1140, wherein the processor 1110, the communication interface 1120 and the memory 1130 communicate with each other through the communication bus 1140. The processor 1110 may call the logic instructions in the memory 1130 to execute the operation and maintenance learning model determination method, the method comprising: obtaining a scenario description of a target operation and maintenance scenario, and determining a scenario feature value of the scenario description; determining a target learning model corresponding to the target operation and maintenance scenario according to the scenario feature value and a label corresponding to at least one learning model pre-trained; wherein the learning model includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning model is obtained by training the learning model with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0095] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0096] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the operation and maintenance learning model determination method provided by the above methods, the method including: obtaining a scenario description of the target operation and maintenance scenario, and determining the scenario feature value of the scenario description; determining a target learning model corresponding to the target operation and maintenance scenario according to the scenario feature value and the label corresponding to at least one pre-trained learning model; wherein the learning model includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning model is obtained by training the learning model with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0097] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the operation and maintenance learning model determination method provided by the above-mentioned methods, the method comprising: obtaining a scenario description of a target operation and maintenance scenario, and determining a scenario feature value of the scenario description; determining a target learning model corresponding to the target operation and maintenance scenario based on the scenario feature value and a label corresponding to at least one pre-trained learning model; wherein the learning model comprises a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model, and the learning model is obtained by training the learning model using an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining an operation and maintenance learning model, characterized in that: include: Obtaining a scenario description of a target operation and maintenance scenario, and determining a scenario feature value of the scenario description; Determine a target learning model corresponding to the target operation and maintenance scenario according to the scenario feature value and a label corresponding to at least one learning model that has been pre-trained; The learning component includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model. The learning component model is obtained by training the learning component with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

2. The method for determining the operation and maintenance learning model according to claim 1, characterized in that: The training steps of the learning model include: Acquire data information of a preset data source, select corresponding indicator features from the data information according to indicator information corresponding to the operation and maintenance scenario, and obtain the indicator feature training set according to the indicator features; Determine at least one training algorithm of the pre-trained model and hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; Training each of the intermediate models based on the indicator feature training set and taking the optimal model as the learning component model; or uploading the indicator feature training set to a preset platform so that the preset platform trains each of the intermediate models according to the indicator feature training set and takes the optimal model as the learning component model, and obtaining the learning component model returned by the preset platform; The label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

3. The method for determining the operation and maintenance learning model according to claim 2, characterized in that: The step of obtaining the indicator feature training set according to the indicator feature includes: Performing full data feature statistics on the indicator features, and clustering and grouping the indicator features according to the statistical results to obtain at least one feature grouping; Data meeting preset conditions are selected in each of the feature groups to obtain the indicator feature training set.

4. The method for determining the operation and maintenance learning model according to claim 2, characterized in that: Before selecting corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario, the method further includes: Preliminarily screening the data source according to preset reliable data source information to obtain a first intermediate data source; Performing a secondary screening on the first intermediate data source based on a preset screening rule or a manual screening method to obtain a second intermediate data source; Performing regular data monitoring on the second intermediate data source based on a preset monitoring time interval to obtain a target data source; Correspondingly, selecting corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario includes: Select corresponding indicator features from the target data source according to the indicator information corresponding to the operation and maintenance scenario.

5. The method for determining the operation and maintenance learning model according to claim 1, characterized in that: After determining the target learning model corresponding to the target operation and maintenance scenario, the method further includes: At least one operation of publishing, starting, pausing, restarting or going offline is executed in the target operation and maintenance scenario based on the target learning model to obtain an execution result corresponding to the target operation and maintenance scenario.

6. A system for determining an operation and maintenance learning model, characterized in that: Including scenario management module and learning model management module; The scenario management module is used to obtain a scenario description of a target operation and maintenance scenario and determine a scenario feature value of the scenario description; Determine a target learning model corresponding to the target operation and maintenance scenario according to the scenario feature value and a label corresponding to at least one learning model that has been pre-trained; The learning piece model management module is used to pre-train the learning piece model. The learning piece includes a pre-trained model and a specification for characterizing the model intent and / or model characteristics of the pre-trained model. The learning piece model is obtained by training the learning piece with an indicator feature training set corresponding to at least one operation and maintenance scenario and at least one training algorithm.

7. The operation and maintenance learning model determination system according to claim 6, characterized in that: The learning model management module and the scenario management module are arranged in a visual interface, and the learning model management module includes a data source management module, an indicator set management module, a training set management module and a learning model generation module; The data source management module is used to manage the data source; The indicator set management module is used to select corresponding indicator features from the data information according to the indicator information corresponding to the operation and maintenance scenario; The training set management module is used to obtain the indicator feature training set according to the indicator feature; The learning model generation module is used to determine at least one training algorithm of the pre-trained model and hyperparameters corresponding to the training algorithm to obtain at least one intermediate model; Based on the indicator feature training set, each of the intermediate models is trained and the optimal model is used as the learning model. Alternatively, the indicator feature training set is uploaded to a preset platform so that the preset platform trains each of the intermediate models according to the indicator feature training set and uses the optimal model as the learning model, and obtains the learning model returned by the preset platform; wherein the label of the learning model is determined based on the label of the training algorithm and the label of the indicator feature training set.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the operation and maintenance learning model determination method as described in any one of claims 1 to 5.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for determining the operation and maintenance learning model as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the method for determining the operation and maintenance learning model as described in any one of claims 1 to 5.

Images