Training method, device and equipment of fault detection model under micro-service architecture

Abstract

The present disclosure provides a training method and device of a fault detection model under a micro-service architecture, equipment, medium and program product, which can be applied to the fields of artificial intelligence and finance. The training method comprises: calling a historical transaction log of a full link under a micro-service architecture from a database, the historical transaction log having a feature field with a representation meaning; preprocessing the historical transaction log by using an isolation forest algorithm model, and generating a training sample data set according to an output result of the isolation forest algorithm model, wherein the historical transaction log is unsupervisedly trained by using the isolation forest algorithm to eliminate abnormal values; and training the training sample data set by using a logistic regression algorithm to obtain a fault detection model for identifying transaction link abnormalities. The technical problem that the fault detection under the micro-service architecture is more difficult and the specific node position of the fault cannot be accurately located in the related art is solved, and the technical effect of improving the accuracy of fault detection and localization is achieved.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence and finance, and in particular to a training method, apparatus, device, medium and program product for a fault detection model under a microservice architecture. Background Technology

[0002] With the development of internet technology and fintech, online transactions are becoming increasingly frequent, and the transaction amounts are constantly increasing. While online transactions bring convenience to people, they also bring more and more transaction problems. Some small errors can lead to major transaction problems, making the diagnosis of transaction anomalies particularly important. However, due to the complexity of the chain, fault detection in microservice architecture systems is more difficult, and it is not possible to accurately locate the specific node where the fault occurred.

[0003] Current problem diagnosis relies on alarms with fixed thresholds. Fixed thresholds are prone to false alarms and missed alarms, resulting in low accuracy of problem diagnosis. On the other hand, they may cause alarm storms, where real alarm information is masked by a large number of false alarms, which seriously affects the efficiency of problem analysis and localization. Summary of the Invention

[0004] In view of the above problems, this disclosure provides training methods, apparatus, devices, media and program products for fault detection models under microservice architecture.

[0005] According to the first aspect of this disclosure, a method for training a fault detection model under a microservice architecture is provided, comprising:

[0006] The database retrieves the end-to-end historical transaction logs under the microservice architecture, and the historical transaction logs have characteristic fields that represent the transaction process.

[0007] The historical transaction logs are preprocessed using the Isolation Forest algorithm model, and a training sample dataset is generated based on the output of the Isolation Forest algorithm model. The historical transaction logs are unsupervised trained using the Isolation Forest algorithm to remove outliers.

[0008] The training sample dataset is trained using a logistic regression algorithm to obtain a fault detection model for identifying anomalies in the transaction chain.

[0009] According to some embodiments of this disclosure, the training sample set includes link call time, call success or failure indicators, and fault label information.

[0010] According to some embodiments of this disclosure, before preprocessing the historical transaction logs using the isolated forest algorithm model, the method further includes:

[0011] Feature extraction is performed on the historical transaction logs to obtain templated transaction logs, wherein transaction logs of different transaction links with the same characterization feature fields are the same template.

[0012] According to some embodiments of this disclosure, after preprocessing the historical transaction logs using the isolated forest algorithm model, the method further includes:

[0013] Clustering algorithms are used to perform clustering analysis on the templated transaction logs to obtain multiple node types, and the label information of whether or not the node is faulty is used as the label of the node type.

[0014] According to some embodiments of this disclosure, the feature fields include average transaction volume per second, average transaction time per second, and transaction success rate per second.

[0015] According to some embodiments of this disclosure, training the training sample dataset using a logistic regression algorithm includes:

[0016] The classifier is trained in a supervised manner using templated logs carrying node type labels to obtain the weight coefficients corresponding to each feature field in order to construct the fault detection model.

[0017] According to some embodiments of this disclosure, the classifier is a CATBOOST classifier.

[0018] According to some embodiments of this disclosure, the step of calling the entire historical transaction log of the microservice architecture from the database includes:

[0019] Another aspect of this disclosure provides a fault detection method in a microservice architecture, including:

[0020] Read transaction log data from multiple nodes in the dynamic log information table stored in the information management system under a microservice architecture; and

[0021] The transaction log data is input into the fault detection model, which outputs information about faulty nodes in the transaction chain.

[0022] The fault detection model is trained using the fault detection model training method described above for microservice architecture.

[0023] According to some embodiments of this disclosure, after outputting the faulty node information in the transaction link, the method further includes:

[0024] The transaction logs of faulty nodes are compared with the templated logs pre-stored in the pattern library, and the fault type is determined based on the comparison results.

[0025] The pattern library is established by analyzing the historical transaction logs of the entire chain stored in the database and storing them according to the fault type through the training method.

[0026] According to some embodiments of this disclosure, constructing the pattern library includes:

[0027] The system retrieves historical transaction log data from the database, which is part of the entire microservice architecture. The historical transaction log data includes the transaction log's call duration, call success or failure indicators, node type information, fault label information, and fault type information.

[0028] Historical transaction logs are categorized according to node type, and the probability of each type of fault in the transaction logs across all link templates is calculated; and

[0029] The pattern library is constructed using the probability values ​​and the historical transaction log data.

[0030] A third aspect of this disclosure provides a training apparatus for a fault detection model in a microservice architecture, comprising:

[0031] The data retrieval module is suitable for retrieving historical transaction logs from a database within a microservice architecture, where the historical transaction logs have characteristic fields that represent the full chain of transactions.

[0032] The sample generation module is suitable for preprocessing the historical transaction logs using the Isolation Forest algorithm model, and generating a training sample dataset based on the output of the Isolation Forest algorithm model; and

[0033] The training module is suitable for training the training sample dataset using a logistic regression algorithm to obtain a fault detection model for identifying anomalies in the transaction chain.

[0034] The fourth aspect of this disclosure provides a training apparatus for a fault detection model in a microservice architecture, comprising:

[0035] The transaction log acquisition module is suitable for reading transaction log data from multiple nodes in a dynamic log information table stored in an information management system under a microservice architecture; and

[0036] The fault detection model module is trained using the above-mentioned training method. The fault detection model is suitable for determining the node information of faults in the transaction link.

[0037] The fifth aspect of this disclosure provides an electronic device comprising:

[0038] One or more processors;

[0039] Storage device for storing one or more programs.

[0040] When the one or more programs are executed by the one or more processors, the one or more processors perform the training method described above.

[0041] A sixth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the training method described above.

[0042] The seventh aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the training method described above.

[0043] Based on the training method, apparatus, equipment, medium, and program products for a fault detection model under a microservice architecture provided in this disclosure, the historical transaction logs of each link of the server are analyzed and processed to extract the characteristic fields with representational significance from the historical transaction logs. First, the isolated forest algorithm model is used to perform unsupervised learning to filter the historical transaction logs. Then, the output of the isolated forest algorithm is used as the input of logistic regression to train a fault detection model that can identify abnormalities in the transaction link. Therefore, it at least partially solves the technical problem in related technologies that fault detection under a microservice architecture is more difficult and that it is not possible to accurately locate the specific node where the fault occurs, and achieves the technical effect of improving the accuracy of fault detection and delimitation. Attached Figure Description

[0044] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0045] Figure 1 The illustration schematically depicts an application scenario of a training method, apparatus, device, medium, and program product for a fault detection model under a microservice architecture according to embodiments of the present disclosure.

[0046] Figure 2 The flowchart illustrates a method for training a fault detection model under a microservice architecture according to an embodiment of the present disclosure.

[0047] Figure 3 A flowchart illustrating a fault detection method under a microservice architecture according to an embodiment of the present disclosure is shown schematically.

[0048] Figure 4 A flowchart illustrating a fault detection method under a microservice architecture according to another embodiment of this disclosure is shown schematically.

[0049] Figure 5 A flowchart illustrating a fault detection method under a microservice architecture according to an embodiment of the present disclosure is shown schematically.

[0050] Figure 6 This schematically illustrates a structural block diagram of a training apparatus for a fault detection model under a microservice architecture according to an embodiment of the present disclosure;

[0051] Figure 7 This schematically illustrates a structural block diagram of a training apparatus for a fault detection model under a microservice architecture according to another embodiment of the present disclosure; and

[0052] Figure 8 A block diagram of an electronic device suitable for implementing a training method for a fault detection model under a microservice architecture, according to an embodiment of the present disclosure, is shown schematically. Detailed Implementation

[0053] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0054] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0055] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0056] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0057] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of data (including but not limited to user personal information) comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0058] Embodiments of this disclosure provide a training method, apparatus, device, medium, and program product for a fault detection model under a microservice architecture.

[0059] Figure 1 The diagram illustrates an application scenario of a training method for a fault detection model under a microservice architecture according to an embodiment of the present disclosure.

[0060] like Figure 1 As shown, application scenario 100 according to this embodiment may include the Internet, local area network, e-commerce platform, etc. Network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0061] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0062] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0063] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0064] It should be noted that the training method and fault detection method under the microservice architecture provided in this embodiment can generally be executed by server 105. Correspondingly, the training device for the fault detection model under the microservice architecture provided in this embodiment can generally be located in server 105. The training method and fault detection method under the microservice architecture provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the training device for the fault detection model under the microservice architecture provided in this embodiment can also be located in a server or server cluster that is different from server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0065] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0066] The following will be based on Figure 1 The described scene, through Figure 2 The training method of the fault detection model under the microservice architecture of the disclosed embodiments is described in detail.

[0067] Figure 2 A flowchart illustrating a training method for a fault detection model under a microservice architecture according to an embodiment of the present disclosure is shown.

[0068] like Figure 2 As shown, the training method for the fault detection model under the microservice architecture in this embodiment includes operations S201 to S203.

[0069] In operation S201, the historical transaction logs of the entire chain under the microservice architecture are retrieved from the database. The historical transaction logs have characteristic fields that are representative.

[0070] In operation S202, the historical transaction logs are preprocessed using the isolated forest algorithm model, and a training sample dataset is generated based on the output of the isolated forest algorithm model. In this process, the isolated forest algorithm is used to perform unsupervised training on the historical transaction logs to remove outliers.

[0071] In operation S203, the training sample dataset is trained using a logistic regression algorithm to obtain a fault detection model for identifying anomalies in the transaction chain.

[0072] In this embodiment, by analyzing and processing the historical transaction logs of each link in the server, characteristic fields with representational significance are extracted from the historical transaction logs. First, an unsupervised learning process is used to filter the historical transaction logs using the Isolation Forest algorithm. Then, the output of the Isolation Forest algorithm is used as the input to logistic regression to train a fault detection model capable of identifying anomalies in the transaction link. Therefore, this at least partially solves the technical problem in related technologies where fault detection is more difficult in microservice architectures and the specific location of faulty nodes cannot be accurately pinpointed, thus achieving the technical effect of improving the accuracy of fault detection and delimitation.

[0073] According to some embodiments of this disclosure, the training sample set includes link call time, call success or failure indicators, and fault label information.

[0074] According to some embodiments of this disclosure, the label for whether a link is faulty includes a timeout link. For example, the start and end times of the application logs of each link are obtained to obtain the transaction duration of the link. The transaction duration is compared with a preset threshold, and the result is used to determine whether the transaction log of the link is a timeout link and to mark the result.

[0075] According to some embodiments of this disclosure, before preprocessing the historical transaction logs using the isolated forest algorithm model, the method further includes: extracting features from the historical transaction logs to obtain templated transaction logs, wherein transaction logs of different transaction links with the same characterizing feature fields are the same template.

[0076] According to some optional embodiments of this disclosure, historical transaction logs of all links within a certain period of time are obtained, for example, transaction log data of 1 to 3 weeks are obtained, and multiple feature fields (e.g., 3, 4, 5 or 6 fields) are filtered out from these data. Transaction logs with these feature fields and the same feature field values ​​are classified into the same templated log, thereby simplifying the subsequent data analysis and processing process.

[0077] According to some embodiments of this disclosure, after preprocessing the historical transaction logs using the isolated forest algorithm model, the method further includes: performing cluster analysis on the templated transaction logs using a clustering algorithm to obtain multiple node types, and using the fault label information as the label of the node type.

[0078] According to some embodiments of this disclosure, the feature fields include average transaction volume per second, average transaction time per second, and transaction success rate per second.

[0079] According to some embodiments of this disclosure, training the training sample dataset using the logistic regression algorithm includes: supervising the classifier using templated logs carrying node type labels to obtain the weight coefficients corresponding to each feature field, so as to construct a fault detection model.

[0080] According to some embodiments of this disclosure, the classifier is a CATBOOST classifier.

[0081] The following will be based on Figure 1 The described scene, through Figures 3-5 The fault detection method under the microservice architecture disclosed in the present embodiment is described in detail.

[0082] Figure 3 A flowchart illustrating a fault detection method under a microservice architecture according to an embodiment of the present disclosure is shown.

[0083] like Figure 3 As shown, the fault detection method under the microservice architecture in this embodiment includes operations S301 to S302.

[0084] In operation S301, transaction log data from multiple nodes is read from the dynamic log information table stored in the information management system under the microservice architecture.

[0085] In operation S302, the transaction log data is input into the fault detection model, which outputs information about faulty nodes in the transaction chain. The fault detection model is trained using the fault detection model training method described above for a microservice architecture.

[0086] According to some embodiments of this disclosure, after outputting the information of faulty nodes in the transaction chain, the method further includes: comparing the transaction logs of the faulty nodes with pre-stored templated logs in the pattern library, and determining the fault type (faulty node location) based on the comparison result. The pattern library is established by analyzing the historical transaction logs of the entire transaction chain stored in the database using the training method and storing them according to fault type.

[0087] Figure 4 A flowchart illustrating a fault detection method under a microservice architecture according to another embodiment of this disclosure is shown.

[0088] like Figure 4 As shown, another embodiment of the fault detection method under the microservice architecture includes operations S301 to S303.

[0089] In operation S301, transaction log data from multiple nodes is read from the dynamic log information table stored in the information management system under the microservice architecture.

[0090] In operation S302, the transaction log data is input into the fault detection model, which outputs information about faulty nodes in the transaction chain. The fault detection model is trained using the fault detection model training method described above for a microservice architecture.

[0091] In operation S303, the transaction logs of the faulty node are compared with the templated logs pre-stored in the pattern library, and the fault type is determined based on the comparison result; wherein, the pattern library is established by analyzing the historical transaction logs of the entire chain stored in the database through the training method and storing them according to the fault type.

[0092] According to some embodiments of this disclosure, constructing the pattern library includes: sub-operations S401 to S403.

[0093] In sub-operation S401, historical transaction log data of the entire chain under the microservice architecture is retrieved from the database. The historical transaction log data includes the chain call time of the transaction log, the call success or failure identifier, node type information, failure label information, and failure type information.

[0094] In sub-operation S402, historical transaction logs are classified according to node type, and the probability value of transaction logs of each type of failure in all link templates is calculated.

[0095] In sub-operation S402, the pattern library is constructed using the probability value and the historical transaction log data.

[0096] According to some embodiments of this disclosure, the log text of the transaction to be detected is obtained, the log text is preprocessed, and the preprocessed result is input into the fault detection model to determine whether the transaction is abnormal. When the determination result indicates that the transaction is abnormal, the preprocessed result is matched with data in the pattern library, and the specific node where the abnormality occurred is determined based on the matching result, and the result is output. For example, in actual fault monitoring, operation S302 is first executed to detect whether the transaction is abnormal. If the transaction is abnormal, then based on the pattern library established by operation S402, the transaction link logs within that time period are templated and matched with the pattern library. If no match is found or the matching probability is lower than a preset threshold, such as lower than 0.6, it indicates that the transaction identified by the log is abnormal, and the specific link where the abnormality occurred is obtained. Combined with the detection of link indicators, fault delimitation at the link node level is achieved.

[0097] Figure 5 A flowchart illustrating a fault detection method under a microservice architecture according to an embodiment of the present disclosure is shown.

[0098] like Figure 5As shown, the technical solution of this application includes two parallel branches (which can be simultaneous or not). The first branch is for model training and the establishment of algorithm and pattern libraries (MySQL, Redis). Training is performed by calling historical data. Optionally, the training process can be carried out offline, which does not consume bandwidth and provides greater flexibility. The second branch is for real-time acquisition of current transaction logs for online diagnosis. After acquiring the log text, it is processed by stream processing (Spark Stream), including data preprocessing and log feature extraction. It monitors in real time whether there are faults in the current transaction and determines the specific node location where the fault occurs, and stores the results (Elastic Search).

[0099] Based on the training method for the fault detection model under the microservice architecture described above, this disclosure also provides a training device for the fault detection model under the microservice architecture. The following will combine... Figure 6 The device is described in detail.

[0100] Figure 6 The diagram illustrates the structure of a training apparatus for a fault detection model under a microservice architecture according to an embodiment of the present disclosure.

[0101] like Figure 6 As shown, the training device 600 for the fault detection model under the microservice architecture in this embodiment includes: a data retrieval module 601, a sample generation module 602, and a training module 603.

[0102] The data retrieval module 601 is used to retrieve the historical transaction logs of the entire microservice architecture from the database. These historical transaction logs have characteristic fields that represent the transaction process. In one embodiment, the data retrieval module 601 can be used to execute the operation S201 described above, which will not be repeated here.

[0103] The sample generation module 602 is used to preprocess the historical transaction logs using the Isolation Forest algorithm model, and generate a training sample dataset based on the output of the Isolation Forest algorithm model. In one embodiment, the sample generation module 602 can be used to perform the operation S202 described above, which will not be repeated here.

[0104] The training module 603 is used to train the training sample dataset using a logistic regression algorithm to obtain a fault detection model for identifying anomalies in the transaction chain. In one embodiment, the training module 603 can be used to perform the operation S203 described above, which will not be repeated here.

[0105] According to some embodiments of this disclosure, the characteristic fields of the historical transaction logs invoked by the data retrieval module 601 include average transaction volume per second, average transaction time per second, and transaction success rate per second.

[0106] According to some embodiments of this disclosure, the training sample set generated by the sample generation module 602 includes link call time, call success or failure identifiers, and fault label information.

[0107] According to some embodiments of this disclosure, the label for whether a training sample set generated by the sample generation module 602 is faulty includes timeout links. For example, the start and end times of the application logs of each link are obtained to obtain the transaction duration of the link. The transaction duration is compared with a preset threshold, and the transaction log of the link is determined and marked as a timeout link based on the comparison result.

[0108] According to some embodiments of this disclosure, before the sample generation module 602 preprocesses the historical transaction logs using the isolated forest algorithm model, it further includes: extracting features from the historical transaction logs to obtain templated transaction logs, wherein transaction logs of different transaction links with the same characterization feature fields are the same template.

[0109] According to some optional embodiments of this disclosure, the sample generation module 602 obtains historical transaction logs of all links within a certain period of time, for example, obtains transaction log data of 1 to 3 weeks, filters out multiple feature fields (e.g., 3 fields) from these data, and classifies transaction logs with rewritten feature fields and the same feature field values ​​into the same templated log, thereby simplifying the subsequent analysis process.

[0110] According to some embodiments of this disclosure, after the sample generation module 602 preprocesses the historical transaction logs using the isolated forest algorithm model, it further includes: performing cluster analysis on the templated transaction logs using a clustering algorithm to obtain multiple node types, and using the fault label information as the label of the node type.

[0111] According to some embodiments of this disclosure, the training module 603 trains the training sample dataset using a logistic regression algorithm by: performing supervised training of the classifier using templated logs carrying node type labels to obtain the weight coefficients corresponding to each feature field.

[0112] According to some embodiments of this disclosure, the classifier trained by the training module 603 is the CATBOOST classifier.

[0113] Training module 603 primarily extracts patterns from historical transaction logs and trains models. By extracting feature fields from historical transaction logs, each log entry is templated. The templated logs are then clustered using the density clustering algorithm DBSCAN. The clustering results and the templated logs are stored in Redis as an online diagnostic pattern library. The clustering results are then used as training data for the CatBoost classification algorithm to train the classification model. The trained classification algorithm is then stored in MySQL. The migration of the offline-trained pattern library and model is completed by exporting the pattern library and algorithm data from Redis and MySQL in the test environment to a data file and importing the data file into the production database using a script.

[0114] According to some embodiments of this disclosure, the training apparatus for the fault detection model under a microservice architecture further includes a data sending module and a data buffering module. The data sending module uses Fluentd to distribute data to the data buffering module (Kafka), and then the data buffering module distributes the data through a Kafka subscription. Optionally, the data sending module uses Fluentd to distribute data to the data buffering module (Kafka), and this data is transaction log data, including logs from each link of the entire transaction. The data buffering module uses a Kafka cluster for data buffering.

[0115] According to some optional embodiments of this disclosure, the training device for the fault detection model under the microservice architecture further includes a data consumption and online real-time diagnosis module. This module obtains log data from Kafka in real time via Spark, performs online real-time pattern diagnosis based on the offline-trained model and pattern library, and stores the diagnosis results in Elasticsearch. Optionally, the data consumption and online real-time diagnosis module uses the Spark Streaming architecture, acting as a consumer of the Kafka cluster to subscribe to topics from the Kafka cluster, obtain transaction log data in real time, and perform online real-time data processing. It identifies whether a transaction has an anomaly by matching the most similar log sequence type for a given transaction. If an anomaly occurs, it classifies the anomaly according to the anomaly judgment criteria and provides information to assist in problem diagnosis. This module includes a Spark stream processing cluster and real-time log diagnosis. Spark subscribes to data from Kafka in real time and sends the data to the online real-time diagnosis module. The real-time diagnosis module obtains the offline-trained model from the MySQL database and the offline-established pattern library from Redis. It identifies problematic transactions through model diagnosis, provides the problematic transaction steps in the transaction chain, and finally stores the diagnosis results in the Elasticsearch cluster.

[0116] According to some optional embodiments of this disclosure, the training apparatus for the fault detection model under the microservice architecture further includes a result display module, which retrieves and displays the corresponding result data from ElasticSearch through rule query.

[0117] According to some optional embodiments of this disclosure, the training device for the fault detection model under the microservice architecture further includes a data storage module. The log mode diagnostic results are indexed and stored in the ElasticSearch cluster on a daily basis. The results data are stored for a preset duration, such as 1 week. Data stored for more than 1 week will be deleted by a preset instruction, such as an automated script.

[0118] According to embodiments of this disclosure, any plurality of modules among the data retrieval module 601, sample generation module 602, and training module 603 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules can be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the data retrieval module 601, sample generation module 602, and training module 603 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the data retrieval module 601, sample generation module 602, and training module 603 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0119] Figure 7 The diagram illustrates a structural block diagram of a training apparatus for a fault detection model under a microservice architecture according to another embodiment of the present disclosure.

[0120] like Figure 7 As shown, the training device 700 for the fault detection model under the microservice architecture in this other embodiment includes a transaction log acquisition module 701 and a fault detection model module 702.

[0121] The transaction log acquisition module is suitable for reading transaction log data from multiple nodes in the dynamic log information table stored in the information management system under a microservice architecture. The fault detection model, trained using the methods described above, is suitable for identifying faulty nodes in the transaction chain.

[0122] Figure 8A block diagram of an electronic device suitable for implementing a training method for a fault detection model under a microservice architecture, according to an embodiment of the present disclosure, is shown schematically.

[0123] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0124] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0125] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 808 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0126] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0127] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0128] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0129] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0130] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0131] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0135] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A training method for a fault detection model under a microservice architecture, characterized in that, include: The database retrieves the historical transaction logs of the entire microservice architecture. The historical transaction logs have characteristic fields that represent the characteristics of the transaction logs. These characteristic fields include the average number of transactions per second, the average transaction time per second, and the transaction success rate per second. The historical transaction logs are preprocessed using the Isolation Forest algorithm, and a training sample dataset is generated based on the output of the Isolation Forest algorithm. Specifically, the historical transaction logs are trained unsupervised using the Isolation Forest algorithm to remove outliers. The training sample dataset is trained using a logistic regression algorithm to obtain a fault detection model for identifying faulty nodes in the transaction chain; The process of preprocessing the historical transaction logs using the isolated forest algorithm model includes the following steps: Feature extraction is performed on the historical transaction logs to obtain templated transaction logs, wherein transaction logs of different transaction links with the same characterization feature fields are the same template.

2. The training method according to claim 1, characterized in that, The training sample dataset includes link call time, call success or failure indicators, and fault label information.

3. The training method according to claim 1, characterized in that, The process of preprocessing the historical transaction logs using the isolated forest algorithm model also includes: Clustering algorithms are used to perform clustering analysis on the templated transaction logs to obtain multiple node types, and the label information of whether or not the node is faulty is used as the label of the node type.

4. The training method according to claim 3, characterized in that, The step of training the training sample dataset using the logistic regression algorithm includes: The classifier is trained in a supervised manner using templated logs carrying node type labels to obtain the weight coefficients corresponding to each feature field in order to construct the fault detection model.

5. The training method according to claim 4, characterized in that, The classifier is the CATBOOST classifier.

6. A fault detection method under a microservice architecture, characterized in that, include: Read transaction log data from multiple nodes in the dynamic log information table stored in the information management system under the microservice architecture; as well as The transaction log data is input into the fault detection model, which outputs information about faulty nodes in the transaction chain. The fault detection model is obtained by training using the training method described in any one of claims 1 to 5.

7. The fault detection method according to claim 6, characterized in that, The information about faulty nodes in the output transaction chain also includes: The transaction logs of faulty nodes are compared with the templated logs pre-stored in the pattern library, and the fault type is determined based on the comparison results. The pattern library is established by analyzing the historical transaction logs of the entire chain stored in the database and storing them according to the fault type through the training method.

8. The fault detection method according to claim 7, characterized in that, Building the pattern library includes: The system retrieves historical transaction log data from the database, which is part of the entire microservice architecture. The historical transaction log data includes the transaction log's call duration, call success or failure indicators, node type information, fault label information, and fault type information. Historical transaction logs are categorized according to node type, and the probability of each type of fault in the transaction logs across all link templates is calculated; and The pattern library is constructed using the probability values ​​and the historical transaction log data.

9. A training device for a fault detection model under a microservice architecture, characterized in that, include: The data retrieval module is suitable for retrieving historical transaction logs from a database within a microservice architecture. These historical transaction logs have characteristic fields that represent the entire transaction chain. These characteristic fields include average transaction volume per second, average transaction time per second, and transaction success rate per second. The sample generation module is suitable for preprocessing the historical transaction logs using the Isolation Forest algorithm model, and generating a training sample dataset based on the output of the Isolation Forest algorithm model; and The training module is suitable for training the training sample dataset using a logistic regression algorithm to obtain a fault detection model for identifying faulty nodes in the transaction chain. The process of preprocessing the historical transaction logs using the isolated forest algorithm model includes the following steps: Feature extraction is performed on the historical transaction logs to obtain templated transaction logs, wherein transaction logs of different transaction links with the same characterization feature fields are the same template.

10. A training device for a fault detection model under a microservice architecture, characterized in that, include: The transaction log acquisition module is suitable for reading transaction log data from multiple nodes in the dynamic information log table stored in the information management system under a microservice architecture; as well as The fault detection model module is trained by the training method described in any one of claims 1 to 5, and the fault detection model is suitable for determining the node information of faults in the transaction link.

11. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 8.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images