Isolated Tree Recognition and Alarm Method, Device, Equipment and Medium for Full-Link Tracing

Abstract

The present application discloses an isolated tree recognition and warning method, device, equipment and medium for full-link tracing, which relates to the field of artificial intelligence technology. It includes building a full-link tracing system, performing system configuration and warning configuration on the full-link tracing system, and obtaining the access volume and elapsed time data of customer business access through the data interface of the configured full-link tracing system; and performing data slicing, calculating the average elapsed time according to the sliced access volume and elapsed time data to generate training data; cleaning the training data, constructing each initial isolated tree using the isolated forest algorithm, and training each initial isolated tree to obtain each isolated tree; inputting the data to be recognized into each isolated tree for data recognition to obtain the recognition result. If the recognition result is abnormal, the recognition result is sent to the administrator for warning. The present application can avoid false warnings or missed warnings, improve the recognition efficiency and accuracy of customer business access data, and improve the intelligent monitoring level of operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to an isolated tree recognition and warning method, device, equipment and medium for full-link tracking. Background Art

[0002] At present, the system mainly relies on threshold settings to monitor and alarm the access volume and elapsed time data of customer business access. When a certain threshold is exceeded, an alarm is triggered, or alarm is based on the task status. The inspection process or the alarm process is usually relatively simple. At the same time, due to different time periods, the business access volume may vary greatly, resulting in the use of fixed thresholds not being able to well identify whether there are abnormalities in customer business access, thus reducing the intelligent monitoring level of operation and maintenance. And currently, traditional methods using thresholds or alarms based on task status cannot adaptively adjust the thresholds, so false alarms or missed alarms may occur.

[0003] As can be seen from the above, how to avoid false alarms or missed alarms, improve the recognition efficiency and accuracy of customer business access data, and improve the intelligent monitoring level of operation and maintenance are problems to be solved in this field. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an isolated tree recognition and warning method, device, equipment and medium for full-link tracking, which can avoid false alarms or missed alarms, improve the recognition efficiency and accuracy of customer business access data, and improve the intelligent monitoring level of operation and maintenance. The specific solutions are as follows:

[0005] In the first aspect, the present application discloses an isolated tree recognition and warning method for full-link tracking, including:

[0006] Build a full-link tracking system, perform system configuration and alarm configuration on the full-link tracking system, and obtain the access volume and elapsed time data of customer business access by using the data interface of the configured full-link tracking system;

[0007] Slice the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the average elapsed time according to the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time;

[0008] Clean the training data to obtain the cleaned training data, construct each initial isolated tree by using the isolated forest algorithm, and train each of the initial isolated trees by using the cleaned training data to obtain each isolated tree;

[0009] Obtain the data to be recognized, input the data to be recognized into each of the isolation trees for data recognition to obtain a recognition result. If the recognition result is abnormal, send the recognition result to the administrator for alarm.

[0010] Optionally, build a full-link tracing system and perform system configuration and alarm configuration on the full-link tracing system, including:

[0011] Build a full-link tracing system and deploy pre-probes for all servers in the full-link tracing system;

[0012] Perform open-source database configuration and alarm administrator information configuration on the full-link tracing system after deploying the pre-probes; the system configuration includes open-source database configuration; the alarm configuration includes alarm administrator information configuration.

[0013] Optionally, use the data interface of the configured full-link tracing system to obtain the access volume and elapsed time data of customer business access; perform data slicing on the access volume and the elapsed time data, including:

[0014] Use the data interface of the configured full-link tracing system to obtain the access volume and elapsed time data of customer business access, and send the access volume and the elapsed time data to the message queue;

[0015] Perform data slicing on the access volume and the elapsed time data according to the preset data slicing rules and using the message queue.

[0016] Optionally, calculate the average elapsed time based on the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time, including:

[0017] Calculate the quotient between the number of elapsed times in the elapsed time data and the access volume, and use the quotient as the average elapsed time;

[0018] Generate training data based on the access volume and the average elapsed time, and store the training data in the cache database.

[0019] Optionally, use the cleaned training data to train each of the initial isolation trees to obtain each isolation tree, including:

[0020] Set the maximum depth of the initial isolation tree;

[0021] Randomly select a target training data from the cleaned training data, use the target training data as the root node of the initial isolation tree, and randomly select a cutting point from the target training data. Cut the target training data according to the target training data and the cutting point to obtain the target training data branches of two groups of leaf nodes;

[0022] Calculate the depths corresponding to the target training data branches of each leaf node, determine whether the depths corresponding to the target training data branches of each leaf node are less than the maximum depth, and determine whether the number of the target training data branches of each leaf node is greater than the preset number;

[0023] If the depths corresponding to the target training data branches of each leaf node are not less than the maximum depth, or the number of the target training data branches of each leaf node is not greater than the preset number, stop the cutting operation and complete the training process to obtain the corresponding isolation tree.

[0024] Optionally, the determining whether the depths corresponding to the target training data branches of each leaf node are less than the maximum depth, and determining whether the number of the target training data branches of each leaf node is greater than the preset number further includes:

[0025] If the depths corresponding to the target training data branches of each leaf node are less than the maximum depth, and the number of the target training data branches on each leaf node is greater than the preset number, randomly select the next target training data from the cleaned training data, and randomly select the corresponding cutting point from the next target training data to cut the next target training data to obtain the target training data branches of two groups of the next leaf nodes;

[0026] Jump to the process of calculating the depths corresponding to the target training data branches of each leaf node until the depths corresponding to the target training data branches of the next leaf node are not less than the maximum depth, or the number of the target training data branches of the next leaf node is not greater than the preset number, then stop the cutting operation and complete the training process to obtain the corresponding isolation tree.

[0027] Optionally, the if the recognition result is abnormal, then send the recognition result to the administrator for alarm includes:

[0028] If the recognition result is abnormal, determine the administrator information from the configured full-link tracing system; the administrator information includes the administrator's name, phone number, and system name;

[0029] Send the recognition result to the corresponding administrator for alarm according to the administrator information.

[0030] In a second aspect, the present application discloses an isolated tree recognition and alarm device for end-to-end tracing, including:

[0031] A data acquisition module, configured to build an end-to-end tracing system, perform system configuration and alarm configuration on the end-to-end tracing system, and obtain the access volume and elapsed time data of customer service access through the data interface of the configured end-to-end tracing system;

[0032] A training data generation module, configured to slice the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the average elapsed time according to the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time;

[0033] An isolated tree training module, configured to clean the training data to obtain the cleaned training data, construct each initial isolated tree using the isolated forest algorithm, and train each initial isolated tree using the cleaned training data to obtain each isolated tree;

[0034] A recognition module, configured to obtain data to be recognized, input the data to be recognized into each isolated tree for data recognition to obtain a recognition result, and if the recognition result is abnormal, send the recognition result to the administrator for alarm.

[0035] In a third aspect, the present application discloses an electronic device, including:

[0036] A memory, configured to store a computer program;

[0037] A processor, configured to execute the computer program to implement the foregoing end-to-end tracing isolated tree recognition and alarm method.

[0038] In a fourth aspect, the present application discloses a computer storage medium, configured to store a computer program; wherein, when the computer program is executed by a processor, the steps of the foregoing end-to-end tracing isolated tree recognition and alarm method are implemented.

[0039] It can be seen that the present application provides an isolated tree recognition and alarm method for full-link tracing, including building a full-link tracing system, performing system configuration and alarm configuration on the full-link tracing system, and obtaining the access volume and time-consuming data of customer business access through the data interface of the configured full-link tracing system; slicing the access volume and the time-consuming data to obtain the sliced access volume and time-consuming data, calculating the average time-consuming according to the sliced access volume and time-consuming data, and generating training data based on the access volume and the average time-consuming; cleaning the training data to obtain the cleaned training data, constructing each initial isolated tree using the isolation forest algorithm, and training each initial isolated tree using the cleaned training data to obtain each isolated tree; obtaining the data to be recognized, inputting the data to be recognized into each isolated tree for data recognition to obtain the recognition result, and if the recognition result is abnormal, sending the recognition result to the administrator for alarm. The present application uses the built and configured full-link tracing system to obtain the access volume and time-consuming data of customer business access, can identify anomalies from the perspective of customer access, effectively implement an intelligent operation and maintenance alarm system, slice the access volume and time-consuming data to avoid false alarms caused by business access characteristics, generate training data and clean it, construct and train each isolated tree using the isolation forest algorithm, combine the full-link tracing system with the artificial intelligence model to intelligently identify abnormal data. The isolation forest algorithm has high accuracy in anomaly detection, is fast in processing big data, and has a wide application range. Using the isolated tree for data recognition and anomaly alarm can effectively improve the intelligent level and operation and maintenance efficiency of operation and maintenance, and combining with the artificial intelligence model can also achieve early prediction, so as to discover business anomalies in advance, reduce the operation and maintenance labor cost and the economic loss caused by the occurrence of faults, and improve the intelligent monitoring level of operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0041] Figure 1 Flowchart of an isolated tree recognition and alarm method for full-link tracing disclosed in the present application;

[0042] Figure 2 Flowchart of an isolated tree recognition and alarm method for full-link tracing disclosed in the present application;

[0043] Figure 3Schematic structural diagram of an isolated tree recognition and alarm device for full - link tracing disclosed in this application;

[0044] Figure 4 Structural diagram of an electronic device provided by this application. Detailed implementation manners

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] Currently, the system mainly relies on threshold settings to monitor and alarm the access volume and elapsed time data of customer service access. When a certain threshold is exceeded, an alarm is triggered, or alarm is based on the task status. The inspection process or the alarm process is usually relatively simple. At the same time, due to different time periods, the difference in business access volume may be large, resulting in the use of fixed thresholds not being able to well identify whether there are abnormalities in customer service access, thus reducing the intelligent monitoring level of operation and maintenance. And currently, traditional methods using thresholds or alarms based on task status cannot adaptively adjust the thresholds, so false alarms or missed alarms will occur. As can be seen from the above, how to avoid false alarms or missed alarms, improve the recognition efficiency and accuracy of customer service access data, and improve the intelligent monitoring level of operation and maintenance are problems to be solved in this field.

[0047] See Figure 1 As shown, the embodiments of the present invention disclose an isolated tree recognition and alarm method for full - link tracing, which may specifically include:

[0048] Step S11: Build a full - link tracing system, perform system configuration and alarm configuration on the full - link tracing system, and obtain the access volume and elapsed time data of customer service access by using the data interface of the configured full - link tracing system.

[0049] Step S12: Perform data slicing on the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the average elapsed time according to the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time.

[0050] In this embodiment, perform data slicing on the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the quotient between the number of elapsed times in the elapsed time data and the access volume, and use the quotient as the average elapsed time. Generate training data based on the access volume and the average elapsed time, and store the training data in the cache database.

[0051] Specifically, data slicing is performed according to the request link and the hour of each day. For example, the access volume and average response time at 12:00 noon for the request link api / login of the login page: the variables of api / login_12 are stored in the file database and accumulated. Finally, the access quantity and average response time for one hour of each request link are calculated. For example, if a user accesses once through the request link api / login and the response time is 0.5 seconds, then the access quantity is incremented by 1 and the response time is incremented by 0.5 seconds. Each access is incremented once. Finally, based on the quantity accumulated for one hour, the access quantity for one hour and the accumulated response time quantity for one hour can be obtained. The average response time is calculated by dividing the accumulated response time quantity by the access volume. Training data is generated based on the access volume and average response time.

[0052] Step S13: Clean the training data to obtain the cleaned training data. Construct each initial isolation tree using the isolation forest algorithm, and use the cleaned training data to train each initial isolation tree to obtain each isolation tree.

[0053] In this embodiment, the process of training the initial isolation tree is as follows: Set the maximum depth of the initial isolation tree; randomly select a target training data from the cleaned training data, use the target training data as the root node of the initial isolation tree, and randomly select a cutting point from the target training data, and cut the target training data according to the target training data and the cutting point to obtain the target training data branches of two sets of leaf nodes; calculate the depth corresponding to each of the target training data branches of the leaf nodes, determine whether the depth corresponding to each of the target training data branches of the leaf nodes is less than the maximum depth, and determine whether the number of each of the target training data branches of the leaf nodes is greater than a preset number; if the depth corresponding to each of the target training data branches of the leaf nodes is not less than the maximum depth, or the number of each of the target training data branches of the leaf nodes is not greater than the preset number, stop the cutting operation and complete the training process to obtain the corresponding isolation tree; if the depth corresponding to each of the target training data branches of the leaf nodes is less than the maximum depth, and the number of each of the target training data branches on the leaf nodes is greater than the preset number, randomly select the next target training data from the cleaned training data, and randomly select the corresponding cutting point from the next target training data to cut the next target training data to obtain the target training data branches of two sets of the next leaf nodes; jump to the process of calculating the depth corresponding to each of the target training data branches of the leaf nodes until the depth corresponding to the target training data branches of the next leaf node is not less than the maximum depth, or the number of the target training data branches of the next leaf node is not greater than the preset number, then stop the cutting operation and complete the training process to obtain the corresponding isolation tree.

[0054] Specifically, to clean the training data and save the computational cost of training the Isolation Forest algorithm model later, request links with less than a certain number of accesses in one hour can be discarded, thereby reducing the computational cost and also reducing the interference of data generated by request connections with too little access data, resulting in false alarms of non-exceptions. The cleaned training data is obtained, and then the Isolation Forest algorithm is used to construct each initial isolation tree. The steps for training the initial isolation tree are as follows: (1) Randomly select N points from the cleaned training data as subsamples and put them into the root node of an isolation tree; (2) Randomly specify a dimension (i.e., randomly select a target training data), and within the data range of the current node, randomly generate a cut point C. The cut point is randomly generated between the maximum and minimum values of the specified dimension in the current node data. The selection of this cut point generates a hyperplane that divides the current node data space into 2 subspaces (i.e., obtains the target training data branches of two groups of leaf nodes). Put the points less than C in the current selected dimension on the left branch of the current node, and put the points greater than or equal to C on the right branch of the current node; (3) Determine whether the depth corresponding to each of the target training data branches of the leaf node is less than the maximum depth, and determine whether the number of target training data branches of the leaf node is greater than the preset number (i.e., determine whether there is only one data on the leaf node or determine whether the isolation tree has grown to the set maximum depth); (4) If the depth corresponding to each of the target training data branches of the leaf node is less than the maximum depth, and the number of target training data branches on the leaf node is greater than the preset number, then recursively perform steps (2) and (3) on the left and right branch nodes of the node, continuously constructing new leaf nodes until there is only one data on the leaf node (unable to continue cutting) or the tree has grown to the set height (the reason for restricting the height of the tree is that we only care about points with shorter path lengths, which are more likely to be outliers, and do not care about normal points with very long paths), stop the cutting operation and complete the training process to obtain the corresponding isolation tree. In this way, the training of a single tree is completed, and then the generated isolation tree can be used to evaluate and predict abnormal data.

[0055] For example, for the same one-hour period and the same access request such as the login page request link api / login, at 12 noon on a certain day, the dimension randomly selected first is "traffic volume". A cut-off point of 1000 is randomly selected. The time interval with a traffic volume less than 1000 is assumed to be March 1, 2024, so it is "isolated" first. The second randomly selected feature is "average response time". Assume that only on March 2, 2024, it is higher than 0.8 seconds, so March 2, 2024 is also "isolated". Set the height of the tree to 2, then the training of this tree ends. On this tree, the path length of March 1, 2024 is 1, and that of March 2, 2024 is 2. Looking at this single tree alone, the degree of abnormality of March 2, 2024 is the highest. However, obviously, the reason why this day is isolated first is related to the order in which the features are randomly selected. By training multiple trees, this randomness can be removed to make the results converge as much as possible.

[0056] Step S14: Obtain the data to be recognized, input the data to be recognized into each of the isolation trees for data recognition to obtain a recognition result. If the recognition result is abnormal, send the recognition result to the administrator for warning.

[0057] In this embodiment, if the recognition result is abnormal, determine the administrator information from the configured full-link tracing system; the administrator information includes the administrator's name, phone number, and system name; send the recognition result to the corresponding administrator for warning according to the administrator information.

[0058] That is to say, if the recognition result is abnormal, the name, phone number, etc. of the corresponding administrator can be found from the configured full-link tracing system, and the recognition result can also be sent to the administrator for warning through various warning methods. For example, a DingTalk message can be sent to the system administrator through the DingTalk group message.

[0059] In this embodiment, a full-link tracing system is built, and system configuration and alarm configuration are performed on the full-link tracing system. The access volume and elapsed time data of customer business access are obtained through the data interface of the configured full-link tracing system; the access volume and the elapsed time data are sliced to obtain the sliced access volume and elapsed time data, the average elapsed time is calculated based on the sliced access volume and elapsed time data, and training data is generated based on the access volume and the average elapsed time; the training data is cleaned to obtain the cleaned training data, each initial isolation tree is constructed using the isolation forest algorithm, and each of the initial isolation trees is trained using the cleaned training data to obtain each isolation tree; the data to be recognized is obtained, the data to be recognized is input into each of the isolation trees for data recognition to obtain a recognition result, and if the recognition result is abnormal, the recognition result is sent to the administrator for alarm. This application uses the built and configured full-link tracing system to obtain the access volume and elapsed time data of customer business access, can identify anomalies from the customer's access perspective, effectively implement an intelligent operation and maintenance alarm system, slice the access volume and elapsed time data to avoid false alarms caused by business access characteristics, generate training data and clean it, construct and train each isolation tree using the isolation forest algorithm, combine the full-link tracing system with the artificial intelligence model to intelligently identify abnormal data. The isolation forest algorithm has high accuracy in anomaly detection, fast speed in processing big data, and a wide range of applications. Using the isolation tree for data recognition and anomaly alarm can effectively improve the level of operation and maintenance intelligence and operation and maintenance efficiency, and combining with the artificial intelligence model can also achieve early prediction, so as to discover business anomalies in advance, reduce operation and maintenance labor costs and economic losses caused by faults, and improve the level of intelligent monitoring of operation and maintenance.

[0060] See Figure 2 As shown, an isolation tree recognition and alarm method for full-link tracing according to an embodiment of the present invention specifically may include:

[0061] Step S21: Build a full-link tracing system, deploy a front-end probe for all servers in the full-link tracing system, and perform open-source database configuration and alarm administrator information configuration on the full-link tracing system after deploying the front-end probe; the system configuration includes open-source database configuration; the alarm configuration includes alarm administrator information configuration.

[0062] This application first builds a full-link tracing system, deploys and configures the full-link tracing system, such as the Pinpoint server, and deploys the probes of the full-link tracing system to all the servers where the applications are located. The steps for building Pinpoint are as follows: build a Java running environment; access the Pinpoint official website to download the latest version of Pinpoint; build and configure HBase (a distributed, column-oriented open-source database); configure Pinpoint, mainly to set database connection information, etc.; start HBase and Pinpoint; configure the alarm administrator information, develop a background management page, through which the system name, system alarm person, and phone number that need to be alarmed can be configured, and finally stored in the database for finding the corresponding system administrator contact information during subsequent exception alarms.

[0063] Step S22: Use the data interface of the configured full-link tracing system to obtain the access volume and elapsed time data of customer business access, send the access volume and the elapsed time data to a message queue, and perform data slicing on the access volume and the elapsed time data according to a preset data slicing rule and using the message queue to obtain the sliced access volume and elapsed time data, calculate the average elapsed time based on the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time.

[0064] For example, it is executed once per minute through scheduling. The access volume and elapsed time data are obtained through the data interface of the full-link tracing system, including the system name, request link, and elapsed time (in seconds), and then the access volume and the elapsed time data are sent to the message queue.

[0065] Step S23: Clean the training data to obtain the cleaned training data, construct each initial isolation tree using the isolation forest algorithm, and train each of the initial isolation trees using the cleaned training data to obtain each isolation tree.

[0066] Step S24: Obtain the data to be recognized, input the data to be recognized into each of the isolation trees for data recognition to obtain a recognition result. If the recognition result is abnormal, send the recognition result to the administrator for alarm.

[0067] This application uses the data interface of the full-link tracing to obtain the access volume and elapsed time data of customer business access. Combining with the time period slice data, it generates training data for the access volume and elapsed time data of each time period and stores it in the cache database. It uses the Isolation Forest algorithm to construct isolation trees and conducts training, so as to be able to identify the data. If an anomaly is identified, an alarm is issued. And by combining the Isolation Forest algorithm with artificial intelligence, it can also realize the prediction of data, enabling operation and maintenance personnel to discover business anomalies in advance, reducing operation and maintenance labor costs and economic losses caused by faults. The Isolation Forest algorithm is an anomaly detection method with a linear time complexity and relatively high accuracy. It is fast in processing big data, so it has a wide range of applications in the industrial field currently. Combining the data of the full-connection tracing system with the Isolation Forest algorithm artificial intelligence model can well identify and predict user access anomalies, thereby improving the level of operation and maintenance intelligence and operation and maintenance efficiency.

[0068] In this embodiment, a full-link tracing system is built, and system configuration and alarm configuration are performed on the full-link tracing system. The data interface of the configured full-link tracing system is used to obtain the access volume and elapsed time data of customer business access; the access volume and the elapsed time data are sliced to obtain the sliced access volume and elapsed time data, the average elapsed time is calculated according to the sliced access volume and elapsed time data, and training data is generated based on the access volume and the average elapsed time; the training data is cleaned to obtain the cleaned training data, each initial isolation tree is constructed using the Isolation Forest algorithm, and each initial isolation tree is trained using the cleaned training data to obtain each isolation tree; the data to be identified is obtained, and the data to be identified is input into each isolation tree for data identification to obtain an identification result. If the identification result is abnormal, the identification result is sent to the administrator for alarm. This application uses the built and configured full-link tracing system to obtain the access volume and elapsed time data of customer business access, can identify anomalies from the perspective of customer access, effectively realizes an intelligent operation and maintenance alarm system, slices the access volume and elapsed time data, so as to avoid false alarms caused by business access characteristics, generates training data and cleans it, constructs and trains each isolation tree using the Isolation Forest algorithm, combines the full-connection tracing system with the artificial intelligence model to intelligently identify abnormal data. The Isolation Forest algorithm has relatively high accuracy in anomaly detection and is fast in processing big data with a wide range of applications. Using the isolation tree for data identification and anomaly alarm can effectively improve the level of operation and maintenance intelligence and operation and maintenance efficiency. And combined with the artificial intelligence model, it can also realize early prediction, thereby discovering business anomalies in advance, reducing operation and maintenance labor costs and economic losses caused by faults, and improving the level of intelligent operation and maintenance monitoring.

[0069] See Figure 3As shown in the figure, an isolated tree recognition and warning device for full-link tracing disclosed in an embodiment of the present invention may specifically include:

[0070] A data acquisition module 11, configured to build a full-link tracing system, perform system configuration and warning configuration on the full-link tracing system, and obtain the access volume and time-consuming data of customer service access by using the data interface of the configured full-link tracing system;

[0071] A training data generation module 12, configured to perform data slicing on the access volume and the time-consuming data to obtain the sliced access volume and time-consuming data, calculate the average time-consuming according to the sliced access volume and time-consuming data, and generate training data based on the access volume and the average time-consuming;

[0072] An isolated tree training module 13, configured to clean the training data to obtain the cleaned training data, construct each initial isolated tree by using the isolated forest algorithm, and train each initial isolated tree by using the cleaned training data to obtain each isolated tree;

[0073] An identification module 14, configured to obtain data to be identified, input the data to be identified into each isolated tree for data identification to obtain an identification result, and if the identification result is abnormal, send the identification result to an administrator for warning.

[0074] In this embodiment, a full-link tracing system is built, and system configuration and alarm configuration are performed on the full-link tracing system. The data interface of the configured full-link tracing system is used to obtain the access volume and elapsed time data of customer business access; the access volume and the elapsed time data are sliced to obtain the sliced access volume and elapsed time data, and the average elapsed time is calculated based on the sliced access volume and elapsed time data. Training data is generated based on the access volume and the average elapsed time; the training data is cleaned to obtain the cleaned training data, the initial isolation trees are constructed using the isolation forest algorithm, and the cleaned training data is used to train each of the initial isolation trees to obtain each isolation tree; the data to be recognized is obtained, and the data to be recognized is input into each of the isolation trees for data recognition to obtain a recognition result. If the recognition result is abnormal, the recognition result is sent to the administrator for alarm. This application uses the built and configured full-link tracing system to obtain the access volume and elapsed time data of customer business access, can identify abnormalities from the customer's access perspective, effectively implement an intelligent operation and maintenance alarm system, slice the access volume and elapsed time data, so as to avoid false alarms caused by business access characteristics, generate training data and clean it, construct and train each isolation tree using the isolation forest algorithm, combine the full-link tracing system with the artificial intelligence model to intelligently identify abnormal data. The isolation forest algorithm has high accuracy in anomaly detection, is fast in processing big data, and has a wide application range. Using the isolation tree for data recognition and anomaly alarm can effectively improve the intelligent level and operation and maintenance efficiency of operation and maintenance, and combined with the artificial intelligence model can also achieve early prediction, so as to discover business anomalies in advance, reduce operation and maintenance labor costs and economic losses caused by faults, and improve the intelligent monitoring level of operation and maintenance.

[0075] In some specific embodiments, the data acquisition module 11 may specifically include:

[0076] A system building and deployment module, which is used to build a full-link tracing system and deploy a front-end probe for all servers in the full-link tracing system;

[0077] A configuration module, which is used to perform open-source database configuration and alarm administrator information configuration on the full-link tracing system after deploying the front-end probe; the system configuration includes open-source database configuration; the alarm configuration includes alarm administrator information configuration.

[0078] In some specific embodiments, the training data generation module 12 may specifically include:

[0079] A data acquisition and sending module, which is used to obtain the access volume and elapsed time data of customer business access by using the data interface of the configured full-link tracing system, and send the access volume and the elapsed time data to the message queue;

[0080] A data slicing module, configured to slice the access volume and the time-consuming data according to a preset data slicing rule and by using the message queue.

[0081] In some specific embodiments, the training data generation module 12 may specifically include:

[0082] An average time-consuming calculation module, configured to calculate the quotient between the number of time-consuming data and the access volume in the time-consuming data, and use the quotient as the average time-consuming.

[0083] A training data generation module, configured to generate training data based on the access volume and the average time-consuming, and store the training data in a cache database.

[0084] In some specific embodiments, the isolation tree training module 13 may specifically include:

[0085] A maximum depth setting module, configured to set the maximum depth of the initial isolation tree.

[0086] A cutting module, configured to randomly select a target training data from the cleaned training data, use the target training data as the root node of the initial isolation tree, randomly select a cutting point from the target training data, and cut the target training data according to the target training data and the cutting point to obtain respective target training data branches of two groups of leaf nodes.

[0087] A judgment module, configured to calculate the depth corresponding to each of the target training data branches of the leaf nodes, judge whether the depth corresponding to each of the target training data branches of the leaf nodes is less than the maximum depth, and judge whether the number of each of the target training data branches of the leaf nodes is greater than a preset number.

[0088] A training completion module, configured to, if the depth corresponding to each of the target training data branches of the leaf nodes is not less than the maximum depth, or each of the target training data branches of the leaf nodes is not greater than the preset number, stop the cutting operation and complete the training process to obtain a corresponding isolation tree.

[0089] In some specific embodiments, the isolation tree training module 13 may specifically include:

[0090] The next cutting module is used to, if the depth corresponding to each of the target training data branches of the leaf node is less than the maximum depth, and the number of each of the target training data branches on the leaf node is greater than the preset number, randomly select the next target training data from the cleaned training data, and randomly select a corresponding cutting point from the next target training data, so as to cut the next target training data to obtain two groups of target training data branches of the next leaf node;

[0091] The isolated tree training completion module is used to jump to the process of calculating the depth corresponding to each of the target training data branches of the leaf node until the depth corresponding to the target training data branch of the next leaf node is not less than the maximum depth, or the number of the target training data branches of the next leaf node is not greater than the preset number, then stop the cutting operation and complete the training process to obtain the corresponding isolated tree.

[0092] In some specific embodiments, the recognition module 14 may specifically include:

[0093] The information determination module is used to, if the recognition result is abnormal, determine the administrator information from the configured full-link tracing system; the administrator information includes the administrator's name, phone number, and system name;

[0094] The alarm module is used to send the recognition result to the corresponding administrator for alarm according to the administrator information.

[0095] Figure 4 The figure is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the isolated tree recognition and alarm method for full-link tracing executed by the electronic device disclosed in any of the foregoing embodiments.

[0096] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows is any communication protocol applicable to the technical solution of the present application, and no specific limitation is imposed on it here; the input / output interface 25 is used to obtain external input data or output data to the outside, and its specific interface type can be selected according to specific application needs, and no specific limitation is made here.

[0097] In addition, as a carrier for storing resources, the memory 22 can be a read-only memory, a random access memory, a magnetic disk, an optical disc, etc. The resources stored thereon include an operating system 221, a computer program 222, data 223, etc. The storage method can be temporary storage or permanent storage.

[0098] Among them, the operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, so as to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. It can be Windows, Unix, Linux, etc. In addition to the computer program that can be used to complete the isolated tree recognition and warning method for full-link tracking executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include computer programs that can be used to complete other specific tasks. In addition to the data that can include the data transmitted by external devices received by the isolated tree recognition and warning device for full-link tracking, the data 223 can also include the data collected by its own input / output interface 25, etc.

[0099] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0100] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium. When the computer program stored in the storage medium is loaded and executed by a processor, the steps of the isolated tree recognition and warning method for full-link tracking disclosed in any of the foregoing embodiments are implemented.

[0101] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the said element.

[0102] The above has introduced in detail a method, device, equipment and storage medium for identifying and alarming isolated trees in full-link tracing. In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. An isolated tree recognition and warning method for full-link tracing, characterized in that, Including: Build a full - link tracing system, perform system configuration and alarm configuration on the full - link tracing system, and obtain the access volume and elapsed time data of customer business access through the data interface of the configured full - link tracing system; Slice the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the average elapsed time based on the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time; Clean the training data to obtain the cleaned training data, construct each initial isolation tree using the isolation forest algorithm, and train each initial isolation tree using the cleaned training data to obtain each isolation tree; Obtain the data to be recognized, input the data to be recognized into each isolation tree for data recognition to obtain a recognition result. If the recognition result is abnormal, send the recognition result to the administrator for alarm; Among them, building the full - link tracing system and performing system configuration and alarm configuration on the full - link tracing system includes: building the full - link tracing system and deploying pre - probes for all servers in the full - link tracing system; performing open - source database configuration and alarm administrator information configuration on the full - link tracing system after deploying the pre - probes; the system configuration includes open - source database configuration; the alarm configuration includes alarm administrator information configuration; Calculating the average elapsed time based on the sliced access volume and elapsed time data and generating training data based on the access volume and the average elapsed time includes: calculating the quotient between the number of elapsed times in the elapsed time data and the access volume, and using the quotient as the average elapsed time; generating training data based on the access volume and the average elapsed time and storing the training data in the cache database; Obtain the access volume and elapsed time data of customer business access through the data interface of the configured full - link tracing system, send the access volume and elapsed time data to the message queue, and slice the access volume and elapsed time data according to the preset data slicing rule and using the message queue.

2. The method for identifying and alarming isolated trees in full-link tracing according to claim 1, wherein Training each initial isolation tree using the cleaned training data to obtain each isolation tree includes: Setting the maximum depth of the initial isolation tree; Randomly select a target training data from the cleaned training data, use the target training data as the root node of the initial isolation tree, and randomly select a cutting point from the target training data. Cut the target training data according to the target training data and the cutting point to obtain each target training data branch of two groups of leaf nodes; Calculate the depth corresponding to each target training data branch of the leaf node, determine whether the depth corresponding to each target training data branch of the leaf node is less than the maximum depth, and determine whether the number of each target training data branch of the leaf node is greater than the preset number; If the depth corresponding to each of the target training data branches of the leaf node is not less than the maximum depth, or the number of each of the target training data branches of the leaf node is not greater than a preset number, stop the cutting operation and complete the training process to obtain the corresponding isolation tree.

3. The method for identifying and alarming isolated trees in full-link tracing according to claim 2, wherein, The step of determining whether the depth corresponding to each of the target training data branches of the leaf node is less than the maximum depth, and determining whether the number of each of the target training data branches of the leaf node is greater than a preset number, further includes: If the depth corresponding to each of the target training data branches of the leaf node is less than the maximum depth, and the number of each of the target training data branches on the leaf node is greater than the preset number, randomly select the next target training data from the cleaned training data, and randomly select a corresponding cutting point from the next target training data to cut the next target training data to obtain two groups of target training data branches of the next leaf node; Jump to the process of calculating the depth corresponding to each of the target training data branches of the leaf node until the depth corresponding to the target training data branches of the next leaf node is not less than the maximum depth, or the number of the target training data branches of the next leaf node is not greater than the preset number, then stop the cutting operation and complete the training process to obtain the corresponding isolation tree.

4. The method for identifying and alarming isolated trees in full-link tracing according to any one of claims 1 to 3, characterized in that, The step of, if the recognition result is abnormal, sending the recognition result to the administrator for warning, includes: If the recognition result is abnormal, determine the administrator information from the configured full-link tracing system; the administrator information includes the administrator's name, phone number, and system name; Send the recognition result to the corresponding administrator for warning according to the administrator information.

5. An isolated tree recognition and warning device for full-link tracing, characterized in that, It includes: A data acquisition module, configured to build a full-link tracing system, perform system configuration and warning configuration on the full-link tracing system, and obtain the access volume and elapsed time data of the customer business access by using the data interface of the configured full-link tracing system; A training data generation module, configured to slice the access volume and the elapsed time data to obtain the sliced access volume and elapsed time data, calculate the average elapsed time according to the sliced access volume and elapsed time data, and generate training data based on the access volume and the average elapsed time; An isolation tree training module, configured to clean the training data to obtain the cleaned training data, construct each initial isolation tree by using the isolation forest algorithm, and train each of the initial isolation trees by using the cleaned training data to obtain each isolation tree; A recognition module, configured to obtain the data to be recognized, input the data to be recognized into each of the isolation trees for data recognition to obtain a recognition result, and if the recognition result is abnormal, send the recognition result to the administrator for warning; Among them, building the full-link tracing system and performing system configuration and alarm configuration on the full-link tracing system includes: building the full-link tracing system and deploying pre-probes for all servers in the full-link tracing system; performing open-source database configuration and alarm administrator information configuration on the full-link tracing system after deploying the pre-probes; the system configuration includes open-source database configuration; the alarm configuration includes alarm administrator information configuration; Calculating the average time-consuming according to the sliced access volume and time-consuming data, and generating training data based on the access volume and the average time-consuming, includes: calculating the quotient between the number of time-consuming in the time-consuming data and the access volume, and using the quotient as the average time-consuming; generating training data based on the access volume and the average time-consuming, and storing the training data in the cache database; Using the data interface of the configured full-link tracing system to obtain the access volume and time-consuming data of customer business access, sending the access volume and time-consuming data to the message queue, and slicing the access volume and time-consuming data according to the preset data slicing rule and using the message queue.

6. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for executing the computer program to implement the method for identifying and alarming isolated trees in full-link tracing according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, For storing a computer program; wherein, when the computer program is executed by the processor, it implements the method for identifying and alarming isolated trees in full-link tracing according to any one of claims 1 to 4.

Images