Business state prediction method and device, electronic equipment and storage medium
By constructing a Hidden Markov Model to analyze historical data from business system logs, the probability distribution and state sequence of system anomalies can be predicted, solving the problem that existing technologies can only provide post-event remedies and improving the system's maintenance and continuity assurance capabilities.
Patent Information
- Application Number
- CN202511565041.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-30
- Publication Date
- 2025-11-28
- Estimated Expiration
- 2045-10-30
AI Technical Summary
Existing technologies can only provide reactive solutions for maintaining the operational status of business systems, resulting in poor maintenance capabilities and business continuity assurance capabilities.
An initial hidden Markov model is constructed, and the initial parameters of the model are iteratively calculated using historical log datasets. The probability of observed anomalies is calculated using forward and Viterbi algorithms, and weighted fusion and backtracking analysis are performed to predict business state sequences.
It enables proactive prediction of business system failures, thereby improving system maintenance capabilities and business continuity assurance capabilities.
Smart Images

Figure CN121029545A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of network device management, and particularly relates to a service state prediction method and device, electronic equipment and a storage medium. BACKGROUND
[0002] A business system is a hardware and software integrated platform built by an enterprise to support daily operation and achieve specific business objectives, and efficiency is improved and decisions are optimized through automatic processes and data integration. Therefore, the running state of the business system is the core of the enterprise digital transformation, and its stability and efficiency directly affect the enterprise operation efficiency, risk control ability and market competitiveness.
[0003] Currently, the maintenance of the running state of the business system mainly analyzes the running data before the fault occurs, resulting in low maintenance efficiency, and obviously, the risk of business interruption is increased. Therefore, in the process of maintaining the running state of the business system, the prior art has the problem of poor system maintenance ability and business continuity guarantee ability due to the ability to only remedy after the event. SUMMARY
[0004] The embodiments of the present application provide a business state prediction method and device, electronic equipment and a storage medium, which aims to advance the fault recovery and intervention of the business system from after the event to during the event or even before the event, so as to improve the system maintenance ability and business continuity guarantee ability.
[0005] To solve the above technical problems, the embodiments of the present application provide the following technical solutions: A business state prediction method, comprising: obtaining a log historical data set of a system and an initial observation probability of each log; constructing an initial hidden Markov model, inputting the log historical data set into the initial hidden Markov model, iteratively calculating and determining model initial parameters of the initial hidden Markov model, determining a target hidden Markov model according to the model initial parameters, the target hidden Markov model comprising a forward algorithm module and a Viterbi algorithm module; calculating an observation anomaly probability of any log observation sequence according to the forward algorithm module, the initial observation probability and the model initial parameters, to obtain an anomaly probability distribution result of the log; performing weighted fusion on the anomaly probability distribution result, and determining a business state sequence length of the system according to the weighted anomaly probability distribution result; obtaining an initial business state of the system; performing backtracking analysis on the initial business state according to the Viterbi algorithm module, the model initial parameters and the business state sequence length, to obtain a target business state sequence of the system.
[0006] Optionally, the model initial parameters include an initial state probability vector, a state transition probability matrix and an observation probability matrix; and the inputting of the log historical data set into the initial hidden Markov model and the iterative calculation and determination of the model initial parameters of the initial hidden Markov model include: acquiring a normal state occurrence frequency, an abnormal state occurrence frequency, a state transition probability set and a log state conditional probability set of the log historical data set; determining the initial state probability vector according to the normal state occurrence frequency and the abnormal state occurrence frequency; determining the state transition probability matrix according to a plurality of state transition probabilities in the state transition probability set; determining the observation probability matrix according to a plurality of log state conditional probabilities in the log state conditional probability set.
[0007] Optionally, the model initial parameters include an initial state probability vector, a state transition probability matrix and an observation probability matrix; and the calculation of the observation abnormal probability of any log observation sequence according to the forward algorithm module, the initial observation probability and the model initial parameters to obtain an abnormal probability distribution result of the log includes: recursively calculating the observation probability of any log at any time based on the initial observation probability of any log, the initial state probability vector, the state transition probability matrix and the observation probability matrix; arraying the observation probability to obtain the abnormal probability distribution result of the log; wherein the abnormal probability distribution result includes the observation probability of any log at any time.
[0008] Optionally, after the target business state sequence of the system is obtained, the method further includes: determining an abnormal business time of the system according to the target business state sequence; screening out abnormal observation probabilities of all logs at the abnormal business time according to the abnormal probability distribution result; comparing the abnormal observation probabilities of all logs, and determining a log corresponding to a maximum abnormal observation probability as an abnormal log.
[0009] Optionally, the system includes a plurality of logs; and the weighted fusion of the abnormal probability distribution result and the determination of the business state sequence length of the system according to the weighted abnormal probability distribution result include: respectively acquiring a weight of each log and the abnormal probability distribution result of all logs at a certain time; weighting calculation of the weight of each log and the abnormal probability distribution result of the log, to determine a log weight value of all logs at a time, and to determine a log weight value sequence according to the log weight values at all times; acquire a preset target time, and determine the service state sequence length according to a time value in the log weight value sequence and the preset target time.
[0010] Optionally, the determination of the service state sequence length further includes: acquiring an abnormal probability maximum value in the log weight value sequence, and determining the service state sequence length according to a time value corresponding to the abnormal probability maximum value.
[0011] Optionally, the model initial parameters include an initial state probability vector, a state transition probability matrix, and an observation probability matrix; and the backtracking analysis of the initial service state according to the Viterbi algorithm module, the model initial parameters, and the service state sequence length to obtain the target service state sequence of the system includes: calculating a service state probability of the system at a first time according to the initial service state, the initial state probability vector, the state transition probability matrix, and the observation probability matrix, and determining a target service state corresponding to a maximum value of the service state probability as the target service state at the first time; iteratively and recursively calculating a service state probability of the system at a second time according to the target service state at the first time, the initial state probability vector, the state transition probability matrix, and the observation probability matrix, and determining a target service state corresponding to a maximum value of the service state probability as the target service state at the second time, until a plurality of target service states at a plurality of times equal to the number of the service state sequence length are obtained; arranging the target service state at the first time, the target service state at the second time, and the target service states at the plurality of times equal to the number of the service state sequence length in time sequence to obtain the target service state sequence.
[0012] A service state prediction device includes: a historical data set acquisition module configured to acquire a log historical data set of a system; a hidden Markov model construction module configured to construct an initial hidden Markov model, input the log historical data set into the initial hidden Markov model, iteratively calculate and determine model initial parameters of the initial hidden Markov model, determine a target hidden Markov model according to the model initial parameters, and the target hidden Markov model includes a forward algorithm module and a Viterbi algorithm module; an abnormal probability distribution result calculation module configured to calculate an observation abnormal probability of any log observation sequence according to the forward algorithm module and the model initial parameter, and obtain an abnormal probability distribution result of the log; a business state sequence length determination module configured to perform weighted fusion on the abnormal probability distribution result, and determine a business state sequence length of the system according to the weighted abnormal probability distribution result; an initial business state acquisition module configured to acquire an initial business state of the system; a target business state sequence module configured to perform backtracking analysis on the initial business state according to the Viterbi algorithm module, the model initial parameter and the business state sequence length, and obtain a target business state sequence of the system.
[0013] An electronic device includes a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the following steps: acquire a log historical data set of a system; construct an initial hidden Markov model, input the log historical data set into the initial hidden Markov model, iteratively calculate and determine model initial parameters of the initial hidden Markov model, determine a target hidden Markov model according to the model initial parameters, and the target hidden Markov model includes a forward algorithm module and a Viterbi algorithm module; calculate an observation abnormal probability of any log observation sequence according to the forward algorithm module and the model initial parameter, and obtain an abnormal probability distribution result of the log; perform weighted fusion on the abnormal probability distribution result, and determine a business state sequence length of the system according to the weighted abnormal probability distribution result; acquire an initial business state of the system; perform backtracking analysis on the initial business state according to the Viterbi algorithm module, the model initial parameter and the business state sequence length, and obtain a target business state sequence of the system.
[0014] A computer readable storage medium stores a computer program, and the computer program is loaded by a processor to execute the steps in the business state prediction method described above.
[0015] In this embodiment of the invention, an initial Hidden Markov Model (HMM) is constructed to parse the system's historical log dataset, determining the initial parameters of the HMM to obtain the target HMM. This ensures the synergy and relevance between the target HMM and the historical log dataset. Based on the forward algorithm module, initial observation probabilities, and initial model parameters, the anomaly probability of any log observation sequence is calculated, predicting the anomaly probability distribution of the logs. This enables the prediction of the anomaly probability of the system's logs at any given time, based on any log observation sequence, ensuring the data integrity of the anomaly probability data. The anomaly probability distribution results are weighted and fused, and the length of the system's business state sequence is determined based on the weighted anomaly probability distribution results. This automatically determines the length of the business state sequence, i.e., the point in time where the system state needs attention, and is the recursive endpoint of the Viterbi algorithm. By backtracking the initial business state, the target business state sequence of the system is obtained, determining the system's business state at the point in time where attention is needed. Finally, the target HMM predicts and outputs the system's business state at a specific point in time, which helps to shift fault recovery and intervention from post-event remediation to during-event or even pre-event measures, improving the system's maintenance capabilities and business continuity assurance capabilities. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a schematic diagram of a scenario of an embodiment of the business status prediction system provided in this invention. Figure 2 This is a schematic diagram of another embodiment of the business status prediction system provided in this invention. Figure 3 This is a flowchart illustrating an embodiment of the business status prediction method provided by the present invention. Figure 4 A flowchart illustrating an embodiment of the business status prediction method provided by this invention. Figure 5 This is a schematic diagram of the structure of an embodiment of the business status prediction device provided in this invention. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0018] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0019] In the following description, specific embodiments of the present application will be described with reference to steps and symbolic representations of operations that are performed by one or more computers, unless indicated otherwise. As such, it will be understood that such steps and operations, while being described as being performed by a computer, include the manipulation by a processing unit of the computer of electrical signals representing data in a structured format and the maintenance of data in computer memory systems. Such description and representations are the means used by those skilled in the art of the data processing arts to most effectively convey the substance of their work to others skilled in the art. A
[0020] The term "module" or "unit" used herein can be regarded as a software object executed on the operating system. The different components, modules, engines and services described herein can be regarded as implemented objects on the operating system. The apparatus and method described herein are preferably implemented in software, but can also be implemented in hardware, both of which are within the scope of the present application.
[0021] The embodiments of the present application provide a service state prediction method and device, electronic equipment and a storage medium.
[0022] Please refer to Figure 1 , Figure 1 The scene schematic diagram of an embodiment of the service state prediction system provided by the embodiments of the present application can include a client 100 and a server 200, the client 100 and the server 200 are connected through a network, and the server 200 is integrated with a service state prediction device. The server 200 can be a work platform server (i.e., a server loaded with a work platform), such as Figure 1The server 200 can be accessed by the client 100. The server 200 in the embodiment of the present application is mainly used to obtain a log history data set and an initial observation probability of each log of the system; construct an initial hidden Markov model, input the log history data set into the initial hidden Markov model, iteratively calculate and determine model initial parameters of the initial hidden Markov model, determine a target hidden Markov model according to the model initial parameters, the target hidden Markov model including a forward algorithm module and a Viterbi algorithm module; calculate an observation anomaly probability of any log observation sequence according to the forward algorithm module, the initial observation probability and the model initial parameters, and obtain an anomaly probability distribution result of the log; perform weighted fusion on the anomaly probability distribution result, determine a business state sequence length of the system according to the weighted anomaly probability distribution result; obtain an initial business state of the system; and perform backtracking analysis on the initial business state according to the Viterbi algorithm module, the model initial parameters and the business state sequence length, and obtain a target business state sequence of the system.
[0023] In the embodiment of the present application, the server 200 can be a stand-alone server, or a server network or a server cluster composed of servers. For example, the server 200 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing. In the embodiment of the present application, communication between the server and the client can be realized through any communication mode, including but not limited to mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP) and User Datagram Protocol (UDP) protocol.
[0024] It can be understood that the client 100 used in the embodiments of the present application can be understood as a client device, which includes a device receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a client device can include a cellular or other communication device with a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific client 100 can be a desktop terminal or a mobile terminal, and the client 100 can be one of a mobile phone, a tablet computer, a notebook computer, etc.
[0025] Those skilled in the art can understand that Figure 1 The application environment shown in the above Figure 1 The application environment shown in the above Figure 1 The application environment shown in the above The application environment shown in the above
[0026] The application environment shown in the above The application environment shown in the above
[0027] In addition, as shown in the above Figure 2 The application environment shown in the above Figure 2 The application environment shown in the above The application environment shown in the above
[0028] It should be noted that Figures 1-2The scenario schematic diagram of the business state prediction system shown is merely an example, and the business state prediction system and scenario described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It can be known by a person of ordinary skill in the art that, as the business state prediction system evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.
[0029] The embodiments will be described in detail below with reference to specific examples.
[0030] In this embodiment, the business state prediction device will be described from the perspective of the business state prediction device, which can be specifically integrated in the server 200.
[0031] The present application provides a business state prediction method, please refer to Figure 3 , Figure 3 The flowchart of an embodiment of the business state prediction method provided by the embodiments of the present application includes: S301: obtaining a log historical data set of a system and an initial observation probability of each log; In a specific embodiment, log data is the "operation fingerprint" of a modern digital system, and almost all software and hardware systems with computing functions will generate log records in the working process. Therefore, the system in this application can be any modern digital system containing log data, which is not limited here.
[0032] The logs of the system are classified into operating system logs, database logs and middleware logs according to the hierarchical classification of the system architecture; classified into server operation logs according to the resource dimension; classified into business system logs according to the business dimension; and the logs of the system can also be additional information obtained by the system. The classification standard of the logs in this application is not limited, and the types and quantities of the logs are also not limited. However, it is obvious that the more abundant the types and quantities of the logs are, the more accurate the subsequent system business state prediction result is.
[0033] The log historical data set refers to the complete set of all recorded and collected log events of various types generated by the system in the past period of time, and in order to improve the effect of business state prediction, the log historical data set covers all related logs that can be generated by the system.
[0034] Since the log historical data set includes different log types, and different types of logs represent the characteristics of the system in different aspects, and because the same type of log can also contain different state information of the system, each log needs to be analyzed separately. Each log here is the smallest computing unit of log analysis, which can be a certain type of log or a certain log, and is subject to the system state analysis, which is not limited here.
[0035] The initial observation probability of each log is the observation state of the log at zero time obtained by analyzing the historical data set, which can be normal log, abnormal log, or other observation states of the log, which are not described here. For example, by statistical analysis of historical data, it can be calculated that when the "disk space is insufficient" log appears, the proportion (probability) of the system actually being in the "high-risk" state is how much.
[0036] S302: Construct an initial hidden Markov model, and input the log historical data set into the initial hidden Markov model, iteratively calculate and determine the model initial parameters of the initial hidden Markov model, determine the target hidden Markov model according to the model initial parameters, and the target hidden Markov model includes a forward algorithm module and a Viterbi algorithm module; In a specific embodiment, the hidden Markov model (HMM) is a statistical model used to describe a double random process composed of a hidden state sequence and an observable sequence. Its core assumption is that the current state of the system only depends on the previous state (Markov property), and the observation result only depends on the current state. In this application, HMM is used to predict the abnormal state of the business system through log data.
[0037] The hidden Markov model is defined by the following five-tuple: 1. Hidden state set .
[0038] Among them, the hidden state set is the actual state of the system (normal / abnormal), which cannot be directly obtained by the observer.
[0039] 2. Observation state set .
[0040] Measurable output (such as observable data of the system).
[0041] 3. State transition probability matrix (A) A = [a ij ] ).
[0042] Among them, a ij represents the probability of transitioning from state s i to s j , which satisfies the Markov property: the current state only depends on the previous state.
[0043] 4. Observation probability matrix (B) B = [b j (k)] ).
[0044] bj (k) indicates in the state s j the observation o k probability.
[0045] 5. initial state probability distribution (pi). π [ π i ]).
[0046] the probability of the system being in each hidden state at the initial time.
[0047] It should be noted that for a specific type or category of system, the hidden state set and the observation state set are determined and relatively fixed, so after determining the state transition probability matrix (A), the observation probability matrix (B) and the initial state probability vector (π) of the hidden Markov model, a reliable target hidden Markov model can be obtained.
[0048] In this embodiment, the model initial parameters include the state transition probability matrix (A), the observation probability matrix (B) and the initial state probability vector (π), so that by inputting the log historical data set into the initial hidden Markov model, the model initial parameters of the initial hidden Markov model can be directly iteratively calculated, and then the target hidden Markov model can be directly obtained.
[0049] The forward algorithm (Forward Algorithm), also known as the forward algorithm, is a key dynamic programming method for efficiently calculating the probability P(O|λ) of the observation sequence in the hidden Markov model. It avoids enumerating all possible hidden state paths, reduces the exponential computational complexity to polynomial, and is a key tool for solving the "probability calculation problem" of the hidden Markov model.
[0050] The Viterbi algorithm is a dynamic programming algorithm for solving the most likely state sequence in the hidden Markov model. Its core goal is to backtrack and deduce the optimal hidden state sequence that produces the observation based on the observation sequence. The Viterbi algorithm calculates the conditional probability of all possible hidden state sequences under a given observation sequence through dynamic programming, and finally selects the path with the maximum probability as the optimal solution.
[0051] S303: Calculate the observation anomaly probability of any log observation sequence according to the forward algorithm module, the initial observation probability and the model initial parameters, and obtain the anomaly probability distribution result of the log; In an embodiment, the log observation sequence refers to an observation data set of a log or a group of logs of the same type arranged in chronological order. By analyzing the abnormal probability of each observation element in the log observation sequence, whether the system has abnormal behavior at the time point corresponding to the observation element can be identified, and thus real-time detection and early warning of the system running state can be realized.
[0052] The abnormal probability distribution result shows the observation abnormal probability of any log or observation sequence of a log of interest in the system, and is sorted in a standard manner, so as to timely discover logs and time points with high observation abnormal probability, and thus accurately locate potential risk points of the system.
[0053] Further, since the forward algorithm is provided with a recursive mechanism of dynamic programming, the path enumeration problem with exponential complexity can be converted into polynomial-level calculation, so that the abnormal degree of each observation sequence can be accurately evaluated by layer-by-layer accumulation of probability, which not only greatly improves the calculation efficiency and ensures the completion of abnormal detection of large-scale log data within a limited time, but also adaptively obtains a variety of reasonable log observation sequences and observation abnormal probability of each log observation sequence according to the characteristics of the system itself, so as to obtain the abnormal probability distribution result of the system as a whole.
[0054] S304: Weighted fusion is performed on the abnormal probability distribution result, and the length of the business state sequence of the system is determined according to the weighted abnormal probability distribution result; In an embodiment, the length of the business state sequence refers to the number or sum of steps of different "states" experienced by a business entity in its life cycle. In the present application, since the log observation sequence and the business state sequence also contain time point information, after the time point of interest is determined, the time difference value can be determined in combination with the known time information, so as to determine the specific length of the business state sequence.
[0055] In addition, since the abnormal probability distribution result contains the observation abnormal probability of multiple logs, and different log types have different influences on the system business state, and the observation abnormal probability of different logs at the same time point is also different, by weighted fusion of the abnormal probability distribution result, the abnormal influence of various logs of the system can be comprehensively considered, so as to intuitively and accurately show the overall business state of the system, so as to determine the length of the business state sequence.
[0056] S305: Obtain the initial business state of the system; In a specific embodiment, the initial service state refers to a basic state of the system at the beginning of the service, generally normal service of the system or abnormal service of the system, and the initial service state is determined to provide a reference for subsequent service state conversion, and ensure the analyzability of the service state sequence.
[0057] S306: According to the Viterbi algorithm module, the model initial parameter and the service state sequence length, the initial service state is analyzed in a backward manner to obtain the target service state sequence of the system.
[0058] In a specific embodiment, the backward analysis refers to reiteratively and progressively determining the initial service state on the basis of the service state sequence length, so as to determine a service state sequence equal in number to the service state sequence length and having the maximum abnormal probability value, obtain the target service state sequence of the system, and further determine the service state of the system at the time point that needs to be focused on.
[0059] It should be noted that, since the forward algorithm can only predict the observation abnormal probability of the log at a time point, and cannot directly show the hidden state, that is, the service state of the system, in order to reveal the change of the service state of the system, the Viterbi algorithm is needed for backward analysis, and the service state of the system at each time point is gradually deduced by means of the dynamic programming principle and in combination with the model initial parameter, and finally a complete target service state sequence is constructed.
[0060] In the embodiment of the application, the initial hidden Markov model is constructed to analyze the log historical data set of the system, the model initial parameter of the initial hidden Markov model is determined to obtain the target hidden Markov model, which can ensure the synergy and correlation of the target hidden Markov model and the log historical data set; the observation abnormal probability of any log observation sequence is calculated according to the forward algorithm, the initial observation probability and the model initial parameter, the abnormal probability distribution result of the log is predicted, the observation abnormal probability of the log at any time point based on any log observation sequence is predicted, the data integrity of the observation abnormal probability is ensured; the abnormal probability distribution result is weighted and fused, the service state sequence length of the system is determined according to the weighted abnormal probability distribution result, the service state sequence length can be automatically determined, that is, the time point of the system state that needs to be focused on, the recursion end point of the Viterbi algorithm; the initial service state is analyzed in a backward manner according to the Viterbi algorithm, the model initial parameter and the service state sequence length, the target service state sequence of the system is obtained, the service state of the system at the time point that needs to be focused on can be determined, finally, the service state of the system at a specific time point is predicted and output by the target hidden Markov model, which is conducive to advancing the fault recovery and intervention from post-repair to in-process or even pre-process, and improving the maintenance ability and service continuity guarantee ability of the system.
[0061] In a specific embodiment, in S301, the log history dataset includes logs of an operating system database, a database, middleware, a server, and a business system of the system, and also includes logs of environmental control, a business system state, and a server, without limitation.
[0062] Specifically, the logs of the operating system database, the database, the middleware, the server, and the business system are collected using rsyslog or nxlog, and the environmental control, the business system state, and the server load information are obtained through an API. The various logs are integrated through a log collection tool, ensuring the comprehensiveness and real-time performance of the log data.
[0063] In a specific embodiment, in S302, the model initial parameters include an initial state probability vector, a state transition probability matrix, and an observation probability matrix; the log history dataset is input to an initial hidden Markov model, and the model initial parameters of the initial hidden Markov model are iteratively calculated and determined, specifically including: The normal state occurrence frequency, the abnormal state occurrence frequency, the state transition probability set, and the log state conditional probability set of the log history dataset are obtained; the initial state probability vector is determined according to the normal state occurrence frequency and the abnormal state occurrence frequency; the state transition probability matrix is determined according to the plurality of state transition probabilities in the state transition probability set; and the observation probability matrix is determined according to the plurality of log state conditional probabilities in the log state conditional probability set.
[0064] In a specific embodiment, the log state conditional probability refers to the conditional probability of generating an observation log event under different log states, and is a probability mapping set from a state to an observation. For example, given a state "error", the probability of observing a log event "memory overflow" can be 0.7; given a state "normal", the probability of observing an event "user login" can be 0.9, and so on. The log state conditional probability set refers to a set of all log state conditional probabilities obtained based on the log history dataset.
[0065] It should be noted that the initial state probability vector refers to the probability distribution of the hidden state of the system. In this embodiment, the hidden state of the system includes a normal state and an abnormal state, and thus the initial state probability vector is (normal state probability, abnormal state probability). The normal state occurrence frequency and the abnormal state occurrence frequency are used to represent the normal state probability and the abnormal state probability of the system, respectively, so as to obtain the initial state probability vector.
[0066] In a specific embodiment, when the normal state occurrence frequency of the log history dataset is 0.6 and the abnormal state occurrence frequency is 0.4, the initial state probability vector is (0.6, 0.4).
[0067] The state transition probability matrix is a matrix composed of probabilities of transition from a state to another state, representing the likelihood of transition between different states of the system. Specifically, the elements of the state transition probability matrix include the probability of transition from a normal state to a normal state, the probability of transition from a normal state to an abnormal state, the probability of transition from an abnormal state to a normal state, and the probability of transition from an abnormal state to an abnormal state, which are obtained by statistics of state transition in the log historical data set, ensuring that the matrix reflects the state change trend in the actual operation of the system.
[0068] The observation probability matrix represents the probability of generating a specific observation event under each state, and its elements are determined by the log state conditional probability set.
[0069] In this embodiment, based on the log historical data set of the system, the data and data conversion in the log historical data set are analyzed, the initial parameters of the model are directly determined according to the log historical data set, and the target hidden Markov model highly consistent with the system itself is constructed, thereby improving the state recognition accuracy and the system anomaly detection efficiency, and ensuring the reliability of the target hidden Markov model.
[0070] In a specific embodiment, in S303, the observation anomaly probability of any log observation sequence is calculated according to the forward algorithm module, the initial observation probability and the initial parameters of the model, and the abnormal probability distribution result of the log is obtained, specifically including: based on the initial observation probability, the initial state probability vector, the state transition probability matrix and the observation probability matrix of any log, the observation probability of any log at any time is recursively calculated; the observation probability is arranged in an array to obtain the abnormal probability distribution result of the log; wherein the abnormal probability distribution result includes the observation probability of any log at any time.
[0071] In this embodiment, the forward algorithm module decomposes the global path probability calculation into the superposition of local state transition through the recursive mechanism of dynamic programming, retaining the joint probability information of all paths. Since the recursive calculation can be stopped at any time point, the forward algorithm module can determine the observation probability of any log at any time. Therefore, by arranging the observation probability in an array according to a certain order, the abnormal probability distribution result representing the observation probability of the system can be obtained.
[0072] In a specific embodiment, the abnormal probability distribution result can be represented in the form of a matrix, where the rows of the matrix represent the observation probabilities of a certain log at different times, and the columns of the matrix correspond to different logs. Other forms can also be used, which are not limited herein.
[0073] In a specific embodiment, in S304, the system includes multiple logs; the abnormal probability distribution results are weighted and fused, and the service state sequence length of the system is determined according to the weighted abnormal probability distribution results, specifically including: acquiring the weight of each log and the abnormal probability distribution results of all logs at a certain time; performing weighted calculation on the weight of each log and the abnormal probability distribution results of the log to determine the log weight value of all logs at a certain time, and determining the log weight value sequence according to the log weight values at all times; acquiring a preset target time, and determining the service state sequence length according to the time value of the preset target time in the log weight value sequence.
[0074] In a specific embodiment, the system logs are divided into operating system logs, database logs and middleware logs, and the weights of the operating system logs, the database logs and the middleware logs are set to 0.7, 0.2 and 0.1 respectively according to the needs, so as to distinguish the importance of different log types, and flexibly modify the offset direction of the log weight value according to the needs, so that the log weight value sequence is more in line with the actual needs.
[0075] The log weight value sequence is a time series data, and the value at each time point represents the comprehensive abnormal possibility score obtained by weighting and fusing all system log types (such as operating system, database, middleware logs) according to their importance weights (such as 0.7, 0.2, 0.1) at the time.
[0076] After obtaining the log weight value sequence, since the log weight value sequence itself contains time information, according to actual needs, it is generally necessary to determine the log state of the system at a specific time, therefore, the preset target time is matched with the time value in the log weight value sequence, the time difference of the preset target time compared with the current time or a certain reference time is found, and the service state sequence length is directly determined according to the time difference.
[0077] In another specific embodiment, after obtaining the log weight value sequence, the maximum abnormal probability value in the log weight value sequence can also be acquired, and the service state sequence length is determined according to the time value corresponding to the maximum abnormal probability value.
[0078] By taking the maximum abnormal probability value as the reference to determine the time at which the system is most likely to have an abnormality, the time difference of the time at which the system is most likely to have an abnormality compared with the current time or a certain reference time is determined, and the service state sequence length is further accurately locked.
[0079] In a specific embodiment, after determining the length of the service state sequence in S305, the initial service state is analyzed by backtracking according to the Viterbi algorithm module, the initial model parameters and the length of the service state sequence, to obtain the target service state sequence of the system, including: calculating the service state probability of the system at the first time according to the initial service state, the initial state probability vector, the state transition probability matrix and the observation probability matrix, and determining that the service state corresponding to the maximum service state probability is the target service state at the first time; iteratively and recursively calculating the service state probability of the system at the second time according to the target service state at the first time, the initial state probability vector, the state transition probability matrix and the observation probability matrix, and determining that the service state corresponding to the maximum service state probability is the target service state at the second time, until a plurality of target service states at multiple times equal to the number of the length of the service state sequence are obtained; arranging the target service state at the first time, the target service state at the second time and the target service states at the multiple times equal to the number of the length of the service state sequence in chronological order to obtain the target service state sequence.
[0080] In a specific embodiment, the initial service state refers to the service state of the system at the starting time, which is usually set to the normal state.
[0081] It should be noted that the initial service state and the initial state probability vector are respectively the service state and the probability distribution of the system at the zeroth time, which are the basic input parameters of the Viterbi algorithm. After determining the length of the service state sequence, the number of iterations of the Viterbi algorithm is directly determined. Since the Viterbi algorithm is iterated based on the service state of the system, the target service state sequence obtained finally directly reflects the service state of the system at the target time.
[0082] In this embodiment, the service state of the system is analyzed by the Viterbi algorithm to directly determine the service state of the system at the target time, which realizes intuitive acquisition of the service state information of the system, solves the problem that the service state of the system cannot be directly displayed in the forward algorithm, and improves the efficiency and intuitiveness of service state recognition.
[0083] In a specific embodiment, after obtaining the service state of the system at the target time, since the service state of the system is generally represented as normal or abnormal, it is difficult to quickly troubleshoot the system through the service state of the system. Therefore, after obtaining the target service state sequence of the system, it further includes: determining the abnormal service time of the system according to the target service state sequence; filtering out the abnormal observation probability of all logs at the abnormal service time from the abnormal probability distribution result; comparing the abnormal observation probabilities of all logs, and determining that the log corresponding to the maximum abnormal observation probability is the abnormal log.
[0084] The abnormal business time refers to a time at which a target business state sequence result of the system is an abnormal state.
[0085] In this embodiment, in order to find problems of the system in time and pertinently, the abnormal business time of the system is taken as a breakthrough point, all logs corresponding to the time are found, and the log entries with higher abnormal observation probability are analyzed in depth, so as to determine the abnormal log causing the system to be abnormal, and thus the system is quickly maintained.
[0086] In a specific embodiment, the state of business abnormality has two kinds (normal / abnormal), as a hidden state, the log information found by the viewpoint has database logs, middleware logs, business system logs, and monitoring index data server load, computer room dynamic ring information, these information can be used as observation sequences in a hidden Markov model (HMM), by analyzing these data, the state (normal or abnormal) of the current business system is inferred, and the specific steps are as follows: 1) Hidden state: normal / abnormal; 2) Observation state: database logs (normal log / abnormal log), middleware logs (normal log / abnormal log), business system logs (normal log / abnormal log), server load (low / high), and ring control state (normal / abnormal); 3) Initial state distribution: p1=P(business normal state), p2=P(business abnormal state); 4) State transition probability matrix: transition probability matrix of two states; 5) Observation probability matrix: ; (B1=P(database log normal | business system normal)); (B2=P(database log abnormal | business system normal)); (B3=P(middleware log normal | business system normal)); (B4=P(middleware log abnormal | business system normal)); (B5=P(server load high | business system normal)); (B6=P(server load low | business system normal)); (B7=P(ring control state normal | business system normal)); (B8=P(ring control state abnormal | business system normal)).
[0087] Please refer to Figure 4 , Figure 4The flow framework diagram of an embodiment of the service state prediction method provided by the embodiment of the application clearly shows the double-layer structure of the hidden Markov model (HMM): taking the service system state (normal / abnormal) as the hidden state, and taking the multi-element data such as the database log, the middleware log and the server load as the observation sequence. The flow framework diagram essentially constructs a complete technical framework from the log data collection to the intelligent prediction of the service state, and realizes the dynamic coupling optimization of the prediction system and the log analysis system, so that the service abnormality prediction is changed from passive response to active early warning, and the operation and maintenance response time window is moved forward.
[0088] Based on the above service state prediction method, the prediction result is reversely applied to the log analysis strategy adjustment through the dynamic feedback mechanism, forming an iterative optimization path of "prediction-optimization-re-prediction", and improving the model prediction efficiency.
[0089] In order to better implement the service state prediction method provided by the embodiment of the application, the embodiment of the application further provides a device based on the above service state prediction method. The meanings of the terms are the same as those in the above service state prediction method, and the specific implementation details can be referred to the description in the method embodiment.
[0090] Please refer to Figure 5 , Figure 5 The structural schematic diagram of an embodiment of the service state prediction device provided by the embodiment of the application, wherein the service state prediction device 500 can include: A historical data set acquisition module 501 configured to acquire the log historical data set of the system; A hidden Markov model construction module 502 configured to construct an initial hidden Markov model, input the log historical data set into the initial hidden Markov model, iteratively calculate and determine the model initial parameters of the initial hidden Markov model, determine a target hidden Markov model according to the model initial parameters, and the target hidden Markov model includes a forward algorithm module and a Viterbi algorithm module; An abnormal probability distribution result calculation module 503 configured to calculate the observation abnormal probability of any log observation sequence according to the forward algorithm module and the model initial parameters, and obtain the abnormal probability distribution result of the log; A service state sequence length determination module 504 configured to weight and fuse the abnormal probability distribution result, and determine the service state sequence length of the system according to the weighted abnormal probability distribution result; An initial service state acquisition module 505 configured to acquire the initial service state of the system; A target service state sequence module 506 configured to backtrack and analyze the initial service state according to the Viterbi algorithm module, the model initial parameters and the service state sequence length, and obtain the target service state sequence of the system.
[0091] This invention also provides an electronic device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, specifically: The electronic device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 601 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operation of the storage medium, user interface, and application programs, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.
[0092] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store applications required for operating the storage medium and at least one function (such as sound playback function, image playback function, etc.); the data storage area may store data created according to the use of the electronic device. In addition, the memory 602 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.
[0093] The electronic device also includes a power supply 603 that supplies power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 via a power management storage medium, thereby enabling functions such as charging, discharging, and power consumption management through the power management storage medium. The power supply 603 may also include one or more DC or AC power supplies, recharge storage media, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0094] The electronic device can further include an input unit 604 which can be used to receive inputted digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.
[0095] Although not shown, the electronic device can further include a display unit or the like, which will not be described here. In particular in the present embodiment, the processor 601 in the electronic device will load one or more executable files corresponding to processes of one or more application programs into the memory 602 according to the following instructions, and run the application programs stored in the memory 602 by the processor 601, thereby implementing various functions as follows: Obtain a log history data set of a system and an initial observation probability of each log; construct an initial hidden Markov model, input the log history data set into the initial hidden Markov model, iteratively calculate and determine model initial parameters of the initial hidden Markov model, determine a target hidden Markov model according to the model initial parameters, the target hidden Markov model including a forward algorithm module and a Viterbi algorithm module; calculate an observation anomaly probability of any log observation sequence according to the forward algorithm module, the initial observation probability and the model initial parameters, and obtain an anomaly probability distribution result of the log; perform weighted fusion on the anomaly probability distribution result, determine a business state sequence length of the system according to the weighted anomaly probability distribution result; obtain an initial business state of the system; perform backtracking analysis on the initial business state according to the Viterbi algorithm module, the model initial parameters and the business state sequence length, and obtain a target business state sequence of the system.
[0096] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by related hardware controlled by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.
[0097] To this end, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute steps in any one of the business state prediction methods provided by the embodiments of the present application. For example, the computer program loaded by the processor can execute the following steps: The log history data set of the system and initial observation probability of each log are acquired; an initial hidden Markov model is constructed, and the log history data set is input into the initial hidden Markov model, model initial parameters of the initial hidden Markov model are determined by iterative calculation, a target hidden Markov model is determined according to the model initial parameters, the target hidden Markov model includes a forward algorithm module and a Viterbi algorithm module; observation abnormal probability of any log observation sequence is calculated according to the forward algorithm module, the initial observation probability and the model initial parameters, and abnormal probability distribution results of the logs are obtained; the abnormal probability distribution results are weighted and fused, and the business state sequence length of the system is determined according to the weighted abnormal probability distribution results; the initial business state of the system is acquired; and the initial business state is analyzed by backtracking according to the Viterbi algorithm module, the model initial parameters and the business state sequence length, and a target business state sequence of the system is obtained.
[0098] The specific implementation of each operation can refer to the foregoing embodiments, and will not be described here.
[0099] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0100] The computer program stored in the computer readable storage medium can execute the steps of any one of the business state prediction methods provided by the embodiments of the present application, and thus can achieve the beneficial effects of any one of the business state prediction methods provided by the embodiments of the present application. Details are described in the foregoing embodiments, and will not be described here.
[0101] The above describes in detail a business state prediction method, device, electronic equipment and storage medium provided by the embodiments of the present application. The principle and implementation manner of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed; and the above description should not be understood as a limitation of the present application.
Claims
1. A business status prediction method, characterized in that, include: Obtain the system's historical log dataset and the initial observation probability for each log entry; An initial Hidden Markov Model is constructed, and the log history dataset is input into the initial Hidden Markov Model. The initial parameters of the initial Hidden Markov Model are iteratively calculated and determined. The target Hidden Markov Model is determined based on the initial parameters. The target Hidden Markov Model includes a forward algorithm module and a Viterbi algorithm module. The abnormal probability of any log observation sequence is calculated based on the forward algorithm module, the initial observation probability, and the initial parameters of the model, and the abnormal probability distribution of the log is obtained. The abnormal probability distribution results are weighted and fused, and the length of the system's business state sequence is determined based on the weighted abnormal probability distribution results; Obtain the initial business state of the system; Based on the Viterbi algorithm module, the initial parameters of the model, and the length of the business state sequence, a backtracking analysis is performed on the initial business state to obtain the target business state sequence of the system.
2. The business status prediction method according to claim 1, characterized in that, The initial parameters of the model include an initial state probability vector, a state transition probability matrix, and an observation probability matrix; the step of inputting the log history dataset into the initial hidden Markov model, iteratively calculating and determining the initial parameters of the initial hidden Markov model includes: Obtain the frequency of normal states, frequency of abnormal states, state transition probability set, and log state conditional probability set of the log historical dataset; The initial state probability vector is determined based on the frequency of occurrence of the normal state and the frequency of occurrence of the abnormal state. The state transition probability matrix is determined based on multiple state transition probabilities in the state transition probability set; The observation probability matrix is determined based on multiple log state condition probabilities in the log state condition probability set.
3. The business status prediction method according to claim 1, characterized in that, The initial parameters of the model include an initial state probability vector, a state transition probability matrix, and an observation probability matrix; the calculation of the observation anomaly probability of any log observation sequence based on the forward algorithm module, the initial observation probability, and the initial parameters of the model, to obtain the log anomaly probability distribution result, includes: Based on the initial observation probability of any log, the initial state probability vector, the state transition probability matrix, and the observation probability matrix, the observation probability of any log at any time is recursively calculated. The observation probabilities are arranged in an array to obtain the anomaly probability distribution of the logs; The anomaly probability distribution result includes the observation probability of any log at any time.
4. The business status prediction method according to claim 3, characterized in that, After obtaining the target service state sequence of the system, the following is also included: The abnormal service time of the system is determined based on the target service state sequence; The abnormal observation probability of all logs at the time of the abnormal business is selected from the abnormal probability distribution results. Compare the anomaly observation probabilities of all logs and determine the log with the highest anomaly observation probability as the anomaly log.
5. The business status prediction method according to claim 1, characterized in that, The system includes multiple logs; the weighted fusion of the anomaly probability distribution results, and the determination of the system's business state sequence length based on the weighted anomaly probability distribution results, includes: Obtain the weight of each type of log and the anomaly probability distribution of all logs at a certain moment; The weight of each type of log and the anomaly probability distribution result of that type of log are weighted and calculated to determine the log weight of all logs at a certain time, and the log weight sequence is determined based on the log weight of all time. Obtain a preset target time, and determine the length of the business status sequence based on the time value of the preset target time in the log weight sequence.
6. The business status prediction method according to claim 5, characterized in that, Determining the length of the service state sequence further includes: Obtain the maximum anomaly probability in the log weight sequence, and determine the length of the business status sequence based on the time value corresponding to the maximum anomaly probability.
7. The business status prediction method according to claim 1, characterized in that, The initial parameters of the model include an initial state probability vector, a state transition probability matrix, and an observation probability matrix; the backtracking analysis of the initial business state based on the Viterbi algorithm module, the initial parameters of the model, and the length of the business state sequence yields the target business state sequence of the system, including: Based on the initial service state, the initial state probability vector, the state transition probability matrix, and the observation probability matrix, the service state probability of the system at the first moment is calculated, and the service state corresponding to the maximum service state probability is determined as the target service state at the first moment. Based on the target service state at the first moment, the initial state probability vector, the state transition probability matrix, and the observation probability matrix, the service state probability of the system at the second moment is calculated iteratively and recursively, and the service state corresponding to the maximum service state probability is determined as the target service state at the second moment, until multiple target service states at multiple moments with a number equal to the length of the service state sequence are obtained. The target service state at the first time point, the target service state at the second time point, and the target service states at multiple times with a length equal to that of the service state sequence are arranged in chronological order to obtain the target service state sequence.
8. A business status prediction device, characterized in that, include: The historical dataset acquisition module is used to acquire the system's historical log dataset. The Hidden Markov Model (HMM) construction module is used to construct an initial HMM and input the log history dataset into the initial HMM. Iteratively calculates and determines the initial parameters of the initial HMM, and determines the target HMM based on the initial parameters. The target HMM includes a forward algorithm module and a Viterbi algorithm module. The anomaly probability distribution result calculation module is used to calculate the observation anomaly probability of any log observation sequence based on the forward algorithm module and the model initial parameters, and obtain the anomaly probability distribution result of the log; The business state sequence length determination module is used to perform weighted fusion on the abnormal probability distribution results and determine the business state sequence length of the system based on the weighted abnormal probability distribution results. The initial business status acquisition module is used to acquire the initial business status of the system; The target business state sequence module is used to perform backtracking analysis on the initial business state based on the Viterbi algorithm module, the initial parameters of the model, and the length of the business state sequence to obtain the target business state sequence of the system.
9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the following steps: Obtain the system's historical log dataset; An initial Hidden Markov Model is constructed, and the log history dataset is input into the initial Hidden Markov Model. The initial parameters of the initial Hidden Markov Model are iteratively calculated and determined. The target Hidden Markov Model is determined based on the initial parameters. The target Hidden Markov Model includes a forward algorithm module and a Viterbi algorithm module. The anomaly probability of any log observation sequence is calculated based on the forward algorithm module and the initial parameters of the model to obtain the anomaly probability distribution result of the log; The abnormal probability distribution results are weighted and fused, and the length of the system's business state sequence is determined based on the weighted abnormal probability distribution results; Obtain the initial business state of the system; Based on the Viterbi algorithm module, the initial parameters of the model, and the length of the business state sequence, a backtracking analysis is performed on the initial business state to obtain the target business state sequence of the system.
10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to execute the steps in the business status prediction method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Method for automatically analyzing logs
CN111367964A
Abnormal traffic detection method and system, storage medium and terminal
CN114363061A
Business handling probability prediction method and device, computer equipment and storage medium
CN115099896A
Safety analysis method and system based on user behavior data
CN117633787A
Business topology detection analysis method and device, equipment and storage medium
CN119011446A
Cited By
Pole number plate missing detection method, device and equipment based on space-time fusion perception and medium
CN121366416A
