Intelligent diagnosis model reliability modeling and evaluation method based on Markov chain

Abstract

The invention discloses a method for modeling and evaluating reliability of an intelligent diagnosis model, which comprises the following steps of: firstly, regarding an output sequence of the intelligent diagnosis model as a Markov chain, defining a correct classification state and an error state according to a classification result, and calculating a corresponding state transition space and a transition probability of each state; and the transition probability matrix can accurately reflect the classification performance of the model. Furthermore, the long-term stability probability of each state is determined by solving the steady-state distribution equation, the steady-state probability of the correct classification state is defined as the reliability index of the model, and the long-term classification stability of the model is quantitatively evaluated. And finally, calculating the steady-state distribution, so that the method can update the reliability index in real time in the model operation process to adapt to the dynamic change of different tasks and data sets. The invention provides an accurate and efficient intelligent diagnosis model reliability evaluation means, model design optimization and parameter adjustment are guided, and the stability of the diagnosis model in practical engineering application is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of reliability evaluation, in particular to reliability modeling and evaluation for intelligent diagnosis algorithm. BACKGROUND

[0002] With the rapid development of prognostics and health management (PHM) technology, intelligent diagnosis technology is increasingly widely used in various industrial equipment. Intelligent diagnosis technology realizes fault detection, fault diagnosis and fault prediction by real-time monitoring of equipment operating state and using data-driven algorithms, which significantly improves the safety and reliability of equipment and is widely used in key fields such as power equipment, mechanical equipment and aero-engine. In modern industrial systems, equipment failure may not only lead to a decline in production efficiency, but also may cause serious safety accidents, so early fault identification and health management of equipment through intelligent diagnosis technology has become a core demand in the industrial field.

[0003] In the field of intelligent diagnosis, data-driven PHM technology has become a research hotspot. Data-driven PHM technology relies on big data, deep learning and machine learning algorithms to extract fault features from equipment operating data, build intelligent diagnosis models, and realize automatic identification and prediction of equipment fault state. This method no longer depends on traditional physical models, but trains deep neural networks or other data-driven models to automatically learn the mapping relationship between fault features and fault state, and has shown excellent fault detection and diagnosis performance in key equipment such as turbines and bearings.

[0004] However, the evaluation of existing intelligent diagnosis algorithms mostly focuses on static indicators such as classification accuracy, precision, recall rate, ignoring the reliability and stability of the model in long-term operation. In actual application, intelligent diagnosis models do not always maintain high accuracy, and their classification performance may fluctuate with changes in equipment operating conditions, fault mode switching, data distribution fluctuations and other factors, and even experience a decline in classification accuracy in long-term operation. Especially in complex industrial environments, intelligent diagnosis models face various uncertainties, such as fluctuations in equipment operating conditions, changes in sensor accuracy, fluctuations in data quality and the emergence of new fault modes, making it difficult to guarantee the reliability and stability of the model's classification results. Existing methods only evaluate based on the classification accuracy on a single test set, which cannot effectively reflect the stability and reliability of the model in long-term operation.

[0005] To address the issue of long-term reliability assessment of intelligent diagnostic models, current methods primarily focus on static evaluation based on test sets and performance statistics from multiple tests. However, these methods struggle to comprehensively reflect the model's reliability over long-term operation. Currently, in industry and academia, common performance evaluation methods for intelligent diagnostic models mainly fall into three categories: 1) Static evaluation method based on test set This type of method measures model performance by evaluating metrics such as classification accuracy, recall, and F1 score on a test set. Typically, evaluation is performed on a fixed dataset after model training, such as CN 11894094's evaluation of a wind turbine equipment state model using a test set. While this method effectively measures classification performance in the short term, it fails to reflect the model's stability and reliability over long-term operation. Especially in practical applications, test sets often differ in data distribution from the actual application environment; high classification accuracy on the test set does not necessarily indicate long-term reliability.

[0006] 2) Statistical evaluation method based on multiple test sets To compensate for the shortcomings of static testing and evaluation, some studies employ multiple test set evaluation, which involves testing the model performance multiple times on multiple test sets and statistically analyzing metrics such as average classification accuracy and standard deviation. For example, CN 114792209 assesses engineering construction risks based on test set statistical probabilities. This method evaluates model stability through multiple tests and can reflect performance fluctuations under different data distributions to some extent. However, this method still relies on test set data and requires a large amount of test data and testing time, making it unable to effectively assess the dynamic reliability of the model during long-term online operation.

[0007] 3) Incremental evaluation method based on dataset updates This type of method, by adding new data to the test set in batches and continuously evaluating the model's classification performance on the new data, can simulate the model's dynamic performance over long-term operation to some extent. For example, the CN 117833351 incremental learning paradigm risk assessment model performs online real-time assessment of the operational risks of new energy power systems. However, this method can still only evaluate the static performance of the model and cannot quantify the model's state changes and reliability level over long-term operation.

[0008] While the above methods can evaluate the performance of intelligent diagnostic models to some extent, they have the following two shortcomings that limit their application in long-term reliability assessment: 1) The evaluation result depends on the number of test sets and data distribution, and cannot effectively reflect the dynamic classification performance change of the model in long-term online operation. In particular, in actual application, the classification performance of the model may fluctuate with the change of equipment working condition, data distribution and fault mode, and the existing method cannot capture this dynamic change.

[0009] 2) The transition rule between correct classification and error classification of the model in long-term operation cannot be quantified, and the description of the stability of the classification performance of the model is lacking. Even if the model has a high classification accuracy in the short term, over time, the model may gradually show performance degradation or classification fluctuation problems, and the existing method cannot effectively identify and quantify this long-term performance change.

[0010] Therefore, it is urgent to establish a structured and dynamic model performance evaluation mechanism that can reveal the state change trend of the model in actual application and quantitatively evaluate the classification reliability of the model in long-term operation, thereby providing theoretical support and technical means for the reliability guarantee of intelligent diagnosis models in actual industrial scenes. SUMMARY

[0011] The present application aims to solve the problem of lack of long-term operation reliability evaluation of existing intelligent diagnosis models, and proposes a Markov chain-based intelligent diagnosis model reliability modeling and evaluation method, which realizes the quantitative evaluation of long-term classification stability of the model by regarding the classification results of the model as state sequences and establishing state transition relationship. This method not only reflects the classification fluctuation characteristics of the model in actual operation, but also has good adaptability and real-time performance, and is suitable for various industrial intelligent diagnosis scenes.

[0012] The purpose of the present application is achieved by the following technical solution: a Markov chain-based intelligent diagnosis model reliability modeling and evaluation method, which comprises the following steps: (1) Based on the actual application task, the classification judgment result of whether the bearing is faulty is obtained by the intelligent diagnosis model, which is recorded as correct classification state and error classification state; (2) The classification states of whether the bearing is faulty are arranged in order to construct a state chain reflecting the running dynamics of the Markov model; (3) The number of transitions between adjacent classification states is counted, the conversion of the Markov model between different states is recorded, and the state transition probability matrix of the model is calculated by using maximum likelihood estimation; (4) The stable probability of the Markov model in each state during operation is solved based on the state transition probability matrix of the model, i.e. the steady-state distribution of the Markov chain; (5) The steady-state probability of the correct classification state is extracted based on the steady-state distribution, and the reliability index of the intelligent diagnosis model is defined as the steady-state probability of the correct classification state, and the performance of the model is judged.

[0013] Further, the intelligent diagnosis model outputs a series of bearing fault prediction labels for the input bearing operation data, and the consistent labels are correct classification states, and the inconsistent labels are error classification states.

[0014] Further, in step (2), the output sequence of the intelligent diagnosis model is regarded as a Markov chain, and the output result of the diagnosis model is defined as a plurality of limited Markov states for a specific fault diagnosis task, the transition between the states is only related to the current classification output, and a determined state transition space is formed.

[0015] Further, it specifically includes four possible state transitions: 1) the current is an error classification state, and the next time is still an error classification state; 2) the current is an error classification state, and the next time is changed to a correct classification state; 3) the current is a correct classification state, and the next time is changed to an error classification state; 4) the current is a correct classification state, and the next time is still a correct classification state.

[0016] Further, based on the state sequence generated by the actual operation result of the intelligent diagnosis model, the occurrence frequency of the four types of state transitions is counted, the state transition probability is equal to the frequency of the type of transition in the state based on the optimal solution given by the maximum likelihood estimation, and then the state transition probability matrix is constructed.

[0017] Further, in step (4), the steady-state distribution corresponding to the state transition probability matrix is solved, the probability distribution law of each state under long-term operation is determined, so as to evaluate the stable classification performance of the model when time tends to infinity.

[0018] Further, the power iteration method is used to calculate the steady-state distribution, that is, an arbitrary initial distribution vector is selected for iterative calculation until the convergence condition is met.

[0019] Further, in step (5), according to the reliability index evaluation result, a quantitative decision basis is provided for the selection, optimization and deployment of the intelligent diagnosis model.

[0020] Compared with the prior art, the beneficial effects of the present application are: 1) from static accuracy evaluation to dynamic reliability modeling, which can effectively depict the evolution trend of model performance.

[0021] The performance evaluation of existing intelligent diagnosis models generally relies on static indicators such as classification accuracy and F1 score on the test set, which cannot reflect the classification stability and performance fluctuation of the model in the long-term running process. The present application realizes the dynamic modeling of the classification behavior of the model by constructing a Markov chain state model and converting the diagnosis results into a state sequence, effectively making up for the deficiency of traditional methods that cannot depict the time evolution characteristics. The output results of the intelligent diagnosis model are abstracted into two basic states of "correct classification state" and "error classification state", and four transition relationships between them are defined, systematically revealing the transition law between the states of the model in the running process. This state modeling method raises the evaluation of diagnosis performance from the result level to the behavior level, enhancing the interpretability and theoretical depth of the evaluation results.

[0022] 2) The evaluation index is structured and interpretable, and can quantitatively present the long-term reliability of the model.

[0023] Existing intelligent diagnosis systems generally lack structure-oriented reliability indicators, and usually can only indirectly estimate the reliability of the model through multiple rounds of precision testing, which is time-consuming and lacks a unified standard. The present application proposes a long-term reliability evaluation method based on Markov steady-state distribution, which calculates the probability of the model being in the "correct classification state" in the long-term running by solving the steady-state distribution of the state transition matrix, and defines a reliability index based on it. The index has a clear probability meaning and can directly quantify the reliability of data-driven intelligent diagnosis models, guiding the design, optimization, selection of diagnosis models and parameters in engineering applications, and can set threshold values for hierarchical management according to actual diagnosis task requirements.

[0024] 3) It has real-time monitoring and dynamic updating capability, and is suitable for actual industrial deployment requirements.

[0025] Compared with the data statistical method which is only applicable to offline testing, the present application supports real-time statistical state transition information during model running, and efficiently solves the reliability index based on the power iteration method, which can quickly respond to model performance changes and is suitable for various deployment environments such as edge computing, embedded systems and online prediction. The method of the present application does not depend on the specific structure of the model and can adapt to various classification models such as support vector machine, random forest, convolutional neural network, and Transformer. At the same time, it does not depend on the physical source of the data and is suitable for various fault diagnosis scenarios such as machinery, electrical, thermal energy, and aviation, with high universality and engineering promotion potential.

[0026] In summary, the reliability modeling and evaluation method for intelligent diagnostic models based on Markov chains transforms the model output into long-term state stability in a probabilistic sense, thus completing the shift from "outcome-oriented" to "process modeling." It constructs a complete link that runs through "classification output—state evolution—probabilistic modeling—performance evaluation," providing a systematic theoretical framework and an operable technical path for the reliability evaluation of intelligent diagnostic models. Attached Figure Description

[0027] Figure 1 System overall block diagram.

[0028] Figure 2 Workflow for reliability modeling and evaluation of intelligent diagnostic models based on Markov chains.

[0029] Figure 3 : Schematic diagram of state transition process and probability matrix.

[0030] Figure 4 : Reliability comparison of three models under the same diagnostic accuracy. Detailed Implementation

[0031] The implementation steps of the present invention will now be described in detail with reference to the accompanying drawings.

[0032] As attached Figure 1 The system's overall block diagram illustrates how the intelligent diagnostic model constructs a corresponding discrete state sequence by acquiring continuous output results from actual bearing fault diagnosis tasks. A correct bearing fault type identification represents a correctly classified state, while an incorrect identification represents a misclassified state. The number of state transitions for each type in the state sequence is statistically analyzed to identify the state transition probability matrix. Subsequently, the steady-state distribution of the Markov chain is solved based on this transition matrix to obtain the stable probability of the intelligent bearing fault diagnosis model in each state during long-term operation. The steady-state probability of the "correctly classified state" is defined as the reliability index of the intelligent diagnostic model, used to quantify its long-term classification stability. Finally, the model's reliability level is evaluated based on this index, providing a reference for the performance analysis and dynamic maintenance of the intelligent bearing fault diagnosis model.

[0033] The specific technical implementation of this invention is as follows: 1) Markov State Definition Based on Diagnostic Tasks In the intelligent diagnosis task of bearing fault, the intelligent diagnosis model classifies and judges the bearing operation data to output the corresponding health state or fault type. The traditional method usually uses these output results to calculate the precision indicators such as accuracy, recall rate and the like, but these static indicators cannot describe the dynamic behavior change of the intelligent diagnosis model in the long-term running process. Therefore, the present application proposes a state modeling method based on Markov chain, which abstracts the result sequence generated by the intelligent diagnosis model in the continuous classification task of bearing fault into a Markov state sequence composed of discrete states, thereby providing a mathematical basis for subsequent reliability modeling.

[0034] The Markov chain is a random process for describing the change of state over time, and its core feature is "memorylessness", that is, the next state of the system is only related to the current state and has nothing to do with the past state. This property makes it suitable for modeling the state transition behavior of the diagnosis model in the running process. In the intelligent diagnosis system, the classification results of the model on the input data have obvious time sequence and randomness, which can be abstracted as a state sequence composed of "correct" and "incorrect" classification results. In order to describe this dynamic process from a structured perspective, the present application proposes a Markov state modeling method based on the diagnosis task, which maps the output results of the intelligent diagnosis model to the finite state space of the Markov chain, thereby providing a unified time sequence expression framework for reliability modeling.

[0035] Considering a typical bearing fault classification diagnosis task, the intelligent diagnosis model outputs a series of bearing fault prediction labels for the input bearing operation data sequence , and the corresponding bearing fault true labels are . According to the correctness of each prediction result, it is converted into a state sequence , wherein the state is defined as: Therefore, the entire bearing fault diagnosis process can be converted into a random process evolving on the finite state set . This process satisfies the Markov property, that is, the transition of the current state only depends on the state at the last time, and has nothing to do with the earlier historical state: This "memorylessness" enables the model to describe the dynamic classification behavior of the diagnosis model in the long-term running process through the state transition probability matrix. From the state space point of view, the following four possible state transitions are defined: : indicates that the model correctly classifies twice in succession; : indicates that the model changes from correct classification to incorrect classification; : indicates that the model recovers from misclassification to correct; : indicates that the model misclassifies twice in a row.

[0036] This structured two-state Markov model is not only simple and clear, but also facilitates subsequent modeling and analysis with probability tools, and is suitable for tracking classification performance fluctuations.

[0037] Since the basic assumption of Markov chain modeling is a finite state space and no memory, the state sequence defined in the present application naturally meets the modeling requirements of Markov chain. It is worth noting that although the intelligent diagnosis model can output multiple categories (such as multiple bearing fault types), the categories themselves are not distinguished in the present modeling framework, but only the correctness of the output is concerned in order to uniformly measure the overall reliability performance of the diagnosis model.

[0038] Through the above state definition method, the classification behavior which is originally difficult to measure is converted into a mathematical modelable and quantifiable random process, so that the performance evaluation of the intelligent diagnosis model is expanded from static indicators to dynamic evolution characteristics, and long-term reliability modeling can be further realized through probability tools.

[0039] 2) State transition probability matrix identification based on maximum likelihood estimation After defining the Markov state for the intelligent diagnosis task, the next step is to establish the state transition law to capture the transition characteristics between the "correct classification state" and the "misclassification state". The state transition law is the core content of Markov chain modeling, which determines the probability distribution of the system in a certain state at future time. In order to accurately depict this dynamic process, the present application adopts the method of maximum likelihood estimation (Maximum Likelihood Estimation, MLE) to construct the state transition probability matrix, and makes the matrix truly reflect the statistical characteristics of the state change of the diagnosis model in actual operation.

[0040] The state space used in the present application includes two types of states: correct classification state and misclassification state . Therefore, the state transition probability matrix is a real matrix of , which represents the transition probability between any states of the model. The matrix is defined as follows: wherein, represents the probability that the current state is a misclassification state and the next time is still a misclassification state; represents the probability that the current state is a misclassification state and the next time is a correct classification state; denotes the probability that the current state is correct classification state, and the next state is error classification state; denotes the probability that the current state is correct classification state, and the next state is still correct classification state.

[0041] To estimate the transition probability described above, the state sequence generated based on the actual operation result of the intelligent diagnosis model , the occurrence times of the following four types of state transitions are counted: : the number of times of transition from error state to error state; : the number of times of transition from error state to correct state; : the number of times of transition from correct state to error state; : the number of times of transition from correct state to correct state.

[0042] Maximum likelihood estimation makes the probability of observed data given the transition path maximum. In this problem, the optimal solution given by maximum likelihood estimation is that the state transition probability is equal to the frequency of this type of transition in the state: The accurate construction of the state transition probability matrix is the core intermediate link of the entire reliability evaluation system, which provides a direct basis for the solution of the steady-state distribution and the definition of the reliability index. Compared with the method of static evaluation based only on the accuracy of the test set, the matrix not only contains the "accuracy" information, but also captures the dynamics and change trend of the model behavior, providing a solid foundation for subsequent long-term evaluation.

[0043] 3) Intelligent diagnosis model reliability evaluation based on steady-state distribution After the state space and the state transition probability matrix are constructed, the present application further proposes an intelligent diagnosis model reliability evaluation method based on the steady-state distribution of Markov chain, to quantify the probability that the model is in the "correct classification state" during long-term operation, thereby providing a structured, interpretable, traceable long-term reliability index.

[0044] The intelligent diagnosis model for bearing fault often runs in the form of continuous reasoning in actual application, and its classification state will continue to fluctuate with the change of data input. By constructing a two-state Markov model and obtaining the state transition probability matrix , the long-term stability of Markov chain is used to solve the steady-state distribution of the model , so as to evaluate the stable classification performance of the model when time tends to infinity.

[0045] The steady-state distribution is defined as the probability vector that satisfies the following relationship: wherein , represents the stable probability of the intelligent diagnosis model being in the "correct classification state" during the long-term operation. After the system is operated for a long time, the probability of entering a certain state no longer changes. To further enhance the understandability and operability of the index, the present application defines the model reliability index (Reliability Index, RI) as: The index value ranges from , the closer to 1 indicates that the long-term classification performance of the model is more stable, and the closer to 0 indicates that the model has a continuous error risk. In practical applications, a reasonable threshold can be set according to the task requirements, for example: , the model is considered to have high reliability; , the model performance is considered to be medium, which needs to be adjusted as appropriate; when , the model reliability is insufficient, and retraining or structure optimization should be considered.

[0046] To improve the calculation efficiency and realize online monitoring, the power iteration method is used to calculate the steady-state distribution, that is, an arbitrary initial distribution vector is selected, and iteration is performed: until the convergence condition is met: wherein, is the number of iterations, is the set error tolerance.

[0047] The specific implementation steps of the present application are shown in Figure 2 , mainly including 9 steps: Step 1: Collecting the output of the intelligent diagnosis model Obtain the continuous classification results of the intelligent diagnosis model for bearing fault in the actual bearing fault diagnosis application task. Usually, the bearing fault intelligent diagnosis model is based on the bearing operation data collected by the sensor to identify the running state of the bearing and output the corresponding bearing fault prediction label . This step needs to record the bearing fault prediction label sequence output by the intelligent diagnosis model synchronously with the actual label of the bearing fault , forming a one-to-one corresponding bearing fault type classification judgment pair. This data can come from the offline inference results in the model test phase, or from the online running log after the model is deployed. This step provides a data basis for the subsequent state mapping, and requires continuous data and accurate labeling to ensure the integrity and reliability of the state evolution process.

[0048] Step 2: Mapping Markov states After obtaining the prediction results of the intelligent diagnosis model and the true labels, each bearing fault type classification judgment result is mapped to a discrete state in the Markov chain. According to formula (1), when the model prediction label is consistent with the true label, the model state at this time is marked as correct classification state ; when the prediction is inconsistent with the actual, it is marked as error classification state . This state mapping process simplifies the originally complex multi-classification diagnosis problem into a two-state discrimination task, facilitating modeling through the Markov model.

[0049] Step 3: Constructing discrete state sequence The mapped discrete classification states are arranged in order to form a state chain reflecting the dynamic operation of the model. This sequence represents the classification behavior pattern of the model at discrete time points, retaining the basic information of the model output while having time evolution characteristics, and can be used to depict the performance fluctuation trend of the model. This step ensures that the state sequence is continuous and uninterrupted, facilitating complete statistical analysis and reliable modeling.

[0050] Step 4: Counting state transition times From the constructed state sequence, adjacent state pairs in the state sequence are traversed, and the transition times between adjacent states are counted to record the conversion of the model between different states. As shown in Figure 3 , it specifically includes four possible state transitions: , , , , respectively denoted as , , , . These counts reflect the classification change frequency of the model within a period of time, and also reveal the behavior characteristics of the model such as error correction ability and correct maintenance ability.

[0051] Step 5: Calculating state transition probability matrix After obtaining the four state transition times in step 4, the maximum likelihood estimation method is used to calculate the model state transition probability matrix based on formulas (3)-(4). As shown in Figure 3 , each element in the matrix represents the conditional probability of transitioning from the current state to another state. By normalizing the counts of each transition type, four probability values are obtained to form a complete second-order state transition matrix, which systematically describes the transition trend of the model's classification behavior during operation.

[0052] Step 6: Solving steady-state distribution Based on the transition probability matrix constructed in step 5 , the steady-state distribution of the Markov chain is solved , i.e., the stable probability of each state in the long-term operation of the model. According to formulas (7)-(8), the steady-state distribution is obtained by numerical calculation using the power iteration method . The power iteration method has the advantages of simple calculation and stable iteration, and is suitable for industrial scenarios with high real-time requirements for bearing fault diagnosis. The steady-state distribution results will be used to extract the long-term performance characteristics of the intelligent diagnosis model, especially its ability to remain in the "correct classification state", reflecting the stability of the intelligent diagnosis model under complex working conditions.

[0053] Step 7: Determine whether the error condition is met To ensure the stability and accuracy of the steady-state distribution calculation results, set an error threshold to determine whether the power iteration process converges. After each iteration, the difference between the current state distribution vector and the previous round is calculated according to formula (8). If the difference is below the threshold, it is considered to have converged, and the final steady-state distribution is output; if the condition is not met, continue iteration. This process avoids the accumulation of distribution errors due to insufficient iterations, ensuring the scientificity of the final reliability index. The error threshold can be set according to engineering requirements, taking into account the calculation overhead and evaluation accuracy.

[0054] Step 8: Extract the reliability index After the steady-state distribution is solved , the steady-state probability of the "correct classification state" is extracted , defined as the reliability index of the intelligent diagnosis model . It is between 0 and 1, and the larger the value, the higher the probability that the intelligent diagnosis model will maintain correct classification in the long-term bearing fault diagnosis task. This index not only has a clear probability meaning, but also can be used for horizontal comparison of the performance of different models or different versions of the same model.

[0055] Step 9: Model reliability evaluation After obtaining the reliability index, this step analyzes and interprets the model reliability level in combination with the actual diagnosis task requirements. Users can divide the bearing fault intelligent diagnosis model into high reliability, medium reliability, or low reliability levels according to the threshold setting, and trigger model retraining, alarm prompts, or operation optimization measures accordingly. In addition, the continuous recording of the reliability index can also form a model behavior log, providing a basis for abnormal detection and trend analysis of bearing fault diagnosis. This step realizes the closed-loop connection from quantitative calculation to diagnosis decision, enhancing the autonomy and safety of the intelligent diagnosis system.

[0056] The above merely describes preferred embodiments of the present application and does not constitute a limitation on the protection scope of the present application. Any modification, equivalent replacement and scheme improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application. The power iteration method in the invention is merely one possible implementation scheme and can be replaced by other schemes such as an analytical method.

[0057] Figure 3 The structure diagram of the Markov state transition process and the state transition probability matrix proposed in the present application. The classification result of the intelligent diagnosis model is defined as two basic states in the Markov chain, i.e., an "error classification state" ( ) and a "correct classification state" ( ). The initial operation of the system starts from any state, and continuously transitions in the state space with a certain probability as time progresses. The diagram intuitively represents all possible state transition paths, including four state transition relationships. A ring structure is used to represent the state itself maintenance behavior (e.g. ), and the transition direction and probability symbol are marked on the directed edge to clearly show the relationship between the states. The start and end of state transition are indicated by "Start" and "End" icons, which facilitate understanding of the process of state evolution.

[0058] Figure 4 Table 1 shows the implementation effect of the present application on three intelligent diagnosis models with the same diagnostic accuracy, demonstrating the effectiveness of the present application in quantifying model reliability. As shown in Figure 4 , the reliability indices of the three models (CNN-1, CNN-2 and CNN-3) differ significantly, although their classification accuracies are all 98%. In the figure, the model accuracy is represented by a gray column, and the corresponding reliability index is represented by a black line and a triangular marker. It can be observed that although the three models perform consistently in terms of static test accuracy, their reliability differs significantly, with the of CNN-1 being the lowest and the of CNN-3 being the highest. This phenomenon indicates that it is difficult to reveal the stability of the model in long-term operation based on static accuracy alone, verifying the effectiveness of the present application in reliability identification. Table 1 further shows the state transition matrix, steady-state distribution and value calculated finally for the three models.

[0059] Table 1: Reliability index calculation of three models with the same diagnostic accuracy As can be seen from Table 1, although the three types of models have consistent accuracy, the transition matrices representing the state changes in actual operation of the diagnostic models are different, resulting in different steady-state distribution results finally calculated. In combination with the above results, under the premise of the same diagnostic accuracy, the Markov state modeling and steady-state distribution analysis can quantify the stability of the model classification behavior from the dynamic evolution perspective, reveal the reliability difference behind the model accuracy, provide an evaluation index of the model performance from another perspective, provide a reliability index with better depth and explanatory power for model deployment, safety decision and dynamic maintenance, and have good generalizability and engineering application value.

[0060] The above examples are used to explain and illustrate the present application, but not to limit the present application, and any modification and change made to the present application within the spirit and protection scope of the claims of the present application all fall into the protection scope of the present application.

Claims

1. A method for reliability modeling and evaluation of intelligent diagnostic models based on Markov chains, characterized in that, The method includes the following steps: (1) Based on the actual application task, the classification judgment result of whether the bearing is faulty is obtained through the intelligent diagnostic model, and recorded as the correct classification state and the incorrect classification state; (2) Arrange the classification states of whether the bearing is faulty in order to construct a state chain that reflects the dynamic operation of the Markov model; (3) Count the number of transitions between adjacent classification states, record the transitions of the Markov model between different states, and use maximum likelihood estimation to calculate the model state transition probability matrix; (4) Solve the stable probability of the Markov model in each state during operation based on the model state transition probability matrix, that is, the steady-state distribution of the Markov chain; (5) The steady-state probability of the correct classification state extracted based on the steady-state distribution is defined as the reliability index of the intelligent diagnostic model to judge the model performance.

2. The reliability modeling and evaluation method according to claim 1, characterized in that: The intelligent diagnostic model outputs a series of bearing fault prediction labels based on the input bearing operation data. These labels are compared with the actual labels; if they match, the bearing is correctly classified; otherwise, it is incorrectly classified.

3. The reliability modeling and evaluation method according to claim 1, characterized in that: In step (2), the output sequence of the intelligent diagnostic model is regarded as a Markov chain. For a specific fault diagnosis task, the output result of the diagnostic model is defined as a number of finite Markov states. The transition between states is only related to the current classification output, forming a definite state transition space.

4. The reliability modeling and evaluation method according to claim 1, characterized in that: Specifically, it includes four possible state transitions: 1) The current state is misclassified, and the next moment will also be a misclassified state; 2) The current state is misclassified; the next moment, it will be correctly classified. 3) The current state is correctly classified, but the next moment it will become incorrectly classified; 4) The current state is the correct classification state, and the next moment will also be the correct classification state.

5. The reliability modeling and evaluation method according to claim 4, characterized in that: Based on the state sequence generated from the actual operation results of the intelligent diagnostic model, the occurrence frequency of four types of state transitions is counted. Based on the optimal solution given by maximum likelihood estimation, the probability of each state transition is equal to the frequency of that type of transition in the state, and then a state transition probability matrix is ​​constructed.

6. The reliability modeling and evaluation method according to claim 1, characterized in that: In step (4), the steady-state distribution corresponding to the state transition probability matrix is ​​solved to determine the probability distribution law of each state under long-term operation, thereby evaluating the stable classification performance of the model when time tends to infinity.

7. The reliability modeling and evaluation method according to claim 6, characterized in that: The steady-state distribution is calculated using the power iteration method, which involves selecting any initial distribution vector and iteratively calculating until the convergence condition is met.

8. The reliability modeling and evaluation method according to claim 1, characterized in that: In step (5), based on the reliability index evaluation results, quantitative decision-making basis is provided for the selection, optimization and deployment of intelligent diagnostic models.

Images