Integrated markov decision process framework for asset-management decision-making

Abstract

Described herein are systems and methods for asset-management decision-making for large industrial facilities. The disclosed systems and methods can be used to determine the current state of components and systems at a facility, and the likelihood of their degradation or failure in the future, and ultimately determine an asset management strategy that optimizes value returns and / or safety constraints.

Description

Atty. Dkt. No.103361-637WO1 INTEGRATED MARKOV DECISION PROCESS FRAMEWORK FOR ASSET-MANAGEMENT DECISION-MAKING CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to, and the benefit of, U.S. Provisional Application Serial No.63 / 621,384, filed January 16, 2024, the disclosure of which is expressly incorporated herein by reference in its entirety. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] This invention was made with government support under grant / contract number DE-AC02-06CH11357 awarded by the Department of Energy. The government has certain rights in the invention.  BACKGROUND

[0003] Economic competitiveness is the greatest barrier facing the U.S. nuclear power sector. The U.S. operating light water reactor (LWR) fleet is struggling to remain profitable in a challenging market environment, leading the federal and state governments to provide incentives to ensure that the U.S. retains its largest source of carbon-free electricity. While these efforts aid in addressing the issue in the short-term, sustainable changes are needed to improve the long-term economic outlook.

[0004] Nuclear power plant operating costs are constrained by many factors, such as security requirements or fuel costs. However, improvements in asset-management and operational strategies offer one avenue for cost reduction that is within the control of utilities. For advanced reactor designs, this includes maintenance activities associated with systems and components, along with potential changes in operational modes, such as power reductions. The difficulty is optimizing asset-management and operational decision-making in a complex and interconnected environment. There are numerous factors that can influence the course of action, including the current status of plant components, projected revenue, regulatory compliance, etc. A comprehensive assessment of these factors must be conducted for a truly optimized solution to be found.

[0005] As advanced reactors are early in the development life-cycle, online monitoring systems and associated sensor networks can be incorporated directly into the design withoutAtty. Dkt. No.103361-637WO1 constraints related to retrofitting and system upgrades. However, due to their innovative designs and lack of operating experience, advanced reactors have large uncertainties regarding component reliability, potential failure modes, and long-term maintenance needs. Therefore, it is necessary to develop an online monitoring system that is capable of multifaceted plant performance cost-benefit analyses but which is both flexible and robust to tolerate operational uncertainties. SUMMARY

[0006] Described herein are systems and methods for asset-management decision-making (or planning) for large industrial facilities. The disclosed systems and methods can be used to determine the current state of components and systems at a facility, and the likelihood of their degradation or failure in the future, and ultimately determine an asset management strategy that optimizes value returns and / or safety constraints. The following factors are incorporated into the disclosed system and method for decision-making: the estimated current status of all components / systems; the probability of components / systems failing or becoming degraded in the future; the cost of components / systems and all associated repair / replacement activities; the planned maintenance / repair activities; the value associated with different levels of plant operation; and the regulatory acceptability of the plant for different operational states.

[0007] In some aspects, described herein is a method including: determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component, wherein the sensor data is obtained from a plurality of sensors in the facility; generating a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior models for each of the plurality of physical components; and determining, using a Markov decision process, an optimal asset- management strategy based on the real-time risk profile and the statuses of the plurality of physical component.

[0008] In some aspects, the status includes a real-time probability of a state of the physical component.

[0009] In some aspects, the real-time risk profile includes a plurality of risk models related to one or more probable future states.

[0010] In some aspects, the one or more probable future states are one of healthy, derate, or failure.

[0011] In some aspects, the future state is determined by a user (e.g. planned maintenance).Atty. Dkt. No.103361-637WO1

[0012] In some aspects, each of the plurality of risk models is associated with one or more of the plurality of physical components (i.e. subsystems).

[0013] In some aspects, at least one of the plurality of risk models is based on a probabilistic risk assessment (PRA) indicative of safety risk.

[0014] In some aspects, at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk.

[0015] In some aspects, at least two of the plurality of risk models are based on a probabilistic risk assessment (PRA) and on a generation risk assessment (GRA).

[0016] In some aspects, a fault tree method is used to configure each of the plurality of risk models.

[0017] In some aspects, determining an optimal asset-management strategy includes evaluating a plurality of asset-management strategies for the facility based on the real-time risk profile and the statuses of the plurality of physical components.

[0018] In some aspects, the optimal asset-management strategy includes an optimal subset of a plurality of asset management strategies.

[0019] In some aspects, the method further includes evaluating a probabilistic risk assessment (PRA) for each of the optimal subset of a plurality of asset management strategies, wherein the PRA is associated with the assessment of safety parameters.

[0020] In some aspects, the method further includes operating the plurality of physical components in the facility based on the optimal asset-management strategy.

[0021] In some aspects, described herein is a method including: determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component, wherein the sensor data is obtained from a plurality of sensors in the facility; generate a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior model for each of the plurality of physical component; evaluating a plurality of asset-management strategies for the plurality of physical components of the facility based on the real-time risk profile and the statuses of the plurality of physical components; and determining, using a Markov decision process, a set of value-optimized asset-management strategies; and determining a subset of value-optimized asset-management strategies that satisfy a set of safety requirements.

[0022] In some aspects, the status includes a real-time probability of a state of the physical component.

[0023] In some aspects, the real-time risk profile includes a plurality of risk models related to one or more probable future states.Atty. Dkt. No.103361-637WO1

[0024] In some aspects, a future state is one of healthy, derate, or failure.

[0025] In some aspects, the future state is determined by a user (e.g. planned maintenance).

[0026] In some aspects, each of the plurality of risk models is associated with one or more of the plurality of physical components (i.e. subsystems).

[0027] In some aspects, at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk.

[0028] In some aspects, a fault tree method is used to configure each of the plurality of risk models.

[0029] In some aspects, determining the set of value-optimized asset-management strategies includes evaluating a plurality of asset-management strategies for the facility based on the real-time risk profile and the statuses of the plurality of physical components.

[0030] In some aspects, a probabilistic risk assessment (PRA) is used to determine the subset of value-optimized asset-management strategies.

[0031] In some aspects, the method further includes operating the plurality of physical components in the facility based on one of the subset of value-optimized asset-management strategies that satisfy the set of safety requirements. BREIF DESCRIPTION OF DRAWINGS

[0032] Fig.1 shows an exemplary method of the disclosure.

[0033] Fig.2 shows an exemplary method of the disclosure.

[0034] Fig.3A and 3B show schematics of exemplary methods.

[0035] Fig.4 shows a state transition process.

[0036] Figs.5A-5B shows a schematic of an exemplary three state Markov model.

[0037] Figs.6A-6C show GRA examples of fault tree model of plant derated states (Trip model, Fig.6A), fault tree model of FW subsystem degraded states (Fig.6B), and fault tree model of transformer subsystem degraded states (Fig.6C).

[0038] Fig.7 shows an example of component Markov model with POMDP optimal policy.

[0039] Fig.8 shows an example PRA Theory diagram.

[0040] Fig.9 shows LMP frequency versus consequence curve.

[0041] Fig.10 shows LMP frequency versus consequence curve with risk significant region.Atty. Dkt. No.103361-637WO1

[0042] Figs.11A-11C show example plant F-C curve results for baseline (Fig.11A), option A (Fig.11B), and option B (Fig.11C). DETAILED DESCRIPTION

[0043] Described herein are systems and methods for asset-management decision-making (or planning) for large industrial facilities. The disclosed systems and methods can be used to determine the current state of components and systems at a facility, and the likelihood of their degradation or failure in the future, and ultimately determine an asset management strategy that optimizes value returns and / or safety constraints. The following factors are incorporated into the disclosed system and method for decision-making: the estimated current status of all components / systems; the probability of components / systems failing or becoming degraded in the future; the cost of components / systems and all associated repair / replacement activities; the planned maintenance / repair activities; the value associated with different levels of plant operation; and the regulatory acceptability of the plant for different operational states.

[0044] wide variety of techniques have been proposed to address the problem stated above; however, they generally only incorporate a subset of the above factors. To develop an optimized asset-management strategy, all factors must be incorporated together into the decision-making framework. This disclosure contemplates a method that utilizes Markov Decision Processes (MDPs) to create an integrated decision-making framework. While MDPs are a common and popular choice for addressing such problems, the developed approach directly integrates the following factors into the MDP solution framework: real- time monitoring of component and system status; the likelihood of future component / system failure or degradation based on Markov component models; the potential costs or revenue associated with each possible future plant state and their likelihood of occurrence; currently planned maintenance and repair activities. The direct inclusion of these factors into the MDP framework is a novel approach that was developed as part of the current project.

[0045] A central aspect of the system and methods is the utilization of an intelligent decision-making approach to optimize asset-management strategies. A system of interest (e.g. a nuclear power plant) can be in any of a finite number of states, and the transition between system states follows a Markov process. At discrete time steps, the decision-maker can take actions to influence system state transition. So, the transitions between system states depend not only on “nature,” i.e., the inherent randomness in system state transition, but also on decision-maker actions. At each time step, different decision-maker actions and different system state transitions lead to varying rewards for the decision-maker. The decision-maker’sAtty. Dkt. No.103361-637WO1 objective is to maximize the sum of the rewards that will be received from the current time step into the future.

[0046] Referring now to Fig.1, an exemplary method is shown. The method includes determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component 110, wherein the sensor data is obtained from a plurality of sensors in the facility; generating a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior models for each of the plurality of physical components 120; and determining, using a Markov decision process, an optimal asset-management strategy based on the real-time risk profile and the statuses of the plurality of physical component 130.

[0047] In some aspects, the status includes a real-time probability of a state of the physical component.

[0048] In some aspects, the real-time risk profile includes a plurality of risk models related to one or more probable future states. In some aspects, the one or more probable future states are one of healthy, derate, or failure. In some aspects, the future state is determined by a user (e.g. planned maintenance). In some aspects, each of the plurality of risk models is associated with one or more of the plurality of physical components (i.e. subsystems).

[0049] In some aspects, at least one of the plurality of risk models is based on a probabilistic risk assessment (PRA) indicative of safety risk, and at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk. In another aspect, at least two of the plurality of risk models is based on a probabilistic risk assessment (PRA) and on a generation risk assessment (GRA).

[0050] In some aspects, a fault tree method is used to configure each of the plurality of risk models.

[0051] In some aspects, determining an optimal asset-management strategy 110 includes evaluating a plurality of asset-management strategies for the facility based on the real-time risk profile and the statuses of the plurality of physical components.

[0052] In some aspects, the optimal asset-management strategy includes an optimal subset of a plurality of asset management strategies.

[0053] In some aspects, the method further includes evaluating a probabilistic risk assessment (PRA) for each of the optimal subset of a plurality of asset management strategies, wherein the PRA is associated with the assessment of safety parameters.

[0054] In some aspects, the method further includes operating the plurality of physical components in the facility based on the optimal asset-management strategy.Atty. Dkt. No.103361-637WO1

[0055] Referring now to Fig.2, an exemplary method is disclosed. The method includes determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component 210, wherein the sensor data is obtained from a plurality of sensors in the facility; generate a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior model for each of the plurality of physical component 220; evaluating a plurality of asset-management strategies for the plurality of physical components of the facility based on the real-time risk profile and the statuses of the plurality of physical components 230; determining, using a Markov decision process, a set of value-optimized asset-management strategies 240; and determining a subset of value-optimized asset-management strategies that satisfy a set of safety requirements 250.

[0056] In some aspects, at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk.

[0057] In some aspects, determining the set of value-optimized asset-management strategies 210 includes evaluating a plurality of asset-management strategies for the facility based on the real-time risk profile and the statuses of the plurality of physical components.

[0058] In some aspects, a probabilistic risk assessment (PRA) is used to determine the subset of value-optimized asset-management strategies. MARKOV DECISION PROCESSES

[0059] The Formulation of a Markov Decision Process. A Markov decision process can be formally defined by the following five elements, i.e., ^^, ^^, T, R, γ.

[0060] The element, ^^ is a discrete and finite space for the states of a system under study; A is specific state in S at time step t is denoted by st.

[0061] The element, ^^ is a discrete and finite space for decision-maker actions. A specific action in A at time step t is denoted by at.

[0062] The element, T represents ^^ × ^^ × ^^ →[0,1], the system state transition probability function, where [0,1] is the interval between 0 and 1. For example, T(s(t+1)|st, at) denotes the probability of system state s(t+1) ∈ ^^ at time step t+1 given the system is in state st ∈ ^^ at time step t and the decision-maker takes action at ∈ A at time step t.

[0063] The element, R represents ^^ × ^^ →(-∞,+∞), the reward function. For example, R(st, at) denotes the reward that the decision-maker receives at time step t if the system is in state st∈ ^^ and the decision-maker takes action at ∈ ^^.Atty. Dkt. No.103361-637WO1

[0064] The element γ ∈ (0,1) is the discount factor used in the calculation of the cumulative rewards.

[0065] To illustrate the above notations, take the maintenance of a valve used in nuclear power plants as an example. Decisions on the maintenance may be made at discrete time steps, with an interval of one month. The state space for such a valve may include perfect, degraded, and failed. The states also describe the levels of degradation of the valve. At each time step, the possible actions (i.e., the action space) that maintenance staff can take may include do nothing and repair. The transition between system states depends on both the inherent randomness of the degradation of the valve, and the action taken by the maintenance staff. For example, if at time step t the valve is in the degraded state and the maintenance staff decides to do nothing, then the valve will be in the failed state at time step t+1 with probability 1×10-2. However, for this same situation, if the maintenance staff takes action repair, the transition probability may be reduced to 1×10-3. Depending on the system state and the decision-maker action at time step t, the decision-maker will receive a certain reward. For example, for the same decision-maker action, the reward in the case of a perfect state will typically be higher than the one in the case of a failed state. The discount factor γ may be determined as the real number (smaller than 1.0) that discounts future rewards back to the present value by referring to financial models used in maintenance management.

[0066] Objective Function. Since decision-maker actions will not only influence the reward for the current time step, but also influence the system state in the next time step, hence the reward in the next time step, in making decisions at each time step, the decision-maker should not only consider the immediate effect of the action, but also consider the long-term effect of the action. In an MDP, given the system state s0 at time step t=0, the decision-maker aims to develop a policy σ that maximizes the following expected discounted cumulative reward, ^ ^ ^^^^^ ^ ^^ ௧ ௧^ ^^^^ ^^ ^^^^

[0067] Thetransitions between system states. At each time step t, action atis taken according to the policy σ. The policy σ is defined as a mapping from the state space to the action space, i.e., σ: ^^ → ^^. The above objective function takes into account both the reward to be gained for the current time step t=0, but also the rewards to be received in all future time steps. The discount factor γ is used to ensure that the sum of the rewards is finite. The policy that maximizes the objectiveAtty. Dkt. No.103361-637WO1 function in the above equation is denoted by σ*. The maximum expected discounted cumulative reward following the optimal policy σ* is denoted by J*(s0) for the starting state s0. This maximum number is also called the value for state s0, and this notation will be used in the next section when introducing the value iteration method for obtaining σ*.

[0068] Solution Methods. Dynamic programming serves as one of the most promising methods for solving an MDP problem, i.e., to obtain the optimal policy σ*. Dynamic programming methods can typically be classified into two categories, value iteration based methods and policy iteration based methods. In this section, a basic value iteration method is briefly introduced. This method relies on the following Bellman equation, ^ ^^∗^^^ ௧^^ ൌ m^బa∈^x^ ^^^^^^^,^^^^ ^ ^^^^^^^ ^^^^^௧ ,^^௧^൩ ൌ m^ a∈^x^ ^^^^^^^,^^^^ ^ ^^^^^^∗^^^^^^௧ୀ^బ

[0069] A more straightforward view of the above Bellman equation can be obtained by replacing s0and s1in the above equation with s denoting the current system state and s' denoting the next system state. The Bellman equation is shown below, ^^∗^^^^^ ൌ m^∈a^x^ ^^^^^^,^^ ^ ^ ^^^^^ᇲ^^∗^^^ᇱ^^

[0070] To use the valuevalues are assigned for all system states s∈ ^^. Then, the value for each state can be updated following the above Bellman equation as follows, ^^∗^^^^^ ← m^∈a^x^ ^^^^^^,^^ ^ ^ ^^^^^ᇲ^^∗^^^ᇱ^^

[0071] The abovevalue for any system state is below a predefined threshold.

[0072] Once the values for all system states are obtained, then the optimal action a*=σ*(s) under any system state s can be obtained as follows, ^^∗ ← max ^^^^^^,^^ ^ ^ ^^^^^ᇲ^^∗^^^ᇱ^^ ^∈^^

[0073] Partially Observable. As can be seen in the introduction to MDPs in Section 2.1, the system state is fully observable or known. In practical applications, the actual system state may only be partially observable, and can only be inferred through observations, such is often the case with equipment monitoring at nuclear power plants. Decision optimization under such situations can be formulated as and solved by partially observable Markov decision processes (POMDPs).Atty. Dkt. No.103361-637WO1

[0074] The Formulation of a POMDP. A POMDP can be formally defined by the following seven elements, i.e., ., ^^, ^^, ^^, T,O, R, γ.. The only differences between a basic MDP and a POMDP is the addition of ^^ and O. The other elements are defined in the similarly, and the additional elements are defined as:

[0075] The element ^^ is the observation space. The observations can be either discrete and finite, or continuous. A specific observation in ^^ at time step t is denoted by ot.

[0076] The element O is the observation probability function. It can be defined as either ^^ →[0,1] for discrete observations where [0,1] denotes the space of all possible probability mass values for discrete observations, or ^^ →[0,+∞) for continuous observations where [0,+∞] denotes the space of all possible probability density values for continuous observations.

[0077] Objective Function in a POMDP. In a POMDP, the actual system state is not directly observable. So, the objective function in a POMDP is defined based on the belief in the system states. Denote the observations collected from time step 0 to time step t by o(0:t). The belief bt (st) in system state st at time step t is defined as the posterior probability of system state stat time step t, i.e., p(st|o(0:t)).

[0078] Suppose the belief bt (st) for each system state st has been obtained at time step t, then the beliefs bt+1(st+1) for any system state st+1at time step t +1 can be obtained recursively as follows: ^^^^^ , ^^ ^ ∑^ ^^^^^௧, ^^௧ା , ^^ , ^^ ^^^ ^^^ ^ ^ | ^ ௧ା^ ^:௧ା^ ^ ^ ^:௧ ௧ା^௧ା^ ௧ା^ ൌ ^^ ^^௧ା^ ^^^:௧ା^ ൌൌ ^ .following objective function, ^ ^ ^^ ^ ^^ ^^ ^0, i.e., the probability distribution over system states at time step 0. The expectation is over the possible observations at each time step. In an MDP, a policy is defined as a mapping from the system state space to the action space, i.e., σ: ^^ → ^^. But in a POMDP, a policy is defined asAtty. Dkt. No.103361-637WO1 a mapping from the belief state space ^^ to the action space ^^, i.e., σ: ^^ → ^^. In the above objective function, σ(bt) denotes the action at provided by the policy σ for belief state bt. Similar to the notations in an MDP, the optimal policy is denoted by σ* and the maximum expected cumulative reward J(b0) following policy σ* is denoted by J*(b0) and is called the value for belief state b0.

[0081] Solution Methods in POMDP. The optimal policy in a POMDP can also be obtained using value iteration or policy iteration, similar to the methods described above for an MDP. However, the major difference between solving an MDP and solving a POMDP is that in a POMDP the policy is a function of the belief state, i.e., the posterior probability distribution over system states, instead of the actual system state. In the value iteration algorithm introduced above for an MDP, the key to obtaining the optimal policy is to obtain the values for all system states. In the case of an MDP, the system state space is discrete and finite, so the values for system states can be enumerated. However, in the case of a POMDP, even for discrete and finite system state space, the belief state space is continuous. So, it is not feasible to enumerate the values for an effectively infinite number of belief states.

[0082] A typical solution to this problem is to sample the belief state space and use these samples in the value iteration algorithm. The details of such algorithms are not provided in this report but can be found in the literature . ASSET-MANAGEMENT DECISION-MAKING APPROACH

[0083] As shown in Error! Reference source not found., the approach for asset- management decision-making during plant operation requires multiple steps and tools but fundamentally relies on an MDP / POMDP optimization assessment. The steps before the MDP are necessary to supply the MPD calculation with the information required to form a real-time assessment of plant status.

[0084] First, sensor information from a system is received and provided to an online monitoring and system diagnosis module, which assesses component status based on the sensor data and physical system models. To inform this calculation, Markov component models provide additional insights regarding component behavior (such as estimated failure rates). Both the online monitoring and system diagnosis module and the Markov component models work in tandem to assess the condition of components within the system.

[0085] The output of the online monitoring and system diagnosis module are real-time probabilities regarding component status (healthy, degraded, failed, etc.). The output from the online monitoring and system diagnosis module and the Markov component models are utilized to develop a real-time plant risk profile, which consists of a probabilistic riskAtty. Dkt. No.103361-637WO1 assessment (PRA) and a generation risk assessment (GRA). The PRA analyzes plant risk from a safety perspective, while the GRA assesses economic risk.

[0086] Lastly, the output from the online monitoring and system diagnosis module and the real-time plant risk profile are fed to the MDP analysis. The MDP analyzes different operational strategies to determine the optimal asset-management strategy to maximize revenue (see Fig.3A). The integration of these differing aspects is detailed in the following subsections.

[0087] Markov Component Models. The Markov component models are utilized to generate the likelihood (probabilities) for system components to be in specific operating / degraded / failed states at any point in time.

[0088] Theoretical Approach. In probability theory, a Markov model is a stochastic model used to model randomly changing systems. It is assumed that future states depend only on the current state, not on the events that occurred before it (that is, it assumes the Markov property). The models are named after Russian mathematician Andrei Markov in late 1800’s early 1900.

[0089] For any given component, a Markov model consists of a list of the possible states the component could be at any specific time, the transition paths between those states, the rate parameters of those transitions, and the initial conditions describing the chance of the component to be in the states at some initial time point. Consider a simple Markov model of a component with two states, Healthy and Failed. Graphical representation of this model is shown in Fig.4.

[0090] The symbol ^ here denotes failure rate which describes the transition of the component from state Healthy to state Failed. Consider that at time t = 0 the component is in the Healthy state, meaning the probability of the component being in that state at that specific time is one, PH(t=0) =1. Because this example the component can only be in one of the two states, then the probability of the component to be in state Failed at time t = 0 equals zero, PF(t=0) = 0. The probability of state Healthy decreases at the constant rate λ, which means that if the component is in state Healthy at any given time, the probability of making the transition to state Failed during the next increment of time dt is λdt. Therefore, the overall probability that the transition from state Healthy to state Failed will occur during a specific incremental interval of time dt is given by multiplying (I) the probability of being in state Healthy at the beginning of that interval, and (II) the probability of the transition during theAtty. Dkt. No.103361-637WO1 interval dt given that it was in state Healthy at the beginning of that increment. This represents the incremental change dPH(t) in probability of state Healthy at any given time: ^^^^ு^^^^ ൌ ^^ு^^^^ ∙ ^െ^^^^^^^Dividing both sides by dt, results in the simple differential equation ^^^^ு^^^^ ^^^^ൌ െ^^^^ு^^^^Similarly,^^^^ி^^^^ ^^^^ൌ ^^^^ி^^^^

[0091] The solutions of theseset of initial conditions are: ^^ ^ ^ ିఒ௧ ^ ^ ିఒ௧ு ^^ ൌ ^^ ^^^^^^ ^^ி ^^ ൌ 1 െ ^^

[0092] The Markova fully operational state (Healthy) and a set of intermediate states representing partially failed condition (Degraded), leading to the fully failed state, i.e., the state in which the component is unable to perform its design function (Failed). The model may include repair transition paths as well as failure transition paths. The state equation for each state equates the rate of change of the probability of that state (dP(t) / dt) with the “probability flow into and out of” that state. The total probability flow into a given state is the sum of all transition rates into that state, each multiplied by the probability of the state at the origin of that transition. The probability flow out of the given state is the sum of all transitions out of the state multiplied by the probability of that given state.

[0093] The ability of Markov models to estimate the future state(s) of components is an important factor in the MDP asset-management analysis, as the change in component status will impact the likelihood of being in different plant states. Direct incorporation of the Markov component models into the MDP is detailed in the following subsections.

[0094] Implementation Approach. As described above, MDPs include a set of transition probabilities (T) that depict the likelihood of the system transitioning from one state to another during the next time interval. In the developed approach, the Markov component models are utilized to develop the transition probabilities.

[0095] While the Markov component models only assess a single component in the plant, the results of the individual component models can be combined to provide a comprehensive assessment of the likelihood of the plant transitioning states. To complete this analysis, eachAtty. Dkt. No.103361-637WO1 individual Markov component model is solved for the next time interval under consideration, in a manner consistent with the assumptions of that action. For example, if the action is to repair that component, then that is reflected in the state of the Markov component model.

[0096] For a simple Markov component model with three states (healthy (H), degraded (D), and failed (F), shown in Fig.5A), can be modeled by equations (1), (2), and (3), respectively. Fig.5B shows the change in the likelihood of component condition over time, assuming it started in the healthy state. The analytical results shown in Fig.5B from the Markov component model can be combined with the results for other components in the plant to form the MDP transition probabilities (T). ௗ^ಹ^௧^ ௗ௧ൌ െ^^^ு^ ^ ^^ுி^ ∙ ^^ு^^^^, ^^ு^^^ ൌ 0^ ൌ 1 (1)(2)(3)

[0097] Theoretical Approach. A GRA is the process of predicting the risk of generation loss during future operation by estimating the probability and duration of plant trip or derate due to equipment degradation or failure. GRA is a key activity in assuring productivity and profitability as plants worldwide face increasingly competitive power markets. Nuclear power plant operators require tools to assist management in making decisions involving the operation and maintenance of equipment whose failure can cause reactor trips or down-power events. A GRA model, whether rudimentary or detailed, is an important element of nuclear asset management risk-informed tools for analyzing effects of equipment reliability and availability on plant value and resource allocation decision-making.

[0098] Central to the assessment of generation risk is the development of a trip model. A trip model is similar in function and construction to that used for PRA with the exception that the end-state of the trip model is the frequency of plant trip as opposed to the frequency of core damage or offsite dose consequence. The trip model is generally used to estimate the frequency of instantaneous trip and down-power at the plant based on actual plant configuration and condition.

[0099] Another model important for a GRA analysis is a derate power model, a model where the end-state is the frequency of a facility to operate at decreased (derated) power level. The two models, trip and derate power, when built, help to identify different facilityAtty. Dkt. No.103361-637WO1 states and the awards (generation) associated with them which are key input parameters for any asset management decision at the facility level.

[0100] For the MDP asset-management decision-making framework, the integration of the GRA provides insight into the potential reward associated with different facility states, in terms of generation and revenue. The likelihood of a reward associated with a facility state can be derived from the GRA analysis.

[0101] Implementation Approach. The information from the GRA is important to the MDP analysis as it provides a key information regarding the potential reward, R. Consider a K-state facility that consists of N subsystems. At any particular time t the facility can be in any one of K facility states with corresponding probability Pk(t) and the reward associated with kthstate rk. The overall plant reward at time t can in general be estimated as ^ ^^^^^^ ൌ ^ ^^^ ∙ ^^^^^^^

[0102] where, rkis the facilityk, k = 1, 2, …, K; and Pk(t) is the probability for the facility to be in state k.

[0103] As will be shown, Pk is a function of not only time t, but also of pi(t), the probability for subsystem i (i = 1, 2, .., N) to be in a state that would allow the facility to be in state k. The exact formula to calculate Pk(t) is strongly dependent on configuration and arrangement of the facility subsystems and should be derived from that knowledge for each particular case. The implementation is shown in the forgoing examples.

[0104] The aforementioned trip and derate models can help with estimating facility state probabilities. In the proposed approach, the fault tree method to build the derate / trip model is applied (i.e., one fault tree for each plant state so, that probability of the plant to be in a particular derate / trip state would be the top event probability of fault tree corresponding to that state). One of the benefits of such approach is that such models can be constructed rather easily, and once built, are straightforward to use. REAL-TIME DIAGNOSTIC INFORMATION

[0105] Theoretical Approach. Optimized asset-management decision-making relies on an accurate, real-time picture of facility conditions. In the developed methodology, this process begins with the sensor network within the operating facility. The sensor network developed through utilizing ISFA in conjunction with the selected optimization criteria (cost, system penetrations, etc.), provides information to online monitoring and system diagnosis moduleAtty. Dkt. No.103361-637WO1 (e.g. an online monitoring tool capable of detecting and discriminating faults through a combination of physics-based models and reference-condition measurements), which assesses the data and diagnoses component operating states.

[0106] The online monitoring and system diagnosis framework consists of quantitative model-based diagnosis, statistical change detection and probabilistic reasoning that allows detecting both component faults and sensor faults. The use of physics-based diagnostic models provides high detection sensitivity and allows noise and measurement uncertainty to be incorporated robustly. For each component model, there are two major sources of uncertainty that need to be taken into account: (1) the measurement uncertainty (uncertainty in the reading value of each sensor), and (2) the model uncertainty (uncertainty in the output of each component model). As a result, the computed residuals, for example the difference between the model predictions and the measurements, will be affected by uncertainty. To this aim, Bayesian inference is used to detect and localize possible faults starting from the observed fault symptoms, such as the relation between the posterior probability and prior probability of a fault can be written as the following equation. A generalized version of this formula is implemented to account for multiple-fault event scenarios. ^| ^ ^^^^^|^^^ ∙ ^^^^^^^^ ^^ ^^ ൌ^^^^^^

[0107] Where, ^^ is the set of ^^(S) is the prior stateprobabilities; ^^(^^|^^) is the likelihood of the observed data, i.e. the probability to have observations described by ^^ for state S to occur; ^^(^^) is the probability to have the observations in ^^ regardless of whether any faults occurred; and ^^(^^|^^) is the posterior probability of the fault, i.e., the probability that state S has occurred given the observed residuals ^^.

[0108] As a result of the online monitoring process, at every time-step, the online monitoring and system diagnosis module evaluates the conditions of the different Piping and Instrumentation Diagram (P&ID) items (sensor or components), i.e., operating, faulted, degraded.

[0109] In case inconsistencies between the set of observations and the fault-free system model are detected, the compatible fault scenarios are identified. For each one of them, the posterior probabilities are calculated by using the Bayesian network method. Possible diagnoses are then ranked in a meaningful order so that unlikely events can be eliminated.

[0110] As was described previously, the probability of the component to be in a given state at a future time can be calculated from the Markov component model. The obtainedAtty. Dkt. No.103361-637WO1 component state probabilities are passed to the online monitoring and system diagnosis module to be utilized as the prior probabilities for its internal Bayesian models. The online monitoring and system diagnosis module then utilizes sensor information to perform a Bayesian update and returns posterior state probabilities to the Markov component models. The component state probabilities are then used within the Markov models to predict the state probabilities at the next time-step. The component status information is then passed to the facility risk profile, which is updated to reflect both the real-time risk profile of the facility, along with an estimate of the risk profile of the facility in the future.

[0111] Implementation Approach. The linking of the online monitoring and system diagnosis module and the Markov component models provides a real-time estimate of the status of facility components. This information can be utilized in the MDP to inform the current facility state (i.e., ^^). The result is a POMDP, since the current state is not known but only estimated. In a POMDP, observations O of the system state are made but are uncertain. This directly aligns with the estimated state probabilities that are obtained through the utilization of the online monitoring and system diagnosis module. The following examples demonstrate how this information is integrated into the MDP framework. PROBABILISTIC RISK ASSESSMENT

[0112] Theoretical Approach. Incorporation of the PRA into the intelligent asset- management decision-making approach is a necessary step to ensure that facility operations remain within acceptable safety bounds. The insights from the PRA provide critical insights into the acceptability of proposed asset-management strategies, including whether such actions would preserve facility operating status within the limits of the facility license. To accomplish this task, the real-time PRA can be utilized in conjunction with the risk-informed, performance-based licensing approach, such as the Licensing Modernization Project (LMP).

[0113] Probabilistic Risk Assessment (PRA) Overview. A PRA is a systematic and comprehensive study that addresses a risk triplet: I) what can go wrong; II) how likely it is: and III) what are its consequences.

[0114] The PRA model utilizes an event tree / fault tree methodology, where event trees are used to graphically depict potential event sequences, with fault trees typically utilized to develop the probabilities of success / failure of the top events. Success criteria are established through understanding the system perturbations (initiating events) and how the system responds to them.Atty. Dkt. No.103361-637WO1

[0115] Fig.8 shows a simplified diagram of PRA process for a facility consisting of two systems (system 1 and system 2) arranged in parallel. The facility’s main operation goal is to produce output, depending on systems availability different levels of output is possible with failure assigned when no output is being produced.

[0116] The middle section of Fig.8 shows an event tree for one of the identified initiating events. The initiating event frequency could be obtained from past industry data. The frequencies for the systems (top events) are taken from the faut tree analysis for the corresponding systems. Note here, if the top event is not a system but rather, for example, an operator action describing whether or not the operator fails to perform certain action after receiving an alarm signal from the control system, the frequency for such event could be taken from human reliability analysis of past experience. The input for fault trees (basic event frequencies) are taken from the industry equipment reliability data or assigned based on expert judgement.

[0117] Implementation Approach. The results of the PRA provide important insights into facility safety and licensing. There are different approaches for the development of a PRA, but only one of them (the real-time PRA approach, as used herein) utilizes online monitoring and diagnostic information to provide real-time updates of failure probabilities and component status. At a high-level, the real-time PRA provides an avenue for assessing the safety impact of different operational strategies. Since the LMP approach to licensing is fundamentally risk-informed, changes to the PRA and the associated facility risk profile can provide direct insights into the status of the facility within its licensing basis. The real-time PRA approach is utilized to aid in the asset-management decision-making framework by incorporating the PRA analysis into the Markov decision process reward or evaluating the output of the MDP in light of the PRA analysis.

[0118] The development of the real-time PRA is achieved similarly as disclosed previously with respect to real-time diagnostic information. The PRA utilizes component state probabilities which are estimated by the Markov Model module and updated (with input from the sensor network) within an online monitoring and system diagnosis module. This process of passing information from one module to another is repeated at each time step and requires linking several different computer codes.

[0119] First, an event tree / fault tree solver (e.g. SAPHIRE) is used for the PRA and GRA analyses. While the code includes component failure models, it does not contain explicit Markov model solvers. Therefore, second, a separate Markov analysis module is necessary. The Markov analysis module is capable of assessing a Markov model for each of theAtty. Dkt. No.103361-637WO1 components of interest in the facility. If an analytical solution for a particular Markov model is not available, this module solves the Markov model numerically. Third, the online monitoring and system diagnosis module is needed to perform the online monitoring and diagnostics. Finally, the MDP module conducts the asset-management decision-making optimization.

[0120] It is contemplated that any of the modules can be integrated in a single executable module. For example, an integrated event tree / fault tree solver and Markov model solver can be used. For example, an integrated Markov model solver and MDP module can be used.

[0121] In some embodiments, the MDP module utilizes the PRA and GRA analysis in the asset-management decision-making optimization. For example, both the PRA and GRA analysis are used to evaluate the rewards component of the Markov model.

[0122] In other embodiments, the MDP module utilizes only the GRA analysis in the asset- management decision-making optimization, where the GRA analysis is used to evaluate the rewards component of the Markov model. After the MDP-based optimization, the PRA analysis can be applied to determine which solutions satisfy the safety and licensing requirements of the PRA analysis.

[0123] As for the communication between the modules, there is a two-way communication between the online monitoring and system diagnosis and Markov modules and one-way from the online monitoring and system diagnosis module and PRA and the online monitoring and system diagnosis and the MDP modules, executed at each time step. Both the PRA and MDP modules do not send back any data to either Markov or online monitoring and system diagnosis modules. The PRA module uses the data received to update the current state of facility risk profile as long with making profile predictions for the future. The MDP module utilizes the updated component state probabilities in predicting the best optimized asset management facility policy while checking that the facility stays within risk or safety limits.

[0124] Different asset-management strategies are assessed throughout facility operation utilizing the MDP approach described in the previous sections. In one embodiment, the result of the MDP analysis is one or more asset-management strategies that are optimized to maximize revenue for the facility. Because, the MDP calculation itself does not provide a bound on available actions from a safety perspective, after the optimization process has taken place, the strategy must be analyzed by the real-time PRA to assess whether safety / licensing constraints would be violated over the next operating window. In this embodiment, the asset- management strategy options are simulated utilizing the real-time PRA. The frequency and consequence of potential event sequences is updated based on the planned status ofAtty. Dkt. No.103361-637WO1 components and systems in the facility. This includes actions taken immediately but also those planned in the future. The full list of impacted parameters includes: the mission time of analysis (i.e., operating period of the reactor, includes planned downtime associated with strategy under examination); status of components (in service, out of service, degraded state, repaired, etc.); and failure rate of components (depending on state, repair / maintenance, etc.).

[0125] In another embodiment, the result of the MDP analysis is one or more asset- management strategies that are optimized to maximize revenue for the facility and compliance of safety / licensing constraints. EXAMPLES

[0126] Nuclear Reactor

[0127] This example shows the use of the described asset-management decision-making approach to optimize the cost and plant performance of advanced nuclear reactors. The current research is focused on the optimization of advanced reactor operation and asset management through the use of online monitoring and diagnostics.

[0128] First, during the reactor design phase, a sensor network was developed that properly monitored and diagnosed important component faults and degradation throughout the lifetime of the plant. This was a difficult task as there were many unknowns regarding long-term operational reliability and the associated costs of additional sensors and system penetrations ccould be prohibitive. Therefore, development of the sensor network was optimized based on these criteria while ensuring necessary system diagnostic capabilities. For the current project, the Integrated System Failure Analysis (ISFA) method was utilized for this assessment

[0129] In the second step, the capabilities of the optimized sensor network design were validated through analysis utilizing the Argonne tool PRO-AID. PRO-AID is an online monitoring tool capable of detecting and discriminating faults through a combination of physics-based models and reference-condition measurements. As PRO-AID is the tool that was used for online monitoring and diagnosis of component faults during plant operations (described in the following step), the validation process ensured that PRO-AID was fully capable of leveraging the designed sensor network for the diagnosis of all required faults or degradation.

[0130] Once reactor operation began, PRO-AID was utilized for online monitoring and diagnosis of system conditions. In particular, both slow degradation phenomena (wear and tear of components and sensors) and abrupt events (leakages, valves failures, etc.) were diagnosed. At every time-step, PRO-AID evaluated the conditions of the different P&IDAtty. Dkt. No.103361-637WO1 items (sensor or components), i.e., operating, faulted, degraded. In case inconsistencies were detected between the set of measurements and the fault-free system models, the compatible fault scenarios were identified. For each one of the scenarios, the corresponding probabilities were calculated by using the Bayesian network method and then ranked so that unlikely events were eliminated.

[0131] Based on the analysis performed by PRO-AID, the plant risk profile was updated to accurately represent the real-time condition of the plant. The plant risk profile included safety considerations, which are evaluated through the probabilistic risk assessment (PRA), and productivity concerns, which are gauged through the use of a generation risk assessment (GRA).

[0132] Utilizing the real-time plant risk profile, a risk-informed decision-making process optimized plant operations and asset management plans. The challenges of this task included cost-benefit decision-making in multivariate space while ensuring the plant did not approach risk or safety limits. Markov decision processes were utilized to perform this task in an efficient and intelligent manner.

[0133] Component Markov Models. As a simple example to illustrate the integration of Markov component models into the MDP framework, consider a system of two components arranged in series, a pump and a valve. Both components have two states: Healthy (H) and Failed (F) with a constant transition rate, ^^^0.05 / time^. For this component configuration, at any point in time the system can be in one of four possible states. In State 1, both pump and valve are in Healthy state. In State 2, the pump is in Healthy state, and the valve is in Failed state. In State 3, the pump is in a Failed state, and the valve is in a Healthy state. In State 4, both the pump and valve are in a Failed state.

[0134] Because the Markov components are arranged in series, both the pump and the valve are required to be in the Healthy state for the system, as a whole, to operate. To introduce this condition into the model, a reward map is used. The system generates revenue only when it is in State 1, and is assigned a positive reward value, while all other states have a zero reward value.

[0135] The four maintenance actions listed in Table 1 are considered for this example. For the simplicity of the example the repair is considered instantaneous and perfect, meaning that after repair the component is as-good-as-new. Each of the maintenance actions is assigned its associated cost (treated as negative reward in the model), shown in Table 1. The do-nothing action (A1) has no associated cost as the name suggests, some cost to repair each of theAtty. Dkt. No.103361-637WO1 system components were assigned (A2 and A3), and cost of the last maintenance action (A4) is being sum of ones from A1 and A2. Table 1. Component Markov Model Example – Maintenance Actions and Cost Action Number Action Reward A1 Do Nothing 0 ,the probability of the system to be in a particular state at the present is one, with a zero probability for the other system states. For example, if both components are in the Healthy state, the vector of system state probabilities would be p(S) =

[1000] . This set of values defines the system states initial conditions for the next time interval. In real life, there is always some uncertainty in the exact system state, and some value between zero and one would be assigned to the system state probabilities.

[0137] Assuming the system starts in State 1, over the next time interval, Dt, the system could move from state 1 to state 2 (if Valve failed during the Dt), state 3 (if Pump failed), or state 4 (if both components failed). For this example, these transitional probabilities can be easily calculated based on the Markov component models, as shown in Table through Error! Reference source not found.. Each table contains calculations performed for all four maintenance actions for one set of initial conditions (Table contains results for the case when system in State 1 at the beginning of interval ^t, Table 3. Component Markov Model System State Transition Probability (system begins in State 2) Component StateState Probability i m () 2 8 2 8Atty. Dkt. No.103361-637WO1 4F FPP(F) = 1-0 PP(H) time = Δt 1 H H 1Table 3. Component Markov Model System State Transition Probability (system begins in State 2) Component StateState Probability S stemMaintenancem () 2 8 2 8

[0139] – in State 3, and Table 3. Component Markov Model System State Transition Probability (system begins in State 2) Component StateState Probability m () 2 8Atty. Dkt. No.103361-637WO1 time = Δt 1 H H PP(H) = e-λp*Δt0.9512 S(Δt|(S1(0), 2 H F Replace PV(H)=1 0 A3)) 3 F H Valve P (F) = 1- 00488S(^t|Si(0), Ak), is the probability for the system to be in the particular state at the end of time interval ^t given that in the beginning of the time interval ^t it was in state i and maintenance activity Ak was performed during ^t. After the calculations are done and a new vector of system state probabilities is found, those values would be used as initial conditions for the next time interval. The process is repeated for the entirely of the system mission time or for a certain time interval of the interest. Table 2. Component Markov Model System State Transition Probability (system begins in State 1) Component StateState Probability Maintenancem () 8 4 4 4 2 8 2 8Table 3. Component Markov Model System State Transition Probability (system begins in State 2)Atty. Dkt. No.103361-637WO1 Component StateState Probability SystemMaintenanceAti m () 2 8 2 8. p y y y g 3) Component StateState Probability Maintenancem () 2 8 2 8Table 5. Component Markov Model System State Transition Probability (system begins in State 4) ComponentMaintenance State ProbabilityAtty. Dkt. No.103361-637WO1 System Pump VSystem Statealve Pump ValvePP()*PV()(variable T). At each time interval, the Markov component models provide a new set of transition probabilities to be utilized in the MDP calculation. In this way, the likelihood of the plant being in a future state is directly integrated with the Markov component models. For the example analysis, an optimal maintenance policy can be found using an MDP calculation, however the calculation is trivial given the simplicity of the problem (i.e., if a component is in a failed state then it should be repaired). Additional complexity is added in the example for real-time monitoring.

[0142] Generation Risk Assessment. An extremely simplified GRA model was developed to demonstrate how the GRA is integrated into the MDP framework through the modification of the rewards R. Consider a three-state plant model – 100% power, 80% power, and 0% power (plant trip). Fig.6A shows a fault tree representation of the degraded power states. The plant consisted of two subsystems – feedwater (FW) subsystem and Transformer subsystem. Only the FW system was directly considered in the resulting MDP, but the Transformer subsystem was also provided to demonstrate how additional systems can be incorporated into the framework.

[0143] The FW subsystem had three components, they were a pump and two FW heaters (heater 1 and heater 2). The heaters were arranged in parallel. Each component had two possible operational states, healthy (H) and failed (F). The FW system needed all three components in healthy state to be at 100% power. With either one of the heaters downAtty. Dkt. No.103361-637WO1 (failed) the FW system can only operate at 80% power. If both heaters are down or pump failed, the FW system is at 0% power level. Fault tree representation of FW subsystem power levels is shown in Fig.6B.

[0144] The transformer subsystem also had three components, one transformer and two coolers (cooler 1 and cooler 2). The coolers were arranged in parallel. Each component in the transformer subsystem had two possible operational states – healthy (H) and failed (F). The Transformer system needed all three components in healthy state to be at 100% power. Failure of either one of the coolers causes transformer cooling power to be reduced to 80% of maximum power. When both coolers were down or the transformer was in a failure state, the transformer system was at 0% power level. Fault tree representation of transformer system power levels is shown in Error! Reference source not found.6C.

[0145] As has been mentioned above, each FW component had only two operational states – healthy and failed. Table shows possible states of FW subsystem, where FWH-1 and FWH-2 stand for FW heater 1 and FW heater 2, respectively. Table 6. GRA Example – FW Subsystem State Map State Number Pump FWH-1 FWH-2

[0146] T ponents inthis example. Do-nothing option (action A1) is followed by three individual component repair options (repair pump (A2), repair FWH-1 (A3), and repair FWH-2 (A4). Option to repair both FW heaters (A5) and repair all three FW components (A6) conclude the list. Table 7. GRA Example - Maintenance Actions Action Number Action A1 D N thi

[0147] Out of eight possible plant states, only top three represent conditions when the FW subsystem was at power (100 % or 80% of maximum power). When the FW subsystem wasAtty. Dkt. No.103361-637WO1 in either one of the remaining five states, it operates at 0%. Based on this information, the corresponding rewards shown in Table 8 column 2 are developed to be considered by the MDP process. Note here, the rewards shown in column 2 are the ones the whole plant would get if there were no other plant subsystems besides the FW system taken into account. In other words, the plant will get reward 10 if it is in the 100% power plant state, reward 8 for being in the 80% power state, and reward 0 otherwise. Table 8. GRA Example - Rewards Map State Number Reward (FW indep) Reward (whole reactor) 1 10 10×P1-100 + 8×P1-80 + 0×(1-P1-100-P1-80)m must be taken into account while calculating the rewards at the plant level. Column 3 in Error! Reference source not found.8 shows a possible way to accomplish that. P1100 is the global probability of 100% power at t=t+1 given state 1 of FW subsystem at that time. Similarly, P180is the global probability of 80% power at t=t+1 given state 1 of FW subsystem at that time. ^^^భబబ ൌ ^^ி^భ ∙ ^^்ோேభబబwhere PFW1is the probability of FW subsystem to be in state 1 (all three components are in healthy state). ^^ி^భ ൌ ^^^௨^^^^^^ ∙ ^^ி^ு^^^^^ ∙ ^^ி^ுଶ^^^^PTRN100 is the(all three components are in healthy state). It can be found using component Markov model in a similar way as was described above, i.e., ^^்ோேభబబ ൌ ^^்ோே^^^^ ∙ ^^்^^^^^^ ∙ ^^்^ଶ^^^^or from the GRA fault tree model as ^^்ோேభబబ ൌ 1 െ ^^்ோே^బെ ^^்ோேబAtty. Dkt. No.103361-637WO1 here, PTRN80 and PTRN0 are top gates from the transformer fault tree model shown in Fig.6C calculated with a mission time equal to the time interval in the MDP process.

[0149] An additional benefit of this framework for GRA integration into the MDP is that planned maintenance activities can also be considered. It was assumed that at some known time in the future transformer cooler 2 will be taken offline for maintenance. With one transformer cooler out of service, the plant can only operate at a max of 80% power. In this case, the whole plant rewards can be calculated using the same approach as above but with P1100 = 0. Table 9 below shows the updated reward calculations. Table 9. GRA Example - Transformer System Maintenance State Number Reward (whole reactor) 1 10 × 0 + 8×P1-80+ 0 × (1- P1-80)

[0150] Reasidered here. In this example, the current state of the components (and therefore the state of the system) was known. However, in reality, the true status of the components is not known, but sensor data and diagnostic software estimate the current status. In the simplified pipe and valve example, there are four system states where the components are in either Healthy (H) or Failed (F) state. In an observable MDP, the current system state is known (for example, p(S) =

[1000] ). In a POMDP, a probability is assigned to each system state (for example, p(S) = [0.60.3 0.10]).

[0151] As described previously, the POMDP can still be solved to identify an optimal policy. However, here the policy is also dependent on the uncertainty regarding system state. For the simplified pipe and valve example, Fig.7 shows an optimal policy map based on the current system state probabilities. As can be seen, the optimal action changes as the likelihood of being in a certain plant state changes. For example, if there is a high level of confidence that the system is in a healthy-healthy state (HH), noted in the upper most region of Fig.7, then the optimal policy is to not take any repair actions. While this is a simple example, the dimensions of the decision-making space increase as the number of components (and therefore system states) grows.Atty. Dkt. No.103361-637WO1

[0152] Licensing Modernization Project. The use of PRA in reactor licensing has evolved since its introduction in the 1970s. The use of risk-informed decision-making in regulatory matters has gradually expanded and a new risk-informed performance-based licensing approach (the LMP approach) has been developed and endorsed for use by advanced reactor vendors. The LMP approach utilizes risk information to inform key licensing decisions, such as the identification and categorization of licensing basis events (LBEs), the safety classification of structures, systems, and components (SSCs), and the evaluation of the adequacy of defense-in-depth.

[0153] As this example focuses on the economics of advanced reactors, the implementation approach assumes the utilization of the LMP process for reactor licensing. Of particular importance for this example is the LMP’s use of the frequency versus consequence (F-C) curve shown in Error! Reference source not found.9. When utilizing the LMP approach, event sequences or event sequence families from the PRA are plotted on the F-C curve. The frequency of the event sequences is utilized for LBE categorization (anticipated operational occurrence – AOO, design basis event – DBE, beyond design basis event – BDBE). In addition, the event sequences and their associated offsite dose are compared to a consequence target, which is developed based on applicable regulation and guidance. The distance from the event sequences to the consequence target helps demonstrate the available safety margin of the design. As depicted in Error! Reference source not found.10, the F-C curve also has a designated “risk significant” region, which identified LBEs that may be located close to the consequence target.

[0154] The F-C curve is also utilized to aid in the safety classification of SSCs. The importance of different SSCs is determined through PRA sensitivity studies, such as not crediting an SSC (i.e., assuming it is in the failed state) and assessing the impact of event sequence placement on the F-C curve. For example, if not crediting an SSC results in an event sequence exceeding the consequence target, than that SSC may be designated as “Safety Related,” the highest safety classification for SSCs.

[0155] Probabilistic Risk Assessment. The following example describes how the PRA is incorporated into the integrated asset-management decision-making approach. For this example, it is assumed that the advanced reactor design under consideration utilized the LMP approach for licensing and the original results of the F-C curve assessment are shown in Fig. 11A. As the figure demonstrates, all LBEs are within the consequence target by considerable margin.Atty. Dkt. No.103361-637WO1

[0156] During operation, the example plant utilizes the MDP approach discussed in the previous sections to assess the condition of components within the plant and potential actions for component maintenance and / or replacement. Based on this assessment, two potential operational plans are identified, options A and B, that optimize the generation (revenue) of the plant. The two options have similar predicted revenue but utilize different approaches to asset-management, such as repair / replacement timing.

[0157] Before one of the asset-management options is selected, a PRA assessment is conducted of each proposal to assess the impact on plant safety and licensing. To conduct this analysis, the real-time PRA is utilized in conjunction with the asset-management plan to estimate the safety risk of the facility over the next time interval (such as the interval to next plant shutdown). To do this, the attributes of the asset-management option are integrated into the PRA model. For example, if option A includes the continued operation of a degraded piece of equipment, then the increased likelihood of the failure of that component is reflected in the PRA. In contrast, if option B represents an immediate shutdown to repair the component, then the decrease in the likelihood of component failure is represented in the PRA.

[0158] The results of the PRA assessment for each asset-management option are reflected in changes to the LBE location on the F-C curve. For this example, the results for option A are presented in Fig.11B, which highlights several major changes. First, an LBE that was located in the BDBE region has now moved into the DBE region due to an increase in frequency. Second, an event sequence that was below the 5×10-7per year threshold value has increased in frequency and now exceeds the consequence target. Both of these changes could have significant repercussions on plant licensing. For example, the movements of an LBE from the BDBE to DBE region will impact the SSC classification decisions, as the importance of certain SSCs will likely change. In addition, under the LMP approach, design basis accidents (DBAs) are derived from those LBEs in the DBE region. Option A results in a new LBE in the DBE region and could result in an additional DBA not currently in the plant license. Lastly, the movement of an LBE to a location that exceeds the consequence target would require further assessment to justify the regulatory acceptable of such a plant posture (if a justification is possible).

[0159] In contrast, the example LBE results for asset-management option B are reflected in Fig.11C. As can be seen, while there is movement in the frequency of certain LBEs, none of the LBEs switch categorization or exceed the consequence target. While additional analysesAtty. Dkt. No.103361-637WO1 are necessary to confirm that there is no impact on SSC classification decisions (due to the movement of an LBE into a risk significant region), these results indicate that such a strategy is likely acceptable from a plant safety / licensing standpoint. DEFINITIONS

[0160] The construction and arrangement of the systems and methods as shown in the various implementations are illustrative only. Although only a few implementations have been described in detail in this disclosure, many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes, and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.). For example, the position of elements may be reversed or otherwise varied, and the nature or number of discrete elements or positions may be altered or varied. Accordingly, all such modifications are intended to be included within the scope of the present disclosure. The order or sequence of any process or method steps may be varied or re-sequenced according to alternative implementations. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the implementations without departing from the scope of the present disclosure.

[0161] The present disclosure contemplates methods, systems, and program products on any machine-readable media for accomplishing various operations. The implementations of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwired system. Implementations within the scope of the present disclosure include program products including machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine- readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine- executable instructions or data structures, and which can be accessed by a general purpose or special purpose computer or other machine with a processor.

[0162] When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired orAtty. Dkt. No.103361-637WO1 wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general-purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.

[0163] Although the figures show a specific order of method steps, the order of the steps may differ from what is depicted. Also, two or more steps may be performed concurrently or with partial concurrence. Such variation will depend on the software and hardware systems chosen and on designer choice. All such variations are within the scope of the disclosure. Likewise, software implementations could be accomplished with standard programming techniques with rule-based logic and other logic to accomplish the various connection steps, processing steps, comparison steps and decision steps.

[0164] It is to be understood that the methods and systems are not limited to specific synthetic methods, specific components, or to particular compositions. It is also to be understood that the terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting.

[0165] As used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and / or to “about” another particular value. When such a range is expressed, another implementation includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another implementation. It will be further understood that the endpoints of each of the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.

[0166] “Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0167] Throughout the description and claims of this specification, the word “comprise” and variations of the word, such as “comprising” and “comprises,” means “including but not limited to,” and is not intended to exclude, for example, other additives, components, integersAtty. Dkt. No.103361-637WO1 or steps. “Exemplary” means “an example of” and is not intended to convey an indication of a preferred or ideal implementation. “Such as” is not used in a restrictive sense, but for explanatory purposes.

[0168] Disclosed are components that can be used to perform the disclosed methods and systems. These and other components are disclosed herein, and it is understood that when combinations, subsets, interactions, groups, etc. of these components are disclosed that while specific reference of each various individual and collective combinations and permutation of these may not be explicitly disclosed, each is specifically contemplated and described herein, for all methods and systems. This applies to all aspects of this application including, but not limited to, steps in disclosed methods. Thus, if there are a variety of additional steps that can be performed it is understood that each of these additional steps can be performed with any specific implementation or combination of implementations of the disclosed methods.

Claims

Atty. Dkt. No.103361-637WO1 WHAT IS CLAIMED IS:

1. A method comprising: determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component, wherein the sensor data is obtained from a plurality of sensors in the facility; generating a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior models for each of the plurality of physical components; and determining, using a Markov decision process, an optimal asset-management strategy based on the real-time risk profile and the statuses of the plurality of physical component.

2. The method of claim 1, wherein the status comprises a real-time probability of a state of the physical component.

3. The method of claim 1 or 2, wherein the real-time risk profile comprises a plurality of risk models related to one or more probable future states.

4. The method of claim 3, wherein the one or more probable future states are one of healthy, derate, or failure.

5. The method of any one of claims 3-4, wherein the future state is determined by a user.

6. The method of any one of claims 3-5, wherein each of the plurality of risk models is associated with one or more of the plurality of physical components.

7. The method of any one of claims 3-6, wherein at least one of the plurality of risk models is based on a probabilistic risk assessment (PRA) indicative of safety risk.

8. The method of any one of claims 3-6, wherein at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk.

9. The method of any one of claims 3-6, wherein at least two of the plurality of risk models is based on a probabilistic risk assessment (PRA) and on a generation risk assessment (GRA).Atty. Dkt. No.103361-637WO1 10. The method of any one of claims 3-9, wherein a fault tree method is used to configure each of the plurality of risk models.

11. The method of claim 1, wherein determining an optimal asset-management strategy comprises evaluating a plurality of asset-management strategies for the facility based on the real- time risk profile and the statuses of the plurality of physical components.

12. The method of claim 11, wherein the optimal asset-management strategy comprises an optimal subset of a plurality of asset management strategies.

13. The method of claim 12, the method further comprising evaluating a probabilistic risk assessment (PRA) for each of the optimal subset of a plurality of asset management strategies, wherein the PRA is associated with the assessment of safety parameters.

14. The method of any one of claims 1-13, the method further comprising operating the plurality of physical components in the facility based on the optimal asset-management strategy.

15. A method comprising: determining, for each of a plurality of physical components in a facility, a status using sensor data corresponding to the physical component, wherein the sensor data is obtained from a plurality of sensors in the facility; generate a real-time risk profile for the facility based on the statuses of the plurality of physical components and Markov component behavior model for each of the plurality of physical component; evaluating a plurality of asset-management strategies for the plurality of physical components of the facility based on the real-time risk profile and the statuses of the plurality of physical components; determining, using a Markov decision process, a set of value-optimized asset- management strategies; and determining a subset of value-optimized asset-management strategies that satisfy a set of safety requirements.

16. The method of claim 15, wherein the status comprises a real-time probability of a state of the physical component.Atty. Dkt. No.103361-637WO1 17. The method of claim 15 or 16, wherein the real-time risk profile comprises a plurality of risk models related to one or more probable future states.

18. The method of claim 17, wherein a future state is one of healthy, derate, or failure.

19. The method of any one of claims 17-18, wherein the future state is determined by a user.

20. The method of any one of claims 17-19, wherein each of the plurality of risk models is associated with one or more of the plurality of physical components.

21. The method of any one of claims 17-20, wherein at least one of the plurality of risk models is based on a generation risk assessment (GRA) indicative of economic risk.

22. The method of any one of claims 17-21, wherein a fault tree method is used to configure each of the plurality of risk models.

23. The method of any one of claims 15-22, wherein determining the set of value-optimized asset-management strategies comprises evaluating a plurality of asset-management strategies for the facility based on the real-time risk profile and the statuses of the plurality of physical components.

24. The method of any one of claims 15-23, wherein a probabilistic risk assessment (PRA) is used to determine the subset of value-optimized asset-management strategies.

25. The method of any one of claims 15-24, the method further comprising operating the plurality of physical components in the facility based on one of the subset of value-optimized asset-management strategies that satisfy the set of safety requirements.

Images