A method and apparatus for autonomous optimization of FMECA risk assessment model

By combining deep Q-networks with FMECA risk assessment, an autonomous optimization model is constructed, which solves the problem of performance degradation in dynamic environments in existing technologies and achieves autonomous optimization and improved accuracy of risk assessment.

CN123067418APending Publication Date: 2026-11-10CHINA ORDNANCE EQUIP GRP AUTOMATION RES INST CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611084056.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-21
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing FMECA risk assessment models are ill-suited to adapting to equipment aging, process parameter adjustments, and production cycle changes in dynamic production environments. They have fixed failure propagation weights, manually set risk warning thresholds, and lack real-time optimization and online learning capabilities, resulting in decreased assessment performance.

Method used

By combining deep Q-networks with FMECA risk assessment, a failure propagation network is constructed. The failure propagation edge weights and risk warning thresholds are optimized through Markov decision processes. Real-time data and historical information are used for autonomous optimization, and a dual deep Q-network is constructed for online learning and pre-training.

Benefits of technology

It improved the accuracy of risk assessment results, reduced false alarm and false negative rates, enhanced the model's adaptability to complex operating conditions, and achieved autonomous optimization of risk assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123067418A_ABST
    Figure CN123067418A_ABST
Patent Text Reader

Abstract

This invention discloses an autonomous optimization method and apparatus for an FMECA risk assessment model, relating to the fields of intelligent manufacturing and industrial IoT technology. This method transforms the parameter optimization process of the FMECA risk assessment model into a reinforcement learning problem. Through a deep Q-network, it adaptively adjusts the weights of failure propagation edges, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights, reducing reliance on human experience and improving the consistency between risk assessment results and actual failure results. Through experience replay, target networks, and online learning mechanisms, the system can continuously correct its risk assessment strategy in dynamic operating environments, thereby improving the accuracy of risk warnings, reducing false alarms and false negatives, and enhancing the adaptability of the FMECA risk assessment model to complex operating conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of intelligent manufacturing and industrial Internet of Things (IIoT) technology, and in particular to an autonomous optimization method and apparatus for an FMECA risk assessment model based on deep Q-networks. Background Technology

[0002] In the fields of intelligent manufacturing and industrial Internet of Things, production equipment is highly interconnected, its operating status changes dynamically, and the impact of failures propagates to each other. Accurately identifying equipment failure modes and conducting timely risk assessments and early warnings are of great significance for ensuring reliable system operation and production safety.

[0003] In existing technologies, FMECA has been widely used for risk assessment. For example, it identifies failure modes, analyzes failure causes and effects, and calculates risk priority numbers or classifies risk levels based on severity, occurrence and detectability. Some technologies also combine failure propagation diagrams to analyze the propagation path of failures between devices or functional modules.

[0004] The closest existing technology to this invention is "A Method for Calculating the Probability of Single-Event Failure Based on Extended FMECA and Fault Propagation Map" (Application No.: 202110028127.4, Publication No.: CN112782549A). This invention analyzes the system composition and single-event effects of airborne electronic equipment, conducts extended FMECA analysis, constructs a fault propagation map, and calculates the probability of single-event failure by combining the fault propagation path. However, this invention mainly relies on pre-established FMECA results and fault propagation relationships for static probability calculation, without considering the dynamic characteristics of fault changes over time. It lacks real-time optimization and online learning capabilities, making it difficult to continuously optimize the risk assessment model according to changes in equipment status, process parameters, and production environment.

[0005] However, existing technologies face key challenges in practical applications: First, risk parameters are usually determined based on historical data or rely directly on expert experience, making it difficult to adapt to equipment aging, process parameter adjustments, and changes in production cycle time. Second, failure propagation weights are usually fixed and cannot reflect changes in the strength of fault association and propagation path under different operating conditions in a timely manner. In addition, risk warning thresholds are mainly set manually, making it difficult to dynamically balance warning accuracy, recall, false alarm rate, and false negative rate. Finally, existing models lack a closed-loop autonomous optimization mechanism that uses actual warning results and new production data for feedback learning, which may cause risk assessment performance to decline with changes in the production environment. Summary of the Invention

[0006] In view of the above problems, the present invention provides an autonomous optimization method and apparatus for an FMECA risk assessment model to overcome or at least partially solve the above problems. By combining deep Q-networks and FMECA risk assessment, the method addresses the shortcomings of existing technologies in dynamic production environments and autonomous optimization mechanisms.

[0007] This invention provides the following solution: An autonomous optimization method for the FMECA risk assessment model includes: A failure propagation network is defined, which includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The optimization process of the FMECA risk assessment model is modeled as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalties, missed alarm penalties, and risk handling costs. A dual deep Q-network consisting of a policy network and a target network is constructed, and pre-training and online updates are performed based on an experience replay buffer; The system acquires real-time operating data, historical failure data, risk event records, and environmental context data of the target system to construct the current state. The current state is then input into the policy network to output optimization actions. The parameters of the FMECA risk assessment model are adjusted according to the optimization actions. Based on the adjusted FMECA risk assessment model, the risk level, key failure modes, key failure propagation chains, and risk handling priorities are output. Real-time rewards are calculated based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs. Experience samples are generated and stored in the experience playback cache. When the number of experience samples reaches a preset batch size, an online learning process is initiated to update the policy network and the target network.

[0008] Preferably, the historical risk event records include risk type, risk level, severity, frequency of occurrence, and handling results; the equipment status parameters include health index, performance degradation trend, fault history, and status of key components; the communication status includes latency, packet loss rate, bandwidth utilization, and communication anomaly records; and the environmental context includes production task type, process parameters, environmental conditions, and operating load.

[0009] Preferably, the adjustment strategy includes increasing the weight of the failure propagation edge, decreasing the weight of the failure propagation edge, increasing the risk warning threshold, decreasing the risk warning threshold, adjusting the severity weight, adjusting the occurrence weight, adjusting the detectability weight, and maintaining the current parameters unchanged.

[0010] Preferably, each adjustment strategy in the action space corresponds to the discrete action output of the strategy network, and the execution of the adjustment strategy is constrained by the mathematical relationship of Risk Priority Number RPN = Severity × Occurrence × Detectability. When the adjustment strategy changes the severity weight or occurrence weight, the Risk Priority Number is updated synchronously.

[0011] Preferably, the instant reward is calculated using the following formula:

[0012] In the formula: Indicates the first The accuracy rate of risk warnings during the second operation; Recall rate, indicating the risk warning; Indicates the penalty for false alarms; This indicates a penalty for failure to report. Indicates the cost of risk management; The reward weighting coefficient is used to balance the importance of different evaluation indicators.

[0013] Preferably: the reward weighting coefficient The recall rate weight is dynamically adjusted based on the risk level, with high-risk failure modes corresponding to higher recall rates. The recall weight is greater than that of low-risk failure modes.

[0014] Preferably, the pre-training and online updating based on the experience replay buffer includes: Randomly sample a small batch of samples from the experience replay buffer; calculate the current state action value through the policy network, and select the optimal action for the next state through the policy network; calculate the target Q value through the target network; update the policy network parameters according to the loss function, and update the target network parameters in a soft update manner.

[0015] Preferably, the online learning initiation process includes: When the number of experience samples reaches the preset batch size, a small batch of samples is randomly sampled from the experience replay buffer to update the policy network and the target network; When the preset retraining period is reached, the deep Q network is periodically retrained using historical samples in the experience replay buffer, and the FMECA risk assessment model is updated based on the retraining results.

[0016] Preferably, the current state of construction includes: Historical risk event records, device status parameters, and communication status are arranged according to the effective step size of historical data and combined with the current environmental context to form a state vector; when the length of historical data is less than the effective step size of historical data, zero padding is used to keep the dimension of the state vector consistent.

[0017] An autonomous optimization device for an FMECA risk assessment model, used to execute the aforementioned autonomous optimization method for an FMECA risk assessment model, the device comprising: The FMECA risk assessment model determination unit is used to determine the FMECA risk assessment model. The FMECA risk assessment model includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The risk analysis and assessment modeling unit is used to model the optimization process of the FMECA risk assessment model as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalty, false negative penalty, and risk handling cost. The Deep Q-Network Pre-training Unit is used to construct a dual Deep Q-Network that includes a policy network and a target network, and is pre-trained and updated online based on the experience replay buffer; The risk assessment model online optimization unit is used to acquire real-time operating data, historical failure data, risk event records and environmental context data of the target system to construct the current state, input the current state into the policy network to output optimization actions, adjust the parameters of the FMECA risk assessment model according to the optimization actions, and output the risk level, key failure modes, key failure propagation chains and risk handling priorities based on the adjusted FMECA risk assessment model. The network update unit is used to calculate real-time rewards based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs, and to generate experience samples to be stored in the experience replay cache. When the number of experience samples reaches a preset batch size, the online learning process is initiated to update the policy network and the target network.

[0018] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention provides an autonomous optimization method and apparatus for an FMECA risk assessment model. The method transforms the parameter optimization process of the FMECA risk assessment model into a reinforcement learning problem. Through a deep Q-network, it adaptively adjusts the weights of failure propagation edges, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. This reduces reliance on human experience and improves the consistency between risk assessment results and actual failure results. Through experience replay, target networks, and online learning mechanisms, the system can continuously correct its risk assessment strategy in dynamic operating environments, thereby improving the accuracy of risk warnings, reducing false alarms and false negatives, and enhancing the adaptability of the FMECA risk assessment model to complex operating conditions.

[0019] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a flowchart of an autonomous optimization method for the FMECA risk assessment model provided in an embodiment of the present invention; Figure 2 This is a flowchart of the FMECA risk analysis and assessment modeling method provided in this embodiment of the invention; Figure 3 This is a flowchart of the deep Q-network pre-training method provided in the embodiments of the present invention; Figure 4 This is a flowchart of the online optimization method for the FMECA risk assessment model provided in this embodiment of the invention; Figure 5 This is a schematic diagram of the autonomous optimization method and system example of the FMECA risk assessment model based on deep Q-network provided in the embodiments of the present invention; Figure 6 This is a schematic diagram of an autonomous optimization device for an FMECA risk assessment model provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of an FMECA risk assessment model autonomous optimization device provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0023] See Figure 1 This invention provides an autonomous optimization method for the FMECA risk assessment model, as illustrated in this embodiment. Figure 1 As shown, the method may include: S101: Determine the FMECA risk assessment model, which includes a failure propagation network. The failure propagation network uses equipment units, key components, or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. S102: The optimization process of the FMECA risk assessment model is modeled as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall rate, false alarm penalty, false negative penalty, and risk handling cost. In specific implementations, this embodiment of the invention can provide that the historical risk event records include risk type, risk level, severity, occurrence frequency, and handling results; the equipment status parameters include health index, performance degradation trend, fault history, and key component status; the communication status includes latency, packet loss rate, bandwidth utilization, and communication anomaly records; and the environmental context includes production task type, process parameters, environmental conditions, and operating load.

[0024] The adjustment strategies include increasing the weight of the failure propagation edge, decreasing the weight of the failure propagation edge, increasing the risk warning threshold, decreasing the risk warning threshold, adjusting the severity weight, adjusting the occurrence weight, adjusting the detectability weight, and maintaining the current parameters unchanged.

[0025] Each adjustment strategy in the action space corresponds to the discrete action output of the strategy network, and the execution of the adjustment strategy is constrained by the mathematical relationship of Risk Priority Number RPN = Severity × Occurrence × Detectability. When the adjustment strategy changes the severity weight or occurrence weight, the Risk Priority Number is updated synchronously.

[0026] S103: Construct a dual deep Q-network containing a policy network and a target network, and perform pre-training and online updates based on the experience replay buffer; S104: Acquire real-time operating data, historical failure data, risk event records and environmental context data of the target system to construct the current state, input the current state into the policy network to output optimization actions, adjust the parameters of the FMECA risk assessment model according to the optimization actions, and output the risk level, key failure modes, key failure propagation chains and risk handling priorities based on the adjusted FMECA risk assessment model. S105: Calculate immediate rewards based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs, and generate experience samples to store in the experience playback cache. When the number of experience samples reaches a preset batch size, start the online learning process to update the policy network and the target network.

[0027] In a specific implementation, embodiments of the present invention can provide that the instant reward is obtained by calculation using the following formula:

[0028] In the formula: Indicates the first The accuracy rate of risk warnings during the second operation; Recall rate, indicating the risk warning; Indicates the penalty for false alarms; This indicates a penalty for failure to report. Indicates the cost of risk management; The reward weighting coefficient is used to balance the importance of different evaluation indicators.

[0029] The pre-training and online updating based on the experience replay cache includes: Randomly sample a small batch of samples from the experience replay buffer; calculate the current state action value through the policy network, and select the optimal action for the next state through the policy network; calculate the target Q value through the target network; update the policy network parameters according to the loss function, and update the target network parameters in a soft update manner.

[0030] The process for initiating online learning includes: When the number of experience samples reaches the preset batch size, a small batch of samples is randomly sampled from the experience replay buffer to update the policy network and the target network; When the preset retraining period is reached, the deep Q network is periodically retrained using historical samples in the experience replay buffer, and the FMECA risk assessment model is updated based on the retraining results.

[0031] The FMECA risk assessment model autonomous optimization method provided in this invention constructs a reinforcement learning framework using a deep Q-network with a failure propagation network as the environment and risk warning effect as the reward. It autonomously optimizes failure propagation weights and warning thresholds using production feedback data. Based on FMECA, it identifies failure modes, analyzes propagation relationships, and assesses risk levels. This method can be widely applied in fields such as ammunition, chemical engineering, energy, and intelligent manufacturing, and is suitable for dynamic risk assessment and warning scenarios under continuously changing equipment status, process parameters, and production environment. Compared with traditional methods, this invention can dynamically adapt to changes in equipment process parameters and autonomously optimize. While introducing the adaptive decision-making capability of deep reinforcement learning, it retains the failure mode, cause, impact, and risk level analysis logic of FMECA, demonstrating significant practical value and innovation, and providing a novel solution for risk assessment in equipment manufacturing processes.

[0032] The following is a detailed description of the autonomous optimization method for the FMECA risk assessment model provided in the embodiments of the present invention.

[0033] To address the problems of existing FMECA risk assessment models, such as reliance on human experience, difficulty in adaptively adjusting scoring parameters to changes in operating conditions, and discrepancies between risk warning results and actual failure results, this invention proposes an autonomous optimization method and system for FMECA risk assessment models based on deep Q-networks. This method models the FMECA risk assessment process as a Markov decision process, using the target system's operating state, historical failure data, risk event records, and environmental context as state inputs, FMECA model parameter adjustments as agent actions, and risk warning accuracy, missed detections, false alarms, and handling costs as reward feedback. The FMECA risk assessment model is continuously optimized through a deep Q-network.

[0034] 1. This invention provides a modeling method for FMECA risk analysis and assessment based on Markov decision processes. It models the autonomous optimization process of the manufacturing equipment FMECA risk assessment model as a Markov decision process. The state space is composed of historical risk events, equipment state parameters, network transmission status, and production environment context. The action space consists of adjustments to failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The payoff function is composed of risk warning accuracy, recall rate, false alarm penalty, false negative penalty, and handling cost. The key to this modeling method is that the system does not directly output a one-time risk level, but rather treats the parameters in the FMECA model that affect the assessment results as learnable objects, enabling the risk assessment process to continuously correct itself based on actual production feedback.

[0035] The system incorporates a failure propagation network into the FMECA risk assessment model. The system uses failure modes, failure causes, failure effects, and failure propagation paths as the foundation for risk analysis. Equipment units, key components, or failure modes are treated as nodes, and failure-impact relationships are considered as propagation edges. Adjustable weights for these propagation edges and risk warning thresholds are set. Furthermore, FMECA evaluation parameters such as severity, occurrence, detectability, and hazard are also included in the optimizable scope. This not only preserves the interpretable analytical framework of FMECA but also enables the model to reflect changes in the strength of fault associations under different operating conditions.

[0036] Specifically, such as Figure 2 As shown, the optimization process of the FMECA risk assessment model is modeled as a Markov decision process, represented by a quintuple: .in, Representing the state space, Represents the action space. Represents the state transition probability. Represents the reward function, This represents the long-term reward discount factor, and The unit of the XX equipment is equated with an intelligent agent, which makes action decisions based on real-time observed status data during the manufacturing process.

[0037] Step 1.1: Construct the state space.

[0038] In the During the next operation or production process, the intelligent agent acquires the state information of the target system:

[0039] in, Indicates the first The historical risk event records corresponding to each operation include risk type, risk level, severity, frequency of occurrence, and handling results; Indicates the first The equipment status parameters corresponding to each run include health index, performance degradation trend, fault history and status of key components; Indicates the first The communication status corresponding to each run includes latency, packet loss rate, bandwidth utilization, and communication anomaly records; This indicates the current environmental context, including production task type, process parameters, environmental conditions, and operating load; This indicates the effective step size for historical data.

[0040] The agent, based on state information From the preset motion space Select optimization action ,Right now

[0041] in, Instead of directly representing continuous parameter values, it indicates adjustment strategies for the parameters of the FMECA risk assessment model, including at least one of the following: increasing the weight of the failure propagation side, decreasing the weight of the failure propagation side, increasing the risk warning threshold, decreasing the risk warning threshold, adjusting the severity weight, adjusting the occurrence weight, adjusting the detectability weight, and maintaining the current parameters unchanged. Indicates the first The weight of each edge that fails to propagate. Indicates the first Each risk warning threshold Indicates the number of failed propagation edges. This indicates the number of risk thresholds.

[0042] Step 1.3: Construct the reward function.

[0043] The agent performs actions Afterwards, an immediate reward will be given based on the effectiveness of the risk assessment:

[0044] in, Indicates the first The accuracy rate of risk warnings during the second operation; Recall rate, indicating the risk warning; Indicates the penalty for false alarms; This indicates a penalty for failure to report. Indicates the cost of risk management; The reward weighting coefficient is used to balance the importance of different evaluation indicators.

[0045] Step 1.4: Construct state transition relationships.

[0046] The agent performs optimization actions Afterwards, the system status was updated to:

[0047] Among them, the next state From the current state Optimize actions The state transitions are determined jointly by the actual operational feedback of the target system and the Markov property.

[0048] Step 1.5: Determine the optimization objective.

[0049] The goal of an intelligent agent is to find the optimal strategy. To maximize long-term cumulative rewards:

[0050] in, This represents the parameter optimization strategy of the agent. This represents the expectation operation. Indicates the index of a future time step.

[0051] 2. This invention provides a deep Q-network pre-training method. It employs DDQN to train the parameter adjustment strategy of the FMECA risk assessment model. The system constructs a policy network and a target network, wherein the policy network parameters are... The target network parameters are The policy network is used to select FMECA parameters to optimize actions in the current state, while the target network is used to calculate the target Q-value, thereby reducing the problem of overestimation of Q-values ​​in ordinary DQN. During training, empirical samples are represented as... This method randomly samples mini-batch samples from the experience replay buffer, updates the policy network parameters based on the TD objective and loss function, and updates the target network parameters through soft or periodic updates. This approach uses reinforcement learning to replace manual adjustments to FMECA scoring parameters and warning thresholds.

[0052] like Figure 3 As shown, the specific implementation includes the following steps: Step 2.1: Sample training data from the risk event experience base to construct an experience replay cache. Each experience sample is represented as:

[0053] in, Indicates the current state. This represents the parameter optimization action performed by the agent. Indicates an immediate reward. This indicates the next state after an action is performed.

[0054] Step 2.2: Construct a dual-depth Q-network. The dual-depth Q-network consists of a policy network and a target network. The policy network parameters are as follows: The target network parameters are .

[0055] Step 2.3, from the sample set Randomly sampled batches of training samples from the training set ,in This represents a mini-batch sample set. This indicates the number of training samples in a batch.

[0056] Step 2.4: Calculate the current state action value using the policy network. .in, Indicates the state Next action The value of the action.

[0057] Step 2.5: Select the next state through the policy network. The optimal action is as follows:

[0058] Step 2.6: Calculate the target using the target network. value: .in, This represents the learning objective of the temporal difference.

[0059] Step 2.7: Update the policy network parameters according to the loss function:

[0060] in, Represents the loss function. Indicates the learning rate. The loss function represents the loss function with respect to the policy network parameters. The gradient.

[0061] Step 2.8: Update the target network parameters using the soft update method.

[0062] in, Denotes the target network soft update coefficient, and .

[0063] Step 2.9: Repeat steps 2.3 to 2.8 until the preset number of training rounds, loss function convergence, or validation set performance meets the preset requirements.

[0064] 3. This invention provides an online optimization method for an FMECA risk assessment model. It is a closed-loop optimization method consisting of production data collection, FMECA risk assessment, risk result confirmation, benefit calculation, experience playback, and model updating. During production line operation, the system continuously collects equipment status, process parameters, communication status, and risk event data. Based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs, it calculates real-time benefits and writes the resulting experience samples into an experience playback buffer. When the number of samples reaches a preset batch size, online learning is initiated; when a set retraining cycle is reached, the DDQN agent is periodically retrained using accumulated samples. This mechanism enables the failure propagation edge weights, risk warning thresholds, and FMECA evaluation weights to be continuously updated with equipment aging, changes in operating conditions, and changes in production cycle time.

[0065] like Figure 4 , Figure 5 As shown, the specific implementation includes the following steps: Step 3.1: Deploy the pre-trained deep Q network optimized agent into the FMECA risk assessment module of the target system.

[0066] Step 3.2: Obtain the target system's real-time operating data, historical failure data, risk event records, and environmental context data.

[0067] Step 3.3: Construct the current state based on the data. If the length of historical data is less than the effective step size of historical data. Then, zero-padding is used to keep the dimension of the state vector consistent.

[0068] Step 3.4: Set the current state Input the policy network and output the optimization actions for each candidate. Value, and select The action with the highest value is selected as the current optimization action. .

[0069] Step 3.5: Based on the optimized actions Adjust the parameters of the FMECA risk assessment model, including the failure propagation edge weight, risk warning threshold, severity weight, occurrence weight, detectability weight, and hazard weight.

[0070] Step 3.6: Output the risk level, critical failure modes, critical failure propagation chains, and risk response priorities based on the adjusted FMECA risk assessment model.

[0071] Step 3.7: Calculate immediate rewards based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs. And generate empirical samples:

[0072] Step 3.8: Develop the experience sample. Store the data in the experience replay cache. When the number of experience samples reaches the preset batch size, initiate the online learning process to update the policy network and the target network.

[0073] Step 3.9: When the preset retraining period is reached, the deep Q network is periodically retrained using historical samples in the experience replay buffer, and the FMECA risk assessment model is updated based on the retraining results.

[0074] The present invention will be further described below with reference to the accompanying drawings and examples.

[0075] Please refer to Figure 4This embodiment takes the risk assessment of process equipment in a smart manufacturing production line as an example. The production line includes multiple interconnected production units, each of which is equipped with an equipment status monitoring device, a process parameter acquisition device, a communication status acquisition device, and a risk event recording module.

[0076] In this embodiment, the system first collects historical risk events, equipment operating status, network transmission status, and current production context data for the target process equipment. Historical risk events include fault type, risk level, frequency of occurrence, and handling results; equipment operating status includes equipment health index, key component temperature, vibration data, performance degradation trend, and historical fault records; network transmission status includes data transmission latency, packet loss rate, and bandwidth utilization; and the current production context includes production task type, process parameter settings, ambient temperature and humidity, and equipment load.

[0077] The system constructs the current state vector from the above data:

[0078] in, This is the effective step size for historical data. When historical data is insufficient... During each cycle, the system uses zero-padding to maintain the consistency of the state vector dimension.

[0079] Subsequently, the pre-trained policy network receives the state. Output the Q-value of each candidate action, and select the action with the largest Q-value as the current action to optimize. The actions include increasing or decreasing the weight of failure propagation edges, increasing or decreasing the risk warning threshold, adjusting the severity weight, adjusting the occurrence weight, adjusting the detectability weight, or maintaining the current parameters unchanged. For example, when the system detects an increased correlation between abnormal vibration of a device unit and subsequent temperature rise faults, the agent chooses to increase the edge weight corresponding to that failure propagation path; when the system finds that there are many false alarms for a certain type of risk warning, the agent chooses to appropriately increase the corresponding risk warning threshold.

[0080] After performing optimization actions, the FMECA risk assessment module assesses the current production process based on updated failure propagation edge weights, risk warning thresholds, and weighted parameters such as severity, occurrence, and detectability. It then outputs the risk level, critical failure modes, critical failure propagation chains, and risk handling priorities. For example, the system might output, "The bearing wear risk in equipment unit A is high, which may increase the assembly accuracy of equipment unit B through transmission errors. Priority should be given to maintenance or parameter compensation."

[0081] After a risk warning is generated, the system confirms it offline or verifies it online based on actual production feedback to determine whether the warning accurately reflects an actual risk event, and calculates false alarms, missed alarms, and handling costs. The revenue calculation module calculates the immediate reward using the following formula:

[0082] in, To improve the accuracy of early warning, To improve the early warning recall rate, For false alarm penalties, For the penalty items for failure to report, For risk management costs, to This is the reward weighting coefficient.

[0083] The system will display the current state. Execution of actions Instant rewards and the next state Composition of empirical samples:

[0084] The data is then stored in the experience replay buffer. When the number of experience samples reaches a preset batch size, the system randomly samples a mini-batch of samples from the experience replay buffer and updates the dual-depth Q-network online. Specifically, the policy network parameters are as follows: The target network parameters are The policy network is used to select the optimal action in the next state:

[0085] The target network is used to calculate the target Q value:

[0086] in, This is a long-term reward discount factor.

[0087] The system is based on the loss function:

[0088] Update the network parameters using gradient descent: And update the target network parameters using a soft update method: ,in, For learning rate, denoted as the target network soft update coefficient.

[0089] When the system reaches the preset retraining cycle, it periodically retrains the deep Q-network using historical samples accumulated in the experience replay buffer, and deploys the optimal strategy network parameters to the FMECA risk assessment system. Thus, the system can continuously optimize failure propagation edge weights, risk warning thresholds, and FMECA-related weight parameters based on changes in equipment status, process parameters, and actual risk feedback, achieving collaborative updates between the risk assessment model and the dynamic production environment.

[0090] In summary, the autonomous optimization method for the FMECA risk assessment model provided by this invention transforms the parameter optimization process of the FMECA risk assessment model into a reinforcement learning problem. Through a deep Q-network, it adaptively adjusts the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights, reducing reliance on human experience and improving the consistency between risk assessment results and actual failure results. Through experience replay, target networks, and online learning mechanisms, the system can continuously correct its risk assessment strategy in dynamic operating environments, thereby improving the accuracy of risk warnings, reducing false alarms and false negatives, and enhancing the adaptability of the FMECA risk assessment model to complex operating conditions.

[0091] See Figure 6 The present invention can also provide an autonomous optimization device for the FMECA risk assessment model, such as... Figure 6 As shown, the device may include: FMECA risk assessment model determination unit 601 is used to determine the FMECA risk assessment model. The FMECA risk assessment model includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The risk analysis and assessment modeling unit 602 is used to model the optimization process of the FMECA risk assessment model as a Markov decision process, defining a state space, an action space, and a reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalty, false negative penalty, and risk handling cost. The Deep Q-Network Pre-training Unit 603 is used to construct a dual Deep Q-Network containing a policy network and a target network, and is pre-trained and updated online based on the experience replay buffer; The online optimization unit 604 for risk assessment model is used to acquire real-time operating data, historical failure data, risk event records and environmental context data of the target system to construct the current state, input the current state into the policy network to output optimization actions, adjust the parameters of the FMECA risk assessment model according to the optimization actions, and output the risk level, key failure modes, key failure propagation chains and risk handling priorities based on the adjusted FMECA risk assessment model. The network update unit 605 is used to calculate real-time rewards based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs, and to generate experience samples to be stored in the experience playback cache. When the number of experience samples reaches a preset batch size, the online learning process is initiated to update the policy network and the target network.

[0092] This invention can also provide an autonomous optimization device for the FMECA risk assessment model, the device including a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the above-described autonomous optimization method for the FMECA risk assessment model according to the instructions in the program code.

[0093] like Figure 7 As shown in the figure, an embodiment of the present invention provides an autonomous optimization device for an FMECA risk assessment model, which may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other through the communication bus 13.

[0094] In this embodiment of the invention, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0095] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the FMECA risk assessment model autonomous optimization method.

[0096] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 11 stores at least a program for implementing the following functions: A failure propagation network is defined, which includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The optimization process of the FMECA risk assessment model is modeled as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalties, missed alarm penalties, and risk handling costs. A dual deep Q-network consisting of a policy network and a target network is constructed, and pre-training and online updates are performed based on an experience replay buffer; The system acquires real-time operating data, historical failure data, risk event records, and environmental context data of the target system to construct the current state. The current state is then input into the policy network to output optimization actions. The parameters of the FMECA risk assessment model are adjusted according to the optimization actions. Based on the adjusted FMECA risk assessment model, the risk level, key failure modes, key failure propagation chains, and risk handling priorities are output. Real-time rewards are calculated based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs. Experience samples are generated and stored in the experience playback cache. When the number of experience samples reaches a preset batch size, an online learning process is initiated to update the policy network and the target network.

[0097] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0098] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0099] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the FMECA risk assessment model autonomous optimization device in the embodiments of the present invention. In practical applications, the FMECA risk assessment model autonomous optimization device may include devices that are more advanced than those described above. Figure 7 More or fewer components as shown, or combinations of certain components.

[0100] This invention also provides a computer-readable storage medium for storing program code that executes the steps of the above-described FMECA risk assessment model autonomous optimization method.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0102] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0103] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A self-optimization method for an FMECA risk assessment model, characterized in that, The method includes: A failure propagation network is defined, which includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The optimization process of the FMECA risk assessment model is modeled as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalties, missed alarm penalties, and risk handling costs. A dual deep Q-network consisting of a policy network and a target network is constructed, and pre-training and online updates are performed based on an experience replay buffer; The system acquires real-time operating data, historical failure data, risk event records, and environmental context data of the target system to construct the current state. The current state is then input into the policy network to output optimization actions. The parameters of the FMECA risk assessment model are adjusted according to the optimization actions. Based on the adjusted FMECA risk assessment model, the risk level, key failure modes, key failure propagation chains, and risk handling priorities are output. Real-time rewards are calculated based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs. Experience samples are generated and stored in the experience playback cache. When the number of experience samples reaches a preset batch size, an online learning process is initiated to update the policy network and the target network.

2. The method according to claim 1, characterized in that, The historical risk event records include risk type, risk level, severity, frequency of occurrence, and handling results; the equipment status parameters include health index, performance degradation trend, fault history, and status of key components; the communication status includes latency, packet loss rate, bandwidth utilization, and communication anomaly records; the environmental context includes production task type, process parameters, environmental conditions, and operating load.

3. The method according to claim 1, characterized in that, The adjustment strategies include increasing the weight of the failure propagation edge, decreasing the weight of the failure propagation edge, increasing the risk warning threshold, decreasing the risk warning threshold, adjusting the severity weight, adjusting the occurrence weight, adjusting the detectability weight, and maintaining the current parameters unchanged.

4. The method according to claim 1, characterized in that, Each adjustment strategy in the action space corresponds to the discrete action output of the strategy network, and the execution of the adjustment strategy is constrained by the mathematical relationship of Risk Priority Number RPN = Severity × Occurrence × Detectability. When the adjustment strategy changes the severity weight or occurrence weight, the Risk Priority Number is updated synchronously.

5. The method according to claim 1, characterized in that, The instant reward is calculated using the following formula: In the formula: Indicates the first The accuracy rate of risk warnings during the second operation; Recall rate, indicating the risk warning; Indicates the penalty for false alarms; This indicates a penalty for failure to report. Indicates the cost of risk management; These are all reward weighting coefficients, used to balance the importance of different evaluation indicators.

6. The method according to claim 5, characterized in that, The reward weight coefficient The recall rate weight is dynamically adjusted based on the risk level, with high-risk failure modes corresponding to higher recall rates. The recall weight is greater than that of low-risk failure modes.

7. The method according to claim 1, characterized in that, The pre-training and online updating based on the experience replay cache includes: Randomly sample a small batch of samples from the experience replay buffer; calculate the current state action value through the policy network, and select the optimal action for the next state through the policy network; calculate the target Q value through the target network; update the policy network parameters according to the loss function, and update the target network parameters in a soft update manner.

8. The method according to claim 1, characterized in that, The process for initiating online learning includes: When the number of experience samples reaches the preset batch size, a small batch of samples is randomly sampled from the experience replay buffer to update the policy network and the target network; When the preset retraining period is reached, the deep Q network is periodically retrained using historical samples in the experience replay buffer, and the FMECA risk assessment model is updated based on the retraining results.

9. The method according to claim 1, characterized in that, The current state of construction includes: Historical risk event records, device status parameters, and communication status are arranged according to the effective step size of historical data and combined with the current environmental context to form a state vector; when the length of historical data is less than the effective step size of historical data, zero padding is used to keep the dimension of the state vector consistent.

10. An autonomous optimization device for an FMECA risk assessment model, characterized in that, The apparatus for performing the autonomous optimization method of the FMECA risk assessment model according to any one of claims 1 to 9, the apparatus comprising: The FMECA risk assessment model determination unit is used to determine the FMECA risk assessment model. The FMECA risk assessment model includes a failure propagation network. The failure propagation network uses equipment units, key components or failure modes as nodes and failure impact relationships as propagation edges. Each propagation edge is set with an adjustable failure propagation edge weight and a risk warning threshold. The risk analysis and assessment modeling unit is used to model the optimization process of the FMECA risk assessment model as a Markov decision process, defining a state space, action space, and reward function. The state space includes historical risk event records, equipment status parameters, communication status, and environmental context. The action space includes adjustment strategies for the failure propagation edge weights, risk warning thresholds, severity weights, occurrence weights, detectability weights, and hazard weights. The reward function is calculated based on risk warning accuracy, recall, false alarm penalty, false negative penalty, and risk handling cost. The Deep Q-Network Pre-training Unit is used to construct a dual Deep Q-Network that includes a policy network and a target network, and is pre-trained and updated online based on the experience replay buffer; The risk assessment model online optimization unit is used to acquire real-time operating data, historical failure data, risk event records and environmental context data of the target system to construct the current state, input the current state into the policy network to output optimization actions, adjust the parameters of the FMECA risk assessment model according to the optimization actions, and output the risk level, key failure modes, key failure propagation chains and risk handling priorities based on the adjusted FMECA risk assessment model. The network update unit is used to calculate real-time rewards based on actual risk events, early warning hit rates, false alarms, missed alarms, and handling costs, and to generate experience samples to be stored in the experience replay cache. When the number of experience samples reaches a preset batch size, the online learning process is initiated to update the policy network and the target network.

Citation Information

Patent Citations

  • Single-particle fault occurrence probability calculation method based on extended FMECA and fault propagation graph

    CN112782549A