A tank defect fault propagation space-time prediction method and system based on physical constraint reinforcement learning

Abstract

The present application relates to oil and gas storage and transportation equipment safety monitoring and fault diagnosis technical field, specifically a kind of tank defect fault propagation space-time prediction method and system based on physical constraint reinforcement learning.The present application collects storage tank acoustic emission, material mechanics and working condition multi-source monitoring data, constructs multi-type defect state space by fuzzy clustering;Rely on the physical information neural network of embedded mechanics differential equation regular term to represent fault state transition kernel, combined with Metropolis-Hastings algorithm to realize state iteration update;Joint action penalty near-end strategy optimization reinforcement learning agent multi-objective collaborative training, rely on physical model output state prior, agent optimization operation and maintenance scheme, form whole-process management and control system, overcome the defects such as mechanism loss, low prediction accuracy of traditional method.

Description

Technical Field

[0001] This invention relates to the field of safety monitoring and fault diagnosis technology for oil and gas storage and transportation equipment, specifically a spatiotemporal prediction method and system for tank defect propagation based on physical constraint reinforcement learning. Background Technology

[0002] As core pressure-bearing equipment in the oil and gas gathering, transportation, and storage process, oilfield storage tanks operate under corrosive media, alternating internal pressure, complex temperature variations, and external stress coupling conditions for extended periods. They are prone to various levels of damage, including elastic deformation, plastic deformation, wall corrosion, stress cracking, and even media leakage. A failure in these tanks can easily trigger significant safety risks such as flammability, explosions, environmental pollution, and personal injury. Therefore, achieving accurate prediction of the spatiotemporal evolution of damage throughout the entire lifecycle of storage tanks, dynamic simulation of fault states, and intelligent operation and maintenance decision-making is crucial for ensuring the safety of oilfield storage tank clusters, reducing operation and maintenance costs, and managing failure risks.

[0003] Currently, the operation and maintenance management model for oilfield storage tanks in China still relies mainly on fixed-cycle manual inspections, threshold alarms, and passive post-incident repairs. The supporting fault monitoring and diagnosis technologies suffer from several inherent technical shortcomings. Existing monitoring methods largely depend on acoustic emission sensor arrays to collect sound source signals of tank damage, combined with AOA and DOA array localization algorithms, and supplemented by single deep learning models such as CNN, LSTM, and Transformer to complete fault location and damage prediction. However, the acoustic emission signals corresponding to tank defects are weak, non-stationary, broadband signals in high-noise environments. Conventional noise reduction and feature extraction algorithms have poor anti-interference capabilities, making them prone to missed detections and false detections. Furthermore, traditional data-driven deep learning models rely solely on historical monitoring data for training, failing to incorporate prior physical mechanisms related to tank material mechanics and damage evolution. The lack of physical constraints in model training prevents the depiction of the nonlinear spatiotemporal evolution of defects from elastic deformation to plastic cracking, resulting in weak extrapolation and generalization capabilities that are difficult to adapt to complex field conditions.

[0004] Meanwhile, the current tank operation and maintenance decision-making mechanism presents significant contradictions: on the one hand, excessively increasing inspection frequency and premature shutdown for maintenance can reduce safety risks, but it also significantly increases operation and maintenance costs and affects the continuity of oil and gas production; on the other hand, extensive periodic maintenance and delayed fault handling can reduce operation and maintenance costs, but they cannot predict the propagation and evolution of hidden defects in advance, resulting in a persistently high risk of equipment failure. Conventional reinforcement learning decision-making models rely solely on data reward functions to update strategies, without considering the physical boundary constraints of fault propagation. Their decision-making actions lack rationality and interpretability, failing to achieve a dynamic balance among equipment lifespan, economic operation and maintenance costs, and safety risks, making it difficult to output smooth and implementable operation and maintenance strategies adapted to industrial sites. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a spatiotemporal prediction method and system for tank defect propagation based on physical constraint reinforcement learning. It integrates acoustic emission array signal denoising, cloud-edge-device collaborative data transmission, physical information neural networks, Markov state transitions, and multi-objective reinforcement learning decision-making techniques. Using partial differential equations of mechanics of materials as physical canonical constraints, and combining fuzzy clustering state space partitioning with the Metropolis-Hastings state update mechanism, it constructs a comprehensive management and control system integrating signal preprocessing, spatiotemporal fault deduction, risk quantification assessment, and intelligent operation and maintenance decision-making. This fundamentally solves the industry problems of missing physical mechanisms, insufficient prediction accuracy, unbalanced decision schemes, and poor anti-interference capabilities inherent in traditional technologies.

[0006] To achieve the above objectives, the technical solution specifically adopted by the present invention is as follows: A spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning includes the following steps: Step 1: Data Acquisition and Preprocessing Real-time monitoring data is acquired through a sensor array deployed on the surface of the oil storage tank. The sensor array includes acoustic emission sensors and ultrasonic thickness gauges. The collected data includes, but is not limited to: the residual wall thickness h at different axial heights z of the tank, the internal pressure P under current operating conditions, the service time t, and acoustic emission timing signals. The raw data undergoes noise reduction and signal conditioning via a field-programmable gate array (FPGA) parallel conditioning circuit before being sent to an embedded processor (preferably RK3588) for data normalization and standardization, serving as input for subsequent physical inversion and reinforcement learning state space processing.

[0007] Step 2: Construct the defect state space based on the fuzzy clustering coarse-scale method Fuzzy clustering with both coarse and fine-scale clustering methods was used to perform cluster analysis on the defect and failure states of oilfield storage tanks, establishing a state space S containing multiple defect states. These defect states include at least: corrosion, fatigue strength degradation, shear strain stress accumulation, plastic deformation, crack initiation and propagation, and leakage.

[0008] Specifically, the defect states are first initially classified at a coarse scale (e.g., into macroscopic levels such as mild, moderate, and severe), and then refined at a fine scale based on different time scales and spatial locations. A configurable trade-off exists between the degree of state refinement and the complexity of the differential equations describing the defect states: the more refined the states, the more accurate the description of fault state changes, but the complexity of establishing the corresponding physical process differential equations increases significantly. Fuzzy membership functions are used to quantitatively characterize the degree to which each monitoring sample belongs to each defect state.

[0009] Step 3: Construct a Physical Information Neural Network (PINN) to characterize the fault state transition kernel. A multilayer fully connected neural network is constructed to approximate the unknown physical field of defect propagation. The input coordinates of the neural network are the spatiotemporal combination z = (x, t), and the output is the physical field approximation u_θ(z), where θ is the network parameter.

[0010] The loss function of the physical information neural network incorporates finite element differential equations describing the root causes of defect propagation as physical regularization terms, specifically including the circumferential stress equation (Lame formula) and empirical corrosion kinetic equation from mechanics of materials.

[0011] The specific construction method of PINN is as follows: Automatic differentiation calculation of partial derivatives: Utilizing the automatic differentiation mechanism of frameworks such as PyTorch, this method calculates the first-order partial derivatives of the neural network output with respect to the input coordinates. and the second-order Laplace operator Thus, the physical residuals of the partial differential equation are constructed:

[0012] in This is the operator form of a partial differential equation.

[0013] Collocational sampling: Generate a set of collocational points within the solution domain Ω. It is used to calculate physical residuals. The collocation generation method supports uniform random sampling, Latin hypercube sampling, or Sobol low-difference sequence sampling.

[0014] Total loss function: The total loss function is constructed by weighting and summing the mean square error of the physical residuals, the boundary condition error, the initial condition error, and the optional supervision data error.

[0015] in, ; For boundary / initial condition loss, To monitor the fitting error of the data points; , , These are the corresponding weighting coefficients.

[0016] Adaptive loss weight mechanism: To avoid one type of loss term dominating the training process, the weight coefficients are dynamically adjusted based on the gradient norm of each loss term with respect to the model parameters.

[0017] in For the first The gradient norm of each loss term, To prevent small amounts from being divided by zero, This is the smoothing coefficient for the exponential moving average.

[0018] High-frequency feature enhancement (optional): Supports Fourier feature embedding, mapping input coordinates to high-dimensional features in sine and cosine forms. B follows a normal distribution This is to improve the ability of neural networks to express high-frequency physical fields.

[0019] Built-in PDE Residual Module: The neural network includes four types of directly callable partial differential equation residual interfaces, including the Burgers equation, heat conduction equation, Poisson equation, and wave equation. Among them, the residual expression for the heat conduction equation, used to adapt to the simulation of corrosion defect evolution in storage tanks, is as follows:

[0020] in Where is the thermal diffusivity, This is an adaptive heat source term.

[0021] Step 4: Bidirectional transfer and update of missing states based on Metropolis-Hastings and fuzzy membership. By combining the Metropolis-Hastings method with fuzzy membership functions and leveraging the bidirectional transfer and update kernel function for oilfield storage tank defect states, a spatiotemporal scheduling process for fault states is constructed.

[0022] definition: Current defect status ; Action sets of various participants under multiple scenarios This represents a set of interfering noise and specific scene conditions; Suggested distribution function This indicates that the current defective state will transition to a potential candidate state in the next time step. The probability density; State optimization accepts boundary functions , satisfying 0 ≤ α ≤ 1.

[0023] Given the current defect state and setting multiple scenarios The state transitions from the previous moment to a potential candidate defect state. The probability density function (i.e., the state transition probability) is expressed as:

[0024] This function represents the reachability between various defective and faulty states in a spatiotemporal state deduction model related to a specific scenario.

[0025] Furthermore, in order to express the propagation and transition process of each fault state using a state-space-time deductive kernel model, the state transition probability is decomposed into the product of the proposal distribution function and the state optimization acceptance boundary function:

[0026] This formula is an explicit expression of the kernel function for the state transition of corrosion defects in oilfield storage tanks.

[0027] To decide whether to accept a potential candidate defect state Introducing an indicator function to represent the state at the next moment:

[0028] in The comparison value is preset or randomly generated (usually a uniformly distributed random number). This formula represents: with probability... accept As the state at the next moment, with probability Refuse to transfer and maintain the current state This mechanism enables Markov chains to adaptively update their states during nonlinear defect evolution.

[0029] Step 5: Construct a reinforcement learning agent based on proximal policy optimization (PPO) Construct a reinforcement learning environment that conforms to the physical logic of oil storage tank failure.

[0030] State space definition: The state vector includes the current maximum damage depth, minimum residual wall thickness, normalized circumferential stress, and the current physical phase (elastic stage / plastic stage). The system monitors the tank phase in real time according to mechanical criteria. When the stress exceeds the yield limit, the environment automatically switches from the "elastic stage" to the "plastic stage," at which point the damage growth slope changes nonlinearly.

[0031] Action space definition: The agent selects the optimal action from five operational actions, namely: Maintain the status quo (Do Nothing); Reduce pressure operation; Apply corrosion inhibitor. Patch Repair; Emergency Shutdown.

[0032] Reward function design: An action switching penalty term is introduced into the reward function. When the decision action changes in consecutive time steps, a negative reward value is applied to the reward function to avoid frequent start-stop of actuators such as pumps, valves, and spraying equipment, ensuring the smooth executability of decision commands in the industrial field.

[0033] Value Function and Bellman Equation: Defining the State-Value Function The quantitative relationship between the two is as follows: ; The action value function satisfies the Bellman equation: ; in As a discount factor, Let be the revenue function, representing the current state of damage to oilfield storage tanks. and under multiple scenarios The state transitions to a potential candidate optimization state. The expected return obtained at that time.

[0034] Training parameters: The total number of iterations for agent training is set to 1.25 × 10^6, and convergence of the average reward value is used as the training termination condition. Gradient pruning and cosine annealing learning rate scheduling are used during training to improve stability.

[0035] Step 6: Multi-objective collaborative optimization and Pareto front screening Within a multi-objective optimization framework that maximizes equipment lifespan, minimizes maintenance costs, and controls safety risks, the physical information neural network and the reinforcement learning agent are jointly trained.

[0036] Inverting the defect depth evolution field using PINN (axial height) Over time The corrosion depth heatmap provides a prior probability distribution for state transitions for reinforcement learning agents.

[0037] The reinforcement learning agent selects the optimal operation and maintenance action based on the current state and the prior probability distribution, forming a closed-loop decision.

[0038] After offline training, a feasible domain for operational decisions is generated through multiple rounds of evaluation, resulting in a set of Pareto non-dominated solutions. This set covers multiple operational strategy choices, ranging from an "economy-first mode" to an "extreme security-first mode." Operations managers can flexibly select the optimal operational route from the set of non-dominated solutions based on budget or security rating requirements.

[0039] Step 7: Real-time Deployment and KPI Monitoring The trained strategy model is lightweight and deployed on the tank monitoring terminal (RK3588 embedded system). The system outputs the full life cycle failure probability P(fail) in real time and keeps it below a preset warning threshold (e.g., 0.03). Simultaneously, it displays safety index, average maintenance cost, and other indicators through an industrial KPI dashboard.

[0040] As a further optimization, the system also includes a fault tracing auxiliary module. This module integrates an LSTM+CNN hybrid machine learning model, takes the AE acoustic emission signal as input, and uses the PINN physical model to construct the posterior probability density distribution function of network parameters and physical parameters. It analyzes the root causes of defect propagation and the statistical dependencies between parameters to achieve accurate fault location.

[0041] This invention also provides a spatiotemporal prediction system for tank defect propagation based on physical constraint reinforcement learning, used in the above-mentioned method, comprising: Data acquisition module: used to acquire multi-source monitoring data of oilfield storage tanks. The data is acquired through an acoustic emission sensor array, conditioning circuit and field programmable gate array parallel data acquisition circuit arranged at key locations of the storage tank, and sent to the embedded processor. State space construction module: used to perform cluster analysis on the defect and failure states of oilfield storage tanks using fuzzy clustering coarse and fine scale clustering methods, and to establish a state space containing multiple defect states; Physical Information Neural Network Module: Used to construct and store physical information neural networks that characterize the fault state transition kernel. Its loss function incorporates finite element differential equations describing the root causes of defective fault propagation as physical regularization terms. State transition and update module: Based on the Metropolis-Hastings method and fuzzy membership function, combined with the state transition kernel represented by the physical information neural network, it performs bidirectional transfer and update of the oilfield storage tank defect state; Reinforcement learning decision-making module: Constructs a reinforcement learning agent based on the proximal policy optimization algorithm, and perceives the optimal operation and maintenance action based on the current state; the reward function of this module includes an action switching penalty term; Multi-objective collaborative optimization module: Under the multi-objective optimization framework of maximizing equipment lifespan, minimizing operation and maintenance costs, and controlling safety risks, it jointly trains the physical information neural network and the reinforcement learning agent to generate the optimal operation and maintenance strategy.

[0042] This invention has the following characteristics and beneficial effects: 1) Significantly Extends Tank Equipment Lifespan: Through the coordinated control of Physical Information Neural Network (PINN) and Proximal Policy Optimization (PPO), the critical inflection point of tank damage transitioning from elastic deformation to plastic deformation can be accurately identified, and the control gain can be dynamically adjusted. Experimental data shows that after implementing this method, the average safety index of the tank reaches 82.4, the damage evolution process is effectively suppressed, and the entire equipment lifespan is significantly extended.

[0043] 2) Provides a complete set of multi-objective operation and maintenance decisions: Under the three conflicting objectives of "maximizing equipment lifespan", "minimizing operation and maintenance costs" and "controlling safety risks", the system obtains four or more non-dominated solutions through multi-objective Pareto optimization screening, covering a variety of operation and maintenance strategies from "economic priority mode" to "ultimate safety priority mode", providing on-site operation and maintenance management personnel with quantitative and flexible decision-making solutions.

[0044] 3) Achieving full lifecycle risk transparency and dynamic control: The system outputs the tank failure probability P(fail) in real time and dynamically controls it to around 0.03, far below the industry warning threshold. Through the full lifecycle failure probability curve and risk scoring curve, it achieves risk visualization, proactive early warning, and closed-loop management.

[0045] 4) Smooth and executable decision instructions: By introducing an action switching penalty term into the reinforcement learning reward function, the action oscillation between adjacent decision steps is effectively suppressed, ensuring smooth and continuous instructions for industrial equipment such as pump and valve start-up and shutdown, corrosion inhibitor spraying, and pressure reduction operation. This avoids hardware wear and tear caused by frequent actuator start-up and shutdown, and significantly improves the executability of the decision scheme in actual industrial control systems.

[0046] 5) Integrating physical constraints to synergistically improve prediction accuracy and reliability: The circumferential stress equation (Lame formula) and corrosion kinetic equation from mechanics of materials are embedded as physical regularization terms into the neural network loss function. Automatic differentiation technology is used to directly solve the residuals of the partial differential equations, avoiding the shortcomings of purely data-driven models that lack mechanical priors. This mechanism can still accurately invert the defect evolution field even with small sample anomalous signals, resolving the contradiction between prediction accuracy and reliability inherent in traditional deep models.

[0047] 6) Adaptive state transition modeling with strong robustness: The Metropolis-Hastings method and fuzzy membership function are introduced to decompose the defective state transition kernel into the product of the proposal distribution function and the state optimization acceptance boundary function. Adaptive acceptance / rejection of states is achieved through the indicator function. This mechanism enables the Markov chain to dynamically respond to the nonlinear evolution of defects (such as elastic to plastic abrupt change), maintaining high-precision spatiotemporal deduction capabilities in complex noisy environments and variable scenarios.

[0048] 7) Strong interpretability of root causes of failures: Through feature sensitivity analysis and joint parameter posterior probability density distribution modeling of LSTM+CNN and PINN, the driving role of key features such as residual wall thickness hmin and circumferential stress on operation and maintenance decisions is quantitatively revealed, and the basis for interpretability of decisions is output, which improves the trust of on-site personnel in the output results of the intelligent system and the reliability of system implementation. Attached Figure Description

[0049] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic diagram of an acoustic emission (AE) oilfield storage tank monitoring system; Figure 2 Clustering for fault propagation in damaged oilfield storage tanks; Figure 3 A two-dimensional spatiotemporal state deductive model and a diagram of the defective fault propagation mode; Figure 4 Schematic diagram of PINN; Figure 5 The feature importance analysis plot reveals the contribution of input variables to policy generation.

[0050] Figure 6 This is a graph showing the importance of strategy interpretability features.

[0051] Figure 7 Training curve of PPO throughout the entire life cycle of oil storage tanks.

[0052] Figure 8 This is a thermogram showing the spatiotemporal evolution of the corrosion damage field.

[0053] Figure 9 This is a statistical chart showing the distribution of decision-making actions.

[0054] Figure 10 This is a multi-objective Pareto front plot.

[0055] Figure 11 This is a risk assessment and early warning curve for the service life period.

[0056] Figure 12 This is a failure risk scoring chart for the entire life cycle. Detailed Implementation

[0057] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0058] Example 1: Spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning This embodiment provides a spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning, and its overall architecture is based on a collaborative "cloud-network-edge-device" system. For example... Figure 1As shown, weak signals under non-stationary strong noise are first acquired by an acoustic emission (AE) sensor array deployed on the surface of oilfield storage tanks. Noise reduction and anomaly detection are then achieved through parallel processing by a front-end conditioning circuit and FPGA. The anomaly signals are then transmitted to a cloud server. In the cloud, federated learning and large models are used for fine-tuning and inference of defect location.

[0059] Specifically, this embodiment includes the following steps performed in sequence: Step 1: Data Acquisition and Preprocessing Real-time monitoring data is acquired through a sensor array (including AE acoustic emission sensors and ultrasonic thickness gauges) deployed at key locations on the oil storage tank. The collected data includes: residual wall thickness h at different axial heights z, internal pressure P under current operating conditions, service time t, and acoustic emission timing signals. The raw signals are denoised, amplified, and filtered by an FPGA parallel conditioning circuit before being sent to an embedded processor (using an RK3588 chip in this embodiment) for data normalization and standardization, serving as input for subsequent physical inversion and reinforcement learning state space processing.

[0060] Step 2: Constructing the defect state space based on the fuzzy clustering coarse-scale method Fuzzy clustering with both coarse and fine-scale clustering methods was used to perform cluster analysis on the defect and failure states of oilfield storage tanks, establishing a state space S containing multiple defect states. These defect states include at least: corrosion, fatigue strength degradation, shear strain stress accumulation, plastic deformation, crack initiation and propagation, and leakage.

[0061] Specifically, the damage is first initially classified at a coarse scale (e.g., mild, moderate, severe), and then refined at a fine scale based on different time scales (e.g., day, week, month) and spatial locations (different axial heights z). Figure 2 As shown, the vertical axis represents the seven defect states of the oilfield storage tank, and the horizontal axis represents the transfer and update process between defect states within a cycle. A configurable trade-off exists between the degree of state refinement and the complexity of the differential equations describing the defect states: the more refined the states, the more accurately the description of the fault state changes, but the complexity of establishing the corresponding physical process differential equations increases significantly. The degree to which each monitoring sample belongs to each defect state is quantitatively characterized using a fuzzy membership function.

[0062] Step 3: Construct a Physical Information Neural Network (PINN) to characterize the fault state transition kernel. The core of this step is to use PINN to represent the transition core of the defective fault state. The specific construction process is as follows: (1) Approximation of physical field in neural network: Construct a multi-layer fully connected neural network with input coordinates as a spatiotemporal combination. (in Represents spatial coordinates, (Represents time), the output is an approximation of the physical field. These are the network parameters. The neural network architecture supports tanh, ReLU, GELU, Swish, Sigmoid, ELU, Softplus, or SIREN sinusoidal activation functions, and can optionally include residual connections and Fourier feature embeddings.

[0063] (2) Automatic differentiation and differential operators: Using the automatic differentiation mechanism of the PyTorch framework, the first-order partial derivatives of the neural network output with respect to the input coordinates are calculated. And the second-order Laplace operator Δu_θ. From this, the physical residual of the partial differential equation is constructed: ; (3) Embedding of physical criteria: The circumferential stress equation (Lame formula) and empirical corrosion kinetic equation from mechanics of materials are added to the loss function as physical regularization terms. For example, for the evolution of corrosion defects, a heat conduction type equation is used: ; in Where is the thermal diffusivity, This is an adaptive heat source term. The code includes four typical PDE residual examples: the Burgers equation, the heat conduction equation, the Poisson equation, and the wave equation, and retains a general residual interface.

[0064] (4) Collocation sampling: Generate a set of collocation points in the solution domain Ω. It is used to calculate physical residuals. The collocation generation method supports uniform random sampling, Latin hypercube sampling, or Sobol low-difference sequence sampling.

[0065] (5) Total loss function: The total loss function is formed by weighted summation of the mean square error of the physical residuals, the boundary condition error, the initial condition error, and the optional supervision data error.

[0066] in, ; For boundary / initial condition loss, To monitor the fitting error of the data points; , , These are the corresponding weighting coefficients.

[0067] (6) Adaptive loss weight mechanism: To avoid one type of loss term dominating the training process, the weight coefficients are dynamically adjusted based on the gradient norm of each loss term with respect to the model parameters.

[0068] in For the first The gradient norm of each loss term, To prevent small amounts from being divided by zero, This is the smoothing coefficient for the exponential moving average.

[0069] (7) High-frequency feature enhancement (optional): Supports Fourier feature embedding, mapping the input coordinates to γ(z) = [sin(zB), cos(zB)], B ~ N(0,σ²), to improve the neural network's ability to express high-frequency physical fields.

[0070] (8) Training optimization: The Adam optimizer is used in conjunction with cosine annealing learning rate scheduling and gradient pruning to minimize the total loss function L_total, so that the network output satisfies both the physical equation and the boundary / initial condition constraints.

[0071] Step 4: Bidirectional transfer and update of missing states based on Metropolis-Hastings and fuzzy membership. By combining the Metropolis-Hastings method with fuzzy membership functions and utilizing the bidirectional transfer and update kernel function of oilfield storage tank defect states, a spatiotemporal grading process of fault states is constructed.

[0072] definition: Current defect status ; Action sets of various participants under multiple scenarios This represents a set of interfering noise and specific scene conditions; Suggested distribution function This indicates that the current defective state will transition to a potential candidate state in the next time step. The probability density; State optimization accepts boundary functions , satisfying 0 ≤ α ≤ 1.

[0073] Given the current defect state and setting multiple scenarios The state transitions from the previous moment to a potential candidate defect state. The probability density function (i.e., the state transition probability) is expressed as:

[0074] This function represents the reachability between various defective and faulty states in a spatiotemporal state deduction model related to a specific scenario.

[0075] Furthermore, in order to express the propagation and transition process of each fault state using a state-space-time deductive kernel model, the state transition probability is decomposed into the product of the proposal distribution function and the state optimization acceptance boundary function:

[0076] This formula is an explicit expression of the kernel function for the state transition of corrosion defects in oilfield storage tanks.

[0077] Furthermore, the state transition probability is decomposed into the product of the proposal distribution function and the state optimization acceptance boundary function: This formula is the explicit expression of the state transition kernel function for corrosion and defect faults in oilfield storage tanks.

[0078] To decide whether to accept a potential candidate defect state Introducing an indicator function to represent the state at the next moment:

[0079] in The comparison value is preset or randomly generated (usually a uniformly distributed random number). This formula represents: with probability... accept As the state at the next moment, with probability Refuse to transfer and maintain the current state This mechanism enables Markov chains to adaptively update their states during nonlinear defect evolution.

[0080] Step 5: Construct a reinforcement learning agent based on Proximal Policy Optimization (PPO). Construct a reinforcement learning environment that conforms to the physical logic of oil storage tank failure.

[0081] State space definition: The state vector includes the current maximum damage depth, minimum residual wall thickness, normalized circumferential stress, and the current physical phase (elastic stage / plastic stage). The system monitors the tank phase in real time based on mechanical criteria. When the stress exceeds the yield limit, the environment automatically switches from the "elastic stage" to the "plastic stage," at which point the damage growth slope changes nonlinearly (see appendix). Figure 2 ).

[0082] Action space definition: The agent selects the optimal option from five maintenance actions: maintain the status quo, reduce pressure, apply corrosion inhibitor, patch repair, and emergency shutdown.

[0083] Reward function design: An action switching penalty term is introduced into the reward function. When the decision action changes in consecutive time steps, a negative reward value is applied to avoid frequent start-stop of actuators such as pumps, valves, and spraying equipment, ensuring the smooth executability of decision commands in the industrial field.

[0084] Value Function and Bellman Equation: Defining the State-Value Function The quantitative relationship between the two is as follows: ; The action value function satisfies the Bellman equation: ; in As a discount factor, Let be the revenue function, representing the current state of damage to oilfield storage tanks. and under multiple scenarios The state transitions to a potential candidate optimization state. The expected return obtained at that time.

[0085] Training parameters: The total number of iterations for agent training is set to 1.25 × 10^6, and convergence of the average reward value is used as the training termination condition (see appendix). Figure 1 Gradient pruning and cosine annealing learning rate scheduling are used during training to improve stability.

[0086] Step Six: Multi-objective Collaborative Optimization and Pareto Front Screening Within a multi-objective optimization framework that maximizes equipment lifespan, minimizes maintenance costs, and controls safety risks, a physical information neural network and a reinforcement learning agent are jointly trained.

[0087] Inverting the defect depth evolution field using PINN (axial height) Over time The corrosion depth heatmap provides a prior probability distribution for state transitions for reinforcement learning agents.

[0088] The reinforcement learning agent selects the optimal operation and maintenance action based on the current state and the prior probability distribution, forming a closed-loop decision.

[0089] After offline training is completed, a feasible domain for operational decisions is generated through multiple rounds of evaluation, resulting in a set of Pareto non-dominated solutions (see Appendix). Figure 5 This solution set covers multiple operation and maintenance strategy options, ranging from "economy-first mode" to "ultimate security-first mode." Operation and maintenance managers can flexibly select the optimal operation and maintenance route from the non-inferior solution set based on budget or security rating requirements.

[0090] Step 7: Real-time Deployment and KPI Monitoring The trained strategy model is lightweight and deployed on the tank monitoring terminal (RK3588 embedded system). The system outputs the full life cycle failure probability P(fail) in real time, keeping it below a preset warning threshold (e.g., 0.03) (see Appendix). Figure 6 Meanwhile, the industrial KPI dashboard displays indicators such as safety index and average operation and maintenance cost (see appendix). Figure 7 Experimental data shows that after implementing this method, the average safety index can be stabilized at over 82.4, and the average operation and maintenance cost can be controlled at around 1.157 million yuan.

[0091] As a further optimization, the system also includes a fault tracing auxiliary module. This module integrates an LSTM+CNN hybrid machine learning model, using the AE acoustic emission signal as input, and combines it with the PINN physical model to construct the posterior probability density distribution function of network parameters and physical parameters. It analyzes the root causes of defect propagation and the statistical dependencies between parameters to achieve precise fault location (see appendix). Figure 8 To be continued Figure 12 ).

[0092] Example 2: Spatiotemporal prediction system for tank defect propagation based on physical constraint reinforcement learning This embodiment provides a system for performing the above method, the system comprising: Data acquisition and preprocessing module: used to collect multi-dimensional operational monitoring data of storage tanks and complete noise reduction and standardization preprocessing.

[0093] Physical perception and prediction module: It has a built-in optimized PINN model, which is used to combine the physical constraint equations of the storage tank to invert the spatiotemporal field of global corrosion damage and quantify the propagation law of defects and faults.

[0094] Phase Recognition and Reinforcement Learning Environment Module: Used to build a simulation environment for tank fault evolution, divide elastic / plastic physical phases, and define standardized state space and action space.

[0095] Smooth Decision Evaluation Module: Based on the PPO algorithm, an intelligent decision-making body is constructed. Relying on a reward function with action penalty terms, smooth and implementable operation and maintenance actions are output.

[0096] Multi-objective optimization screening module: used to solve multi-objective operation and maintenance optimization models, generate Pareto non-dominated solution sets, and adapt to operation and maintenance needs with different budgets and security levels.

[0097] Terminal monitoring and control module: used to deploy models, display failure probability, security index and operation and maintenance cost KPI indicators in real time, and realize full life cycle failure management.

[0098] The specific functions and interaction processes of each module have been described in detail in Example 1, and will not be repeated here.

[0099] Example 3: Computer-readable storage medium and electronic device This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in Embodiment 1 above.

[0100] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of any one of the methods described in Embodiment 1 above.

[0101] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning, characterized in that, Includes the following steps: S1: Acquire multi-source monitoring data of oilfield storage tanks, including time-series signals, material mechanical parameters, and environmental and stress condition parameters collected by an acoustic emission sensor array; S2: Fuzzy clustering with coarse and fine scales is used to perform cluster analysis on the defect and failure states of oilfield storage tanks, and a state space containing multiple defect states is established; the defect states include at least corrosion, fatigue strength degradation, shear strain stress accumulation, plastic deformation, crack initiation and propagation, and leakage; S3: Construct a physical information neural network to characterize the fault state transition kernel; the loss function of the physical information neural network embeds a finite element differential equation describing the root cause of the defect propagation as a physical regularization term, the finite element differential equation including the circumferential stress equation and the defect evolution equation in mechanics of materials; wherein, the physical information neural network calculates the residual of the partial differential equation based on the automatic differentiation mechanism, and constructs the residual as a loss term, which is weighted and combined with other constraint loss terms to form the total loss function, and the residual is calculated by generating collocation points in the solution region; S4: Based on the Metropolis-Hastings method and fuzzy membership function, combined with the state transition kernel represented by the physical information neural network, a bidirectional transfer and update mechanism for the defective state of oilfield storage tanks is constructed. This mechanism generates potential candidate defective states through a proposal distribution function and determines whether to accept the candidate defective states through a state optimization acceptance boundary function, thereby simulating the spatiotemporal scheduling process of the fault state. S5: Construct a reinforcement learning agent based on the near-end policy optimization algorithm; the state perception input of the agent includes the residual wall thickness and stress level, its action space is a discrete action set containing multiple operation and maintenance actions, and its reward function introduces an action switching penalty term to constrain the smoothness of decision instructions. S6: The physical information neural network is combined with the reinforcement learning agent and trained under a multi-objective optimization framework that maximizes equipment lifespan, minimizes operation and maintenance costs, and controls safety risks; the physical information neural network is used to invert the deep evolution field of the defect and provide the reinforcement learning agent with a prior probability distribution of state transitions; the reinforcement learning agent selects the optimal operation and maintenance action based on the current state and the prior probability distribution to achieve spatiotemporal prediction and decision-making on the propagation of tank defect faults.

2. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S2, the specific process of establishing the state space using the fuzzy clustering coarse-scale clustering method includes: initially dividing the state of damage to oilfield storage tanks according to a coarse scale, and then refining it according to different time scales and spatial locations; a configurable trade-off is formed between the degree of state refinement and the complexity of the differential equations describing the state of damage.

3. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, The state optimization acceptance boundary function in step S4 satisfy The bidirectional transfer and update mechanism for the fault status of the oilfield storage tank is expressed as follows: in, s This is the current defective state. s c For potential candidate defect states, Δ is a preset or randomly generated comparison value; with probability... α ( s → s c )accept s c As the state at the next moment, with a probability of 1 α ( s → s c Refuse to transfer and maintain the current state. s .

4. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S5: the action space includes five action dimensions: "maintain the status quo", "decompression operation", "spray corrosion inhibitor", "patch repair" and "emergency shutdown"; the action switching penalty is set to apply a negative reward value to the reward function when the decision action changes in consecutive moments, so as to avoid frequent start and stop of the actuator.

5. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, The goal of joint training in step S6 is to seek a set of Pareto non-dominated solutions to a multi-objective optimization problem; this set of solutions covers multiple operation and maintenance strategy choices from an economy-first mode to an extreme safety-first mode; the goal of controlling safety risks is to keep the failure probability below a preset warning threshold in real time throughout the entire life cycle.

6. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S3, when constructing the physical information neural network, the first-order partial derivatives of the neural network output with respect to the input coordinates and the second-order Laplace operator are calculated through an automatic differentiation mechanism to construct the physical residuals of the partial differential equations. ,in This is the approximate output of the neural network for the physical field; the mean square error of the physical residual is... The total loss function is formed by weighting and summing the boundary condition error, initial condition error, and optional supervision data error. .

7. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, In step S3, the method for generating coordinate points is at least one of uniform random sampling, Latin hypercube sampling, or Sobol low-difference sequence sampling.

8. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, The physical information neural network constructed in step S3 also includes Fourier feature embedding, which is used to map the input coordinates into high-dimensional features to improve the neural network's ability to express high-frequency physical fields.

9. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, The physical information neural network constructed in step S3 also includes an adaptive loss weight mechanism: dynamically adjusting the weight coefficients based on the gradient norm of each loss term with respect to the model parameters, expressed as: ; in For the first The gradient norm of each loss term, To prevent small amounts from being divided by zero, This is the smoothing coefficient for the exponential moving average.

10. The spatiotemporal prediction method for tank defect propagation based on physical constraint reinforcement learning according to claim 1, characterized in that, The physical information neural network in step S3 supports a built-in partial differential equation residual interface, wherein the partial differential equations include at least one of the Burgers equation, the heat conduction equation, the Poisson equation, and the wave equation; during training, at least one of the following strategies is used to stabilize training: gradient pruning, cosine annealing learning rate scheduling, or multi-stage learning rate decay.

11. A spatiotemporal prediction system for tank defect propagation based on physical constraint reinforcement learning, used to execute the method according to any one of claims 1 to 10, characterized in that, include: Data acquisition module: used to acquire multi-source monitoring data of oilfield storage tanks. The data is acquired through an acoustic emission sensor array, conditioning circuit and field programmable gate array parallel data acquisition circuit arranged at key locations of the storage tank, and sent to the embedded processor. State space construction module: used to perform cluster analysis on the defect and failure states of oilfield storage tanks using fuzzy clustering coarse and fine scale clustering methods, and to establish a state space containing multiple defect states; Physical Information Neural Network Module: Used to construct and store physical information neural networks that characterize the fault state transition kernel. Its loss function incorporates finite element differential equations describing the root causes of defective fault propagation as physical regularization terms. State transition and update module: Based on the Metropolis-Hastings method and fuzzy membership function, combined with the state transition kernel represented by the physical information neural network, it performs bidirectional transfer and update of the oilfield storage tank defect state; Reinforcement learning decision-making module: Constructs a reinforcement learning agent based on the proximal policy optimization algorithm, and perceives the optimal operation and maintenance action based on the current state; the reward function of this module includes an action switching penalty term; Multi-objective collaborative optimization module: Under the multi-objective optimization framework of maximizing equipment lifespan, minimizing operation and maintenance costs, and controlling safety risks, it jointly trains the physical information neural network and the reinforcement learning agent to generate the optimal operation and maintenance strategy.

Images