Industrial agent equipment analysis system and method based on reinforcement learning

Abstract

The invention relates to the technical field of industrial intelligent manufacturing and equipment health management, in particular to an industrial intelligent agent equipment analysis system and method based on reinforcement learning, and the method comprises the steps: collecting multi-dimensional real-time sensor data of industrial equipment; mapping multi-dimensional real-time sensor data to a continuous potential Riemannian manifold space by using a configured variational auto-encoder embedding module; manifold state coordinates and measurement tensors are output; constructing a directional degradation gradient vector field by using a configured virtual aging operator; generating an anti-fact fault sample; calculating a geodesic distance and a curvature gradient; generating a geometric curvature risk field signal representing the deformation degree; generating an equipment control instruction for guiding the equipment to avoid the high-curvature area; executing the equipment control instruction, and collecting actual moving track data generated by the equipment; calibrating calculation parameters of the measurement tensor; according to the method, the fault prediction problem under the zero sample condition is effectively solved, and advanced deduction of the potential evolution trend of the equipment is realized.

Description

Technical Field

[0001] This invention relates to the field of industrial intelligent manufacturing and equipment health management technology, specifically to an industrial intelligent agent equipment analysis system and method based on reinforcement learning. Background Technology

[0002] As modern industrial production develops towards high precision and intelligence, the system architecture and operating mechanism of industrial equipment are becoming increasingly complex. This complexity makes the data on the operating status of equipment exhibit high-dimensional, heterogeneous and nonlinear characteristics, which brings great challenges to equipment fault diagnosis and health management. Currently, equipment status is generally analyzed by monitoring whether the time-series signal values ​​collected by sensors exceed preset safety thresholds, or by training supervised learning models based on existing historical fault data. Technicians typically rely on the statistical characteristics of single physical quantities such as vibration, temperature, and current for judgment. However, traditional monitoring and analysis methods are mainly based on the linear assumption of Euclidean space or fitting known fault modes. When dealing with high-precision equipment, there are often latent faults where the values ​​of a single sensor are still within the normal range, but the coupling relationship between multidimensional parameters has undergone structural distortion. In addition, for zero-sample faults or extreme operating conditions that have never occurred before, due to the lack of corresponding historical training samples, existing methods are unable to accurately identify the potential topological evolution trend of the equipment, and cannot effectively defend and control before physical faults appear. Therefore, how to mine the deep geometric features of multidimensional data to cope with the risk of zero-sample faults in the absence of fault samples has become an urgent problem to be solved in this field. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for analyzing industrial intelligent equipment based on reinforcement learning. This method can mine deep geometric features from multi-dimensional data, identify latent faults where a single sensor reading is normal but the system structure is already unstable, and address zero-sample fault risks through counterfactual inference even in the absence of historical fault samples. This enables preventative control of industrial equipment. Specifically, the technical solution of this invention is as follows: A reinforcement learning-based method for analyzing industrial intelligent agent equipment includes: S1. Collect multidimensional real-time sensor data from industrial equipment; use the configured variational autoencoder embedding module to map the multidimensional real-time sensor data to a continuous latent Riemannian manifold space; based on the mapping result, output the manifold state coordinates and metric tensor containing local geometric properties. S2. Receive the manifold state coordinates and the metric tensor; construct a directional degradation gradient vector field in the latent Riemannian manifold space using the configured virtual aging operator; lock the tangent vector direction with the largest degradation gradient according to the metric tensor; perform geodesic extension calculation on the tangent vector direction to generate counterfactual fault samples. S3. Based on the manifold state coordinates and the counterfactual fault samples, calculate the geodesic distance and curvature gradient; based on the geodesic distance and the curvature gradient, quantify the degree of deformation of the manifold surface topology; generate a geometric curvature risk field signal characterizing the degree of deformation; S4. The geometric curvature risk field signal is used as a negative feedback numerical matrix and input to the geometric instinct reinforcement learning strategy unit; based on the configured negative curvature reward mechanism, a device control command is generated to guide the device to avoid high curvature regions. S5. Execute the device control command and collect the actual operating trajectory data generated by the device; feed the actual operating trajectory data back to the variational autoencoder embedding module and calibrate the calculation parameters of the metric tensor.

[0004] Optionally, the output includes manifold state coordinates and metric tensors containing local geometric properties, including: The flat region of the potential Riemannian manifold surface is defined as the healthy operating state of the device, and the flat region represents high degree of freedom and low energy confinement; The high curvature twisted regions or singularities of the potential Riemannian manifold spatial surface are defined as equipment failures or physical limit states; The geometric parameters describing the degree of curvature in the local space are calculated and combined with the manifold state coordinates to form a sensing input that reflects the potential topology of the device.

[0005] Optionally, generate counterfactual fault samples, including: Configure the virtual aging operator by combining prior knowledge of the physical degradation of the equipment or historical full life cycle data; The state evolution trajectory of the simulated device under counterfactual conditions is calculated by extending the geodesic lines. The coordinates of the high curvature points of the manifold on the state evolution trajectory are extracted as the counterfactual fault samples.

[0006] Optionally, generating a geometric curvature risk field signal characterizing the degree of deformation includes: Using the configured geodesic deviation calculation module, the concept of curvature flow is introduced to process the manifold state coordinates and the counterfactual fault samples; Risk is defined as the degree of severe deformation of the manifold surface topology, and the degree of deformation is positively correlated with the extent to which the device approaches its physical limits; The degree of deformation is quantified to generate the geometric curvature risk field signal that directly acts on the geometric instinct reinforcement learning strategy unit.

[0007] Optionally, generate device control commands to guide the device to avoid high curvature regions, including: The geometric curvature risk field signal is introduced as a penalty term into the reward function of the reinforcement learning algorithm; During training and inference, the agent is forced to develop a geometric instinct to automatically search for and lock onto the flat region; When the counterfactual fault sample causes a trough in the value terrain, the device control command that changes the operating strategy is generated to avoid potential fault trajectories before the real fault signal appears.

[0008] An industrial intelligent agent equipment analysis system based on reinforcement learning, comprising: A multidimensional state manifold mapping unit is used to collect the multidimensional real-time sensor data, map it to the latent Riemannian manifold space using the variational autoencoder embedding module, and output the manifold state coordinates and the metric tensor. The counterfactual stress projection unit is used to receive the manifold state coordinates and the metric tensor, construct the degradation gradient vector field using the virtual aging operator, and perform the geodesic extension calculation to generate the counterfactual fault sample. A geometric curvature sensing and measurement unit is used to calculate the geodesic distance and the curvature gradient, quantify the degree of deformation of the manifold surface topology, and generate the geometric curvature risk field signal. A geometric instinct reinforcement learning strategy unit is used to receive the geometric curvature risk field signal and generate the device control command based on the negative curvature reward mechanism. The reverse correction coupling module is used to collect the actual running trajectory data generated after the execution of the device control command, and feed it back to the multidimensional state manifold mapping unit to calibrate the metric tensor.

[0009] Optionally, the multidimensional state manifold mapping unit is configured with an embedded module based on a variational autoencoder; The counterfactual stress projection unit is configured with a virtual aging operator based on prior knowledge of physical decay or historical full life cycle data. The geometric instinct reinforcement learning strategy unit is configured with a negative curvature reward mechanism to convert the geometric curvature risk field signal into a negative reward.

[0010] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention maps high-dimensional heterogeneous sensor data to a latent Riemannian manifold space using a variational autoencoder and analyzes the local geometric properties of the data using a metric tensor. Compared with traditional linear monitoring methods based on Euclidean space, this technology can capture the subtle distortions in the nonlinear coupling relationships between multidimensional parameters at the topological level, thereby effectively identifying hidden faults where the values ​​of a single sensor are within the normal range, but the internal structure of the system has become unstable, significantly improving the state perception accuracy of high-precision equipment. 2. This invention introduces a virtual aging operator and geodesic extension calculation, which can actively construct counterfactual fault samples in the absence of historical fault data; by simulating the evolution trajectory of the equipment along the degradation gradient direction in manifold space, the system can generate physically feasible but not yet occurred limit state points; this mechanism gets rid of the dependence of traditional supervised learning models on historical fault labels, effectively solves the problem of fault prediction under zero sample conditions, and realizes the advanced inference of the potential evolution trend of the equipment. 3. This invention establishes a risk quantification system based on geometric topology, which assesses the degree of deformation of manifold surfaces by calculating geodesic distances and curvature gradients; the system converts spatial bending characteristics into a highly sensitive geometric curvature risk field signal, which can reflect the subtle decoupling trends between equipment parameters in real time; this assessment method based on geometric shape changes has an extremely fast response speed and can provide early warnings before physical parameters reach safety thresholds, thus gaining valuable buffer time for equipment maintenance; 4. This invention incorporates geometric curvature risk into a reinforcement learning strategy, constructing a negative curvature reward mechanism that forces the agent to develop a geometric instinct to seek out flat areas with low curvature. When counterfactual inference detects a potential risk that reduces the value of the terrain, the system can automatically generate control commands to change the operating strategy. This achieves a leap from passive fault diagnosis to proactive preventive control, ensuring that the equipment actively avoids potential fault trajectories before real faults appear, thus guaranteeing the continuity and safety of the production process. Attached Figure Description

[0011] The present invention will be further explained below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. Example 1:

[0013] Please see Figure 1 A reinforcement learning-based method for analyzing industrial intelligent agent equipment includes: S1. Collect multidimensional real-time sensor data from industrial equipment; use the configured variational autoencoder embedding module to map the multidimensional real-time sensor data to a continuous latent Riemannian manifold space; based on the mapping result, output the manifold state coordinates and metric tensor containing local geometric properties. S2. Receive manifold state coordinates and metric tensor; construct a directional degradation gradient vector field in the latent Riemannian manifold space using the configured virtual aging operator; lock the tangent vector direction with the largest degradation gradient based on the metric tensor; perform geodesic extension calculation on the tangent vector direction to generate counterfactual fault samples. S3. Based on the manifold state coordinates and counterfactual fault samples, calculate the geodesic distance and curvature gradient; based on the geodesic distance and curvature gradient, quantify the degree of deformation of the manifold surface topology; generate a geometric curvature risk field signal characterizing the degree of deformation. S4. Input the geometric curvature risk field signal as a negative feedback numerical matrix into the geometric instinct reinforcement learning strategy unit; based on the configured negative curvature reward mechanism, generate device control commands to guide the device to avoid high curvature regions; S5. Execute equipment control commands and collect actual operating trajectory data generated by the equipment; feed the actual operating trajectory data back to the variational autoencoder embedding module to calibrate the calculation parameters of the metric tensor.

[0014] In this embodiment, an industrial intelligent agent equipment analysis method based on reinforcement learning constructs a set of industrial control logic based on differential geometry, aiming to solve the defense problem of high-precision equipment under zero-sample faults. In the data mapping and geometric modeling stage, the system executes step S1 to collect multi-dimensional real-time sensor data of industrial equipment, covering high-frequency time-series signals of vibration frequency, temperature gradient, multiphase current, and fluid pressure. This step uses a configured variational autoencoder embedding module to process heterogeneous data. This module is pre-trained based on a full lifecycle dataset containing normal equipment operation and known historical faults to ensure that the potential space can cover healthy and unhealthy areas. This module is configured with a specific loss function, which forces the discrete, high-dimensional sensor data to be mapped into a continuous potential Riemannian manifold space by minimizing reconstruction error and KL divergence. In this process, the system does not simply perform dimensionality reduction, but uses the Jacobian matrix of the decoder network to calculate the metric tensor of each point. The metric tensor is a positive definite symmetric matrix that precisely defines the minute distance and angle between two points in the manifold space. Specifically, the metric tensor... The calculation formula is: ;in, Represents decoder network function Regarding latent variables Jacobian matrix, It is the identity matrix. The regularization constant, set to prevent matrix singularity, is set to a value of [value missing] in this embodiment. This formula ensures that along any vector in the latent space... The actual change in distance in the original data space during movement. satisfy ; This outputs the manifold state coordinates and metric tensor containing local geometric properties, providing a mathematical basis for subsequent non-Euclidean geometric operations; Based on this, the system enters the counterfactual deduction stage, i.e., executing step S2. The system receives the manifold state coordinates and metric tensor, and loads the configured virtual aging operator. This operator is essentially a vector field function defined based on physical mechanisms. Its function is to construct a directional degradation gradient vector field in the potential Riemannian manifold space. This vector field characterizes the trend of natural degradation of the device over time under different states. The system calculates the Riemann gradient based on the metric tensor and locks the direction along the tangent vector with the largest degradation gradient, i.e., the evolution direction in which the device is most likely to experience performance degradation. In this direction, the system calls the exponential mapping algorithm to perform geodesic extension calculation. This calculation simulates the state position of the device under assumed extreme operating conditions or after long-term operation by solving the geodesic differential equation, thereby generating counterfactual failure samples. These samples represent physically feasible but currently unrealized extreme state points. In step S3, the risk quantification and assessment phase involves the system calculating the shortest path length (geodesic distance) between the manifold state coordinates and counterfactual fault samples using Riemannian metrics. Simultaneously, it calculates the curvature change (curvature gradient) along this path. Within this analysis framework, the system quantifies the degree of deformation of the manifold surface topology based on the geodesic distance and curvature gradient. When the curvature gradient exceeds a preset safety threshold, it indicates that the geometry of the region has distorted, corresponding to a significant increase in the instability of the physical equipment. Accordingly, the system generates a geometric curvature risk field signal characterizing the degree of deformation, which quantifies the geometric cost of evolving from the current state to a fault state. In the decision-making and control phase, i.e. step S4, the system imports the geometric curvature risk field signal as a negative feedback numerical matrix into the control loop and inputs it into the geometric instinct reinforcement learning strategy unit. This unit runs an improved reinforcement learning algorithm, the core of which is to reshape the value function based on the configured negative curvature reward mechanism. Under this mechanism, the risk field signal corresponding to the high curvature region is transformed into a very large penalty value, which forces the policy network to automatically find the path with the minimum curvature during the optimization process, thereby generating equipment control instructions to guide the device to avoid the high curvature region. The closed-loop optimization stage corresponds to step S5, where the system executes equipment control commands to drive the actuators and simultaneously collects actual operating trajectory data generated by the equipment. Using this real posterior data, the system calculates the geometric deviation between the predicted manifold structure and the actual observed values, and feeds this deviation as a loss signal back to the variational autoencoder embedding module. This geometric deviation loss signal... Defined using the Mahalanobis distance form: ;in, To predict manifold coordinates, The manifold coordinates corresponding to the actual observations; total loss function Updated to: ,in is the KL divergence weight coefficient, used to adjust the decoupling characteristics of the latent space; in this embodiment, it is set to 4.0. The geometric constraint weight is set to 0.5 in this embodiment; The encoder weights are adjusted using the backpropagation algorithm, and the computational parameters of the metric tensor are calibrated to ensure that the constructed Riemannian manifold dynamically approximates the actual physical characteristics of the device, achieving adaptive evolution of the model. To prevent frequent and drastic changes in the manifold coordinate system from causing the reinforcement learning strategy to fail, the above weight adjustments are performed using offline batch updates or periodic fine-tuning modes, for example, updating once every 24 hours. Furthermore, after updating the variational autoencoder weights, if the average variation norm of the metric tensor exceeds a preset threshold... The system will automatically trigger the online transfer learning process of the reinforcement learning policy unit, and use the latest manifold coordinates and reward signals to fine-tune the policy network with few samples to ensure that the control policy remains aligned with the updated manifold space. Example 2:

[0015] The output includes the manifold state coordinates and metric tensor containing local geometric properties, including: The flat region on the surface of a potential Riemannian manifold is defined as the healthy operating state of the device, and the flat region represents high degree of freedom and low energy confinement. Define the high-curvature twisted regions or singularities on the potential Riemannian manifold surface as equipment failures or physical limit states. The geometric parameters describing the degree of curvature in the local space are calculated and combined with the manifold state coordinates to form a sensing input that reflects the potential topology of the device.

[0016] In the geometric semantic definition of this embodiment, in order to establish an accurate mapping between mathematical space and physical entity, the flat region of the potential Riemannian manifold space surface is defined as the healthy operating state of the device; in a topological sense, a flat region means that the Riemann curvature tensor at that location is close to the zero tensor; in a physical sense, a flat region represents high degree of freedom and low energy constraint, that is, the device operating parameters are in the linear response range, and there is no nonlinear coupling effect caused by component wear or stress concentration. In contrast, this embodiment defines a high curvature distortion region or singularity on the potential Riemannian manifold surface as a device failure or physical limit state; wherein, a high curvature distortion region corresponds to a strong nonlinear correlation between device parameters, while a singularity corresponds to the determinant of the metric tensor tending to be singular, representing the failure of physical constraints or system collapse. In the specific calculation process, the system calculates the second derivative of the decoder's Jacobian matrix based on the automatic differentiation framework. It then calculates the geometric parameters describing the curvature of the local space by calculating the Christofer symbol and the Riemann curvature tensor. To avoid the curse of dimensionality caused by directly inputting high-order Riemann curvature tensors, the system extracts the scalar invariants of the curvature tensor, including the Ricci scalar curvature. and average cross-sectional curvature By employing vector concatenation, the system combines these scalarized or tensorized curvature features with manifold state coordinates to assemble them into high-dimensional feature vectors, forming a sensing input that reflects the potential topological structure of the equipment. This process upgrades the condition monitoring of industrial equipment from traditional numerical threshold judgment to geometric analysis of the overall topological shape of the system, enabling the identification of hidden faults where a single sensor reading is normal but the system structure has become unstable. Example 3:

[0017] Generate counterfactual fault samples, including: Configure virtual aging operators by combining prior knowledge of the physical degradation of the equipment or historical full life cycle data; The state evolution trajectory of the equipment under counterfactual conditions is simulated by geodesic extension calculations. The coordinates of high curvature points on the manifold on the state evolution trajectory are extracted as counterfactual fault samples.

[0018] In this embodiment, for the crucial step of generating counterfactual fault samples, the system employs a physically guided data augmentation strategy. During the configuration phase, the system combines prior knowledge of the device's physical degradation or historical full lifecycle data to configure a virtual aging operator; this operator is concretized as a drift vector function. Its parameters are derived from fitting historical failure data or from material mechanics formulas. In this embodiment, the drift vector function... It is constructed as a residual neural network module: ; in, By maximizing likelihood estimation The network weights are trained on the historical full lifecycle dataset. During the preprocessing stage of the training data, while ensuring that the sensor sampling frequency satisfies the Nyquist sampling theorem and is sufficiently high, the system uses the finite difference method to construct the supervision signal and calculate the difference vector between adjacent state points in the historical time series. During training, this difference vector As network output The pseudo-labels, by minimizing the mean square error between the predicted drift vector and the actual difference vector, enable the neural network to... It can fit a continuous deterioration velocity field, thus enabling modeling from discrete historical data to continuous manifold dynamics; the function outputs a... A velocity vector of the same dimension represents a point on the manifold. Tangential velocity evolving over time; ensuring that the inferred direction conforms to objective physical laws rather than random perturbations; During simulation execution, the system operates in its current state. Starting from the aging operator, the gradient direction is calculated. Using an initial velocity, the state evolution trajectory of the device under counterfactual conditions is simulated through geodesic extension calculations. This calculation obtains the state evolution trajectory of the device under the assumed time by solving a second-order differential equation through numerical integration. The projected position of the device on the manifold; During the sampling phase, the system scans the curvature changes along the generated trajectory and extracts the coordinates of high curvature points on the manifold of the state evolution trajectory as counterfactual fault samples. The specific extraction criteria are set as follows: when the cross-sectional curvature value of a point on the trajectory exceeds three times the standard deviation of the historical operating benchmark, the point is marked as a counterfactual fault sample. These samples supplement the missing extreme operating condition data in the training set and solve the model generalization problem under small sample conditions. Example 4:

[0019] Generate a geometric curvature risk field signal characterizing the degree of deformation, including: Using the configured geodesic deviation calculation module, the concept of curvature flow is introduced to process manifold state coordinates and counterfactual fault samples; Risk is defined as the degree of severe deformation of the manifold surface topology, and the degree of deformation is positively correlated with the degree to which the device approaches its physical limits; The degree of deformation is quantified to generate a geometric curvature risk field signal that directly acts on the geometric instinct reinforcement learning strategy unit.

[0020] In this embodiment, the generation of a geometric curvature risk field signal characterizing the degree of deformation aims to construct a highly sensitive risk assessment system; the system utilizes a configured geodesic deviation calculation module and introduces the concept of curvature flow to process manifold state coordinates and counterfactual fault samples; through the Jacobi field equation, the system calculates the separation or convergence velocity of adjacent geodesics during the evolution process, thereby measuring the curvature characteristics of space; In this assessment system, risk is defined as the degree of severe deformation of the manifold surface topology; this degree of deformation is quantified by measuring the second derivative norm of the tensor, and the degree of deformation is positively correlated with the degree to which the device approaches its physical limits; when minor damage occurs inside the device, wrinkles will occur in the corresponding local manifold, resulting in a significant deflection of the geodesics; To convert this geometric feature into a control signal, the system quantizes the degree of deformation by using a modified Sigmoid function to quantize the Riemann curvature magnitude. The risk probability mapped to the interval [0,1] The specific formula is as follows: ;in, The F-norm is calculated by integrating the components of the Riemann curvature tensor, i.e. Let be the norm of the Riemann curvature tensor. To establish a preset safety curvature threshold for the system, this embodiment uses historical operating data. Upper Realm The sensitivity coefficient is set to 2.0 in this embodiment. The geometric curvature risk field signal generated by this formula directly acts on the geometric instinct reinforcement learning strategy unit; this signal has The response speed is at a certain level, which means it can reflect the slight decoupling trend between parameters in real time and provide early warning information for control strategies that is ahead of the threshold alarm. Example 5:

[0021] Generate equipment control commands to guide the device to avoid high curvature regions, including: The geometric curvature risk field signal is introduced as a penalty term into the reward function of the reinforcement learning algorithm; During training and inference, the agent is forced to develop a geometric instinct to automatically search for and lock onto flat areas; When counterfactual fault samples cause a trough in the value terrain, equipment control instructions are generated to change the operating strategy in order to avoid potential fault trajectories before real fault signals appear.

[0022] In this embodiment, generating device control commands to guide the device to avoid high curvature regions is achieved by reconstructing the optimization objective; the system incorporates the geometric curvature risk field signal as a penalty term into the reward function of the reinforcement learning algorithm; the specific reward function is designed as follows: ,in Incentives for production efficiency For geometric curvature risk field signals, The penalty coefficient is denoted as ; where It is a dimensionless value after Min-Max normalization, and its value range is constant [0,1]. Specifically, The calculation formula is defined as follows: ;in, This refers to the current spindle speed or production line throughput of the equipment. The theoretical maximum speed or throughput designed for the equipment; This represents the instantaneous energy consumption or current value at the current time step. This is the upper limit of the rated energy consumption; The preset efficiency and energy consumption weighting coefficients are set to 0.7 and 0.3 respectively in this embodiment; this formula ensures that the intelligent agent does not excessively sacrifice production efficiency while pursuing high geometrical safety. That is, the risk probability calculated above. The value range is [0,1]; the penalty coefficient. Set as dynamic weights: ,in , , This represents the number of training iterations; and it is set to... Upper limit cutoff value ,Right now This dynamic weight design ensures that the focus is on exploration in the early stages of training, while in the later stages... When the value increases slightly, The value quickly turns negative, forcing the strategy network to avoid high curvature regions; During training and inference, this mechanism forces the agent to develop a geometric instinct to automatically search for and lock onto flat regions. This geometric instinct refers to the fact that after a large number of iterations and optimizations, the agent's policy network weights converge to a specific distribution, so that the output action sequence naturally tends to keep the device state in a region with a small Riemann curvature. In real-time control, when counterfactual fault samples cause the value terrain to form a trough, the agent generates equipment control commands to change the operating strategy based on the gradient guidance of the value function. For example, when an increase in the manifold curvature of the bearing vibration is detected, the agent will actively reduce the speed before the vibration amplitude exceeds the limit, thereby avoiding potential fault trajectories before the real fault signal appears, and achieving preventive control. Example 6:

[0023] Please see Figure 2 An industrial intelligent agent equipment analysis system based on reinforcement learning, comprising: The multidimensional state manifold mapping unit is used to acquire multidimensional real-time sensor data, map it to the latent Riemannian manifold space using a variational autoencoder embedding module, and output the manifold state coordinates and metric tensor. The antifactual stress projection unit is used to receive manifold state coordinates and metric tensors, construct a degradation gradient vector field using a virtual aging operator, and perform geodesic extension calculations to generate antifactual fault samples. The geometric curvature sensing and measurement unit is used to calculate geodesic distance and curvature gradient, quantify the degree of deformation of the manifold surface topology, and generate a geometric curvature risk field signal. A geometric instinctual reinforcement learning strategy unit is used to receive geometric curvature risk field signals and generate device control commands based on a negative curvature reward mechanism. The reverse correction coupling module is used to collect the actual running trajectory data generated after the execution of equipment control commands and feed it back to the multidimensional state manifold mapping unit to calibrate the metric tensor. The multidimensional state manifold mapping unit is configured with an embedded module based on a variational autoencoder; The counterfactual stress projection unit is equipped with a virtual aging operator based on prior knowledge of physical degradation or historical full life cycle data; The geometric instinct reinforcement learning strategy unit is equipped with a negative curvature reward mechanism, which is used to convert geometric curvature risk field signals into negative rewards.

[0024] In this embodiment, an industrial intelligent agent equipment analysis system based on reinforcement learning integrates five core computing modules that work collaboratively in the form of hardware or software logic. The multidimensional state manifold mapping unit serves as the data entry point and is equipped with an embedding module based on a variational autoencoder. This unit is used to acquire multidimensional real-time sensor data and, through forward propagation of a deep neural network, maps it to the latent Riemannian manifold space using the embedding module. This unit contains a differentiator for calculating the Jacobian matrix, thereby outputting the manifold state coordinates and metric tensor. The counterfactual stress projection unit is the system's deduction engine, equipped with a virtual aging operator based on prior knowledge of physical degradation or historical full lifecycle data. This unit receives manifold state coordinates and metric tensors, and uses the virtual aging operator to construct a degradation gradient vector field through numerical calculation methods. This unit calls the ODE solver to perform geodesic extension calculations to generate counterfactual fault samples, providing the system with virtual negative sample data. The geometric curvature sensing and measurement unit is responsible for risk calculation, which is used to calculate geodesic distance and curvature gradient. This unit has a built-in differential geometry calculation library to quantify the degree of deformation of the manifold surface topology and generate a geometric curvature risk field signal, which transforms abstract geometric features into specific control variables. The geometric instinct reinforcement learning strategy unit serves as the decision-making core and is equipped with a negative curvature reward mechanism to convert the geometric curvature risk field signal into a negative reward. This unit is used to receive the geometric curvature risk field signal and generate equipment control commands based on the negative curvature reward mechanism during the inference process of the reinforcement learning strategy network, and output them to the PLC or the underlying controller. The reverse correction coupling module is responsible for the closed-loop calibration of the system. It is used to collect the actual running trajectory data generated after the execution of equipment control commands and feed it back to the multidimensional state manifold mapping unit to calibrate the metric tensor. By continuously comparing the difference between the predicted manifold and the real manifold, the system can correct the weight parameters of the variational autoencoder to ensure the continuous improvement of analysis accuracy.

[0025] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for analyzing industrial intelligent agent equipment based on reinforcement learning, characterized in that, include: S1. Collect multi-dimensional real-time sensor data from industrial equipment; Using the configured variational autoencoder embedding module, the multidimensional real-time sensor data is mapped to a continuous latent Riemannian manifold space; based on the mapping result, the manifold state coordinates and metric tensor containing local geometric properties are output. S2. Receive the manifold state coordinates and the metric tensor; construct a directional degradation gradient vector field in the latent Riemannian manifold space using the configured virtual aging operator; lock the tangent vector direction with the largest degradation gradient according to the metric tensor; perform geodesic extension calculation on the tangent vector direction to generate counterfactual fault samples. S3. Based on the manifold state coordinates and the counterfactual fault sample, calculate the geodesic distance and curvature gradient; Based on the geodesic distance and the curvature gradient, the degree of deformation of the manifold surface topology is quantified; Generate a geometric curvature risk field signal characterizing the degree of deformation; S4. The geometric curvature risk field signal is used as a negative feedback numerical matrix and input into the geometric instinct reinforcement learning strategy unit. Based on the configured negative curvature feedback mechanism, generate equipment control commands to guide the device to avoid high curvature regions; S5. Execute the device control command and collect the actual operating trajectory data generated by the device; The actual running trajectory data is fed back to the variational autoencoder embedding module to calibrate the calculation parameters of the metric tensor.

2. The method for analyzing industrial intelligent agents based on reinforcement learning according to claim 1, characterized in that, The output includes the manifold state coordinates and metric tensor containing local geometric properties, including: The flat region of the potential Riemannian manifold surface is defined as the healthy operating state of the device, and the flat region represents high degree of freedom and low energy confinement; The high curvature twisted regions or singularities of the potential Riemannian manifold spatial surface are defined as equipment failures or physical limit states; The geometric parameters describing the degree of curvature in the local space are calculated and combined with the manifold state coordinates to form a sensing input that reflects the potential topology of the device.

3. The method for analyzing industrial intelligent agent equipment based on reinforcement learning according to claim 1, characterized in that, Generate counterfactual fault samples, including: Configure the virtual aging operator by combining prior knowledge of the physical degradation of the equipment or historical full life cycle data; The state evolution trajectory of the simulated device under counterfactual conditions is calculated by extending the geodesic lines. The coordinates of the high curvature points of the manifold on the state evolution trajectory are extracted as the counterfactual fault samples.

4. The method for analyzing industrial intelligent agent equipment based on reinforcement learning according to claim 1, characterized in that, Generating a geometric curvature risk field signal characterizing the degree of deformation includes: Using the configured geodesic deviation calculation module, the concept of curvature flow is introduced to process the manifold state coordinates and the counterfactual fault samples; Risk is defined as the degree of severe deformation of the manifold surface topology, and the degree of deformation is positively correlated with the extent to which the device approaches its physical limits; The degree of deformation is quantified to generate the geometric curvature risk field signal that directly acts on the geometric instinct reinforcement learning strategy unit.

5. The method for analyzing industrial intelligent agent equipment based on reinforcement learning according to claim 1, characterized in that, Generate equipment control commands to guide the device to avoid high curvature regions, including: The geometric curvature risk field signal is introduced as a penalty term into the reward function of the reinforcement learning algorithm; During training and inference, the agent is forced to develop a geometric instinct to automatically search for and lock onto the flat region; When the counterfactual fault sample causes a trough in the value terrain, the device control command that changes the operating strategy is generated to avoid potential fault trajectories before the real fault signal appears.

6. An industrial intelligent agent equipment analysis system based on reinforcement learning, based on the industrial intelligent agent equipment analysis method based on reinforcement learning as described in any one of claims 1-5, characterized in that, include: A multidimensional state manifold mapping unit is used to collect the multidimensional real-time sensor data, map it to the latent Riemannian manifold space using the variational autoencoder embedding module, and output the manifold state coordinates and the metric tensor. The counterfactual stress projection unit is used to receive the manifold state coordinates and the metric tensor, construct the degradation gradient vector field using the virtual aging operator, and perform the geodesic extension calculation to generate the counterfactual fault sample. A geometric curvature sensing and measurement unit is used to calculate the geodesic distance and the curvature gradient, quantify the degree of deformation of the manifold surface topology, and generate the geometric curvature risk field signal. A geometric instinct reinforcement learning strategy unit is used to receive the geometric curvature risk field signal and generate the device control command based on the negative curvature reward mechanism. The reverse correction coupling module is used to collect the actual running trajectory data generated after the execution of the device control command, and feed it back to the multidimensional state manifold mapping unit to calibrate the metric tensor.

7. The industrial intelligent agent equipment analysis system based on reinforcement learning according to claim 6, characterized in that: The multidimensional state manifold mapping unit is configured with an embedding module based on a variational autoencoder; The counterfactual stress projection unit is configured with a virtual aging operator based on prior knowledge of physical decay or historical full life cycle data. The geometric instinct reinforcement learning strategy unit is configured with a negative curvature reward mechanism to convert the geometric curvature risk field signal into a negative reward.

Images