Equipment failure mode-based hydrogen refueling station multi-modal data acquisition method and terminal equipment

Abstract

The invention discloses a hydrogen refueling station multi-mode data acquisition method based on an equipment failure mode and terminal equipment, and belongs to the technical field of hydrogen energy infrastructure safety monitoring. The multi-level image intelligent detection algorithm comprises three levels of bottom-layer multi-mode feature extraction, middle-layer failure feature classification and high-layer multi-mode fusion decision; (2) analyzing a risk level corresponding to the failure mode in combination with a cloud equipment knowledge graph, wherein the knowledge graph cloud equipment knowledge graph comprises equipment manufacturing, installation, process, maintenance and detection data; and (3) dynamically scheduling the collection frequency and range of the terminal data collector according to the risk level and failure mechanism sorting result. The terminal device comprises an image acquisition module, a sensor module and a communication module. The problems that an existing hydrogen refueling station data acquisition method is fixed in strategy, low in failure recognition precision and insufficient in risk association are solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hydrogen energy infrastructure safety monitoring, and particularly relates to a hydrogen refueling station multi-modal data acquisition method and terminal equipment based on equipment failure modes. BACKGROUND

[0002] Hydrogen refueling station equipment (hydrogen storage tanks, compressors, hydrogen dispensers, etc.) is in a high-pressure, hydrogen-rich environment for a long time, which is prone to failure modes such as hydrogen embrittlement, corrosion, and mechanical fatigue, directly threatening the safety of operation. The existing data acquisition methods have the following limitations: 1. Fixed acquisition strategy: Traditional terminal equipment uses a fixed frequency (e.g., once an hour) and a fixed range (e.g., only acquiring pressure and temperature) data acquisition mode, which cannot dynamically adjust to the actual failure risk of the equipment, resulting in insufficient data for high-risk failure modes (e.g., early hydrogen embrittlement) and redundant data in low-risk states.

[0003] 2. Low failure mode recognition accuracy: Existing image detection methods rely on single features (such as surface cracks) to identify failures, without considering the multi-level features of hydrogen refueling station equipment failure (such as "microscopic lattice changes -> macroscopic surface bulging -> performance parameter decay" of hydrogen embrittlement), resulting in an accuracy rate of less than 60% for identifying hidden failure modes such as hydrogen embrittlement and vibration fatigue.

[0004] 3. Lack of risk correlation: Data acquisition is disconnected from equipment failure mechanisms and historical risk data, and does not analyze the impact of equipment life cycle information (manufacturing materials, installation records, maintenance history, etc.) on failure, making it impossible to achieve closed-loop control of "failure mode - risk level - acquisition strategy".

[0005] 4. Lack of multi-modal coordination: Image data, sensor data (vibration, hydrogen concentration), and text data (inspection reports) do not form a coordinated acquisition mechanism, making it difficult for single data dimensions to support the identification and risk assessment of complex failure modes (such as "external damage + corrosion" coupled failure). SUMMARY

[0006] The present application aims to solve the problems of fixed acquisition strategy, low failure recognition accuracy, and insufficient risk correlation in existing hydrogen refueling station data acquisition methods, and provides a hydrogen refueling station multi-modal data acquisition method and terminal equipment based on equipment failure modes.

[0007] The technical solution adopted by the present application is as follows: A hydrogen refueling station multi-modal data acquisition method based on equipment failure modes, comprising the following steps: (1) Using a multi-level image intelligent detection algorithm to identify equipment failure modes, the multi-level image intelligent detection algorithm including bottom-level multi-modal feature extraction, middle-level failure feature classification, and high-level multi-mode fusion decision-making; (2) Analyze the risk level corresponding to the failure mode in combination with the cloud device knowledge graph, wherein the cloud device knowledge graph comprises device manufacturing, installation, process, maintenance and detection data; (3) According to the risk level and the failure mechanism ranking result, the collection frequency and range of the terminal data collector are dynamically scheduled.

[0008] Further, the bottom layer multi-modal feature extraction is specifically: based on the image acquisition module, the features of the device surface image are extracted, the lattice distortion, surface bulge and hydrogen concentration distribution features are extracted for hydrogen embrittlement; the geometric parameters, texture features and color features of corrosion spots are extracted for corrosion; the image edge blur and bolt displacement features are extracted for mechanical vibration fatigue.

[0009] Further, the middle layer failure feature classification is specifically: a CNN model with a fusion attention mechanism is used to classify the bottom layer multi-modal features, a hydrogen embrittlement double-channel CNN is constructed for hydrogen embrittlement, and the hydrogen embrittlement exclusive features are strengthened through the attention mechanism; a ResNet-50 model is used for corrosion, and a classification branch of corrosion degree is added in the fully connected layer; finally, the preliminary confidence of each failure mode is output.

[0010] Further, the high layer multi-modal fusion decision is based on the D-S evidence theory, and the middle layer failure feature result and the feature similarity are fused to output the final failure mode and the comprehensive confidence.

[0011] A terminal device for implementing a hydrogenation station multi-modal data acquisition method based on device failure modes, comprising an image acquisition module, a sensor module and a communication module, which can dynamically adjust the collection parameters according to the cloud scheduling instructions; The image acquisition module comprises a visible light camera, an ultrasonic imager and an infrared thermal imager. The sensor module comprises a vibration sensor and a hydrogen concentration detector.

[0012] As described above, due to the adoption of the above technical solutions, the beneficial effects of the present application are: 1. The failure mode recognition accuracy is significantly improved: the multi-level image intelligent detection algorithm greatly improves the recognition accuracy of hidden failure modes such as hydrogen embrittlement and vibration fatigue through three-level feature extraction and fusion, solving the problem of high misjudgment rate of traditional methods for complex failure modes.

[0013] 2. The data acquisition pertinence is enhanced: based on the risk level, the collection strategy is dynamically scheduled, the data integrity is greatly improved in high-risk scenarios, the redundant data is significantly reduced in low-risk scenarios, and the data transmission and storage costs are reduced.

[0014] 3. Enhanced Risk Correlation Capabilities: By combining cloud-based device knowledge graphs, the system enables correlation analysis between device lifecycle data and failure modes, significantly reducing risk level assessment errors and providing accurate basis for safety decisions.

[0015] 4. Multimodal collaborative optimization: Integrating image, sensor, and text data to achieve a closed loop of "failure identification - risk analysis - data replenishment", significantly improving the detection rate of complex coupled failure modes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart illustrating the overall technical process of the present invention. Figure 2 This is a schematic diagram of the device failure association rules of the cloud-based knowledge graph of the present invention; Figure 3 This is a dynamic scheduling logic diagram of the terminal data collector of the present invention; Figure 4 This is a flowchart illustrating an implementation example of the hydrogen embrittlement failure monitoring method for hydrogen storage tanks according to the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0019] It should be noted that the labels and letters in the following figures represent similar items, therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0020] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use. They are only used for the purpose of simplifying the description of this invention and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. In addition, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] Furthermore, terms such as "horizontal" and "vertical" do not imply that components must be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0022] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0023] Refer to the instruction manual. Figures 1-4 , A multimodal data acquisition method for hydrogen refueling stations based on equipment failure modes includes three main steps: multi-level image equipment failure mode recognition, cloud-based risk analysis, and terminal dynamic data acquisition. The specific process is as follows: 1. Failure Modes of Multi-Level Image Intelligent Detection and Recognition Equipment For typical failure modes of hydrogen refueling stations (hydrogen embrittlement, mechanical vibration fatigue, corrosion, external damage, etc.), a three-level detection algorithm is designed, consisting of low-level feature extraction, mid-level failure feature classification, and high-level multi-mode fusion decision-making. 1) Multimodal feature extraction (bottom layer) Feature extraction from equipment surface images (visible light, infrared, and ultrasonic imaging): Hydrogen embrittlement characteristics: extraction of microscopic lattice distortion (through high-resolution ultrasonic images), macroscopic surface bulging (grayscale gradient changes in visible light images), and hydrogen concentration distribution (temperature anomaly areas in infrared thermal imaging); Corrosion features: Extract geometric parameters (area, perimeter), texture features (LBP local binary mode), and color features (HSV color shift) of corrosion spots; Mechanical vibration fatigue: combining vibration sensor data with image edge blurring (image edge diffusion caused by high-frequency vibration) and bolt loosening displacement (pixel offset after image registration). External damage: Extract the contour features of scratches / dents (Canny edge detection) and estimate the depth (binocular visual disparity calculation).

[0024] 2) Failure Feature Classifier (Intermediate Layer) An improved CNN model (with fused attention mechanism) is used to classify low-level features: To address hydrogen embrittlement: a dual-channel CNN with a "lattice-bulge" structure was designed to enhance hydrogen embrittlement-specific features (such as the irregularity of the bulge edges) through an attention mechanism. For corrosion: The ResNet-50 model is used, and a classification branch for the degree of corrosion (mild / moderate / severe) is added to the fully connected layer; Output: Preliminary identification results and confidence levels of various failure modes (e.g., "hydrogen embrittlement: confidence level 85%" and "corrosion: confidence level 60%").

[0025] 3) Multi-mode integrated decision-making (decision-making level) Based on the DS evidence theory, the results of the intermediate layer are fused to resolve feature conflicts (such as the confusion between the features of "hydrogen embrittlement bulge" and "corrosion bulge"): Input: Confidence of each failure mode and feature similarity (e.g., the matching degree between the current feature and historical hydrogen embrittlement cases); Decision rules: When the confidence level of hydrogen embrittlement is >80% and the lattice distortion feature matching degree is >75%, it is judged as "hydrogen embrittlement failure"; when the confidence level of corrosion is >70% and the texture feature conforms to the corrosion law, it is judged as "corrosion failure". Output: Final failure mode (e.g., "hydrogen embrittlement + mild corrosion") and overall confidence level (≥90%).

[0026] 2. Cloud-based device knowledge graph association and risk level analysis The entire lifecycle of data from cloud-based knowledge graph storage devices includes: Basic attributes: manufacturing material (e.g., 316L stainless steel), design parameters (e.g., rated pressure of hydrogen storage tank 100MPa), installation records (e.g., welding process); Operational data: historical maintenance records (e.g., safety valve replacement in 2023), test reports (e.g., wall thickness test results), process parameters (e.g., hydrogenation frequency); Failure association rules: such as "hydrogen embrittlement + material is 304 stainless steel → risk level increases by 20%" and "vibration fatigue + operation for more than 5 years → risk level increases by 30%".

[0027] Risk analysis process: 1) Match the failure modes identified by image recognition (such as "hydrogen embrittlement") with the basic attributes of the device in the knowledge graph (such as "material = 304 stainless steel" and "service life = 6 years"). 2) Calculate the risk value using the equipment risk assessment model: Risk value = Probability of failure × Consequences of failure Failure probability = Failure probability of similar equipment × (Failure technology factor + Process factor) Failure technology factor = f 失效模式 (Detection parameters, rate), such as the "hydrogen embrittlement" technology factor = f 氢脆 (Crack length, 2.5 mm / month), "corrosion" technology factor = f 腐蚀 (Crack length, 0.05 mm / year) Failure consequences = personal injury / casualty loss + property loss 3) Divide the risk level into four levels: low (R4), medium (R3), medium-high (R2), and high (R1). First, sort the equipment from high to low risk and output the main failure modes of the corresponding risks (e.g., "hydrogen embrittlement is the core failure mechanism of the current high risk").

[0028] 3. Dynamic scheduling of terminal data collectors The terminal equipment (equipped with cameras, vibration sensors, hydrogen concentration detectors, etc.) dynamically adjusts its data acquisition strategy based on the risk level and failure mechanism ranking results. Data acquisition frequency adjustment: For high-risk failure mechanisms (such as hydrogen embrittlement at level 4 risk), increase the image acquisition frequency from once per hour to once per 10 minutes, and increase the vibration data acquisition frequency from 1 kHz to 5 kHz; for low-risk (level 1) failure mechanisms, reduce the frequency to once per day. Expanded acquisition scope: For the coupled failure of "hydrogen embrittlement + corrosion", near-infrared image acquisition (to identify micro-corrosion) and ultrasonic wall thickness detection (to supplement the wall thickness change data caused by hydrogen embrittlement) have been added. Multimodal collaboration: When an image detects external damage, it triggers the acquisition of text data from the terminal (automatically retrieving inspection reports from the last 3 months) to achieve multi-dimensional verification of "image + sensor + text".

[0029] Example The following uses the monitoring of hydrogen embrittlement failure in hydrogen storage tanks at hydrogen refueling stations as an example to illustrate the implementation process of this invention in detail: 1. Terminal equipment deployment Terminal equipment includes: Image acquisition module: High-definition visible light camera (4K resolution), ultrasonic imager (0.1mm accuracy), infrared thermal imager (temperature measurement range -20~150℃); Sensor modules: vibration sensor (range 0~10mm / s), hydrogen concentration detector (range 0~2000ppm). Communication module: 5G combined with edge computing gateway, supporting real-time data transmission and local preliminary processing.

[0030] 2. Multi-level intelligent image detection process (1) Low-level feature extraction: An ultrasonic imager was used to acquire images of the weld area of ​​the hydrogen storage tank and extract lattice distortion features (distortion area S=5mm²). A visible light camera captures surface images, and the bulge features are extracted (diameter D=3mm, grayscale gradient change rate=0.8). The infrared thermal imager detected a local temperature anomaly (5°C higher than the ambient temperature).

[0031] (2) Middle-level classification: A hydrogen embrittlement dual-channel CNN classifies the "lattice distortion + bulge" feature with an output confidence level of 88%. The corrosion classifier analyzes texture features and outputs a confidence level of 30%.

[0032] (3) High-level integration decision-making: DS evidence theory fusion result: hydrogen embrittlement feature matching degree 82% > 75%, overall confidence level 91% → judged as "hydrogen embrittlement failure".

[0033] 3. Cloud-based risk analysis (1) Knowledge graph matching: Hydrogen storage tank material = 304 stainless steel (high sensitivity to hydrogen embrittlement), service life = 7 years (50% over the design life), historical maintenance records include "weld repair in 2022"; (2) Risk calculation: Severity = 0.8 (hydrogen embrittlement may lead to leakage), Probability = 0.7 (material + service life), Importance = 0.9 (hydrogen storage tank is the core equipment) → Risk value = 0.8×0.7×0.9=0.504 → Risk level 4 (extremely high); (3) Ranking of main failure mechanisms: hydrogen embrittlement (70% weight) > welding defects (30% weight).

[0034] 4. Terminal data acquisition and scheduling (1) Modal type and acquisition equipment Visual modality: High-definition camera (20 megapixels) captures the appearance, defects, and markings of the equipment; Thermal modes: Temperature field distribution was acquired using an infrared thermal imager (640×512 resolution); Mechanical modes: Vibration sensors (±16g range) collect vibration frequency and amplitude data of the equipment; Chemical mode: A hydrogen sensor (0-2000ppm) collects the leak concentration and analyzes the rate of concentration increase; (2) Risk-oriented modal combination R1 level equipment: adopts a full-modal combination of "vision + thermal + mechanical + chemical"; R2 level equipment: adopts a combination of "visual + thermal + mechanical / chemical" methods (selected according to the failure mechanism of the equipment); R3 level equipment: adopts a basic combination of "vision + thermal"; R4 level devices: use only the "vision" single modality.

[0035] (3) Dynamic acquisition frequency adjustment R1 level: Real-time acquisition (key parameters once per second) + timed full parameter acquisition (once per hour); R2 level: Timed data collection (1 time / 4 hours); R3 level: Daily collection (1 time / day); R4 level: Periodic data collection (1 time / week).

[0036] (4) Priority setting The terminal automatically sorts data collection tasks according to the real-time risk level. R1 level tasks interrupt the current non-urgent tasks and are executed first. R2 level tasks are executed immediately after the current task is completed. R3 / R4 level tasks are executed as planned.

[0037] (5) Adaptive adjustment to risk changes The terminal receives real-time updates on the risk level from the backend (e.g., a device is upgraded from R2 to R1). Automatically adjust the data acquisition strategy within 10 seconds: increase the number of parameter types (e.g., from 14 to 18) and increase the data acquisition frequency (from once / 4 hours to real-time + once / hour). Generate a risk change log to record the parameter comparison benchmarks before and after the data collection strategy adjustment.

[0038] The above description constitutes an embodiment of the present invention. The foregoing descriptions are preferred embodiments of the present invention. Unless there is a clear contradiction or a prerequisite for a particular preferred embodiment, the preferred embodiments can be arbitrarily combined and used. The embodiments and specific parameters described are merely for clearly illustrating the verification process of the invention and are not intended to limit the scope of patent protection of the present invention. The scope of patent protection of the present invention is still determined by its claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention should also be included within the scope of protection of the present invention.

Claims

1. A multimodal data acquisition method for hydrogen refueling stations based on equipment failure modes, characterized in that, Includes the following steps: (1) A multi-level image intelligent detection algorithm is used to identify the failure mode of the equipment. The multi-level image intelligent detection algorithm includes three levels: bottom-level multi-modal feature extraction, middle-level failure feature classification, and high-level multi-modal fusion decision. (2) Combine the cloud-based device knowledge graph to analyze the risk level corresponding to the failure mode. The cloud-based device knowledge graph includes equipment manufacturing, installation, process, maintenance, and testing data. (3) Based on the risk level and failure mechanism ranking results, dynamically schedule the collection frequency and range of the terminal data collector.

2. The method for multimodal data acquisition of hydrogen refueling stations based on equipment failure modes according to claim 1, characterized in that, The underlying multimodal feature extraction specifically involves: extracting features from the equipment surface image based on the image acquisition module; extracting lattice distortion, surface bulging, and hydrogen concentration distribution features for hydrogen embrittlement; extracting geometric parameters, texture features, and color features of corrosion spots for corrosion; and extracting image edge blurring and bolt displacement features for mechanical vibration fatigue.

3. The method for multimodal data acquisition of hydrogen refueling stations based on equipment failure modes according to claim 1, characterized in that, The mid-level failure feature classification is specifically as follows: a CNN model with an attention mechanism is used to classify the multimodal features of the bottom layer. For hydrogen embrittlement, a hydrogen embrittlement dual-channel CNN is constructed, and the hydrogen embrittlement-specific features are enhanced through the attention mechanism. For corrosion, a ResNet-50 model is used, and a classification branch for the degree of corrosion is added to the fully connected layer. Finally, the preliminary confidence scores of various failure modes are output.

4. The method for multimodal data acquisition of hydrogen refueling stations based on equipment failure modes according to claim 1, characterized in that, The high-level multi-mode fusion decision is based on DS evidence theory, which integrates mid-level failure feature results and feature similarity to output the final failure mode and overall confidence level.

5. A terminal device for implementing the method according to claims 1-4, characterized in that, It includes an image acquisition module, a sensor module, and a communication module, and can dynamically adjust acquisition parameters according to cloud scheduling instructions; The image acquisition module includes a visible light camera, an ultrasonic imager, and an infrared thermal imager; The sensor module includes a vibration sensor and a hydrogen concentration detector.

Images