Chemical tank multi-modal leakage monitoring method and system based on large model

Abstract

This invention relates to the field of industrial intelligent monitoring and chemical safety equipment technology, and discloses a multimodal leakage monitoring method and system for chemical storage tanks based on a large model. The key technical points are: extracting acoustic time-series signals and capturing hardware physical time differences, generating a modulated infrared thermo-spatiotemporal differential feature matrix, aggregating and updating the dimensionless IoT cyber-physical risk index scalar, performing channel-level spatial feature recalibration, performing rasterized perspective projection and cross-modal semantic reconstruction, performing inverse three-dimensional calculation and real-time temperature and sound velocity compensation, and performing nonlinear physical clamping calculation and hard-wired interlocking early warning; adopting a software-hardware collaborative cyber-physical fusion architecture, through the underlying cross-modal physical cascade modulation and state-space semantic reconstruction of heterogeneous sensor data, combined with a hardware time difference verification mechanism based on inverse three-dimensional calculation and real-time sound velocity and temperature compensation, to achieve rigid physical law clamping and highly reliable explosion-proof closed loop for leakage early warning decisions.

Description

Technical Field

[0001] This invention relates to the field of industrial intelligent monitoring and chemical safety equipment technology, and more specifically, to a method and system for multimodal leakage monitoring of chemical storage tanks based on a large model. Background Technology

[0002] Existing multimodal leakage monitoring methods for chemical storage tanks largely rely excessively on purely data-driven black-box algorithm models, lacking deep collaboration with macroscopic physical conservation laws and underlying hardware sensing mechanisms. As a result, when faced with complex operating conditions such as strong low-frequency mechanical vibrations, dynamic and complex heat source backgrounds, rarefied aerosol interference, and extreme temperature changes causing drift in the physical properties of metallic media (such as ultrasonic velocity), conventional algorithms cannot adaptively extract and amplify weak leakage features. At the same time, due to the spatial misalignment of coordinate dimensions during cross-modal data fusion in existing technologies, and the lack of a reverse hardware time-difference verification mechanism based on real three-dimensional physical space, pure semantic networks are highly susceptible to nondeterministic output biases (false positives) induced by environmental noise. They cannot rigidly constrain the model output with physical laws, resulting in persistently high overall false alarm and refusal-to-operate rates in high-risk industrial sites, making it difficult to meet the stringent engineering requirements for absolute reliability in explosion-proof linkage systems.

[0003] Therefore, this invention provides a method and system for multimodal leakage monitoring of chemical storage tanks based on a large model, which improves the above-mentioned technical problems. Summary of the Invention

[0004] This invention aims to address the shortcomings of existing technologies by providing a multimodal leakage monitoring method and system for chemical storage tanks based on a large model. The invention adopts a software-hardware collaborative cyber-physical fusion architecture, and achieves rigid physical constraints and highly reliable explosion-proof closed loop for leakage early warning decisions by combining the underlying cross-modal physical cascade modulation and state space semantic reconstruction of heterogeneous sensor data with a hardware time difference verification mechanism based on inverse three-dimensional calculation and real-time sound velocity and temperature compensation.

[0005] To achieve the above objectives, the present disclosure proposes the following technical solutions:

[0006] In a first aspect, this disclosure proposes a multimodal leakage monitoring method for chemical storage tanks based on a large model, applied to a cyber-physical fusion architecture composed of an edge computing gateway and multi-source heterogeneous sensors. The edge computing gateway integrates a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit. The method includes the following steps:

[0007] S1. Extracting acoustic timing signals and capturing hardware physical time difference: Through a digital signal processor, a two-dimensional time-frequency reconstruction is performed on the one-dimensional acoustic timing digital signal collected by the acoustic emission sensor, which is aggregated to generate an acoustic anomaly attention weight scalar, and the real hardware physical time difference of the two stress wave signals is captured simultaneously through the time-to-digital converter built into the field programmable gate array.

[0008] S2. Generate the modulated infrared thermal spatiotemporal difference feature matrix: Perform differential convolution operation on the infrared thermal image frame sequence acquired by the explosion-proof infrared thermal imager through the graphics processing unit, and inject the acoustic anomaly attention weight scalar as the physical gain coefficient to generate the modulated infrared thermal spatiotemporal difference feature matrix.

[0009] S3. Aggregate and update the dimensionless IoT cyber-physical risk index scalar: The central processing unit performs time-series feature extraction and standardization processing on the working condition data collected by heterogeneous IoT sensors, and combines it with the partial regression calibration coefficient to aggregate and generate the dimensionless IoT cyber-physical risk index scalar.

[0010] S4. Perform channel-level spatial feature recalibration: Extract the initial three-dimensional spatial feature matrix of the solid-state lidar through the field-programmable gate array, and use the dimensionless IoT cyber-physical risk index scalar to map the gate control weight vector to perform channel-level spatial feature recalibration on the initial three-dimensional spatial feature matrix.

[0011] S5. Perform rasterized perspective projection and cross-modal semantic reconstruction: The recalibrated three-dimensional spatial feature matrix is ​​rasterized through the spatial alignment coprocessor built into the field programmable gate array to generate a two-dimensional radar depth feature map. After the map is stitched together with the modulated infrared thermal spatiotemporal difference feature matrix, the hidden state is iterated through the input selective state space model via the neural processing unit, and the two-dimensional cross-modal fusion leaky semantic feature tensor is output.

[0012] S6. Perform inverse 3D calculation and real-time temperature and sound velocity compensation: The central processing unit extracts the local maximum leakage probability and the corresponding horizontal and vertical coordinates of the leakage extreme pixel from the 2D cross-modal fusion leakage semantic feature tensor. The inverse projection matrix is ​​called to reverse calculate the difference of the Euclidean distance of the leakage extreme pixel into the theoretical 3D propagation path. The infrared radiation of the extreme pixel region is extracted to calculate the real temperature difference, and real-time temperature compensation correction is performed on the ultrasonic reference sound velocity.

[0013] S7. Perform nonlinear physical clamping operation and hard-wired interlocking early warning: The central processing unit uses the dimensionless IoT information physical risk index scalar as a dynamic confidence weight, performs nonlinear physical clamping operation by combining the difference between the local maximum leakage probability, the Euclidean distance of the theoretical three-dimensional propagation path and the actual hardware physical time difference in a closed loop, outputs the overall leakage joint early warning confidence score scalar, and triggers the field loop relay to close when the overall leakage joint early warning confidence score scalar exceeds the preloaded leakage early warning confidence threshold.

[0014] As a preferred embodiment of the present invention, the extraction of acoustic timing signals and the physical time difference of the capture hardware in step S1 specifically includes:

[0015] A short-time Fourier transform is performed on the one-dimensional acoustic time-series digital signal, and the pre-loaded Mel filter bank weight matrix is ​​used for filtering to generate a two-dimensional acoustic time-frequency energy matrix. After integration and summation along the frequency dimension, the normalized activation function is applied to obtain the acoustic anomaly attention weight scalar. The absolute arrival time scalar of the two stress wave signals breaking through the pre-loaded amplitude threshold is captured by the time-to-digital converter, and the absolute value of the difference between the two is calculated to obtain the actual hardware physical time difference.

[0016] As a preferred embodiment of the present invention, the modulated infrared thermo-spatiotemporal difference feature matrix is ​​generated in step S2, specifically calculated using the following formula:

[0017] ;

[0018] in, Indicates time step The generated modulated infrared thermo-spatiotemporal difference feature matrix; This represents the acoustic anomaly attention weight scalar; Represented by natural constant An exponential function with base 0; This represents a pre-loaded feature extraction operator for a deep convolutional neural network; Indicates time step The current frame infrared thermal image matrix; Indicates time step Historical frame infrared thermal image matrix; The element-wise squaring operator represents the tensor level; This represents the preloaded Gaussian smoothing adjustment parameter.

[0019] As a preferred embodiment of the present invention, the aggregation and updating of the dimensionless Internet of Things cyber-physical risk index scalar in step S3 includes:

[0020] The system collects raw operating data from heterogeneous IoT sensors based on the Modbus protocol; it calls pre-loaded sliding median filter parameters to clean outliers in the raw operating data; and it calculates the instantaneous first derivative of the current pipeline pressure with respect to time and the spatial flow difference at the inlet and outlet of the storage tank based on the sensor polling sampling time interval.

[0021] The pre-loaded calibration mean and calibration standard deviation are respectively called to perform Z-score standardization on the instantaneous first derivative, the spatial flow difference, and the collected local volatile gas highest concentration measurement value;

[0022] The standardized values ​​are multiplied by their corresponding dimensionless partial regression calibration coefficients and summed. Combined with the environmental background drift compensation constant, the dimensionless Internet of Things cyber-physical risk index scalar is generated.

[0023] As a preferred embodiment of the present invention, the channel-level spatial feature recalibration performed in step S4 is specifically calculated using the following formula:

[0024] ;

[0025] in, Indicates time step The three-dimensional spatial feature matrix after risk recalibration; This represents the initially extracted three-dimensional spatial feature matrix; This represents the Hadamard product operator; This represents the preloaded recalibration gated mapping vector; This represents the dimensionless Internet of Things cyber-physical risk index scalar; This represents the normalized activation function.

[0026] As a preferred embodiment of the present invention, the rasterized perspective projection and cross-modal semantic reconstruction performed in step S5 specifically include:

[0027] The pre-loaded infrared camera intrinsic parameter matrix and solid-state lidar extrinsic parameter rotation matrix and translation vector are called to perform rasterization perspective projection and out-of-bounds truncation processing on the recalibrated three-dimensional spatial feature matrix, generating a two-dimensional radar depth feature map with pixel-by-pixel corresponding physical depth values.

[0028] Generate the corresponding depth matrix based on the mapping. After concatenating it with the modulated infrared thermo-spatial-differential feature matrix along the feature channel depth dimension, cross-modal semantic reconstruction is performed using the following tensor iteration formula:

[0029] ;

[0030] ;

[0031] in, Represents the depth matrix; Indicates time step Update the hidden state tensor; and This represents the preloaded implicit state transition matrix and input projection matrix; Indicates time step The historical implicit state tensor; This represents the tensor concatenation operator along the depth dimension of the feature channel; This represents the modulated infrared thermo-spatial-differential feature matrix; Indicates time step The two-dimensional cross-modal fusion leaky semantic feature tensor output by the mapping; This represents the preloaded output projection matrix; This represents the preloaded residual adaptive scaling matrix.

[0032] As a preferred embodiment of the present invention, the reverse three-dimensional solution and real-time temperature and sound velocity compensation performed in step S6 specifically include:

[0033] The two-dimensional cross-modal fusion leakage semantic feature tensor is mapped to a pixel-level probability distribution by a linear segmentation decoder, class probability normalization is performed and global spatial extrema are searched to obtain the local maximum leakage probability and the corresponding horizontal and vertical coordinates of the leakage extremum pixel.

[0034] The difference between the actual temperature difference and the Euclidean distance of the theoretical three-dimensional propagation path is calculated using the following formula:

[0035] ;

[0036] ;

[0037] in, This represents the difference between the actual average temperature of the tank wall and the standard temperature in the area of ​​the leakage extreme point. , This represents the slope and intercept of the preloaded thermal imager linear mapping operator; The x and y coordinates of the pixel representing the leakage extreme value; This represents the pixel value of the radiation intensity at the horizontal and vertical coordinates of the leakage extreme pixel in the current frame of the infrared image; Indicates the standard ambient temperature of the storage tank; This represents the difference in Euclidean distance between the theoretical three-dimensional propagation paths; Represents the L2 norm operator for three-dimensional Euclidean distance; This represents the inverse matrix of the pre-loaded infrared camera intrinsic parameters; A homogeneous vector representing the two-dimensional pixel coordinates of extreme values; Represents the depth matrix The precise physical depth scalar value at the horizontal and vertical coordinates of the leaking extreme pixel; and This represents the three-dimensional position vectors of the pre-loaded first and second high-frequency acoustic emission sensors.

[0038] As a preferred embodiment of the present invention, in step S7, nonlinear physical clamping calculation and hard-wired blocking early warning are performed. Specifically, the overall leakage joint early warning confidence score scalar is calculated using the following formula. When the score scalar exceeds the pre-loaded leakage early warning confidence threshold, the field circuit relay is triggered to close via the general-purpose input / output watchdog register:

[0039] ;

[0040] ;

[0041] ;

[0042] in, This represents the real-time ultrasonic velocity after temperature drift difference compensation. and This indicates the pre-loaded ultrasonic reference velocity and the velocity-temperature attenuation coefficient. This represents the actual temperature difference; The dimensionless Internet of Things cyber-physical risk index scalar represents the... The dynamic confidence weight coefficient scalar is determined; This represents the overall joint leak warning confidence score scalar; This represents the probability of leakage from the local maximum. Represents a negative exponential physical penalty operator; This represents the preloaded physical time difference tolerance attenuation coefficient; This represents the actual physical time difference of the hardware. This represents the difference in Euclidean distances of the theoretical three-dimensional propagation paths.

[0043] Secondly, this disclosure proposes a multimodal leakage monitoring system for chemical storage tanks based on a large model, deployed in a cyber-physical fusion architecture composed of an edge computing gateway and multi-source heterogeneous sensors. The edge computing gateway integrates a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit. The system includes:

[0044] The hardware initialization and parameter preloading module is used to preload the external prior model, physical calibration constants and network weight matrix into the high-speed storage medium corresponding to each computing unit through the central processing unit, and to complete clock synchronization and pipeline activation.

[0045] The acoustic timing signal extraction and hardware physical time difference capture module is used to perform two-dimensional time-frequency reconstruction on the one-dimensional acoustic timing digital signal collected by the acoustic emission sensor through a digital signal processor, aggregate and generate an acoustic anomaly attention weight scalar, and simultaneously capture the real hardware physical time difference of the two stress wave signals through the time-to-digital converter built into the field programmable gate array.

[0046] The infrared thermal spatiotemporal differential modulation module is used to perform differential convolution operation on the infrared thermal image frame sequence acquired by the explosion-proof infrared thermal imager through the graphics processing unit, and inject the acoustic anomaly attention weight scalar as the physical gain coefficient to generate the modulated infrared thermal spatiotemporal differential feature matrix.

[0047] The IoT operating condition feature extraction and dimensionless conversion module is used to collect raw operating condition values ​​based on the Modbus protocol, call the pre-loaded sliding median filter parameters to clean outliers, and perform time series feature extraction and standardization processing through the central processing unit. Combined with the partial regression calibration coefficient, it aggregates and generates a dimensionless IoT cyber-physical risk index scalar.

[0048] The channel-level spatial feature recalibration module is used to extract the initial three-dimensional spatial feature matrix of solid-state LiDAR through field-programmable gate array, and to perform channel-level spatial feature recalibration on the initial three-dimensional spatial feature matrix by mapping the dimensionless IoT cyber-physical risk index scalar to the gate control weight vector.

[0049] The rasterized perspective projection and cross-modal semantic reconstruction module is used to perform rasterized perspective projection on the recalibrated three-dimensional spatial feature matrix through the spatial alignment coprocessor built into the field programmable gate array, generate a two-dimensional radar depth feature map, and then stitch it with the modulated infrared thermal spatiotemporal difference feature matrix. After inputting the selective state space model through the neural processing unit, the hidden state is iterated, and the two-dimensional cross-modal fusion leaky semantic feature tensor is output.

[0050] The inverse 3D solution and real-time temperature and sound velocity compensation module is used to extract the local maximum leakage probability and the corresponding leakage extreme pixel horizontal and vertical coordinates from the 2D cross-modal fusion leakage semantic feature tensor through the central processing unit, call the inverse projection matrix to reverse solve the horizontal and vertical coordinates of the leakage extreme pixel into the difference of the Euclidean distance of the theoretical 3D propagation path; extract the infrared radiation of the extreme pixel area to calculate the real temperature difference, and perform real-time temperature compensation correction on the ultrasonic reference sound velocity.

[0051] The nonlinear physical clamping and hard-wired interlocking early warning module is used to use the dimensionless IoT information physical risk index scalar as a dynamic confidence weight through the central processing unit. It performs nonlinear physical clamping calculation by combining the difference between the local maximum leakage probability, the Euclidean distance of the theoretical three-dimensional propagation path, and the actual hardware physical time difference in a closed loop. It outputs an overall leakage joint early warning confidence score scalar and triggers the closing of the field circuit relay when the pre-loaded leakage early warning confidence threshold is exceeded.

[0052] Thirdly, this disclosure proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor comprises a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit that form an edge computing gateway. When the processing units work together to run the computer program, they realize a multimodal leakage monitoring method for chemical storage tanks based on a large model.

[0053] In summary, the present invention has the following beneficial effects:

[0054] Firstly, addressing the technical deficiency of traditional pure vision algorithms being highly susceptible to artifacts caused by dynamic background heat sources, this application utilizes a digital signal processor to extract the one-dimensional acoustic temporal digital signal from a high-frequency acoustic emission sensor, reconstructs and aggregates it to generate an acoustic anomaly attention weight scalar, and directly injects it as a physical gain coefficient into the pixel-by-pixel differential convolution of the graphics processing unit. This mechanism uses the underlying stress wave abrupt change as a physical prior signal, adaptively amplifying the sensitivity of deep networks to the extraction of small temperature gradients (such as heat-absorbing cold spots from leaking gas vaporization), achieving acoustic-thermal physics synergy at the hardware level.

[0055] Secondly, addressing the issue that conventional point cloud filtering algorithms tend to treat minute leakage plumes as noise and discard them, this application constructs a dimensionless IoT cyber-physical risk index scalar by polling multi-source underlying operating data through a central processing unit, and maps it to a gating weight vector using a field-programmable gate array. When the macroscopic pipeline network presents a risk of depressurization or high-concentration leakage, the system uses spatial broadcasting and Hadamard product to forcibly increase the gain of specific channels at the feature extraction layer, adaptively preserving and amplifying the extremely weak spatial features of aerosol diffuse reflection, and accurately preserving the spatial diffusion morphology of the leakage plume in the three-dimensional topology.

[0056] Thirdly, addressing the issues of spatial coordinate misalignment and excessive computational consumption of large models caused by direct splicing of heterogeneous data, this application utilizes a spatial alignment coprocessor to call intrinsic and extrinsic parameter matrices, performing hardware-level rasterization perspective projection and out-of-bounds truncation processing on the recalibrated 3D spatial feature matrix to generate a 2D radar depth feature map that strictly matches the infrared resolution. Furthermore, the neural processing unit employs a selective state-space model to iterate the implicit states of the spliced ​​heterogeneous tensors and integrates a residual adaptive scaling branch at the output, thus avoiding the enormous computational overhead of attention and effectively mitigating feature degradation in deep networks.

[0057] Fourth, addressing the issues of nondeterministic output bias (false positives) and false alarms caused by ultrasonic velocity drift under extreme weather conditions in purely data-driven algorithms, this application uses a linear segmentation decoder to lock the two-dimensional leakage extreme pixel and calls the inverse projection matrix to calculate the theoretical three-dimensional propagation path Euclidean distance difference. Simultaneously, it extracts the infrared radiation of the extreme pixel region to calculate the true temperature difference and performs real-time temperature compensation correction on the ultrasonic reference velocity. Finally, the system uses the operating condition risk index as a dynamic confidence weight to perform a closed-loop combination of the local maximum leakage probability and the actual hardware physical time difference captured by the field-programmable gate array, executing a nonlinear physical clamping operation. This mechanism completely eliminates the "illusion" false alarms of the algorithm model, ensuring the absolute engineering reliability of the field relay explosion-proof interlocking logic. Attached Figure Description

[0058] Figure 1 A flowchart of a multimodal leakage monitoring method for chemical storage tanks based on a large model, provided in an embodiment of the present invention;

[0059] Figure 2 This is a framework diagram of a multimodal leakage monitoring system for chemical storage tanks based on a large model, provided for an embodiment of the present invention. Detailed Implementation

[0060] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0063] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0064] This disclosure aims to address the problem that existing multimodal leakage monitoring algorithms for chemical storage tanks rely excessively on pure data-driven approaches and lack deep collaboration with underlying hardware and physical laws. This leads to highly uncertain output deviations (false positives) under complex industrial interference and parameter drift conditions, making it difficult to meet the stringent reliability requirements of explosion-proof linkage systems. Therefore, this disclosure proposes a large-model-based multimodal leakage monitoring method and system for chemical storage tanks to reliably monitor leakage status and execute a safe physical closed loop. The method and system employ a software-hardware collaborative cyber-physical fusion architecture. Through underlying physical cascade modulation of heterogeneous sensor data, inverse three-dimensional spatial calculation, and a hardware time-difference verification mechanism based on real-time sound velocity and temperature compensation, the semantic output of the large model is rigidly constrained by physical laws. This achieves the goal of completely blocking false alarms from pure algorithms and significantly improving the absolute engineering reliability of explosion-proof early warning systems.

[0065] Please refer to Figure 1 , Figure 1 A flowchart of a multimodal leakage monitoring method for chemical storage tanks based on a large model, provided in this embodiment, is illustrated. This embodiment constructs a cyber-physical fusion architecture consisting of an edge computing gateway and multi-source heterogeneous underlying sensors (a high-frequency acoustic emission sensor matrix, an explosion-proof infrared thermal imager, a solid-state lidar, and an IoT sensor array), and completely isolates the visible light sensing mode in the data processing pipeline. The method executes the following steps according to time sequence and hardware triggering logic:

[0066] Step S0: System hardware initialization and parameter preloading.

[0067] S01: During the system power-on phase, the operating system kernel of the edge computing gateway performs hardware abstraction layer initialization, establishes physical communication links with each underlying sensor, and allocates independent direct memory access (DMA) channels and circular buffer queues for the data streams of each sensor.

[0068] S02: The main control central processing unit initiates a read instruction to the non-volatile memory, and accurately preloads all external prior models, physical calibration constants and network weight matrices required for system operation into the high-speed storage medium corresponding to each computing unit according to the hardware architecture characteristics.

[0069] The system specifically instantiates the following values ​​and matrices:

[0070] Digital Signal Processor (DSP) Static Memory Preloading: Mel filter bank weight matrix matching the resonant characteristics of the tank's metal material. Hamming window function for short-time Fourier transform (STFT), window length 1024 points, frame shift 512 points; cutoff frequency of high-pass filter at acoustic emission front end 20kHz; sampling frequency of analog-to-digital converter (ADC) in acoustic emission acquisition channel calibrated to not less than 1MHz.

[0071] Pre-loaded on-chip static random access memory (SRAM) of the Field Programmable Gate Array (FPGA); high-precision time-to-digital converter (TDC) with a clock sampling frequency of at least 100MHz and a signal arrival time detection amplitude threshold set to 30% of full scale; solid-state lidar extrinsic parameter rotation matrix. With translation vector Infrared camera intrinsic parameter matrix and its inverse matrix ; dimension is Recalibration gated mapping vector Weight matrix of point cloud encoding network based on voxel hashing (output feature channels number is) The point cloud coding network is used to extract voxelized features from the original 3D point cloud data and output a spatial feature matrix with a specified number of channels.

[0072] Graphics Processing Unit (GPU) memory preloading: includes 3-layer depthwise separable convolutional structures, with an output feature dimension of... Deep convolutional neural network feature extraction operators and its weight matrix; Gaussian smoothing adjustment parameters .

[0073] Pre-loading of dedicated video memory for the Neural Processing Unit (NPU): Selective state-space model parameter matrix, specifically including the hidden state transition matrix. Input projection matrix Output projection matrix With residual adaptive scaling matrix Initial Implicit State Tensor of Selective State-Space Model Initialize as dimension The zero tensor; the weight parameter matrix of the linear segmentation decoder. With bias vector .

[0074] Central Processing Unit (CPU) main memory preloading: Sensor data polling sampling time interval Set to 100ms; frequency domain energy adaptive scaling factor The sliding median filter window length is set to 5; the environmental background drift constant is... Historical calibration averages of pressure, flow rate, and concentration with standard deviation ;satisfy Dimensionless partial regression calibration coefficients under constraints Standard ambient temperature for storage tanks Slope of the linear mapping operator from thermal imager radiation to temperature With intercept The position vectors of the first and second high-frequency acoustic emission sensors in the three-dimensional world coordinate system. Ultrasonic reference sound velocity and its sound velocity temperature attenuation coefficient in Q345R steel Physical time difference fault tolerance attenuation coefficient The unit of measurement is the reciprocal of a second; the confidence threshold for leak warning. .

[0075] S03: After the above parameters are instantiated, the central processing unit broadcasts the initialization completion flag to the system data bus, and the underlying clock synchronization module sends a global microsecond-level alignment pulse to activate the hardware data acquisition pipeline.

[0076] Step S1: Extract the acoustic timing signal and capture the physical time difference of the hardware.

[0077] S101: A high-frequency acoustic emission sensor deployed on the outer wall of the storage tank captures transient elastic stress waves in real time. The signal is processed by a hardware preamplifier and a pre-loaded high-pass filter, then converted into a one-dimensional acoustic time-series digital signal by an analog-to-digital converter. The digital signal processor within the edge computing gateway performs sliding truncation using a pre-loaded Hamming window configured in the cache, maps it to the complex frequency domain using a short-time Fourier transform, and calls the pre-loaded Mel filter bank weight matrix to perform matrix multiplication, outputting a two-dimensional acoustic time-frequency energy matrix.

[0078] ;

[0079] in: Indicates time step Frequency Index The two-dimensional acoustic time-frequency energy matrix below; This represents the logarithmic compression operator for suppressing broadband white noise; This represents the preloaded Mel filter bank weight matrix; Represents the short-time Fourier transform operation in the complex field; This represents the acquired one-dimensional acoustic time-series digital signal, with the measurement dimension in millivolts; This represents the operation of squaring the modulus of a complex number.

[0080] S102: The digital signal processor performs a definite integral operation along the frequency axis on the above two-dimensional acoustic time-frequency energy matrix, aggregating and generating scalar weights that digitally represent the abrupt resonant state of the physical metal wall:

[0081] ;

[0082] in: Indicates time step The generated acoustic anomaly attention weight scalar; This represents the normalized activation function; This represents the preloaded frequency domain energy adaptive scaling factor; This represents the integral summation operation along the frequency dimension.

[0083] S103: During the synchronization period of acoustic signal digitization, the high-precision time-to-digital converter (TDC) built into the FPGA compares the pre-loaded amplitude threshold to capture the characteristic arrival times of the two high-frequency acoustic emission sensor signals, and calculates the true hardware physical time difference between the two stress waves.

[0084] ;

[0085] in: This represents the actual physical time difference in the reception of transient stress wave signals by two high-frequency acoustic emission sensors, with the dimension of seconds. and These represent the absolute arrival time scalars of the first and second acoustic wave signals, locked by the TDC hardware timestamp, when they exceed the set amplitude threshold.

[0086] S104: Edge computing gateway performs parallel bus transmission operation: Digital signal processor transfers acoustic anomaly attention weight scalar via DMA channel. Control registers written to shared memory; FPGA synchronization reduces physical time differences. Write to the specified address of the shared memory segment; after the dual write verification is completed, the bus controller jointly triggers a hardware-level interrupt request for Fast Peripheral Component Interconnect (PCIe) to activate the GPU feature extraction link.

[0087] Step S2: Generate the modulated infrared thermo-spatiotemporal difference feature matrix.

[0088] S201: The explosion-proof infrared thermal imager continuously writes thermal radiation image queues to the gateway's video memory; in response to the hardware interrupt issued in step S104, the GPU memory controller extracts the current frame and adjacent historical frames from the head of the queue.

[0089] S202: The GPU executes parallel pixel-wise differential convolution operations using a single-instruction multithreaded (SIMT) architecture, extracts acoustic anomaly attention weights from shared memory and configures them as physical gain coefficients for the activation function; this mechanism dynamically amplifies the network's extraction gain for small temperature gradients using acoustic control, and preserves the spatial feature array by calculating element-wise alignment.

[0090] ;

[0091] in: Indicates time step The generated modulated infrared thermo-spatiotemporal difference feature matrix has a spatial dimension of ; Represented by natural constant An exponential function with base 0; This represents a pre-loaded feature extraction operator for a deep convolutional neural network; Indicates time step The current frame infrared thermal image matrix; Indicates time step Historical frame infrared thermal image matrix; The element-wise squaring operator represents the tensor level; This represents the preloaded Gaussian smoothing adjustment parameter.

[0092] S203: Modulated feature matrix The GPU resides in the GPU memory and waits to be assembled; the GPU sends a thermal branch data ready signal to the main control CPU, and the CPU allocates an independent thread to start the IoT physical quantity polling process through the underlying industrial serial port.

[0093] Step S3: Aggregate and update the dimensionless IoT cyber-physical risk index scalar.

[0094] S301: The edge computing gateway acquires raw sensor values ​​based on the Modbus protocol; the central processing unit's built-in streaming data processing engine calls a pre-loaded sliding median filter to remove outliers, and calculates the difference between the first-order time derivative of pipeline pressure and spatial flow rate based on time step intervals.

[0095] ;

[0096] ;

[0097] in: Indicates time step The instantaneous first derivative of internal pipeline pressure with respect to time; and These represent the measured values ​​of the absolute pressure inside the pipeline network obtained at the current time step and the previous time step, respectively. This indicates the polling sampling time interval for the pre-loaded sensors; Indicates time step The spatial flow difference between the inlet and outlet of the storage tank; and These represent the instantaneous measured values ​​of the electromagnetic flowmeters at the feed inlet and discharge outlet, respectively, representing the current time step.

[0098] S302: To eliminate the dimensional and order-of-magnitude differences between heterogeneous physical quantities, the central processing unit explicitly expands the Z-score normalization equation, calls the pre-loaded historical calibration mean and standard deviation, and performs a dot product with the partial regression calibration coefficients to aggregate them into a cyber-physical risk index.

[0099] ;

[0100] in: Indicates time step An updated dimensionless scalar index for the cyber-physical risk of the Internet of Things; Indicates time step The highest concentration of locally emitted volatile gases was measured. These represent the calibrated average values ​​of the first derivative of the preloaded pressure, the flow rate difference, and the gas concentration, respectively. These represent the calibration standard deviations of the corresponding physical quantities preloaded; This represents the dimensionless partial regression calibration coefficients for the preloaded data. This represents the preloaded environmental background drift compensation constant.

[0101] S303: Scalar The message is pushed into the central processing unit's message queue; the FPGA listens for this scalar through the internal bus and triggers the point cloud channel recalibration state machine.

[0102] Step S4: Perform channel-level spatial feature recalibration.

[0103] S401: The solid-state LiDAR streams 3D point cloud data to the FPGA, and extracts the number of channels through a point cloud encoding network. The initial three-dimensional spatial feature matrix .

[0104] S402: The FPGA uses the received information physical risk index scalar to linearly map a gate weight vector along the channel dimension; this operation performs a Hadamard product with the feature matrix through a spatial broadcast mechanism, adaptively amplifying the extremely weak channel features of aerosol diffuse reflection.

[0105] ;

[0106] in: Indicates time step The three-dimensional spatial feature matrix after risk recalibration; This represents the initially extracted three-dimensional spatial feature matrix; This represents the Hadamard product operator; The dimension indicating preloading is The recalibrated gated mapping vector.

[0107] S403: Feature Matrix The data is transmitted to the system's main memory via the PCIe channel; after the central processing unit's main control thread performs memory barrier verification, it wakes up the NPU to start the projection alignment task.

[0108] Step S5: Perform rasterized perspective projection and cross-modal semantic reconstruction.

[0109] S501: The FPGA's built-in spatial alignment coprocessor calls the pre-loaded intrinsic and extrinsic parameter matrices, executes the 3D-to-2D physical mapping projection equation, and generates a 2D radar depth feature matrix that strictly matches the resolution of the infrared image.

[0110] ;

[0111] in: and Represents the horizontal and vertical coordinate indices of pixels in a two-dimensional photosensitive array; This represents the depth scalar value mapped to the camera's physical coordinate system. This represents the pre-loaded intrinsic parameter matrix of the infrared camera; and These represent the rotation matrix and translation vector of the pre-loaded solid-state lidar extrinsic parameters, respectively. , , This represents the three-dimensional spatial components of point cloud data in the radar coordinate system.

[0112] Based on the above mapping, the system generates a two-dimensional radar depth feature matrix. , where matrix elements The physical depth value is stored pixel by pixel.

[0113] S502: To model the temporal evolution of leakage characteristics and suppress false detections in single frames, the NPU tensor core converts the two-dimensional radar depth feature matrix... Tensor concatenation is performed along the feature channel dimension with the modulated infrared thermo-spatiotemporal difference feature matrix, and the input is a selective state-space model with a pre-loaded zero tensor. Iteratively update the hidden state tensor for the initial value:

[0114] ;

[0115] in: Indicates time step The updated hidden state tensor has dimension . ; , This represents the preloaded implicit state transition matrix and input projection matrix; Indicates time step The historical implicit state tensor; This represents the tensor concatenation operator along the depth dimension of the feature channel.

[0116] S503: To preserve the detailed features of the underlying infrared differential model, a residual adaptive scaling branch is introduced and fused with the output features of the selective state-space model; the NPU performs decoding feature projection and outputs a two-dimensional cross-modal fused leaked semantic feature tensor.

[0117] ;

[0118] in: Indicates time step The two-dimensional cross-modal fusion leaky semantic feature tensor output by the mapping; This represents the preloaded output projection matrix; This represents the preloaded residual adaptive scaling matrix.

[0119] S504: NPU will use tensors Write to the memory-mapped I / O zone, the central processing unit's daemon process captures data changes, and schedules the temperature compensation module to start the reverse spatial physical verification stream in step S6.

[0120] Step S6: Perform reverse three-dimensional solution and real-time temperature and sound velocity compensation.

[0121] S601: The cross-modal fusion features are mapped to a pixel-level probability distribution using a linear segmentation decoder. After performing class probability normalization along the feature channel dimension, a two-dimensional spatial probability distribution map of a specific leakage category is extracted. Then, the global spatial extremum is searched to pinpoint the precise pixel location where the leakage occurs.

[0122] ;

[0123] ;

[0124] ;

[0125] in: Indicates time step The two-dimensional pixel-level leakage segmentation probability matrix; This represents an activation operator that extracts the two-dimensional spatial probability distribution of a specific class after performing class probability normalization along the feature channel dimension. , This represents the weight parameter matrix and bias vector of the pre-loaded segment decoder; This represents a scalar value indicating the probability of leakage of local maxima in the extracted two-dimensional image. , This represents the global extremum search operator and the extremum independent variable coordinate extraction operator; Indicates time step In pixel coordinates The scalar value of the leakage probability at the location; The x and y coordinates of the two-dimensional pixels corresponding to the probability of the maximum value of the locked leakage are represented.

[0126] S602: Based on the principle of acoustic emission triangulation, the central processing unit extracts the infrared physical radiation values ​​of extreme pixel regions and converts them into actual temperature differences. It then uses the inverse projection matrix to calculate the theoretical propagation path difference from the leakage source to the two sensors, providing a theoretical benchmark value for subsequent physical consistency verification.

[0127] ;

[0128] in: It represents the difference between the actual average temperature of the tank wall in the leakage extreme point area and the standard temperature, with the measurement dimension being degrees Celsius; , This represents the slope and intercept of the preloaded thermal imager linear mapping operator; Indicates the extreme coordinates of the current frame infrared image. The pixel value of the radiation intensity at that location; This indicates the standard ambient temperature of the storage tank.

[0129] ;

[0130] in: The difference in Euclidean distance between the three-dimensional spatial coordinates of the leakage source calculated by reverse engineering and the theoretical three-dimensional propagation path of the two sets of high-frequency acoustic emission sensors is expressed in meters. Represents the L2 norm operator for three-dimensional Euclidean distance; This represents the inverse matrix of the pre-loaded infrared camera intrinsic parameters; A homogeneous vector representing the two-dimensional pixel coordinates of extreme values; This represents the precise physical depth scalar value at the extreme coordinates of the two-dimensional radar depth feature matrix generated in step S501. , This represents the three-dimensional position vector of the pre-loaded high-frequency acoustic emission sensor.

[0131] Step S7: Perform nonlinear physical clamping calculations and hard-wired blocking warnings.

[0132] S701: Based on the physical characteristic that the ultrasonic velocity in Q345R steel decreases linearly with increasing temperature, the central processing unit performs real-time temperature correction on the reference sound velocity; a dynamic weighted fusion mechanism is adopted, using the working condition risk index as the confidence weight, fusing semantic probability and physical time difference matching degree to achieve physical hard constraints on the model output.

[0133] ;

[0134] ;

[0135] in: This represents the real-time ultrasonic velocity after temperature drift difference compensation, with the measurement dimension being meters per second; , This indicates the preloaded reference sound velocity and the sound velocity temperature attenuation coefficient. This represents the temperature drift difference in the leakage extreme point region calculated previously; This represents a scalar of dynamic confidence weight coefficients determined by physical conditions.

[0136] Closed-loop joint overall leakage joint early warning confidence score scalar:

[0137] ;

[0138] in: The final output is represented by the overall leakage joint early warning confidence score scalar, normalized to the interval (0,1]. Represents a negative exponential physical penalty operator; This represents the preloaded physical time difference tolerance attenuation coefficient.

[0139] S702: After the floating-point unit of the central processing unit completes the fixed-point calculation, it will evaluate the scalar value. Write to the watchdog register of the independent security monitoring module's general-purpose input / output (GPIO); respond to the value exceeding the pre-loaded leak warning confidence threshold. The GPIO pin outputs a strong current pulse, which hardwires the field circuit relay to close, cutting off the power supply to the pneumatic regulating valve and activating the explosion-proof alarm system; the whole method completes a highly reliable and safe closed loop in the information processing space and the physical execution space.

[0140] Please refer to Figure 2 , Figure 2This diagram illustrates a framework of a large-model-based multimodal leak monitoring system for chemical storage tanks, as provided in this embodiment. The system is deployed within a cyber-physical fusion architecture comprised of an edge computing gateway and multi-source heterogeneous underlying sensors. The edge computing gateway integrates a digital signal processor, a field-programmable gate array (FPGA), a graphics processing unit (GPU), a neural processing unit (NNF), and a central processing unit (CPU). Specifically, the system includes eight modules that correspond entirely to the steps described above:

[0141] The hardware initialization and parameter preloading module is used to preload the external prior model, physical calibration constants and network weight matrix into the high-speed storage medium corresponding to each computing unit through the central processing unit, and to complete clock synchronization and pipeline activation.

[0142] The acoustic timing signal extraction and hardware physical time difference capture module is used to perform two-dimensional time-frequency reconstruction on the one-dimensional acoustic timing digital signal collected by the acoustic emission sensor through a digital signal processor, aggregate and generate an acoustic anomaly attention weight scalar, and simultaneously capture the real hardware physical time difference of the two stress wave signals through the time-to-digital converter built into the field programmable gate array.

[0143] The infrared thermal spatiotemporal differential modulation module is used to perform differential convolution operation on the infrared thermal image frame sequence acquired by the explosion-proof infrared thermal imager through the graphics processing unit, and inject the acoustic anomaly attention weight scalar as the physical gain coefficient to generate the modulated infrared thermal spatiotemporal differential feature matrix.

[0144] The IoT operating condition feature extraction and dimensionless conversion module is used to collect raw operating condition values ​​based on the Modbus protocol, call the pre-loaded sliding median filter parameters to clean outliers, and perform time series feature extraction and standardization processing through the central processing unit. Combined with the partial regression calibration coefficient, it aggregates and generates a dimensionless IoT cyber-physical risk index scalar.

[0145] The channel-level spatial feature recalibration module is used to extract the initial three-dimensional spatial feature matrix of solid-state LiDAR through field-programmable gate array, and to perform channel-level spatial feature recalibration on the initial three-dimensional spatial feature matrix by mapping the dimensionless IoT cyber-physical risk index scalar to the gate control weight vector.

[0146] The rasterized perspective projection and cross-modal semantic reconstruction module is used to perform rasterized perspective projection on the recalibrated three-dimensional spatial feature matrix through the spatial alignment coprocessor built into the field programmable gate array, generate a two-dimensional radar depth feature map, and then stitch it with the modulated infrared thermal spatiotemporal difference feature matrix. After inputting the selective state space model through the neural processing unit, the hidden state is iterated, and the two-dimensional cross-modal fusion leaky semantic feature tensor is output.

[0147] The inverse 3D solution and real-time temperature and sound velocity compensation module is used to extract the local maximum leakage probability and the corresponding leakage extreme pixel horizontal and vertical coordinates from the 2D cross-modal fusion leakage semantic feature tensor through the central processing unit, call the inverse projection matrix to reverse solve the horizontal and vertical coordinates of the leakage extreme pixel into the difference of the Euclidean distance of the theoretical 3D propagation path; extract the infrared radiation of the extreme pixel area to calculate the real temperature difference, and perform real-time temperature compensation correction on the ultrasonic reference sound velocity.

[0148] The nonlinear physical clamping and hard-wired interlocking early warning module is used to use the dimensionless IoT information physical risk index scalar as a dynamic confidence weight through the central processing unit. It performs nonlinear physical clamping calculation by combining the difference between the local maximum leakage probability, the Euclidean distance of the theoretical three-dimensional propagation path, and the actual hardware physical time difference in a closed loop. It outputs an overall leakage joint early warning confidence score scalar and triggers the closing of the field circuit relay when the pre-loaded leakage early warning confidence threshold is exceeded.

[0149] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multimodal leakage monitoring method for chemical storage tanks based on a large model, applied to a cyber-physical fusion architecture composed of an edge computing gateway and multi-source heterogeneous sensors, wherein the edge computing gateway integrates a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit, characterized in that... The method includes the following steps: S1. Extracting acoustic timing signals and capturing hardware physical time difference: Through a digital signal processor, a two-dimensional time-frequency reconstruction is performed on the one-dimensional acoustic timing digital signal collected by the acoustic emission sensor, which is aggregated to generate an acoustic anomaly attention weight scalar, and the real hardware physical time difference of the two stress wave signals is captured simultaneously through the time-to-digital converter built into the field programmable gate array. S2. Generate the modulated infrared thermal spatiotemporal difference feature matrix: Perform differential convolution operation on the infrared thermal image frame sequence acquired by the explosion-proof infrared thermal imager through the graphics processing unit, and inject the acoustic anomaly attention weight scalar as the physical gain coefficient to generate the modulated infrared thermal spatiotemporal difference feature matrix. S3. Aggregate and update the dimensionless IoT cyber-physical risk index scalar: The central processing unit performs time-series feature extraction and standardization processing on the working condition data collected by heterogeneous IoT sensors, and combines it with the partial regression calibration coefficient to aggregate and generate the dimensionless IoT cyber-physical risk index scalar. S4. Perform channel-level spatial feature recalibration: Extract the initial three-dimensional spatial feature matrix of the solid-state lidar through the field-programmable gate array, and use the dimensionless IoT cyber-physical risk index scalar to map the gate control weight vector to perform channel-level spatial feature recalibration on the initial three-dimensional spatial feature matrix. S5. Perform rasterized perspective projection and cross-modal semantic reconstruction: The recalibrated three-dimensional spatial feature matrix is ​​rasterized through the spatial alignment coprocessor built into the field programmable gate array to generate a two-dimensional radar depth feature map. After the map is stitched together with the modulated infrared thermal spatiotemporal difference feature matrix, the hidden state is iterated through the input selective state space model via the neural processing unit, and the two-dimensional cross-modal fusion leaky semantic feature tensor is output. S6. Perform inverse 3D calculation and real-time temperature and sound velocity compensation: The central processing unit extracts the local maximum leakage probability and the corresponding horizontal and vertical coordinates of the leakage extreme pixel from the 2D cross-modal fusion leakage semantic feature tensor. The inverse projection matrix is ​​called to reverse calculate the difference of the Euclidean distance of the leakage extreme pixel into the theoretical 3D propagation path. The infrared radiation of the extreme pixel region is extracted to calculate the real temperature difference, and real-time temperature compensation correction is performed on the ultrasonic reference sound velocity. S7. Perform nonlinear physical clamping operation and hard-wired interlocking early warning: The central processing unit uses the dimensionless IoT information physical risk index scalar as a dynamic confidence weight, performs nonlinear physical clamping operation by combining the difference between the local maximum leakage probability, the Euclidean distance of the theoretical three-dimensional propagation path and the actual hardware physical time difference in a closed loop, outputs the overall leakage joint early warning confidence score scalar, and triggers the field loop relay to close when the overall leakage joint early warning confidence score scalar exceeds the preloaded leakage early warning confidence threshold.

2. The large model-based chemical tank multi-modal leakage monitoring method according to claim 1, characterized in that, The extraction of acoustic timing signals and the physical time difference of the capture hardware in S1 specifically includes: A short-time Fourier transform is performed on the one-dimensional acoustic time-series digital signal, and the pre-loaded Mel filter bank weight matrix is ​​used for filtering to generate a two-dimensional acoustic time-frequency energy matrix. After integration and summation along the frequency dimension, the normalized activation function is applied to obtain the acoustic anomaly attention weight scalar. The absolute arrival time scalar of the two stress wave signals breaking through the pre-loaded amplitude threshold is captured by the time-to-digital converter, and the absolute value of the difference between the two is calculated to obtain the actual hardware physical time difference.

3. The large model-based chemical tank multi-modal leakage monitoring method according to claim 1, characterized in that, The modulated infrared thermo-spatiotemporal difference feature matrix generated in S2 is specifically calculated using the following formula: ； in, Indicates time step The generated modulated infrared thermo-spatiotemporal difference feature matrix; This represents the acoustic anomaly attention weight scalar; Represented by natural constant An exponential function with base 0; This represents a pre-loaded feature extraction operator for a deep convolutional neural network; Indicates time step The current frame infrared thermal image matrix; Indicates time step Historical frame infrared thermal image matrix; The element-wise squaring operator represents the tensor level; This represents the preloaded Gaussian smoothing adjustment parameter.

4. The method for multimodal leakage monitoring of chemical storage tanks based on a large model according to claim 1, characterized in that, The S3 method for aggregating and updating the dimensionless IoT cyber-physical risk index scalar includes: The system collects raw operating data from heterogeneous IoT sensors based on the Modbus protocol; it calls pre-loaded sliding median filter parameters to clean outliers in the raw operating data; and it calculates the instantaneous first derivative of the current pipeline pressure with respect to time and the spatial flow difference at the inlet and outlet of the storage tank based on the sensor polling sampling time interval. The pre-loaded calibration mean and calibration standard deviation are respectively called to perform Z-score standardization on the instantaneous first derivative, the spatial flow difference, and the collected local volatile gas highest concentration measurement value; The standardized values ​​are multiplied by their corresponding dimensionless partial regression calibration coefficients and summed. Combined with the environmental background drift compensation constant, the dimensionless Internet of Things cyber-physical risk index scalar is generated.

5. The large model-based chemical storage tank multi-modal leakage monitoring method according to claim 1, characterized in that, In S4, channel-level spatial feature recalibration is performed, specifically calculated using the following formula: ； in, Indicates time step The three-dimensional spatial feature matrix after risk recalibration; This represents the initially extracted three-dimensional spatial feature matrix; This represents the Hadamard product operator; This represents the preloaded recalibration gated mapping vector; This represents the dimensionless Internet of Things cyber-physical risk index scalar; This represents the normalized activation function.

6. The large model-based chemical storage tank multi-modal leakage monitoring method according to claim 1, characterized in that, The S5 process performs rasterized perspective projection and cross-modal semantic reconstruction, specifically including: The pre-loaded infrared camera intrinsic parameter matrix and solid-state lidar extrinsic parameter rotation matrix and translation vector are called to perform rasterization perspective projection and out-of-bounds truncation processing on the recalibrated three-dimensional spatial feature matrix, generating a two-dimensional radar depth feature map with pixel-by-pixel corresponding physical depth values. generating a corresponding depth matrix according to the mapping , which is spliced along the feature channel depth dimension with the modulated infrared thermal space-time difference feature matrix, and then the cross-modal semantic reconstruction is performed through the following tensor iteration formula: ； ； in, Represents the depth matrix; Indicates time step Update the hidden state tensor; and This represents the preloaded implicit state transition matrix and input projection matrix; Indicates time step The historical implicit state tensor; This represents the tensor concatenation operator along the depth dimension of the feature channel; This represents the modulated infrared thermo-spatial-differential feature matrix; Indicates time step The two-dimensional cross-modal fusion leaky semantic feature tensor output by the mapping; This represents the preloaded output projection matrix; This represents the preloaded residual adaptive scaling matrix.

7. The large model-based chemical storage tank multi-modal leakage monitoring method according to claim 6, characterized in that, The reverse three-dimensional solution and real-time temperature and sound speed compensation performed in S6 specifically include: The two-dimensional cross-modal fusion leakage semantic feature tensor is mapped to a pixel-level probability distribution by a linear segmentation decoder, class probability normalization is performed and global spatial extrema are searched to obtain the local maximum leakage probability and the corresponding horizontal and vertical coordinates of the leakage extremum pixel. The difference between the actual temperature difference and the Euclidean distance of the theoretical three-dimensional propagation path is calculated using the following formula: ； ； in, This represents the difference between the actual average temperature of the tank wall and the standard temperature in the area of ​​the leakage extreme point. , This represents the slope and intercept of the preloaded thermal imager linear mapping operator; The x and y coordinates of the pixel representing the leakage extreme value; This represents the pixel value of the radiation intensity at the horizontal and vertical coordinates of the leakage extreme pixel in the current frame of the infrared image; Indicates the standard ambient temperature of the storage tank; This represents the difference in Euclidean distance between the theoretical three-dimensional propagation paths; Represents the L2 norm operator for three-dimensional Euclidean distance; This represents the inverse matrix of the pre-loaded infrared camera intrinsic parameters; A homogeneous vector representing the two-dimensional pixel coordinates of extreme values; Represents the depth matrix The precise physical depth scalar value at the horizontal and vertical coordinates of the leaking extreme pixel; and This represents the three-dimensional position vectors of the pre-loaded first and second high-frequency acoustic emission sensors.

8. The large model-based chemical storage tank multi-modal leakage monitoring method according to claim 7, characterized in that, In S7, nonlinear physical clamping operations and hard-wired interlocking early warning are performed. Specifically, the overall leakage joint early warning confidence score scalar is calculated using the following formula. When this score scalar exceeds the pre-loaded leakage early warning confidence threshold, the field loop relay is triggered to close via the general-purpose input / output watchdog register: ； ； ； in, This represents the real-time ultrasonic velocity after temperature drift difference compensation. and This indicates the pre-loaded ultrasonic reference velocity and the velocity-temperature attenuation coefficient. This represents the actual temperature difference; This represents the dimensionless Internet of Things cyber-physical risk index scalar. The dynamic confidence weight coefficient scalar is determined; This represents the overall leak joint early warning confidence score scalar; This represents the probability of leakage from the local maximum. Represents a negative exponential physical penalty operator; This represents the preloaded physical time difference tolerance attenuation coefficient; This represents the actual physical time difference of the hardware. This represents the difference in Euclidean distance between the theoretical three-dimensional propagation paths.

9. A multimodal leakage monitoring system for chemical storage tanks based on a large model, deployed in a cyber-physical fusion architecture composed of an edge computing gateway and multi-source heterogeneous sensors, wherein the edge computing gateway integrates a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit, characterized in that... The system is used to implement the multimodal leakage monitoring method for chemical storage tanks based on a large model as described in any one of claims 1 to 8, and the system includes: The hardware initialization and parameter preloading module is used to preload the external prior model, physical calibration constants and network weight matrix into the high-speed storage medium corresponding to each computing unit through the central processing unit, and to complete clock synchronization and pipeline activation. The acoustic timing signal extraction and hardware physical time difference capture module is used to perform two-dimensional time-frequency reconstruction on the one-dimensional acoustic timing digital signal collected by the acoustic emission sensor through a digital signal processor, aggregate and generate an acoustic anomaly attention weight scalar, and simultaneously capture the real hardware physical time difference of the two stress wave signals through the time-to-digital converter built into the field programmable gate array. The infrared thermal spatiotemporal differential modulation module is used to perform differential convolution operation on the infrared thermal image frame sequence acquired by the explosion-proof infrared thermal imager through the graphics processing unit, and inject the acoustic anomaly attention weight scalar as the physical gain coefficient to generate the modulated infrared thermal spatiotemporal differential feature matrix. The IoT operating condition feature extraction and dimensionless conversion module is used to collect raw operating condition values ​​based on the Modbus protocol, call the pre-loaded sliding median filter parameters to clean outliers, and perform time series feature extraction and standardization processing through the central processing unit. Combined with the partial regression calibration coefficient, it aggregates and generates a dimensionless IoT cyber-physical risk index scalar. The channel-level spatial feature recalibration module is used to extract the initial three-dimensional spatial feature matrix of solid-state LiDAR through field-programmable gate array, and to perform channel-level spatial feature recalibration on the initial three-dimensional spatial feature matrix by mapping the dimensionless IoT cyber-physical risk index scalar to the gate control weight vector. The rasterized perspective projection and cross-modal semantic reconstruction module is used to perform rasterized perspective projection on the recalibrated three-dimensional spatial feature matrix through the spatial alignment coprocessor built into the field programmable gate array, generate a two-dimensional radar depth feature map, and then stitch it with the modulated infrared thermal spatiotemporal difference feature matrix. After inputting the selective state space model through the neural processing unit, the hidden state is iterated, and the two-dimensional cross-modal fusion leaky semantic feature tensor is output. The inverse 3D solution and real-time temperature and sound velocity compensation module is used to extract the local maximum leakage probability and the corresponding leakage extreme pixel horizontal and vertical coordinates from the 2D cross-modal fusion leakage semantic feature tensor through the central processing unit, call the inverse projection matrix to reverse solve the horizontal and vertical coordinates of the leakage extreme pixel into the difference of the Euclidean distance of the theoretical 3D propagation path; extract the infrared radiation of the extreme pixel area to calculate the real temperature difference, and perform real-time temperature compensation correction on the ultrasonic reference sound velocity. The nonlinear physical clamping and hard-wired interlocking early warning module is used to use the dimensionless IoT information physical risk index scalar as a dynamic confidence weight through the central processing unit. It performs nonlinear physical clamping calculation by combining the difference between the local maximum leakage probability, the Euclidean distance of the theoretical three-dimensional propagation path, and the actual hardware physical time difference in a closed loop. It outputs an overall leakage joint early warning confidence score scalar and triggers the closing of the field circuit relay when the pre-loaded leakage early warning confidence threshold is exceeded.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor comprises a digital signal processor, a field-programmable gate array, a graphics processing unit, a neural processing unit, and a central processing unit that form an edge computing gateway. When each processing unit works together to run the computer program, it implements the multimodal leakage monitoring method for chemical storage tanks based on a large model as described in any one of claims 1 to 8.

Images