Optical device for online monitoring of equipment gas leakage and monitoring method thereof

Abstract

The present application relates to the field of intelligent optical equipment, and specifically to an optical device for online monitoring of equipment gas leakage and a monitoring method thereof. The optical device includes: an infrared camera and a data processor, wherein the infrared camera is used to collect infrared monitoring videos of the monitored equipment, and the data processor is used to process the infrared monitoring videos to obtain monitoring results, wherein the monitoring results are location data of equipment leakage points. Then, by introducing image processing and analysis algorithms based on deep learning at the back end for processing, more intelligent equipment leakage monitoring and leakage point location can be achieved, so that when equipment leakage occurs, a rapid response can be made according to the location of the leakage point, providing safety protection for enterprises and reducing potential environmental risks and economic losses.

Description

Technical Field

[0001] The present application relates to the field of intelligent optical equipment, and in particular to an optical equipment for online monitoring of gas leakage and a monitoring method thereof. Background Art

[0002] With the acceleration of industrialization, equipment systems are widely used in energy, chemical, petroleum and other fields due to their high efficiency and economy. At present, equipment leakage incidents occur from time to time. Equipment leakage problems not only cause economic losses, but also may pose serious threats to the environment and human health. Therefore, it is particularly important to monitor equipment leakage in real time and take corresponding measures in time.

[0003] However, traditional equipment leak detection methods mainly rely on manual inspections and acoustic monitoring. Manual inspections usually take up a lot of time and human resources to cover a wide network of equipment, which results in relatively low detection efficiency. In addition, manual inspections are susceptible to fatigue, distraction, or professional skill levels of staff, which may lead to inaccurate or missed equipment leak detection results. Although the acoustic monitoring method does not require manual inspections to spend a lot of time and energy, it may be affected by background noise in a noisy environment and cannot accurately distinguish different types of sound signals, reducing the accuracy of equipment leak detection. At the same time, the positioning accuracy of equipment leak points is limited.

[0004] Therefore, an optimized optical device for on-line monitoring of gas leaks is desired. Summary of the invention

[0005] This application is made in view of the above problems. One object of this application is to provide an optical device for online monitoring of gas leakage and a monitoring method thereof.

[0006] An embodiment of the present application provides an optical device for online monitoring of equipment gas leakage, which includes an infrared camera and a data processor. The infrared camera is used to collect infrared monitoring video of the monitored equipment, and the data processor is used to process the infrared monitoring video to obtain monitoring results, and the monitoring results are the location and flow data of the equipment leakage point.

[0007] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the data processor includes:

[0008] A moving object shielding module, used for shielding moving objects in each infrared image frame in the infrared surveillance video to obtain a shielded infrared surveillance video;

[0009] An infrared image frame normalization module, used for normalizing each infrared image frame in the infrared surveillance video after the shielding process to obtain a normalized infrared surveillance video;

[0010] A morphological processing module, used for performing morphological processing on each infrared image frame in the normalized infrared surveillance video to obtain a morphologically processed infrared surveillance video;

[0011] A threshold binarization module, used for performing a threshold-based binarization process on each infrared image frame in the infrared surveillance video after the morphological process to obtain a binarized infrared surveillance video;

[0012] A contour extraction module, used for performing contour extraction on each infrared image frame in the infrared monitoring video after the binarization process to obtain an infrared monitoring video of an air mass contour;

[0013] The equipment leakage point location detection module is used to determine the location data of the equipment leakage point based on the infrared monitoring video of the air mass contour.

[0014] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the moving object shielding module is used to:

[0015] A mixed Gaussian background modeling algorithm is used to perform statistical analysis on the pixel brightness values ​​of each infrared image frame, and pixel points that exceed the normal brightness distribution range are shielded to obtain the infrared monitoring video after shielding processing.

[0016] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the infrared image frame normalization module is used to:

[0017] Each infrared image frame in the infrared surveillance video after the shielding process is normalized by the following normalization processing formula to obtain the infrared surveillance video after the normalization process; wherein the normalization processing formula is:

[0018]

[0019] in, for each infrared image frame, represents the minimum value of each infrared image frame, represents the maximum value of each infrared image frame, It is each infrared image frame in the infrared surveillance video after the normalization processing.

[0020] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the contour extraction module is used to:

[0021] The findContour function is used to extract the contour of each infrared image frame in the binarized infrared monitoring video to obtain the air mass contour infrared monitoring video.

[0022] For example, according to an optical device for online monitoring of gas leakage in an embodiment of the present application, the equipment leakage point location detection module includes:

[0023] An optical flow algorithm processing unit, used to process the infrared monitoring video of the air mass contour using an optical flow algorithm to obtain a set of pixel optical flow feature vectors;

[0024] A semantic association enhancement unit, configured to perform semantic association enhancement on each pixel optical flow feature vector in the set of pixel optical flow feature vectors based on feature energy level measurement associated radiation to obtain a set of enhanced pixel optical flow feature vectors;

[0025] A feature aggregation unit, used for inputting the set of enhanced pixel optical flow feature vectors into a feature aggregation network based on feature local energy distribution gating to obtain an air mass motion optical flow aggregation representation vector;

[0026] The decoding unit is used to input the air mass movement optical flow aggregation representation vector into a decoder-based leakage source locator to obtain the position data and flow data of the equipment leakage point.

[0027] For example, in an optical device for online monitoring of gas leakage according to an embodiment of the present application, the semantic association strengthening unit includes:

[0028] A characteristic energy level coefficient calculation subunit, configured to calculate, for the set of pixel optical flow feature vectors, a characteristic energy level coefficient of each pixel optical flow feature vector in the set of pixel optical flow feature vectors to obtain a sequence of pixel optical flow characteristic energy level coefficients;

[0029] A coefficient arrangement subunit, used for arranging the sequence of pixel optical flow feature energy level coefficients into a pixel optical flow feature energy level coefficient input vector;

[0030] A one-dimensional convolutional encoding subunit, used for performing one-dimensional convolutional encoding on the pixel optical flow feature energy level coefficient input vector to obtain a pixel optical flow feature energy level radiation correlation vector;

[0031] A point convolution encoding subunit, used for performing point convolution encoding on the pixel optical flow feature energy level radiation association vector to obtain a pixel optical flow feature energy level radiation convolution association vector;

[0032] A probabilistic mapping subunit, used for inputting the pixel optical flow feature energy level radiation convolution association vector into a sigmoid function for probabilistic mapping to obtain a pixel optical flow feature energy level radiation association weight vector;

[0033] The enhancement subunit is used to use the characteristic value of each position in the pixel optical flow characteristic energy level radiation associated weight vector as the weight value, perform vector-by-vector point multiplication on the set of pixel optical flow characteristic vectors by position, and add the set of pixel optical flow characteristic vectors to obtain the set of enhanced pixel optical flow characteristic vectors.

[0034] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the characteristic energy level coefficient calculation subunit is used to:

[0035] Extracting the maximum value of the feature at each position in the pixel optical flow feature vector to obtain the maximum value of the pixel optical flow feature;

[0036] Calculating the mean and variance of the pixel optical flow feature vector respectively to obtain a pixel optical flow feature mean and a pixel optical flow feature variance;

[0037] The value obtained by adding the pixel optical flow feature variance and the preset hyperparameter is multiplied by four to obtain the first energy level coefficient of the pixel optical flow feature;

[0038] Calculate the square of the difference between the maximum value of the pixel optical flow feature and the mean value of the pixel optical flow feature to obtain a pixel optical flow feature difference value;

[0039] The pixel optical flow feature second energy level coefficient is obtained by adding a modulated pixel optical flow feature variance obtained by multiplying the pixel optical flow feature variance by two, a value obtained by multiplying the preset hyperparameter by two, and the pixel optical flow feature difference value;

[0040] The first energy level coefficient of the pixel optical flow feature is divided by the second energy level coefficient of the pixel optical flow feature to obtain the pixel optical flow feature energy level coefficient.

[0041] For example, in the optical device for online monitoring of gas leakage according to an embodiment of the present application, the feature aggregation unit includes:

[0042] A coefficient acquisition subunit, configured to determine the characteristic energy level coefficients of each enhanced pixel optical flow feature vector in the set of enhanced pixel optical flow feature vectors based on the maximum value, minimum value, mean value and variance of each enhanced pixel optical flow feature vector to obtain a sequence of pixel optical flow characteristic energy level coefficients;

[0043] A local neighborhood mean calculation subunit is used to determine the scale of the local neighborhood of the pixel optical flow, use the scale of the local neighborhood of the pixel optical flow as the mean calculation range, and respectively calculate the local neighborhood mean of each pixel optical flow feature energy level coefficient to obtain a sequence of pixel optical flow feature energy level significance descriptors;

[0044] A mask subunit, used for inputting the sequence of pixel optical flow feature energy level significance descriptors into a mask module based on a gating function to obtain a sequence of mask-gated probabilistic pixel optical flow feature energy level significance descriptors;

[0045] A position-based weighted sum subunit is used to calculate the position-based weighted sum of the set of enhanced pixel optical flow feature vectors using the sequence of the mask-gated probabilistic pixel optical flow feature energy level significance descriptors as a sequence of weights to obtain the air mass motion optical flow aggregate representation vector;

[0046] Wherein, the coefficient acquisition subunit includes:

[0047] A multi-item calculation secondary subunit, used to calculate the maximum value, minimum value, average value and variance of the enhanced pixel optical flow feature vector to obtain the maximum value of the pixel optical flow, the minimum value of the pixel optical flow, the average value of the pixel optical flow and the variance of the pixel optical flow;

[0048] A first statistical factor calculation secondary subunit is used to calculate the sum of the pixel optical flow variance and the hyperparameter to obtain a first pixel optical flow local energy statistical factor;

[0049] A maximum difference calculation secondary subunit, used for calculating the difference between the maximum value of the pixel optical flow and the average value of the pixel optical flow to obtain a maximum deviation value of the pixel optical flow;

[0050] A minimum difference calculation secondary subunit, used for calculating the difference between the minimum value of the pixel optical flow and the average value of the pixel optical flow to obtain a minimum deviation value of the pixel optical flow;

[0051] A second statistical factor calculation secondary subunit is used to calculate the square sum of the maximum deviation value of the pixel optical flow and the minimum deviation value of the pixel optical flow and then add the square sum to the hyperparameter to obtain a second pixel optical flow local energy statistical factor;

[0052] The second-level division operation subunit is used to calculate the division between the first pixel optical flow local energy statistical factor and the second pixel optical flow local energy statistical factor to obtain the pixel optical flow characteristic energy level coefficient.

[0053] The embodiment of the present application also provides an equipment gas leakage online monitoring method, which includes:

[0054] Collect infrared surveillance video of the monitored equipment;

[0055] Shielding the moving objects in each infrared image frame in the infrared surveillance video to obtain a shielded infrared surveillance video;

[0056] Normalizing each infrared image frame in the infrared surveillance video after the shielding process to obtain a normalized infrared surveillance video;

[0057] Performing morphological processing on each infrared image frame in the normalized infrared surveillance video to obtain a morphologically processed infrared surveillance video;

[0058] Performing a threshold-based binarization process on each infrared image frame in the infrared surveillance video after the morphological processing to obtain a binarized infrared surveillance video;

[0059] Performing contour extraction on each infrared image frame in the binarized infrared monitoring video to obtain an infrared monitoring video of an air mass contour;

[0060] Based on the infrared monitoring video of the air mass contour, the location data and flow data of the equipment leakage point are determined.

[0061] According to the optical equipment and monitoring method for online monitoring of equipment gas leakage in the embodiments of the present application, it can be processed by introducing deep learning-based image processing and analysis algorithms at the back end, thereby achieving more intelligent equipment leakage monitoring and leakage point positioning, so that when equipment leakage occurs, it can respond quickly according to the location of the leakage point, provide safety protection for the enterprise, and reduce potential environmental risks and economic losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0063] Figure 1 A schematic diagram of the structure of an online gas leakage monitoring device in an embodiment of the present application is shown.

[0064] Figure 2 A schematic diagram of the structure of a data processor equipped with an online gas leakage monitoring device in an embodiment of the present application is shown.

[0065] Figure 3 A structural schematic diagram of an equipment leakage point position detection module of an equipment gas leakage online monitoring device in an embodiment of the present application is shown.

[0066] Figure 4 A flow chart of an equipment gas leakage online monitoring method in an embodiment of the present application is shown.

[0067] Figure 5 A diagram showing an application scenario of the equipment gas leakage online monitoring method in an embodiment of the present application is shown.

[0068] Figure 6 A schematic diagram of black smoke detection in an online monitoring method for equipment gas leakage in an embodiment of the present application is shown.

[0069] Figure 7 A diagram showing an application scenario of the equipment gas leakage online monitoring method in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0071] The terms used in this specification are those common terms currently widely used in the art in consideration of the functions of the present application, but these terms may vary according to the intentions of those skilled in the art, precedents, or new technologies in the art. In addition, specific terms may be selected, and in this case, their detailed meanings will be described in the detailed description of the present application. Therefore, the terms used in the specification should not be understood as simple names, but rather as a general description based on the meaning of the terms and the present application.

[0072] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0073] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0074] In addition, the application architecture diagram in the embodiment of the present application is intended to more clearly illustrate the technical solution in the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Of course, for other application architectures and business applications, the technical solution provided in the embodiment of the present application is also applicable to similar problems.

[0075] The following is a non-restrictive description of an optical device for online monitoring of gas leaks provided according to at least one embodiment of the present application through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present application.

[0076] With the advancement of sensor technology and image processing technology, leak detection systems based on video monitoring have gradually become an emerging solution. Among them, infrared imaging technology can use the radiation characteristics within the infrared spectrum to effectively monitor gas leaks and detect leak points under different environmental conditions. By capturing infrared images around the monitored equipment, the system can identify the characteristics of gas leaks and further extract the location data and flow data of the leak point through data processing algorithms.

[0077] Based on this, in the technical solution of this application, an optical device for online monitoring of gas leakage is proposed, such as Figure 1 As shown, it includes: an infrared camera 810 and a data processor 820, wherein the infrared camera 810 is used to collect infrared monitoring videos of the monitored equipment, and the data processor 820 is used to process the infrared monitoring videos to obtain monitoring results, wherein the monitoring results are location data of equipment leakage points.

[0078] In some optional embodiments, a visible light camera 830 is further included, and the visible light camera 830 can further collect environmental information in the environment and perform key image information fusion to facilitate further analysis and optimization of the collected information.

[0079] In the above optical equipment for online monitoring of gas leakage, such as Figure 2As shown, the data processor 820 includes: a moving object shielding module 821, which is used to shield the moving objects in each infrared image frame in the infrared monitoring video to obtain the infrared monitoring video after shielding; an infrared image frame normalization module 822, which is used to normalize each infrared image frame in the infrared monitoring video after shielding to obtain the infrared monitoring video after normalization; a morphological processing module 823, which is used to perform morphological processing on each infrared image frame in the infrared monitoring video after normalization to obtain the infrared monitoring video after morphological processing; a threshold binarization module 824, which is used to perform threshold-based binarization processing on each infrared image frame in the infrared monitoring video after morphological processing to obtain the infrared monitoring video after binarization processing; a contour extraction module 825, which is used to perform contour extraction on each infrared image frame in the infrared monitoring video after binarization processing to obtain the air mass contour infrared monitoring video; an equipment leakage point position detection module 826, which is used to determine the position data of the equipment leakage point based on the air mass contour infrared monitoring video.

[0080] In a specific example, the moving object shielding module 821 is used to: use a mixed Gaussian background modeling algorithm to perform statistical analysis on the pixel brightness values ​​of each infrared image frame, and shield pixels that exceed the normal brightness distribution range to obtain the infrared surveillance video after shielding.

[0081] In a specific example, the infrared image frame normalization module 822 is used to: perform normalization processing on each infrared image frame in the infrared surveillance video after the shielding processing according to the following normalization processing formula to obtain the normalized infrared surveillance video; wherein the normalization processing formula is:

[0082]

[0083] in, for each infrared image frame, represents the minimum value of each infrared image frame, represents the maximum value of each infrared image frame, It is each infrared image frame in the infrared surveillance video after the normalization processing.

[0084] In a specific example, the contour extraction module 825 is used to: use the findContour function to perform contour extraction on each infrared image frame in the binarized infrared monitoring video to obtain the air mass contour infrared monitoring video.

[0085] In particular, after performing moving object shielding, normalization processing, morphological processing, binarization processing and contour extraction on each infrared image frame in the collected infrared monitoring video, the necessary clarity and accuracy are provided for the subsequent equipment leakage monitoring and leakage point location tasks. The positioning step is the key link in the entire monitoring process. Only by accurately identifying equipment leakage and determining the leakage location can timely measures be taken to repair it and prevent the accident from expanding, thereby ensuring the effectiveness and reliability of the system.

[0086] In a specific example, the morphological processing module 823 is used to perform morphological processing on each infrared image frame in the normalized infrared surveillance video to obtain the morphologically processed infrared surveillance video; a feature point extraction function can be added to extract and describe key feature points of the morphologically processed video frame. It further includes the following steps: feature point detection, applying feature point detection algorithms (such as ORB, SIFT, SURF, etc.) on the morphologically processed image to identify key feature points in the image; feature description, generating descriptors for each detected feature point, which can represent the local characteristics of the feature point and are used in subsequent matching or analysis processes; feature screening, screening out the most significant or stable feature points according to specific standards (such as the response strength of the feature point, the quality of the corner point, etc.) to improve the accuracy and robustness of subsequent processing; feature storage / transmission, storing or transmitting the extracted feature points and their descriptors to the next processing stage for further analysis or matching; visualization of feature points, if necessary, feature points can be marked on the image for debugging and visualization.

[0087] In a specific example, the morphological processing module 823 collects image information from an infrared camera 810 and / or a visible light camera 830, and identifies positioning points in specific areas of the equipment. In most scenarios, the areas where equipment is prone to leakage mainly come from the gaskets or welds of containers or pipes, equipment outlets or inlets, and bends or corners with low structural strength. These areas are often concentrated at the junction of two materials or components. By using the morphological processing module 823 to perform the feature point extraction function, the leakage-prone areas of the equipment are collected in advance, and key leakage source monitoring is performed, which can further reduce the demand for image processing computing power, reduce the probability of misjudgment, and improve recognition accuracy.

[0088] In some specific examples, after the morphological processing module 823 collects image information from the infrared camera 810 and / or a visible light camera 830, it identifies the leak-prone area and performs image segmentation for obtaining the complete acquisition area, focusing on image processing and algorithm analysis for segmented image information with leak-prone areas. For the rest, after the leak source monitoring confirms the occurrence of a leak event, image processing and algorithm analysis are performed on the full-frame or adjacent segmented image information to improve the rationality of computing power allocation and reduce the background data processing pressure.

[0089] Furthermore, in the step of determining the location data of the equipment leakage point based on the infrared monitoring video of the air mass contour, the technical concept of the present application is to analyze the infrared monitoring video of the air mass contour by introducing an image processing and analysis algorithm based on deep learning at the back end, so as to learn and characterize the pixel optical flow characteristics between video image frames, thereby obtaining an optical flow aggregation representation of the air mass movement information, so as to locate the source of the leakage and obtain the leakage point location data. In this way, more intelligent equipment leakage monitoring and leakage point location can be achieved, so that when equipment leakage occurs, a rapid response can be made according to the location of the leakage point, providing safety protection for enterprises and reducing potential environmental risks and economic losses.

[0090] Specifically, in the technical solution of the present application, the infrared monitoring video of the air mass contour is processed by using an optical flow algorithm to obtain a set of pixel optical flow feature vectors. In particular, in a specific example of the present application, the Farneback optical flow method or the Lucas-Kanade optical flow method can be used to calculate the motion vector between adjacent frames in the infrared monitoring video of the air mass contour to provide the motion direction and speed of each pixel. This is crucial for detecting the movement of air masses during gas leaks, because the diffusion of leaking gas is usually accompanied by obvious motion characteristics. By analyzing this motion information, the diffusion pattern and dynamic changes of the gas can be better understood, thereby improving the real-time and accuracy of leak detection.

[0091] Accordingly, if Figure 3As shown, the equipment leakage point location detection module 826 includes: an optical flow algorithm processing unit 8261, which is used to use the optical flow algorithm to process the air mass contour infrared monitoring video to obtain a set of pixel optical flow feature vectors; a semantic association enhancement unit 8262, which is used to perform semantic association enhancement on each pixel optical flow feature vector in the set of pixel optical flow feature vectors based on feature energy level metric associated radiation to obtain a set of enhanced pixel optical flow feature vectors; a feature aggregation unit 8263, which is used to input the set of enhanced pixel optical flow feature vectors into a feature aggregation network based on feature local energy distribution gating to obtain an air mass movement optical flow aggregation representation vector; a decoding unit 8264, which is used to input the air mass movement optical flow aggregation representation vector into a decoder-based leak source locator to obtain the location data of the equipment leakage point.

[0092] Then, since each pixel optical flow feature vector in the set of pixel optical flow feature vectors represents the pixel optical flow features between each group of adjacent frames, it reflects the movement direction and speed of each pixel between adjacent frames, and this information is crucial for equipment gas leakage monitoring and leakage point location positioning. However, in the process of equipment gas leakage monitoring and leakage point location positioning, the complexity of the environment may cause a large amount of irrelevant information to be mixed into the pixel optical flow feature vector. At the same time, the pixel optical flow features between different adjacent frames will have different importance and contribution to the subsequent leakage point identification and location tasks. Based on this, in the technical solution of the present application, each pixel optical flow feature vector in the set of pixel optical flow feature vectors is further semantically enhanced based on the feature energy level measurement associated radiation to obtain a set of enhanced pixel optical flow feature vectors. Through the semantic association enhancement processing based on the feature energy level measurement associated radiation, the focus can be focused on those features that truly reflect the movement of equipment leakage gas through detailed feature energy level analysis and fine-grained semantic association enhancement, which significantly improves the expressiveness of the pixel optical flow semantic features between adjacent frames and the semantic understanding ability of the model, thereby improving the accuracy of leakage point identification and location, and reducing false alarms and missed alarms.

[0093] Specifically, the processing steps for semantic association enhancement based on feature energy level metric associated radiation are as follows: first, the feature energy level coefficients are calculated for the set of pixel optical flow feature vectors to quantify the energy distribution of each pixel optical flow feature vector. In the technical solution of the present application, the feature energy level coefficients are related to the statistical characteristics of the scale and feature distribution of each pixel optical flow feature vector, which is used to represent the semantic information content density of each pixel optical flow feature vector itself and its influence on the semantic information of other pixel optical flow feature vectors in the surrounding neighborhood. Furthermore, one-dimensional convolutional coding is applied to capture the local dependencies in the feature energy level coefficient sequence of pixel optical flow semantics, and point convolution coding is used to further enhance the expression ability of spatial features. On this basis, a function-based probabilistic mapping is used to generate a pixel optical flow semantic feature energy level radiation associated weight vector to achieve adaptive weighting of feature energy levels. In particular, the pixel optical flow semantic feature energy level radiation association weight vector is used as the weight, and the feature vector sequence is weighted and residual processed to obtain the set of enhanced pixel optical flow feature vectors. This process not only strengthens the semantic association in the set of pixel optical flow feature vectors, but also improves the significance of the features and enhances the model's ability to capture key pixel optical flow semantic information. Therefore, by dynamically adjusting feature weights and strengthening feature representation, the model can more flexibly cope with the diversity and complexity of different equipment leakage environments, thereby better locating the leakage point in various equipment leakage environments.

[0094] Correspondingly, the semantic association strengthening unit 8262 includes: a feature energy level coefficient calculation subunit, which is used to calculate the feature energy level coefficient of each pixel optical flow feature vector in the set of pixel optical flow feature vectors to obtain a sequence of pixel optical flow feature energy level coefficients; a coefficient arrangement subunit, which is used to arrange the sequence of pixel optical flow feature energy level coefficients into a pixel optical flow feature energy level coefficient input vector; a one-dimensional convolution encoding subunit, which is used to perform one-dimensional convolution encoding on the pixel optical flow feature energy level coefficient input vector to obtain a pixel optical flow feature energy level radiation association vector; and a point convolution encoding subunit, which is used to The pixel optical flow feature energy level radiation association vector is point convolution encoded to obtain a pixel optical flow feature energy level radiation convolution association vector; a probabilistic mapping subunit is used to input the pixel optical flow feature energy level radiation convolution association vector into a sigmoid function for probabilistic mapping to obtain a pixel optical flow feature energy level radiation association weight vector; an enhancement subunit is used to use the characteristic value of each position in the pixel optical flow feature energy level radiation association weight vector as the weight value, perform vector-by-vector point multiplication on the set of pixel optical flow feature vectors by position, and add the set of pixel optical flow feature vectors to obtain the set of enhanced pixel optical flow feature vectors.

[0095] Among them, the characteristic energy level coefficient calculation subunit is used to: extract the characteristic maximum value of each position in the pixel optical flow feature vector to obtain the pixel optical flow feature maximum value; respectively calculate the mean and variance of the pixel optical flow feature vector to obtain the pixel optical flow feature mean and the pixel optical flow feature variance; multiply the value obtained by adding the pixel optical flow feature variance and the preset hyperparameter by four to obtain the first energy level coefficient of the pixel optical flow feature; calculate the square of the difference between the pixel optical flow feature maximum value and the pixel optical flow feature mean to obtain the pixel optical flow feature difference value; add the modulated pixel optical flow feature variance obtained by multiplying the pixel optical flow feature variance by two, the value obtained by multiplying the preset hyperparameter by two, and the pixel optical flow feature difference value to obtain the second energy level coefficient of the pixel optical flow feature; divide the first energy level coefficient of the pixel optical flow feature by the second energy level coefficient of the pixel optical flow feature to obtain the pixel optical flow feature energy level coefficient.

[0096] In a specific example, the semantic association enhancement unit 8262 is used to: perform semantic association enhancement based on feature energy level measurement associated radiation on each pixel optical flow feature vector in the set of pixel optical flow feature vectors using the following feature association radiation enhancement formula to obtain the set of enhanced pixel optical flow feature vectors; wherein the feature association radiation enhancement formula is:

[0097]

[0098] in, is the set of pixel optical flow feature vectors, and are the first The pixel optical flow feature vector and the pixel optical flow feature vector, represents the number of feature vectors in the set of pixel optical flow feature vectors, and are respectively the first The mean and variance of the pixel optical flow feature vector, For the said The maximum value of the feature in the pixel optical flow feature vector, is a preset hyperparameter, It is the The characteristic energy level coefficient of the pixel optical flow feature vector, and They are respectively pixel optical flow feature vector and the The characteristic energy level coefficient of the pixel optical flow feature vector, is the pixel optical flow feature level coefficient input vector, is a one-dimensional convolutional code, is the scale of the one-dimensional convolution kernel, is the pixel optical flow feature energy level radiation association vector, is point convolutional coding, yes function, is the pixel optical flow feature energy level radiation associated weight vector, It means point multiplication by position. is the set of enhanced pixel optical flow feature vectors.

[0099] It should be understood that since each enhanced pixel optical flow feature vector in the set of enhanced pixel optical flow feature vectors contains waveform semantic feature information about the pixel optical flow semantics of each two adjacent key frames after being enhanced by key features, in the task of equipment leakage point identification and location positioning, a single enhanced pixel optical flow feature vector is not sufficient to fully describe the temporal changes of pixel optical flow semantics and the temporal movement of equipment leakage gas, so that it is difficult to effectively identify and detect when the equipment leakage point is subsequently located. Based on this, in the technical solution of the present application, the set of enhanced pixel optical flow feature vectors is further input into a feature aggregation network based on feature local energy distribution gating to obtain an air mass motion optical flow aggregation representation vector. It should be understood that the feature aggregation network based on feature local energy distribution gating can utilize the feature energy local significance gating mechanism to dynamically adjust the weights of different pixel optical flow features in the aggregation process, so that the model can focus on those pixel optical flow features with higher energy and more contribution to equipment leakage point detection and location tasks. This mechanism can filter out noise and redundant information and improve the effectiveness of features. Therefore, through the processing of the feature aggregation network based on the feature local energy distribution gating, the local energy distribution of each pixel optical flow feature can be analyzed in the process of aggregating different pixel optical flow features, which helps to identify and strengthen those features with significant energy distribution in the local neighborhood, thereby improving the model's sensitivity to local changes. This analysis of local energy distribution helps to identify the dynamic changes of key pixel optical flow features, which may be related to the movement and propagation information of the leaking gas, and is of great significance to the task of locating the source of the leak. Then, using the gating function and mask module, the significance of different pixel optical flow features can be dynamically adjusted. This adaptive screening mechanism allows the model to dynamically adjust the focus according to the optical flow information of the air mass movement, thereby more effectively highlighting the important pixel optical flow semantics and suppressing unimportant features in the aggregation results of the pixel optical flow features to form an optical flow aggregation representation of the air mass movement, which helps the model to more accurately identify and locate equipment leakage points.

[0100] Accordingly, the feature aggregation unit 8263 includes: a coefficient acquisition subunit, which is used to determine the characteristic energy level coefficients of each enhanced pixel optical flow feature vector in the set of enhanced pixel optical flow feature vectors based on the maximum value, minimum value, mean value and variance of each enhanced pixel optical flow feature vector to obtain a sequence of pixel optical flow characteristic energy level coefficients; a local neighborhood mean calculation subunit, which is used to determine the scale of the local neighborhood of the pixel optical flow, and use the scale of the local neighborhood of the pixel optical flow as the mean calculation range to calculate each of the pixel optical flow characteristic energy level coefficients. to obtain a sequence of pixel optical flow feature energy level significance descriptors; a mask subunit, used to input the sequence of pixel optical flow feature energy level significance descriptors into a mask module based on a gating function to obtain a sequence of mask-gated probabilistic pixel optical flow feature energy level significance descriptors; a position-weighted sum subunit, used to calculate the position-weighted sum of the set of enhanced pixel optical flow feature vectors using the sequence of mask-gated probabilistic pixel optical flow feature energy level significance descriptors as a sequence of weights to obtain the air mass motion optical flow aggregation representation vector.

[0101] Wherein, the coefficient acquisition subunit includes: a multi-item calculation secondary subunit, which is used to calculate the maximum value, minimum value, average value and variance of the enhanced pixel optical flow feature vector to obtain the pixel optical flow maximum value, pixel optical flow minimum value, pixel optical flow average value and pixel optical flow variance; a first statistical factor calculation secondary subunit, which is used to calculate the sum of the pixel optical flow variance and the hyperparameter to obtain the first pixel optical flow local energy statistical factor; a maximum difference calculation secondary subunit, which is used to calculate the difference between the pixel optical flow maximum value and the pixel optical flow average value to obtain the pixel optical flow maximum deviation value; a minimum difference calculation secondary subunit, which is used to calculate the difference between the pixel optical flow minimum value and the pixel optical flow average value to obtain the pixel optical flow minimum deviation value; a second statistical factor calculation secondary subunit, which is used to calculate the square sum of the pixel optical flow maximum deviation value and the pixel optical flow minimum deviation value and then add it to the hyperparameter to obtain the second pixel optical flow local energy statistical factor; a division operation secondary subunit, which is used to calculate the division between the first pixel optical flow local energy statistical factor and the second pixel optical flow local energy statistical factor to obtain the pixel optical flow characteristic energy level coefficient.

[0102] The mask subunit is used to: use the negative number of each pixel optical flow feature energy level saliency descriptor in the sequence of pixel optical flow feature energy level saliency descriptors as the exponential power, calculate the exponential function value with the natural constant e as the base to obtain a sequence of pixel optical flow feature energy level class support saliency descriptors; calculate the positional sum between the sequence of pixel optical flow feature energy level class support saliency descriptors and the constant 1 to obtain a sequence of linear modulation pixel optical flow feature energy level class support saliency descriptors; calculate the reciprocal of each linear modulation pixel optical flow feature energy level class support saliency descriptor in the sequence of linear modulation pixel optical flow feature energy level class support saliency descriptors to obtain a pixel optical flow gated probabilistic feature energy level saliency descriptor. A sequence of pixel optical flow gated probabilistic feature energy level significance descriptors; performing masking processing on each pixel optical flow gated probabilistic feature energy level significance descriptor in the sequence of pixel optical flow gated probabilistic feature energy level significance descriptors to obtain the sequence of masked gated probabilistic pixel optical flow feature energy level significance descriptors; wherein, performing masking processing on each pixel optical flow gated probabilistic feature energy level significance descriptor in the sequence of pixel optical flow gated probabilistic feature energy level significance descriptors to obtain the sequence of masked gated probabilistic pixel optical flow feature energy level significance descriptors comprises: in response to the pixel optical flow gated probabilistic feature energy level significance descriptor being greater than a predetermined threshold, taking the original value of the pixel optical flow gated probabilistic feature energy level significance descriptor, otherwise setting it to 0.

[0103] In a specific example, the feature aggregation unit 8263 is used to: input the set of enhanced pixel optical flow feature vectors into the feature aggregation network based on feature local energy distribution gating to process it with the following feature aggregation formula to obtain the air mass motion optical flow aggregation representation vector; wherein the feature aggregation formula is:

[0104]

[0105] in, is the set of enhanced pixel optical flow feature vectors, is the first in the set of enhanced pixel optical flow feature vectors Enhanced pixel optical flow feature vector, For the The mean of the optical flow feature vectors of enhanced pixels, For the The variance of the optical flow feature vector of the enhanced pixel, is a hyperparameter, Indicates the The maximum value of the optical flow feature vector of the enhanced pixels, Indicates the The minimum value of the enhanced pixel optical flow feature vector, For the The characteristic energy level coefficient of the enhanced pixel optical flow feature vector, Indicated by the characteristic energy level coefficient The number of characteristic energy coefficients in the local neighborhood centered at For the The characteristic energy level coefficient of the enhanced pixel optical flow feature vector, For the The feature energy level saliency descriptor of the enhanced pixel optical flow feature vector, For the A gated probabilistic feature level saliency descriptor of enhanced pixel optical flow feature vectors, For masking operation, is the predetermined threshold, For the mask-gated probabilistic feature level saliency descriptor of the enhanced pixel optical flow feature vector, For the The salient enhanced pixel optical flow feature vector corresponding to the enhanced pixel optical flow feature vector is To significantly enhance the number of sequences of pixel optical flow feature vectors, The optical flow aggregation representation vector is for the air mass motion.

[0106] Then, the air mass motion optical flow aggregation representation vector is input into the decoder-based leak source locator to obtain the location data of the equipment leak point. In other words, the optical flow semantic aggregation features of air mass motion are used for decoding regression to locate the leak source and obtain the leak point location data. In this way, more intelligent equipment leak monitoring and leak point location can be achieved, so that when equipment leaks occur, a rapid response can be made based on the location of the leak point, providing safety protection for enterprises and reducing potential environmental risks and economic losses.

[0107] Preferably, the decoding unit 8264 is used to: subtract the 0 norm of the air mass motion optical flow aggregation representation vector from the length of the air mass motion optical flow aggregation representation vector to obtain an air mass motion optical flow aggregation isolated representation value; calculate a power function with each eigenvalue of the air mass motion optical flow aggregation representation vector as the base, and the difference of the air mass motion optical flow aggregation isolated representation value minus one as the exponent, and multiply it by the logarithm of the product of the air mass motion optical flow aggregation isolated representation value plus one and the air mass motion optical flow aggregation isolated representation value with base 2 to obtain an air mass motion optical flow aggregation leading value; After multiplying each eigenvalue of the air mass movement optical flow aggregation representation vector by the difference of the air mass movement optical flow aggregation isolated representation value minus one and dividing it by the air mass movement optical flow aggregation isolated representation value, an exponential function with a natural constant as the base is calculated to obtain the air mass movement optical flow aggregation bias value; after adding the air mass movement optical flow aggregation leading value and the air mass movement optical flow aggregation bias value, each eigenvalue of the optimized air mass movement optical flow aggregation representation vector is obtained; and, the optimized air mass movement optical flow aggregation representation vector is input into a decoder-based leakage source locator to obtain the location data of the equipment leakage point.

[0108] Specifically, the air mass motion optical flow aggregation represents a vector, for example, Each eigenvalue of ,in The optimization expression is:

[0109]

[0110] in, represents the length of the air mass motion optical flow aggregation representation vector, The optical flow aggregation representation vector of the air mass motion is represented, represents the zero norm of the optical flow aggregation representation vector of the air mass motion, Indicates the optical flow aggregation isolated representation value of the air mass movement, The first vector representing the optical flow aggregation of the air mass motion eigenvalues, represents an exponential operation of a value, wherein the exponential operation of the value represents calculating the value of a natural exponential function with the value as a power, represents the logarithmic function value with base 2, The first vector representing the optimized air mass motion optical flow aggregation representation is feature values.

[0111] Here, the set of pixel optical flow feature vectors expresses the optical flow-based image semantic features of the image frames of the air mass contour infrared monitoring video. Therefore, after performing temporal energy level measurement image semantic feature association enhancement and temporal local energy distribution gated aggregation between optical flow frames, it is still expected to further improve the regression comprehensibility of the aggregated mapping expression of the air mass motion optical flow aggregation representation vector, thereby improving the decoding regression comprehensibility and improving the accuracy of the decoding results.

[0112] Therefore, for the high-dimensional feature manifold of the air mass movement optical flow aggregation representation vector, the eigenvalues ​​of the feature set of the air mass movement optical flow aggregation representation vector are used as the vector field representation of the aggregation dimension, and the value of the derivative of the vector field of the air mass movement optical flow aggregation representation vector at the isolated zero position is used as the order information to fix the local position of the eigenvalues ​​of the feature set of the air mass movement optical flow aggregation representation vector, and a bias for the reversibility of the feature regression distribution of the air mass movement optical flow aggregation representation vector is added as a reward to achieve the mapping target tracking of the regression distribution of the air mass movement optical flow aggregation representation vector for the eigenvalue position, so that the feature set of the air mass movement optical flow aggregation representation vector perceives the mapping migration to the aggregation distribution, thereby promoting the decoding regression comprehensibility of the air mass movement optical flow aggregation representation vector by improving the mapping comprehensibility of the air mass movement optical flow aggregation representation vector, and improving the accuracy of the location data of the equipment leakage point obtained by the leakage source locator based on the decoder input of the air mass movement optical flow aggregation representation vector. In this way, the source of equipment leakage can be identified and located more accurately to obtain leakage point location data, so that when equipment leakage occurs, a rapid response can be made based on the location of the leakage point, providing safety protection for the enterprise and reducing potential environmental risks and economic losses.

[0113] Based on the above embodiments, see Figure 4 As shown in FIG. 1 , it is a flow chart of an online monitoring method for equipment gas leakage in an embodiment of the present application. Figure 4As shown, the online monitoring method for gas leakage of equipment according to the embodiment of the present application includes the following steps: S510, collecting infrared monitoring video of the monitored equipment; S520, shielding the moving objects in each infrared image frame in the infrared monitoring video to obtain the shielded infrared monitoring video; S530, normalizing each infrared image frame in the shielded infrared monitoring video to obtain the normalized infrared monitoring video; S540, morphologically processing each infrared image frame in the normalized infrared monitoring video to obtain the morphologically processed infrared monitoring video; S550, binarizing each infrared image frame in the morphologically processed infrared monitoring video based on a threshold value to obtain the binarized infrared monitoring video; S560, extracting the contour of each infrared image frame in the binarized infrared monitoring video to obtain the air mass contour infrared monitoring video; S570, determining the location data of the equipment leakage point based on the air mass contour infrared monitoring video.

[0114] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned equipment gas leakage online monitoring method have been referred to above. Figures 1 to 3 The optical device 800 for online monitoring of gas leakage has been described in detail, and therefore, its repeated description will be omitted.

[0115] Figure 5 FIG. 1 is an application scenario diagram of the equipment gas leakage online monitoring method according to an embodiment of the present application. Figure 5 As shown, in this application scenario, first, the infrared monitoring video of the monitored equipment is collected, and then the infrared monitoring video is input into a server deployed with an equipment leakage online monitoring algorithm, wherein the server can use the equipment leakage online monitoring algorithm to process the infrared monitoring video to determine the location data of the equipment leakage point.

[0116] In one example, the explosion-proof camera 1 and the explosion-proof camera 2 (or one of the cameras, such as the explosion-proof camera 1 or the explosion-proof camera 2) can respectively collect infrared monitoring videos of the monitored equipment in the monitoring scene area, and output corresponding intelligent visual signals 1 and intelligent visual signals 2 (not shown in the figure). In some examples, the explosion-proof camera 1 and the explosion-proof camera 2 can synchronously collect visible light monitoring videos of the monitored equipment and perform image fusion and splicing with the above-mentioned intelligent visual signals 1 and intelligent visual signals 2. Specifically, the explosion-proof camera 1 and the explosion-proof camera 2 are respectively equipped with explosion-proof distribution boxes and powered by 220V AC. The above-mentioned explosion-proof camera 1 and the explosion-proof camera 2 further include the infrared camera 810, the visible light camera 830 and the data processor 820.

[0117] Further integration Figure 6 As shown, the infrared data from the background is obtained and represented as I B , the infrared data after the suspected infrared ray passes through the black smoke (representing the suspicious air mass) is expressed as I G , I B >I G Therefore, by calculating the detected infrared data, it is possible to determine whether black smoke is generated, and thus whether there is a leak in the equipment. Accordingly, the calculation formula can be expressed as: ΔI = I B -I G ΔI represents the difference between the infrared data from the background and the infrared data after the suspected infrared passes through the black smoke. Through the relationship between ΔI and the predetermined threshold, it can be determined whether black smoke is generated, and then whether the equipment is leaking.

[0118] like Figure 7 FIG. 1 is an application scenario diagram of the equipment leakage online monitoring method according to an embodiment of the present application. Figure 7 As shown, in this application scenario, first, the infrared monitoring video of the monitored pipeline is collected (for example, Figure 7 D), and then, the infrared monitoring video is input to a server (for example, Figure 7 In S) as shown in , the server can use the pipeline leakage online monitoring algorithm to process the infrared monitoring video to determine the location data of the pipeline leakage point.

[0119] It is worth mentioning that in this application, the original video data is obtained from the infrared camera, and the video data can be input into the processing module of the algorithm in the form of a single-frame grayscale image. In addition, moving objects (such as people, vehicles, trees, etc.) in the video frame are shielded to eliminate interference with the estimation of the direction of air mass movement. The mixed Gaussian background modeling algorithm (MOG2) in OpenCV can be used, and the parameters can be optimized to adapt it to industrial scenarios. MOG2 performs statistical analysis on the brightness of each pixel in each frame of the image, judges and shields pixels that exceed the normal brightness distribution range, thereby shielding moving objects.

[0120] Furthermore, it is worth mentioning that in this application, the original video data is obtained from the infrared camera, and the video data can be input into the processing module of the algorithm in the form of a single-frame grayscale image. In addition, the moving objects (such as people, vehicles, trees, etc.) in the video frame are shielded to eliminate interference with the estimation of the direction of air mass movement. The mixed Gaussian background modeling algorithm (MOG2) in OpenCV can be used, and the parameters can be optimized to adapt it to industrial scenarios. MOG2 performs statistical analysis on the brightness of each pixel in each frame of the image, judges and shields pixels that exceed the normal brightness distribution range, thereby shielding moving objects.

[0121] Furthermore, in another example of the present application, a reverse optical flow algorithm can also be used to reversely calculate the motion trajectory of the air mass. By reversely processing the video frames in a time series, the reverse motion vector of the air mass in each frame is determined to trace the motion path of the air mass. Then, all the reverse motion vectors are merged to generate a complete reverse motion trajectory of the air mass, and the trajectory is analyzed to identify the starting point of the trajectory, that is, the source position of the air mass. Finally, by analyzing the spatial position of the starting point of the trajectory, combined with environmental information and possible sources of air mass generation, the specific source position and volume of the air mass are finally determined.

[0122] Specifically, the inverse optical flow algorithm is a computer vision technology used to estimate the inverse process of object motion in an image sequence. Unlike traditional optical flow algorithms, the inverse optical flow algorithm does not predict the motion of the next frame from the current frame, but reversely calculates the motion from the next frame of the sequence to the previous frame. The advantage of this is that when the final position of the object is known, the motion trajectory of the object can be tracked in reverse, so as to find the starting position or source of the object, and further analyze and obtain the volume information of the air mass.

[0123] The application scenarios of the inverse optical flow algorithm include but are not limited to environmental monitoring, meteorology, and other fields that require tracking the path of an object. For example, when tracking the spread of pollutants in the atmosphere, the inverse optical flow algorithm can be used to determine the source of the pollutants. In addition, the inverse optical flow algorithm can also be used for specific applications in video analysis, such as target tracking and behavior recognition.

[0124] When implementing the inverse optical flow algorithm, the following key steps need to be considered: 1. Starting from the next frame of the video sequence, select a target region. 2. Use the optical flow algorithm to calculate the position of the region in the previous frame, which involves calculating the gradient of the image and estimating the motion of the pixels. 3. Reversely update the position of the target region and repeat the above process until the first frame of the sequence is reached. 4. Determine the starting point or source position of the object by analyzing the entire reverse motion trajectory.

[0125] It is worth noting that the inverse optical flow algorithm may face some challenges in its implementation, such as the need to deal with occlusion problems in image sequences, illumination changes, etc. In addition, the efficiency and accuracy of the algorithm also depend on the specific optical flow estimation method and parameter settings used.

[0126] Based on the above embodiments, another exemplary embodiment of an electronic device is also provided in the embodiments of the present application. In some possible implementations, the electronic device in the embodiments of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the equipment gas leakage online monitoring method in the above embodiments when executing the program.

[0127] The embodiment of the present application also provides a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the online monitoring method for equipment gas leakage according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.

[0128] The embodiment of the present application also provides a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer readable storage medium. The processor of the computer device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the computer device executes the equipment gas leakage online monitoring method according to the embodiment of the present application.

[0129] Those skilled in the art will appreciate that the contents disclosed in this application may be subject to various variations and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0130] In addition, although the present application makes various references to certain units in the system according to embodiments of the present application, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0131] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present application is not limited to any particular form of combination of hardware and software.

[0132] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0133] The above is an explanation of the present application and should not be considered as limiting thereof. Although several exemplary embodiments of the present application have been described, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present application.

Claims

1. An optical device for online monitoring of gas leakage, characterized in that: It includes an infrared camera and a data processor, wherein the infrared camera is used to collect infrared monitoring video of the monitored equipment, and the data processor is used to process the infrared monitoring video to obtain monitoring results, wherein the monitoring results are the location and flow data of the equipment leakage point, wherein the data processor includes: A moving object shielding module, used for shielding moving objects in each infrared image frame in the infrared surveillance video to obtain a shielded infrared surveillance video; An infrared image frame normalization module, used for normalizing each infrared image frame in the infrared surveillance video after the shielding process to obtain a normalized infrared surveillance video; A morphological processing module, used for performing morphological processing on each infrared image frame in the normalized infrared surveillance video to obtain a morphologically processed infrared surveillance video; A threshold binarization module, used for performing a threshold-based binarization process on each infrared image frame in the infrared surveillance video after the morphological process to obtain a binarized infrared surveillance video; A contour extraction module, used for performing contour extraction on each infrared image frame in the infrared monitoring video after the binarization process to obtain an infrared monitoring video of an air mass contour; An equipment leakage point location detection module is used to determine the location data of the equipment leakage point based on the infrared monitoring video of the air mass contour; Wherein, the equipment leakage point location detection module includes: An optical flow algorithm processing unit, used to process the infrared monitoring video of the air mass contour using an optical flow algorithm to obtain a set of pixel optical flow feature vectors; A semantic association enhancement unit, configured to perform semantic association enhancement on each pixel optical flow feature vector in the set of pixel optical flow feature vectors based on feature energy level measurement associated radiation to obtain a set of enhanced pixel optical flow feature vectors; A feature aggregation unit, used for inputting the set of enhanced pixel optical flow feature vectors into a feature aggregation network based on feature local energy distribution gating to obtain an air mass motion optical flow aggregation representation vector; A decoding unit, used for inputting the air mass motion optical flow aggregation representation vector into a leak source locator based on a decoder to obtain the position data of the equipment leak point; Wherein, the semantic association strengthening unit includes: A characteristic energy level coefficient calculation subunit, configured to calculate, for the set of pixel optical flow feature vectors, a characteristic energy level coefficient of each pixel optical flow feature vector in the set of pixel optical flow feature vectors to obtain a sequence of pixel optical flow characteristic energy level coefficients; A coefficient arrangement subunit, used for arranging the sequence of pixel optical flow feature energy level coefficients into a pixel optical flow feature energy level coefficient input vector; A one-dimensional convolutional encoding subunit, used for performing one-dimensional convolutional encoding on the pixel optical flow feature energy level coefficient input vector to obtain a pixel optical flow feature energy level radiation correlation vector; A point convolution encoding subunit, used for performing point convolution encoding on the pixel optical flow feature energy level radiation association vector to obtain a pixel optical flow feature energy level radiation convolution association vector; A probabilistic mapping subunit, used for inputting the pixel optical flow feature energy level radiation convolution association vector into a sigmoid function for probabilistic mapping to obtain a pixel optical flow feature energy level radiation association weight vector; An enhancement subunit, configured to use the characteristic value of each position in the pixel optical flow characteristic energy level radiation associated weight vector as a weight value, perform vector-by-vector point multiplication on the set of pixel optical flow characteristic vectors by position, and add the set of pixel optical flow characteristic vectors to obtain the set of enhanced pixel optical flow characteristic vectors; Wherein, the feature aggregation unit includes: A coefficient acquisition subunit, configured to determine the characteristic energy level coefficients of each enhanced pixel optical flow feature vector in the set of enhanced pixel optical flow feature vectors based on the maximum value, minimum value, mean value and variance of each enhanced pixel optical flow feature vector to obtain a sequence of pixel optical flow characteristic energy level coefficients; A local neighborhood mean calculation subunit is used to determine the scale of the local neighborhood of the pixel optical flow, use the scale of the local neighborhood of the pixel optical flow as the mean calculation range, and respectively calculate the local neighborhood mean of each pixel optical flow feature energy level coefficient to obtain a sequence of pixel optical flow feature energy level significance descriptors; A mask subunit, used for inputting the sequence of pixel optical flow feature energy level significance descriptors into a mask module based on a gating function to obtain a sequence of mask-gated probabilistic pixel optical flow feature energy level significance descriptors; The position-weighted sum subunit is used to calculate the position-weighted sum of the set of enhanced pixel optical flow feature vectors using the sequence of the mask-gated probabilistic pixel optical flow feature energy level significance descriptors as a sequence of weights to obtain the air mass motion optical flow aggregation representation vector.

2. The optical device for online monitoring of gas leakage according to claim 1, characterized in that: The morphological processing module performs a feature point extraction function, identifies positioning points in leak-prone areas of equipment, and monitors key leak sources.

3. The optical device for online monitoring of gas leakage according to claim 1, characterized in that: The moving object shielding module is used for: A mixed Gaussian background modeling algorithm is used to perform statistical analysis on the pixel brightness values ​​of each infrared image frame, and pixel points that exceed the normal brightness distribution range are shielded to obtain the infrared monitoring video after shielding processing.

4. The optical device for online monitoring of gas leakage according to claim 3, characterized in that: The infrared image frame normalization module is used to: Each infrared image frame in the infrared surveillance video after the shielding process is normalized by the following normalization processing formula to obtain the infrared surveillance video after the normalization process; wherein the normalization processing formula is: ； in, for each infrared image frame, represents the minimum value of each infrared image frame, represents the maximum value of each infrared image frame, It is each infrared image frame in the infrared monitoring video after the normalization processing.

5. The optical device for online monitoring of gas leakage according to claim 4, characterized in that: The contour extraction module is used to: The findContour function is used to extract the contour of each infrared image frame in the binarized infrared monitoring video to obtain the air mass contour infrared monitoring video.

6. The optical device for online monitoring of gas leakage according to claim 5, characterized in that: The characteristic energy level coefficient calculation subunit is used for: Extracting the maximum value of the feature at each position in the pixel optical flow feature vector to obtain the maximum value of the pixel optical flow feature; Calculating the mean and variance of the pixel optical flow feature vector respectively to obtain a pixel optical flow feature mean and a pixel optical flow feature variance; The value obtained by adding the pixel optical flow feature variance and the preset hyperparameter is multiplied by four to obtain the first energy level coefficient of the pixel optical flow feature; Calculate the square of the difference between the maximum value of the pixel optical flow feature and the mean value of the pixel optical flow feature to obtain a pixel optical flow feature difference value; The pixel optical flow feature second energy level coefficient is obtained by adding a modulated pixel optical flow feature variance obtained by multiplying the pixel optical flow feature variance by two, a value obtained by multiplying the preset hyperparameter by two, and the pixel optical flow feature difference value; The first energy level coefficient of the pixel optical flow feature is divided by the second energy level coefficient of the pixel optical flow feature to obtain the pixel optical flow feature energy level coefficient.

7. The optical device for online monitoring of gas leakage according to claim 6, characterized in that: The coefficient acquisition subunit includes: A multi-item calculation secondary subunit, used to calculate the maximum value, minimum value, average value and variance of the enhanced pixel optical flow feature vector to obtain the maximum value of the pixel optical flow, the minimum value of the pixel optical flow, the average value of the pixel optical flow and the variance of the pixel optical flow; A first statistical factor calculation secondary subunit is used to calculate the sum of the pixel optical flow variance and the hyperparameter to obtain a first pixel optical flow local energy statistical factor; A maximum difference calculation secondary subunit, used for calculating the difference between the maximum value of the pixel optical flow and the average value of the pixel optical flow to obtain a maximum deviation value of the pixel optical flow; A minimum difference calculation secondary subunit, used for calculating the difference between the minimum value of the pixel optical flow and the average value of the pixel optical flow to obtain a minimum deviation value of the pixel optical flow; A second statistical factor calculation secondary subunit is used to calculate the square sum of the maximum deviation value of the pixel optical flow and the minimum deviation value of the pixel optical flow and then add the square sum to the hyperparameter to obtain a second pixel optical flow local energy statistical factor; The second-level division operation subunit is used to calculate the division between the first pixel optical flow local energy statistical factor and the second pixel optical flow local energy statistical factor to obtain the pixel optical flow characteristic energy level coefficient.

8. A method for online monitoring of equipment gas leakage, using the optical device for online monitoring of equipment gas leakage according to claim 1, characterized in that: include: Collect infrared surveillance video of the monitored equipment; Shielding the moving objects in each infrared image frame in the infrared surveillance video to obtain a shielded infrared surveillance video; Normalizing each infrared image frame in the infrared surveillance video after the shielding process to obtain a normalized infrared surveillance video; Performing morphological processing on each infrared image frame in the normalized infrared surveillance video to obtain a morphologically processed infrared surveillance video; Performing a threshold-based binarization process on each infrared image frame in the infrared surveillance video after the morphological processing to obtain a binarized infrared surveillance video; Performing contour extraction on each infrared image frame in the binarized infrared monitoring video to obtain an infrared monitoring video of an air mass contour; Based on the infrared monitoring video of the air mass contour, the location data of the equipment leakage point is determined.

Images