Pipeline oil leakage detection method and system based on multiple modes and GLCM

Abstract

The invention belongs to the technical field of pipeline oil leakage detection, and particularly discloses a crude oil and product oil pipeline oil leakage detection method and system based on multiple modes and a GLCM. The method comprises the following steps: acquiring a multispectral image and a common RGB image; performing image preprocessing on the common RGB image to obtain a preprocessed RGB image; generating a GLCM texture feature image for the multispectral image; performing feature extraction on the preprocessed RGB image to generate an RGB image feature map; performing feature extraction on the GLCM texture feature image to generate a GLCM texture feature graph; superposing the RGB image feature map and the GLCM texture feature map on a channel level to generate a fused feature map; and performing target detection by using the fused feature map. According to the scheme of the invention, the common RGB image and the multispectral image are subjected to feature-level fusion according to the spectral features of the crude oil and the product oil, and the texture features are extracted by using the gray-level co-occurrence matrix (GLCM), so that the external interference resistance of the detection model is improved, and the oil leakage detection is more accurate and reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of pipeline oil leakage detection, and in particular relates to a crude oil and refined oil pipeline leakage detection method and system based on multi-modality and GLCM. Background Art

[0002] Crude oil and refined oil are important energy resources, and also flammable and explosive dangerous goods. During the production and transportation process, oil leaks may have serious impacts in many aspects. For example, oil leaks may cause safety accidents such as fires and explosions, threatening the lives and property of residents and the public. The prices of crude oil and refined oil are high. Once an oil leak occurs, it may lead to a large amount of waste of crude oil and refined oil and property losses, which will have a serious impact on related companies and economic interests. Therefore, the detection of crude oil and refined oil leaks has always been highly valued by industrial production.

[0003] At present, the detection of crude oil and refined oil leakage can be divided into three aspects:

[0004] 1. Visual inspection: Monitor the pipelines through manual inspections or installation of cameras, and visually identify oil leaks, such as crude oil and refined oil leaks and drips. However, this method relies on manual operation or the setting of fixed cameras, and has the disadvantages of high labor costs, limited monitoring range, and susceptibility to interference from environmental factors. Most pipelines are outdoors, which will have a great impact on the accuracy of ordinary visual inspections.

[0005] 2. Acoustic detection: By installing acoustic sensors or microphones to monitor pipelines, oil leaks can be identified through acoustic signals, such as the sound produced by the leakage of crude oil or refined oil. However, this method is affected by environmental noise and may have low detection sensitivity for small-scale or low-frequency oil leaks.

[0006] 3. Gas detection: By installing gas sensors to monitor the air around the pipeline, oil leaks can be identified by detecting the smell of crude oil or refined oil or changes in the concentration of volatile organic compounds (VOCs) in the air. However, this method may be affected by factors such as environmental meteorological conditions and sensor sensitivity, and there may be detection errors for tiny oil leaks. This method is not applicable if it is outdoors.

[0007] Based on the above, it is found that the existing crude oil and refined oil leakage detection methods have the following shortcomings:

[0008] 1. The detection sensitivity is not high, and small-scale oil leaks or low-frequency oil leaks may not be detected in a timely and accurate manner;

[0009] 2. Interference from environmental factors, such as environmental noise, meteorological conditions, etc., may cause unstable test results;

[0010] 3. Relying on manual operation or fixed camera settings, the labor cost is high and the monitoring range is limited;

[0011] 4. The detection method is single, and it is difficult to comprehensively utilize multiple information sources to improve the accuracy and reliability of oil leak detection.

[0012] In view of this, there is an urgent need in the art for a sensitive, stable, efficient and accurate method for detecting leakage of crude oil and refined oil. Summary of the invention

[0013] In order to solve at least one of the shortcomings of the existing crude oil and refined oil leakage detection methods mentioned in the background technology, the main purpose of the present invention is to provide a crude oil and refined oil pipeline leakage detection method and system based on multimodality and GLCM, so as to improve the sensitivity, accuracy, reliability and efficiency of oil leakage detection, and realize efficient monitoring and early warning of oil leakage during the transportation of crude oil and refined oil in pipelines within the factory, thereby protecting the environment, human health and property safety.

[0014] According to a first aspect of the present invention, a method for detecting oil leakage in crude oil and refined oil pipelines based on multimodality and GLCM is provided, which comprises the following steps:

[0015] Step S1: Acquire a multispectral image and a normal RGB image;

[0016] Step S2: performing image preprocessing on the common RGB image to obtain a preprocessed RGB image;

[0017] Step S3: generating a GLCM texture feature image for the multispectral image;

[0018] Step S4: extracting features from the preprocessed RGB image to generate an RGB image feature map;

[0019] Step S5: extracting features from the GLCM texture feature image to generate a GLCM texture feature map;

[0020] Step S6: superimpose the RGB image feature map and the GLCM texture feature map at the channel level to generate a fused feature map;

[0021] Step S7: Use the fused feature map to perform target detection.

[0022] According to some embodiments of the present invention, in step S1, a multispectral image and a common RGB image are acquired by a multispectral camera for real-time shooting.

[0023] According to some embodiments of the present invention, in step S2, performing image preprocessing on the common RGB image includes denoising, adjusting brightness and contrast, and cropping and scaling the common RGB image.

[0024] According to some embodiments of the present invention, in step S3, generating a GLCM texture feature image for a multispectral image includes:

[0025] Step S31: limiting the grayscale of the multispectral image to a specific range and quantizing it into discrete grayscales to obtain a quantized image;

[0026] Step S32: for the quantized image, by traversing each pixel in the image, calculating the occurrence frequency of each pair of grayscale levels at a specific distance and angle;

[0027] Step S33: normalizing the statistical results of the grayscale pairs to obtain a GLCM matrix;

[0028] Step S34: Generate a GLCM texture feature image based on the GLCM matrix.

[0029] According to some embodiments of the present invention, the generated GLCM texture feature images include mean texture feature images, variance texture feature images, homogeneity texture feature images, contrast texture feature images, dissimilarity texture feature images, entropy texture feature images, angular second moment texture feature images, correlation texture feature images, and autocorrelation texture feature images.

[0030] According to some embodiments of the present invention, in step S3, generating a GLCM texture feature image for a multispectral image further includes:

[0031] Step S35: Select three images that can highlight the characteristics of crude oil and refined oil from the generated GLCM texture feature images as three-channel data for feature extraction in step S5.

[0032] According to some embodiments of the present invention, the three images selected to highlight the characteristics of crude oil and refined oil are a mean texture feature image, an entropy texture feature image, and an autocorrelation texture feature image.

[0033] According to some embodiments of the present invention, in step S4 and step S5, the backbone network Darknet-53 of YOLOv3 is used as a feature extraction network to perform feature extraction.

[0034] According to some embodiments of the present invention, in step S4, three RGB image feature maps of different sizes are generated by feature extraction; in step S5, three GLCM texture feature maps of different sizes are generated by feature extraction; in step S6, the three RGB image feature maps of different sizes and the three GLCM texture feature maps of different sizes are superimposed at the channel level to generate fused feature maps of three sizes.

[0035] According to a second aspect of the present invention, there is provided a crude oil and refined oil pipeline leakage detection system based on multimodality and GLCM, which comprises:

[0036] Multispectral cameras; and

[0037] A GPU computing device is communicatively connected to the multispectral camera and is configured to execute the method according to the first aspect of the present invention.

[0038] Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0039] The solution of the present invention performs feature-level fusion of ordinary RGB images and multispectral images according to the spectral characteristics of crude oil and refined oil, and uses gray-level co-occurrence matrix (GLCM) to extract texture features, thereby improving the ability of the detection model to resist external interference and making oil leakage detection more accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flow chart of a crude oil and refined oil pipeline leakage detection method based on multimodality and GLCM provided by the present invention;

[0042] Figure 2 This is an image of oil leakage from a pipeline;

[0043] Figure 3 This is the YOLOv3 network structure diagram;

[0044] Figure 4 It is a schematic diagram of the GLCM sliding window;

[0045] Figure 5 is the GLCM texture feature image;

[0046] Figure 6 Generate a flow chart for GLCM;

[0047] Figure 7 This is the Darknet-53 network structure diagram;

[0048] Figure 8 This is the flow chart of the crude oil and refined oil detection model. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0050] Crude oil and refined oil are important energy resources and also dangerous goods that are flammable and explosive. Therefore, the detection of crude oil and refined oil leakage has always been of great importance in industrial production. Nowadays, computer vision has been applied to many fields and has shown good results. Computer vision technology has been widely used in the field of security alarm monitoring. By using computer vision algorithms, it is possible to quickly discover potential safety hazards and provide timely alarms. At the same time, applying computer vision to monitor safety conditions can improve safety while reducing labor costs. The present invention aims to use computer vision monitoring methods to achieve simple and accurate monitoring of crude oil and refined oil leakage.

[0051] In outdoor conditions, there are often dark, rainy, and other contaminant interferences. This requires an analysis and understanding of the unique characteristics of crude oil and refined oil, and target detection design based on their specific characteristics:

[0052] 1. Utilization of texture features: The surfaces of crude oil and refined oil usually have certain texture features, which can be used to detect crude oil and refined oil targets.

[0053] 2. Optical properties: The reflection, absorption and transmission characteristics of oil in optical bands such as visible light, infrared light and ultraviolet light are different from those of the surrounding environment, and the optical difference between it and the environment can be detected by optical sensors.

[0054] 3. Micro-infrared characteristics: Oil has a unique absorption band in the micro-infrared band (usually in the range of 1.4-1.7μm), called the CH stretching vibration absorption band, and the micro-infrared difference between it and the environment can be detected by micro-infrared sensors such as micro-infrared spectrometers.

[0055] 4. Utilization of spectral information: Crude oil and refined oil have unique absorption and reflection characteristics in the visible light and near-infrared spectrum range. Spectral information can be used to detect crude oil and refined oil targets.

[0056] Through the analysis of the physical properties of crude oil and refined oil, it is found that under normal circumstances without interference, ordinary target detection and identification can be used. When encountering rainy days and dim conditions, the texture characteristics and optical characteristics of crude oil and refined oil can be used to improve the accuracy of target detection.

[0057] To this end, the present invention proposes a crude oil and refined oil pipeline leakage detection method based on multimodality and GLCM, which is mainly used for video automatic monitoring of whether there is oil leakage in the pipelines transporting crude oil and refined oil. The multimodal target detection method of the present invention fuses the features of ordinary RGB images and multispectral images, and uses gray-level co-occurrence matrix (GLCM) to extract texture features, thereby improving the ability of the detection model to resist external interference and making oil leakage detection more accurate and reliable. The proposal of the multimodal oil leakage detection model aims to overcome the above-mentioned defects of the existing technology or products, improve the sensitivity, accuracy and reliability of oil leakage detection through the comprehensive utilization of multiple detection means, and realize efficient monitoring and early warning of oil leakage in the process of transporting crude oil and refined oil in pipelines in factories, thereby protecting the environment, human health and property safety.

[0058] Multimodal object detection is an object detection method based on multiple different sensors, modalities or data sources. Traditional object detection methods usually only use data of a single modality, such as images or videos. Multimodal object detection can more comprehensively understand the target features and improve the accuracy and robustness of object detection by fusing information from multiple modalities.

[0059] Modal fusion is to fuse the features of multispectral images and ordinary RGB images to generate a fused feature representation. The fusion method can be implemented in a variety of ways, such as feature-level fusion, decision-level fusion, deep fusion, etc. Feature-level fusion can form a new feature representation by splicing the features of multispectral images and ordinary images together. Decision-level fusion can be performed at the output layer of the target detection algorithm, such as using weight coefficients to perform weighted averaging on the outputs of different modalities. Deep fusion can simultaneously process multimodal inputs through a deep neural network model to achieve end-to-end fusion. The present invention selects to fuse ordinary RGB images and multispectral images at the feature level according to the spectral features of crude oil and refined oil, which can better distinguish crude oil, refined oil and background, and better improve accuracy and anti-interference ability.

[0060] Gray Level Co-occurrence Matrix (GLCM) is a statistical method used to describe image texture features. It is widely used in the fields of image processing and computer vision, especially in tasks such as texture analysis, image recognition, image classification, and image segmentation.

[0061] The gray-level co-occurrence matrix method calculates the gray-level image to obtain its co-occurrence matrix, and then calculates the co-occurrence matrix to obtain some eigenvalues ​​of the matrix to represent certain texture features of the image. The gray-level co-occurrence matrix can reflect the comprehensive information of the image grayscale about direction, adjacent intervals, and change amplitude. It is the basis for analyzing the local patterns of the image and their arrangement rules. The gray-level co-occurrence matrix needs to use the direction θ, step size d, and the order N of the gray-level co-occurrence matrix when calculating.

[0062] Direction θ: The calculation process is usually performed in several different directions, usually horizontal 0°, vertical 90°, 45° and 135°;

[0063] Step d: The distance between elements required to calculate the co-occurrence matrix;

[0064] The order N of the gray-level co-occurrence matrix is ​​the same as the order of the gray-level value of the gray-level image. That is, when the gray-level value order of the gray-level image is N, the gray-level co-occurrence matrix is ​​an N×N matrix.

[0065] P(i,j|θ,d): represents the frequency of occurrence of pixel pairs between gray level i and gray level j at a specific step size d and angle θ.

[0066] The calculation process of GLCM includes the following steps:

[0067] 1. Grayscale quantization: The grayscale of the original image is limited to a specific range and quantized into discrete grayscale levels to obtain a quantized image.

[0068] 2. Grayscale pair statistics: For the quantized image, by traversing each pixel in the image, the frequency of occurrence of each pair of grayscale pairs at a specific distance d and angle θ is calculated.

[0069] Assume: the grayscale is divided into N = 4 levels (grayscale from 0 to 3); the window size is 6×6, and then slide the entire image, each time you can slide the original Figure 6 ×6 range, such as Figure 4 As shown. Taking (1,2) as an example, calculate the frequency of (1,2) when the distance d = 1 and the angle θ = 0 when the gray level i = 1 and the gray level j = 2. The result is shown in formula 2-1.

[0070]

[0071] Boundary problems may occur in sliding windows. The present invention adopts a method of expanding boundaries. According to the size and step size of the window, the boundary of the source image is expanded with a value of 0 or a boundary value, so that the dimension of the source image is expanded, and the added boundary pixel value is also included in the calculation, so that the original boundary element can eventually become the central element for calculation, and the final processing result still maintains the original image size.

[0072] 3. Normalization: Normalize the results of the grayscale pair statistics to obtain the GLCM matrix, as shown in Equation 2-2. Common normalization methods include dividing each element in the GLCM matrix by the total number of pixel pairs to obtain frequency or probability.

[0073]

[0074] 4. Calculate texture features: GLCM provides information about the grayscale direction, interval, change amplitude and speed of the image, but it cannot directly provide the characteristics of distinguishing textures. It is necessary to extract statistical attributes that describe specific texture features based on GLCM. There are 9 commonly used texture features.

[0075] Mean, calculate the mean of each grayscale pair, as shown in formula 2-3.

[0076]

[0077] Variance, calculate the variance of each grayscale pair, as shown in formula 2-4.

[0078]

[0079] Homogeneity reflects the magnitude of local changes in image texture. If the image texture is uniform in different regions and changes slowly, the value will be larger, otherwise it will be smaller, as shown in formula 2-5.

[0080]

[0081] Contrast reflects the clarity of the image and the depth of the texture grooves. The deeper the texture grooves, the greater the contrast and the clearer the effect; conversely, if the contrast value is small, the grooves are shallow and the effect is blurred. As shown in formula 2-6.

[0082]

[0083] The difference is as shown in formula 2-7.

[0084]

[0085] Entropy represents the randomness contained in the image and shows the complexity of the image. When all values ​​of the co-occurrence matrix are equal or the pixel values ​​show the greatest randomness, the entropy is the largest, as shown in Formula 2-8.

[0086]

[0087] Angular second moment, also known as energy, is a measure of the uniformity of image grayscale distribution and texture coarseness. When the image texture is uniform and regular, the energy value is large, as shown in formula 2-9.

[0088]

[0089] Correlation reflects the consistency of image texture. If there is horizontal texture in the image, the correlation value of the horizontal co-occurrence matrix is ​​greater than the correlation value of the co-occurrence matrix in other directions. It measures the similarity of the elements of the spatial grayscale co-occurrence matrix in the row or column direction. Therefore, the size of the correlation value reflects the local grayscale correlation in the image. The correlation and autocorrelation are shown in Equation 2-10 and Equation 2-11 respectively.

[0090]

[0091]

[0092] The above statistics can be used to describe the texture information in the image and generate the grayscale texture image of the image. Therefore, the texture of the image can be better extracted in image processing and image analysis tasks, and it can be applied to crude oil and refined oil detection to better locate the detection based on the special texture of crude oil and refined oil.

[0093] YOLOv3 achieves real-time target detection in a single network through a one-time forward propagation (You Only Look Once) method, with the characteristics of high speed and high accuracy. Today, YOLOv3 is widely used in industrial-level target detection tasks. The YOLOv3 network structure is as follows Figure 3 shown.

[0094] YOLOv3 has the following features:

[0095] 1. Multi-scale detection: YOLOv3 uses multiple feature maps of different scales in the network, so that it can detect objects of different scales, including small objects and large objects. This can improve the detection accuracy of objects of different scales.

[0096] 2. Feature Pyramid Network: YOLOv3 introduces a feature pyramid network (FPN) to extract features with different receptive fields from feature maps at different levels, so that objects of different sizes can be detected.

[0097] 3. Multi-scale prediction: YOLOv3 performs target prediction on feature maps of each scale, so that it can detect targets of different scales. This can improve the detection accuracy of small and large targets.

[0098] According to a first aspect of the present invention, a method for detecting oil leakage in crude oil and refined oil pipelines based on multimodality and GLCM is provided. Figure 1 As shown, the method comprises the following steps:

[0099] Step S1: Acquire a multispectral image and a normal RGB image;

[0100] Step S2: performing image preprocessing on the common RGB image to obtain a preprocessed RGB image;

[0101] Step S3: generating a GLCM texture feature image for the multispectral image;

[0102] Step S4: extracting features from the preprocessed RGB image to generate an RGB image feature map;

[0103] Step S5: extracting features from the GLCM texture feature image to generate a GLCM texture feature map;

[0104] Step S6: superimpose the RGB image feature map and the GLCM texture feature map at the channel level to generate a fused feature map;

[0105] Step S7: Use the fused feature map to perform target detection.

[0106] The following is a detailed description of each step.

[0107] In step S1, a multispectral image and a normal RGB image are obtained by a multispectral camera that takes real-time photos. A multispectral image usually contains information of multiple spectral bands, while a normal RGB image usually only needs to extract information of the three RGB channels, and a multispectral image needs to extract information of a specific band.

[0108] A multispectral camera is a camera that can acquire image information in multiple bands (multiple spectral bandwidths). Unlike traditional color cameras, multispectral cameras are able to acquire images in a wider spectral range, from visible light to near-infrared light and even short-wave infrared light. Multispectral cameras are commonly used in remote sensing, agriculture, environmental monitoring, geological exploration, medical diagnosis and other fields, and can provide rich spectral information for analysis and research. A multispectral camera consists of multiple optical filters and corresponding photosensors, each of which selectively transmits light in a specific band and blocks light in other bands. In this way, when the light passes through the filter, only light in a specific band will be transmitted to the corresponding photosensor, forming an image of a spectral band.

[0109] Multispectral cameras can provide rich spectral information by acquiring image information from multiple spectral bands. When the spectral band corresponding to RGB is selected from the spectral band, it is the ordinary RGB image seen in daily life (visible light-blue: 0.450~0.515μm, visible light-green: 0.525~0.600μm, visible light-red: 0.630~0.680μm). Therefore, by extracting different bands of multispectral camera images, RGB images and multispectral images can be obtained.

[0110] In step S2, image preprocessing is performed on the common RGB image, including denoising, adjusting brightness and contrast, and cropping and scaling the common RGB image.

[0111] In step S3, generating a GLCM texture feature image for the multispectral image includes:

[0112] Step S31: limiting the grayscale of the multispectral image to a specific range and quantizing it into discrete grayscales to obtain a quantized image;

[0113] Step S32: for the quantized image, by traversing each pixel in the image, calculating the occurrence frequency of each pair of grayscale levels at a specific distance and angle;

[0114] Step S33: normalizing the statistical results of the grayscale pairs to obtain a GLCM matrix;

[0115] Step S34: Generate a GLCM texture feature image based on the GLCM matrix.

[0116] Figure 2 Given a sample pipeline oil leak image, Figure 5 To correspond to Figure 2 The generated GLCM texture feature image. Figure 5 As shown, the generated GLCM texture feature images include mean texture feature images, variance texture feature images, homogeneity texture feature images, contrast texture feature images, dissimilarity texture feature images, entropy texture feature images, angular second moment texture feature images, correlation texture feature images, and autocorrelation texture feature images. Three images that can highlight the characteristics of crude oil and refined oil are selected from the above texture feature images as three-channel data for the next step of feature extraction (i.e., step S5). In one embodiment, in Figure 5 The nine different characteristic texture generation graphs shown in the figure select the three data with prominent characteristics of crude oil and refined oil, namely [Mean, Entropy, AutoCorrelation], as the input data for the next step (i.e., step S5).

[0117] In one embodiment, when extracting GLCM texture features of multispectral images, the present invention selects gray level N=64, distance d=1 and angle θ=[0°, 45°, 90°, 135°], and the sliding window size is 7. The processing flow is as follows Figure 6 As shown, it includes acquiring a multispectral image with a pixel grayscale level of 256, grayscale quantizing the acquired multispectral image, and then downgrading the grayscale to 64 to obtain a grayscale image with a pixel grayscale level of 64, then performing sliding window processing, generating a co-occurrence matrix, calculating texture statistics, and finally outputting a texture image.

[0118] In step S4 and step S5, feature extraction is performed on the preprocessed RGB image and GLCM texture feature image, respectively, to generate an RGB image feature map and a GLCM texture feature map, which are used as inputs of the target detection algorithm. The present invention mainly uses the backbone network Darknet-53 of YOLOv3 as a feature extraction network, and performs feature extraction on the two types of data, respectively.

[0119] The backbone network Darknet-53 of YOLOv3 has the following structure: Figure 7 As shown in Figure 2, three feature maps of different sizes are generated through the extracted data, which will be used for subsequent data fusion processing.

[0120] In step S6, the feature-level fusion method is used to superimpose the three different-sized feature maps of the multispectral image and the RGB image after feature extraction at the channel level to generate the fused feature maps of the three sizes as the input for the final target detection. The calculation is as shown in formula 3-1.

[0121]

[0122] In step S7, the fused feature representation is used to perform target detection. The present invention passes the feature map after modality fusion into the improved feature pyramid network and the detection head, and finally obtains the detection result.

[0123] Combining the above seven steps, we can obtain the crude oil and refined oil target detection algorithm of this patent, such as Figure 8 As shown. This solution combines the multi-modality of multispectral images and RGB images, as well as the texture feature extraction GMCL algorithm, to achieve stable and accurate target detection. Compared with other solutions, this solution has lower hardware costs, lower requirements for the external environment, and stronger anti-interference ability on the basis of achieving good monitoring effects.

[0124] According to a second aspect of the present invention, there is provided a crude oil and refined oil pipeline leakage detection system based on multimodality and GLCM, which comprises a multispectral camera and a GPU server. The multispectral camera can be installed at a position that ensures that it can capture the position of the pipeline interface that is prone to problems. The GPU server is communicatively connected to the multispectral camera and is configured to execute the method according to the first aspect of the present invention.

[0125] To implement the method according to the first aspect of the present invention, the GPU server can be configured to include the following units or modules: an image acquisition unit, used to acquire a multispectral image and an ordinary RGB image; an image preprocessing unit, used to perform image preprocessing on the ordinary RGB image to obtain a preprocessed RGB image; a GLCM texture feature image generation unit, used to generate a GLCM texture feature image for the multispectral image; an RGB image feature extraction unit, used to perform feature extraction on the preprocessed RGB image to generate an RGB image feature map; a GLCM texture feature extraction unit, used to perform feature extraction on the GLCM texture feature image to generate a GLCM texture feature map; a fusion feature generation unit, used to superimpose the RGB image feature map and the GLCM texture feature map at the channel level to generate a fused feature map; and a target detection unit, used to perform target detection using the fused feature map.

[0126] The scheme of the present invention is to shoot with a multi-spectral camera, study the physical properties of crude oil and refined oil, detect and process the video based on multimodal, GLCM and YOLOv3 target detection algorithms, design special feature extraction and fusion algorithms, and achieve stable, efficient and accurate monitoring. In particular, the scheme of the present invention extracts and fuses ordinary RGB images and multi-spectral images, and different image information sources can provide complementary information, thereby helping to better distinguish crude oil, refined oil targets and background interference. At the same time, when performing feature extraction, a texture extraction algorithm based on gray level co-occurrence matrix (GLCM) is added, which improves the accuracy of model detection and the ability to resist external interference according to the special texture characteristics of crude oil and refined oil. The scheme of the present invention is improved on the basis of the YOLOv3 target detection network. Compared with the existing detection model, this model is more targeted at the physical properties of crude oil and refined oil, making the detection model more accurate and stable.

[0127] Compared with the existing crude oil and refined oil pipeline leakage detection system and method, the crude oil and refined oil pipeline leakage detection system and method based on multimodality and GLCM provided by the present invention has the following advantages:

[0128] 1. Reduce labor costs, deploy multi-spectral cameras and GPU computing equipment, and realize fully automatic detection and alarm.

[0129] 2. The ability to resist environmental interference has been improved. A multimodal fusion method combining ordinary RGB images and multispectral images has been adopted, and a variety of physical characteristic information of crude oil and refined oil has been comprehensively utilized, so that the model can stably detect oil leaks regardless of rainy days, dimness or interference from foreign objects.

[0130] 3. The GLCM texture feature extraction algorithm is used to better locate the texture features of crude oil and refined oil, improve the accuracy and efficiency of the model, and thus more accurately detect crude oil and refined oil leaks.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention, including the combination of various technical features in any other appropriate manner, these simple variations and combinations should also be regarded as the contents disclosed by the present invention and should be included in the protection scope of the present invention.

Claims

1. A crude oil and refined oil pipeline leakage detection method based on multimodal and GLCM, characterized in that: The following steps are involved: Step S1: Acquire a multispectral image and a normal RGB image; Step S2: performing image preprocessing on the common RGB image to obtain a preprocessed RGB image; Step S3: generating a GLCM texture feature image for the multispectral image; Step S4: extracting features from the preprocessed RGB image to generate an RGB image feature map; Step S5: extracting features from the GLCM texture feature image to generate a GLCM texture feature map; Step S6: superimpose the RGB image feature map and the GLCM texture feature map at the channel level to generate a fused feature map; Step S7: Use the fused feature map to perform target detection.

2. The method according to claim 1, characterized in that In step S1, a multispectral image and a common RGB image are acquired by a multispectral camera for real-time shooting.

3. The method according to claim 1, characterized in that In step S2, image preprocessing is performed on the common RGB image, including denoising, adjusting brightness and contrast, and cropping and scaling the common RGB image.

4. The method according to claim 1, characterized in that: In step S3, generating a GLCM texture feature image for the multispectral image includes: Step S31: limiting the grayscale of the multispectral image to a specific range and quantizing it into discrete grayscales to obtain a quantized image; Step S32: for the quantized image, by traversing each pixel in the image, calculating the occurrence frequency of each pair of grayscale levels at a specific distance and angle; Step S33: normalizing the statistical results of the grayscale pairs to obtain a GLCM matrix; Step S34: Generate a GLCM texture feature image based on the GLCM matrix.

5. The method according to claim 4, characterized in that The generated GLCM texture feature images include mean texture feature image, variance texture feature image, homogeneity texture feature image, contrast texture feature image, dissimilarity texture feature image, entropy texture feature image, angular second moment texture feature image, correlation texture feature image, and autocorrelation texture feature image.

6. The method according to claim 5, characterized in that In step S3, generating a GLCM texture feature image for the multispectral image further includes: Step S35: Select three images that can highlight the characteristics of crude oil and refined oil from the generated GLCM texture feature images as three-channel data for feature extraction in step S5.

7. The method according to claim 6, characterized in that The three images selected to highlight the characteristics of crude oil and refined oil are mean texture feature image, entropy texture feature image, and autocorrelation texture feature image.

8. The method according to claim 1, characterized in that In step S4 and step S5, the backbone network Darknet-53 of YOLOv3 is used as the feature extraction network to perform feature extraction.

9. The method according to claim 8, characterized in that In step S4, three RGB image feature maps of different sizes are generated through feature extraction; In step S5, three GLCM texture feature maps of different sizes are generated through feature extraction; In step S6, the RGB image feature maps of three different sizes and the GLCM texture feature maps of three different sizes are superimposed at the channel level to generate fused feature maps of three sizes.

10. A crude oil and refined oil pipeline leakage detection system based on multimodal and GLCM, characterized in that: include: Multispectral cameras; as well as A GPU computing device, the GPU computing device is communicatively connected to the multispectral camera and is configured to execute the method according to any one of claims 1 to 9.

Images