An oil spill detection method based on video monitoring

Abstract

The present application belongs to the technical field of oil spill detection, and particularly relates to an oil spill detection method based on video monitoring. The oil spill detection method overcomes the defects in the prior art, thereby better evaluating the oil spill area and being applicable to multiple working conditions, and having a better application prospect. The oil spill detection method comprises the steps of obtaining video images of the working area where the oil pipeline is located, learning the features of the shadow area and the features of the shadow boundary in the oil spill area, removing the shadow area from the video images, performing image segmentation on the oil spill area in the video images, performing feature extraction and feature combination on the area, complexity, shape factor and grayscale of the suspected oil spill area after the image segmentation, performing target detection on the suspected oil spill area after the feature combination, selecting the classification results, obtaining a total fuzzy set after clustering, performing integrity filtering on the total fuzzy set by using a defuzzification algorithm, and converting to obtain a corrected oil spill boundary.

Description

Technical Field

[0001] This invention belongs to the field of oil spill detection technology, and in particular relates to an oil spill detection method based on video surveillance. Background Technology

[0002] Oil spills from pipelines can cause significant damage to agriculture, forestry, fisheries, tourism, and the natural ecosystem, leading to serious environmental problems and posing significant safety risks to oil and petrochemical companies. Therefore, to promptly detect and monitor oil spills, engineers have proposed an oil spill video monitoring technology based on computer vision and intelligent video analysis. This video monitoring technology has broad application prospects and is currently an important research topic in oil spill monitoring. In recent years, the rapid development of video monitoring technology and the dramatic growth of neural network technology have led to increased attention to target detection, and related research has made significant progress. A video may contain multiple target segments. Detecting target regions in a given, unsegmented long video is crucial for understanding and applying oil spill detection. Current trends focus on segmenting oil spill areas and shadowed areas in the video for more accurate localization of whether an oil spill has occurred. Deep learning-based object detection algorithms can generally be divided into two categories: two-stage detection models and one-stage detection models. Two-stage detection models, such as RCNN, FastRCNN, Faster RCNN, Mask RCNN, and their improved versions, are the main representatives. These algorithms combine Region Proposal Networks (RPNs) with Convolutional Neural Networks (CNNs). One-stage detection models, such as SSD, YOLO, and their improved versions, are characterized by transforming object detection into a regression problem.

[0003] The most common approach is the R-CNN series. In object detection, the boundaries of oil spill areas are usually very clear, so relatively well-defined bounding boxes can be drawn. However, the presence of shadows at certain times makes the boundaries less clear, and defining the start and end points of an action often cannot provide an accurate boundary. This is similar to using an RNN to read features extracted by a CNN from each frame of the image. Existing algorithms, such as SCNN, CDC, and TURN, use and improve methods to generate candidate regions for object classification, suitable for simple and specific scenarios. However, in real-world engineering scenarios with many occlusions and shadows that change over time, the presence of shadows leads to inaccurate or large errors in image analysis, affecting the performance of many computer vision tasks, such as object detection. Therefore, shadow detection and shadow removal have always been fundamental problems in computer vision research. How to achieve accurate and efficient oil spill target detection in real-world engineering scenarios with numerous shadow interference factors and complex backgrounds has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides an oil spill detection method based on video surveillance. This method overcomes the shortcomings of existing technologies, thereby enabling better assessment of oil spill areas and adapting to multiple working conditions, thus having better application prospects.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A video surveillance-based oil spill detection method includes the following steps:

[0007] Step S1: Acquire video images of the work area where the oil pipeline is located, and preprocess the video images;

[0008] Step S2: Using two independent transformation networks, learn to detect the features of the shadowed areas and shadow boundaries in the oil spill area;

[0009] Step S3: Remove shadow areas from the video image;

[0010] Step S4: Perform image segmentation on the oil spill area in the video image;

[0011] Step S5: Extract and combine features from the area, complexity, shape factor, and grayscale of the suspected oil spill area after segmentation of the image;

[0012] Step S6: Perform target detection on the suspected oil spill area obtained from the feature combination in step S5 to obtain the classification result of the target in the two-dimensional image;

[0013] Step S7: Select the classification result obtained in step S6 as the input coordinate parameter of the fuzzy recognition model; generate a fuzzy set through the fuzzy logic system, and obtain the total fuzzy set after clustering;

[0014] Step S8: Use the defuzzification algorithm to perform integrity filtering on the total fuzzy set obtained in step S7, and transform it to obtain the corrected oil spill boundary.

[0015] Preferably, the process of learning and detecting the features of the shaded area in the oil spill area in step S2 can be specifically described as follows:

[0016] Using SLIC technology, by... s ×T s The shadow region features are extracted by spatially centering the centroid of each superpixel (ρ). r ); among which, the shadow region feature (ρ r Satisfy: ρ r (i, j) = W i，j (F slic (x), T s (i, j) represents the centroid space of the shadow region, x is the position of each pixel in the video image, p represents a specific model part, and W i，j (·) is used to extract features around the desired point of interest.

[0017] The preferred method for learning and detecting shadow boundary features in the oil spill area in step S2 can be specifically described as follows:

[0018] Using the gPb technique, traverse along each boundary with a step size λ, and extract a T at each step. s ×T s Spatial features, combined with the local upper and lower environment, determine the shadow boundary features (ρ) b ); among which, the shadow boundary feature (ρ b Satisfy: ρ b (i, j) = W i，j (F gPb (x), T s (i, j) represents the centroid space of the shadow boundary, x is the position of each pixel in the video image, p represents a specific model part, W i，j (·) is used to extract features around the desired point of interest.

[0019] Preferably, the process of removing shadow areas from the video image in step S3 can be specifically described as follows:

[0020] Using a three-color attenuation model and brightness information, shadow regions are segmented from video images;

[0021] The shadow areas are extracted and masked using guided filtering and robust matting algorithms;

[0022] The image is relit pixel by pixel in the shadowed areas of the video image to generate a shadowless image with the shadows removed.

[0023] Preferably, the image without shadows after removing the shadow areas. satisfy:

[0024] These are the average pixel values ​​for the unshaded area and the shaded area, respectively; X γ Let be a true color image containing shaded areas; γ∈{r, g, b}, representing the red, green, and blue channels in the true image, respectively.

[0025] Preferably, the process of image segmentation of the oil spill area in the video image in step S4 can be specifically described as follows:

[0026] Based on the maximum entropy method and the maximum inter-class variance method, the maximum entropy principle is used to select the optimal threshold to achieve image segmentation of oil spill areas in video images.

[0027] Preferably, the process of extracting and combining features of the area, complexity, shape factor, and grayscale of the suspected oil spill area in step S5 after image segmentation can be specifically described as follows:

[0028] A pyramid structure composed of multiple layers of networks is used to extract and combine features of the area, complexity, shape factor, and grayscale of suspected oil spill areas after image segmentation.

[0029] Preferably, the process of target detection of the suspected oil spill area obtained from the feature combination in step S5 in step S6 can be specifically described as follows:

[0030] An improved M2det model is used to detect suspected oil spill areas after feature combination obtained in step S5.

[0031] This invention provides an oil spill detection method based on video surveillance. The method includes the following steps: acquiring video images of the work area where the oil pipeline is located; learning and detecting the features of shadow areas and shadow boundaries in the oil spill area; removing shadow areas from the video images; segmenting the oil spill area in the video images; extracting and combining features of the area, complexity, shape factor, and grayscale of the suspected oil spill areas after segmentation; performing target detection on the suspected oil spill areas after feature combination to obtain a classification result of the presence of targets in the two-dimensional image; selecting the classification result as the input coordinate parameters of a fuzzy recognition model; generating a fuzzy set through a fuzzy logic system; obtaining a total fuzzy set after clustering; and using an anti-fuzzification algorithm to perform integrity filtering on the total fuzzy set to obtain a corrected oil spill boundary.

[0032] The video surveillance-based oil spill detection method with the above-described steps solves the problems of large detection errors and poor real-time performance in complex shadow scenes of existing technologies, and has at least the following characteristics:

[0033] (1) A shadow processing network framework was analyzed and designed, which can perform shadow detection and removal better than previous methods. Multiple transformation networks are used to automatically learn the most relevant feature representations from the original image. Extensive quantitative evaluation proves that the proposed framework is robust and less constrained, with high accuracy and speed, and has a wide range of application scenarios.

[0034] (2) The oil spill detection method provided by this invention can be applied to the complex scenario of oilfield operation area, realize accurate oil spill target detection, and try to combine the pyramid structure composed of multi-layer network for oil spill target detection and analysis in specific practical application scenarios, realizing a general framework for video surveillance to realize oil spill detection tasks.

[0035] (3) In actual engineering scenarios, the fuzzy logic system clustering output is used to process the fuzzy set using the membership function. Through the defuzzification method, the accurate distinction between real oil spill and suspected oil spill can be achieved. Detailed Implementation

[0036] This invention provides an oil spill detection method based on video surveillance. This method overcomes the shortcomings of existing technologies, thereby enabling better assessment of oil spill areas and adapting to multiple working conditions, thus having better application prospects.

[0037] Example 1

[0038] This invention provides a video surveillance-based oil spill detection method, specifically comprising the following steps:

[0039] Step S1: Acquire video images of the work area where the oil pipeline is located, and preprocess the video images;

[0040] It should be noted that, as a preferred embodiment of the present invention, the preprocessing of the video image can be referred to as follows: An untrimmed video V is input, and the video data is converted into a continuous L-frame RGB image through video sequence preprocessing. To generate random extraction frames, a hash function calculation is performed every 24 frames. Each time, a random function is selected, and the frame number of each frame is taken as its hash address, resulting in a randomly generated frame number, which is the extraction frame.

[0041] Step S2: Using two independent transformation networks, learn to detect the features of the shadowed areas and shadow boundaries in the oil spill area;

[0042] It should be noted that during feature learning through the transformation network, the same Conv Net architecture is used for feature learning at each feature layer: specifically, it includes alternating convolutional and subsampling layers as well as a fully connected layer preceding the output layer. This hierarchical structure enables the communication network to learn multi-layered features.

[0043] This network architecture takes an RGB patch as input and processes it to give the posterior distribution of the binary class. The output of each neuron is modeled as a nonlinear network function σ((·)) of its input, which is defined by the logistic sigmoid function: σ(x)=(1+e -x ) -1 .

[0044] The convolutional layers in the transformation network consist of filter banks that convolve with the input feature maps. Subsampling layers merge the outputs of adjacent groups of neurons into the same kernel map. Bilateral filters are applied to enhance edges before boundary extraction. A gPb boundary detector is used to find important boundaries in the image. After each boundary point, overlapping boundary patches are fed into a Conv Net for training along the boundary extraction window. The trained Conv Net distinguishes between shadow and reflection edges and predicts class assignment probabilities based on the trained weights. Next, primary shadow attributes are used to locate possible shadow regions, which are then assigned to the localized regions after prediction by the Conv Net.

[0045] Step S3: Remove shadow areas from the video image;

[0046] It should be noted that step S3 consists of two steps: accurately segmenting the shadow region from the video image and seamlessly reilluminating the segmented shadow region in RGB space (preferably with shadow edge restoration), thereby successfully generating a shadow-free image. In this process, qualitative and quantitative analysis is used to provide the rationale for individual components and their combinations in automatic shadow removal, with the aim of better preserving the color and texture of the shadow-free image.

[0047] As a preferred embodiment of the present invention, given a true color image X containing a shadow region... γ Where γ∈{r,g,b} represent the red, green, and blue channels, respectively; the correspondence between shadows and non-shadows is determined based on the degree of attenuation, i.e. in, This indicates that the shadow pixel vector is included, while Represents the non-shaded pixel vector, [λ r λ g λ b The image represents the attenuation vectors for the three color channels. Based on this, the shadow-free image after removing shadow areas can be calculated. satisfy: These are the average pixel values ​​for the unshaded area and the shaded area, respectively.

[0048] It is worth noting that in step S3 above, the image without shadows accurately preserves the color and texture of the original video image, but some errors appear near the boundaries of shadows. This is due to the significant changes in illumination around the penumbra region. To recover from these boundary errors, we use a sample-based patching technique. This technique replaces each erroneous pixel patch by searching for and copying the most similar patch pixels from the neighborhood, thus replicating the erroneous region with similar texture. Furthermore, applying a threshold to detect shadow regions can cause other non-shadow regions in the image to appear equally dark, affecting the segmentation accuracy of shadow region detection. Therefore, we need to combine intensity information with the image to address the shadow removal problem.

[0049] Step S4: Perform image segmentation on the oil spill area in the video image;

[0050] Step S5: Extract and combine features from the area, complexity, shape factor, and grayscale of the suspected oil spill area after segmentation of the image;

[0051] It should be noted that, as a preferred embodiment of the present invention, VGG technology is used to perform feature recognition (area, complexity, shape factor, and grayscale) and classification combination on the suspected oil spill area (video image) sequence after the entire image segmentation through a pyramid structure composed of multiple layers. The input data is a series of frames, each with a set of coordinates. A feature map is constructed using graph structure rules, utilizing grayscale and texture features. Then, the category features of each oil spill area are extracted, and the features from different convolutional layers are fused and fed as basic features into a U-shaped module for encoding and decoding. These features are then fused with the basic features to generate multi-layer, multi-scale features. Finally, the multi-layer, multi-scale features are fused to construct a detection pyramid for target detection. The feature formula for this construction can be found as follows:

[0052] in, T represents the feature vector of the input image, respectively. l (·) denotes the sampling function, which is used to calculate the inner product of the sampled input feature vectors with dimension l. F(·) denotes the weight function, which is independent of the input vector x. Therefore, all filter weights on the input image are shared. The standard convolution in the image domain fuses features by encoding the rectangular grid in the FFMV (Feature Fusion Module).

[0053] Step S6: Perform target detection on the suspected oil spill area obtained from the feature combination in step S5 to obtain the classification result of the target in the two-dimensional image;

[0054] It should be noted that, as a preferred embodiment of the present invention, the improved M2Det model is preferably used. Because the network structure is very compact and requires little storage, frame-level recognition and classification can be trained directly from the original video, performing oil spill area detection in an end-to-end manner without needing to cache intermediate features. Compared to the SCNN network, although the present invention increases workload by predicting each video frame, and efficiency decreases for higher accuracy as video length increases, when the proposed segments are short, video segment-level methods must process a large number of segments individually because these proposed segments are usually densely overlapping. The present invention, however, only needs to process all video frame images once, thus avoiding redundant computation.

[0055] Furthermore, due to GPU memory limitations, the algorithm actually uses a 16-frame overlapping time window in the video, feeding each window separately into the network to obtain dense predictions in real time. The label for each frame can be obtained from the oil spill boundary annotations; frames within the same window can have different labels. To prevent including too many background frames during training, this method only retains windows with at least one frame belonging to the oil spill area. Therefore, in a given set of video data, a set of oil spill areas corresponding to frame-level labeled windows can be obtained.

[0056] Step S7: Select the classification result obtained in step S6 as the input coordinate parameter of the fuzzy recognition model; generate a fuzzy set through the fuzzy logic system, and obtain the total fuzzy set after clustering;

[0057] It should be noted that the classification results obtained in step S6 may still contain incomplete or redundant instances. Therefore, a specific fuzzy logic system is needed to filter out these fragments to ensure the integrity of the results, and position regression is used to optimize the time interval of the proposal segment itself. Specifically, further screening is performed based on the characteristic that the sample feature values ​​satisfy an approximately normal distribution. At the same time, feature parameters that show a certain distribution pattern among the results are determined. Finally, five feature parameters are selected, including background mean, variance ratio (RBSDO), complexity (COM), shape factor (SF), and angular second moment (ASM). The above five input feature parameters and one output classification result parameter are clustered to define the total output fuzzy set, which satisfies the following formula: Where x represents a fuzzy variable with characteristic significance, σ represents the probability statistics of the characteristic parameter and the possibility of oil spill, and m is a characteristic variable labeled as a suspected oil spill.

[0058] Step S8: Use the defuzzification algorithm to perform integrity filtering on the total fuzzy set obtained in step S7, and transform it to obtain the corrected oil spill boundary.

[0059] Finally, an anti-fuzzy algorithm is used to transform the total fuzzy set of fuzzy outputs into a definite output value. In a preferred embodiment of this invention, the centroid method can be used to take the ordinate function value corresponding to the centroid of the region enclosed by the membership function curve of the fuzzy cluster output fuzzy set and the horizontal axis as the output value. The output value of the fuzzy logic classification system for oil spills and suspected oil spills represents the probability that the target is an oil spill.

[0060] Example 2

[0061] Embodiment 2 includes all the technical features of Embodiment 1; in addition, Embodiment 2 further limits step S2 as follows.

[0062] It is worth noting that the two independent transformation networks used in step S2 of this invention are for extracting shadow region features and determining shadow boundary features, respectively.

[0063] In this context, Conv Nets for locating shadow region features operate on windows of equal size, thus requiring window operations. Considering efficiency, for region pixels, superpixels are extracted by clustering homogeneous pixels. Specifically, this can be described as using the SLIC technique to extract superpixels by clustering T… s ×T s The shadow region features are extracted by spatially centering the centroid of each superpixel (ρ). r ); among which, the shadow region feature (ρ r Satisfy: ρ r (i, j) = W i，j (F slic (x), T s (i, j) represents the centroid space of the shadow region, x is the position of each pixel in the video image, p represents a specific model part, and W i，j (·) is used to extract features around the desired point of interest.

[0064] The localization of shadow boundary features can be specifically described as follows: using gPb technology, traversing along each boundary with a step size λ, and extracting a T at each step. s ×T s Spatial features, combined with the local upper and lower environment, determine the shadow boundary features (ρ) b ); among which, the shadow boundary feature (ρ b Satisfy: ρ b (i, j) = W i，j (F gPb (x), T s (i, j) represents the centroid space of the shadow boundary, x is the position of each pixel in the video image, p represents a specific model part, and W i,j (·) is used to extract features around the desired point of interest.

[0065] Example 3

[0066] Example 3 includes all the technical features of Example 1; in addition, Example 3 further limits step S3 as follows.

[0067] It should be noted that step S3, mentioned earlier, consists of two steps: accurately segmenting the shadow region from the video image and seamlessly reilluminating the segmented shadow region in RGB space. In this process, as a preferred embodiment of the present invention, the shadow region is first segmented from the video image using a three-color attenuation model and luminance information; then, guided filtering and robust matting algorithms are used to extract and mask the shadow region, thus achieving the goal of accurately segmenting the shadow region from the video image. Finally, pixel reillumination (preferably shadow edge restoration) is performed on the masked shadow region in the video image to generate a shadow-free image with the shadow region removed.

[0068] This invention provides an oil spill detection method based on video surveillance. The method includes the following steps: acquiring video images of the work area where the oil pipeline is located; learning and detecting the features of shadow areas and shadow boundaries in the oil spill area; removing shadow areas from the video images; segmenting the oil spill area in the video images; extracting and combining features of the area, complexity, shape factor, and grayscale of the suspected oil spill area after segmentation; performing target detection on the suspected oil spill area after feature combination to obtain a classification result of the presence of targets in the two-dimensional image; selecting the classification result as the input coordinate parameters of a fuzzy recognition model; generating a fuzzy set through a fuzzy logic system; obtaining a total fuzzy set after clustering; and using an anti-fuzzification algorithm to perform integrity filtering on the total fuzzy set obtained in step S7 to obtain a corrected oil spill boundary.

[0069] The video surveillance-based oil spill detection method with the above-described steps solves the problems of large detection errors and poor real-time performance in complex shadow scenes of existing technologies, and has at least the following characteristics:

[0070] (1) A shadow processing network framework was analyzed and designed, which can perform shadow detection and removal better than previous methods. Multiple transformation networks are used to automatically learn the most relevant feature representations from the original image. Extensive quantitative evaluation proves that the proposed framework is robust and less constrained, with high accuracy and speed, and has a wide range of application scenarios.

[0071] (2) The oil spill detection method provided by this invention can be applied to the complex scenario of oilfield operation area, realize accurate oil spill target detection, and try to combine the pyramid structure composed of multi-layer network for oil spill target detection and analysis in specific practical application scenarios, realizing a general framework for video surveillance to realize oil spill detection tasks.

[0072] (3) In actual engineering scenarios, the fuzzy logic system clustering output is used to process the fuzzy set using the membership function. Through the defuzzification method, the accurate distinction between real oil spill and suspected oil spill can be achieved.

[0073] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting oil spills based on video surveillance, characterized in that, The steps include the following: Step S1: Acquire video images of the work area where the oil pipeline is located, and preprocess the video images; Step S2: Using two independent transformation networks, learn to detect the features of the shadowed areas and shadow boundaries in the oil spill area; Step S3: Remove shadow areas from the video image; Step S4: Perform image segmentation on the oil spill area in the video image; Step S5: Extract and combine features from the area, complexity, shape factor, and grayscale of the suspected oil spill area after image segmentation; Step S6: Perform target detection on the suspected oil spill area obtained from the feature combination in step S5 to obtain the classification result of the target in the two-dimensional image; Step S7: Select the classification result obtained in step S6 as the input coordinate parameter of the fuzzy recognition model; Fuzzy sets are generated through a fuzzy logic system, and the final fuzzy set is obtained after clustering. Step S8: Use the defuzzification algorithm to perform integrity filtering on the total fuzzy set obtained in step S7, and transform it to obtain the corrected oil spill boundary; The process of learning and detecting the features of the shaded areas in the oil spill area in step S2 can be specifically described as follows: Using SLIC technology, by... The shadow region features are extracted by spatially centering the centroid of each superpixel. Among them, the characteristics of the shaded area satisfy: ; The centroid space representing the shaded region, It is the position of each pixel in the video image. This represents a specific part of the model. To extract features around the desired point of interest; The process of learning and detecting shadow boundary features in the oil spill area in step S2 can be specifically described as follows: Using gPb technology, with step size Traverse along each boundary and extract one at each step. Spatial features, combined with the local upper and lower environment, determine the shadow boundary features. Among them, shadow boundary features satisfy: ; The centroid space representing the shadow boundary, It is the position of each pixel in the video image. This represents a specific part of the model. To extract features around the desired point of interest.

2. The oil spill detection method based on video surveillance according to claim 1, characterized in that, The process of removing shadow areas from the video image in step S3 can be specifically described as follows: Using a three-color attenuation model and brightness information, shadow regions are segmented from video images; The shadow areas are extracted and masked using guided filtering and robust matting algorithms; The image is relit pixel by pixel in the shadowed areas of the video image to generate a shadowless image with the shadows removed.

3. The oil spill detection method based on video surveillance according to claim 2, characterized in that, The shadowless image after removing the shadow areas ,satisfy: ; ; , These are the average pixel values ​​for the unshaded area and the shaded area, respectively. A true color image that includes shaded areas; , representing the red, green, and blue channels in a real image, respectively.

4. The oil spill detection method based on video surveillance according to claim 1, characterized in that, The process of image segmentation of the oil spill area in the video image in step S4 can be specifically described as follows: Based on the maximum entropy method and the maximum inter-class variance method, the maximum entropy principle is used to select the optimal threshold to achieve image segmentation of oil spill areas in video images.

5. The oil spill detection method based on video surveillance according to claim 1, characterized in that, The process of extracting and combining features of the area, complexity, shape factor, and grayscale of the suspected oil spill area in step S5 can be specifically described as follows: A pyramid structure composed of multiple layers of networks is used to extract and combine features of the area, complexity, shape factor, and grayscale of suspected oil spill areas after image segmentation.

6. The oil spill detection method based on video surveillance according to claim 1, characterized in that, The process of target detection of the suspected oil spill area obtained from the feature combination in step S5 in step S6 can be specifically described as follows: An improved M2det model is used to detect suspected oil spill areas after feature combination obtained in step S5.

Images