Oil leakage detection method and detection system and detection robot based on spatial visual feature detection

Abstract

This invention provides an oil spill detection method, system, and robot based on spatial visual feature detection, relating to the fields of computer vision and image processing. It addresses the problems of poor model generalization ability, low detection accuracy, high hardware requirements, and slow detection speed in existing methods. The key technical points of this invention include the following steps: S1, acquiring oil spill image data, performing data augmentation and image preprocessing to construct an oil spill image dataset; S2, constructing an oil spill detection model based on YOLOv12, introducing a CBAM module to replace the A2C2F module in the model's Neck to improve the model's ability to learn detailed information in oil spill images; simultaneously, inserting a ViT module in the first layer of the model's Head to improve the model's global feature learning ability and multi-scale feature fusion capability; S3, detecting oil spills based on the oil spill detection model.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and more specifically, to an oil spill detection method, detection system, and detection robot based on spatial visual feature detection. Background Technology

[0002] With the rapid development of the petrochemical industry, oil pipeline, storage tank, and transport vehicle leaks occur frequently, causing serious damage to soil, water sources, and the surrounding ecological environment. Currently, oil leak monitoring mainly relies on manual inspections, fixed sensors, or robotic aerial photography. Manual inspections are inefficient and difficult to cover remote areas; fixed sensors are costly to deploy and lack flexibility; while traditional oil leak detection robots mostly rely on visible light image analysis, which is easily affected by changes in lighting, vegetation obstruction, and ground debris, resulting in high false positive and false negative rates. Especially in complex terrain conditions, oil leak areas are often similar in color to soil, water accumulation, or shaded areas, making it difficult for traditional image processing methods (such as threshold segmentation and color space conversion) to achieve stable identification.

[0003] In recent years, deep learning-based target detection technology has begun to be applied in the field of environmental monitoring, but existing algorithms still have significant shortcomings in oil spill detection scenarios. Taking the YOLO series algorithms as an example, although their detection speed meets real-time requirements, standard versions (such as YOLOv5 / v7) have limited feature extraction capabilities for land spills: on the one hand, the boundaries of the contaminated areas formed after oil seepage are blurred and irregular in shape, and have low contrast with the ground background, making it difficult for the model to learn effective features; on the other hand, interference objects in the ground environment, such as water reflections and oil deposits, are easily misidentified as leak targets. In addition, existing public datasets are mostly based on marine oil spills, lacking diverse samples for land scenarios (such as leak images under different soil types and vegetation cover conditions), further limiting the generalization performance of the models. Therefore, developing an oil spill detection algorithm that balances high accuracy and real-time performance is of great significance for realizing automated oil spill detection. Summary of the Invention

[0004] The technical problem to be solved by this invention is:

[0005] Existing oil spill detection methods suffer from poor model generalization ability, low detection accuracy, high hardware requirements, and slow detection speed. To address these issues, this invention provides an oil spill detection method, system, and robot based on spatial visual feature detection.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] This invention provides a method for detecting oil spills based on spatial visual features, comprising the following steps:

[0008] S1. Collect oil spill image data, perform data augmentation and image preprocessing on the data, and construct an oil spill image dataset;

[0009] S2. Construct an oil spill detection model based on YOLOv12. In the model's Neck, introduce the CBAM module to replace the A2C2F module to improve the model's ability to learn detailed information in oil spill images. At the same time, insert the ViT module in the first layer of the model's Head to improve the model's global feature learning ability and multi-scale feature fusion ability.

[0010] S3. Detect oil leaks based on the oil leak detection model.

[0011] Furthermore, the data augmentation process described in step S1 involves randomly scaling the four input images (I1, I2, I3, I4) and then stitching them together into a single image I. mosaic .

[0012] Furthermore, the data augmentation process described in step S1 also includes introducing improved Copy-Paste enhancement, specifically: from the source image I s Segment the oil spill area M, add semantic segmentation labels, and paste region M into the target image I. t middle:

[0013] I paste =I t ⊙(1-M)+I s ⊙M, where I paste This is the new image after pasting;

[0014] Poisson fusion is used to eliminate edge artifacts on the new image.

[0015] Furthermore, the functional implementation process of the ViT module is as follows:

[0016] First, the input feature map is segmented and embedded, dividing the input feature map F into blocks. There are 3 patches, each patch being P×P in size, i.e. Where B is the batch size; the feature map is linearly mapped to D-dimensional space to obtain: X embed =X·W P ,in,

[0017] Then, learnable positional encodings are added to the feature map X in D-dimensional space. embed Position labeling is performed, at which point the Patch embedding sequence is X. pos =X embed +Epos ,in, Added positional encoding; the feature map X with learnable positional encoding will be added. pos The input from the Transformer encoder is X. out =TransformerEncoder(X pos ), where TransformerEncoder represents the nonlinear transformation processing of the multi-head attention mechanism MHSA and the feedforward network FFN;

[0018] Finally, the output of the Transformer encoder is linearly projected, mapped back to the Patch space, and reassembled to obtain F. out =Resh ape(X out This converts the output sequence into a two-dimensional image.

[0019] Furthermore, during the training process of the YOLOv12-based oil spill detection model, simulated annealing algorithm is used to dynamically adjust the network learning rate: Where k is the number of restarts, and These are the maximum and minimum values ​​of the learning rate, T. now For the current number of training iterations, T k This represents the total number of training iterations.

[0020] This invention provides an oil spill detection system based on spatial visual feature detection. The system has a program module corresponding to the steps of any of the above-described technical solutions, and executes the steps of the above-described oil spill detection method based on spatial visual feature detection when running.

[0021] The present invention also provides a computer-readable storage medium storing a computer program configured to, when invoked by a processor, implement the steps of the oil spill detection method based on spatial visual feature detection as described in any of the above technical solutions.

[0022] The present invention also provides an oil spill detection robot based on spatial visual feature detection. The detection robot includes at least one processor, a memory communicatively connected to the at least one processor, and a device for acquiring oil spill image data and sending the data information to the processor. The memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the oil spill detection method based on spatial visual feature detection as described in any of the above technical solutions, thereby realizing oil spill detection.

[0023] Compared with the prior art, the beneficial effects of the present invention are:

[0024] This invention constructs and augments an oil spill image dataset, enriching its quantity and content and enhancing the model's learning richness. The YOLOv12-based oil spill detection model addresses the challenges of a small dataset with numerous target samples and complex image backgrounds by replacing some of the original A2C2F modules with CBAM modules. This improves the model's ability to learn spatial and channel information within the dataset, enhancing its ability to capture detailed features and thus increasing detection accuracy. Furthermore, considering the large size and training difficulty of the YOLOv12 model, and the rich multi-scale feature information in the training set, the ViT (Vision Transformer) module is integrated into YOLOv12. Its residual learning characteristics improve the model's generalization ability and learning efficiency, reduce hardware requirements, and simplify the learning process and deployment. Attached Figure Description

[0025] Figure 1 This is a flowchart of the oil spill detection method based on spatial visual feature detection in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram illustrating the Mosaic enhancement and improved Copy-Paste enhancement effects in the embodiments of the present invention;

[0027] Figure 3 This is an example diagram of the network model structure of the improved YOLOv12 in an embodiment of the present invention;

[0028] Figure 4 This is an example diagram of the ViT module structure in an embodiment of the present invention;

[0029] Figure 5 This is an example diagram of the structure of an oil spill detection system based on spatial visual feature detection in an embodiment of the present invention;

[0030] Figure 6 This is a rendering of an embodiment of the oil spill detection system based on spatial visual feature detection according to the present invention.

[0031] Figure 7 This is an example diagram of the structure of an oil spill detection robot based on spatial visual feature detection in an embodiment of the present invention;

[0032] Figure 8 This is an example diagram of the detection results in an embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, exemplary embodiments or examples of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments or examples are merely some, not all, of the embodiments or examples of the present invention. All other embodiments or examples obtained by those skilled in the art based on the embodiments or examples of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0034] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0035] Specific Implementation Plan 1: Combining Figure 1 As shown, this invention provides an oil spill detection method based on spatial visual feature detection, comprising the following steps:

[0036] S1. Collect oil spill image data, perform data augmentation and image preprocessing on the data, and construct an oil spill image dataset;

[0037] S2. Construct an oil spill detection model based on YOLOv12. In the model's Neck, introduce the CBAM module to replace the A2C2F module to improve the model's ability to learn detailed information in oil spill images. At the same time, insert the ViT module in the first layer of the model's Head to improve the model's global feature learning ability and multi-scale feature fusion ability.

[0038] S3. Detect oil leaks based on the oil leak detection model.

[0039] Because the original model is large and difficult to modify, after introducing a new module, it is necessary to ensure that the computational dimensions of the front and back layers are consistent, and also to ensure that the improved network can improve computational accuracy, computational speed, and the ability to analyze datasets.

[0040] Specific Implementation Scheme Two: In step S1, oil spill image data is collected using robot photography and manual on-site photography to obtain an initial dataset; the data augmentation process is performed using Mosaic enhancement: the four input images (I1, I2, I3, I4) are randomly scaled and then stitched together to form a single image I. mosaic To improve the model's ability to detect small targets and enhance background diversity, such as Figure 2 As shown in (a).

[0041] Among them, (x i y i The random cropping coordinates are used to preserve the geometric transformation of the original annotation box. This implementation scheme is otherwise the same as specific implementation scheme one.

[0042] Specific implementation plan three: The data augmentation process also includes introducing improved Copy-Paste enhancement, specifically: from source image I s Segment the oil spill area M, add semantic segmentation labels, and paste region M into the target image I. t In order to improve the richness and diversity of the samples and enhance the generalization ability of the model, such as... Figure 2 As shown in (b).

[0043] I paste =I t ⊙(1-M)+I s ⊙M, where I paste This is the new image after pasting;

[0044] Meanwhile, Poisson Blending technology is applied to the new image to eliminate edge artifacts, making the transition between the oil stain area and the target background more natural.

[0045] This implementation scheme uses Mosaic enhancement and improved Copy-Paste enhancement techniques to perform data augmentation operations on the image dataset, increasing the quantity and richness of the dataset, and improving the feature diversity of the dataset;

[0046] Step S1 also includes: using the Labelimg plugin to annotate all processed images, labeling the oil spill portions of the images with "Oil spills", and saving the annotated images and labels to generate an oil spill image dataset, which is then divided into training, validation, and test sets proportionally. The rest of this implementation scheme is the same as in specific implementation scheme two.

[0047] Specific Implementation Plan Four: (e.g.) Figure 3 As shown, the YOLOv12-based oil spill detection model in step S2 consists of three parts: a backbone feature extraction network, a Neck feature fusion network, and a head detection network. The backbone feature extraction network includes a Conv module, a C3K2 module, and an A2C2F module. The Conv module has a 1×1 convolutional kernel size and a stride of 1. The C3K2 module is set to False and consists of a 1×1 Conv module and a Bottleneck module. The feature information is processed by the Conv module and the Bottleneck module respectively and then residually connected to reduce the computational cost of the model. The A2C2F module is set to True and consists of a 1×1 Conv module and an ABlock module. The ABlock module contains an AAttn attention mechanism module and an MLP multilayer perceptron module.

[0048] The model's neck comprises a Conv module, a C3K2 module, a CBAM module, and an upsampling module. The Conv module has a 1×1 kernel and a stride of 1. The C3K2 module, set to True, consists of 1×1 Conv and C3K modules. Feature information is processed by the Conv and Bottleneck modules and then stitched together at multiple scales to improve the model's ability to learn multi-scale features. The CBAM module enhances the model's ability to learn detailed information in oil spill images. The A2C2F module, set to False, consists of 1×1 Conv and C3K modules. The CBAM attention mechanism module replaces the original A2C2F module, further improving the model's ability to learn detailed information in oil spill images.

[0049] The model's detection head includes the Conv module, the Conv2d module, and the DWConv module; the Vision Transformer (ViT) module is inserted at the first layer position in the head.

[0050] In this implementation scheme, the CBAM module includes two sub-modules: a channel attention mechanism module and a spatial attention mechanism module.

[0051] For channel attention mechanisms, M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))), that is

[0052] in, The input feature map is C, where C is the number of channels, H is the height, and W is the width. AvgPool(F) and MaxPool(F) represent global average pooling and max pooling along the spatial dimensions, respectively. The output is... The MLP is a two-layer fully connected layer with shared weights, containing ReLU activation. W1 and W0 correspond to the weights of the two layers, respectively. σ is the Sigmoid function, and M... c (F) represents the output weights of the channel attention mechanism.

[0053] For the spatial attention mechanism, based on the channel attention mechanism, the results of global max pooling and global average pooling are concatenated according to channels, and a convolution operation is performed on the concatenated result to obtain the output weight M. s (F′) is the final output. M s (F′)=σ(f[AvgPool(F′);MaxPool(F′)])

[0054] Where F′ is the output feature map of the channel attention mechanism, and f is the convolution operation, typically using a 7×7 convolution kernel. Specifically, This indicates multiplication by channel.

[0055] The final output F″ of CBAM can be written as This implementation plan is otherwise the same as Implementation Plan 3.

[0056] Specific Implementation Plan Five: (e.g.) Figure 4 As shown, the functional implementation process of the ViT module is as follows:

[0057] First, the input feature map is segmented and embedded, dividing the input feature map F into blocks. There are 3 patches, each patch having a size of P×P, i.e. Where B is the batch size; the feature map is linearly mapped to D-dimensional space to obtain: X embed =X·W P ,in,

[0058] Then, learnable positional encodings are added to the feature map X in D-dimensional space. embed Position labeling is performed, at which point the Patch embedding sequence is X. pos =X embed +E pos ,in, Added positional encoding; the feature map X with learnable positional encoding will be added. pos The input from the Transformer encoder is X. out =TransformerEncoder(X pos The TransformerEncoder represents the nonlinear transformation processing of the Multi-Head Attention (MHSA) mechanism and the Feedforward Network (FFN), thereby enhancing the model's feature representation capabilities. Specifically, Where Q is the query, K is the key, K is the value, T is the sequence length, Softmax represents the normalization operation, and D... h This represents the dimension of each Head, and the output value of the multi-head attention mechanism is fed into the FFN to calculate X. out .

[0059] Finally, the output of the Transformer encoder is linearly projected, mapping the output value of the feedforward network FFN back to the patch space and reassembling it to obtain F. out =Resh ape(X out This ensures the output dimension while converting the output sequence into a two-dimensional image. Other aspects of this implementation scheme are the same as in specific implementation scheme four.

[0060] Specific Implementation Plan Six: The weight files trained on the official COCO dataset are used as the initial training weights for the YOLOv12-based oil spill detection model. The image training set from step S1 is used for model training. Simulated annealing is employed to dynamically adjust the network learning rate, and K-fold cross-validation is used to adjust model parameters and reduce overfitting. The network learning rate is calculated as follows during training: Where k is the number of restarts, and These are the maximum and minimum values ​​of the learning rate, T. now For the current number of training iterations, T k This represents the total number of training iterations.

[0061] The trained model and all trained weights are applied to the validation and test sets, and the optimal weights are saved. This implementation scheme is otherwise the same as specific implementation scheme five.

[0062] The oil spill detection method (algorithm) based on spatial visual feature detection proposed in this invention is the underlying technical core of this invention, and various products can be derived based on the algorithm.

[0063] Based on the method proposed in this invention, an oil spill detection system based on spatial visual feature detection has been developed using a programming language. This system has program modules corresponding to the steps of the aforementioned technical solution, and executes the steps in the above-described oil spill detection method based on spatial visual feature detection during runtime. This system meets the operating environment requirements of the method, improves the smoothness of its operation, and simplifies the operation process.

[0064] Based on the oil spill detection model and its optimal weights, an oil spill detection system based on spatial visual feature detection is built. For example... Figure 5 As shown, it specifically includes a user interface module, an oil spill detection module, and a data management module. The user interface module includes account registration and login functions; the oil spill detection module includes image capture and oil spill detection functions; and the data management module includes account information management, detection data query, and remarks functions.

[0065] Functional test results are as follows Figure 6 As shown. Among them, Figure 6 (a) This shows the implementation effect of the user interface module; the account login and registration functions are running stably. Figure 6 (b) The implementation effect of the oil leak detection module, which deploys the oil leak detection method based on spatial visual feature detection proposed in this invention, and can operate the corresponding functions stably and with high accuracy; Figure 6 (c) The data management module is implemented effectively, and functions such as account information management and log query are running stably.

[0066] An oil spill detection system based on spatial visual feature detection is deployed on an oil spill detection robot. The system controls the robot to capture images for actual detection. In this embodiment, an oil spill detection robot is used as the system's platform. The robot's structure is as follows... Figure 7 As shown, it features a WIFI communication system, a vision camera, a fill light, an infrared sensing system, a GPS module, and an embedded AI processor such as a Raspberry Pi / Rockchip. The infrared sensing system is used to assist in detecting temperature differences in the oil film.

[0067] The developed system (software) computer program is stored on a computer-readable storage medium. This computer program is configured to implement the steps of the aforementioned oil spill detection method based on spatial visual feature detection when invoked by a processor. In other words, the invention is materialized on a carrier, becoming a computer program product. Examples of detection results are shown below. Figure 8 As shown, by Figure 8 (a) It can be seen that the present invention is capable of detecting a large-area oil leak from a single target in close-range, conventional scenarios; Figure 8 (b) It is evident that the present invention is capable of detecting multi-target oil leaks in complex scenarios involving pipelines, instruments, etc.; Figure 8 (c) It can be seen that the present invention can detect oil leaks from multiple targets in a scene containing sand and shadows that are highly similar to oil at a distance; the accuracy and feasibility of the present invention are demonstrated from multiple perspectives.

[0068] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0069] The computational programs (also referred to as programs, software, software applications, or code) of this invention include machine instructions of a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0070] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for detecting oil spills based on spatial visual features, characterized in that: Includes the following steps: S1. Collect oil spill image data, perform data augmentation and image preprocessing on the data, and construct an oil spill image dataset; S2. Construct an oil spill detection model based on YOLOv12. In the model's Neck, introduce the CBAM module to replace the A2C2F module to improve the model's ability to learn detailed information in oil spill images. At the same time, insert the ViT module in the first layer of the model's Head to improve the model's global feature learning ability and multi-scale feature fusion ability. S3. Detect oil leaks based on the oil leak detection model; The CBAM module comprises two sub-modules: a channel attention mechanism module and a spatial attention mechanism module. The functional implementation process of the ViT module is as follows: First, the input feature map is segmented and embedded. Divided into There are 1 patch, and the size of each patch is 1. ,Right now ,in, Set the batch size; linearly map the feature map to... We obtain the following from dimensional space: ,in, ; Then, learnable positional encodings are added to the feature maps in D-dimensional space. Position labeling is performed, at which point the Patch embedding sequence is... ,in, Added positional encoding; feature maps with learnable positional encodings will be added. The input from the Transformer encoder is as follows: ,in This represents the nonlinear transformation processing of the Multi-Head Attention (MHSA) mechanism and the Feedforward Network (FFN). Finally, the output of the Transformer encoder is linearly projected, mapped back to the Patch space, and reassembled to obtain... This converts the output sequence into a two-dimensional image.

2. The oil spill detection method based on spatial visual feature detection according to claim 1, characterized in that: The data augmentation process described in step S1 involves processing the four input images... Randomly scaled and stitched together into a single image .

3. The oil spill detection method based on spatial visual feature detection according to claim 2, characterized in that: The data augmentation process described in step S1 also includes introducing improved copy-paste enhancement, specifically: from the source image Demarcate the oil spill area Includes semantic segmentation labels, dividing regions Paste to target image middle: ,in, This is the new image after pasting; Poisson fusion is used to eliminate edge artifacts on the new image.

4. The oil spill detection method based on spatial visual feature detection according to claim 3, characterized in that: During the training process of the YOLOv12-based oil spill detection model, simulated annealing algorithm is used to dynamically adjust the network learning rate. ,in, For the number of restarts, and These are the maximum and minimum values ​​of the learning rate, respectively. For the current number of training iterations, This represents the total number of training iterations.

5. An oil spill detection system based on spatial visual feature detection, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 4, and executes the steps in the above-described oil spill detection method based on spatial visual feature detection when it is run.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of the oil spill detection method based on spatial visual feature detection as described in any one of claims 1 to 4.

7. An oil spill detection robot based on spatial visual feature detection, characterized in that: The detection robot includes at least one processor, a memory communicatively connected to the at least one processor, and a device for acquiring oil spill image data and sending the data information to the processor; wherein, the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the oil spill detection method based on spatial visual feature detection as described in any one of claims 1 to 4, thereby realizing oil spill detection.

Images