Unmanned aerial vehicle aerial photography petroleum leakage intelligent detection method and system fused with MobileNetV4 lightweight network

Abstract

The invention provides an unmanned aerial vehicle aerial photography petroleum leakage intelligent detection method and system fused with a MobileNetV4 lightweight network, and belongs to the field of petroleum leakage detection. The method aims to solve the problems of low accuracy and high missing report / false report rate caused by excessive background interference, low resolution and noise of a data set in the petroleum leakage detection process. And the model parameter quantity is large and cannot be deployed on the unmanned aerial vehicle equipment. The method comprises the following steps: constructing a petroleum leakage data set, and carrying out data preprocessing; a MobileNetV4 network model is constructed; building an experimental environment, initializing grid parameters, and training a model; and evaluating the effectiveness of the model in petroleum leakage detection through a standard evaluation index. According to the method, the accuracy of petroleum leakage detection can be remarkably improved, the missing report rate and the false report rate are remarkably reduced, meanwhile, the parameter quantity and the calculation complexity of the model are reduced, and real-time detection tasks can be conveniently deployed and efficiently executed on the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of oil spill detection. Specifically, it relates to an intelligent oil spill detection method and system for UAV aerial photography that integrates the MobileNetV4 lightweight network. Background Art

[0002] Marine oil detection technology is the core defense against the dual ecological and economic problems of oil spills, due to the irreversible long-term harm of oil spills: the spilled oil spreads rapidly under the action of ocean currents and wind, which will not only directly poison marine life and destroy sensitive habitats such as coral reefs, leading to a cliff-like decline in biodiversity, but also be amplified through the food chain, polluting seafood such as shellfish and fish, and ultimately threatening human health (for example, long-term ingestion of contaminated seafood can increase the risk of inducing liver cancer).

[0003] With the continuous development of computer vision technology, object detection technology based on deep learning has been widely used in the field of offshore oil detection. There are two existing architectures for object detection: based on convolutional neural network CNN (such as the YOLO series) and based on Transformer (such as the DETR series). The object detector based on Transformer (DETR series) eliminates some components (anchor box design and non-maximum suppression (NMS) in traditional object detectors) and achieves remarkable performance, simplifies the object detection process, and realizes end-to-end object detection. However, DETR requires extremely high computational overhead and is difficult to perform real-time detection. The YOLO (You Only Look Once) series of models show significant application value in oil spill detection due to their unique architecture design and performance advantages. Compared with two-stage object detection models such as Faster R-CNN and Mask R-CNN, the YOLO series of models adopt a single-stage detection paradigm and directly complete leak detection and localization through end-to-end feature mapping. Its unique grid prediction mechanism can greatly improve the detection speed and meet the stringent requirements for real-time detection in industrial scenarios. However, although the YOLO series of models have many advantages in oil spill detection, they still face the following problems: (1) Oil spill datasets often contain interfering objects such as waves, ships, floating ice, and algae. The similarity between the oil film and the water texture is high, which affects the ability to identify oil. (2) There are small targets in oil spills. These small targets are often difficult to clearly identify due to low resolution and background noise. Therefore, the existing YOLO models often experience a decrease in accuracy when dealing with small targets. (3) Although the YOLO model has a certain degree of lightweight, it is still difficult to operate efficiently on resource-constrained drones, which limits the practical application of real-time detection. (4) Oil spills include different scales. However, the YOLO model has certain limitations in dealing with defective targets of different scales. Especially when detecting targets with significant size differences, the traditional IoU loss is difficult to provide sufficient adaptability. Summary of the Invention

[0004] The technical problem to be solved by the present invention is:

[0005] To solve the problems in the process of oil spill detection, such as excessive background interference in the dataset, low resolution, noise leading to low accuracy and high false negative / false positive rates; and the large number of model parameters making it impossible to be deployed on drone devices.

[0006] The technical solution adopted by the present invention to solve the above technical problems:

[0007] The present invention provides an intelligent detection method for oil spills in drone aerial photography that integrates the MobileNetV4 lightweight network, including the following steps:

[0008] S100. Construct an oil spill dataset by building a dataset from the oil spill pictures captured by drones, and then process the dataset using image enhancement methods to obtain the oil spill dataset;

[0009] S200. Construct a MobileNetv4 model, including an input layer, a shallow layer, a middle layer, two deep layers, and an output layer. Among them, the UIB module is adopted in the deep layer, introducing multi-spectral channel attention, using the dynamic upsampling module DySample for network upsampling, and using the Inner_CIoU loss function as the optimization objective of the network;

[0010] S300. Based on the oil spill dataset obtained in step S100, train the MobileNetv4 model described in step S200 to obtain the trained MobileNetv4 model;

[0011] S400. Evaluate the effectiveness of the MobileNetv4 model obtained in step S300 through evaluation metrics. When the evaluation result is effective, input the actually collected oil spill pictures into the trained MobileNetv4 model for monitoring.

[0012] Further, in step S100, the image enhancement methods include blur effect, median blur, and CLAHE; applying the blur effect to simulate the slight blur problem that occurs in the actual detection process; applying median blur to remove salt-and-pepper noise and small spots in the image; applying CLAHE to enhance the local contrast of the image.

[0013] Further, in step S200, the UIB module includes: the starting depth convolution performs spatial mixing on the input features; the channel expansion layer uses a 1×1 convolutional layer to expand the number of channels; the middle depth convolution performs spatial feature interaction on the expanded features; the projection layer uses a 1×1 convolutional layer to compress the number of channels.

[0014] Further, the calculation process of the UIB module is as follows:

[0015] Among them, is to process the input feature map T through the UIB module, Conv is convolution, is a convolution with a kernel size of 1×1, DW is depthwise separable convolution, is a depthwise separable convolution with a kernel size of 5×5, is to perform a depthwise separable convolution with a kernel size of 3×3 on the input feature map T.

[0016] Further, when adding multi-spectral channel attention, first perform discrete cosine transform on the input features to convert the spatial information to the frequency domain; in the frequency domain, extract different frequency components and re-encode them according to their importance; select a limited but important frequency component to re-compress and encode each channel.

[0017] Further, when using the dynamic upsampling module DySample for network upsampling, generate an initial offset and limit its range, and use grid sampling for upsampling; through the grid_sample function of PyTorch, resample the input feature map according to the generated sampling points, and finally obtain the upsampled feature map.

[0018] Further, when using the Inner_CIoU loss function as the optimization objective of the network, calculate the IoU loss by introducing an auxiliary bounding box, where the size of the auxiliary box is controlled by a scale factor; the size of the auxiliary box is dynamically adjusted according to different IoU samples; the specific calculation formula is as follows:

[0019] (1)

[0020] (2)

[0021] (3)

[0022] (4)

[0023] (5)

[0024] (6)

[0025] Among them, represents the Inner_CIoU loss function, r and l respectively represent the right and left boundaries of the box, represents the right boundary of the ground truth box, represents the right boundary of the predicted box, represents the left boundary of the ground truth box, represents the left boundary of the predicted box, represents the bottom boundary of the ground truth box, represents the bottom boundary of the predicted box, represents the top boundary of the ground truth box, the top boundary of the predicted box; and b respectively represent the ground truth box and the predicted box; 1 - represents the loss value, in represents the overlapping area of the ground truth box and the predicted box, un represents the total coverage area of the ground truth box and the predicted box, ratio represents the scale factor, and respectively represent the height and width of the ground truth box, h and w respectively represent the height and width of the predicted box, p represents the distance between the centers of the ground truth box and the predicted box, c represents the diagonal length of the minimum bounding rectangle between the predicted box and the ground truth box, a represents a positive trade-off parameter for balancing the weights of different loss terms, v represents a positive geometric metric term for measuring the aspect ratio consistency between the predicted box and the ground truth box; min() is to take the minimum value of two values; max() is to take the maximum value of two values.

[0026] Further, in step S400, standard evaluation metrics are used to evaluate the performance of the trained MobileNetv4 model; if the set performance threshold is reached, it is deployed in the actual scenario to intelligently detect the oil spill images collected by the drone to determine whether there is a spill; otherwise, continue to optimize the model parameters and train until the model meets the performance requirements.

[0027] An intelligent oil spill detection system for drone aerial photography integrating the MobileNetV4 lightweight network, the system has program modules corresponding to the above steps, and executes the steps in the above-mentioned intelligent oil spill detection method for drone aerial photography integrating the MobileNetV4 lightweight network when running.

[0028] A computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of an intelligent oil spill detection method for drone aerial photography integrating the MobileNetV4 lightweight network when called by a processor.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] The present invention designs an oil spill model based on Mobilenetv4. By introducing the Mobilenetv4 backbone network, multi-spectral channel attention mechanism, dynamic upsampling module and Inner_CIoU loss function, it effectively improves the accuracy of oil spill detection and reduces the model parameter quantity and computational overhead, so as to be easily deployed to the drone real-time monitoring.

[0031] The present invention optimizes the loss function (Inner_CIoU), accelerates the network convergence speed, and improves the positioning accuracy. Compared with the traditional CIoU loss function, Inner_CIoU performs more excellently when dealing with targets of different scales, effectively improving the training efficiency and the robustness of the network.

[0032] The present invention adopts a multi-level data augmentation technology, including blur effect, median blur and CLAHE, to improve the adaptability of the model under different environmental conditions. Description of the Drawings

[0033] Figure 1 This is a partial oil spill data map actually captured by the drone in the embodiment of the present invention. Among them, (a)-(f) are the oil spill data maps captured by the drone;

[0034] Figure 2 This is a flowchart of an intelligent detection method for aerial photography of oil spills by a drone integrating the MobileNetV4 lightweight network in the embodiment of the present invention;

[0035] Figure 3 For using the present invention to Figure 1 The effect diagram after recognition. Among them, (a)-(f) are the corresponding effect diagrams after recognition of (a)-(f) in Figure 1 using the present invention;

[0036] Figure 4 This is the P-R (Precision-Recall) curve diagram of Mobilenetv4 in the embodiment of the present invention. Specific embodiments

[0037] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings.

[0038] Specific implementation plan one: Combining Figures 1 to 3 As shown, the present invention provides an intelligent detection method for aerial photography of oil spills by a drone integrating the MobileNetV4 lightweight network, including the following steps:

[0039] S100. Construct a data set. Construct a data set from the oil spill pictures captured by the drone, and then use a multi-level image enhancement technology, including blur effect, median blur, and CLAHE (Contrast Limited Adaptive Histogram Equalization), to process the oil spill data to obtain an oil spill data set, including,

[0040] S110. Collect 9618 pieces of oil spill data and randomly allocate them according to the ratio of training set: validation set: test set of 8:1:1;

[0041] S120. Apply the blur effect to simulate the slight blur that may occur during actual detection, such as image defocus caused by the movement of the drone camera or environmental light changes;

[0042] S130. Apply median blur to effectively remove salt-and-pepper noise and small spots in the image, improve the image quality, and enhance the robustness of the model to noisy images;

[0043] S140. Apply CLAHE to enhance the local contrast of the image, making the oil features more obvious and improving the detection ability of the model under different brightness and contrast conditions;

[0044] S200. Build a MobileNetv4 model, including,

[0045] S210. Use MobileNetv4 to build the backbone network. MobileNetv4 includes six layers: an input layer, a shallow layer, a middle layer, two deep layers, and an output layer. The UIB (Universal Inverted Bottleneck) module used in the deep layer is the core module of the MobileNetv4 model of the present invention. The UIB module consists of four key components: the starting depth convolution performs spatial mixing on the input features to expand the receptive field; the channel expansion layer uses a 1×1 convolutional layer to expand the number of channels and improve the expression ability; the middle depth convolution performs spatial feature interaction on the expanded features to enhance the ability to capture local details; the projection layer uses a 1×1 convolutional layer to compress the number of channels, reduce the computational overhead, and retain key information; finally, when the input and output channels are the same, a residual connection is maintained to stabilize the training;

[0046] The overall calculation process of the UIB module is:

[0047] Among them, is to process the input feature map T through the UIB module, T is the input feature map, Conv is the convolution, is the convolution with a convolution kernel size of 1×1, DW is the depthwise separable convolution, is the depthwise separable convolution with a convolution kernel size of 5×5, is to perform a depthwise separable convolution with a convolution kernel size of 3×3 on the input feature map T;

[0048] S220. Add multi-spectral channel attention. First, perform DCT (Discrete Cosine Transform) on the input features to convert the spatial information to the frequency domain; in the frequency domain, different frequency components are extracted and re-encoded according to their importance; select a limited but important frequency component to re-compress and encode each channel. In this way, the compressed channels can represent the original information more comprehensively;

[0049] S230. Use the dynamic upsampling module DySample as the network upsampling. DySample generates an initial offset and limits its range, and uses grid sampling for upsampling; through the grid_sample function of PyTorch (an open-source deep learning framework), re-sample the input feature map according to the generated sampling points to finally obtain the upsampled feature map;

[0050] S240. The Inner_CIoU loss function is used as the optimization objective of the network. The Inner_CIoU loss function calculates the IoU (Intersection over Union) loss by introducing an auxiliary bounding box, where the size of the auxiliary box is controlled by a scale factor; the size of the auxiliary box will be dynamically adjusted according to different IoU samples to accelerate the convergence process of the network, thereby improving the overall network performance. The specific calculation formula is as follows:

[0051] (1)

[0052] (2)

[0053] (3)

[0054] (4)

[0055] (5)

[0056] (6)

[0057] Among them, represents the Inner_CIoU loss function, r and l respectively represent the right and left boundaries of the box, represents the right boundary of the ground truth box, represents the right boundary of the predicted box, represents the left boundary of the ground truth box, represents the left boundary of the predicted box, represents the bottom boundary of the ground truth box, represents the bottom boundary of the predicted box, represents the top boundary of the ground truth box, the top boundary of the predicted box; and b respectively represent the ground truth box and the predicted box; 1 - represents the loss value, in represents the area of the overlapping part between the ground truth box and the predicted box, un represents the total covered area of the ground truth box and the predicted box, ratio represents the scale factor, and respectively represent the height and width of the ground truth box, h and w respectively represent the height and width of the predicted box, p represents the distance between the centers of the ground truth box and the predicted box, c represents the diagonal length of the minimum bounding rectangle between the predicted box and the ground truth box, a represents a positive trade-off parameter used to balance the weights of different loss terms, v is a positive geometric metric term used to measure the aspect ratio consistency between the predicted box and the ground truth box; min() takes the minimum value of two values; max() takes the maximum value of two values; in formula (1), the width of the intersection is obtained on the left side of *, and the length of the intersection is obtained on the right side of *;

[0058] S300. Based on the oil spill dataset obtained in step S100, train and construct the MobileNetv4 model according to the following path:

[0059] S310. Set up the experimental environment. The experimental platform is the Windows11 operating system, the CPU is i5-12490f, and the graphics card is NVIDIA RTX 4060TI with 8G video memory. Environment configuration: The Python version is 3.9.21, the torch version is 2.6.0, and the cuda version is 12.6;

[0060] S320. Initialize the hyperparameters of the model, including the input image size, the batch size of the training set, the initial learning rate and the learning rate decay coefficient, the SGD optimizer parameters, and the number of training iterations;

[0061] S330. During the training process, use the validation set to evaluate the model, and adjust the hyperparameters of the model (such as the learning rate, batch size, etc.) to ensure the stability of the training process and prevent overfitting;

[0062] S400. Prove the effectiveness of the model through standard evaluation metrics such as mAP and the number of parameters; after the model training is completed, evaluate its performance using standard evaluation metrics (such as mAP and the number of model parameters); if the model reaches the set performance threshold, deploy it to the actual scenario to perform intelligent detection on the oil spill images collected by the drone to determine whether there is a spill; otherwise, continue to optimize the model parameters and train until the model meets the performance requirements;

[0063] Specific implementation plan two: An intelligent oil spill detection system for drone aerial photography integrating the MobileNetV4 lightweight network of the present invention. This system has program modules corresponding to the above steps and executes the steps in the above-mentioned intelligent oil spill detection method for drone aerial photography integrating the MobileNetV4 lightweight network when running.

[0064] The other combinations and connection relationships in this implementation plan are the same as those in the specific implementation plan one.

[0065] Specific Embodiment 3: A computer-readable storage medium of the present invention stores a computer program, and the computer program is configured to implement the steps of an intelligent detection method for drone aerial photography of oil spills integrating the MobileNetV4 lightweight network when called by a processor.

[0066] The other combinations and connection relationships of this embodiment are the same as those of Specific Embodiment 1.

[0067] Experiment

[0068] Combined Figure 1 、 Figure 3 and Figure 4 As shown, this experiment was carried out at the Institute of Artificial Intelligence and Energy of Northeast Petroleum University. Based on the drone aerial photography dataset collected in the laboratory, by introducing the MobileNetV4 lightweight backbone network, multi-spectral channel attention mechanism, DySample dynamic upsampling and Inner_CIoU loss function structure optimization strategy, while significantly compressing the model parameter quantity and computational complexity, the mAP@0.5 average precision of oil spill detection was increased to 67.4% (an absolute increase of 10.6% compared to the baseline of 56.8%).

[0069] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will all fall within the protection scope of the present invention.

Claims

1. An intelligent detection method for oil spills in UAV aerial photography that integrates the MobileNetV4 lightweight network, characterized in that, It includes the following steps: S100. Construct an oil spill dataset. Build a dataset from the oil spill pictures captured by the drone, and then use image enhancement methods to process the dataset to obtain the oil spill dataset; S200. Construct a MobileNetv4 model, including an input layer, a shallow layer, a middle layer, two deep layers, and an output layer. The UIB module is adopted in the deep layer, multi-spectral channel attention is introduced, the dynamic upsampling module DySample is used as the network upsampling, and the Inner_CIoU loss function is used as the optimization target of the network; S300. Based on the oil spill dataset obtained in step S100, train the MobileNetv4 model described in step S200 to obtain the trained MobileNetv4 model; S400. Evaluate the effectiveness of the MobileNetv4 model obtained in step S300 through evaluation metrics. When the evaluation result is valid, input the actually collected oil spill pictures into the trained MobileNetv4 model for monitoring.

2. The intelligent detection method for oil spills in UAV aerial photography integrating the MobileNetV4 lightweight network according to claim 1, characterized in that: In step S100, the image enhancement methods include blur effect, median blur, and CLAHE; apply the blur effect to simulate the slight blur problem that occurs in the actual detection process; apply median blur to remove salt-and-pepper noise and small spots in the image; apply CLAHE to enhance the local contrast of the image.

3. An intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network according to claim 1, characterized in that: In step S200, the UIB module includes: the starting depth convolution performs spatial mixing on the input features; the channel expansion layer uses a 1×1 convolutional layer to expand the number of channels; the middle depth convolution performs spatial feature interaction on the expanded features; the projection layer uses a 1×1 convolutional layer to compress the number of channels.

4. An intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network according to claim 3, characterized in that: The calculation process of the UIB module is as follows: Among them, UIB module processes the input feature map T, Conv is convolution, is convolution with a kernel size of 1×1, DW is depthwise separable convolution, is depthwise separable convolution with a kernel size of 5×5, is depthwise separable convolution with a kernel size of 3×3 on the input feature map T.

5. An intelligent detection method for oil spills in UAV aerial photography integrating the MobileNetV4 lightweight network, characterized in that: When adding multi-spectral channel attention, first perform discrete cosine transform on the input features to convert the spatial information to the frequency domain; in the frequency domain, extract different frequency components and re-encode them according to their importance; Select limited but important frequency components to re-compress and encode each channel.

6. An intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network, characterized in that: When using the dynamic upsampling module DySample as the network upsampling, generate an initial offset and limit its range, and use grid sampling for upsampling; through the grid_sample function of PyTorch, re-sample the input feature map according to the generated sampling points to finally obtain the upsampled feature map.

7. An intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network according to claim 1, characterized in that: When using the Inner_CIoU loss function as the optimization target of the network, calculate the IoU loss by introducing an auxiliary bounding box, where the size of the auxiliary box is controlled by a scale factor; the size of the auxiliary box is dynamically adjusted according to different IoU samples; the specific calculation formula is as follows: （1） （2） （3） （4） （5） （6） Among them, represents the Inner_CIoU loss function, where r and l respectively represent the right and left boundaries of the bounding box, represents the right boundary of the ground truth bounding box, represents the right boundary of the predicted bounding box, represents the left boundary of the ground truth bounding box, represents the left boundary of the predicted bounding box, represents the bottom boundary of the ground truth bounding box, represents the bottom boundary of the predicted bounding box, represents the top boundary of the ground truth bounding box, the top boundary of the predicted bounding box; and b respectively represent the ground truth bounding box and the predicted bounding box; 1 - represents the loss value, in represents the area of the overlapping part between the ground truth bounding box and the predicted bounding box, un represents the total covered area of the ground truth bounding box and the predicted bounding box, ratio represents the scaling factor, and respectively represent the height and width of the ground truth bounding box, h and w respectively represent the height and width of the predicted bounding box, p represents the distance between the center points of the ground truth bounding box and the predicted bounding box, c represents the diagonal length of the minimum bounding rectangle between the predicted bounding box and the ground truth bounding box, a represents a positive trade-off parameter used to balance the weights of different loss terms, v represents a positive geometric metric term used to measure the aspect ratio consistency between the predicted bounding box and the ground truth bounding box; min() is to take the minimum value of two values; max() is to take the maximum value of two values.

8. The intelligent oil spill detection method for drone aerial photography integrated with the MobileNetV4 lightweight network according to claim 1 is characterized by: In step S400, use standard evaluation metrics to evaluate the performance of the trained MobileNetv4 model; if the set performance threshold is reached, deploy it to the actual scenario to perform intelligent detection on the oil spill images collected by the drone to determine whether there is a leak; otherwise, continue to optimize the model parameters and train until the model meets the performance requirements.

9. An intelligent oil spill detection system for UAV aerial photography that integrates the MobileNetV4 lightweight network, characterized in that: The system has program modules corresponding to the steps described in any one of the above claims 1-8, and executes the steps in the above-mentioned intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network when running.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the intelligent detection method for oil spill in UAV aerial photography integrating MobileNetV4 lightweight network described in any one of claims 1-8 when called by a processor.

Images