An unmanned aerial vehicle aerial oil leakage intelligent detection method and system fusing a MobileNetV4 lightweight network

Abstract

The application provides a UAV aerial oil leakage intelligent detection method and system fusing a MobileNetV4 lightweight network, and belongs to the field of oil leakage detection. In order to solve the problems that in the process of oil leakage detection, the data set has too much background interference, low resolution and noise, which leads to low accuracy and high false negative / false positive rate, and the model parameter quantity is too large to be deployed on a UAV device, the application comprises the following steps: constructing an oil leakage data set and performing data preprocessing; constructing a MobileNetV4 network model; building an experimental environment, initializing grid parameters, and training the model; and evaluating the effectiveness of the model in oil leakage detection through standard evaluation indicators. The application can significantly improve the accuracy of oil leakage detection, significantly reduce the false negative rate and false positive rate, and reduce the parameter quantity and computational complexity of the model, so as to facilitate the deployment on a UAV and the efficient execution of real-time detection tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of oil leakage detection, in particular to a UAV aerial oil leakage intelligent detection method and system fusing a MobileNetV4 lightweight network. BACKGROUND

[0002] Marine oil detection technology is the core defense line for dealing with the ecological and economic dual problems of oil leakage, which is derived from the irreversible long-term harm of oil leakage: the leaked oil pollution rapidly spreads under the action of ocean currents and wind, not only directly killing marine organisms and destroying sensitive habitats such as coral reefs, leading to a cliff-like decline in biodiversity, but also polluting marine products such as shellfish and fish through the food chain amplification, ultimately threatening human health (such as long-term consumption of contaminated seafood can increase the risk of liver cancer).

[0003] With the continuous development of computer vision technology, target detection technology based on deep learning has been widely applied in offshore oil detection. There are two existing architectures for target detection: convolutional neural network (CNN) based (such as YOLO series) and Transformer based (such as DETR series). The target detector based on Transformer (DETR series) eliminates some components (anchor design in traditional target detector, non-maximum suppression (NMS)) and achieves significant performance, simplifies the target detection process, and realizes end-to-end target detection. However, DETR requires a large amount of computational overhead, making real-time detection difficult. The YOLO (You Only Look Once) series model has shown significant application value in oil spill detection due to its unique architecture design and performance advantages. Compared with two-stage target detection models such as Faster R-CNN and Mask R-CNN, the YOLO series model adopts a single-stage detection paradigm and directly completes the leak detection positioning through end-to-end feature mapping. Its unique grid prediction mechanism can greatly improve detection speed, meeting the stringent real-time detection requirements of industrial scenarios. However, despite the many advantages of the YOLO series model in oil spill detection, it still faces the following problems: (1) Oil spill datasets often contain waves, ships, ice floes, and algae, which interfere with the identification of oil films and water bodies, affecting the ability to identify oil; (2) Oil spills contain small targets, which are often difficult to identify clearly due to low resolution and background noise, so existing YOLO models often have accuracy problems when processing small targets; (3) Although the YOLO model is lightweight to some extent, it is still difficult to run efficiently on resource-limited drones, limiting the practical application of real-time detection; (4) Oil spills include different scales, but the YOLO model has limitations when processing targets of different scales, especially when detecting targets with significant size differences, the traditional IoU loss is difficult to provide sufficient adaptability. SUMMARY

[0004] The technical problem to be solved by the present application is:

[0005] To solve the problem of excessive background interference, low resolution, and noise in the oil spill detection process, resulting in low accuracy and high false positive / negative rate; and the problem of large model parameter size that cannot be deployed on a drone device.

[0006] The technical solution adopted by the present application to solve the above technical problems is:

[0007] The present application provides an unmanned aerial vehicle aerial oil spill intelligent detection method based on MobileNetV4 lightweight network, comprising the following steps:

[0008] S100, construct an oil leakage dataset, construct an oil leakage picture taken by a drone into a dataset, and then process the dataset by using a picture enhancement method to obtain an oil leakage dataset;

[0009] S200, construct a MobileNetv4 model, including an input layer, a shallow layer, a middle layer, two deep layers, and an output layer, wherein the deep layer adopts a UIB module, introduces multi-spectral channel attention, adopts a dynamic upsampling module DySample as network upsampling, and adopts an Inner_CIoU loss function as the optimization target of the network;

[0010] S300, based on the oil leakage dataset obtained in step S100, train the MobileNetv4 model in step S200 to obtain a trained MobileNetv4 model;

[0011] S400, evaluate the effectiveness of the MobileNetv4 model obtained in step S300 by using an evaluation index, and when the evaluation result is effective, input an actually collected oil leakage picture into the trained MobileNetv4 model for monitoring.

[0012] Further, in step S100, the picture enhancement method includes a blur effect, a median blur, and CLAHE; the blur effect is applied to simulate slight blur problems that occur in actual detection processes; the median blur is applied to remove pepper and salt noise and small spots in the image; and the CLAHE is applied to enhance the local contrast of the image.

[0013] Further, in step S200, the UIB module includes: a starting depth convolution that performs spatial mixing on input features; a channel expansion layer that uses a 1×1 convolution layer to expand the number of channels; an intermediate depth convolution that performs spatial feature interaction on the expanded features; and a projection layer that uses a 1×1 convolution layer to compress the number of channels.

[0014] Further, the calculation process of the UIB module is as follows:

[0015] wherein, is the processing of the input feature map T by the UIB module, Conv is convolution, is a convolution with a kernel size of 1×1, DW is a depth separable convolution, is a depth separable convolution with a kernel size of 5×5, is a depth separable convolution with a kernel size of 3×3 on the input feature map T.

[0016] Further, when adding multi-spectral channel attention, first, the input features are subjected to discrete cosine transform to convert spatial information to the frequency domain; in the frequency domain, different frequency components are extracted and re-encoded according to their importance; limited but important frequency components are selected to re-compress and encode each channel.

[0017] Further, when the dynamic upsampling module DySample is used as the network upsampling, the initial offset is generated and its range is limited, and grid sampling is used for upsampling; through the grid_sample function of PyTorch, the input feature map is resampled according to the generated sampling points, and finally the upsampled feature map is obtained.

[0018] Further, when the Inner_CIoU loss function is used as the optimization objective of the network, an auxiliary bounding box is introduced to calculate the IoU loss, where the size of the auxiliary box is controlled by a scale factor; the size of the auxiliary box is dynamically adjusted according to different IoU samples; the specific calculation formula is as follows:

[0019] (1)

[0020] (2)

[0021] (3)

[0022] (4)

[0023] (5)

[0024] (6)

[0025] wherein, represents the Inner_CIoU loss function, r and l represent the right and left boundaries of the box respectively, represents the right boundary of the real box, represents the right boundary of the predicted box, represents the left boundary of the real box, represents the left boundary of the predicted box, represents the bottom boundary of the real box, represents the bottom boundary of the predicted box, represents the top boundary of the real box, represents the top boundary of the predicted box; and b represent the real box and the predicted box respectively; 1- represents the loss value, in represents the overlapping area of the real box and the predicted box, un represents the total coverage area of the real box and the predicted box, and ratio represents the scale factor, and respectively, h and w represent the height and width of the prediction box respectively, p represents the distance between the center points of the real box and the prediction box, c represents the diagonal length of the minimum circumscribed rectangle between the prediction box and the real box, a represents a positive weighting parameter for balancing the weights of different loss terms, v represents a positive geometric metric term for measuring the consistency of the aspect ratio between the prediction box and the real box, min() is the minimum of the two values, and max() is the maximum of the two values.

[0026] Further, in step S400, the performance of the trained MobileNetv4 model is evaluated using a standard evaluation index; if a set performance threshold is reached, the model is deployed in an actual scenario to intelligently detect oil leakage images collected by a UAV to determine whether there is leakage; otherwise, the model parameters are continuously optimized and trained until the model meets the performance requirements.

[0027] A UAV aerial oil leakage intelligent detection system fusing a MobileNetV4 lightweight network, which has program modules corresponding to the above steps, and executes the steps of the above UAV aerial oil leakage intelligent detection method fusing a MobileNetV4 lightweight network when running.

[0028] A computer-readable storage medium stores a computer program configured to implement the steps of the UAV aerial oil leakage intelligent detection method fusing a MobileNetV4 lightweight network when called by a processor.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] The present application designs an oil leakage model based on Mobilenetv4, which effectively improves the accuracy of oil leakage detection and reduces the model parameter quantity and computational overhead by introducing a Mobilenetv4 backbone network, a multi-spectral channel attention mechanism, a dynamic up-sampling module, and an Inner_CIoU loss function, so as to facilitate deployment to a UAV for real-time monitoring.

[0031] The present application accelerates the convergence speed of the network and improves the positioning accuracy by optimizing the loss function (Inner_CIoU). Compared with the traditional CIoU loss function, Inner_CIoU has superior performance when processing targets of different scales, effectively improving the training efficiency and the robustness of the network.

[0032] The present application adopts multi-level data enhancement technology, including blur effect, median blur, and CLAHE, which improves the adaptability of the model under different environmental conditions. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 Part of the oil spill data graph actually photographed by the unmanned aerial vehicle in the embodiment of the application, wherein (a)-(f) are oil spill data graphs photographed by the unmanned aerial vehicle;

[0034] Figure 2 A flowchart of a kind of unmanned aerial vehicle aerial oil spill intelligent detection method fusing MobileNetV4 lightweight network in the embodiment of the application;

[0035] Figure 3 The effect graph after identification, wherein (a)-(f) are corresponding effect graphs after identification of (a)-(f) in the application by the application; Figure 1 The effect graph after identification, wherein (a)-(f) are corresponding effect graphs after identification of (a)-(f) in the application by the application; Figure 1

[0036] Figure 4 The P-R (precision-recall) curve of MobileNetV4 in the embodiment of the application. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0038] Specific implementation scheme one: as shown in the drawings, the application provides a kind of unmanned aerial vehicle aerial oil spill intelligent detection method fusing MobileNetV4 lightweight network, comprising the following steps: Figures 1 to 3 S100, construct data set, the oil spill picture photographed by unmanned aerial vehicle is constructed as data set, then a multi-level picture enhancement technique is used, including blur effect, median blur and CLAHE (contrast limited adaptive histogram equalization) to process oil spill data to obtain oil spill data set, including,

[0039] S110, 9618 oil spill data are collected and randomly distributed according to the ratio of 8:1:1 as training set: verification set: test set;

[0040] S120, apply blur effect, simulate the slight blur that may occur in the actual detection process of image, such as image defocus caused by unmanned aerial vehicle camera movement or environmental light change;

[0041] S130, apply median blur, effectively remove salt and pepper noise and small spots in the image, improve image quality, and enhance the robustness of the model to noisy images;

[0042]

[0043] ​​S140, apply CLAHE, enhance the local contrast of the image, make the oil features more obvious, and improve the detection ability of the model under different brightness and contrast conditions;

[0044] S200, construct a MobileNetv4 model, including,

[0045] S210, use MobileNetv4 to construct a backbone network, MobileNetv4 includes six layers: an input layer, a shallow layer, a middle layer, two deep layers, and an output layer, wherein the UIB (Universal Inverted Bottleneck) module adopted in the deep layer is the core module of the MobileNetv4 model of the application, the UIB module is composed of four key components: the initial depth convolution performs spatial mixing on the input features to expand the receptive field; the channel expansion layer uses a 1x1 convolution layer to expand the channel number to improve the expression ability; the intermediate depth convolution performs spatial feature interaction on the expanded features to enhance the local detail capture ability; the projection layer uses a 1x1 convolution layer to compress the channel number to reduce the calculation overhead and retain the key information; finally, when the input and output channels are the same, the residual connection is maintained to stabilize the training;

[0046] The overall calculation process of the UIB module is:

[0047] Among them, is the processing of the input feature map T through the UIB module, T is the input feature map, Conv is the convolution, is the convolution with a kernel size of 1x1, DW is the depth separable convolution, is the depth separable convolution with a kernel size of 5x5, is the depth separable convolution with a kernel size of 3x3 on the input feature map T;

[0048] S220, add multispectral channel attention, first perform DCT (Discrete Cosine Transform) on the input features, convert the spatial information to the frequency domain; in the frequency domain, different frequency components are extracted and re-encoded according to their importance; select limited but important frequency components to re-compress and encode each channel; in this way, the compressed channel can more comprehensively represent the original information;

[0049] S230, use a dynamic upsampling module DySample as the network upsampling, DySample generates an initial offset and limits its range, and uses grid sampling for upsampling; through the grid_sample function of PyTorch (an open source deep learning framework), the input feature map is resampled according to the generated sampling points, and finally the upsampled feature map is obtained.

[0050] In S240, an Inner_CIoU loss function is used as an optimization objective of the network. The Inner_CIoU loss function calculates an IoU (Intersection over Union) loss by introducing an auxiliary bounding box, wherein the size of the auxiliary box is controlled by a scale factor. The size of the auxiliary box will be dynamically adjusted according to different IoU samples, which is used to accelerate the convergence process of the network, and thus improve the overall network performance. The specific calculation formula is as follows:

[0051] (1)

[0052] (2)

[0053] (3)

[0054] (4)

[0055] (5)

[0056] (6)

[0057] wherein, represents the Inner_CIoU loss function, r and l represent the right and left boundaries of the box respectively, represents the right boundary of the real box, represents the right boundary of the predicted box, represents the left boundary of the real box, represents the left boundary of the predicted box, represents the bottom boundary of the real box, represents the bottom boundary of the predicted box, represents the top boundary of the real box, represents the top boundary of the predicted box; and b represent the real box and the predicted box respectively; 1- represents a loss value, in represents the area of the overlapping part of the real box and the predicted box, un represents the total coverage area of the real box and the predicted box, and ratio represents a scale factor, and respectively, h and w represent the height and width of the prediction box respectively, p represents the distance between the center points of the real box and the prediction box, c represents the diagonal length of the minimum circumscribed rectangle between the prediction box and the real box, a represents a positive weighting parameter for balancing the weights of different loss terms, v is a positive geometric measurement term for measuring the consistency of the aspect ratio between the prediction box and the real box; min() is the minimum of the two values; max() is the maximum of the two values; the left side of formula (1) obtains the intersection width, and the right side obtains the intersection length;

[0058] S300, based on the oil leakage data set obtained in step S100, the MobileNetv4 model constructed according to the following path is trained:

[0059] S310, build an experimental environment, the experimental platform is Windows 11 operating system, CPU i5-12490f, and the graphics card is NVIDIA RTX 4060TI with 8G video memory. Environment configuration: the Python version is 3.9.21, the torch version is 2.6.0, and the cuda version is 12.6;

[0060] S320, initialize the hyperparameters of the model, including the input picture size, the batch size of the training set, the initial learning rate and the learning rate decay coefficient, the SGD optimizer parameters, and the training iteration number;

[0061] S330, in the training process, the model is evaluated using the validation set, the hyperparameters of the model (such as learning rate, batch size, etc.) are adjusted, the stability of the training process is ensured, and overfitting is prevented;

[0062] S400, prove the effectiveness of the model by standard evaluation indicators such as mAP and parameter quantity; after the model training is completed, the performance is evaluated by standard evaluation indicators (such as mAP and model parameter quantity); if the model reaches the set performance threshold, it is deployed in the actual scene to intelligently detect the oil leakage images collected by the unmanned aerial vehicle to determine whether there is leakage; otherwise, continue to optimize the model parameters and train until the model meets the performance requirements;

[0063] Specific implementation scheme two: the present application is a kind of unmanned aerial vehicle aerial oil leakage intelligent detection system for fusing MobileNetV4 light weight network, the system has the program module corresponding to the above steps, and when running, the steps in the above one kind of unmanned aerial vehicle aerial oil leakage intelligent detection method for fusing MobileNetV4 light weight network are executed.

[0064] The other combinations and connection relationships of the present embodiment are the same as those of the first specific embodiment.

[0065] Specific embodiment three: a computer readable storage medium of the application, the computer readable storage medium stores a computer program, the computer program is configured to realize the steps of a kind of unmanned aerial vehicle aerial oil spill intelligent detection method of fusion MobileNetV4 light network when being called by processor.

[0066] Other combinations and connection relationships of the present embodiment are the same as specific embodiment one.

[0067] Experiment

[0068] Combination Figure 1 、 Figure 3 And Figure 4 As shown in the experiment, the experiment was carried out in the artificial intelligence energy research institute of northeast petroleum university, based on the laboratory collected unmanned aerial vehicle aerial data set, by introducing MobileNetV4 light main network, multispectral channel attention mechanism, DySample dynamic up-sampling and Inner_CIoU loss function structure optimization strategy, while significantly compressing model parameter quantity and computational complexity, the mAP@0.5 average precision of oil spill detection is improved to 67.4% (absolute improvement of 10.6% compared with baseline 56.8%).

[0069] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. The person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A UAV aerial oil spill intelligent detection method based on a fusion MobileNetV4 lightweight network, characterized in that, The method comprises the following steps: S100, constructing an oil leakage data set, constructing oil leakage pictures taken by a drone into a data set, and then processing the data set by using a picture enhancement method to obtain an oil leakage data set; The picture enhancement method comprises a blur effect, a median blur, and CLAHE; the blur effect is applied to simulate the blur problem of the image in the actual detection process; the median blur is applied to remove the salt and pepper noise and small spots in the image; and the CLAHE is applied to enhance the local contrast of the image; S200, constructing a MobileNetv4 model, comprising an input layer, a shallow layer, a middle layer, two deep layers, and an output layer, wherein the deep layer adopts a UIB module, introduces multi-spectral channel attention, adopts a dynamic upsampling module DySample as network upsampling, and adopts an Inner_CIoU loss function as the optimization target of the network; The UIB module comprises: a starting deep convolution that performs spatial mixing on the input features; a channel expansion layer that expands the channel number by using a 1x1 convolution layer; and an intermediate deep convolution that performs spatial feature interaction on the expanded features; and a projection layer that compresses the channel number by using a 1x1 convolution layer; The computing process of the UIB module is: wherein, is a processing of the input feature map T by the UIB module, Conv is a convolution, is a convolution with a kernel size of 1x1, DW is a depthwise separable convolution, is a depthwise separable convolution with a kernel size of 5x5, is a depthwise separable convolution with a kernel size of 3x3 on the input feature map T; When the multi-spectral channel attention is introduced, first, the input features are subjected to discrete cosine transformation to convert the spatial information to the frequency domain; in the frequency domain, different frequency components are extracted and re-encoded according to their importance; and limited but important frequency components are selected to re-compress and encode each channel; When the dynamic upsampling module DySample is adopted as the network upsampling, an initial offset is generated and its range is limited, and grid sampling is used for upsampling; by using the grid_sample function of PyTorch, the input feature map is resampled according to the generated sampling points, and finally the upsampled feature map is obtained; S300, training the MobileNetv4 model of step S200 based on the oil leakage data set obtained in step S100 to obtain a trained MobileNetv4 model; initializing the hyperparameters of the model, including the input picture size, the batch size of the training set, the initial learning rate and the learning rate decay coefficient, the SGD optimizer parameters, and the training iteration number; in the training process, the model is evaluated by using the validation set, and the hyperparameters of the model are adjusted; S400, evaluating the effectiveness of the MobileNetv4 model obtained in step S300 by using an evaluation index, and when the evaluation result is effective, inputting the actually collected oil leakage pictures into the trained MobileNetv4 model for monitoring.

2. The unmanned aerial vehicle aerial oil spill intelligent detection method of fusing MobileNetV4 lightweight network according to claim 1, characterized in that: When the Inner_CIoU loss function is adopted as the optimization target of the network, the IoU loss is calculated by introducing an auxiliary bounding box, wherein the size of the auxiliary box is controlled by a scale factor; the size of the auxiliary box is dynamically adjusted according to different IoU samples; and the specific calculation formula is as follows: （1） （2） （3） （4） （5） （6） wherein, represents the Inner_CIoU loss function, r and l represent the right and left boundaries of the frame, respectively, represents the right boundary of the real frame, represents the right boundary of the predicted frame, represents the left boundary of the real frame, represents the left boundary of the predicted frame, represents the bottom boundary of the real frame, represents the bottom boundary of the predicted frame, represents the top boundary of the real frame, the top boundary of the predicted frame; and b represent the real frame and the predicted frame, respectively; 1- represents the loss value, in represents the area of the overlapping part of the real frame and the predicted frame, un represents the total coverage area of the real frame and the predicted frame, and ratio represents a proportion factor, and represent the height and width of the real frame, respectively, h and w represent the height and width of the predicted frame, respectively, p represents the distance between the center points of the real frame and the predicted frame, c represents the diagonal length of the minimum circumscribed rectangle between the predicted frame and the real frame, a represents a positive weighting parameter for balancing the weights of different loss terms, v represents a positive geometric metric term for measuring the consistency of the length-width ratio between the predicted frame and the real frame; min() is the minimum of two values; max() is the maximum of two values.

3. The unmanned aerial vehicle aerial oil spill intelligent detection method of claim 1, wherein: In step S400, the performance of the trained MobileNetv4 model is evaluated using standard evaluation indicators; if the performance threshold is reached, the model is deployed in the actual scene to intelligently detect the oil leakage images collected by the unmanned aerial vehicle to determine whether there is leakage; otherwise, the model parameters are continuously optimized and trained until the model meets the performance requirements.

4. An unmanned aerial vehicle aerial oil spill intelligent detection system based on a fusion MobileNetV4 lightweight network. The system has program modules corresponding to the steps of any one of claims 1-3, and when running, the steps of the above-mentioned unmanned aerial vehicle aerial oil leakage intelligent detection method of fusing MobileNetV4 lightweight network are executed.

5. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program configured to be called by the processor to implement the steps of the unmanned aerial vehicle aerial oil leakage intelligent detection method of fusing MobileNetV4 lightweight network of any one of claims 1-3.

Images