Marine oil spill detection method based on image segmentation, unmanned aerial vehicle, storage medium

Abstract

The embodiment of the application discloses a kind of offshore oil spill detection method based on image segmentation, unmanned plane, storage medium, wherein, method includes: determining inspection area, inspection area includes multiple inspection nodes;Determine initial inspection path according to multiple inspection nodes, initial inspection path passes each inspection node once;With initial inspection path as the initialization population of artificial bee colony algorithm, determine the target path of optimal energy consumption by artificial bee colony algorithm;According to target path, inspection is carried out, and the image to be detected is photographed at each inspection node by SAR image imager;The image to be detected is input to pre-trained oil spill dark spot segmentation model and is subjected to image segmentation processing, to obtain segmented image;Determine offshore oil spill detection result of inspection node according to segmented image.The embodiment of the application can be combined with the unmanned plane provided with SAR image imager and oil spill dark spot segmentation model to carry out offshore oil spill detection on inspection area, to effectively improve the efficiency of offshore oil spill detection.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of image processing technology, and in particular to a marine oil spill detection method based on image segmentation, an unmanned aerial vehicle (UAV), and a storage medium. Background Technology

[0002] Oil spills at sea pose a serious threat to marine ecological environment safety and the development of marine industries. To mitigate the impact of oil spills, it is necessary to strengthen their detection. Currently, marine oil spill detection is mainly carried out using Synthetic Aperture Radar (SAR). However, SAR image generation takes a long time, which is not conducive to timely early warning. Furthermore, creating labeled SAR image datasets is difficult and requires a lot of manpower and resources, resulting in low efficiency in marine oil spill detection. Summary of the Invention

[0003] This application provides an image segmentation-based method for detecting marine oil spills, an unmanned aerial vehicle (UAV), and a storage medium, which can effectively improve the efficiency of marine oil spill detection.

[0004] In a first aspect, embodiments of this application provide a marine oil spill detection method based on image segmentation, applied to an unmanned aerial vehicle (UAV), wherein the UAV is equipped with a SAR imager, and the method includes:

[0005] Determine the inspection area, which includes multiple inspection nodes;

[0006] An initial inspection path is determined based on the multiple inspection nodes, and the initial inspection path passes through each of the inspection nodes once.

[0007] The initial inspection path is used as the initial population for the artificial bee colony algorithm, and the target path with optimal energy consumption is determined by the artificial bee colony algorithm.

[0008] The inspection is carried out according to the target path, and the SAR image imager is used to capture images to be detected at each of the inspection nodes.

[0009] The image to be detected is input into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image;

[0010] The marine oil spill detection results of the inspection node are determined based on the segmented image.

[0011] In some embodiments, the number of initial inspection paths is multiple; determining the energy-optimal target path using the artificial bee colony algorithm includes:

[0012] Determine the initial path energy consumption for each of the initial inspection paths;

[0013] Obtain a preset fitness function, and select multiple nodes to be adjusted from the inspection nodes according to the fitness function and the initial path energy consumption. Each node to be adjusted has a corresponding fitness value.

[0014] Clustering is performed on the nodes to be adjusted corresponding to each initial inspection path according to the preset clustering algorithm and the fitness value to obtain the first intermediate inspection path;

[0015] The first intermediate inspection path is iterated, and after completing a preset number of iterations, a second intermediate inspection path is obtained.

[0016] The target path is determined from the second intermediate inspection path, wherein the target path represents the second intermediate inspection path that is different from the first intermediate inspection path before the iteration operation.

[0017] In some embodiments, the oil spill dark spot segmentation model includes a feature extractor and a transformer encoder; the step of inputting the image to be detected into the pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image includes:

[0018] The image to be detected is input into the feature extractor for feature extraction, resulting in multiple first feature maps;

[0019] Perform linear projection processing on each of the first feature maps to obtain the first feature sequence;

[0020] Each of the first feature sequences is input into the transformer encoder for position encoding to obtain the second feature sequence;

[0021] The second feature map is obtained by linearly projecting each of the second feature sequences.

[0022] Each of the second feature maps is upsampled to obtain a third feature map, and the size of each third feature map is the same as the size of the image to be detected.

[0023] The segmented image is obtained by fusing all the third feature maps.

[0024] In some embodiments, determining the marine oil spill detection result of the inspection node based on the segmented image includes:

[0025] When the segmented image represents the presence of dark oil spill spots in the image to be detected, the marine oil spill detection result is determined to be a failure.

[0026] or,

[0027] When the segmented image represents the absence of oil spill dark spots in the image to be detected, the marine oil spill detection result is determined to be a successful detection.

[0028] In some embodiments, the drone communicates with a client, and the method further includes:

[0029] When the segmented image represents an oil spill dark spot image in the image to be detected, the target location information corresponding to the segmented image is determined;

[0030] The target location information and the marine oil spill detection results are sent to the client so that the client can display the target location information and the marine oil spill detection results on the interface of a preset application.

[0031] In some embodiments, the oil spill dark spot segmentation model is trained using an exponential logarithmic loss function.

[0032] In some embodiments, before inputting the image to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image, the method further includes:

[0033] The image to be detected is preprocessed according to preset preprocessing rules.

[0034] Secondly, embodiments of this application provide a drone, including:

[0035] An inspection area determination module is used to determine an inspection area, which includes multiple inspection nodes.

[0036] The inspection module is used to determine an initial inspection path based on multiple inspection nodes, wherein the initial inspection path passes through each inspection node once.

[0037] The path planning module is used to use the initial inspection path as the initial population of the artificial bee colony algorithm, and to determine the energy-optimal target path through the artificial bee colony algorithm.

[0038] An image acquisition module is used to perform inspections according to the target path and to capture images to be detected at each inspection node using a SAR image imager installed on the UAV.

[0039] The detection module is used to input the image to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image;

[0040] A marine oil spill detection result determination module is used to determine the marine oil spill detection result of the inspection node based on the segmented image.

[0041] Thirdly, embodiments of this application provide an unmanned aerial vehicle (UAV), including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image segmentation-based marine oil spill detection method as described in the first aspect.

[0042] Thirdly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for performing the image segmentation-based marine oil spill detection method as described in the first aspect.

[0043] This application provides a method, a drone, and a storage medium for marine oil spill detection based on image segmentation. The method includes: determining an inspection area, which includes multiple inspection nodes; determining an initial inspection path based on the multiple inspection nodes, wherein the initial inspection path passes through each inspection node once; using the initial inspection path as the initial population for an artificial bee colony algorithm, and determining an energy-optimal target path through the artificial bee colony algorithm; conducting inspections according to the target path, and capturing images to be detected at each inspection node using a SAR image imager; inputting the images to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain segmented images; and determining the marine oil spill detection result of the inspection node based on the segmented images. According to the solution provided by this application, a drone equipped with a SAR image imager and an oil spill dark spot segmentation model can be used to detect marine oil spills in an inspection area, thereby effectively improving the efficiency of marine oil spill detection. Attached Figure Description

[0044] Figure 1 This is a flowchart of the steps of a marine oil spill detection method based on image segmentation provided in one embodiment of this application;

[0045] Figure 2 This is a flowchart of the steps for determining a target path provided in another embodiment of this application;

[0046] Figure 3 This is a flowchart of the steps for obtaining a segmented image provided in another embodiment of this application;

[0047] Figure 4 This is a flowchart of the steps for determining the results of marine oil spill detection provided in another embodiment of this application;

[0048] Figure 5 This is a flowchart illustrating the steps for detecting marine oil spills, provided in another embodiment of this application.

[0049] Figure 6 This is a flowchart of the steps for image preprocessing of the image to be detected provided in another embodiment of this application;

[0050] Figure 7 This is a schematic diagram of a drone module provided in another embodiment of this application;

[0051] Figure 8 This is a schematic diagram of the model structure of an oil spill dark spot segmentation model provided in another embodiment of this application;

[0052] Figure 9 This is a structural diagram of a transformer encoder provided in another embodiment of this application;

[0053] Figure 10 This is a structural diagram of a drone provided in another embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0056] This application provides a method, a drone, and a storage medium for marine oil spill detection based on image segmentation. The method includes: determining an inspection area, which includes multiple inspection nodes; determining an initial inspection path based on the multiple inspection nodes, wherein the initial inspection path passes through each inspection node once; using the initial inspection path as the initial population for an artificial bee colony algorithm, and determining an energy-optimal target path through the artificial bee colony algorithm; conducting inspections according to the target path, and capturing images to be detected at each inspection node using a SAR image imager; inputting the images to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain segmented images; and determining the marine oil spill detection result of the inspection node based on the segmented images. According to the solution provided by this application, a drone equipped with a SAR image imager and an oil spill dark spot segmentation model can be used to detect marine oil spills in an inspection area, thereby effectively improving the efficiency of marine oil spill detection.

[0057] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0058] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of an image segmentation-based marine oil spill detection method according to an embodiment of this application. This application provides an image segmentation-based marine oil spill detection method, which includes, but is not limited to, the following steps:

[0059] Step S110: Determine the inspection area, which includes multiple inspection nodes;

[0060] Step S120: Determine the initial inspection path based on multiple inspection nodes. The initial inspection path passes through each inspection node once.

[0061] Step S130: Use the initial inspection path as the initial population for the artificial bee colony algorithm, and determine the target path with the optimal energy consumption through the artificial bee colony algorithm.

[0062] Step S140: Conduct inspections according to the target path and capture images to be detected at each inspection node using a SAR image imager.

[0063] Step S150: Input the image to be detected into the pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image;

[0064] Step S160: Determine the marine oil spill detection results of the inspection node based on the segmented image.

[0065] It should be noted that the drone in this embodiment can be of any type, as long as its endurance is sufficient to complete the inspection. Furthermore, to acquire the images to be inspected, a SAR imager can be mounted on the drone. The inspection area can be set according to actual needs, including a starting point and multiple inspection nodes. During the inspection process, the drone starts from the starting point, passes through all nodes within the area, and finally returns to the starting point, using the SAR imager to collect images at each node.

[0066] It's important to note that the Artificial Bee Colony Algorithm (APA) is a biomimetic swarm segmentation algorithm that draws inspiration from the behavior of bee colonies in nature. It finds the optimal solution by simulating how bees search for nectar, making it suitable for inspection tasks in complex environments. This is because it doesn't require specific information about the problem; only a comparison of the best and worst solutions is needed. The APA starts by randomly initializing the population and uses neighborhood search and clustering updates to generate inspection routes that are more adapted to the environment. This process continues until the optimal path converges, yielding the optimal solution to the problem. Using the target path obtained through the APA can effectively reduce energy consumption during UAV inspections.

[0067] It is understandable that after the UAV determines the target path, it can conduct multiple inspections within the inspection area according to the target path, and use the SAR imager on the UAV to capture images to be detected. By inputting the images to be detected into the UAV's pre-trained oil spill dark spot segmentation model for image segmentation processing, segmented images are obtained. The marine oil spill detection results of the inspection node are determined based on the segmented images. This application embodiment combines a UAV equipped with a SAR imager and an oil spill dark spot segmentation model to conduct marine oil spill detection in the inspection area. The UAV is small in size, highly flexible, and easy to carry, thereby solving the problem of untimely early warning from satellite SAR images. Its efficiency is particularly prominent when a large number of UAVs are used. It does not require additional maintenance personnel, saving manpower, material resources, and costs. Moreover, through the high-speed flight of the UAV and the pre-trained oil spill dark spot segmentation model, it can quickly and accurately identify oil spill targets, ensuring the safety of nearshore ecology and nearshore fire fighting. Compared with the scheme of marine oil spill detection through SAR, the technical solution of this application can more effectively improve the efficiency and accuracy of marine oil spill detection.

[0068] Additionally, refer to Figure 2 In one embodiment, the number of initial inspection paths is multiple. Figure 1 Step S130 in the illustrated embodiment also includes, but is not limited to, the following steps:

[0069] Step S210: Determine the initial path energy consumption for each initial inspection path;

[0070] Step S220: Obtain the preset fitness function, and select multiple nodes to be adjusted from the inspection nodes according to the fitness function and the initial path energy consumption. Each node to be adjusted has a corresponding fitness value.

[0071] Step S230: Cluster the nodes to be adjusted corresponding to each initial inspection path according to the preset clustering algorithm and fitness value to obtain the first intermediate inspection path.

[0072] Step S240: Iterate the first intermediate inspection path and obtain the second intermediate inspection path after completing the preset number of iterations;

[0073] Step S250: Determine the target path from the second intermediate inspection path. The target path represents the second intermediate inspection path that is different from the first intermediate inspection path before the iteration operation.

[0074] It should be noted that the initial path energy consumption is obtained using the following formula:

[0075]

[0076] Where P1 represents the distance cost of the drone; P2 represents the turning cost of the drone; |θ N | represents the absolute value of the turning angle of the UAV at the Nth inspection node, cost1 represents the energy consumption generated by the UAV's forward movement, cost2 represents the energy consumption generated by the UAV's turning lock, and w1 and w2 are the energy consumption coefficients of distance cost and turning cost, respectively, where w1 = 0.1072; w2 = 0.0104; (a n b n ) and (a n+1 b n+1 () represents the coordinates of the inspection node.

[0077] It should be noted that the formula for the fitness function is as follows:

[0078]

[0079] Among them, f i For the solution of the fitness function corresponding to the i-th population, f i The calculation formula is as follows:

[0080]

[0081] The inspection nodes of the initial inspection path are represented as follows:

[0082] x ij =x min +r (0，1) (x max -x min ));

[0083] Where, x ij Let x represent the j-th dimension of the i-th inspection node, i∈{1,2,...,N}, j∈{1,2,...,D}, and x min x represents the minimum allowed value for a node. max This indicates the maximum allowed value for a node.

[0084] It should be noted that the formula for selecting multiple nodes to be adjusted from the inspection nodes is as follows:

[0085] v ij =x ij +r (-1，1) (x ij -x kj );

[0086] in, v ij For the new node to be adjusted, v ij The selection is based on the following formula:

[0087]

[0088] Where, p i This represents the probability of selecting the i-th inspection node.

[0089] It should be noted that the formula for the clustering algorithm is as follows:

[0090]

[0091] Where d(x) i C j ) represents the cluster center C j With element x in class i The distance; J represents the sum of various interior distances; C j For the j-th population; x i Represents the i-th inspection node, each x i Both are D-dimensional vectors; where C j The formula is as follows:

[0092]

[0093] Understandably, the specific process of planning a drone inspection path can be as follows: Obstacles within the target area are acquired through remote sensing and a simulated map is constructed; the artificial bee colony algorithm is initialized, and the maximum flight speed, minimum flight speed, maximum turning speed, minimum turning speed, and fitness function weights of the drone are determined; several initial inspection paths are randomly generated, and the initial path energy consumption of each initial inspection path is determined; the fitness value of each inspection node in each initial inspection path is determined through a preset fitness function; multiple nodes to be adjusted are selected from the inspection nodes; multiple first intermediate inspection paths are obtained by combining a clustering algorithm; the first intermediate inspection paths are used to update the optimal inspection path; the first intermediate inspection paths are iterated, and after completing a preset number of iterations, second intermediate inspection paths are obtained; each second intermediate inspection path obtained after iteration corresponds to a first intermediate inspection path before its iteration; when the second intermediate inspection path is different from its corresponding first intermediate inspection path, the second intermediate inspection path is selected as the target path.

[0094] Additionally, refer to Figure 3 In one embodiment, the oil spill dark spot segmentation model includes a feature extractor and a transformer encoder. Figure 1 Step S150 in the illustrated embodiment also includes, but is not limited to, the following steps:

[0095] Step S310: Input the image to be detected into the feature extractor for feature extraction to obtain multiple first feature maps;

[0096] Step S320: Perform linear projection processing on each first feature map to obtain the first feature sequence;

[0097] Step S330: Input each of the first feature sequences into the transformer encoder for position encoding to obtain the second feature sequence;

[0098] Step S340: Perform linear projection processing on each second feature sequence to obtain the second feature map;

[0099] Step S350: Upsample each second feature map to obtain a third feature map. The size of each third feature map is the same as the size of the image to be detected.

[0100] Step S360: Perform fusion processing on all the third feature maps to obtain the segmented image.

[0101] It should be noted that the oil spill dark spot segmentation model can be a Transformers-based multi-UVit model. The UAV's pre-set oil spill dark spot segmentation model can exist in two stages: (1) Offline learning stage: First, download SAR images from public datasets such as MTA for pre-training of the Transformers-based multi-UVit model; then collect SAR images of nearshore oil spill dark spots through web crawling. Then, perform data filtering to select images of nearshore oil spill dark spots in reasonable real-world scenarios; finally, manually label all oil spill dark spot targets and apply them to the Transformers model for offline training; (2) Online monitoring stage: Input the images to be detected obtained by the UAV into the Transformers-based multi-UVit model. By modeling on the original image, the multi-UVit model will return a segmented image, and then determine the marine oil spill detection result based on the segmented image.

[0102] Understandably, the multi-UVit model is a network model designed for training with small sample sizes while maintaining high accuracy. It can effectively generalize to training sets with limited samples and ensure good generalization ability. Transformers for sequence prediction have become an alternative architecture with an inherent global self-attention mechanism, but their localization ability is limited due to insufficient detail in low-level semantic information. Multi-UVit combines the advantages of both Transformers and U-Net, and can utilize the feature information of each layer's feature map for detection and recognition. On one hand, the feature map of Convolutional Neural Networks (CNN) is converted into a feature sequence that retains positional information through positional encoding and input into Transformers. On the other hand, the feature vectors output by each Transformer are upsampled and fused to generate a mask, which is then combined with the high-resolution CNN feature map to achieve accurate localization. Therefore, the combination of Transformers and multi-UVit technology is considered as the network model for marine oil spill detection tasks.

[0103] It should be noted that the embodiments of this application do not limit the specific structure of the oil spill dark spot segmentation model, and can be as follows: Figure 8 As shown, the specific image segmentation process based on this oil spill dark spot segmentation model is as follows: The input image to be detected is first used as a feature extractor to generate three CNN feature maps. These three CNN feature maps are then processed through pattern bedding and linear projection to generate feature sequences. Each feature sequence is then input into its respective Transformer encoder. Finally, the output sequence of each Transformer encoder is reconstructed into a two-dimensional feature map, and then multiple upsampling steps are used to decode the hidden features. Finally, the three segmented images restored to the size of the input image are fused, and the final segmented image containing the segmentation mask is output.

[0104] Understandably, in the oil spill dark spot segmentation model, after the image to be detected is input to the feature extractor for feature extraction and linear projection to obtain multiple first feature sequences, the spatial distribution information of the pixels will be lost. Therefore, it is necessary to perform positional encoding on the first feature sequences. Consider adding specific positional embeddings to the feature sequences to preserve positional information. The specific implementation formula is as follows:

[0105]

[0106] Where E is the representation vector for linearly projecting and embedding the input image patch, and Epos is the representation vector that preserves the spatial location information between the input image patches.

[0107] It should be noted that the embodiments in this application do not limit the specific structure of the Transformers encoder, and can be as follows: Figure 9 As shown, the Transformers encoder consists of L layers of multi-headed self-attention (MSA) and a multilayer perception (MLP); the output of the Lth layer can be expressed as follows:

[0108] z′ L =MSA(LN(z) L-1 ))+Z L-1 ;

[0109] z L =MLP(LN(z′) L ))+z′ L ;

[0110] Where LN refers to layer normalization, and Z... L It is an encoded image representation.

[0111] Understandably, since the resolution of SAR images used to train models is usually much smaller than that of SAR images that actually need to be analyzed, it inevitably leads to the loss of low-level details, such as affecting the size and boundaries of oil spill spots. However, the Transformers-based multi-UVit model, which cascades an upsampler, can achieve precise localization and segmentation to compensate for this loss of information. This improves the accuracy of the oil spill spot segmentation model in obtaining marine oil spill detection results, enabling more accurate detection of nearshore oil spills and improving the accuracy of disaster prevention.

[0112] Additionally, in one embodiment, reference Figure 4 , Figure 1 Step S160 in the illustrated embodiment also includes, but is not limited to, the following steps:

[0113] Step S410: When the segmented image represents the presence of dark oil spill spots in the image to be detected, the marine oil spill detection result is determined to be a failure.

[0114] or,

[0115] Step S420: When the segmented image represents the absence of oil spill dark spot images in the image to be detected, the marine oil spill detection result is determined to be a successful detection.

[0116] Understandably, by analyzing the segmented image obtained through the oil spill dark spot segmentation model, it is possible to determine whether there is an oil spill dark spot image in the image to be detected, and thus determine whether there is an oil spill in the current sea area. If the segmented image indicates that there is an oil spill dark spot image in the image to be detected, then there is an oil spill in the current sea area, and timely investigation and processing are required. If the segmented image indicates that there is no oil spill dark spot image in the image to be detected, then there is no oil spill in the current sea area, thereby effectively improving the efficiency of marine oil spill detection.

[0117] In another embodiment, the drone communicates with the client, referring to... Figure 5 The marine oil spill detection method based on image segmentation provided in this application also includes, but is not limited to, the following steps:

[0118] Step S510: If the segmented image represents an oil spill dark spot image in the image to be detected, determine the target location information corresponding to the segmented image;

[0119] Step S520: Send the target location information and marine oil spill detection results to the client so that the client can display the target location information and marine oil spill detection results on the interface of the preset application.

[0120] Understandably, when the segmented image represents the presence of dark oil spill spots in the image to be detected, indicating an oil spill in the current sea area, the target location information corresponding to the segmented image (the location of the current oil spill and the location of the oil spill) is determined. The target location information and the marine oil spill detection results are then sent to the client so that relevant management personnel can obtain the location of the oil spill based on the information displayed on the client, in order to formulate an oil spill handling plan.

[0121] Additionally, refer to Figure 6 In one embodiment, during execution Figure 1 Before step S150 in the illustrated embodiment, the marine oil spill detection method based on image segmentation provided in this application embodiment also includes, but is not limited to, the following steps:

[0122] Step S610: Perform image preprocessing on the image to be detected according to preset preprocessing rules.

[0123] Understandably, the main purpose of image preprocessing for images to be detected is to eliminate irrelevant information, restore useful real information, enhance the detectability of relevant information, and simplify data to the maximum extent possible, thereby improving the reliability of subsequent applications of the images to be detected.

[0124] It should be noted that the embodiments of this application do not limit the specific method of image preprocessing for the image to be detected. It can be image random number rotation processing, normalization processing, or image binarization processing, etc. Those skilled in the art can select the specific method of image preprocessing according to the actual situation.

[0125] It should be noted that, in order to improve the accuracy of the oil spill spot segmentation model in obtaining marine oil spill detection results, this embodiment of the application trains the oil spill spot segmentation model using an exponential logarithmic loss function. The formula for the exponential logarithmic loss function is as follows:

[0126] L Exp =α1L CE (x)+α2L Dice (X, Y);

[0127] Among them, L CE The expression is as follows:

[0128]

[0129] Where x is the position of a specific pixel, i is the correct label of the corresponding specific pixel x, and p i (x) represents the probability value of a specific pixel after softmax calculation; where w i The specific expression is as follows:

[0130]

[0131] Among them, f k L represents the frequency of occurrence of the corresponding label k; Dice The specific expression is as follows:

[0132]

[0133] Where |X∩Y| represents the common elements between sets X and Y, |X| represents an element in X, and |Y| represents an element in Y. Here, the Sorenson-dice coefficient of the predicted segmentation map is calculated, approximating |X∩Y| as the dot product (i.e., z) between the predicted map and the label map. ij =x ij *y ij The elements in the resulting function are summed to obtain the result of |X∩Y|; the quantization calculation of |X| and |Y| is performed by summing the squares, that is, each term in |X| and |Y| is summed by squaring to obtain the output of |X| and |Y|; where L is the result of summing the squares of both weights α1 and α2. Exp The loss function is represented as L final L final The specific expression is:

[0134]

[0135] In addition, the oil spill dark spot segmentation model uses L to process the final segmentation map output by each Transformer encoder. Exp The loss function is used to calculate the loss, and its specific expression is:

[0136]

[0137] Where i represents the loss of the final output of the i-th Transformers encoder; therefore, the loss function expression for the oil spill dark spot segmentation model is:

[0138]

[0139] Additionally, refer to Figure 7 , Figure 7 This is a schematic diagram of a drone module provided in another embodiment of this application. In one embodiment, this application provides a drone 700, which includes:

[0140] Inspection area determination module 710 is used to determine the inspection area, which includes multiple inspection nodes;

[0141] Inspection module 720 is used to determine the initial inspection path based on multiple inspection nodes. The initial inspection path passes through each inspection node once.

[0142] The path planning module 730 is used to use the initial inspection path as the initial population of the artificial bee colony algorithm, and to determine the target path with the optimal energy consumption through the artificial bee colony algorithm.

[0143] Image acquisition module 740 is used to perform inspections according to the target path and to capture images to be inspected at each inspection node by using a SAR image imager set on the UAV.

[0144] The detection module 750 is used to input the image to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image.

[0145] The marine oil spill detection result determination module 760 is used to determine the marine oil spill detection result of the inspection node based on the segmented image.

[0146] Additionally, refer to Figure 10 , Figure 10This is a structural diagram of a drone provided in another embodiment of this application. Another embodiment of this application also provides a drone 1000, which includes: a memory 1010, a processor 1020, and a computer program stored in the memory 1010 and executable on the processor 1020.

[0147] The processor 1020 and the memory 1010 can be connected via a bus or other means.

[0148] The non-transient software program and instructions required to implement the image segmentation-based marine oil spill detection method of the above embodiments are stored in the memory 1010. When executed by the processor 1020, the image segmentation-based marine oil spill detection method applied to the UAV 1000 in the above embodiments is executed, for example, the method described above is executed. Figure 1 Method steps S110 to S160, Figure 2 Method steps S210 to S250 Figure 3 Method steps S310 to S360, Figure 4 Method steps S410 to S420 Figure 5 Method steps S510 to S520 Figure 6 Method step S610.

[0149] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, one embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described UAV 1000 embodiment. These instructions enable the processor to execute the image segmentation-based marine oil spill detection method applied to a UAV in the above-described embodiment, for example, to execute the methods described above. Figure 1 Method steps S110 to S160, Figure 2 Method steps S210 to S250 Figure 3 Method steps S310 to S360, Figure 4 Method steps S410 to S420 Figure 5 Method steps S510 to S520 Figure 6Method step S610. Those skilled in the art will understand that all or some of the steps in the methods and systems disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A marine oil spill detection method based on image segmentation, applied to an unmanned aerial vehicle (UAV), wherein the UAV is equipped with a SAR image imager, characterized in that, The method includes: Determine the inspection area, which includes multiple inspection nodes; An initial inspection path is determined based on the multiple inspection nodes, and the initial inspection path passes through each of the inspection nodes once. The initial inspection path is used as the initial population for the artificial bee colony algorithm, and the target path with optimal energy consumption is determined by the artificial bee colony algorithm. The inspection is carried out according to the target path, and the SAR image imager is used to capture the image to be detected at each of the inspection nodes; The image to be detected is input into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image; The marine oil spill detection results of the inspection node are determined based on the segmented image; The oil spill dark spot segmentation model includes a feature extractor and a transformer encoder; the step of inputting the image to be detected into the pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image includes: The image to be detected is input into the feature extractor for feature extraction, resulting in multiple first feature maps; Perform linear projection processing on each of the first feature maps to obtain the first feature sequence; Each of the first feature sequences is input into the transformer encoder for position encoding to obtain the second feature sequence; The second feature map is obtained by linearly projecting each of the second feature sequences. Each of the second feature maps is upsampled to obtain a third feature map, and the size of each third feature map is the same as the size of the image to be detected. The segmented image is obtained by fusing all the third feature maps.

2. The marine oil spill detection method based on image segmentation according to claim 1, characterized in that, The number of initial inspection paths is multiple; the determination of the energy-optimal target path using the artificial bee colony algorithm includes: Determine the initial path energy consumption for each of the initial inspection paths; Obtain a preset fitness function, and select multiple nodes to be adjusted from the inspection nodes according to the fitness function and the initial path energy consumption. Each node to be adjusted has a corresponding fitness value. Clustering is performed on the nodes to be adjusted corresponding to each initial inspection path according to the preset clustering algorithm and the fitness value to obtain the first intermediate inspection path; The first intermediate inspection path is iterated, and after completing a preset number of iterations, a second intermediate inspection path is obtained. The target path is determined from the second intermediate inspection path, wherein the target path represents the second intermediate inspection path that is different from the first intermediate inspection path before the iteration operation.

3. The marine oil spill detection method based on image segmentation according to claim 1, characterized in that, The step of determining the marine oil spill detection result of the inspection node based on the segmented image includes: When the segmented image represents the presence of dark oil spill spots in the image to be detected, the marine oil spill detection result is determined to be a failure. or, When the segmented image represents the absence of oil spill dark spots in the image to be detected, the marine oil spill detection result is determined to be a successful detection.

4. The marine oil spill detection method based on image segmentation according to claim 3, characterized in that, The drone communicates with the client, and the method further includes: When the segmented image represents an oil spill dark spot image in the image to be detected, the target location information corresponding to the segmented image is determined; The target location information and the marine oil spill detection results are sent to the client so that the client can display the target location information and the marine oil spill detection results on the interface of a preset application.

5. The marine oil spill detection method based on image segmentation according to claim 1, characterized in that: The oil spill dark spot segmentation model is obtained by training an exponential logarithmic loss function.

6. The marine oil spill detection method based on image segmentation according to claim 1, characterized in that, Before inputting the image to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image, the method further includes: The image to be detected is preprocessed according to preset preprocessing rules.

7. A drone, characterized in that, include: An inspection area determination module is used to determine an inspection area, which includes multiple inspection nodes. The inspection module is used to determine an initial inspection path based on multiple inspection nodes, wherein the initial inspection path passes through each inspection node once. The path planning module is used to use the initial inspection path as the initial population of the artificial bee colony algorithm, and to determine the energy-optimal target path through the artificial bee colony algorithm. An image acquisition module is used to perform inspections according to the target path and to capture images to be detected at each inspection node using a SAR image imager installed on the UAV. The detection module is used to input the image to be detected into a pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image; A marine oil spill detection result determination module is used to determine the marine oil spill detection result of the inspection node based on the segmented image; The oil spill dark spot segmentation model includes a feature extractor and a transformer encoder; the step of inputting the image to be detected into the pre-trained oil spill dark spot segmentation model for image segmentation processing to obtain a segmented image includes: The image to be detected is input into the feature extractor for feature extraction, resulting in multiple first feature maps; Perform linear projection processing on each of the first feature maps to obtain the first feature sequence; Each of the first feature sequences is input into the transformer encoder for position encoding to obtain the second feature sequence; The second feature map is obtained by linearly projecting each of the second feature sequences. Each of the second feature maps is upsampled to obtain a third feature map, and the size of each third feature map is the same as the size of the image to be detected. The segmented image is obtained by fusing all the third feature maps.

8. An unmanned aerial vehicle (UAV), comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the image segmentation-based marine oil spill detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions for performing the image segmentation-based marine oil spill detection method as described in any one of claims 1 to 6.

Images