An oil spill detection algorithm on water surface based on visible light and infrared images

Abstract

The present invention discloses an algorithm for detecting oil spills on the water surface based on visible light and infrared images, belonging to the field of marine remote sensing technology. It includes four steps: obtaining images and performing preprocessing, image classification and recognition of visible light images, oil spill detection in infrared images, and confirmation of oil spill targets on the sea surface. By using visible light and infrared data, the data advantages can be fully utilized to improve the robustness and accuracy of oil spill detection. First, based on visual classification, the water surface is pre-extracted and the oil film is initially extracted from visible images to reduce false alarms in infrared oil spill detection. Finally, the oil spill area on the water surface is confirmed by fusing the results of the two image features.

Description

Technical Field

[0001] The present invention relates to the technical field of marine remote sensing, and particularly to an algorithm for detecting oil spills on the water surface based on visible and infrared images. Background Art

[0002] Oil spills in the ocean not only cause serious damage to the marine ecological environment, but also cause huge economic losses to society. Quickly and accurately obtaining the oil film coverage area in an oil spill accident can provide powerful assistance for accident handling.

[0003] Existing marine oil spill sensors include fluorescence, polarimetric synthetic aperture radar (SAR), infrared, ultraviolet, etc. Among them, visible and infrared sensors are relatively low in cost and wide in use range, and are suitable for large-scale test detection.

[0004] The principle of using infrared images to detect oil spills is that when there is an oil film on the water surface, the radiation temperatures of water and oil are different, which is manifested as a difference in thermal infrared images. The thermal infrared detection method can adapt to all-weather work. However, due to the low contrast, blurred scene edges and serious noise in infrared images, it is vulnerable to environmental interference. In practical applications, water surface suspended impurities such as algae, biofilms, and foams have great similarity with oil films in infrared images, often causing false detections in infrared images.

[0005] On the one hand, the principle of visible light detection is that the difference in reflectivity between the oil film on the water surface and the clean water surface results in different gray levels in the visible light image. In addition, solar flare radiance can also be used for detection. According to the research of C. Cox and W. Munk, there are different coefficient values in the solar flare models of clean sea surfaces and sea surfaces with oil films, that is, visible light images containing flares can be used for oil film detection. Many subsequent studies have also confirmed the feasibility of using flare images for detecting oil spills on the water surface. Whether the solar flare area is rougher or smoother than the background sea water area is manifested as brighter or darker than the background sea water area in the image, and this depends on the angles of these areas relative to the specular reflection direction. Visible light can provide high-resolution images and can better identify the interference of non-oil film areas such as algae, biofilms, and foams that are easily misdetected by infrared. However, the visible light detection conditions are affected by weather and light, and all-weather detection cannot be achieved. Summary of the Invention

[0006] To solve the above problems, the present invention provides an algorithm for detecting oil spills on the water surface based on visible and infrared images. By using visible and infrared data, the data advantages can be fully utilized to improve the robustness and accuracy of oil spill detection. First, based on visual classification, the water surface pre-extraction and initial oil film extraction of visible images are realized, reducing the false alarms of infrared oil spill detection. Finally, the oil spill area on the water surface is confirmed by fusing the results of the two image features. The specific content of the present invention is as follows:

[0007] An algorithm for detecting oil spills on the water surface based on visible light and infrared images, the technical points of which include the following steps:

[0008] Step 1: Obtain images and perform preprocessing: Obtain the visible light image RGB captured by a visible light camera and the infrared image IR captured by an infrared camera at the same time and in the same area, and perform preprocessing on each to obtain the visible light image D RGB and the infrared image D IR ;

[0009] Step 2: Image classification and recognition of the visible light image: Perform target classification and recognition on the preprocessed visible light image D in Step 1 RGB The classification categories include the water surface area V water , the suspected oil spill area V oil and three categories of other areas;

[0010] Step 3: Oil spill detection in the infrared image: Use the water surface area V obtained in Step 2 water as the input image to calculate the gray-level co-occurrence matrix of the infrared image, and select indicators to extract the visual features of the infrared image D in Step 1 IR ;

[0011] Step 4: Confirmation of the oil spill target on the sea surface: Use a random forest sub-model to obtain the oil spill target detection result. The input data is the suspected oil spill area V obtained in Step 2 oil and the visual feature indicators extracted in Step 3, and train the model to obtain the result of the oil spill on the sea surface.

[0012] In some embodiments of the present invention, the preprocessing method in Step 1 of the above algorithm for detecting oil spills on the water surface based on visible light and infrared images is as follows:

[0013] Preprocessing of the visible light image RGB: First, non-uniformity correction needs to be performed on the infrared image obtained by the sensor, and then denoising is performed to obtain an infrared image with clear and smooth edges;

[0014] Preprocessing of the infrared image IR: Perform distortion correction on the visible light image, and perform image cropping according to the field of view relationship with the infrared to ensure that the field of view ranges of the two are the same, and obtain the visible light image D RGB .

[0015] In some embodiments of the present invention, the target classification and recognition method in Step 2 of the above algorithm for detecting oil spills on the water surface based on visible light and infrared images is the deep learning image classification method.

[0016] In some embodiments of the present invention, the selected indicators in Step 3 of the above algorithm for detecting oil spills on the water surface based on visible light and infrared images are mean, variance, angular second moment, texture entropy, contrast, uniformity, texture correlation and dissimilarity.

[0017] The above at least one technical solution adopted in the embodiments of the present invention can achieve the following beneficial effects:

[0018] An oil spill detection algorithm for water surface based on visible light and infrared images of the present invention includes four steps: acquiring images and performing preprocessing, image classification and recognition of visible light images, oil spill detection of infrared images, and confirmation of oil spill targets on the sea surface. By using visible light and infrared data, the data advantages can be fully exploited to improve the robustness and accuracy of oil spill detection. First, based on visual classification, the water surface pre-extraction and initial oil film extraction of visible images are realized to reduce the false alarms of infrared oil spill detection, and finally the oil spill area on the water surface is confirmed by fusing the results of the two image features. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0020] Figure 1 is a flowchart of the algorithm for oil spill detection on the water surface based on visible light and infrared images of the present invention;

[0021] Figure 2 is the oil spill identification structure of the random forest in step 4 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0023] The following will detail the technical solutions provided by the embodiments of the present invention in conjunction with the drawings.

[0024] As Figure 1 shown, the present invention provides an oil spill detection algorithm for water surface based on visible light and infrared images, including the following steps:

[0025] Step 1: Acquire images and perform preprocessing

[0026] Acquire the visible light image RGB captured by a visible light camera and the infrared image IR captured by an infrared camera at the same time and in the same area. If the acquired spatial areas are inconsistent, geometric registration is required, and preprocessing is performed separately to obtain the visible light image D RGB and the infrared image D IR ;

[0027] The above preprocessing methods are as follows:

[0028] Visible light image RGB preprocessing: First, the non-uniformity correction needs to be performed on the infrared image obtained by the sensor, and then denoising is carried out to obtain a clear and smooth infrared image D with distinct edges IR ;

[0029] Infrared image IR preprocessing: Perform distortion correction on the visible light image, and according to the field of view relationship with the infrared, perform image cropping to ensure that the field of view ranges of both are consistent, obtaining the visible light image D RGB .

[0030] Step 2, image classification and recognition of visible light images

[0031] Based on the Cox-Munk solar flare radiance theory and the principle of water-oil reflectivity difference, use the method of deep learning image classification to perform target classification and recognition on the preprocessed visible light image D in Step 1 RGB The classification categories include the water surface area V water , the suspected oil spill area V oil and other areas;

[0032] According to the solar flare model theory of the ocean surface by C. Cox and W. Munk, it does not exclude inputting visible light images containing flares. Optionally, this step uses deep learning to identify the oil spill area V of the visible light image oil , which has a higher classification recognition accuracy compared to general models. It is necessary to prepare the visible light image training data in advance. Use the U-net model for semantic segmentation training, mark 1000 sample labels, and the number, category, and their positions on the image of the training and test samples can be reasonably selected according to actual needs

[0033] Specifically, set the classification category as (M = 3 in the present invention), set the input visible light image D RGB and its annotation ground truth have N, and are expressed in set form as , where represents the input visible light image D RGB , represents the annotation ground truth corresponding to the visible light image D RGB , and H and W respectively represent the height and width of the remote sensing image. The parameters of the network layer are , where L is the number of network layers. Each layer of the network is defined as , then the model is defined as follows:

[0034] where the o-th component of The score indicating that pixel x belongs to class o. The probability that the pixel points of the input image belong to a certain class is calculated using the relu activation function in the output layer. Finally, the binary cross-entropy loss function is used to calculate the difference between the predicted value and the true value and continuously correct it to obtain the optimal parameter W * 。

[0035] After classification and recognition, the dataset of the oil film area detected by the visible image is denoted as V oil As one of the important input data in Step 4, the water surface dataset is denoted as V water As the input data in Step 3, in terms of the dataset relationship 。

[0036] Step 3, Infrared image oil spill detection

[0037] Take the water surface area V obtained in Step 2 water As the input image to calculate the gray-level co-occurrence matrix of the infrared image, and select the mean, variance, angular second moment, texture entropy, contrast, uniformity, texture co-occurrence and dissimilarity indexes to extract the visual features of the infrared image D in Step 1 IR 。

[0038] Step 4, Sea surface oil spill target confirmation

[0039] Considering the variability of the detection accuracy of oil spill targets by visible and infrared sensors in different scenarios, as well as the unknown setting of the feature weights between them. Therefore, compared with decision trees, the random forest that combines multiple decision trees can judge the importance of different features of visible and infrared data by itself, and the final effect is better, and it is not easy to produce overfitting. The present invention adopts the random forest sub-model as shown in Figure 2 to obtain the oil spill target detection result. The input data is the suspected oil spill area V obtained in Step 2 oil and the visual feature indexes extracted in Step 3, and train the model to obtain the result of sea surface oil spill

[0040] The above is only the embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention

Claims

1. An algorithm for detecting oil spills on the water surface based on visible light and infrared images, characterized in that, It includes the following steps: Step 1, acquire images and perform preprocessing: Acquire the visible light image RGB captured by a visible light camera and the infrared image IR captured by an infrared camera at the same time and in the same area, and perform preprocessing on each to obtain the visible light image D RGB and the infrared image D IR ; The preprocessing method is as follows: Visible light image RGB preprocessing: First, it is necessary to perform non-uniformity correction on the infrared image obtained by the sensor, and then perform denoising to obtain an infrared image with clear and smooth edges; Infrared image IR preprocessing: Distortion correction is performed on the visible light image. According to the field of view relationship with the infrared, image cropping is carried out to ensure that the field of view ranges of the two are consistent, and the visible light image D is obtained. RGB； Step 2, image classification and recognition of visible light images: Perform target classification and recognition on the preprocessed visible light image D in Step 1 RGB The classification categories include the water surface area V water , the suspected oil spill area V oil and three categories of other areas; Step 3, Infrared image oil spill detection: Take the water surface area V obtained in Step 2 water as the input image to calculate the gray-level co-occurrence matrix of the infrared image, and select indicators to extract the visual features of the infrared image D IR in Step 1; Step 4, Confirmation of oil spill target on the sea surface: Use the random forest sub-model to obtain the detection results of the oil spill target. The input data is the suspected oil spill area V obtained in Step 2 oil and the visual feature indicators extracted in Step 3. Train the model to obtain the results of the oil spill on the sea surface.

2. The water surface oil spill detection algorithm based on visible light and infrared images according to claim 1, wherein, The target classification and recognition method in the second step is a deep learning image classification method.

3. The water surface oil spill detection algorithm based on visible light and infrared images according to claim 1 is characterized in that, The selected metrics in the third step are mean, variance, angular second moment, texture entropy, contrast, uniformity, texture co-occurrence, and dissimilarity.

Images