Unmanned aerial vehicle-mounted sea surface oil spill detection system, method, equipment, medium and product

Abstract

The invention discloses an unmanned aerial vehicle-mounted sea surface oil spill detection system, method, device, medium and product, and relates to the field of sea surface oil spill detection, the system comprises a polarization characteristic detection module, an image fusion module and a data processing module; the polarization characteristic detection module comprises an unmanned aerial vehicle and an image shooting unit; the image shooting unit is connected with the image fusion module; the image fusion module is connected with the data processing module; the unmanned aerial vehicle hovers at a set position according to a set navigation route; the image shooting unit shoots multi-modal images of the sea surface to be detected during hovering, wherein the multi-modal images comprise a visible light intensity image, a visible light polarization image, an infrared intensity image and an infrared polarization image; the image fusion module performs image registration processing, radiation correction processing and image fusion processing on the multi-modal image to obtain a fused image; and the data processing module determines the polarization degree of the fused image so as to determine the oil spill type according to the polarization degree. The sea surface oil spill detection precision is improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] The present application relates to the field of sea surface oil spill detection, and in particular, to an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system, method, device, medium and product. Background Art

[0002] With the rapid development of modern industry, oil spills from oil tankers, the development of coastal and estuarine oil deposits, the discharge of industrial wastewater from refineries, etc. can pollute water bodies with oil. Especially in estuarine and offshore waters, oil pollution is very prominent. It not only causes serious environmental pollution to the sea surface, severely damages the marine ecosystem, but also affects human health and causes huge economic losses. Therefore, detecting sea surface oil spills and improving the detection accuracy have always been research hotspots for marine environmental ecological security.

[0003] The polarization bidirectional reflectance distribution function (pBRDF) is used to describe the polarization reflection characteristics of materials. It can not only quantify the magnitude of scattering in different directions, but also give the polarization characteristics of scattering. Currently, the observation equipment for pBRDF characteristics usually uses orbital sliding to control the observation angle of the spectrometer to achieve multi-angle spectral reflectance measurement of ground object targets. Such observation equipment generally has the following deficiencies: The equipment is large in volume and mass, not convenient for transportation, and needs to be assembled on site, which is not suitable for sites with poor accessibility; When observing, the equipment needs to be directly erected on the ground, which will damage the observed target and its surrounding environment to a certain extent and affect the natural characteristics of the measurement target; Due to the size of the observation orbit, the distance between the instrument and the target is limited, and it is impossible to effectively observe large-scale targets and mixed pixels, etc. for pBRDF; Spectral measurement requires manual operation, with a large workload, low measurement repeatability, and large random errors. Summary of the Invention

[0004] The purpose of the present application is to provide an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system, method, device, medium and product to accurately distinguish the types of oil spills and reduce the false alarm rate.

[0005] To achieve the above purpose, the present application provides the following solutions: In a first aspect, the present application provides an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system, including: a polarization characteristic detection module, an image fusion module, and a data processing module; the polarization characteristic detection module includes a pod, a UAV, and an image capturing unit; the pod is suspended directly below the UAV; the image capturing unit is placed inside the pod; the image capturing unit is connected to the image fusion module; the image fusion module is connected to the data processing module; The drone is used to hover at a set position according to a set navigation route; the image capture unit is used to capture multi-modal images of the sea surface to be detected during hovering; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images; The image fusion module is used to perform image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image; The data processing module is used to determine the polarization degree of the fused image, and thus determine the type of oil spill according to the polarization degree.

[0006] Optionally, the image capture unit includes: a visible light intensity camera, an infrared intensity camera, a visible light polarization camera, and an infrared polarization camera; The visible light intensity camera is used to capture visible light intensity images of the sea surface to be detected; The infrared intensity camera is used to capture infrared intensity images of the sea surface to be detected; The visible light polarization camera is used to capture visible light polarization images of the sea surface to be detected; The infrared polarization camera is used to capture infrared polarization images of the sea surface to be detected.

[0007] Optionally, the image fusion module includes: An image registration unit, which is used to register the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain a registered visible light intensity image, a registered visible light polarization image, a registered infrared intensity image, and a registered infrared polarization image; An image preprocessing unit, which is used to perform radiation correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain a corrected visible light intensity image, a corrected visible light polarization image, a corrected infrared intensity image, and a corrected infrared polarization image; An image fusion unit, which is used to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image by using a residual network to obtain a fused image.

[0008] In a second aspect, the present application provides an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection method. The UAV-borne sea surface oil spill detection method is applied to the above-mentioned UAV-borne sea surface oil spill detection system. The UAV-borne sea surface oil spill detection method includes: Obtain multi-modal images of the sea surface to be detected; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images; Perform image registration processing, radiometric correction processing, and image fusion processing on the multi-modal image to obtain a fused image; Determine the degree of polarization of the fused image, and thus determine the type of oil spill according to the degree of polarization.

[0009] Optionally, performing image registration processing, radiometric correction processing, and image fusion processing on the multi-modal image to obtain a fused image specifically includes: Register the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain a registered visible light intensity image, a registered visible light polarization image, a registered infrared intensity image, and a registered infrared polarization image; Perform radiometric correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain a corrected visible light intensity image, a corrected visible light polarization image, a corrected infrared intensity image, and a corrected infrared polarization image; Use a residual network to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image to obtain a fused image.

[0010] Optionally, registering the visible light polarization image and the infrared polarization image to obtain a registered visible light polarization image and a registered infrared polarization image specifically includes: Use the ray contour feature point extraction method to extract the first feature point set of the visible light polarization image and the second feature point set of the infrared polarization image; According to the first feature point set and the second feature point set, calculate the first mapping transformation matrix from the visible light polarization image to the infrared polarization image; Determine whether the number of first polarization image pairs is greater than the set number of groups; the first polarization image pair includes the visible light polarization image and the infrared polarization image collected at the same moment; If not, continue to obtain the multi-modal image of the sea surface to be detected; If so, take the mean of the first mapping transformation matrices of the set number of groups to obtain a mapping transformation mean matrix; Determine the root mean square error between the first mapping transformation matrix of the remaining first polarization image pairs and the mapping transformation mean matrix; If the root mean square error is less than the set error value, use the mapping transformation mean matrix as the image registration matrix; Based on the image registration matrix, register the visible light polarization image and the infrared polarization image to obtain a registered visible light polarization image and a registered infrared polarization image.

[0011] Optionally, determining the polarization degree of the fused image specifically includes: Using the formula to determine the polarization degree of the fused image; where DOLP is the polarization degree of the fused image; is the coefficient of the specular reflection component; ; ; is the diffuse reflection coefficient; is the diffuse reflection component; ; is the coefficient of the scattering component; is the scattering component; ; M ij s is the Mueller matrix element; q is the generalized parameter used to determine wide-angle scattering; θ i is the zenith angle of the incident direction; θ r is the zenith angle of the reflection direction; α is the angle between the microfacet normal and the macroscopic normal of the rough surface; β is the angle between the incident light and the microfacet normal; σ is the surface roughness constant of the object; τ is the parameter characterizing the shadow of the rough surface; Ω is the parameter characterizing the occlusion effect, is the coefficient of the multiple scattering component; is the coefficient of the single scattering component; M 10 、M 00 are the elements in the Mueller matrix; v is the incident angle of the scattering upper surface; v0 is the exit angle of the scattering upper surface.

[0012] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the unmanned aerial vehicle-borne sea surface oil spill detection method described in any one of the above.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the unmanned aerial vehicle-borne sea surface oil spill detection method described in any one of the above.

[0014] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the unmanned aerial vehicle-borne sea surface oil spill detection method described in any one of the above.

[0015] According to the specific embodiments provided by the present application, the present application has the following technical effects: The present application provides an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system, method, device, medium and product. The system includes: a polarization characteristic detection module, an image fusion module, and a data processing module; the polarization characteristic detection module includes a pod, a UAV, and an image capturing unit; the pod is suspended directly below the UAV; the image capturing unit is placed inside the pod; the image capturing unit is connected to the image fusion module; the image fusion module is connected to the data processing module; the UAV is used to hover at a set position according to a set navigation route; the image capturing unit is used to capture multi-modal images of the sea surface to be detected during hovering; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images; the image fusion module is used to perform image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image; the data processing module is used to determine the polarization degree of the fused image, so as to determine the type of oil spill according to the polarization degree. By combining visible light detection with polarized infrared detection, and comparing and analyzing the polarization degrees of common oil spills in the ocean, the present application realizes the differentiation of oil spill types. The fused image combining infrared and visible light information in the present application can better utilize the indication characteristics and texture details of the source images, which is convenient for effectively distinguishing oil spills from other marine species such as algae, biofilms, and foams, as well as different types of oil spills during detection, improving the detection accuracy and realizing all-weather detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 It is a schematic structural diagram of an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system provided by an embodiment of the present application; Figure 2 It is a structural block diagram of an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system; Figure 3 It is a schematic diagram of a UAV pod device of an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system; Figure 4 It is a UAV flight trajectory diagram of an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system; Figure 5 It is a flow chart of a registration method for four images in an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system; Figure 6 It is an image fusion block diagram of four images in an unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system; Figure 7Schematic structural diagram of a computer device provided by an embodiment of the present application.

[0018] Reference numerals: 1 - unmanned aerial vehicle; 2 - pod; 3 - runner; 4 - visible light intensity camera; 5 - visible light polarization camera; 6 - infrared intensity camera; 7 - infrared polarization camera. Detailed implementation manners

[0019] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0020] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0021] The pBRDF measurement system can accurately measure the reflection characteristics of the object surface, including the reflection light intensity and polarization state at different angles and wavelengths. When an unmanned aerial vehicle is equipped with such a system, it can perform high-precision measurement and detection on ground targets in the air. The unmanned aerial vehicle-borne pBRDF measurement system has a wide range of application prospects. In the agricultural field, the unmanned aerial vehicle equipped with the pBRDF measurement system can detect the growth status of crops, the occurrence of pests and diseases, and soil humidity, etc., providing data support for precision agriculture; in the military field, the unmanned aerial vehicle-borne pBRDF measurement system can be used for target reconnaissance and camouflage recognition, improving the concealment and accuracy of military operations. The unmanned aerial vehicle equipped with the pBRDF measurement system can transmit measurement data in real time, providing timely and accurate information support for decision-makers. The unmanned aerial vehicle has the characteristics of flexible deployment and can quickly reach the designated area for measurement and detection, without being restricted by ground traffic and terrain. These advantages make the unmanned aerial vehicle-borne polarization BRDF system have a wide range of application prospects and important practical values in multiple fields.

[0022] Spectral polarization imaging technology organically combines spectral imaging technology and polarization imaging technology, enabling the data obtained by the instrument to present a multi-dimensional form, greatly enriching the information of the target. The multi-dimensional information can make the measurement results more accurate. Not only more information can be obtained, but also multi-dimensional data and images such as intensity, polarization, and spectrum can be obtained simultaneously, and the types of spilled oil and the thickness of the spilled oil film can be identified more quickly and accurately. According to the advantageous characteristics of polarization, the influence of environmental factors can also be reduced, improving the detection ability of the detector and the stability of information acquisition. By combining spectral imaging technology and polarization imaging technology, different types of spilled oil can be distinguished through multi-dimensional polarization images, and at the same time, wide-area search and fine identification of multi-dimensional sea surface oil spills can be realized, providing decision-making information for the rapid and accurate monitoring and traceability treatment of sea surface oil spills.

[0023] This application integrates infrared and visible light polarization information, can better utilize the texture details of the source image, is convenient for effectively distinguishing the types of spilled oil during detection, reduces the false alarm rate, and gives full play to the advantages of multi-source information fusion.

[0024] In an exemplary embodiment, as Figures 1 - 3 shown, a drone-borne sea surface oil spill detection system is provided, including: a polarization characteristic detection module (i.e., Figure 2 the polarization characteristic detection system in Figure 2 ), an image fusion module (i.e., Figure 2 the image fusion system in

[0025] ), and a data processing module (i.e., Figure 2 the data processing system in

[0025] ); the polarization characteristic detection module includes a pod 2, a drone 1, and an image capturing unit; the pod 2 is suspended directly below the drone 1; the image capturing unit is placed inside the pod 2; the image capturing unit is connected to the image fusion module; the image fusion module is connected to the data processing module. This system can realize the pBRDF measurement of sea surface oil spills within a hemisphere.

[0026] The visible light intensity camera 4 is used to capture the visible light intensity image of the sea surface to be detected.

[0027] The infrared intensity camera 6 is used to capture the infrared intensity image of the sea surface to be detected.

[0028] The visible light polarization camera 5 is used to capture the visible light polarization image of the sea surface to be detected.

[0029] The infrared polarization camera 7 is used to capture the infrared polarization image of the sea surface to be detected.

[0030] The unmanned aerial vehicle 1 is used to hover at a set position according to a set navigation route; the image capturing unit is used to capture multi-modal images of the sea surface to be detected when hovering; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images.

[0031] In this embodiment, the unmanned aerial vehicle-borne sea surface oil spill detection system of the present application is described by taking the sea surface oil spill conditions simulated in an outdoor large pool as an example for experiments.

[0032] The environmental conditions are outdoor, and natural light is used as the light source.

[0033] The aircraft trajectory diagram is as Figure 4 shown. The unmanned aerial vehicle 1 first flies to the highest point in the aircraft trajectory diagram, that is, at the zenith angle of 0°, and the camera observes the surface target vertically downward; the set navigation route of the unmanned aerial vehicle 1 is represented by the following formula: The observation points of the unmanned aerial vehicle 1 at specific zenith angles and azimuth angles can be represented by the following formula: .

[0034] where f is a function describing the relationship between the position of the unmanned aerial vehicle 1 and the camera angle, θ is the zenith angle, is the azimuth angle, H is the initial height of the unmanned aerial vehicle 1, and R is the initial flight radius of the unmanned aerial vehicle 1.

[0035] The height and flight radius of the unmanned aerial vehicle 1 follow the following formula: H′ = H - kθ.

[0036] where H′ is the height of the unmanned aerial vehicle 1 at the corresponding zenith angle; k is the zenith angle adjustment coefficient.

[0037] .

[0038] where R′ is the flight radius of the unmanned aerial vehicle 1 at the corresponding azimuth angle; d is the azimuth angle adjustment coefficient.

[0039] The zenith angle θ and the azimuth angle satisfy the following relationship: .

[0040] .

[0041] where, The symbol represents rounding, and θ0 is the initial zenith angle.

[0042] During the observation, the range of the zenith angle θ is 0° - 60°, and the azimuth angle The observation range is 0° - 360°, the azimuth angle observation interval is 30°, and the hovering observation at different azimuth angles is completed by changing the pitch angle and flight radius of the camera on the stable platform. The camera lens is always facing the oil spill surface.

[0043] Then, the visible light intensity camera 4, the visible light polarization camera 5, the infrared intensity camera 6, and the infrared polarization camera 7 respectively collect the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image of the oil spill surface at each observation point.

[0044] The image fusion module is used to perform image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image.

[0045] As an optional implementation manner, the image fusion module includes: An image registration unit, which is used to register the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image.

[0046] An image preprocessing unit, which is used to perform radiation correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image.

[0047] An image fusion unit, which is used to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image by using a residual network to obtain a fused image.

[0048] In practical applications, during the image registration process of the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image, the first feature point set K and the second feature point set I of the visible light polarization image and the infrared polarization image are extracted by using the method of extracting ray contour feature points.

[0049] The corresponding points of the visible light polarization image and the infrared polarization image to be registered are registered between the image pairs by using the mapping transformation matrix H.

[0050] Furthermore, the mapping transformation matrix H is the homography matrix, which is represented by the following formula: 。

[0051] The conversion relationship of the feature points between the infrared polarization image and the visible light polarization image can be written in the following form: 。

[0052] In the formula, (x, y) are the coordinates of the feature points of the visible light polarization image; (x', y') are the coordinates of the feature points of the infrared polarization image; h 00 -h 33 are the parameters of the homography matrix, and σ H is the scale parameter.

[0053] Taking the visible light polarization image and the infrared polarization image as examples, the registration process is described as follows, as Figure 5 shown, the specific process is as follows: 1) Determine whether the number of image pairs (the first polarization image pair) of the collected visible light polarization image and infrared polarization image is greater than the set number of groups. In this embodiment, the set number of groups is 30 groups. If not, go to step 2); if so, go to step 3).

[0054] 2) Corresponding to take 4 pairs of feature points in the second feature point set I of the infrared polarization image and the first feature point set K of the visible light polarization image, and calculate the H matrix from the visible light polarization image to the infrared polarization image plane; continue step 1).

[0055] 3) Take the mean value M (mapping transformation mean matrix) of 30 groups of H matrices and save it.

[0056] 4) Judge the remaining groups, calculate the H matrix, and output the calibration result: calculate the root mean square error between the H matrix of the remaining groups and the mean value M. If the error value is less than 30%, the mean value M of the H matrix is the image registration matrix of the visible light polarization image and the infrared polarization image, and the visible light polarization image and the infrared polarization image are registered based on this image registration matrix.

[0057] As Figure 6As shown, perform radiometric correction preprocessing on the registered visible light intensity image, registered visible light polarization image, registered infrared intensity image, and registered infrared polarization image. First, use bilinear interpolation to resample the images with mismatched resolutions or pixel coordinates, so that all modal images have the same resolution on the same plane. Subsequently, through histogram matching or other radiometric correction methods, further balance the differences in gray value distribution or contrast between the visible light image and the infrared image, so as to obtain relatively unified and comparable multi-modal input data. Subsequently, on the network structure, merge the registered visible light intensity image and the registered visible light polarization image into 2 channels, and also merge the registered infrared intensity image and the registered infrared polarization image into 2 channels, and send them into two improved residual network branches for multi-scale deep feature extraction. Replace the original 3-channel convolution with a convolution kernel adapted to 2 channels at the input layer and freeze some of the shallow convolution kernels to optimize the degradation of shallow features in the initial stage of training. The extracted multi-scale deep features are compressed to 512 channels respectively through 1×1 convolution and then fused element-wise. Introduce the ECA-Net channel attention and CBAM spatial attention modules to adaptively allocate feature weights in the channel dimension and the spatial dimension, and use residual connections to retain the original information. Then, use transposed convolution layer by layer to upsample to the target resolution and splice the low-level features of each stage of the residual network to enhance texture and edge information. Finally, output the fused image through 3×3 convolution.

[0058] The data processing module is used to determine the polarization degree of the fused image, so as to determine the type of oil spill according to the polarization degree.

[0059] Through the calculation of the data processing module, the pBRDF data of the fused image can be obtained, and the polarization characteristic parameters such as the polarization degree of the fused image can be obtained. By comparing the polarization characteristic parameters of different oil spill types and analyzing their polarization characteristic differences, the oil spill types can be effectively distinguished. The oil spill types include crude oil, diesel, gasoline, kerosene, etc.

[0060] Use pBRDF to describe the polarization degree of an object. The expression of the pBRDF model is as follows: .

[0061] Among them, k s , k m , k scat are the coefficients of the specular reflection component, the coefficients of the diffuse reflection component, and the coefficients of the scattering component respectively; f s is the specular reflection component; f m is the diffuse reflection component; f scat is the scattering component; M ij s are the Mueller matrix elements; qis a general parameter used to determine wide-angle scattering; θ i is the zenith angle of the incident direction; θ r is the zenith angle of the reflection direction; α is the angle between the microfacet normal and the macroscopic normal of the rough surface; β is the angle between the incident light and the microfacet normal; σ is the surface roughness constant of the object; is a parameter characterizing the shadow of the rough surface; Ω is a parameter characterizing the occlusion effect, is the coefficient of the multiple scattering component; is the coefficient of the single scattering component; v is the incident angle on the scattering upper surface; v0 is the exit angle on the scattering upper surface.

[0062] Under the passive illumination conditions of non-polarized light such as natural light, the Stokes vector of the incident light is expressed as , then the expression of the Stokes vector of the corresponding reflected light is as follows: .

[0063] where I is the total light intensity; Q is the linear polarization component in the horizontal and vertical directions; U is the linear polarization component in the ±45° direction; V is the circular polarization component; f 00 -f 33 is the Mueller matrix in the elements; E0, E1, E2, E3 are the parameters in the Stokes vector in.

[0064] Calculate the degree of polarization under natural light illumination, and the expression is as follows: .

[0065] where f s10 , f s20 , f s30 are the elements of the specular reflection component in the Mueller matrix in.

[0066] Since the circular polarization component in natural light is less, this component can be ignored in the reflected light. The incident light and the pBRDF matrix can be reduced in dimension. The incident light , then the expression of the Stokes vector of the reflected light is as follows: .

[0067] Then the degree of polarization satisfying the coplanar relationship after ignoring the circular polarization component can be further simplified, and the expression is as follows:

[0068] where M 10, M 00 represents an element in the Mueller matrix .

[0069] The airborne sea surface oil spill detection system of the present application uses a method combining visible light detection and polarized infrared detection to compare and analyze the visible light / infrared polarization characteristics of common oil spills in the ocean, so as to distinguish the types of spilled oils. The image fusion scheme of the present application can reduce the computational cost and complexity, and the fused image combining infrared and visible light information can better utilize the indicative characteristics and texture details of the source images, facilitating the effective distinction between oil spills and other marine species such as algae, biofilms, and foams, as well as different types of oil spills during detection, improving the detection accuracy, and achieving all-weather detection.

[0070] Based on the same inventive concept, the embodiment of the present application also provides an airborne sea surface oil spill detection method implemented based on the above-mentioned airborne sea surface oil spill detection system.

[0071] In an exemplary embodiment, an airborne sea surface oil spill detection method is provided, including:[[]] Obtaining multimodal images of the sea surface to be detected; the multimodal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images.

[0072] Performing image registration processing, radiometric correction processing, and image fusion processing on the multimodal images to obtain a fused image.

[0073] As an optional implementation manner, performing image registration processing, radiometric correction processing, and image fusion processing on the multimodal images to obtain a fused image specifically includes:[[]] Performing registration on the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain a registered visible light intensity image, a registered visible light polarization image, a registered infrared intensity image, and a registered infrared polarization image.

[0074] Performing radiometric correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain a corrected visible light intensity image, a corrected visible light polarization image, a corrected infrared intensity image, and a corrected infrared polarization image.

[0075] Using a residual network to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image to obtain a fused image.

[0076] As an alternative implementation, the visible light polarization image and the infrared polarization image are registered to obtain a registered visible light polarization image and a registered infrared polarization image, which specifically includes: The first feature point set of the visible light polarization image and the second feature point set of the infrared polarization image are extracted by using the ray contour feature point extraction method.

[0077] According to the first feature point set and the second feature point set, a first mapping transformation matrix from the visible light polarization image to the infrared polarization image is calculated.

[0078] It is determined whether the number of first polarization image pairs is greater than a set number of groups; the first polarization image pairs include the visible light polarization image and the infrared polarization image collected at the same moment.

[0079] If not, the multi-modal images of the sea surface to be detected are continuously acquired.

[0080] If so, the mean value of the first mapping transformation matrices of the set number of groups is taken to obtain a mapping transformation mean matrix.

[0081] The root mean square error between the first mapping transformation matrix of the remaining first polarization image pairs and the mapping transformation mean matrix is determined.

[0082] If the root mean square error is less than a set error value, the mapping transformation mean matrix is used as the image registration matrix.

[0083] Based on the image registration matrix, the visible light polarization image and the infrared polarization image are registered to obtain a registered visible light polarization image and a registered infrared polarization image.

[0084] The polarization degree of the fusion image is determined, and thus the type of oil spill is determined according to the polarization degree.

[0085] As an alternative implementation, determining the polarization degree of the fusion image specifically includes: Using the formula to determine the polarization degree of the fusion image; where DOLP is the polarization degree of the fusion image; is the coefficient of the specular reflection component; ; ; is the diffuse reflection coefficient; is the diffuse reflection component; ; is the coefficient of the scattering component; is the scattering component; 。

[0086] In an exemplary embodiment, a computer device is provided, which includes a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned method for detecting oil spills on the sea surface by an unmanned aerial vehicle is implemented.

[0087] In an exemplary embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for detecting oil spills on the sea surface by an unmanned aerial vehicle is implemented.

[0088] In an exemplary embodiment, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the above-mentioned method for detecting oil spills on the sea surface by an unmanned aerial vehicle is implemented.

[0089] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structural diagram can be as shown in Figure 7 The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting oil spills on the sea surface by an unmanned aerial vehicle is implemented.

[0090] Those skilled in the art can understand that Figure 7 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0091] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0092] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0093] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0094] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0095] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An unmanned aerial vehicle (UAV)-borne sea surface oil spill detection system, characterized in that, Including: A polarization characteristic detection module, an image fusion module, and a data processing module; the polarization characteristic detection module includes a pod, a drone, and an image capture unit; the pod is suspended directly below the drone; the image capture unit is placed inside the pod; the image capture unit is connected to the image fusion module; the image fusion module is connected to the data processing module; The drone is used to hover at a set position according to a set navigation route; the image capture unit is used to capture multi-modal images of the sea surface to be detected during hovering; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images; The image fusion module is used to perform image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image; The data processing module is used to determine the polarization degree of the fused image, and thus determine the type of oil spill according to the polarization degree.

2. The drone-borne sea oil spill detection system according to claim 1, wherein, The image capture unit includes: a visible light intensity camera, an infrared intensity camera, a visible light polarization camera, and an infrared polarization camera; The visible light intensity camera is used to capture visible light intensity images of the sea surface to be detected; The infrared intensity camera is used to capture infrared intensity images of the sea surface to be detected; The visible light polarization camera is used to capture visible light polarization images of the sea surface to be detected; The infrared polarization camera is used to capture infrared polarization images of the sea surface to be detected.

3. The drone-borne sea surface oil spill detection system according to claim 1, wherein, The image fusion module includes: An image registration unit, used to register the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain a registered visible light intensity image, a registered visible light polarization image, a registered infrared intensity image, and a registered infrared polarization image; An image preprocessing unit, used to perform radiation correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain a corrected visible light intensity image, a corrected visible light polarization image, a corrected infrared intensity image, and a corrected infrared polarization image; An image fusion unit, used to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image by using a residual network to obtain a fused image.

4. A method for detecting oil spills on the sea surface carried by an unmanned aerial vehicle, characterized in that, The method for detecting oil spills on the sea surface carried by a drone is applied to the system for detecting oil spills on the sea surface carried by a drone according to any one of claims 1-3. The method for detecting oil spills on the sea surface carried by a drone includes: Obtaining multi-modal images of the sea surface to be detected; the multi-modal images include visible light intensity images, visible light polarization images, infrared intensity images, and infrared polarization images; Performing image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image; Determining the polarization degree of the fused image, and thus determining the type of oil spill according to the polarization degree.

5. The method for detecting sea surface oil spills by an unmanned aerial vehicle according to claim 4, wherein, Performing image registration processing, radiation correction processing, and image fusion processing on the multi-modal images to obtain a fused image, specifically including: Register the visible light intensity image, the visible light polarization image, the infrared intensity image, and the infrared polarization image to obtain the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image; Perform radiometric correction processing on the registered visible light intensity image, the registered visible light polarization image, the registered infrared intensity image, and the registered infrared polarization image to obtain the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image; Use a residual network to perform image fusion on the corrected visible light intensity image, the corrected visible light polarization image, the corrected infrared intensity image, and the corrected infrared polarization image to obtain a fused image.

6. The method for detecting sea surface oil spills by an unmanned aerial vehicle according to claim 5, wherein Register the visible light polarization image and the infrared polarization image to obtain the registered visible light polarization image and the registered infrared polarization image, specifically including: Use the ray contour feature point extraction method to extract the first feature point set of the visible light polarization image and the second feature point set of the infrared polarization image; Calculate the first mapping transformation matrix from the visible light polarization image to the infrared polarization image according to the first feature point set and the second feature point set; Determine whether the number of first polarization image pairs is greater than the set number of groups; the first polarization image pair includes the visible light polarization image and the infrared polarization image collected at the same moment; If not, continue to obtain the multi-modal images of the sea surface to be detected; If so, take the average value of the first mapping transformation matrices of the set number of groups to obtain the mapping transformation average matrix; Determine the root mean square error between the first mapping transformation matrix of the remaining first polarization image pairs and the mapping transformation average matrix; If the root mean square error is less than the set error value, use the mapping transformation average matrix as the image registration matrix; Based on the image registration matrix, register the visible light polarization image and the infrared polarization image to obtain the registered visible light polarization image and the registered infrared polarization image.

7. The method for detecting sea surface oil spills by an unmanned aerial vehicle according to claim 4, wherein Determine the degree of polarization of the fused image, specifically including: Using the formula to determine the polarization degree of the fused image; where DOLP is the polarization degree of the fused image; is the coefficient of the specular reflection component; ; ; is the diffuse reflection coefficient; is the diffuse reflection component; ; is the coefficient of the scattering component; is the scattering component; ; M ij s is the Mueller matrix element; q is the generalized parameter used to determine wide-angle scattering; θ i is the zenith angle of the incident direction; θ r is the zenith angle of the reflection direction; α is the angle between the microfacet normal and the macroscopic normal of the rough surface; β is the angle between the incident light and the microfacet normal; σ is the surface roughness constant of the object; τ is the parameter characterizing the shadow of the rough surface; Ω is the parameter characterizing the occlusion effect, is the coefficient of the multiple scattering component; is the coefficient of the single scattering component; M 10 、M 00 are the elements in the Mueller matrix; v is the incident angle on the scattering upper surface; v0 is the exit angle on the scattering upper surface.

8. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the unmanned aerial vehicle-borne sea surface oil spill detection method according to any one of claims 4-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the unmanned aerial vehicle-borne sea surface oil spill detection method according to any one of claims 4-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the unmanned aerial vehicle-borne sea surface oil spill detection method according to any one of claims 4-7.

Images