Oil Spill Patrol and Detection System Based on Multi-Spectral and LiDAR Fusion Imagery

Abstract

The present invention relates to the technical field of image recognition, and in particular to an oil spill patrol and survey system based on the fusion of multispectral and lidar images, including: a surface patrol group for observing the water surface, which is provided with an image drone cluster carrying multispectral cameras, several consoles for receiving water surface images, a layered patrol group for detecting underwater, which is provided with a radar drone cluster carrying lidar, a central server for responding to the water surface image and the layered image to determine the corresponding oil spill core and sending out corresponding core patrol signals; by recognizing the oil spill state on the sea surface, the oil spill recognition of the part below the sea surface is activated, and a model is built based on the recognition result. While effectively improving the sensitivity of oil spill monitoring, it avoids the energy consumption caused by frequently mobilizing drones equipped with radar. According to several detection data, the oil leakage point is identified, and at the same time, the completed detection model is recorded, thereby effectively improving the reliability of oil leakage patrol and survey.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to an oil spill patrol and detection system based on the fusion of multispectral and lidar images. Background Art

[0002] With the rapid growth of the world economy, the shipping industry has also developed rapidly, and the probability of oil spill accidents has increased accordingly. When an oil spill accident occurs, a large amount of oil spills instantaneously and enters the marine environment, which will cause extremely serious damage to the marine ecological environment.

[0003] In sea areas with severe oil pollution, the probability of red tides will increase. Moreover, with the diffusion and drift of the oil spill, facilities and scenic spots in coastal areas will also be affected, and there is even a possibility of triggering fire and explosion accidents, further exacerbating the losses and impacts of the disasters. Exploring the use of intelligent patrol drones for oil spills on water is an innovative way to enhance the ability to respond to oil spill incidents, reduce the safety risks faced by personnel, and improve the efficiency of oil spill rescue.

[0004] Chinese Patent Grant Publication No. CN114882371B provides a method for constructing a sea surface oil spill detection model based on fully polarized SAR images, including the following steps: extracting the anisotropic features of fully polarized SAR images, selecting the anisotropic features of oil spill pixels and non-oil spill pixels as training samples, forming matrix A and performing normalization processing to obtain a(t), calculating the pixel weight E from a(t), calculating the pixel feature P from a(t) and the pixel weight E, using the pixel feature P as the training set, and 0 and 1 as the training set labels, where 0 represents non-oil spill pixels and 1 represents oil spill pixels, importing the support vector machine model, and training the model to obtain the oil spill detection model. The invention makes full use of the anisotropic features of fully polarized SAR images and constructs the pixel feature P, improving the accuracy of sea surface oil spill detection in fully polarized SAR images, and having the advantages of being scientific, reasonable, easy to implement, and high in accuracy.

[0005] However, the above method has the following problems: The timeliness of the pictures to be recognized cannot be guaranteed during acquisition, making it difficult to detect oil spill incidents in the early stage of their occurrence. Summary of the Invention

[0006] Therefore, the present invention provides an oil spill patrol and detection system based on the fusion of multispectral and lidar images to overcome the problem in the prior art that the timeliness of the pictures to be recognized cannot be guaranteed during acquisition, resulting in a lag in the subsequent response to oil spill incidents and making it difficult to detect oil spill incidents in the early stage of their occurrence, thereby reducing the oil spill recognition efficiency.

[0007] To achieve the above object, the present invention provides an oil spill patrol and survey system based on multi - spectral and lidar fusion image recognition, including:

[0008] A surface patrol group for observing the water surface, which is equipped with an image drone cluster carrying multi - spectral cameras. Each cluster collects water surface images using visible light and / or infrared light according to the acquisition period.

[0009] Several consoles for receiving the water surface images, which are used to determine the corresponding oil - spill - like areas according to the water surface images.

[0010] A layered patrol group for detecting underwater, which is equipped with a lidar - carrying radar drone cluster, and is used to collect layered images of the oil - spill - like areas using radar.

[0011] A central server, which is connected to each console, and is used to determine the corresponding oil - spill core according to the water surface images and the layered images, and send out corresponding core patrol signals.

[0012] Among them, the console responds to the core patrol signal, controls the corresponding radar drone clusters to photograph various oil - spill - like areas to generate corresponding oil - spill images.

[0013] The acquisition period corresponds to the longest patrol duration of each drone constituting the image drone cluster.

[0014] The central server collects the water surface images, the layered images and the oil - spill images and forms an oil - spill monitoring database.

[0015] Further, the console consists of a drifting console and a shore - based console. Among them, any console has a corresponding patrol range, and the patrol ranges of each console cover the sea area to be patrolled in any acquisition period.

[0016] Further, the surface patrol group includes several hangars corresponding to each console, and several image drone clusters set based on each hangar;

[0017] Among them, for a single console, any drone in the corresponding drone cluster starts from the hangar corresponding to this console during a single patrol, and returns to the hangar corresponding to this console when completing the corresponding patrol route;

[0018] The patrol route of the drone is related to the patrol range of the console corresponding to this drone, and the image coverage range of its single - time patrol is not less than a preset ratio of the patrol range;

[0019] The preset ratio is related to the viewport of the drone.

[0020] Further, the radar drone group includes:

[0021] A flight station installed on the shore-based console, and a radar drone cluster composed of several radar drones;

[0022] Wherein, the number of the radar drones is equal to the number of the flight stations. When a single radar drone completes the shooting of the layered image, the radar drone enters the flight station with the shortest relative distance to it.

[0023] Furthermore, the central server is provided with an oil spill color difference value and an oil spill area threshold. When any adjacent areas of the water surface image in any survey range reach the oil spill color difference value, the central server issues a color difference warning;

[0024] When the water surface area of the water surface image in any acquisition cycle reaching the oil spill color difference value is not less than the oil spill area threshold, the central server issues a layered warning, and determines the corresponding oil spill-like area and the console of the corresponding oil spill-like area;

[0025] Wherein, the oil spill color difference value is the gray-scale difference value between the sea surface color and the oil spill color, and it is related to the illumination condition;

[0026] The oil spill area threshold is related to the maximum oil spill range of a single oil spill event in a single acquisition cycle.

[0027] Furthermore, the console corresponding to the oil spill-like area responds to the color difference warning, and sets several adjacent consoles as extended consoles;

[0028] Each extended console responds to the color difference warning and controls the corresponding image drone cluster to collect the corresponding survey range;

[0029] The central server responds to the color difference warning and extends the acquisition cycle until each extended console completes the acquisition of the corresponding water surface image.

[0030] Furthermore, the central server responds to the layered warning, sets at least one shore-based console closest to the center of the oil spill-like area corresponding to the layered warning as the active console, and

[0031] issues a layered image drawing instruction with at least one shore-based console closest to the edge of the oil spill-like area as the slave console.

[0032] Furthermore, the active console and the slave console respond to the layered image drawing instruction, and respectively control the corresponding radar drones to detect the layered image of the oil spill-like area according to a preset route;

[0033] Wherein, the preset route is related to the contour of the oil spill-like area and the corresponding ocean current direction.

[0034] Furthermore, the central server determines several classes of oil spill cores based on the layered images and issues corresponding core patrol signals;

[0035] The layered patrol group responds to the core patrol signal and detects the oil spill core according to the preset route;

[0036] Among them, the core patrol signal is sent to each shore-based console;

[0037] The oil spill core is located at the bottom layer of the layered image.

[0038] Furthermore, when the central server obtains the corresponding layered image of the oil spill core, it generates a corresponding oil spill image;

[0039] Among them, the oil spill image forms a corresponding longitudinal mapping from the layered image and a corresponding water surface mapping from the water surface image.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows. By setting the unmanned aerial vehicles as the surface patrol group and the layered patrol group, and setting several consoles and a central server, the oil spill identification of the part below the sea surface is activated by identifying the oil spill state of the sea surface, and a model is built based on the identification results. While effectively improving the sensitivity of oil spill monitoring, it avoids the energy consumption caused by frequently mobilizing the unmanned aerial vehicle equipped with a radar, and can identify the oil leakage point according to several detection data, and record the completed detection model at the same time, thus effectively improving the reliability of oil spill patrol.

[0041] Furthermore, by setting up the drifting console and the shore-based console to conduct patrols on the sea surface respectively, it avoids the situation that the oil spill phenomenon cannot be observed due to the oil spill position exceeding the patrol radius of the unmanned aerial vehicle, effectively improving the extensibility of the oil spill patrol system and further improving the reliability of the oil spill patrol.

[0042] Furthermore, by setting the corresponding patrol range according to the viewport of the unmanned aerial vehicle, while improving the patrol range of the unmanned aerial vehicle, it avoids the problem that the sea surface cannot be observed due to the mismatch between the shooting viewport size of the unmanned aerial vehicle and the patrol range, thus further improving the reliability of oil spill patrol.

[0043] Furthermore, by observing the color difference of the sea surface and presetting an oil spill area threshold to characterize the degree of oil spill diffusion caused by seawater flow, while effectively improving the timeliness of oil spill patrol, it avoids the situation that when a large oil spill area is found during detection, it is impossible to accurately determine whether the oil spill event is in the early stage or the diffusion stage, thus further improving the reliability of oil spill patrol.

[0044] Furthermore, by using the layered image as the vertical model benchmark and the water surface image as the benchmark of the horizontal model to model the oil spill event, and storing the completed oil spill model, with this model, a reasonable oil spill source area can be found through model analysis, which can provide model support for subsequent oil spill events, thereby further improving the reliability of oil spill patrol and survey. Brief Description of the Drawings

[0045] Figure 1 It is a schematic structural diagram of the oil spill patrol and survey based on the fusion image recognition of multi-spectral and lidar of the present invention;

[0046] Figure 2 It is a schematic connection diagram of the shore-based console of the oil spill patrol and survey system in the embodiment of the present invention;

[0047] Figure 3 It is a schematic connection diagram of the drifting console of the oil spill patrol and survey system in the embodiment of the present invention;

[0048] Figure 4 It is a schematic communication diagram of the oil spill patrol and survey UAV and the ground base station in the embodiment of the present invention;

[0049] Figure 5 It is the oil spill model in the embodiment of the present invention;

[0050] In the figure: 1, water surface image; 11, water surface oil spill range; 2, layered image; 21, layered oil spill range; 3, oil spill target layer; 31, speculated oil spill core; 4, water surface depth; 41, first layered depth; 42, second layered depth; 43, third layered depth; 44, oil spill core layered depth. Detailed Embodiment

[0051] In order to make the purpose and advantages of the present invention clearer, the present invention will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] The preferred embodiments of the present invention will be described below with reference to the drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principle of the present invention and do not limit the protection scope of the present invention.

[0053] It should be noted that in the description of the present invention, the terms indicating the direction or position relationship such as "upper", "lower", "left", "right", "inner", "outer", etc. are based on the direction or position relationship shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0054] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and defined, the terms "installation", "connection", and "linkage" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0055] Please refer to Figure 1 as shown, which is a schematic structural diagram of the oil spill patrol survey based on the fusion image recognition of multispectral and lidar in the present invention, including:

[0056] A surface inspection group for observing the water surface, which is provided with an image drone cluster carrying a multispectral camera, and each cluster collects water surface images according to the acquisition cycle using visible light and / or infrared light;

[0057] Several consoles for receiving the water surface images, which are used to determine the corresponding oil spill-like areas according to the water surface images;

[0058] A layered inspection group for detecting underwater, which is provided with a radar drone cluster carrying a lidar, and is used to collect the layered images of the oil spill-like areas according to the radar;

[0059] A central server, which is connected to each console, and is used to determine the corresponding oil spill core according to the water surface images and the layered images, and send out the corresponding core patrol signals;

[0060] Among them, the console responds to the core patrol signal and controls the corresponding radar drone clusters to take pictures of various oil spill-like areas to generate the corresponding oil spill images;

[0061] The acquisition cycle corresponds to the longest patrol duration of each drone constituting the image drone cluster;

[0062] The central server collects the water surface images, the layered images, and the oil spill images and forms an oil spill monitoring database.

[0063] By setting the drones as the surface inspection group and the layered inspection group, and setting several consoles and the central server, the oil spill identification of the part below the sea surface is activated by identifying the oil spill state of the sea surface, and modeling is carried out according to the identification results. While effectively improving the sensitivity of oil spill monitoring, it avoids the energy consumption caused by frequently mobilizing the drones equipped with radars, and can identify the oil leakage points according to several detection data, and record the completed detection model at the same time, thus effectively improving the reliability of the oil leakage patrol survey.

[0064] Please refer to Figure 2 and Figure 3As shown, they are respectively the connection schematic diagram of the shore-based console of the oil spill survey system according to the embodiment of the present invention and the connection schematic diagram of the drifting console of the oil spill survey system according to the embodiment of the present invention. Among them, the console is composed of a drifting console and a shore-based console. Among them, any console is provided with a corresponding survey range, and the survey ranges of each console cover the sea area to be surveyed in any acquisition period.

[0065] By setting up the drifting console and the shore-based console, inspections are carried out separately on the sea surface, avoiding the situation that the oil spill phenomenon cannot be observed due to the oil spill position exceeding the survey radius of the unmanned aerial vehicle. While effectively improving the extensibility of the oil spill survey system, the reliability of the oil spill survey is further improved.

[0066] Particularly, the surface inspection group includes a number of hangars corresponding to each console, and a number of image unmanned aerial vehicle clusters based on each hangar;

[0067] Among them, for a single console, any unmanned aerial vehicle in the corresponding unmanned aerial vehicle cluster starts from the hangar corresponding to this console during a single survey, and returns to the hangar corresponding to this console when completing the corresponding survey route;

[0068] The survey route of the unmanned aerial vehicle is related to the survey range of the console corresponding to this unmanned aerial vehicle, and the image coverage range of its single survey is not less than a preset ratio of the survey range;

[0069] The preset ratio is related to the viewport of the unmanned aerial vehicle.

[0070] By setting the corresponding survey range according to the viewport of the unmanned aerial vehicle, while improving the survey range of the unmanned aerial vehicle, the problem of being unable to observe the sea surface due to the mismatch between the shooting viewport size of the unmanned aerial vehicle and the survey range is avoided, thereby further improving the reliability of the oil spill survey.

[0071] Particularly, the radar unmanned aerial vehicle group includes:

[0072] The flight station set on the shore-based console, and the radar unmanned aerial vehicle cluster composed of a number of radar unmanned aerial vehicles;

[0073] Among them, the number of radar unmanned aerial vehicles is equal to the number of flight stations. When a single radar unmanned aerial vehicle completes the shooting of the layered image, this radar unmanned aerial vehicle enters the flight station with the shortest relative distance to it.

[0074] Specifically, the central server is provided with an oil spill color difference value and an oil spill area threshold. When any adjacent areas of the water surface image in any survey range reach the oil spill color difference value, the central server issues a color difference warning;

[0075] When the water surface area in any acquisition cycle that reaches the oil spill color difference value is not less than the oil spill area threshold, the central server issues a hierarchical warning, determines the corresponding oil spill-like area and the console for the corresponding oil spill-like area;

[0076] Among them, the oil spill color difference value is the gray-scale difference value between the sea surface color and the oil spill color, which is related to the lighting conditions;

[0077] The oil spill area threshold is related to the maximum oil spill range of a single oil spill event in a single acquisition cycle.

[0078] In one embodiment, under strong daylight lighting conditions, the oil spill color difference value set by the central server is 30, and the oil spill area threshold is 500 square meters. During a patrol survey, the water surface image shows that the gray-scale difference value between the sea surface color and the oil spill color in a certain area has reached 35, exceeding the set oil spill color difference value.

[0079] Warning situation:

[0080] Color difference warning: Since the gray-scale difference value of 35 is greater than the oil spill color difference value of 30, the central server issues a color difference warning.

[0081] Hierarchical warning: After further analysis, it is found that the water surface area that reaches the oil spill color difference value is 600 square meters, exceeding the oil spill area threshold of 500 square meters. The central server issues a hierarchical warning, determines that this area is an oil spill-like area, and notifies the corresponding console for further processing.

[0082] In one embodiment, under weak night lighting conditions, the oil spill color difference value set by the central server is 20, and the oil spill area threshold is 300 square meters. During a patrol survey, the water surface image shows that the gray-scale difference value between the sea surface color and the oil spill color in a certain area is 18, not reaching the set oil spill color difference value.

[0083] Warning situation:

[0084] Color difference warning: Since the gray-scale difference value of 18 is less than the oil spill color difference value of 20, the central server does not issue a color difference warning.

[0085] Hierarchical warning: Since no color difference warning is issued, the central server will not perform further hierarchical warning analysis.

[0086] In one embodiment, under moderately cloudy weather conditions with moderate lighting, the oil spill color difference value set by the central server is 25, and the oil spill area threshold is 400 square meters. During a patrol survey, the water surface image shows that the gray-scale difference value between the sea surface color and the oil spill color in a certain area is 28, exceeding the set oil spill color difference value.

[0087] Warning situation:

[0088] Color difference warning: Since the grayscale difference value of 28 is greater than the oil spill color difference value of 25, the central server issues a color difference warning.

[0089] Stratification warning: After further analysis, it is found that the water surface area reaching the oil spill color difference value is 450 square meters, exceeding the oil spill area threshold of 400 square meters. The central server issues a stratification warning, determines that this area is an oil spill-like area, and notifies the corresponding console for further processing.

[0090] It can be understood that the above embodiments only represent a possible situation. In actual applications, corresponding settings should be made according to the actual situation of the ocean surface.

[0091] Please refer to Figure 4 as shown, which is a communication schematic diagram of the oil spill patrol and survey unmanned aerial vehicle and the ground base station in the embodiment of the present invention, including:

[0092] UAV platform payload: The UAV platform payload generally refers to the airframe of the UAV and its support structure, including components such as wings, fuselage, landing gear, and their connection, support, and loading facilities. It not only has to support the flight of the UAV but also carry various payloads required for mission execution, such as sensors, cameras, communication equipment, etc.

[0093] Multispectral camera: A multispectral camera is an advanced imaging device that can simultaneously obtain spectral characteristics and spatial image information. The multispectral camera can not only capture images in the visible light band but also sense spectral ranges invisible to the human eye, such as near-infrared and short-wave infrared. It can detect the oil film area more accurately.

[0094] Micro lidar: It consists of a laser emitter, an optical system, a scanning system, a receiver, and a signal processing unit. The working principle of lidar is based on the Time of Flight (TOF) technology. The lidar system emits laser pulses and records the time difference between the emission of the laser pulse and the reception of the reflected pulse. By knowing the speed of light, the distance that the laser pulse travels to and from the target can be calculated.

[0095] Thermal infrared video capture card: Usually equipped with a dedicated thermal infrared sensor, it can convert thermal radiation into an electrical signal.

[0096] Visible light video capture card: It can convert the analog video signal output by the camera into a digital signal so that the operating system and software of the computer can process it.

[0097] Radar data acquisition card: It can convert the analog or digital signals generated by the radar system into digital data that can be processed by the computer.

[0098] Radar data processor: The radar data processor is a key component in the radar system. It is responsible for processing the raw signals received by the radar to extract important information about the target. It can reduce the impact of sea waves on oil film detection.

[0099] Oil film thickness inversion module: It calculates the thickness of the oil film by detecting the impact of the oil film on specific signals.

[0100] Thermal infrared image, visible light image and radar image fuser: Integrates the image data from different sensors for subsequent analysis and processing.

[0101] Oil spill detection and analysis card: Used to detect and analyze the oil spill information in the fused images.

[0102] Oil spill detection alarm: The oil spill detection alarm is equipped with an electronic display screen and is installed at the server end of the command center. Its function is to display the location of the suspected oil spill on the electronic nautical chart by flashing, and to prompt the area of the suspected oil spill by sound alarm. After the oil spill detection and analysis card in the ground console detects an oil spill on the sea surface, it sends the suspected oil spill analysis results (including the location and area of the suspected oil spill), thermal infrared, visible light and radar fused images, thermal infrared images, visible light images, and radar images to the server end of the command center.

[0103] Oil spill monitoring database: The oil spill monitoring database is deployed at the server end of the command center and is used to store the suspected oil spill analysis results (including the location and area of the suspected oil spill), thermal infrared, visible light and radar fused images, thermal infrared images, visible light images, radar images and alarm time information.

[0104] Specifically, the console corresponding to the oil spill-like area responds to the color difference warning and sets several adjacent consoles as extended consoles;

[0105] Each extended console responds to the color difference warning and controls the corresponding image unmanned cluster to collect the corresponding patrol range;

[0106] The central server responds to the color difference warning and extends the acquisition period until each extended console completes the corresponding water surface image acquisition.

[0107] By observing the color difference of the sea surface and presetting an oil spill area threshold to characterize the degree of oil spill diffusion caused by seawater flow, while effectively improving the timeliness of oil spill patrol, it avoids the situation where when a large oil spill area is detected during detection, it is impossible to accurately determine whether the oil spill event is in the early stage or the diffusion stage, thus further improving the reliability of oil spill patrol.

[0108] Specifically, the central server responds to the hierarchical warning and sets at least one shore-based console closest to the center of the oil spill-like area corresponding to the hierarchical warning as the active console, and,

[0109] Take at least one shore-based console closest to the edge of the distance-based oil spill area as the slave console and issue layered image drawing instructions.

[0110] Specifically, the master console and the slave console respond to the layered image drawing instructions, and respectively control the corresponding radar drones to detect the layered image of the oil spill area along a preset route;

[0111] Among them, the preset route is related to the contour of the oil spill area and the corresponding ocean current direction.

[0112] Specifically, the central server determines several oil spill cores based on the layered image and issues corresponding core patrol signals;

[0113] The layered patrol group responds to the core patrol signal and detects the oil spill core along the preset route;

[0114] Among them, the core patrol signal is sent to each shore-based console;

[0115] The oil spill core is at the bottom layer of the layered image.

[0116] Specifically, when the central server obtains the corresponding layered image of the oil spill core, it generates a corresponding oil spill image;

[0117] Among them, the oil spill image forms a corresponding longitudinal mapping from the layered image and a corresponding water surface mapping from the water surface image.

[0118] By using the layered image as the longitudinal model benchmark and the water surface image as the horizontal model benchmark to model the oil spill event, and storing the completed oil spill model, with this model, a reasonable oil spill source area can be found through model analysis, which can provide model support for subsequent oil spill events, thereby further improving the reliability of oil spill patrol.

[0119] Please refer to Figure 5 As shown in the figure, it is a schematic diagram of the oil spill model of the embodiment of the present invention. In the figure, when the water surface oil spill range 11 in the water surface image 1 exceeds the set range, the central server divides this range into a rectangular area including several consoles, and controls the corresponding radar drone group of the layered patrol group to detect the images of each depth;

[0120] At this time, based on the water surface depth 4, the radar drone divides the first layered depth 41, the second layered depth 42, and the third layered depth 43 corresponding to different radar frequencies, and respectively detects the layered oil spill range 21 corresponding to each layered image 2, and transmits each layered image 2 to the central server;

[0121] At this time, the central server infers a number of smooth fitting curves based on the stratified oil spill ranges 21 corresponding to each stratified image 2 and the surface oil spill range 11 corresponding to the surface image 1, vertically maps the surface oil spill range 11 and each stratified oil spill range 21 to each other, and forms the corresponding inferred oil spill core 31 and the corresponding oil spill target layer 3, and forms the modeling of this oil spill event.

[0122] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0123] The above are only the preferred embodiments of the present invention and are not used to limit the present invention; for those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An oil spill patrol and survey system based on the recognition of fused images of multispectral and lidar, characterized in that, Including: A surface inspection group for observing the water surface, which is equipped with an image drone cluster carrying a multispectral camera. Each cluster collects water surface images using visible light and / or infrared light according to an acquisition period; A number of consoles for receiving the water surface images, which are used to determine the corresponding oil spill-like areas based on the water surface images; A layered inspection group for detecting underwater, which is equipped with a radar drone cluster carrying a lidar, and is used to collect layered images of the oil spill-like areas using the radar; A central server, which is connected to each console, and is used to determine the corresponding oil spill-like core in response to the water surface images and the layered images, and send out corresponding core survey signals; Wherein, the console responds to the core survey signal and controls the corresponding radar drone clusters to take pictures of various oil spill-like areas to generate corresponding oil spill images; The acquisition period corresponds to the longest survey duration of each drone composing the image drone cluster; The central server collects the water surface images, the layered images and the oil spill images and forms an oil spill monitoring database.

2. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 1, wherein The console consists of a drifting console and a shore-based console. Among them, any console has a corresponding survey range, and the survey ranges of each console cover the sea area to be surveyed in any acquisition period.

3. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 2, characterized in that, The surface inspection group includes a number of hangars corresponding to each console, and a number of image drone clusters set based on each hangar; Wherein, for a single console, any drone in the corresponding drone cluster starts from the hangar corresponding to this console during a single survey, and returns to the hangar corresponding to this console when completing the corresponding survey route; The survey route of the drone is related to the survey range of the console corresponding to the drone, and the image coverage range of its single survey is not less than a preset ratio of the survey range; The preset ratio is related to the spatial resolution of the drone.

4. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 2, wherein, The radar drone cluster includes: A flight station set on the shore-based console, and a radar drone cluster composed of a number of radar drones; Wherein, the number of the radar drones is equal to the number of the flight stations. When a single radar drone completes the shooting of the layered image, the radar drone enters the flight station with the shortest relative distance to it.

5. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to any one of claims 3 or 4, characterized in that, The central server is provided with an oil spill color difference value and an oil spill area threshold. When any adjacent areas of the water surface image in any survey range reach the oil spill color difference value, the central server issues a color difference warning; When the water surface area of the water surface image in any acquisition period reaches the oil spill color difference value and is not less than the oil spill area threshold, the central server issues a layered warning, and determines the corresponding oil spill-like area and the console corresponding to the oil spill-like area; Wherein, the oil spill color difference value is the gray difference value between the sea surface color and the oil spill color, and it is related to the lighting conditions; The oil spill area threshold is related to the maximum oil spill range of a single oil spill event in a single acquisition period.

6. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 5, characterized in that, The console corresponding to the oil spill-like area responds to the color difference warning and sets a number of adjacent consoles as extended consoles; Each extended console responds to the color difference warning and controls the corresponding image drone cluster to collect the corresponding survey range; In response to the color difference warning, the central server extends the acquisition period until each extended console completes the corresponding water surface image acquisition.

7. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 5, characterized in that, In response to the layering warning, the central server designates at least one shore-based console closest to the center of the oil spill area corresponding to the layering warning as the active console, and issues a layering image drawing instruction to at least one shore-based console closest to the edge of the oil spill area as the slave console.

8. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 7, characterized in that, The active console and the slave console respond to the layering image drawing instruction and respectively control the corresponding radar drones to detect the layering image of the oil spill area along a preset route; wherein the preset route is related to the contour of the oil spill area and the corresponding ocean current direction.

9. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 8, wherein, The central server determines a number of oil spill cores based on the layering image and issues corresponding core patrol signals; The layering inspection team responds to the core patrol signal and detects the oil spill core along the preset route; wherein the core patrol signal is sent to each shore-based console; The oil spill core is located at the bottom layer of the layering image.

10. The oil spill patrol and survey system based on multi-spectral and lidar fusion image recognition according to claim 9, characterized in that, When the central server obtains the corresponding layering image of the oil spill core, it generates a corresponding oil spill image; wherein the oil spill image forms a corresponding longitudinal mapping from the layering image and a corresponding water surface mapping from the water surface image.

Images