Optical mine detection in shallow water

Description

[0001] The invention relates to a method for optical mine detection in shallow water, particularly in shallow water areas.

[0002] Sea mines pose a threat not only to shipping, but also to numerous military and civilian activities along coasts and at sea. Historically, and currently, various types of sea mines have been deployed at varying water depths. Clearing them is a time-consuming, costly, and, not least, risky undertaking, requiring up to 200 times more effort than the initial placement of the mines. Currently, various methods exist for detecting sea mines, although a combination of several procedures is usually necessary to derive the most reliable indicators. New methods that enable comprehensive and largely automated mine detection can make an important contribution here, or complement and support existing methods.

[0003] Over the past decade, unmanned aerial vehicles (UAVs), often known by the more common term "drone," have played an increasingly important role in civil and military operations. Surveillance and reconnaissance missions, as well as the assessment of battle damage, are just two examples of a wide range of applications in which increasingly powerful UAVs are of great benefit. In this context, researchers often emphasize the fact that UAVs significantly increase the safety and efficiency of missions and operations. However, "UAV" merely refers to the platform itself, which can only provide truly relevant information in combination with suitable sensor technology such as imaging and / or spectral measurement instruments.

[0004] When discussing mine detection, a clear distinction must be made between sea mines and land mines. Detection of the latter is currently primarily carried out using detection dogs, mine clearance vehicles, or metal detectors.

[0005] The detection of sea mines is significantly more complex in comparison, so that the methods used on land are either not applicable or only of limited use. So far, acoustic methods (sonar) have been used primarily, with the corresponding sensors being carried by various platforms, for example and in particular minehunters, remotely operated or autonomous unmanned surface vehicles (USVs), autonomous underwater vehicles (AUVs), or remotely operated vehicles (ROVs).

[0006] Recently, however, UAVs have also been increasingly used as delivery systems for land and sea mine detection. These are typically unmanned helicopters that tow an underwater sonar and, similar to underwater and surface systems, follow pre-programmed geodetically planned routes. In this combination, the UAV is used less as an actual delivery system and more as a propulsion or traction device, while the sensors themselves continue to operate underwater.

[0007] Sea mines are explosive devices of various shapes, ranging in size from approximately 50 cm to almost 3 m and containing from approximately 50 kg to over 1000 kg of explosive. Aside from their size and explosive force, mines can be further categorized according to their type of use, their position in the water column, and the method of activation or detonation. They can be placed in the water in different ways. There are bottom mines, anchor mines, and floating mines. Anchor mines can be deep or high-lying, meaning they are close to the bottom or near the surface. Drifting mines either float freely on or below the water surface, following the influences of water currents and wind, or are tethered to a fixed location. Although free-floating mines have been banned under international law for over 70 years, some countries still use them for defensive or offensive strategies in conflicts.

[0008] Bottom mines remain on the seabed after deployment. Because they are intended to remain in place, they are loaded with larger explosive charges (approximately 1,000 kg). They are often partially or completely covered with sediment, sometimes with the specific intention of making them more difficult to detect. To ensure that the mine's explosive force can penetrate the ship's hull near the water surface, bottom mines are usually deployed in relatively shallow water (less than 50 m).

[0009] Anchor mines, on the other hand, must be buoyant to enable them to be deployed at a specific depth. A weight (anchor chair) on the seabed serves as a fastening, while the mine itself floats and, depending on the length of the anchor chain, can "stand" at the desired depth. Because anchor mines are closer to the target and to ensure the required buoyancy, anchor mines are typically filled with significantly less explosive than bottom mines.

[0010] Sonar-based methods for detecting sea mines in greater water depths are now quite sophisticated and also enable large area coverage per unit of time. Mines near the water surface are difficult to detect with these methods. Acoustic methods are particularly difficult to apply in shallow water due to ambient noise and reflections. Furthermore, they are usually not accessible to conventional minehunting vessels. Therefore, other approaches must be used for these tasks.

[0011] EP 148 704 B1 discloses a surveillance system using a drone to determine the location of a target.

[0012] A method for detecting drift mines is known from DE 196 33 008 A1.

[0013] EP 0 858 051 B1 discloses a method for segmenting a digital image.

[0014] From DE 10 2004 062 124 B3 a device and a method for tracking an underwater vehicle are known.

[0015] EP 0 148 704 B1 discloses a surveillance system using a drone to determine the location of a target.

[0016] From US 2019 / 011921 A1 a system and a method for the interactive control and monitoring of UAVs is known.

[0017] Bathymetric techniques using satellite images are known from WO 2015 / 013543 A1.

[0018] US 2019 / 014760 A1 discloses a method and devices for object detection in unmanned aircraft.

[0019] A spherical infrared transmitter is known from US 2018 / 343400 A1.

[0020] US 10 377 485 B2 discloses a system and a method for the automatic examination of surfaces.

[0021] US 2015 / 049588 A1 discloses a method for detecting naval mines and a naval mine detection system.

[0022] An optical imaging system and a method therefor are known from US 2018 / 270474 A1.

[0023] From US 2019 / 266712 A1 a system and a method for imaging underwater environments using liquid lenses is known.

[0024] The object of the invention is to provide a method with which mines in shallow water, especially in shallow water, can be located quickly and reliably.

[0025] This object is achieved by the method having the features specified in claim 1. Advantageous further developments emerge from the subclaims, the following description and the drawings.

[0026] The method according to the invention for detecting mines in shallow water below the water surface comprises the following steps, where shallow water is the area with a water depth of up to 50 m: a) Flying over a suspected region with an aircraft, wherein the aircraft selected is an aircraft with at least one first optical detection device, b) Capturing an image series, wherein the image series comprises 3 to 100 individual images, wherein at least one pixel is captured with spectral resolution, c) Carrying out an interference factor analysis of the image series to eliminate surface effects, d) Carrying out a contrast adjustment of the individual images, e) Carrying out image segmentation, f) Selecting image segments with a predetermined size, in particular from 0.3 to 3 m, and a predetermined shape, in particular round or rectangular, g) Identifying the position of a potential mine, wherein the spectral information of the sunlight and the spectral information of the reflected light are recorded, wherein between step d) and step e) the water depth is calculated from the spectral information.

[0027] Shallow water depth within the meaning of the invention is the area with a water depth of up to 50 m, particularly preferably up to 30 m. The invention particularly preferably relates to a method for detecting mines in shallow water, where the bottom lies up to 50 m, particularly preferably up to 30 m, below the water surface. In this case, the shallow water area extends to the land and thus, for example, on a beach, with decreasing water depth, practically to depth 0, which would correspond to the surf zone. The method therefore allows not only the detection of mines that are on the surface or at least partially penetrate the water surface, but above all also the detection of mines that are completely covered with water, in particular bottom mines and anchor mines.

[0028] In step a), in a first preferred embodiment, an area-resolving first optical detection device is selected, for example a CMOS or CCD sensor.

[0029] In step a), in a second preferred embodiment, an area-resolving first optical detection device is selected, for example a CMOS or CCD sensor, wherein the first optical detection device has a slit-shaped inlet and a dispersive or diffractive element behind the slit-shaped inlet, wherein the refractive or diffractive effect is arranged perpendicular to the longitudinal direction of the slit. The advantage of this embodiment is one-dimensional detection, which, together with the movement of the aircraft, which preferably occurs perpendicular to the longitudinal direction of the slit, enables area detection. At the same time, the spectral information in the second dimension can be detected on the sensor. The diffractive element is particularly preferably a grating.

[0030] In step a), in a third preferred embodiment, a first optical detection device with both areal and spectral resolution is selected. An example of this is a Cubert S 185 spectrometer with a resolution of 50 × 50 pixels and a spectral resolution of 450 nm to 1000 nm.

[0031] In step a), in a fourth preferred embodiment, an aircraft with a first optical detection device for area detection and a second optical detection device is selected, with a spectrometer being selected as the second optical detection device. Particularly preferably, the second optical detection device is arranged such that the spectral information of an image area of ​​the first optical detection device can be detected.

[0032] In a further embodiment of the invention, 15 to 40 individual images, preferably 20 to 30 individual images, are captured in step b). The individual images are preferably captured within a period of 10 to 30 seconds.

[0033] Surface effects that should be removed during the disturbance factor analysis include reflections or effects caused by surf, such as foam and spray.

[0034] For example, and in particular, the noise factor analysis of the image series to eliminate surface effects is carried out by comparing the individual images in the image series. Since the water surface is constantly moving during the recording time, the (extremely bright) areas with sunlight reflections shift in each individual image of the series. By selecting the darkest pixel in the image series, it is then possible to minimize or even completely eliminate the reflections. This multi-temporal approach with subsequent minimum filtering makes the final image virtually reflection-free. Similar methods can be used, for example, to remove noise objects that move through the image during the recording.

[0035] Since this procedure frequently reduces contrast, it is advantageous to perform step d) after step c). Theoretically, these two steps could also be performed in reverse order. Alternatively, an initial contrast adjustment of the individual images can be performed before step c) and a contrast adjustment of the image in step d).

[0036] For example, in step d), the contrast adjustment is performed using a minimum-maximum stretch. This can be performed across the entire image or across individual image regions. If the contrast adjustment is performed across image regions, image regions are preferably selected such that the image region is at least a factor of two larger than the largest longitudinal extent of an expected mine. Those skilled in image processing are familiar with other contrast adjustment methods that can also be used.

[0037] In image segmentation in step e), colors, textures, and / or contrasts are grouped into regions with similar properties and distinguished from neighboring regions with different properties. The Felzenszwalb-Huttenlocher algorithm is just one example. In this edge-oriented method, neighboring pixels are examined and compared for their characteristics such as color or brightness. Similar pixels are then grouped into regions, with the choice of parameters for this algorithm significantly influencing the result.

[0038] In step f), the identified image segments are compared with predefined shapes and sizes that would correspond to mines. For example, anti-invasion mines would be round and approximately 1 m in diameter. Ground mines are often cylindrical, with a diameter of 50 cm to 60 cm, particularly 533 mm, and a length of 1 m to 2 m. These would thus appear as rectangles with corresponding edge lengths. Shape and size are thus correlated by the existing mine shape.

[0039] In step g), a potential mine is identified using the captured optical image in the area of ​​the identified image segment. The image data can also be used to determine the exact position of the mine and forward it to a mine countermeasures unit.

[0040] According to the invention, in step b) at least one pixel is captured with spectral resolution. For example, at least 20 spectrally different pieces of information, preferably at least 50 spectrally different pieces of information are captured. Spectrally different pieces of information in this sense are pieces of information that represent intensities in different regions of the spectrum. In the simplest case, different filters can capture different pieces of spectrally different information. The simplest example of this are sensors in commercially available cameras that capture a red, a green, and a blue region of the spectrum, with corresponding filters arranged in front of corresponding pixels of the sensor. To achieve a higher resolution, theoretically several filters with several subpixels can be used, but this is technically very complex. A diffractive element, for example a grating, is preferably used for this purpose.In simple spectrometers, the resolution is partially limited due to their cost-effective design, so that only spectral information with a resolution of 5 nm to 10 nm can be resolved. In this case, spectral information is understood to be the intensity in separate spectral ranges. The spectral information is recorded, for example, and preferably in the range from 200 nm to 10 µm, preferably from 200 nm to 3 µm, particularly preferably from 300 nm to 1.1 µm. The spectral information is preferably recorded at least in the range from 400 nm to 700 nm. The maximum spectral resolution is preferably 0.5 nm, and the maximum spectral resolution is particularly preferably 1 nm.

[0041] In a further embodiment of the invention, the aircraft has a monochromatic light source, in particular a laser. The light source projects a light beam at least temporarily into the area detected by the first optical detection device. The backscattered radiation is used for spectral calibration of the first optical detection device.

[0042] According to the invention, the water depth is calculated from the spectral information between step d) and step e). An important basic property of light is that different wavelengths also have different penetration depths into the water body. The principle applies that light with shorter wavelengths (e.g. blue) can penetrate deeper into the water column than light with longer wavelengths (e.g. red). An exception to this are bodies of water rich in absorbers such as yellow matter, in which green light (medium wavelength) has a greater penetration depth than blue light. Optical sensors measure the reflection from the water, i.e. the light that is reflected back to the sensor from the water surface, the particles within the water column or the lake / seabed.The water depth from which the signal originates is approximately half the penetration depth, as the light has to travel twice through the water column (first downwards, then back up to the sensor). The actual penetration depth of sunlight depends primarily on the water constituents, i.e. the absorbers (e.g., yellow matter) and scatterers (e.g., sediment particles or phytoplankton) contained in the water. As a result, it is subject to spatial and temporal dynamics and can vary greatly from region to region. With increasing concentrations of absorbing and scattering water constituents, the depth of visibility decreases. For the detection of the lake bottom or objects underwater, this means that the use of optical methods is limited to half the maximum penetration depth of sunlight.The water depth can thus be calculated based on the scattering and absorption properties of the water, in particular with the help of a physical model for the creation and inversion of water spectra.

[0043] Determining the water depth has another positive effect. Before a landing, the area in question can not only be examined for anti-invasion mines, but the current depth profile can also be recorded. This can help with concrete planning of a landing, particularly in determining positions where vehicles can land. Especially in shallow water, maps can quickly become outdated due to sediment movement. Furthermore, obstacles may have been introduced undetected into the shallow water, complicating or even preventing a landing.

[0044] In a further embodiment of the invention, in step e) the image segmentation is carried out exclusively on the basis of the water depth or on the basis of a combination of the water depth and the optical information.

[0045] According to the invention, the spectral information of the sunlight and the spectral information of the reflected light are detected. For this purpose, an aircraft is preferably selected in step a) that does not have a further optical detection direction, which is preferably oriented in the opposite direction to the detection direction of the first optical detection device. In this way, it is easily possible to calculate spectral differences that arise, for example, due to different cloud cover or other weather influences, as well as times of day and the like.

[0046] In a further embodiment of the invention, the aircraft detects the geoposition at the time of image acquisition and assigns the geoposition to the image. In particular, the geoposition can be detected using a GNNS satellite navigation system (GPS, Galileo, GLONASS, Beidou).

[0047] In a further embodiment of the invention, the optical detection direction is aligned at an angle of less than 20° to the beam direction of direct sunlight. This preferably places the sun "behind" the first optical detection device. Artifacts and stray light are thus minimized.

[0048] In a further embodiment of the invention, a polarization filter is rotatably mounted in front of the first optical detection device. The polarization filter is rotated (particularly for a constant flight direction) such that the light detected by the first optical detection device is minimized. In this position, reflection on the water surface should be minimized, thus enabling optimal detection of objects below the water surface.

[0049] In a further embodiment of the invention, the method is carried out in the Northern Hemisphere between October 1 and April 30, and between 10:00 a.m. and 3:00 p.m. Optimal sampling is when the sun is at its highest, i.e., during midday. On the other hand, sampling in summer is often hampered by algal blooms, so autumn, winter, and spring are preferred.

[0050] In a further embodiment of the invention, an unmanned aircraft is selected as the aircraft. Particularly preferably, this is an autonomous unmanned aircraft.

[0051] In a further embodiment of the invention, a helicopter is selected as the aircraft, wherein the helicopter additionally has an unmanned underwater vehicle. After identifying a mine, the following steps are performed: u) Deployment of the unmanned underwater vehicle by the helicopter, v) Approaching the mine by the unmanned underwater vehicle, w) Combating the mine by the unmanned underwater vehicle.

[0052] In a further embodiment of the invention, an unmanned aerial vehicle (UAV) is selected as the aircraft, wherein the UAV additionally comprises an unmanned underwater vehicle. After identifying a mine, the following steps are performed: u') Deployment of the unmanned underwater vehicle by the UAV, v') Approaching the mine by the unmanned underwater vehicle, w') Combating the mine by the unmanned underwater vehicle.

[0053] The method according to the invention is explained in more detail below using an embodiment shown in the drawings. Fig. 1 Schematic representation of optical mine detection in shallow water areas Fig. 2 Penetration depth as a function of wavelength Fig. 3 Flow diagram

[0054] In Fig. 1 A coastal area is depicted in a highly schematic manner. An aircraft 10 flies over shallow water 20. Shallow water 20 is characterized by the distance between the water surface 22 and the seabed 24 being less than 50 m.

[0055] The aircraft 10 has a first optical detection device 40, for example, a camera for the area-wide detection of the shallow water area. Furthermore, the aircraft 10 has a second optical detection device 50, for example, a spectrometer, which detects the spectral information, in particular the image center of the image captured by the first optical detection device 40. To exclude influences, such as weather, the aircraft 10 has a third optical detection device 70, which is directed upwards and can thus detect the spectral characteristics of the incident sunlight.

[0056] Furthermore, the aircraft 10 has a GNNS sensor 80. This allows the geopositions to be directly assigned to each captured image and each captured spectrum.

[0057] Furthermore, the watercraft 10 has an unmanned underwater vehicle 60. The unmanned underwater vehicle 60 is an unmanned underwater vehicle 60 for combating mines, for example a sea fox.

[0058] If a mine 30 is detected and located by the first optical detection device 40 and / or the second optical detection device 50, the aircraft 10 can deploy the unmanned underwater vehicle 60. The unmanned underwater vehicle 60 can then further investigate the mine 30 and, if necessary, combat it.

[0059] Fig. 2 shows roughly and schematically how the penetration depth of light of different wavelengths behaves. A maximum penetration depth is reached in the blue-green range and, depending on the incident radiation intensity (especially solar brightness) and detector sensitivity, lies in the range of 20 to 50 m, provided that no attenuating effects, particularly turbidity due to suspended matter, absorption by algae, or the like, negatively influence the light propagation underwater. In the worst case, for example during a severe algal bloom, an effective value can drop below 1 m. In principle, however, it is clear that the depth of an object can be determined from the spectral shift.

[0060] In Fig. 3 the process is shown in a flow chart. a) Flying over a suspected region with an aircraft, wherein the aircraft selected is an aircraft with at least one first optical detection device, b) Capturing an image series, wherein the image series has 3 to 100 individual images, c) Carrying out an interference factor analysis of the image series to eliminate surface effects, d) Carrying out a contrast adjustment of the individual images, e) Carrying out image segmentation, f) Selecting image segments with a predetermined size, in particular from 0.3 to 3 m, and a predetermined shape, in particular round or rectangular, g) Identifying the position of a potential mine, u) Lowering the unmanned underwater vehicle by the helicopter, v) Approaching the mine by the unmanned underwater vehicle, w) Combating the mine by the unmanned underwater vehicle. Reference symbol

[0061] 10Aircraft 20Shallow water 22Water surface 24Seabed 30Mine 40First optical detection device 50Second optical detection device 60Unmanned underwater vehicle 70Third optical detection device 80GNNS sensor

Claims

1. A method for detecting mines (30) in shallow water depth below the water surface, wherein shallow water depth is the area with a water depth of up to 50 metres, the method comprising the following steps: a) Flying over a suspected region with an aircraft (10), wherein the aircraft (10) selected is an aircraft (10) with at least one first optical detection device (40), b) Capturing an image series, wherein the image series comprises 3 to 100 individual images, wherein at least one pixel is captured with spectral resolution, c) Carrying out an interference factor analysis of the image series to eliminate surface effects, d) Perform a contrast adjustment of the individual images, e) Carrying out an image segmentation, f) Selection of image segments with a predefined size and shape, g) Identify the position of a potential mine (30), characterised in that the spectral information of the sunlight and the spectral information of the reflected light is recorded, the water depth being calculated from the spectral information between step d) and step e).

2. Method according to claim 1, characterised in that 15 to 40 individual images are captured in step b).

3. Method according to one of the preceding claims, characterised in that in step e) the image segmentation is carried out exclusively on the basis of the water depth or from a combination of the water depth and the optical information.

4. Method according to one of the preceding claims, characterised in that the aircraft (10) detects the geoposition at the time of image acquisition and assigns the geoposition to the image.

5. Method according to one of the preceding claims, characterised in that the optical detection direction is aligned at an angle of less than 20° to the beam direction of the direct sunlight.

6. Method according to one of the preceding claims, characterised in that the method is carried out in the northern hemisphere between 1 October and 30 April and between 10:00 and 15:00.

7. Method according to one of the preceding claims, characterised in that an unmanned aerial vehicle (10) is selected as the aircraft (10).

8. A method according to any one of claims 1 to 7, characterised in that a helicopter is selected as the aircraft (10), the helicopter additionally comprising an unmanned underwater vehicle (60), wherein after identification of a mine (30) the following steps are carried out: u) Deployment of the unmanned underwater vehicle (60) by the helicopter, v) Approach of the mine (30) by the unmanned underwater vehicle (60), w) Fighting the mine (30) with the unmanned underwater vehicle (60).