Underwater image acquisition device and method for acquiring underwater images

Abstract

An underwater imaging device (1) comprising an LED illumination unit (5), a gated image acquisition unit (4), and a control unit (11) is described. The control unit is configured to determine the ambient illuminance in the image acquisition area of ​​the gated image acquisition unit (4) without illumination from the LED illumination unit (5) and to determine the particle concentration in the image acquisition area of ​​the gated image acquisition unit (4). Image acquisition is performed in operating modes selectable depending on the determined illuminance and particle concentration. a) Image capture without additional illumination if the determined ambient illuminance exceeds a predetermined illuminance threshold, regardless of whether the determined particle concentration exceeds or falls below a predetermined turbidity threshold; b) Additionally, a continuously illuminated image capture if the measured illuminance falls below the specified illumination threshold and the measured particle concentration falls below the specified turbidity threshold; and c) Pulsed multiple illumination with synchronized pulsed gated image acquisition when the determined ambient illuminance falls below the specified illumination threshold and the particle concentration exceeds the specified turbidity threshold.

Description

[0001] The invention relates to an underwater image recording device comprising: - an LED lighting unit; - a gated image acquisition unit that can be exposed multiple times in sequential exposure time windows for image acquisition and is set up for image acquisition synchronized with the LED illumination unit, and - a control unit connected to the LED lighting unit and the gated image acquisition unit and configured to control the LED lighting unit and the gated image acquisition unit to capture underwater images with the gated image acquisition unit.

[0002] The invention further relates to a method for taking underwater pictures with such an underwater image recording device.

[0003] J. Schmidt; E. Peters; M. Stephan; O. Zielinski: “Point spread function measurements for underwater imaging: an analysis of wavelength-specific behavior for image deconvolution.” in: Optics express (2025) 33 (2), pp. 1772-1790. DOI: 10.1364 / OE.541377 show that turbidity and scattering behavior exhibit a high spectral dependence in underwater images.

[0004] The standard equipment for optical camera systems on autonomous underwater vehicles (AUVs) or remotely operated vehicles (ROVs) currently typically includes sensitive CMOS cameras and LED spotlights. If there are increased requirements, such as improved visibility or 3D imaging, the platforms can be expanded with additional systems like laser scanners or range gating, which involve special, proprietary combinations of active illumination and optical receivers, as described, for example, in A. Driewer, I. Abrosimov, J. Alexander, M. Benger, M. O'Farrell, KH Haugholt, C. Softley, JT Thielemann, J. Thorstensen, and C. Yates: “UTOFIA: an underwater time-of-flight image acquisition system”, in: Proc. SPIE 10434, Electro-Optical Remote Sensing XI, 1043404 (5 October 2017). The UTOFIA system described in https: / / doi.org / 10.1117 / 12.2277944.

[0005] Range gating is an established technique for improving underwater visibility. Previous systems are based on nanosecond pulsed laser illuminators combined with image intensifiers, single-photon avalanche diodes (SPADs), or specialized CMOS cameras capable of exposure windows of a few nanoseconds. Two main techniques have become established: single-shot and multiple exposure. In the former, a single, powerful laser pulse (typical pulse energies in the µJ range per pulse) illuminates the scene, and the frame rate is equal to the pulse frequency. In the latter, multiple laser pulses are emitted at a higher pulse frequency, and an image is generated by integrating many individual exposures. This requires a lower pulse energy from the light source but also a camera capable of higher exposure frequencies.

[0006] Comparing the typical view ranges of passive optical systems, as described by J. Ronald V. Zaneveld; W. Scott Pegau (2003): Robust underwater visibility parameter. In: Optics express 11 (23), pp. 2997-3009, with the view ranges of range-gating systems, as described by Maccarone, Aurora; McCarthy, Aongus; Ren, Ximing; Warburton, Ryan E.; Wallace, Andy M.; Moffat, James et al. (2015): Underwater depth imaging using timecorrelated single-photon counting. In: Optics express 23 (26), pp. 33911-33926. As described in DOI: 10.1364 / OE.23.033911, this shows that range gating systems are able to generate a view range gain of about a factor of 2-3 for intensity images under the corresponding conditions (increased turbidity and low ambient light), as well as a significantly greater view range gain for pure 3D depth information.

[0007] The light source of the range-gated system always competes with the ambient light, and in the case of underwater use, unlike above water, spectral filtering is less effective because typically the most ambient light is available at the emission wavelength of the spotlight, as this is designed for the wavelength of maximum transmission (typically around 532 nm in the green range).

[0008] CN 1 04 364 673 A discloses a gated imaging system that can be used on vehicle platforms or underwater platforms. The gated imaging system combines at least two images (frames) that may have been acquired with or without system illumination. Image acquisition can be performed using different image sensors (e.g., thermal sensors, CMOS, CCD, etc.). A passive image, which does not use a light source, and an active image can be used. Active images can have a time sequence with an illumination pulse followed by a delay in sensor exposure, as defined for continuous gating. The illumination pulses can have different durations and be followed by delays. Sensors with different exposure durations in different time sequences can read out up to N cycles per sensor image. The fusion of passive and active images or frames can produce a fused image or frame.To deliver a frame with improved image quality. For example, in gated cameras, each pixel can differ from the others in the continuous gating timing (light accumulation), or each array (i.e., multiple pixels or pixel clusters) can differ from the others. This method allows each gated pixel (or gated array) to accumulate different "slices" by controlling the triggering mechanism of each pixel or pixel cluster.

[0009] CN 1 16 106 928 A discloses a self-adaptive underwater full-gating imaging method that sets the control parameters of the high-repetition-rate underwater laser rangefinder imaging system in full-gating operating mode based on the high underwater repetition rate after the control parameters have been set. The laser rangefinder imaging system performs adaptive underwater full-gating imaging; the set control parameters include...This process supports the calibration of the attenuation coefficient of the working water range, the determination of the working distance range and optical system parameters, the calculation of the minimum achievable spacer disk length and the determination of the initial number of spacer disks and the delay time of each spacer disk, the determination of the gate width of each spacer disk according to the working distance range, the determination of the cumulative pulse count and the gain of each spacer disk based on the control parameter setting database, and the determination of the actual number of spacer disks and the corresponding delay time of each spacer disk. This method for optimal gate distance adjustment leads to enhanced functionality and an improvement in the imaging effect.

[0010] Range gating systems are optimized for pulsed gated operation. They increase underwater visibility by suppressing backscatter from particles in the water column and by providing 3D information. However, this system only offers real added value in underwater photography in a few very specific situations, namely when there is a high particle concentration in the water and little or no ambient light is available. This is the main reason why gated systems are rarely used.

[0011] The frame rate of gated viewing systems is significantly lower compared to continuous-exposure camera systems because the gate (the illuminated area in the image) only covers a limited distance range. Therefore, multiple images must be captured with different gate positions to cover the entire distance range of a scene. This reduces the frame rate considerably, depending on the gate length and maximum distance (typically by a factor of > 5 to 10).

[0012] The power consumption of range-gating systems is significantly higher than that of passive camera systems due to the required pulsed illuminator. This is a disadvantage on autonomous, robotic platforms. Furthermore, space on such systems is always limited, so careful consideration must be given to whether a gated viewing system, which only demonstrates its strengths under very specific environmental conditions, should be included.

[0013] Existing gated viewing systems are based on laser illumination, which entails operational limitations, particularly in the presence of divers. Furthermore, laser illumination increases system complexity, as it requires a hardware shutdown. This necessitates, for example, an additional control line in the tether (a flexible connection element to the vehicle, such as a line, wire, or cable) when operating on an ROV. This additional line transmits the control signal to switch the laser on and off.

[0014] Existing gated viewing systems emit at a single wavelength, which does not allow for color imaging. Furthermore, they typically have very small emitting surfaces, leading to problems with shadows cast by larger particles directly in front of the system.

[0015] The object of the present invention is to provide an improved underwater image acquisition device and an improved method for acquiring underwater images with such an underwater image acquisition device.

[0016] The problem is solved by the underwater image acquisition device having the features of claim 1 and by the method having the features of claim 15. Advantageous embodiments are described in the dependent claims.

[0017] It is proposed that the control unit be configured to determine the ambient illuminance in the image acquisition area of ​​the gated image acquisition unit without illumination by the LED lighting unit, and to determine the particle concentration in the image acquisition area of ​​the gated image acquisition unit, and to acquire images in the operating modes selectable depending on the determined illuminance and particle concentration: a) image acquisition with the gated image acquisition unit without illumination with the LED illumination unit, if the determined illuminance exceeds a predetermined illumination threshold, regardless of whether the determined particle concentration exceeds or falls below a predetermined turbidity threshold; b) image acquisition with the gated image acquisition unit during illumination with the LED lighting, if the measured illuminance falls below the specified illumination threshold and the measured particle concentration falls below the specified turbidity threshold; and c) image acquisition with the pulsed gated image acquisition unit with multiple pulsed illumination by the LED illumination unit over the image acquisition time for an image with the gated image acquisition unit controlled synchronously to the pulsed illumination by the LED illumination unit, wherein the gated image acquisition unit performs an exposure of the gated image acquisition unit delayed after an illumination pulse of the LED illumination unit, if the determined illuminance falls below the specified illumination threshold and the particle concentration exceeds the specified turbidity threshold.

[0018] The invention exploits the fact that gated-viewing cameras typically allow for excellent monochrome image capture in low light conditions. They can also be used as passive cameras in ambient light or as continuous-exposure systems using standard LED spotlights on AUVs or ROVs. The combined system according to the invention, through its control in different operating modes and the LED illumination unit configured for continuous and pulsed operation, achieves the advantages of both image acquisition systems via nanosecond pulse operation and continuous illumination of an LED illumination unit.

[0019] The control unit can be configured to detect illuminance and / or particle concentration from images acquired by the gated image acquisition unit. This utilizes the gated image acquisition unit and the images it captures as a sensor for detecting illuminance and / or particle concentration. This can be achieved through image analysis, for example, by evaluating the brightness of the image pixels, such as determining the brightness distribution. Particle concentration can be determined by analyzing the image pixels, which, based on their size, indicate the presence of particles.

[0020] Alternatively or additionally, at least one environmental sensor can be provided, which is configured to detect the illuminance and / or particle concentration in the water and is connected to the control unit for determining the illuminance and / or particle concentration. Environmental sensors enable the detection of illuminance and / or particle concentration independently of image acquisition. However, environmental sensors can also be combined with the detection of illuminance and / or particle distribution from the captured images. This provides a redundant measurement system, which, for example, allows measurement errors to be identified and, if necessary, compensated for by comparing the measurement results obtained from both methods. The measurement results obtained from image analysis and the environmental sensor can be combined, for example, by averaging (mean, median, etc.).), a very reliable and accurate determination of the illuminance and / or particle concentration can be carried out.

[0021] The at least one environmental sensor used as an alternative or in addition to image evaluation can, for example, be selected from the group of radiometers, turbidity sensors, backscatter sensors, lux meters and particle concentration sensors.

[0022] The control unit can be configured to capture an image in operating mode c) in a recording area directly in front of the gated image acquisition unit, without any objects present, and to determine the particle concentration from the backscatter caused by the particles in the water, which are depicted in the image captured by the gated image acquisition unit. By capturing the image in the area directly in front of the gated image acquisition unit, such as directly in front of the lens of a gated camera, the particle concentration can be determined without being influenced by interfering background objects in the captured image. Since the LED illumination unit (spotlight) is positioned next to the gated camera, and the camera's viewing angle and the spotlight's illumination angle only overlap at a certain distance, there is a minimum distance for the recording area.This depends on the viewing and illumination angle and the distance between the spotlight and the camera. The ideal position for the recording area is therefore within the water column, so that only the backscatter from particles, without any other objects, contributes to the image, and as close as possible to the image capture unit, whereby the minimum distance is limited by the overlap of the spotlight illumination angle and the camera viewing angle.

[0023] The degree of overlap between the headlight illumination area and the camera's field of view can also be used as a measure of particle concentration. If the recording area is located at a point where the overlap of both angular ranges is neither 0% nor close to 100%, an increase in particle concentration, due to increased scattering, also leads to an increase in the degree of overlap, as the light is scattered laterally.

[0024] The control unit can be configured to regularly determine the illuminance and check whether the determined illuminance exceeds the predefined illuminance threshold. It then switches to operating mode a) as soon as the determined illuminance exceeds the predefined illuminance threshold. The illuminance is thus used as a primary control parameter, triggering a switch to operating module a) independently of the particle distribution as soon as sufficient illuminance is present.

[0025] The control unit can be configured to regularly determine the particle concentration and switch to operating mode c) when the measured particle concentration exceeds the predefined turbidity threshold and the measured illuminance falls below the predefined illuminance threshold. A switch to operating mode b) can occur when the particle concentration falls below the predefined turbidity threshold and the measured illuminance falls below the predefined illuminance threshold. The particle concentration is thus used as a second control parameter, enabling switching to operating mode c) with pulsed gated operation of the LED illumination unit and gated image acquisition unit, independent of the particle distribution, as soon as sufficient illuminance is present.

[0026] The LED lighting unit can have color groups of LEDs, with each color group emitting light at different wavelengths. This allows for image capture with illumination in specific, selected wavelength ranges, such as red, green, and blue, or optionally with broadband illumination, such as white light.

[0027] The control unit can be set up to successively capture images by illuminating one color group of the LED lighting unit at a time, such as red, green and blue.

[0028] The control unit can be set up for image capture with simultaneous illumination by all color groups of the LED lighting unit, such as with white light.

[0029] The LED lighting unit can have multiple LED drivers, each controlling a single LED or a group of LEDs. The LED drivers can be controlled by a common trigger signal from the control unit and can include a trigger delay compensation unit for synchronously adjusting the pulse emission times of all LEDs in the LED lighting unit.

[0030] The control unit can be configured for continuous illumination with the LED lighting unit in operating mode b) for at least the duration of the gated image acquisition unit's image capture time for a single image of the surroundings. This allows image acquisition in CW mode (CW = "Continuous Wave" or PWM = "Pulse Width Modulation"). Pulse-width modulated (PWM) operation can be used to control the power (i.e., dimming) of the LEDs and effectively results in continuous illumination for image acquisition, since the pulse lengths are typically in the micro- or millisecond range and the pulse blanking has no significant effect on the exposure during the image capture time. PWM operation for image acquisition differs from pulsed gating operation, which uses pulse lengths in the ineffective nanosecond range and significantly longer pulse blanking times.

[0031] The underwater image acquisition unit can be configured to adjust the delay time between an illumination pulse from the LED lighting unit and the exposure of the gated image acquisition unit, depending on a predefined acquisition distance to the gated image acquisition unit. By selecting the delay based on the propagation speed of the emitted light pulse and the desired acquisition distance of the reflected light pulse, an image can be captured within the range of the acquisition distance.

[0032] The LED lighting unit can be configured for illumination with pulse frequencies exceeding 100 kHz and optical pulse slopes of less than 5 nanoseconds. Such steep and short light pulses enable imaging of underwater areas at selected, precisely illuminated detection distances of just a few meters.

[0033] The procedure for taking underwater pictures with an underwater image acquisition device described above includes a choice of one of the following operating modes: a) for image acquisition with the gated image acquisition unit without illumination of the LED illumination unit, if the determined ambient illuminance in the image acquisition area of ​​the gated image acquisition unit exceeds a predetermined illumination threshold, regardless of whether the particle concentration exceeds or falls below a predetermined turbidity threshold; b) for image acquisition with the gated image acquisition unit during continuous (i.e., CW or PWM) illumination with the LED lighting, when the measured ambient illuminance in the image acquisition area of ​​the gated image acquisition unit falls below the specified illumination threshold and the particle concentration falls below the specified turbidity threshold; and c) for image acquisition with the pulsed gated image acquisition unit with multiple pulsed illuminations over the image acquisition time for an image with the gated image acquisition unit controlled synchronously to the illumination unit, wherein the gated image acquisition unit performs an exposure delayed after an illumination pulse of the LED illumination unit, if the determined illuminance of the environment in the image acquisition area of ​​the gated image acquisition unit falls below the specified illumination threshold and the particle concentration exceeds the specified turbidity threshold.

[0034] The invention is explained in more detail below with reference to an exemplary embodiment and the accompanying drawings. These show: Fig. 1 - Sketch of an autonomous underwater vehicle with underwater image recording device; Fig.2 - Block diagram of an underwater image recording device with three different colored LED lights; Fig. 3 - Block diagram of an underwater image recording device with a multi-colored LED lighting unit; Fig. 4 - Pulse diagram of the control pulses for the separately pulsed different colored LED lights and the pulsed gated image acquisition unit over the exposure time of three differently colored illuminated images; Fig. 5 - Pulse diagram of the control pulses for the jointly pulsed multicolored LED lights and the pulsed gated image acquisition unit over the exposure time of three white light illuminated images; Fig.6 - Pulse diagram of the control pulses for the separately controlled different colored LED lights and the gated image acquisition unit controlled continuously for image acquisition over the exposure time of three differently colored illuminated images; Fig. 7 - Pulse diagram of the control pulses for the multicolored LED lights controlled together in continuous operation and the gated image acquisition unit controlled in continuous operation for image acquisition over the exposure time of three white light illuminated images.

[0035] Fig. Figure 1 shows a sketch of a remotely controlled underwater vehicle 1 operating below the water surface W and equipped with an underwater image acquisition device 2. Alternatively, the device described below can also be used for an autonomously operating underwater vehicle 1.

[0036] The underwater vehicle 1 has a propulsion system 3, such as several thrusters as shown in the diagram. The underwater imaging device 2 has a gated imaging unit 4, i.e., a gated camera, and an LED lighting unit 5. The LED lighting unit 5 has several differently colored LEDs 6 to illuminate the area around the gated imaging unit 4 with light of different wavelengths or, optionally, with white light.

[0037] The underwater vehicle 1 also has a local control unit 7, which can be connected, for example, via a control and supply cable 8 to a ship on the water's surface and a central control unit 17 located there. However, an independent power supply for the underwater vehicle 1 and a wireless communication link are also conceivable.

[0038] This provides a combination of a gated camera (gated image acquisition unit 4) with a special LED spotlight (LED lighting unit 5) and a trigger unit, which is operated according to the method for automatic switching of operating modes already described above. The complete system can be operated in the following operating modes: a) Passive with ambient light. For this purpose, a highly sensitive image intensifier tube or SPAD camera can be provided, which allows images to be taken even at greater depths in low ambient light. b) Continuous LED illumination in the absence of ambient light and with few scattering particles. The emission spectrum can be adjusted depending on the turbidity. c) Gated operation in the absence of ambient light and with many scattering objects, so that active illumination leads to backscattering problems and oversaturation of the sensor or obscuration of the area behind it.

[0039] Preferably, up to four operating modes are provided, encompassing a) RGB-gated operation (capture of multiple images illuminated in different color spectra with pulsed illumination and exposure) b) Monochrome gated operation (illumination with white light or a single color with pulsed illumination and exposure) c) RGB-CW operation (capture of multiple images illuminated in different color spectra with continuous illumination and exposure) d) Monochrome CW operation (capture of one or more images with continuous illumination with white light and continuous exposure)

[0040] When using RGB-gated cameras, RGB operation no longer requires capturing multiple individual color images in sequence; instead, illumination with white light is sufficient. This would allow operating modes b) and d) to directly capture RGB images, rendering operating modes a) and c) unnecessary for this type of camera.

[0041] An automated change occurs depending on the prevailing light intensity and particle concentration in the water.

[0042] For example, three different operating modes are provided: 1. Passive Operation:

[0043] These are image captures where only the camera is activated and the spotlight(s) are / are switched off. This is the preferred starting operating mode and is the method of choice wherever sufficient ambient light is available. The frame rate is significantly increased compared to gated viewing systems, while power consumption is considerably reduced without spotlight operation, thus extending the operating time of autonomous and battery-powered systems. 2. The active non-gated operation:

[0044] The spotlights emit light for the duration of the camera exposure time (or continuously or via PWM modulation). For clear waters without ambient light, this can multiply the light output compared to gated viewing. LEDs are operated efficiently in continuous-wave (CW) or PWM-modulated mode, bypassing the short-pulse generator for nanosecond pulses.

[0045] This reduces power consumption compared to pulsed gated viewing systems. 3. The gated viewing operation:

[0046] In gated mode, a short-pulse generator and a delay unit of a trigger controller are used (the LEDs emit pulses lasting a few nanoseconds). The efficiency of the LEDs is reduced in favor of improved visibility by suppressing backscatter from particles in turbid waters with little or no ambient light.

[0047] Fig.Figure 2 shows a block diagram of an underwater imaging device 2 with three differently colored LED lights 6a, 6b, 6c. LED light 6a has a plurality of red LEDs (LED(R)) in the red color spectrum (in the range of 600 to 780 nm, preferably due to the increasing absorption underwater with increasing wavelength in the near red region from about 600 nm to 660 nm). LED light 6b has a plurality of green LEDs (LED(G)) in the green color spectrum (in the range of 520 to 580 nm, preferably about 530 to 550 nm, as the transmission of water is particularly good in this region). LED light 6c has a plurality of blue LEDs (LED(B)) in the blue color spectrum (in the range of 400 to 500 nm, preferably about 450 nm, as absorption increases again towards the short-wave UV).

[0048] The LED lights 6a, 6b, 6c are each controlled by a trigger signal T1a, T1b, T1c from a short-pulse control unit 9. The short-pulse control unit 9 can optionally be part of the control unit 11 or of LED drivers 15. This allows illumination with individual light pulses of only one selected color (R, G, B) to be achieved by pulsing only one selected LED light 6a, 6b, 6c. It is also possible to control several different colored LED lights 6a, 6b, 6c simultaneously to generate light pulses of mixed wavelengths for illumination. By controlling the three LED lights 6a, 6b, 6c simultaneously, illumination with white light can be achieved.

[0049] The short-pulse control unit 9 is also configured to optionally achieve longer illumination, at least for the entire exposure time of an image capture. This is required for CW mode.

[0050] The short-pulse control unit 9 is further configured to generate a trigger signal T2 synchronous with the pulsed trigger signals T1a, T1b, T1c, which is used to control the gated image acquisition unit 4. The short-pulse control unit 9 is configured to initiate illumination after a pulse in the trigger signal T1a, T1b, T1c and a predefined delay time t. d to generate a trigger pulse T2 to initiate an exposure of an image sensor 10 of the gated image acquisition unit 4. The delay time t d It can also be zero and should be selected variably depending on the desired recording distance. This can be automated by a control unit 11, which is connected to the short-pulse control unit 9.

[0051] The control unit 11 is connected to the gated image acquisition unit 4 in order to control it in continuous operation via a control signal C2 and to achieve continuous exposure for image acquisition in an exposure time during non-pulsed continuous illumination with the LED illumination unit 5 or with sufficient illuminance without illumination with the LED illumination unit 5.

[0052] The gated image acquisition unit 4 can further include a lens 12 positioned in front of the image sensor 10, i.e., the camera. The lens 12 can have a fixed focus or a focus that can be adjusted actuatorically with the control unit 11.

[0053] The adaptive headlight-camera system shown preferably uses three differently colored nanosecond pulsed LED headlights (red, green, blue), with each headlight emitting only in one of the three spectral ranges. These can be mounted arbitrarily on the support system. Thanks to the gated capability, the operator is not limited to a maximum baseline between the camera and the headlight, although this is still recommended to minimize contrast loss due to backscattering in the image.

[0054] The short-pulse control unit 9, which in turn can be controlled via the control unit 11 (e.g., a processor) or directly implemented in an FPGA, microcontroller, etc., provides the necessary trigger signals T1a, T2a, T3a, T2, and the control signal C2 for the various operating modes. It is capable of providing both CW signals and trigger signals in the range of a few nanoseconds with frequencies up to the MHz range.

[0055] The exposure of a gate-capable camera (gated image acquisition unit 4) is also triggered externally via the short pulse control unit 9, while the camera data is transferred to the control unit 11, such as the processor (e.g. a miniaturized computer) or to the interface of the autonomous underwater vehicle and the operator.

[0056] The adaptive switching between the different operating modes (passive, CW and gated) can be implemented automatically, based on the recorded images and / or based on sensor signals from at least one environmental sensor 14, and depends on the detected illuminance and particle concentration.

[0057] Fig. Figure 3 shows a block diagram of an underwater image acquisition device 2 with a multi-colored LED illumination unit 5. The LED illumination unit 5 can be controlled by the short-pulse control unit 9 with a trigger signal T1 for short-term, multiple pulsed illumination for the synchronized pulsed gated operation of the gated image acquisition unit 4. The short-pulse control unit 9 is also configured for continuous-wave (CW) operation to achieve continuous illumination by the LED illumination unit 5 via the control line C1 during exposure for at least one image acquisition.

[0058] In this alternative version, the LEDs of different colors (again, red, green, and blue as examples) are integrated into a single spotlight. This means only one spotlight is needed for RGB operation. However, this requires an additional control cable, and the spotlight would need to be three times larger than the one shown in [reference missing]. Fig. The second variant shown is intended to achieve a comparable light output.

[0059] Basically, this variant offers the same functionalities as the one in Fig. Variant 2 shown.

[0060] The additional control line C1 handles the switching between the different colors in the headlight (LED lighting unit 5). This is done, for example, by switching field-effect transistors (FETs) in the LED driver circuit.

[0061] For all variants of LED lighting units 5 especially from Fig. 2 and Fig.Section 3 states that for adaptive, environmental-condition-dependent automatic switching between passive operating mode, CW operating mode, and gated operating mode, a sensor unit with environmental sensors (brightness sensor, turbidity sensor) can be used as an alternative or additional method to extraction from the captured images. Automatic switching therefore requires either external environmental sensors, image analysis, or both.

[0062] Fig.Figure 4 shows a pulse diagram of the control pulses T1a (here R), T1b (here G), T1c (here B), T2 (here K) for the separately pulsed, differently colored LED lights 6a, 6b, 6c for red light R, green light G, and blue light B, and the pulsed gated image acquisition unit 4 over the exposure time t in seconds of three differently illuminated images P1(R), P2(G), P3(B). The first image P1(R) was illuminated with red light pulses in a first image acquisition period and exposed with a number n of exposure pulses 1, 2, ..., n-1, n. The second image P2(G) was illuminated with green light pulses in a subsequent second image acquisition period and exposed with several exposure pulses 1, 2, ..., n-1, n. The third image P3(B) was illuminated with blue light pulses in a subsequent third image acquisition period and exposed with several exposure pulses 1, 2, ..., n-1, n.

[0063] The diagram for RGB-gated operation in Fig. Figure 4 shows how a color gated image can be generated with a monochromatic gated camera by serially controlling red, green, and blue LEDs in pulsed operation. The pulse frequency f puls The red LEDs are activated first for a period of a few nanoseconds. The gated camera also exposes for a comparable duration in the nanosecond range after the delay time t. d , which determines the gate position (typically between 0 and 10 m distance). This is the typical gated viewing principle. After a number n of pulse emissions and exposures (typically in the range of several thousand exposures), the image is read out (in the case of SPAD cameras, the binary images are stacked until a desired bit depth is reached). Then, only the green LEDs are activated, followed by the blue LEDs. Finally, an RGB-gated image can be recombined from the three channels.

[0064] Alternatively, RGB-gated images could be generated directly using gated-capable RGB cameras (e.g., an RGB SPAD camera) by simultaneously pulsing all three spotlight colors.

[0065] Fig. Figure 5 shows a pulse diagram of the control pulses T1a (here R), T1b (here G), T1c (here B), T2 (here K) for the jointly pulsed different colored LED lights 6a, 6b, 6c and the pulsed gated image acquisition unit 4 over the time t of the exposure K of three white light illuminated images P1, P2, P3.

[0066] In the depicted monochrome-gated operation, the light output can be multiplied by simultaneously controlling all LEDs; here, with three LED lights (6a, 6b, 6c), it is approximately tripled. This significantly increases the frame rate compared to RGB-gated operation. Alternatively, only a single color can be controlled, for example, for contrast reasons.

[0067] Fig. Figure 6 shows a pulse diagram of the control pulses T1a (here R), T1b (here G), T1c (here B), T2 (here K) for the differently colored LED lights 6a, 6b, 6c, and the gated image acquisition unit 4, which are controlled separately in continuous CW operation for image acquisition, over the exposure time of three differently illuminated images P1(R-CW), P2(G-CW), and P3(B-CW). Continuous CW operation also functionally refers to classic PWM operation with pulses in the microsecond or millisecond range, in contrast to the nanosecond range of gated operation.

[0068] For the RGB-CW operation shown, the red, green, and blue LEDs are sequentially activated for the duration of the camera exposure time. This results in significantly higher light output than in pulsed operation, as the efficiency of the LEDs and the driver circuitry increases significantly (e.g., switching losses are reduced).

[0069] Fig. Figure 7 shows a pulse diagram of the control pulses T1a (here R), T1b (here G), T1c (here B), T2 (here K) for the different colored LED lights 6a, 6b, 6c controlled together in continuous operation CW and the gated image acquisition unit 4 controlled in continuous operation CW for an image acquisition over the exposure time of three images P1(CW), P2(CW), P3(CW) illuminated with white light (i.e. simultaneously R+G+B).

[0070] For the depicted monochrome CW operation, the three LED colors (e.g., red, green, and blue) can be operated simultaneously as a continuous white light source to further increase the light output. Here too, it is possible to select to control only a single color, for example, for contrast reasons.

[0071] The various trigger options allow the nanosecond pulsed LED spotlight to implement all operating modes relevant to a wide range of environmental conditions.

[0072] The automatic switching between the in the Fig. The operating modes outlined in points 5 to 7 are dependent on environmental conditions, as shown in the following table: Plenty of ambient light available No / hardly any ambient light available Low particle concentration (turbidity) Passive operation CW operation High particle concentration (turbidity) Passive operation Gated operation

[0073] A decision-making process for the operating mode can either a) on additional environmental sensors (environmental sensor 14) such as radiometers and turbidity sensors b) or are based on regular image analysis.

[0074] It is advantageous to capture an image in gated mode with the gate positioned directly in front of the gated image acquisition device 4, especially the lens 12 (e.g., 10 to 50 cm in front of the lens 12). This ensures with a high degree of probability that no objects are present in this area, only the water column. Accordingly, the backscatter in the image originates solely from the particles in the water.

[0075] The following table illustrates how such a regular image analysis can be performed. It is preferably always started in passive mode. Passive operation (start,) CW operation Gated operation Action / Query 1 Regular evaluation to determine if the image brightness is high enough at a sufficient frame rate. Regularly check whether the particle concentration is too high. Regularly check whether the particle concentration is too high for CW operation (insert a CW image into every n images). Yes, continue in passive mode. No, switch to CW mode. Yes, change to gated operation. No, continue in CW operation. Yes, switch to CW operation. No, continue in gated operation. Action / Query 2 Regularly check whether there is sufficient ambient light for passive operation (insert a passive recording every n recordings). Regularly check whether there is sufficient ambient light for passive operation (insert a passive image into all recordings). Yes, switch to passive operation. No, continue in CW operation. Yes, switch to passive operation. No, continue in gated operation.

[0076] The aforementioned gated cameras, which are known per se, are in particular cameras that can achieve exposure time windows in the nanosecond range or shorter at repetition rates of several hundred kHz up to the MHz range. These typically include image intensifier cameras, SPAD cameras, and special CMOS cameras.

[0077] The LED headlight unit for an adaptive camera system with gated capability preferably has the following additional specific features: a) In pulsed operation (pulse lengths of a few nanoseconds and edge slope of less than 5 ns), the spotlight achieves a comparable average light output to existing underwater gated viewing systems. This output is typically in the milliwatt to watt range. This has previously been achieved using lasers (typically frequency-doubled Nd:YAG lasers at 532 nm). By using light-emitting diodes (LEDs), eye safety-related operational restrictions, e.g., when used in the presence of divers, are avoided, while the larger emitting surface reduces the shadowing effect caused by large particles directly in front of the emitter. b) Pulse energy levels in the range of a few nJ can be achieved with pulsed operation (6 ns pulse length). For power comparable to laser lights, this requires a significant increase in the pulse frequency to more than 100 kHz, up to the double-digit MHz range. An array of LEDs is proposed to achieve the required total power. Because the LED lighting unit has an LED array, pulses in the µJ range can be generated by driving a group of LEDs in parallel. Typically, about 10 to 100 LEDs per color group are needed. The electronic control (pulse generators) must ensure synchronous and homogeneous operation (all LEDs should therefore have similar pulse shapes and low jitter to avoid broadening the overall pulse). c) In the LED array, each LED or a small group of LEDs should have its own LED driver circuit. d) Each LED driver in the array can additionally have its own trigger delay compensation to reduce the difference in the pulse emission time of all LEDs to less than 1 ns. This is helpful to keep the slope of the overall pulse in the range of a few nanoseconds. e) The LEDs should preferably emit in the visible part of the spectrum. f) The LEDs can be switched between pulsed ns operation and continuous (CW) illumination. Continuous illumination in this context refers both to actual continuous illumination and to pulses longer than 10 microseconds. The switching can be based on external control signals or an electronic circuit for detecting the trigger pulse or light pulse duration. g) A combination of blue, green, and red LEDs is advantageous for the entire unit. These can either be implemented in a single headlight ( Fig.3) or be implemented by three different headlights ( Fig. 2) For capturing images illuminated in only one color spectrum, it is possible to control each color individually, but also to have all LEDs emit simultaneously for greater light availability. Depending on the ambient conditions and application, this allows for a variable trade-off between increased light availability and a reduction in image blurring and oversaturation caused by scattering. Since haze and scattering behavior exhibit a high spectral dependence, emission can be selectively directed to the spectral range of lower scattering.

[0078] To control the gated camera and the LED headlight unit, a suitable electronic circuit is provided as a trigger and control unit. This circuit preferably has the following properties and features for the overall operation of the system: a) At least three spotlight outputs for controlling the individual colors and at least one camera output for controlling the exposure time. b) The spotlight outputs are capable of generating trigger pulses in the nanosecond range via a short-pulse generator, with edge slew rates of less than 1 ns at pulse frequencies up to the tens of MHz. They can be individually controlled, activated, and deactivated. The edges exhibit jitter in the nanosecond range or less. However, the same output can also be used, for example, by bypassing the short-pulse generator, to output a continuous signal for continuous illumination or a pulse signal (optionally PWM modulated) with pulse durations greater than 1 microsecond. c) The camera output has the same characteristics as the spotlight outputs, but additionally features a programmable delay circuit that can generate delays between the spotlight outputs and the camera output of at least 0 to 150 ns in steps smaller than 5 ns. The jitter compared to the undelayed signal is in the range of a few nanoseconds or less. These times are derived from the typical operating range of up to 10 m. The distance R traveled by the light to the sensor corresponds to half the delay t multiplied by the speed of light c in water (refractive index n = 1.33). Therefore, a delay time of 150 ns corresponds to a maximum operating distance of 17 meters in gated operation.

[0079] The underwater image acquisition device according to the invention can be advantageously used, for example, in the following areas: - Maritime system equipment (equipment for ROVs, AUVs and divers). - Offshore service sector (primarily in the area of ​​oil / gas / wind power / pipeline / power line / data cable inspection tasks). - Research equipment. - Equipping government organizations, especially those with security responsibilities. - Fisheries and fish farming, as well as environmental organizations, e.g. to find lost fishing equipment such as nets. - Ordnance disposal (contaminated sites). - Underwater archaeology. - Wreck inspection. - Maintenance and servicing of coastal structures (bridges, quays, port terminals). Reference symbol list 1 underwater vehicle 2 Underwater imaging device 3 Drive 4 Gated image acquisition unit 5 LED lighting unit 6 LED lights 6a, 6b, 6c LED lights 7 local control unit 8 control and power cables 9 Short pulse control unit 10 Image sensor 11 Control unit 12 Lens 14 Environmental sensor 15 LED drivers 16 Trigger runtime compensation unit 17 central control unit C1 control line C2 control signal LED(R) red light-emitting diodes LED(G) green light-emitting diodes LED(B) blue light-emitting diodes T1a, T2a, T3a trigger signals T2 trigger signal / trigger pulse t d Delay time W Water surface

Claims

[1] Underwater image acquisition device (1) with: - an LED lighting unit (5); - a gated image acquisition unit (4) which can be exposed multiple times in sequential exposure time windows for image acquisition and is set up for image acquisition synchronized with the LED illumination unit (5), and - a control unit (11) connected to the LED lighting unit (5) and the gated image acquisition unit (4) and configured to control the LED lighting unit (5) and the gated image acquisition unit (4) to take underwater images with the gated image acquisition unit (4), characterized by, that the control unit (11) is used to determine the ambient light intensity in the image acquisition area of ​​the gated image acquisition unit (4) without illumination by the LED lighting unit (5) and to determine the particle concentration in the image acquisition area of ​​the gated image acquisition unit (4) and to acquire images in the operating modes selectable depending on the determined light intensity and particle concentration: a) the image acquisition with the gated image acquisition unit (4) without illumination with the LED illumination unit (5), if the determined illuminance exceeds a predetermined illumination threshold, regardless of whether the determined particle concentration exceeds or falls below a predetermined turbidity threshold; b) the image acquisition with the gated image acquisition unit (4) during illumination with the LED lighting, if the determined illuminance falls below the specified illumination threshold and the determined particle concentration falls below the specified turbidity threshold; and c) the image acquisition with the pulsed gated image acquisition unit (4) with multiple pulsed illumination by the LED illumination unit (5) over the image acquisition time for an image with the gated image acquisition unit (4) controlled synchronously to the pulsed illumination with the LED illumination unit (5), wherein the gated image acquisition unit (4) performs an exposure of the gated image acquisition unit (4) with a delay after an illumination pulse of the LED illumination unit (5) if the determined illuminance falls below the specified illumination threshold and the particle concentration exceeds the specified turbidity threshold. [2] Underwater image acquisition device (1) according to claim 1, characterized by , that the control unit (11) is set up to detect the illuminance and / or the particle concentration from the images acquired with the gated image acquisition unit (4). [3] Underwater image acquisition device (1) according to claim 1 or 2, characterized by , that the control unit (11) is set up to take an image in operating mode c) in a recording area located immediately in front of the gated image acquisition unit (4) without recorded objects and to determine the particle concentration from the backscattering caused by the particles in the water, which are shown in the image taken with the gated image acquisition unit (4). [4] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by at least one environmental sensor (14) which is set up to detect the illuminance and / or the particle concentration in the water and is connected to the control unit (11) for determining the illuminance and / or the particle concentration. [5] Underwater image acquisition device (1) according to claim 4, characterized by, that at least one environmental sensor (14) is selected from the group consisting of radiometer, turbidity sensor, backscatter sensor, particle concentration sensor and lux meter. [6] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by , that the control unit (11) is set up to regularly determine the illuminance and check whether the determined illuminance exceeds the specified illuminance threshold and is set up to switch to operating mode a) as soon as the determined illuminance exceeds the specified illuminance threshold. [7] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by, that the control unit (11) is set up to regularly determine the particle concentration and to switch to operating mode c) when the determined particle concentration exceeds the specified turbidity threshold and the determined illuminance falls below the specified illuminance threshold, and to switch to operating mode b) when the particle concentration falls below the specified turbidity threshold and the determined illuminance falls below the specified illuminance threshold. [8] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by , that the LED lighting unit (5) has color groups of light-emitting diodes (LED(R), LED(G), LED(B)), wherein the color groups of the light-emitting diodes (LED(R), LED(G), LED(B)) each emit light with different wavelengths. [9] Underwater image acquisition device (1) according to claim 8, characterized by, that the control unit (11) is set up for successive recording of images, each time with illumination of a color group of the LED lighting unit (5). [10] Underwater image acquisition device (1) according to claim 8 or 9, characterized by , that the control unit (11) is set up for image acquisition with illumination by all color groups of the LED lighting unit (5) simultaneously. [11] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by , that the LED lighting unit (5) has several LED drivers (15) for controlling one light-emitting diode (LED) or a group of light-emitting diodes (LEDs), wherein the LED drivers (15) are controlled with a common trigger signal (T1a, T1b, T1c, C1) of the control unit (11) and have a trigger time-of-flight compensation unit (16) for synchronously adjusting the pulse emission times of all light-emitting diodes (LEDs) of the LED lighting unit (5). [12] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by , that the control unit (11) is set up for continuous illumination (CW / PWM) with the LED lighting unit (5) in operating mode b) for at least the image acquisition time of the gated image acquisition unit (4) for an image of the environment. [13] Underwater image acquisition device (1) according to any one of the preceding claims, characterized by , that the gated image acquisition unit (4) is capable of exposure depending on a predetermined acquisition distance of an image to the underwater image acquisition device (1) and an adjustable delay time (t). d ) is set up after an illumination pulse of the LED illumination unit (5) and the exposure of the gated image acquisition unit (4). [14] Underwater image acquisition device (1) according to any one of the preceding claims characterized by, that the LED lighting unit (5) is set up for illumination with pulse frequencies of more than 100 kHz and edge steepness of the optical pulses of less than 5 nanoseconds. [15] Method for taking underwater pictures with an underwater image acquisition device (1) according to any one of claims 1 to 14, characterized by Optional selection of an operating mode: a) for image acquisition with the gated image acquisition unit (4) without illumination of the LED illumination unit (5), if the determined ambient illuminance in the image acquisition area of ​​the gated image acquisition unit (4) exceeds a predetermined illumination threshold, regardless of whether the particle concentration exceeds or falls below a predetermined turbidity threshold; b) for image acquisition with the gated image acquisition unit (4) during illumination with the LED illumination unit (5), if the determined illuminance falls below the specified illumination threshold and the particle concentration falls below the specified turbidity threshold; and c) for image acquisition with the pulsed gated image acquisition unit (4) with multiple pulsed illumination over the image acquisition time for an image with the gated image acquisition unit (4) controlled synchronously to the pulsed illumination with the LED illumination unit (5), wherein the gated image acquisition unit (4) performs an exposure of the gated image acquisition unit (4) after a delay following an illumination pulse of the LED illumination unit (5) if the determined illuminance falls below the specified illumination threshold and the particle concentration exceeds the specified turbidity threshold.

Images