Active illumination image capture device for low ambient light intensity

Abstract

This invention relates to an active illumination image capturing device for low ambient light intensity, used to capture images of target objects within a predetermined distance in low ambient light intensity environments with clutter interference. The device includes: a high-frequency pulsed light source, whose emitted light beam can illuminate the predetermined distance; an optical image capture unit having a light receiver and an electronic shutter with a frequency higher than a multiple of 1 / 15 second, thereby controlling the exposure time and outputting an image data signal of one frame; an image processor for receiving the image data signal; a drive signal generator for controlling the illumination time and synchronously outputting a control signal to the image processor; and the image processor calculating the illumination time based on the control signal and adding a clutter removal time, used to exclude frames corresponding to the illumination time and clutter removal time from the received frame images and then superimpose them.

Description

Technical Field

[0001] The present invention provides an image capturing device, and more particularly an active illumination image capturing device for use in low ambient light intensity. Prior Technology

[0002] Traditional cars rely on the driver's vision to observe road conditions and control the vehicle. Newer vehicles have head-up displays or even autonomous vehicles. Various unmanned vehicles are also increasingly relying on external image acquisition equipment for forward observation. Whether it is human vision or images or information about the external environment provided by equipment, determining the distance of people, deer or other obstacles ahead is undoubtedly based on optical principles.

[0003] However, the presence of external debris such as raindrops, fog, insects, and silt in the water can interfere with visual or imaging system judgments. Since image data judgment primarily relies on light illuminating the object being observed, and the light entering the eye or camera through diffuse reflection from the object's surface, the intensity of diffused light decreases inversely with the square of the distance. Raindrops, fog, insects, or silt in the water, being closer to the observer or camera, cause diffuse reflection with a stronger beam than distant objects like people, deer, fallen trees, chariots, or underwater rocks. This can lead to objects that need to be observed at a distance being obscured by the diffuse reflection from these objects, preventing clear observation. Alternatively, it can cause sensors and autopilot systems to misinterpret the presence of obstacles and overreact, affecting the smoothness and safety of vehicle operation. When there is abundant natural light or bright streetlights in the external environment, the ambient light intensity is high. Even if there are raindrops, fog, insects or sandstorms in front, the objects to be observed behind can generally still be barely observed or images captured for accurate interpretation. However, problems are more likely to occur when the ambient light intensity is low.

[0004] For ease of explanation, we will use, for example, the lens of a camera and the pixels of the image sensor as the calculation standard. If we use a lens with a half angle of view of 15 degrees, according to the trigonometric function tan(15°)=0.2679, at a distance of 500m, the radius of the lens field of view is 500m * 0.2679=134m, and the diameter of the field of view is 268m. Consider a sensor size of 1 / 2.8" (approximately 6.46 mm diagonally), exemplified by 1920 cells in width and 1080 cells in height. Each cell senses and transmits a signal, representing one pixel in the image. In this case, each pixel at a distance of 500m would cover an area of ​​268m / 1920 = 13.95cm. This means that at a distance of 500m from the camera, an object smaller than 13.95cm would not occupy any pixel space in the image sensor and would not be accurately sensed and recorded. Following a similar triangle scaling principle, at a distance of 50m from the camera, an object of 1.395cm would still be needed to occupy a complete pixel.

[0005] Considering the actual size of raindrops, which ranges from 0.2-0.5 mm in diameter for drizzle to 5-7 mm for torrential rain, especially when the diameter is greater than 7 mm, the air resistance during the fall will cause it to break into smaller raindrops. Therefore, in meteorology, optical measurement, or the design of raindrop detection instruments, a raindrop diameter of 1-2 mm is often used as a common reference value. In other words, even considering the process of raindrops falling and forming rain streaks, the impact on actual photography will occur within 50 meters of the camera. Raindrops beyond this distance will not affect the sensing of other objects to be observed in the background.

[0006] On the other hand, if considering unmanned vehicles such as drones and unmanned surface vessels (USVs) for detecting enemy drones, tanks, warships, or submarines at a greater distance, the lens must be changed to a telephoto lens, as shown in Figure 1, covering only a half-angle of, for example, 2 degrees. In this case, each pixel at a distance of 500m from the camera corresponds to an actual observation size of approximately 17.53m / 1920 = 0.91cm. That is, raindrops or silt at a distance of 500m from the camera are generally much smaller than the size covered by one pixel. Considering that the terminal velocity of raindrops falling through the atmosphere is approximately 2-10m / s, the size of rain streaks at a frame rate of 15-30fps can be estimated. As shown in Figure 2, at a distance of 500m, even by the most stringent standard, small raindrops (Φ...) (2mm) will not obstruct the image (occlusion rate) 5%); medium and large raindrops (Φ2-4mm) can be completely ignored at a distance of 1000m without affecting image quality.

[0007] Current technology typically uses headlights or laser light sources to supplement illumination in dimly lit environments, enhancing the diffuse reflection of the object and making it easier to identify. However, these supplemental lights can also illuminate nearby raindrops, fog, insects, or silt, making it difficult to accurately identify objects in the background. For distant objects with weak diffuse reflection, another common solution is continuous exposure, accumulating diffuse light data over a longer period and repeatedly overlaying it onto the same position in the image to slightly enhance the image of the object. However, during long exposures, objects at close and medium distances often exhibit ghosting due to rapid movement, meaning their images appear to sweep across the screen, causing blurring. These problems are particularly severe in low-light environments such as at night or underwater.

[0008] The main problem this invention aims to solve is how to provide accurate, noise-free external environmental image information in situations with low ambient light intensity and the presence of interfering clutter, thereby increasing the safety of car driving, self-driving, submarine or drone operation, while avoiding sensor overreaction to clutter that would make handling difficult.

[0009] The following detailed description of the features and advantages of the present invention is sufficient to enable anyone skilled in the art to understand the technical content of the present invention and implement it accordingly. Based on the content disclosed in this specification, the scope of the patent application and the drawings, anyone skilled in the art can easily understand the relevant objectives and advantages of the present invention. Summary of the Invention

[0010] One objective of this invention is to provide an active supplemental lighting image capturing device with low ambient light intensity, which uses periodic instantaneous supplemental lighting to eliminate images of nearby clutter by means of time difference, thereby improving image quality.

[0011] Another objective of this invention is to provide an active supplemental lighting image acquisition device with low ambient light intensity, which uses periodic instantaneous supplemental lighting to enhance the image of distant objects by superimposing multiple frames of images over a long period of time, thereby greatly improving the recognition accuracy.

[0012] Another objective of this invention is to provide an active supplemental lighting image capturing device with low ambient light intensity, which reduces image superposition of frame frames by determining reliable image blocks, thereby making the image of moving objects clear.

[0013] To achieve the above objectives, the present invention provides an active illumination image capturing device for low ambient light intensity, used to capture images of target objects within a predetermined distance in a low ambient light intensity environment with clutter interference, wherein the clutter has a predetermined size. The active illumination image capturing device includes a high-frequency pulsed light source, the emitted light beam of which can illuminate the predetermined distance; an optical image capture device with an electronic shutter, having a light receiver and an electronic shutter with a frequency higher than 1 / 15 second by a multiple of that, thereby controlling each exposure period of the light receiver, and outputting a corresponding image after each exposure period. The image data signal of the frame and the aforementioned optical receiver have a predetermined number of pixels; an image processor for receiving the aforementioned image data signal of the aforementioned optical image capturer; a drive signal generator for controlling the emission period of the aforementioned high-frequency pulse light source and synchronously outputting a control signal for controlling the emission of the aforementioned high-frequency pulse light source to the aforementioned image processor; and the aforementioned image processor calculates the aforementioned emission period based on the aforementioned control signal and adds a debris removal period to exclude the frame corresponding to the aforementioned emission period and the aforementioned debris removal period from the received aforementioned frame and then superimpose them.

[0014] By employing the active supplemental lighting image acquisition device described above, this invention can accurately capture images in environments with insufficient ambient light and where general lighting is completely obscured by debris. It utilizes periodic, instantaneous supplemental lighting to synchronously and precisely select image capture times and overlay selected frames. By using the time difference, it eliminates image interference from nearby debris, allowing for accurate capture of the image of the object to be observed. In particular, during the process, by identifying reliable image blocks within the image, sufficiently clear parts of the image do not require excessive overlay, while parts of the image where the object is too far away and diffuse light is insufficient can be overlaid for a longer period, resulting in a sufficiently recognizable image. Therefore, whether it is a manually driven vehicle, an autonomous vehicle, a drone, or an unmanned vessel, it can obtain more accurate images of the external environment in environments with extremely low ambient light and debris interference, enabling drivers and autonomous driving systems to identify objects without being disturbed or obstructed by raindrops, insects, or mud. Simple Explanation of the Diagram

[0015] Figure 1 is a schematic diagram showing the size of the image covered by a telephoto lens camera at different distances.

[0016] Figure 2 shows the occlusion rate of raindrops of different sizes falling at different distances.

[0017] Figure 3 is a side view of a car, illustrating the spatial relationship between the vehicle and nearby objects, medium distance objects, and distant objects to be observed, according to a first preferred embodiment of the active supplementary lighting image capturing device with low ambient light intensity of the present invention.

[0018] Figure 4 is an image capture timing diagram of the embodiment in Figure 3, illustrating the time sequence relationship between the illumination period, the debris removal period, the exposure period, and the frame cycle.

[0019] Figure 5 is a block diagram of the image capturing device of the embodiment in Figure 3, illustrating the main structures.

[0020] Figure 6 is a flowchart of the image acquisition process in the embodiment of Figure 3.

[0021] Figure 7 is a side view of the second preferred embodiment of the active supplementary lighting image capturing device with low ambient light intensity of the present invention applied to a submarine.

[0022] Figure 8 is a side view of the third preferred embodiment of the active supplementary lighting image capturing device with low ambient light intensity of the present invention applied to a drone, illustrating the spatial relationship between the drone and nearby clutter, medium distance and long distance objects to be observed.

[0023] Figure 9 is an image capture timing diagram of the embodiment in Figure 8, illustrating the time sequence relationship between the illumination period, the debris removal period, the exposure period, and the frame cycle. Implementation

[0024] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

[0025] The structures, proportions, and sizes illustrated in the accompanying drawings are solely for illustrative purposes and to aid those skilled in the art in understanding and reading the invention. They are not intended to limit the scope of the invention and therefore have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, provided they do not affect the effectiveness or purpose of the invention, should still fall within the scope of the technical content disclosed herein. Furthermore, terms such as "one," "two," and "above" used in this specification are merely for clarity and not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention's implementation.

[0026] As shown in Figures 3 to 5, a first preferred embodiment of the low ambient light intensity active illumination image capturing device of the present invention involves installing the aforementioned device in a car, with raindrops as the debris 9 in this example, wherein the predetermined size 90 of the aforementioned debris 9, i.e., the raindrop, is set to 7 mm. In this example, the light receiver 20 is exemplified by a sensing element cell of 1920×1080, i.e., a predetermined number of pixels 200 per frame.

[0027] In this example, taking nighttime and heavy rain as an example, considering that the image size formed by the aforementioned debris 9, at the light receiver 20 in the optical image capture device 2, can cover at least half the area of ​​a single pixel 200, the distance to the aforementioned optical image capture device 2 is approximately 60 meters. Therefore, 60 meters is taken as the debris interference distance 620 in this example, and debris beyond this debris interference distance 620 will not cause image interference. Considering that the speed of light in a vacuum is approximately 300,000 kilometers per second, slightly slower in air but still approximately 300,000 kilometers per second (rounded up), and reduced to 225,000 kilometers per second in water, the speed of light is generally considered to be constant unless it enters or exits different media or the same medium but the density changes significantly.

[0028] In this case, a pulsed light source is used as the main supplementary lighting source. The time it takes for the emitted light beam to reflect off a near-end obstacle and be imaged by a light receiver such as a CMOS is approximately 0.00000004 seconds (60m × 2 / 300,000,000m / s = 0.00000004s). For ease of explanation, in this invention, the time interval from the origin of the supplementary light beam to its reflection or diffuse reflection back to the light receiver is defined as the obstruction removal period 62, that is, the time required for the light beam 10 to travel from the high-frequency pulsed light source 1 to the aforementioned obstruction interference distance 620 and then reflect back to the aforementioned light receiver 20. At a distance farther from the vehicle, there may be, for example, a deer 82 500m away, and an oncoming vehicle 84 1km away.

[0029] Referring to Figure 6, in step 70, a drive signal generator 3 outputs a control signal to drive a high-frequency pulsed light source 1, such as a vertical-cavity surface-emitting laser (VCSEL), to emit a beam 10. After a short period, the light emission stops; this time interval is called the emission period 60. The control signal is also simultaneously provided to the image processor 4. Since continuous images are generally based on, for example, 30 frames per second, in this case, at least 15 processed frames per second are required to provide the dynamic images, that is, the duration of one frame period 6 is 1 / 15 of a second. Therefore, the opening and closing frequency of the electronic shutter 22 of the optical image capture unit 2 is higher than 1 / 15 of a second and is a multiple of a number not limited to integers; this is defined as the exposure period 64. In this example, each exposure period is only 1 / 100 of a second. In step 71, the optical image sensed by one opening and closing process is converted into an electrical signal, and an image data signal 50 of one frame is output to the subsequent image processor 4.

[0030] On the other hand, in step 72, the comparator 40 in the image processor 4 starts to add the calculated number from the end of the light emission period 60 according to the control signal, and checks whether a debris removal period 62 has been reached; before the debris removal period 62 is completed, the image data signal 50 of the image frame received in this period is removed; and when it is confirmed that the debris removal period 62 has been exceeded, this period is defined as the receiving period 66. In step 73, the image processor 4 starts to receive and process the image data signal 50 of the image frame corresponding to the diffuse reflected light of the object at a distance from the vehicle in the receiving period 66.

[0031] In this example, one frame of image data needs to be provided every 1 / 15 second. For example, each exposure period 64 is 1 / 100 second, meaning that within each second, the optical image capture device 2 actually outputs 100 frame images. Among them, any frame image data that is synchronously calculated and confirmed to be in the light emission period 60 or the object removal period 62 will not be included in the reception and superposition. Since the optical image capture device 2 only needs to continuously sense images according to its own frequency, the image capturing device of the present invention has extremely high compatibility and can be used with any high-frequency switching camera, camcorder, or even a high-performance smartphone.

[0032] Since the exposure of a single frame may not be sufficient to analyze and identify distant objects in a dark environment, multiple frames need to be superimposed to form a single image. This involves excluding frames from each illumination period 60 and debris removal period 62. In step 74, it is determined whether four frames have been superimposed after the current exposure period 64 ends. If not, the image processor 4 continues superimposing. If four frames have been received, in step 75, the four frames are superimposed to produce a single image 56. Since the image capturing device continuously captures images, steps 70 to 75 are repeated until the device is powered off after one image is output.

[0033] Therefore, when the aforementioned beam 10 shines on the object 9 and is diffusely reflected back to the aforementioned optical image capture unit 2, since it is still within the aforementioned object removal period 62, even if the light receiver 20 outputs the image frame at this time, the image processor 4 will still exclude it. After the aforementioned object removal period 62, during the exposure period 64, the diffuse reflection images returned from the deer 82 and the oncoming vehicle 84 can be clearly identified and recorded by the image processor 4. In particular, the raindrop diffuse reflection images returned from the deer 82 or the oncoming vehicle 84 are too small to occupy half a pixel, so the raindrops at that location do not affect image capture and do not interfere with image interpretation.

[0034] As will be readily understood by those skilled in the art, the low ambient light intensity active illumination image acquisition device of this invention is not limited to land vehicles such as vehicles. As shown in Figure 7, the second preferred embodiment is used in a submarine, and the parts that are the same as those described in the first embodiment will not be repeated. Since there may be other natural fish, shrimp, organisms such as whales 84' or artificial instruments operating underwater, this embodiment uses the mud and sand stirred up by the submarine propeller as debris 9'. After the high-frequency pulse light source 1' emits a beam 10', the image processor begins to receive the image from the optical image capturer 2' after the debris removal period has passed. In this example, the high-frequency pulse light source 1' and the optical image capturer 2' are set in different positions. At this time, because the distance between the beam 10' and the light receiver and the target is different, and the speed of light travels in water and air is different, the debris removal period in this example needs to be adjusted accordingly. In addition, the debris interference distance 620' in this embodiment can also be increased or decreased according to the size of the debris 9'.

[0035] Furthermore, given that the diffuse reflection light of a mid-range object is higher than that of a far-range object, and that the angle of view may change significantly at any instant when the mid-range object moves within the field of view, that is, when an image is composed of multiple superimposed frames, the image of the mid-range object may be significantly misaligned in each frame, forming a ghost image. In contrast, the diffuse reflection light of a far-range object will decrease inversely with the square of the distance, making the image in a single frame unclear and difficult to distinguish. Therefore, the third preferred embodiment of this case, as shown in Figure 8, takes the infrared laser beam 10” emitted by a drone flying at night as an example, and the debris 9” in front is smoke, there is another drone 82” in the mid-range in front, and a tank 84 in the far distance.

[0036] As shown in Figure 9, when the image data signal 50” of the first frame is determined by the comparator in the image processor, the image of another UAV 82” at medium distance is clear enough to reach a reliable resolution standard and can be clearly identified from the background. A reliable image block 52” is defined here. Then, the image data signals of the second and subsequent frame images corresponding to this object will be excluded to avoid blurring the final image. Of course, if it is necessary to superimpose two frame images 50” to reach the reliable resolution standard, the image data of this block of the first two frame images can be superimposed to form an image of the UAV at medium distance. As for the other parts 54” that have not yet reached the reliable resolution standard, the image data signals of the subsequent frame images will be superimposed until all the image data signals of the frame cycle are superimposed to form an image data 56.

[0037] According to this embodiment, since the image data signals of each frame in each frame cycle are superimposed one by one, the portion that has already formed a reliable image block is recorded and subsequent superposition is excluded, avoiding ghosting caused by the movement of objects at medium distances; objects at long distances can continue to be superimposed until the end of the frame cycle; thereby effectively improving image quality. Of course, the above examples of light sources are all monochromatic light sources, but even if multiple pulse light sources are used together as high-frequency pulse light sources, each light source has a different wavelength. It is only necessary to limit the driving signal generator to drive each of the aforementioned high-frequency pulse light sources to be enabled and emit light at the same time, and the camera can distinguish the image data of different wavelengths through the front filter or Fourier transform during post-processing and perform superposition or identification analysis on each, which is still within the scope of this case.

[0038] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify the above embodiments without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention should be as set forth in the following claims.

[0039] 1, 1': High-frequency pulsed light source 10, 10', 10”: Beam 2, 2': Optical image capture device 20, 20': Optical receiver 22: Electronic shutter 200:pixel 3: Drive signal generator 4: Image Processor 40: Comparator 50, 50”: Image data signal 52”: Trustworthy Image Block 54”: Part that has not yet met the reliable resolution standard 56, 56”: Footage 6: Frame Cycle 60: Emission Period 62: Debris Removal Period 620: Distance to obstruction 64: Exposure Period 66: Reception Period 70~75: Steps 82: Deer 82”: Drone 84: Oncoming vehicle 84': Whale 84”: Tank 9, 9', 9”: Miscellaneous items 90: Pre-order size

Claims

1. A low-ambient-light-intensity active illumination image capturing device, for capturing images of a target object within a predetermined distance in a low-ambient-light-intensity environment with clutter, wherein the clutter has a predetermined size, the active illumination image capturing device comprising: A high-frequency pulsed light source emits a beam that can illuminate the aforementioned predetermined distance; an optical image capture device with an electronic shutter has a light receiver and an electronic shutter with a frequency higher than 1 / 15 second by a multiple of that frequency, thereby controlling each exposure period of the aforementioned light receiver, and outputting an image data signal corresponding to a frame after each exposure period, and the aforementioned light receiver has a predetermined number of pixels; an image processor for receiving the aforementioned image data signal from the aforementioned optical image capture device; a drive signal generator for controlling the emission period of the aforementioned high-frequency pulsed light source, and synchronously outputting a control signal controlling the emission of the aforementioned high-frequency pulsed light source to the aforementioned image processor; and the aforementioned image processor calculates the aforementioned emission period based on the aforementioned control signal, and adds a debris removal period, for superimposing the frame corresponding to the aforementioned emission period and the aforementioned debris removal period in the received aforementioned frame; wherein the aforementioned high-frequency pulsed light source receives instructions from the aforementioned drive signal generator to provide a multiple emission period that conforms to the aforementioned multiples in one frame cycle at a fixed frequency.

2. The active illumination image capturing apparatus as claimed in claim 1, wherein the aforementioned obscuration removal period refers to the time required for a light beam to be emitted from the aforementioned high-frequency pulsed light source to a distance from an obscuration interference and to be reflected back to the aforementioned light receiver during the aforementioned emission period; and the aforementioned obscuration interference distance refers to the distance between the obscuration and the aforementioned optical image capturer when the image size formed at the aforementioned light receiver by the obscuration having the aforementioned predetermined size can cover at least half the area of ​​a single aforementioned pixel.

3. The active illumination image capturing device as described in claim 1, wherein the aforementioned high-frequency pulsed light source is a vertical cavity surface-emitting laser (VCSEL).

4. The active illumination image capturing apparatus as claimed in claim 1, wherein the aforementioned image processor further includes a comparator for confirming, in each of the aforementioned frame cycles, whether some adjacent pixels, in the aforementioned plurality of frame frames, have reached a reliable resolution standard only in the image data signal of the first received frame, thereby constituting at least one reliable image block in the aforementioned image data.

Images