Image capturing device and image quality correction method for image capturing device

The imaging device uses frame division and block analysis to detect and correct image quality issues caused by raindrops, maintaining clear images despite adverse weather conditions.

JP2024157319APending Publication Date: 2024-11-07JVC KENWOOD CORP
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023071615
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-25
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Raindrops hitting the lens or transparent cover of imaging devices on vehicles, such as drive recorders on motorcycles, cause deterioration in image quality during rainfall.

Method used

An imaging device with an image sensor that detects raindrops based on luminance signal changes between adjacent frames, dividing each frame into blocks to correct image quality by adjusting luminance and sharpness signals.

Benefits of technology

Suppresses image quality deterioration even when raindrops adhere to the lens or transparent cover, ensuring clear image capture.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157319000001_ABST
    Figure 2024157319000001_ABST
Patent Text Reader

Abstract

To provide an image capturing device capable of suppressing deterioration of image quality even if raindrops adhere to a lens or a transparent cover covering the lens.SOLUTION: A video frame generation unit 4 generates each frame of a video signal on the basis of a captured image signal obtained by capturing a subject with an imaging element 2. A raindrop adhesion detection unit 14 divides each frame into a plurality of blocks, and detects whether or not it is in a state where a raindrop adheres to a lens 1 or a transparent cover 1c provided in front of the lens 1 on the basis of change in a luminance signal between adjacent frames for each block. An image quality correction unit 5 corrects image quality of the video signal by correcting the luminance signal in a block in a state where the raindrop is captured, among the plurality of blocks in each frame.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an image pickup apparatus and an image quality correction method for the image pickup apparatus. [Background technology]

[0002] 2. Description of the Related Art Imaging devices (so-called drive recorders) that are mounted on vehicles and capture images of the area in front of or behind the vehicle and record the images have become widespread. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-176398 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the number of motorcycles equipped with drive recorders has increased. In a drive recorder equipped with a motorcycle, raindrops directly hit the lens or the transparent cover covering the lens when it rains, so the image quality of the captured image deteriorates when it rains. This type of problem is not limited to drive recorders equipped with motorcycles, but also exists in an imaging device that is installed outside the vehicle as described in Patent Document 1.

[0005] An object of the present invention is to provide an imaging device and an image quality correction method for an imaging device that can suppress deterioration of image quality even if raindrops adhere to a lens or a transparent cover that covers the lens. [Means for solving the problem]

[0006] The present invention provides an imaging device that includes an imaging element into which light from a subject is incident through a lens, a video frame generation unit that generates each frame of a video signal based on an image signal obtained by imaging the subject with the imaging element, a raindrop adhesion detection unit that divides each frame into a plurality of blocks and detects whether raindrops are attached to the lens or a transparent cover provided in front of the lens based on a change in the luminance signal between adjacent frames for each block, and an image quality correction unit that corrects the image quality of the video signal by correcting the luminance signal in a block in which raindrops are imaged, among the plurality of blocks of each frame.

[0007] The present invention provides an image pickup device that captures an image of a subject using an image pickup element into which light from the subject is incident through a lens, generates each frame of a video signal based on an image pickup image signal obtained by the image pickup element capturing the subject, divides each frame into a plurality of blocks, and detects whether raindrops are attached to the lens or a transparent cover provided in front of the lens based on a change in the luminance signal between adjacent frames for each block. The image pickup device provides an image quality correction method for correcting the image quality of the video signal by correcting the luminance signal in a block in which raindrops are captured among the plurality of blocks of each frame. Effect of the Invention

[0008] According to the imaging device and the image quality correction method for the imaging device of the present invention, even if raindrops adhere to the lens or the transparent cover that covers the lens, deterioration of image quality can be suppressed. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an imaging device according to an embodiment. [Figure 2A] FIG. 2A is a diagram showing an example of a frame generated by an imaging device according to an embodiment. [Figure 2B]FIG. 2B is a diagram illustrating an example of a frame generated by an imaging device according to an embodiment, in which raindrops are attached to a lens or a transparent cover covering the lens. [Figure 3A] FIG. 3A is a diagram showing an example in which a frame is divided into 20 blocks. [Figure 3B] FIG. 3B is a diagram showing an example in which a frame is divided into 80 blocks. [Figure 4] FIG. 4 is a diagram showing an example in which the block size is changed for each level of vehicle speed. [Diagram 5] FIG. 5 is a partial block diagram showing an example of a configuration in which the size of a block is increased as the precipitation level increases. [Figure 6A] FIG. 6A is a diagram showing an example of a block in which raindrops are photographed, with a frame being divided into 20 blocks. [Figure 6B] FIG. 6B is a diagram showing an example of a block in which raindrops are photographed, with the frame being divided into 80 blocks. [Figure 7A] FIG. 7A is a diagram showing an example of gamma characteristics used when the image quality of a video signal is not corrected. [Figure 7B] FIG. 7B is a diagram showing an example of gamma characteristics used when correcting the image quality of a video signal. [Figure 8] FIG. 8 is a flowchart showing the operation of the imaging device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An imaging device and an image quality correction method for an imaging device according to an embodiment will be described below with reference to the accompanying drawings. FIG. 1 shows an imaging device 100 which is an imaging device according to an embodiment. As an example, the imaging device 100 is a drive recorder for a motorcycle. The imaging device 100 is suitable for use as an imaging device mounted on the outside of a vehicle, but is not limited to being mounted on the outside of a vehicle. The imaging device 100 may also be mounted at a predetermined location outside a vehicle, such as a surveillance camera.

[0011] 1, the imaging device 100 includes a lens 1, an imaging element 2, a signal processing unit 3, an encoding unit 6, a write control unit 7, a memory 8, a GNSS receiving unit 11, a position calculation unit 12, a vehicle speed detection unit 13, a raindrop attachment detection unit 14, and an image quality correction intensity determination unit 15. The signal processing unit 3 has a video frame generation unit 4 and an image quality correction unit 5. The image quality correction unit 5 includes a sharpness correction unit 51 and a gamma correction unit 52.

[0012] Although it is shown in a simplified manner in FIG. 1, the lens 1 is made up of multiple lenses. The image sensor 2 is a charge coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. The image sensor 2 may be three image sensors for red, green, and blue. Light from a subject, indicated by a dashed line, enters the image sensor 2 through the lens 1. As indicated by a two-dot dashed line, a transparent cover 1c made of plastic or glass may be provided in front of the lens 1. When it rains, raindrops directly hit the lens 1 or the transparent cover 1c and adhere to it.

[0013] The video frame generator 4 generates each frame of a video signal based on an image signal obtained by capturing an image of a subject with the image sensor 2. The video signal is composed of a luminance signal and a color signal (or a color difference signal). The image quality corrector 5 corrects the luminance signal of each frame to correct the image quality, and supplies the signal to the encoder 6. The signal processor 3 can be configured with a signal processing circuit. The signal processor 3 may be an integrated circuit. The video signal input to the image quality corrector 5 may be converted into a digital signal by an A / D converter (not shown). How the image quality corrector 5 corrects the luminance signal will be described in detail later.

[0014] The encoding unit 6 compresses and encodes the input video signal using a compression encoding method such as H.264 / MPEG-4 AVC. The encoding unit 6 can be configured with an integrated circuit. The write control unit 7 writes the compressed and encoded video signal to the memory 8. The write control unit 7 can be configured with a circuit that writes data to the memory 8. The memory 8 may be a removable memory card. Although not shown in FIG. 1, the video signal written to the memory 8 may be read by a read control unit and displayed on the display unit. Note that FIG. 1 omits the collection of audio and the writing of the audio signal to the memory 8.

[0015] The GNSS receiver 11 receives GNSS signals transmitted from three or more satellites for the Global Navigation Satellite System (GNSS). The GNSS receiver 11 can be configured with a receiving circuit. The position calculation unit 12 calculates the current position (position information) of the vehicle (motorcycle) based on the GNSS signals from three or more satellites received by the GNSS receiver 11. Typically, the GNSS is a Global Positioning System (GPS). The vehicle speed detection unit 13 detects the vehicle speed based on changes in the position information supplied from the position calculation unit 12. The vehicle speed is the speed at which the imaging device 100 moves. The position calculation unit 12 and the vehicle speed detection unit 13 can be configured with a microprocessor.

[0016] Instead of the imaging device 100 including the GNSS receiving unit 11, the position calculating unit 12, and the vehicle speed detecting unit 13 to detect the vehicle speed, the imaging device 100 may obtain the vehicle speed (traveling speed) from an ECU (Electronic Control Unit) of the vehicle.

[0017] The raindrop detection unit 14 is supplied with the luminance signal of each frame from the video frame generation unit 4. The raindrop detection unit 14 is supplied with the luminance signal converted into a digital signal by an A / D converter (not shown). The raindrop detection unit 14 detects whether or not raindrops are present on the lens 1 or the transparent cover 1c based on the luminance signal of each frame.

[0018] FIG. 2A shows a frame F with a video signal in a state where there are no raindrops on the lens 1 or the transparent cover 1c. FIG. 2B shows a frame F with a video signal in a state where there are raindrops on the lens 1 or the transparent cover 1c. In FIG. 2B, raindrops Rd are shown in frame F with four hatched areas. The shape of the raindrops Rd is indefinite. In reality, the raindrops Rd are transparent, and the raindrops Rd attached to the lens 1 or the transparent cover 1c spread due to wind pressure, so that the attached raindrops Rd act to blur the image of the subject being photographed.

[0019] The raindrop detection unit 14 detects the presence or absence of raindrops Rd as follows. The raindrop detection unit 14 divides each frame of the video signal into multiple blocks, and detects the presence or absence of raindrops Rd based on the change in the luminance signal between adjacent frames for each block. When raindrops Rd are present, the edges of the image become smooth and blurred, and the luminance of the mid-tone or low-tone areas of the luminance signal becomes higher.

[0020] The raindrop detection unit 14 can determine whether a block is in a state where raindrops Rd are being imaged, depending on whether a first condition is satisfied, that is, the high-frequency components obtained by differentiating the luminance signal between adjacent frames for each block are reduced by a first threshold or more. The raindrop detection unit 14 can also determine whether a block is in a state where raindrops Rd are being imaged, depending on whether a second condition is satisfied, that is, the luminance of an area below a predetermined gradation of the luminance signal is increased by a second threshold or more between adjacent frames for each block.

[0021] The second threshold may be an individual threshold set for each gradation below the predetermined gradation, or may be a single threshold indicating the overall increase rate of gradations below the predetermined gradation.

[0022] The raindrop detection unit 14 determines that the block is in a state in which the raindrop Rd is being imaged when at least one of the first condition and the second condition is satisfied. In order to further improve the accuracy of the determination, the raindrop detection unit 14 may determine that the block is in a state in which the raindrop Rd is being imaged when both the first condition and the second condition are satisfied. Since the images have correlation between adjacent frames, there is almost no sudden drop in high-frequency components or a sudden increase in brightness in areas below a certain gradation. Therefore, the raindrop detection unit 14 can detect a block in a state in which the raindrop Rd is being imaged by using at least one of the first condition and the second condition.

[0023] The raindrop detection unit 14 may detect that the raindrops Rd are attached to the lens 1 or the transparent cover 1c if the number of blocks determined to be in a state where the raindrops Rd are captured is equal to or greater than a predetermined ratio to the total number of blocks into which each frame is divided. The predetermined ratio is, for example, 10% to 20%. In this way, it is possible to avoid erroneous detection when a block that coincidentally satisfies at least one of the first condition and the second condition occurs.

[0024] Raindrops Rd adhering to the lens 1 or the transparent cover 1c may have a mountain-shaped characteristic in which the brightness is high at the center and decreases toward the periphery. The raindrop attachment detection unit 14 may detect that raindrops Rd are attached to the lens 1 or the transparent cover 1c based on the mountain-shaped brightness characteristic.

[0025] FIG. 3A shows an example in which a frame F is divided into 20 blocks Bk by dividing the frame F into 5 horizontally and 4 vertically. FIG. 3B shows an example in which a frame F is divided into 80 blocks Bk by dividing the frame F into 10 horizontally and 8 vertically. In FIG. 3A and FIG. 3B, the raindrops Rd are simplified into a circular shape. As shown in FIG. 3A, when the raindrops Rd are photographed in a large block Bk, the presence of the raindrops Rd does not have a large effect on the luminance signal in the block Bk. As shown in FIG. 3B, when the raindrops Rd are photographed in a small block Bk, the presence of the raindrops Rd has a large effect on the luminance signal in the block Bk.

[0026] Since the size of the raindrops Rd is not constant, it is preferable that the raindrop detection unit 14 changes the size of the blocks Bk in multiple stages. Changing the size of the blocks Bk means changing the number of divisions into which the frame F is divided. If the number of divisions of the frame F is reduced, the blocks Bk become larger, and if the number of divisions of the frame F is increased, the blocks Bk become smaller.

[0027] On the other hand, the raindrop detection unit 14 needs to make the block Bk the same size in adjacent frames in which the luminance signals are compared in order to detect changes in the luminance signal. Therefore, it is preferable that the raindrop detection unit 14 makes the block Bk the same size in adjacent frames in which the luminance signals are compared, and repeats changing the size of the block Bk in multiple steps between the first size and the second size for every two or more frames. The first size is the minimum size when the size of the block Bk is changed in multiple steps, and the second size is the maximum size when the size of the block Bk is changed in multiple steps.

[0028] As an example, suppose that the frames proceed in the order of F1, F2, F3, F4, F5, F6, ... and that the size of the block Bk is the same for a period of four frames, and the size of the block Bk is changed every four frames. In this case, the raindrop detection unit 14 compares the luminance signals of each block Bk between frames F1 and F2, between frames F2 and F3, and between frames F3 and F4. The size of the block Bk in frames F1 to F4 is a first size.

[0029] The raindrop detection unit 14 sets the size of the blocks Bk in frames F5 to F8 to be larger than the first size. The raindrop detection unit 14 compares the luminance signals of the blocks Bk between frames F5 and F6, between frames F6 and F7, and between frames F7 and F8. If the size of the blocks Bk is changed in three stages, the raindrop detection unit 14 sets the size of the blocks Bk in frames F9 to F12 to a second size larger than the size of the blocks Bk in frames F5 to F8. The raindrop detection unit 14 compares the luminance signals of the blocks Bk between frames F9 and F10, between frames F10 and F11, and between frames F11 and F12.

[0030] Since raindrops Rd attached to the lens 1 or the transparent cover 1c spread due to wind pressure, the size of the raindrops Rd in the attached state increases as the vehicle speed increases. Therefore, it is preferable for the raindrop attachment detection unit 14 to increase the size of the blocks Bk as the vehicle speed increases. As an example, the vehicle speed is divided into a plurality of levels, such as less than 40 km / h (level 1), 40 km / h or more and less than 60 km / h (level 2), 60 km / h or more and less than 80 km / h (level 3), and 80 km / h or more (level 4). The raindrop attachment detection unit 14 increases each size from the first size to the second size of the blocks Bk as the vehicle speed level increases.

[0031] FIG. 4 shows an example in which the raindrop detection unit 14 changes the size of the block for each level of the vehicle speed. As shown in FIG. 4, the raindrop detection unit 14 increases each size from the first size to the second size of the block Bk as the level of the vehicle speed increases. In FIG. 4, it is assumed that the vehicle speed accelerates from 0 km / h to 40 km / h from time t0 to time t1, and from 40 km / h to 60 km / h from time t1 to time t2. It is assumed that the vehicle speed accelerates from 60 km / h to 80 km / h from time t2 to time t3, and from 80 km / h to 100 km / h from time t3 to time t4, and then maintains 100 km / h.

[0032] During the period when the vehicle speed is at level 1, the raindrop detection unit 14 repeatedly changes the size of the block Bk from size 1 to size 3 as a relative value. During the period of level 1, size 1 is the first size, and size 3 is the second size. In the example shown in FIG. 4, the sizes 1, 2, 3, 1, 2, 3, ... are repeated, but the sizes 1, 2, 3, 2, 1, 2, 3, ... may be repeated. The number of stages from the first size to the second size is not limited to three. As an example, when the raindrop detection unit 14 increases the size by 1, it decreases the number of divisions in the horizontal and vertical directions of the frame F by 1, respectively. There is no limitation on how the raindrop detection unit 14 decreases the number of divisions in the horizontal and vertical directions of the frame F every time it increases the size by 1.

[0033] When the vehicle speed is at level 2, the raindrop detection unit 14 repeatedly changes the size of the block Bk, as a relative value, from size 4 to size 6. When the vehicle speed is at level 3, the raindrop detection unit 14 repeatedly changes the size of the block Bk, as a relative value, from size 7 to size 9. When the vehicle speed is at level 4, the raindrop detection unit 14 repeatedly changes the size of the block Bk, as a relative value, from size 10 to size 12.

[0034] During level 2, magnitude 4 is the first magnitude and magnitude 6 is the second magnitude. During level 3, magnitude 7 is the first magnitude and magnitude 9 is the second magnitude. During level 4, magnitude 10 is the first magnitude and magnitude 12 is the second magnitude. During levels 2 to 4, the range from the first magnitude to the second magnitude is not limited to three stages.

[0035] In the example shown in Figure 4, the first magnitude during the level 2 period, magnitude 4, is greater than the second magnitude during the level 1 period, magnitude 3. The first magnitude during the level 3 period, magnitude 7, is greater than the second magnitude during the level 2 period, magnitude 6. The first magnitude during the level 4 period, magnitude 10, is greater than the second magnitude during the level 3 period, magnitude 9.

[0036] The second magnitude during the level 1 period may be the same as the first magnitude during the level 2 period. The second magnitude during the level 2 period may be the same as the first magnitude during the level 3 period. The second magnitude during the level 3 period may be the same as the first magnitude during the level 4 period. In this manner, the sizes of the blocks Bk may overlap between adjacent vehicle speed levels. Also, the first magnitude during the level 2 period may be slightly smaller than the second magnitude during the level 1 period. The first magnitude during the level 3 period may be slightly smaller than the second magnitude during the level 2 period. The first magnitude during the level 4 period may be slightly smaller than the second magnitude during the level 3 period.

[0037] 4, which shows how the size of the block Bk is changed over time, the change in size of the block Bk is shown in a cycle that is much longer than the actual cycle in which the size of the block Bk is changed. The configuration in which the size of the block Bk is increased as the vehicle speed increases is preferably used when the imaging device 100 is mounted on a vehicle.

[0038] Generally, the more the amount of precipitation, the larger the raindrops become, so the larger the raindrops Rd on the lens 1 or the transparent cover 1c become. Therefore, the raindrop detection unit 14 may increase the size of the block Bk as the amount of precipitation increases. As an example, the raindrop detection unit 14 divides the amount of precipitation into a plurality of levels, and increases each size of the block Bk from the first size to the second size as the level of precipitation increases.

[0039] FIG. 5 shows an example of a configuration in which the raindrop adhesion detection unit 14 increases the size of the block Bk as the precipitation level increases. In FIG. 5, parts common to FIG. 1 are omitted. The communication unit 21 acquires meteorological information related to rainfall from a meteorological information providing server 31 via a network 30 and supplies the acquired meteorological information to the raindrop adhesion detection unit 14. The network 30 may be the Internet or a mobile phone communication network. The meteorological information related to rainfall may be a weather forecast related to the amount of rainfall, or may be the actual amount of rainfall at each location and each time. The configuration shown in FIG. 5 may be used when the imaging device 100 is mounted at a fixed location other than a vehicle.

[0040] As another example, the raindrop detection unit 14 may determine the number of pixels in an area where high frequency components suddenly drop or brightness in an area below a certain gradation suddenly increases as the size of the raindrops Rd. The raindrop detection unit 14 may determine the size of the block Bk according to the average value of the sizes of a plurality of raindrops Rd. The raindrop detection unit 14 may also estimate the amount of precipitation based on the number of raindrops Rd in the frame F and determine the size of the block Bk according to the estimated amount of precipitation.

[0041] In Fig. 6A, for example, the hatched blocks Bk show the blocks Bk in which raindrops Rd are photographed when the vehicle speed is level 4 and the frame F is divided into 20 blocks Bk. In Fig. 6B, for example, the vehicle speed is level 1 and the frame F is divided into 80 blocks Bk, and the hatched blocks Bk show the blocks Bk in which raindrops Rd are photographed. The number of divisions for dividing the frame F into a plurality of blocks Bk shown in Figs. 6A and 6B is merely an example.

[0042] Returning to Fig. 1, the image quality correction strength determination unit 15 determines the correction strength for correcting the image quality of the hatched block Bk according to the degree of change in the luminance signal detected by the raindrop detection unit 14. The degree of change in the luminance signal is the degree to which high frequency components are reduced and the degree to which the luminance of the area below a certain gradation is increased.

[0043] The raindrop detection unit 14 and the image quality correction strength determination unit 15 can be configured with a microprocessor. The position calculation unit 12, the vehicle speed detection unit 13, the raindrop detection unit 14, and the image quality correction strength determination unit 15 may be configured with one microprocessor. The microprocessor may be a central processing unit (CPU) of a microcomputer.

[0044] The image quality correction unit 5 corrects the image quality of the video signal by correcting the luminance signal in the hatched blocks Bk in which the raindrops Rd are captured, among the multiple blocks Bk in each frame. The sharpness correction unit 51 applies a first image quality correction to the video signal, which increases the sharpness of the luminance signal in the blocks Bk in which the raindrops Rd are captured. The gamma correction unit 52 applies a second image quality correction to the video signal, which reduces the luminance of the area below a predetermined gradation in the blocks Bk in which the raindrops Rd are captured.

[0045] The video signal input to the image quality correction unit 5 is delayed by a predetermined time by a delay unit (not shown) to match the timing of the frame in which the raindrop detection unit 14 detects the adhesion of raindrops Rd with the frame in which the image quality correction unit 5 corrects the luminance signal. This allows the image quality correction unit 5 to correct the luminance signal of the frame in which the raindrop detection unit 14 detects the block Bk in which the raindrop Rd is imaged.

[0046] The sharpness correction unit 51 corrects the sharpness by adding overshoot and undershoot components to the edges of the image. The sharpness correction unit 51 varies the degree to which the sharpness is increased according to the correction strength determined by the image quality correction strength determination unit 15. By varying the magnitude of the overshoot and undershoot components added to the edges of the image, the degree to which the sharpness is increased can be varied. The sharpness correction unit 51 may be a circuit called an edge enhancement circuit or a contour correction circuit.

[0047] When the image quality of the video signal is not corrected, the gamma correction unit 52 converts the input luminance signal into an output luminance signal with a gamma characteristic as shown in FIG. 7A. In the block Bk in which the raindrop Rd is imaged, the luminance of the area below a predetermined gradation is increased. Therefore, when the image quality of the video signal is corrected, the gamma correction unit 52 converts the input luminance signal into an output luminance signal with a correction gamma characteristic that reduces the luminance of the area below a predetermined gradation as shown in FIG. 7B. FIGS. 7A and 7B show an example in which the video signal input to the image quality correction unit 5 is 8 bits.

[0048] 7B shows only one gamma characteristic, multiple correction gamma characteristics with different degrees of reduction in the luminance in the region below a predetermined gradation are set in the gamma correction unit 52. The gamma correction unit 52 selects an appropriate correction gamma characteristic according to the correction strength determined by the image quality correction strength determination unit 15, and varies the degree of reduction in the luminance in the region below a predetermined gradation.

[0049] The image quality correction unit 5 may have only the sharpness correction unit 51 and apply only the first image quality correction to the video signal. The image quality correction unit 5 may have only the gamma correction unit 52 and apply the second image quality correction to the video signal. It is preferable that the image quality correction unit 5 has the sharpness correction unit 51 and the gamma correction unit 52 and applies both the first and second image quality corrections to the video signal.

[0050] The imaging device 100 may be configured without the image quality correction intensity determination unit 15. In this case, the image quality correction unit 5 uniformly corrects the luminance signal regardless of the degree of change in the luminance signal detected by the raindrop detection unit 14. If the imaging device 100 includes the image quality correction intensity determination unit 15, the luminance signal can be corrected according to the degree of change in the luminance signal, so it is preferable to include the image quality correction intensity determination unit 15.

[0051] When the blocks Bk in which the raindrops Rd are photographed occupy most of all the blocks Bk in the frame F, the image quality correction unit 5 may apply the first image quality correction or the second image quality correction to the entire frame F. When the blocks Bk in which the raindrops Rd are photographed occupy most of all the blocks Bk, for example, the blocks Bk in which the raindrops Rd are photographed occupy 80% or more of all the blocks Bk.

[0052] The operation of the imaging device 100 and the image quality correction method executed by the imaging device 100 will be described with reference to the flowchart shown in Fig. 8. When the imaging device 100 is turned on and processing starts, the video frame generator 4 generates each frame of a video signal based on a captured image in step S1. The raindrop detector 14 acquires the vehicle speed (the moving speed of the imaging device 100) in step S2. The raindrop detector 14 divides each frame into a plurality of blocks according to the level of the vehicle speed in step S3.

[0053] In step S4, the raindrop detection unit 14 determines whether or not there is a sudden change in luminance between adjacent frames for each block. In step S5, the raindrop detection unit 14 determines whether or not the number of blocks in which the luminance has suddenly changed is equal to or greater than a predetermined ratio among all blocks. If the number of blocks in which the luminance has suddenly changed is not equal to or greater than the predetermined ratio (NO), the raindrop detection unit 14 transitions the process to step S6.

[0054] If the number of blocks that have changed suddenly in step S5 is equal to or greater than a predetermined ratio (YES), then in step S11, the image quality correction strength determination unit 15 determines the image quality correction strength according to the degree of change in luminance. In step S12, the image quality correction unit 5 corrects the image quality of the blocks in the frame where the luminance has changed suddenly, and then proceeds to step S6.

[0055] In step S6, the raindrop detection unit 14 acquires the vehicle speed. In step S7, the raindrop detection unit 14 determines whether the level of the vehicle speed has changed. If the level of the vehicle speed has not changed (NO), in step S8, the raindrop detection unit 14 changes the block size within the same level and transitions to step S10. If the level of the vehicle speed has changed (YES), in step S9, the raindrop detection unit 14 sets the block size used within the changed level and transitions to step S10.

[0056] In step S10, the imaging device 100 determines whether or not to end the operation of the imaging device 100 by turning off the power, etc. If the operation of the imaging device 100 is not to be ended (NO), the imaging device 100 repeats the processes from step S4 onwards. If the operation is to be ended (YES), the imaging device 100 ends the process.

[0057] As described above, according to the imaging device 100, even if raindrops Rd adhere to the lens 1 in a configuration that does not have the transparent cover 1c, or even if raindrops Rd adhere to the transparent cover 1c in a configuration that has the transparent cover 1c, deterioration of image quality can be suppressed.

[0058] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0059] 1 Lens 1c Transparent cover 2. Image sensor 3. Signal Processing Section 4. Video Frame Generation Section 5. Image quality correction section 6 Encoding section 7 Write control section 8. Memory 11 GNSS receiver 12 Position calculation section 13 Vehicle speed detection unit 14 Raindrop detection unit 15 Image quality correction strength determination unit 21 Communications Department 30 Network 31 Weather information server 51 Sharpness correction section 52 Gamma correction section 100 Imaging device

Claims

1. an image sensor into which light from a subject is incident via a lens; a video frame generating unit that generates each frame of a video signal based on an image signal obtained by imaging the subject with the imaging element; A raindrop adhesion detection unit that divides each frame into a plurality of blocks and detects whether or not raindrops are attached to the lens or a transparent cover provided in front of the lens based on a change in a luminance signal between adjacent frames for each block; An image quality correction unit that corrects the image quality of the video signal by correcting a luminance signal in a block in which a raindrop is imaged among the plurality of blocks in each frame; An imaging device comprising:

2. The raindrop detection unit compares the luminance signals to detect a change in the luminance signal, and the blocks are the same size in the adjacent frames, and the size of the blocks is changed in multiple steps between a first size and a second size for every two or more frames. The imaging device according to claim 1, wherein the image sensor detects a change in the luminance signal.

3. The imaging device according to claim 2 , wherein the raindrop detection unit increases each of the magnitudes from the first magnitude to the second magnitude as the level of the speed at which the imaging device moves increases.

4. The imaging device according to claim 2 , wherein the raindrop detection unit increases each of the first magnitude to the second magnitude as a precipitation level increases.

5. The raindrop detection unit determines that the block is in a state in which raindrops are imaged when at least one of a first condition in which a high-frequency component obtained by differentiating the luminance signal between the adjacent frames for each block is reduced by a first threshold value or more and a second condition in which the luminance of a region below a predetermined gradation of the luminance signal is increased by a second threshold value or more is satisfied. The image quality correction unit applies at least one of a first image quality correction that increases the sharpness of a luminance signal in a block in which a raindrop is imaged and a second image quality correction that reduces the luminance of an area below the predetermined gradation of the luminance signal in a block in which a raindrop is imaged. The imaging device according to any one of claims 1 to 4.

6. An image of the subject is captured by an image sensor into which light from the subject is incident via a lens; generating each frame of a video signal based on an image signal obtained by the imaging device imaging the subject; Each frame is divided into a plurality of blocks, and based on a change in a luminance signal between adjacent frames for each block, it is detected whether or not raindrops are attached to the lens or a transparent cover provided in front of the lens; The image quality of the video signal is corrected by correcting a luminance signal in a block in which a raindrop is imaged among the plurality of blocks in each frame. A method for correcting image quality of an imaging device.

Citation Information

Patent Citations

  • Vehicle image processing device, vehicle image processing method, and vehicle image processing program

    JP2019176398A