Imaging apparatus, video processing method, and video processing apparatus

Abstract

To enable high-sensitivity photographing in an imaging apparatus without a slow shutter function, and to suppress deterioration in video quality accompanying the high-sensitivity photographing.SOLUTION: There is provided an imaging device including a signal processing unit that applies a filtering process to a video frame generated from an output signal of an image sensor, a first black level correction unit that applies black level correction to the video frame after the application of the filtering process, and a frame storage unit that outputs a high-luminance video frame generated using a plurality of video frames after the application of the black level correction when a high-sensitivity imaging function is turned on.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a video processing method, and a video processing device. [Background technology]

[0002] There are many different types of cameras in the world. Familiar cameras include compact digital cameras and smartphone cameras that are used for snapshots and portraits. Special-purpose cameras, on the other hand, include broadcast cameras used by television stations and surveillance cameras installed at building entrances and outdoors. Other examples include camera systems installed in automobiles, ships, and aircraft, as well as specialized surveillance cameras installed on airport runways to monitor for obstacles. Of course, there are also multipurpose cameras that can be adapted for multiple specialized uses.

[0003] For example, outdoor fixed surveillance cameras are used for long periods of time without an operator present. In this case, the brightness level of the images output from the surveillance camera changes from moment to moment due to environmental changes such as changes in weather and sunlight conditions. Considering the use of surveillance cameras for detecting monitored objects, it is necessary to maintain a certain brightness level of the images. One way to increase the brightness level is to extend the exposure time (hereinafter referred to as the slow shutter function). However, cameras used for specific purposes are not equipped with a slow shutter function, and brightness levels cannot be adjusted using a slow shutter.

[0004] One method for increasing the brightness level of an image without using a slow shutter is to add multiple image frames to generate a high-brightness image frame, as described in Patent Document 1 below. Furthermore, in the method of Patent Document 1, when the number of added image frames is changed, gain adjustment is performed so that the brightness level of the output high-brightness image frame remains constant before and after the change. For example, in the method of Patent Document 1, when the number of added image frames is halved, the gain for the changed image frame is doubled. Furthermore, Patent Document 2 below describes a method for adjusting the gain according to the output of a frame memory to suppress blown-out highlights caused by frame addition. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-263316 [Patent Document 2] Japanese Patent Application Publication No. 11-261880 Summary of the Invention [Problem to be solved by the invention]

[0006] By applying the above-mentioned video frame addition process, it is possible to increase the brightness level of the video even without a slow shutter function. However, with the method described in the above document, a high-brightness video frame is suddenly output after the frame addition operation begins, resulting in a brightness level gap before and after the output. Note that the gain adjustment in Patent Document 1 is a function that adjusts the gain to maintain a constant brightness level depending on the increase or decrease in the number of frames added during the frame addition operation, and is therefore unable to eliminate the brightness level gap described above. The gain adjustment in Patent Document 2 is also a function that adjusts the gain to prevent overexposure during the frame addition operation, and is therefore unable to eliminate the brightness level gap described above.

[0007] The above-described sudden changes in brightness level degrade image quality. In addition to this, reducing factors that degrade image quality due to frame addition, even if only slightly, is important when performing high-sensitivity shooting with an imaging device that does not have a slow shutter function. Therefore, one aspect of the present invention aims to enable high-sensitivity shooting with an imaging device that does not have a slow shutter function, and to suppress the degradation of image quality that accompanies high-sensitivity shooting. [Means for solving the problem]

[0008] According to a first aspect of the present invention, there is provided an imaging device comprising: a signal processing unit that generates a video frame from an output signal of an image sensor; a frame storage unit that generates a high-brightness video frame using a plurality of video frames output from the signal processing unit when a high-sensitivity shooting function is turned on, and switches the video frame to be output to the high-brightness video frame; and a gain adjustment unit that adjusts the brightness level of the video frame output during a plurality of frame periods before switching to the high-brightness video frame so that the brightness level gradually approaches the brightness level of the high-brightness video frame during a plurality of frame periods before switching to the high-brightness video frame.

[0009] Furthermore, according to a second aspect of the present invention, there is provided an imaging device comprising: a signal processing unit that applies filtering processing to video frames generated from an output signal of an image sensor; a first black level correction unit that applies black level correction to the video frames after the filtering processing has been applied; and a frame storage unit that, when a high-sensitivity shooting function is turned on, outputs a high-brightness video frame generated using a plurality of video frames after the black level correction has been applied.

[0010] Furthermore, according to a third aspect of the present invention, there is provided an image processing method in which, when a high-sensitivity shooting function is turned on, a computer executes processing including: generating a high-brightness image frame using a plurality of image frames generated from an output signal of an image sensor, and switching the image frame to be output to the high-brightness image frame; and adjusting the brightness levels of the image frames output during a plurality of frame periods before switching to the high-brightness image frame so that the brightness levels gradually approach the brightness level of the high-brightness image frame during a plurality of frame periods before switching to the high-brightness image frame.

[0011] Furthermore, according to a fourth aspect of the present invention, there is provided an image processing method in which a computer executes processing including applying a filtering process to an image frame generated from an output signal of an image sensor, applying black level correction to the image frame after the filtering process has been applied, and when a high-sensitivity shooting function is turned on, outputting a high-brightness image frame generated using a plurality of image frames after the black level correction has been applied.

[0012] Furthermore, according to a fifth aspect of the present invention, there may be provided a computer program for implementing the video processing method according to either the third or fourth aspect on a computer. Furthermore, according to a sixth aspect of the present invention, there may be provided a computer-readable storage medium storing the computer program according to the fifth aspect. Furthermore, according to a seventh aspect of the present invention, there may be provided a video processing device that executes the video processing method according to either the third or fourth aspect. [Effects of the Invention]

[0013] According to an aspect of the present invention, it is possible to enable high-sensitivity shooting with an imaging device that does not have a slow shutter function, and also to suppress the degradation of image quality that accompanies high-sensitivity shooting. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram schematically illustrating the configuration of an imaging device according to a first embodiment. [Figure 2]10A and 10B are diagrams for explaining pixel addition processing by a signal processing unit. [Figure 3] 10 is a table showing an example of first correction data. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a frame storage unit. [Figure 5] 10 is a timing chart for explaining an example of operation of a frame storage unit (in the case of storing two frames). [Figure 6] 10 is a timing chart for explaining an example of operation of the frame storage unit (in the case of storing three frames). [Figure 7] 10 is a table showing an example of second correction data. [Figure 8] 10 is a timing chart for explaining an example of operation of a gain adjustment unit (in the case of accumulating two frames); [Figure 9] 10 is a timing chart for explaining an example of operation of the gain adjustment unit (in the case of accumulating three frames). [Figure 10] 10 is a flowchart illustrating the flow of video frame processing according to the first embodiment. [Figure 11] FIG. 10 is a block diagram schematically illustrating the configuration of an imaging device according to a second embodiment. [Figure 12] FIG. 10 is a block diagram schematically illustrating the configuration of an imaging device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, elements having substantially the same functions are designated by the same reference numerals, and redundant description may be omitted.

[0016] 1. First Embodiment A first embodiment of the present invention will be described below.

[0017] [1-1. Configuration of imaging device] The configuration of the imaging device according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram that schematically shows the configuration of the imaging device according to the first embodiment. Note that the imaging device 10 shown in Fig. 1 is an example of the imaging device according to the first embodiment. Furthermore, the imaging device 10 does not have the slow shutter function described above.

[0018] 1, the imaging device 10 includes a camera head unit 11, a camera control unit 12, and a display device 13. The camera head unit 11 may be simply referred to as a camera head or as an optical unit. The camera control unit 12 may be referred to as a CCU (Camera Control Unit).

[0019] 1, the camera head unit 11 and the camera control unit 12 are shown separately, but they may be an integrated imaging device in which both are housed in the same housing. In the following explanation, the configuration example shown in Fig. 1 will be referred to, but the physical shape of the imaging device 10 and the combination of functions provided in each of the camera head unit 11 and the camera control unit 12 can be modified as desired depending on the embodiment, and such modifications naturally fall within the technical scope of the first embodiment.

[0020] Camera head unit 11 is connected to camera control unit 12 via a communication line such as an optical fiber cable, a coaxial cable, or a LAN (Local Area Network) cable. Control signals output from camera control unit 12 and video signals output from camera head unit 11 are transmitted via the communication line. For example, when an optical fiber cable is used, a communication line up to a maximum length of approximately 10 km can be laid, which makes it possible to use the cable not only as a broadcast camera but also to easily apply it to a surveillance camera operating in a large area such as an airport.

[0021] The display device 13 is connected to the camera control unit 12 and displays the video output from the camera control unit 12. The display device 13 is, for example, a display device such as an LCD (Liquid Crystal Display) or an ELD (Electro-Luminescence Display). In the example of FIG. 1, the display device 13 is directly connected to the camera control unit 12, but the display device 13 may be external to the imaging device 10 or may be connected to the imaging device 10 via an interface for video output. The installation method and connection method of the display device 13 can be modified as desired depending on the embodiment, and such modifications naturally fall within the technical scope of the first embodiment.

[0022] (Camera head configuration) As shown in FIG. 1, the camera head unit 11 includes an optical system 111, an image sensor 112, and a signal processing unit 113.

[0023] The optical system 111 includes, for example, a lens unit 111a, an optical filter 111b, and a spectroscopic element 111c. The lens unit 111a is composed of one or more lens groups, and each lens group includes one or more lenses. The lens unit 111a may include an iris. The optical filter 111b may be an ND (Neutral Density) filter for dimming, an IR filter that attenuates infrared rays, or a color filter that transmits any of R (red), G (green), or B (blue), and may be disposed on the path of light that passes through the lens unit 111a and reaches the image sensor 112.

[0024] The spectroscopic element 111c is an optical element such as a prism that separates light transmitted through the lens unit 111a, and is disposed between the lens unit 111a and the image sensor 112. In the example of FIG. 1, three image sensors 112 are mounted on the imaging device 10. For example, the three image sensors 112 correspond to R, G, and B, respectively. In this case, the spectroscopic element 111c separates the light transmitted through the lens unit 111a into three components, R, G, and B, and guides these three components to the three image sensors 112, respectively. Note that if the imaging device 10 is equipped with one image sensor 112, the spectroscopic element 111c is omitted.

[0025] The image sensor 112 is a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal-Oxide Semiconductor) sensor. As described above, the image sensor 112 may be composed of multiple sensors. The readout method of the image sensor 112 may be a rolling shutter method or a global shutter method. However, from the viewpoint of suppressing rolling shutter distortion, it is more preferable to employ a global shutter CMOS sensor. The video signal output from the image sensor 112 is input to the signal processing unit 113.

[0026] The signal processing unit 113 has an ADC (Analog to Digital Converter) function 113a and a pixel addition function 113b. The ADC function 113a converts the analog video signal output from the image sensor 112 into a digital signal (hereinafter referred to as a video frame). The pixel addition function 113b applies filtering processing to each video frame to reduce noise and / or increase the brightness level (gain increase). For example, the pixel addition function 113b processes each video frame using the method shown in FIG. 2. FIG. 2 is a diagram for explaining the pixel addition processing by the signal processing unit.

[0027] In the example of Figure 2, n is a parameter indicating the horizontal position of a pixel, and m is a parameter indicating the vertical position of a pixel. Also, p(n,m) represents the pixel value of the pixel located at coordinates (n,m). For example, if the pixel value after filtering is P(n,m), the pixel addition function 113b calculates P(n,m) according to the following equation (1).

[0028]

number

[0029] In equation (1), a0, a1, a2, and a3 are filter coefficients. When a0, a1, a2, and a3 are all set to 1, the brightness level of the video frame increases. In addition, because the pixel values ​​of surrounding pixels are added, the above filtering process also acts as a low-pass filter, reducing high-frequency components. By changing the combination of filter coefficients, various filtering processes become possible. For example, by making the total value of the coefficients greater than 1, the filter can function as a filter with increased gain, and by making the total value less than 1, the filter can function as a noise reduction filter that reduces noise without increasing gain.

[0030] In the example of Fig. 2 and the above formula (1), for the sake of convenience of explanation, four pixels (two pixels horizontally and two pixels vertically) are targeted for filtering processing, but the combination of target pixels is not limited to this. As combinations of pixels targeted for filtering processing, for example, the seven filtering patterns shown in Table 1 below can be set. Note that the +0 dB pattern corresponds to a case where filtering processing by the pixel addition function 113b is not applied.

[0031] [Table 1]

[0032] Furthermore, the above formula (1) can be expanded as the following formula (2). mnis a filter coefficient applied to the pixel value p(m,n) at the coordinate position (m,n). By setting the filter coefficients of pixels other than the target pixel to 0, each of the above patterns can be expressed, and by setting the filter coefficient of the target pixel, the frequency characteristics of the video frame can be manipulated. The signal processing unit 113 calculates P(m,n) for all m and n using the pixel addition function 113b, and outputs a video frame having the calculated P(m,n) as the pixel value to the camera control unit 12.

[0033]

number

[0034] (Configuration of camera control unit) 1 again, the camera control unit 12 includes a control unit 120, a first black level correction unit 121, a frame accumulation unit 122, a second black level correction unit 123, a gain adjustment unit 124, and a video processing unit 125.

[0035] The control unit 120 controls the operations of the camera head unit 11 and the camera control unit 12 according to the first embodiment. The control unit 120 is a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPU (Graphic Processing Unit). The processor is connected to a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and / or a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). A computer program for implementing the functions of the control unit 120 can be stored in the memory and / or the storage device.

[0036] (Regarding the first black level correction) The first black level correction unit 121 corrects the black level of the video frame output from the camera head unit 11 (signal processing unit 113) using a preset correction value M1. The correction value M1 is the average brightness level of the video frame output from the camera head unit 11 in a light-shielded state. The black level can be corrected by subtracting the correction value from the brightness level of the video frame captured in a light-unshielded state. However, when the gain of the sensor output is increased, dark current noise is amplified, and therefore the appropriate black level correction value also changes depending on the magnitude of the gain.

[0037] In the case of the imaging device 10, the pixel addition function 113b of the signal processing unit 113 applies the above-described filtering process to the video frame. If the content of this filtering process changes, the magnitude of the black level to be corrected varies. Therefore, the first black level correction unit 121 obtains a correction value M1 corresponding to the content of the filtering process applied by the signal processing unit 113, and corrects the black level by subtracting the correction value M1 from the luminance level of the video frame.

[0038] In the example of FIG. 1, data on the correction value M1 is held by the first black level correction unit 121 as first correction data 121a. However, the first correction data 121a may be held by the control unit 120 and provided to the first black level correction unit 121. The first correction data 121a may have a structure as shown in FIG. 3, for example. FIG. 3 is a diagram showing an example of the first correction data. Note that, for ease of explanation, FIG. 3 shows the relationship between the contents of the filtering process (the seven filtering patterns shown in Table 1 above) and the correction value M1 in the form of a diagram, but the data structure is not limited to this example.

[0039] For example, when a filtering pattern (see FIG. 2) that simply adds two horizontal pixels and two vertical pixels is applied by the pixel addition function 113b of the signal processing unit 113, the first black level correction unit 121 corrects the black level of the video frame output from the signal processing unit 113 using a correction value M1 (+12 dB) corresponding to that pattern. Note that the correction value M1 corresponding to each filtering pattern can be obtained in advance using a filtered video frame captured in a light-blocking state. Furthermore, the filtering pattern used by the signal processing unit 113 may be determined by the control unit 120, and the correction value M1 applied by the first black level correction unit 121 may be specified by the control unit 120.

[0040] In the above explanation, a method has been introduced in which a correction value M1 corresponding to each filtering pattern is prepared in advance as the first correction data 121a, but as a modified example, a difference value between the correction value M1 (0+dB) when no filtering processing is applied and a correction value (for example, M1 (+12dB)) when another filtering pattern is applied may be held as the first correction data 121a. In this case, the first black level correction unit 121 uses the correction value M1 (0+dB) and the difference value to determine a correction value (for example, M1 (+12dB)) corresponding to the applied filtering pattern, and performs black level correction using the determined correction value.

[0041] (About frame accumulation) The video frame after black level correction by the first black level correction unit 121 is output to the frame accumulation unit 122. When the high-sensitivity shooting mode is on, the frame accumulation unit 122 accumulates multiple video frames in a frame memory and adds the multiple video frames together to generate a high-brightness video frame. When the high-sensitivity shooting mode is off, the frame accumulation unit 122 directly outputs the input video frames without adding them together.

[0042] 1, frame accumulation unit 122 has frame memories 122a and 122b and register 122c. In the following explanation, for ease of explanation, frame memory 122a may be referred to as side A, and frame memory 122b may be referred to as side B. However, please note that the notations "side A" and "side B" are merely used to distinguish between the two, and do not imply any relationship such as priority or higher / lower ranking.

[0043] The frame accumulation unit 122 has, for example, a configuration as shown in Fig. 4. Fig. 4 is a diagram schematically illustrating the configuration of the frame accumulation unit. Note that the configuration example in Fig. 4 is merely an example for the purpose of explanation, and that equivalent functions can be realized in implementation by appropriately combining hardware elements such as circuits and / or software elements, and that the scope of application of the technology according to the first embodiment is not limited to the example in Fig. 4.

[0044] In FIG. 4, as elements of the frame storage unit 122, in addition to frame memories 122a and 122b, switches SWin and SWout and adders 122d and 122e are shown.

[0045] The switch SWin is a switch for switching the frame memory that stores the video frames. When the switch SWin connects the input In to the frame storage unit 122 to the terminal Ain on the frame memory 122a side, the video frames are stored in the frame memory 122a (side A). When the switch SWin connects the input In to the frame storage unit 122 to the terminal Bin on the frame memory 122b side, the video frames are stored in the frame memory 122b (side B).

[0046] The switch SWout is a switch for switching the frame memory from which a video frame is read. When the switch SWout connects the terminal Aout on the frame memory 122a side to the output Out from the frame accumulation unit 122, a video frame is read from the frame memory 122a (side A). When the switch SWout connects the terminal Bout on the frame memory 122b side to the output Out from the frame accumulation unit 122, a video frame is read from the frame memory 122b (side B).

[0047] The adder 122d adds a video frame read from the frame memory 122a and a video frame input to the terminal Ain. The video frame after addition output from the adder 122d is written to the frame memory 122a. Similarly, the adder 122e adds a video frame read from the frame memory 122b and a video frame input to the terminal Bin. The video frame after addition output from the adder 122e is written to the frame memory 122b. The operation of the switches SWin and SWout and the timing of reading and writing the video frames are controlled by the control unit 120.

[0048] (Explanation of operation) Here, the operation of frame accumulation unit 122 will be specifically described with reference to Fig. 5 and Fig. 6. Fig. 5 is a timing chart for explaining an example of operation of the frame accumulation unit (when 2 frames are accumulated). Fig. 6 is a timing chart for explaining an example of operation of the frame accumulation unit (when 3 frames are accumulated).

[0049] The example in Fig. 5 shows a case where, after the accumulation operation of video frames in the frame memory is started, two video frames are added and output. Fig. 5 shows the output timing (frame period) from image sensor 112 [sensor output], the operation of frame memory 122a [frame memory (side A)], the operation of frame memory 122b [frame memory (side A)], the value of register 122c [register (number of accumulated frames)], and which side the video frame is output from [output video]. The horizontal axis of the timing chart shown in Fig. 5 is the time axis, and time progresses to the right.

[0050] The numbers written in the blocks of the sensor output represent frame periods. For example, a block marked k represents the kth frame period. In the example of FIG. 5, the accumulation operation of video frames begins from the kth frame period. The number of accumulated video frames, N, is 2. During the frame period before the accumulation operation begins, no video frames are added, and one video frame is output as is. In this case, the adders 122d and 122e do not operate, and data is written alternately to planes A and B, and the one video frame written is output as is.

[0051] For example, in the (k-2)th frame period, a video frame output from the camera head unit 11 is written to side B, and at the same timing, a video frame on side A is read out and output from the frame storage unit 122. In this case, the switch SWin connects the input In to the terminal Bin on the side B, and the switch SWout connects the terminal Aout on the side A to the output Out.

[0052] During the (k-1)th frame period, the video frame output from the camera head unit 11 is written to side A, and at the same time, the video frame on side B is read out and output from the frame storage unit 122. In this case, the switch SWin connects the input In to the terminal Ain on the side A, and the switch SWout connects the terminal Bout on the side B to the output Out. Note that before the start of the storage period, the value of the register 122c, which indicates the number of stored frames, remains 0.

[0053] When the accumulation operation starts, adders 122d and 122e start functioning and add video frames. During the kth frame period, which occurs immediately after the accumulation operation starts, a new video frame input to frame accumulation unit 122 is written to side B, and at the same time, the video frame on side A is read out and output from frame accumulation unit 122. At this time, switch SWin connects input In to terminal Bin on the side B, and switch SWout connects terminal Aout on the side A to output Out.

[0054] In the following (k+1)th frame period, the video frame on the B side is read out and input to the adder 122e. Also, a new video frame input to the frame storage unit 122 is input to the adder 122e and added to the video frame read out from the B side. The video frame added by the adder 122e is then written to the B side. At this time, the value of the register 122c (the number of stored frames) is rewritten to 1. During this period, the switch SWin connects the input In to the terminal Bin on the B side.

[0055] In this example, the number N of video frames to be added is 2, so when the value of register 122c becomes 1, the video frames stored in the frame memory are output. Therefore, the added video frame on side B is read out and output from frame storage unit 122. At this time, switch SWout connects terminal Bout on the side B to output Out. When frame storage is complete and the added video frames are output, the value of register 122c is reset to 0. Note that no processing is performed on side A during this frame period.

[0056] In the following (k+2)th frame period, the video frame on side B is read out and output from frame accumulation unit 122. Also, a video frame newly input to frame accumulation unit 122 is written to side A. At this time, switch SWin connects input In to terminal Ain on the A side, and switch SWout connects terminal Bout on the B side to output Out.

[0057] In the following (k+3)th frame period, the video frame on plane A is read out and input to adder 122d. Also, a new video frame input to frame storage unit 122 is input to adder 122d and added to the video frame read out from plane A. The video frame added by adder 122d is then written to plane A. At this time, the value of register 122c (number of stored frames) is rewritten to 1. During this period, switch SWin connects input In to terminal Ain on the A plane side.

[0058] In this example, the number N of video frames to be added is 2, so when the value of register 122c becomes 1, the video frames stored in the frame memory are output. Therefore, the added video frame on side A is read out and output from frame storage unit 122. At this time, switch SWout connects terminal Aout on the side A to output Out. When frame storage is complete and the added video frames are output, the value of register 122c is reset to 0. Note that no processing is performed on side B during this frame period.

[0059] Similarly, in the subsequent (k+4)th frame period and thereafter, the frame accumulation unit 122 accumulates video frames while switching between the A and B sides, and adds the video frames using the addition units 122d and 122e, thereby generating high-brightness video frames.

[0060] When the number N of video frames to be added is 3, the operation of frame accumulation unit 122 is as shown in Figure 6. The operation of frame accumulation unit 122 in the frame period before the start of the accumulation operation is the same as in the example of Figure 5. On the other hand, the operation in the (k+1)th and subsequent frame periods differs from the example of Figure 5. For example, in the (k+1)th frame period, the video frame on plane A is output from frame accumulation unit 122. Also, the process of reading out the video frame on plane B, inputting it to adder 122e, and adding it to the video frame newly input to frame accumulation unit 122 is the same as in the example of Figure 5, but the video frame after the addition is not output from frame accumulation unit 122 after being written to plane B. Also, the value of register 122c remains 1.

[0061] In the following (k+2)th frame, the video frame on side B is read out and input to adder 122e. Also, a new video frame input to frame storage unit 122 is input to adder 122e and added to the video frame read out from side B. The video frame added by adder 122e is then written to side B. At this time, the value of register 122c (number of stored frames) is rewritten to 2. During this period, switch SWin connects input In to terminal Bin on the side B.

[0062] In the example of FIG. 6, the number N of video frames to be added is 3, so when the value of register 122c reaches 2, the video frame stored in the frame memory is output. Therefore, the added video frame on side B is read out and output from frame storage unit 122. At this time, switch SWout connects terminal Bout on the side B to output Out. When frame storage is completed and the added video frame is output, the value of register 122c is reset to 0. Note that no processing is performed on side A during this frame period.

[0063] In the following (k+3)th frame period, the video frame on plane B is read out and output from frame accumulation unit 122. Also, a video frame newly input to frame accumulation unit 122 is written to plane A. At this time, switch SWin connects input In to terminal Ain on the A side, and switch SWout connects terminal Bout on the B side to output Out. In the following (k+4)th frame period and thereafter, frame accumulation unit 122 similarly accumulates video frames while switching between plane A and plane B, and adds the video frames using adders 122d and 122e, thereby generating a high-brightness video frame.

[0064] As described above, frame accumulation unit 122 accumulates video frames using frame memories 122a and 122b, and adds multiple video frames using adders 122d and 122e to generate high-brightness video frames. Furthermore, even during a frame period when a video frame is accumulated in one frame memory, a video frame is output from the other frame memory. Therefore, while a typical slow shutter function results in a period when no video is output, the imaging device 10 described above does not experience such a period.

[0065] Here, the function of the adders 122d and 122e will be further explained. One method of adding multiple video frames is to simply add the pixel values ​​of corresponding pixels in multiple video frames. As an applied example, there is also a method of weighting and adding the pixel values ​​of corresponding pixels. The pixel value of a pixel at position (n, m) in the j-th video frame is expressed as q(n, m, j), and the weighting coefficient for the pixel value of the j-th video frame is expressed as r(n, m, j). j Then, the pixel value Q(n, m, t) of the video frame after addition is given by the following equation (3).

[0066]

number

[0067] r0=r1=1, r jWhen (j≧2)=0, a video frame is obtained by simply adding two video frames. Also, the weighting coefficient r j If is greater than 1, a video frame with increased gain is obtained. j Depending on the setting of , it is possible to create a filter effect such as a low-pass filter. j As a method for incorporating the elements of the above into the adders 122d and 122e, for example, a weighting coefficient r j When adding video frames, the adders 122d and 122e calculate a weighting coefficient r according to the value of the register 122c. j is applied to each pixel value.

[0068] As described above, when the high-sensitivity shooting mode is on, the frame accumulation unit 122 generates a high-luminance video frame using multiple video frames. The video frame output from the frame accumulation unit 122 is then input to the second black level correction unit 123.

[0069] (Regarding invalid frame elimination) As described above, the frame accumulation unit 122 performs the addition process of video frames to obtain a high-brightness video frame. However, there is a possibility that an invalid video frame may be output from the camera head unit 11 for some reason, and adding such an invalid video frame will degrade the video quality. Note that recent image sensors are capable of outputting video at various timings, and may intentionally output invalid video frames that produce unwanted video (black). If such invalid video frames are used as is, they may become noise. Therefore, during the above-described accumulation operation, the frame accumulation unit 122 avoids adding invalid video frames among the video frames output from the camera head unit 11 (i.e., video frames generated from the output signal of the image sensor 112) and prevents them from being used to generate a high-brightness video frame. By applying this mechanism, it is possible to avoid degradation of video quality due to invalid video frames. Note that various factors may cause invalid video frames to be output, including, for example, noise contained in the output signal of the image sensor 112, as well as some quality degradation factors that occur during video processing by the signal processing unit 113 and / or the first black level correction unit 121. The reason for invalidation is not particularly limited in this embodiment.

[0070] (Regarding the second black level correction) As already explained, the black level of each video frame is corrected by the first black level correction unit 121 before the frame accumulation unit 122. However, since the frame accumulation unit 122 adds multiple video frames, there is a possibility that the black level may fluctuate due to amplification of remaining noise depending on the number of additions (number of accumulated frames). Therefore, in the first embodiment, the black level is further corrected by the second black level correction unit 123.

[0071] The second black level correction unit 123 corrects the black level of the video frame output from the frame accumulation unit 122 using a correction value M2 corresponding to the number of accumulated frames. The correction value M2 is set to the average brightness level of the video frame output from the frame accumulation unit 122 when a shooting operation is performed in a light-blocking state. By calculating the average brightness level while changing the number of accumulated frames, data of the correction value M2 corresponding to the number of accumulated frames (second correction data 123a) is obtained. FIG. 7 is a chart showing an example of the second correction data.

[0072] When the number of accumulated frames is 1 (for example, a frame period before the kth frame in FIG. 5), the video frame is outputted through, so there is no black level fluctuation due to the addition process, and the black level has already been corrected by the first black level corrector 121. Therefore, when the number of accumulated frames is 1, the second black level corrector 123 does not perform black level correction, and outputs the video frame outputted from the frame accumulation unit 122 to the gain adjustor 124 as is.

[0073] When the number of accumulated frames is two or more (for example, the (k+1)th frame period or later in FIG. 5), the second black level correction unit 123 acquires a correction value M2 corresponding to the number of accumulated frames and corrects the black level by subtracting the correction value M2 from the luminance level of the video frame. For example, when the number of accumulated frames is two, the second black level correction unit 123 corrects the black level of the video frame using the correction value M2(2) of the second correction data 123a. The video frame after the black level correction is output to the gain adjustment unit 124.

[0074] (Regarding gain adjustment) As described above, when imaging device 10 enters high-sensitivity shooting mode, multiple video frames are added together by the function of frame accumulation unit 122 to output a high-brightness video frame. The brightness level of the output video frame is multiplied by the number of added frames (doubled when adding two frames) compared to the brightness level of one video frame input to frame accumulation unit 122. Therefore, a gap in brightness level occurs when switching from an unadded video frame to a high-brightness video frame.

[0075] To eliminate the gap, the gain adjustment unit 124 gradually increases the gain of the unadded video frames during a period of several frames before the high-brightness video frame is output, adjusting the brightness level to approach that of the high-brightness video frame. The gain adjustment unit 124 also adjusts the brightness level of the video frames output during a period of several frames before the high-brightness video frame is output after the start of the accumulation operation. This gain adjustment allows for a smooth change in brightness level when switching to high-sensitivity shooting mode.

[0076] The gain adjustment unit 124 may be configured with a variable gain circuit in which the gain adjustment range can be set arbitrarily, or may be configured with a step gain circuit in which the gain adjustment range is set to a predetermined value (e.g., +6 dB). When a variable gain circuit is used, the gain adjustment range may be determined based on the luminance level of a high-luminance video frame (or the number of accumulated frames) and the gain adjustment period (the number of frame periods in which gain adjustment is performed). When a step gain circuit is used, the gain adjustment period (the number of frame periods in which gain adjustment is performed) may be determined based on the luminance level of a high-luminance video frame (or the number of accumulated frames).

[0077] Here, the operation of the gain adjustment unit 124 will be described in more detail with reference to Fig. 8 and Fig. 9. Fig. 8 is a timing chart for explaining an example of operation of the gain adjustment unit (in the case of accumulating two frames). Fig. 9 is a timing chart for explaining an example of operation of the gain adjustment unit (in the case of accumulating three frames).

[0078] The example in Figure 8 shows a case where, after starting the accumulation of video frames in the frame memory, two video frames are added together and output. In this case, the brightness level of the output high-brightness video frame is twice that of the input single video frame. Figure 8 shows the output timing (frame period) from image sensor 112 [sensor output], the operation of frame memory 122a [frame memory (plane A)], the operation of frame memory 122b [frame memory (plane A)], the value of register 122c [register (number of accumulated frames)], and which plane the video frame is output from [output video].

[0079] In the example of Fig. 8, similar to the example of Fig. 5 already described, the accumulation operation of video frames begins from the kth frame period. In the frame period before the accumulation operation begins, video frames are not added, and unadded video frames are output. On the other hand, after the accumulation operation begins, in the frame period after the first addition process is completed, high-luminance video frames are output. In the example of Fig. 8, high-luminance video frames are output in the (k+1)th and subsequent frame periods, and unadded video frames are output in the frame periods before the kth.

[0080] In the imaging device 10, when the high-sensitivity shooting mode is turned on, the accumulation operation is not started immediately, but a frame period (hereinafter, the gain adjustment period) for gain adjustment by the gain adjustment unit 124 is set. In the example of Fig. 8, the (k-6)th to kth frame periods are the gain adjustment period. In other words, the gain adjustment period is set to be the period consisting of a predetermined number of frame periods (6 in the example of Fig. 8) before the accumulation operation is started and a frame period until a high-luminance video frame is output after the accumulation operation is started.

[0081] 8, the amount of amplification applied in each frame period within the gain adjustment period is set to a constant value, and this constant value may be preset so that the same luminance level as that of a high-luminance video frame is reached in the frame period immediately prior to the start of the accumulation operation (the (k-1)th frame period in the example of FIG. 8). Also, this constant value may be preset so that the same luminance level as that of a high-luminance video frame is reached in the frame period at the end of the gain adjustment period (the kth frame period in the example of FIG. 8).

[0082] When the number N of video frames to be added is 3, the operation of the frame accumulation unit 122 and the gain adjustment unit 124 is as shown in FIG. 9. The operation of the frame accumulation unit 122 and the gain adjustment unit 124 before the start of the accumulation operation is the same as in the example of FIG. 8. After the accumulation operation starts, unadded video frames are output for two frame periods (the kth and (k+1)th frame periods in the example of FIG. 9) until a high-luminance video frame is output. Therefore, in the example of FIG. 9, the frame periods before the (k+1)th frame period are the gain adjustment periods. Note that the amount of amplification applied in each frame period within the frame adjustment period is the same as in the example of FIG. 8.

[0083] The video frames output from gain adjustment unit 124 are input to video processing unit 125. Video frames input before the gain adjustment period are through-output video frames with no gain adjustment, video frames input during the gain adjustment period are gain-adjusted video frames with no addition, and video frames input after the gain adjustment period are high-luminance video frames. Because the luminance levels of these video frames change smoothly, there are no gaps in the luminance levels of the output video, resulting in high-quality video with little discomfort.

[0084] The above explanation describes the operation when the high-sensitivity shooting mode is on. However, even when the high-sensitivity shooting mode is off, gaps in the brightness level can occur if gain adjustment is not performed. Therefore, contrary to the above-mentioned method, during a predetermined number of frame periods (second gain adjustment period) after the accumulation operation ends, the gain adjustment unit 124 gradually decreases the amplification amount so that the amplification amount becomes zero immediately after the second gain adjustment period, while increasing the gain of unadded video frames during each frame period within the second gain adjustment period. As a result, the brightness of the image changes smoothly even when the high-sensitivity shooting mode is off, so that gaps in the brightness level do not occur.

[0085] (Regarding processing and display by the video processing unit) The image processor 125 applies predetermined image processing such as AWB (Auto White Balance), gamma correction, and knee correction to the input image frames. As described above, unlike a slow shutter, the image capture device 10 is configured to increase the brightness level by adding image frames and output image frames during the accumulation period, so there is no period during which images are not output. Furthermore, gain adjustment by the gain adjuster 124 prevents sudden changes in brightness levels. This makes it possible to apply image processing such as AWB, thereby achieving higher image quality. Furthermore, black level correction by the first black level corrector 121 and the second black level corrector 123 further improves image quality.

[0086] The video processing unit 125 applies predetermined video processing to the video frames and then causes the display device 13 to display the series of video frames. Note that the video processing unit 125 may output the series of video frames to a video output interface in addition to or instead of the display device 13. As a variation, the video processing unit 125 may analyze the series of input video frames, detect objects (e.g., obstacles on a runway) included in the video frames, and display information about the detected objects on the display device 13. As another variation, the video processing unit 125 may output the series of video frames to a detection device that analyzes video and detects objects.

[0087] The configuration of the imaging device according to the first embodiment has been described above.

[0088] [1-2. Video frame processing flow] Next, the flow of video frame processing (processing by the camera control unit 12) according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart for describing the flow of video frame processing according to the first embodiment.

[0089] (S101) A video frame output from the camera head unit 11 is input to the first black level correction unit 121. The first black level correction unit 121 references the first correction data 121a and selects a correction value M1 corresponding to the filtering pattern applied by the signal processing unit 113 of the camera head unit 11. The first black level correction unit 121 then corrects the black level of the video frame using the selected correction value M1. The video frame after the black level correction is output to the frame accumulation unit 122.

[0090] (S102) The control unit 120 determines whether the high sensitivity shooting mode is selected. If the high sensitivity shooting mode is selected, the process proceeds to S103. On the other hand, if the high sensitivity shooting mode is not selected, the process proceeds to S107.

[0091] (S103) The control unit 120 determines whether it is a gain adjustment period. For example, when the high-sensitivity shooting mode is turned on, a predetermined number of frame periods set before the start of the accumulation operation and a frame period in which unadded video frames are output after the start of the accumulation operation are set as the gain adjustment period. If it is a gain adjustment period, the process proceeds to S104. On the other hand, if it is not a gain adjustment period, the process proceeds to S105.

[0092] (S104) Frame accumulation unit 122 outputs the video frames without adding them. Gain adjustment unit 124 increases the gain of the unadded video frames output from frame accumulation unit 122. At this time, gain adjustment unit 124 gradually increases the amount of amplification, and increases the gain of the unadded video frames so that they have the same luminance level as the high-luminance video frame to be output subsequently by the last frame period of the gain adjustment period. The gain-adjusted video frames are subjected to predetermined video processing by video processing unit 125 and then output to the outside. When the processing of S104 is completed, the series of processes shown in FIG. 10 ends.

[0093] (S105) Frame accumulation unit 122 reads out and outputs high-luminance video frames from the frame memories. Also, while frame accumulation unit 122 is storing video frames in one frame memory, it is outputting video frames read out from the other frame memory.

[0094] (S106) The video frame output from frame accumulation unit 122 in S105 is input to second black level correction unit 123. Second black level correction unit 123 references second correction data 123a and selects correction value M2 corresponding to the number of accumulated frames. Then, second black level correction unit 123 corrects the black level of the video frame using the selected correction value M2. The video frame after black level correction is output to video processing unit 125 without undergoing gain adjustment by gain adjustment unit 124, and is output to the outside after predetermined video processing is applied by video processing unit 125. When the processing of S106 is completed, the series of processes shown in FIG. 10 ends.

[0095] (S107) The video frame after the black level correction by the first black level correction unit 121 skips the processing by the frame accumulation unit 122, the second black level correction unit 123, and the gain adjustment unit 124, and is output to the outside after predetermined video processing is applied by the video processing unit 125. When the processing of S107 is completed, the series of processes shown in FIG. 10 ends.

[0096] The example in Figure 10 describes the operation when the high-sensitivity shooting mode is turned on, but the gain adjustment method during the second gain adjustment period described above changes when the high-sensitivity shooting mode is turned off. As already explained, during the second gain adjustment period, the gain of unadded video frames is increased while the amplification amount is gradually reduced, so that the amplification amount becomes 0 in the last frame period of the second gain adjustment period. This is different from the example in Figure 10, but those skilled in the art will easily understand the operation when the high-sensitivity shooting mode is turned off from the explanation in Figure 10.

[0097] The first embodiment of the present invention has been described above.

[0098] 2. Second Embodiment Next, the configuration of an imaging device according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a block diagram schematically showing the configuration of the imaging device according to the second embodiment. Note that the imaging device 10a shown in Fig. 11 is an example of the imaging device according to the second embodiment. Furthermore, the imaging device 10a does not have the slow shutter function described above.

[0099] As shown in Fig. 11, the imaging device 10a has a camera head unit 11, a camera control unit 12a, and a display device 13. The configuration of the camera head unit 11 is the same as that of the camera head unit 11 of the first embodiment. The configuration of the display device 13 is also the same as that of the display device 13 of the first embodiment. Therefore, detailed descriptions of the configurations of the camera head unit 11 and the display device 13 will be omitted. Furthermore, among the elements constituting the camera control unit 12a, elements having substantially the same functions as those of the elements constituting the camera control unit 12 of the first embodiment are assigned the same reference numerals.

[0100] Camera control unit 12a has control unit 120, first black level correction unit 121, frame accumulation unit 122, gain adjustment unit 124, and video processing unit 125. Camera control unit 12a differs from camera control unit 12 of the first embodiment in the presence or absence of second black level correction unit 123. Camera control unit 12a does not include second black level correction unit 123, and high-luminance video frames output from frame accumulation unit 122 are also input to video processing unit 125 without undergoing black level correction.

[0101] In the second embodiment, the second black level correction unit 123 is omitted, but sufficient image quality can be obtained depending on the application because the first black level correction unit 121 applies black level correction to each video frame before the frame accumulation unit 122. Furthermore, omitting the second black level correction unit 123 makes the circuit configuration slightly simpler during implementation, which may contribute to reducing design costs and manufacturing costs.

[0102] The configuration of the imaging device according to the second embodiment has been described above.

[0103] 3. Third Embodiment Next, the configuration of an imaging device according to a third embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram schematically illustrating the configuration of an imaging device according to the third embodiment. Note that the imaging device 10b shown in Fig. 12 is an example of the imaging device according to the third embodiment. Furthermore, the imaging device 10b does not have the slow shutter function described above.

[0104] 12, the imaging device 10b has a camera head unit 11, a camera control unit 12b, and a display device 13. The configuration of the camera head unit 11 is the same as that of the camera head unit 11 of the first embodiment. The configuration of the display device 13 is also the same as that of the display device 13 of the first embodiment. Therefore, detailed descriptions of the configurations of the camera head unit 11 and the display device 13 will be omitted. Furthermore, among the elements constituting the camera control unit 12b, elements having substantially the same functions as those of the elements constituting the camera control unit 12 of the first embodiment are assigned the same reference numerals.

[0105] Camera control unit 12b has control unit 120, first black level correction unit 121, frame accumulation unit 122, second black level correction unit 123, and video processing unit 125. Camera control unit 12b differs from camera control unit 12 of the first embodiment in the presence or absence of gain adjustment unit 124. Camera control unit 12b does not include gain adjustment unit 124, and unadded video frames output from frame accumulation unit 122 during the gain adjustment period are also input to video processing unit 125 without undergoing gain adjustment.

[0106] In the third embodiment, the gain adjustment unit 124 is omitted, so a gap in the brightness level occurs when a high-brightness video frame is first output. On the other hand, the first black level correction unit 121 and the second black level correction unit 123 correct the black level before and after the frame accumulation unit 122, significantly improving video quality compared to conventional methods. By omitting the gain adjustment unit 124, the circuit configuration becomes slightly simpler during implementation, which may contribute to reducing design costs and manufacturing costs. Therefore, if the application does not involve a gap in the brightness level, there is an advantage to adopting the configuration of the third embodiment.

[0107] The configuration of the imaging device according to the third embodiment has been described above.

[0108] <4. Notes> The above-described first to third embodiments are intended to explain the technical ideas described in the following supplementary notes, and in other words, the technical ideas described in the following supplementary notes can be derived from the above-described explanations of the first to third embodiments. Of course, it should be noted that the descriptions in the supplementary notes shown here are not intended to limit the technical scope of the present invention, but are intended to aid in understanding the technical ideas disclosed in this specification.

[0109] [Appendix 1] a signal processing unit that generates a video frame from an output signal of the image sensor; a frame storage unit that generates a high-luminance image frame using the plurality of image frames output from the signal processing unit when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame; a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame; An imaging device comprising: [Appendix 2] The frame storage unit a first frame memory and a second frame memory; storing a plurality of video frames output from the signal processing unit in one of the first frame memory and the second frame memory to generate the high-luminance video frame, outputting the video frame in the other frame memory during the storage operation, and after outputting the high-luminance video frame, switching the frame memory used for storage with the frame memory used for output during the storage operation. 2. The imaging device of claim 1. [Appendix 3] the number of video frames whose luminance levels are adjusted by the gain adjustment unit is determined based on the number of video frames used to generate the high-luminance video frame and a preset adjustment amount of luminance level to be applied to one video frame. 3. The imaging device according to claim 1 or 2. [Appendix 4] The storage operation by the frame storage unit includes a process of adding together the video frame to be stored output from the signal processing unit and the video frame read from the frame memory used for storage, with a weighting factor applied thereto, and a process of writing the video frame after the addition into the frame memory used for storage. 3. The imaging device according to claim 2. [Appendix 5] the frame storage unit does not use invalid video frames among the video frames generated from the output signal of the image sensor for generating the high-luminance video frames; 5. The imaging device according to any one of Supplementary Notes 1 to 4. [Appendix 6] a first black level correction unit that corrects a black level of a video frame output from the signal processing unit and outputs the corrected video frame to the frame storage unit; when the high-sensitivity shooting function is turned on, the frame storage unit generates a high-luminance video frame using the plurality of video frames output from the first black level correction unit. 6. The imaging device according to any one of Supplementary Notes 1 to 5. [Appendix 7] a second black level correction unit that corrects the black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame; 7. The imaging device according to claim 6. [Appendix 8] the first black level correction unit corrects the black level of the video frame output from the signal processing unit using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit; 8. The imaging device according to claim 6 or 7.

[0110] [Appendix 9] a signal processing unit that applies filtering to video frames generated from the output signals of the image sensor; a first black level correction unit that applies black level correction to the video frame after the filtering process is applied; a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on; An imaging device comprising: [Appendix 10] a second black level correction unit that corrects the black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame; 10. The imaging device according to claim 9. [Appendix 11] the first black level correction unit corrects the black level of the video frame output from the signal processing unit using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit; 11. The imaging device according to claim 9 or 10. [Appendix 12] the signal processing unit applies the filtering process to a pixel group in a rectangular region including a target pixel in the video frame, a pixel group arranged in a horizontal direction including the target pixel, or a pixel group arranged in a vertical direction including the target pixel, by multiplying the pixel group by a filter coefficient and performing an addition process; 12. The imaging device according to any one of Supplementary Notes 9 to 11.

[0111] [Appendix 13] When the high-sensitivity photographing function is turned on, a high-luminance image frame is generated using a plurality of image frames generated from the output signal of the image sensor, and the image frame to be output is switched to the high-luminance image frame; adjusting the luminance levels of the video frames output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance levels gradually approach the luminance level of the high-luminance video frame; A video processing method in which a computer executes a process including the steps of:

[0112] [Appendix 14] applying a filtering process to a video frame generated from the output signal of the image sensor; applying black level correction to the filtered video frames; When the high-sensitivity shooting function is turned on, outputting a high-brightness video frame generated using the plurality of video frames after the black level correction is applied; A video processing method in which a computer executes a process including the steps of:

[0113] [Appendix 15] a frame storage unit that generates a high-luminance image frame using a plurality of image frames generated from an output signal of the image sensor when the high-sensitivity image capturing function is turned on, and switches the image frame to be output to the high-luminance image frame; a gain adjustment unit that adjusts the luminance level of the video frame output during a plurality of frame periods before switching to the high-luminance video frame so that the luminance level gradually approaches the luminance level of the high-luminance video frame; A video processing device comprising:

[0114] [Appendix 16] a first black level correction unit that applies black level correction to the video frame that has been generated from the output signal of the image sensor and that has been subjected to the filtering process; a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on; A video processing device comprising:

[0115] While the preferred embodiments of the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that those skilled in the art can conceive of various modifications and alterations within the scope of the claims, and these modifications and alterations naturally fall within the technical scope of the present invention. [Explanation of symbols]

[0116] 10, 10a, 10b Imaging device 11 Camera head 12, 12a, 12b Camera control unit 13 Display device 111 Optical system 112 Image Sensor 113 Signal Processing Unit 120 control section 121 First black level correction section 122 Frame storage unit 123 Second black level correction section 124 Gain adjustment section 125 Video Processing Unit

Claims

1. a signal processing unit that applies filtering to video frames generated from the output signals of the image sensor; a first black level correction unit that applies black level correction to the video frame after the filtering process; a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on; An imaging device comprising:

2. a second black level correction unit that corrects a black level of the high-luminance video frame by using a second correction value corresponding to the number of video frames used to generate the high-luminance video frame; The imaging device according to claim 1 .

3. the first black level correction unit corrects a black level of the video frame output from the signal processing unit by using a first correction value corresponding to the content of the filtering process applied to the video frame by the signal processing unit; 3. The imaging device according to claim 1.

4. the signal processing unit applies the filtering process to a pixel group in a rectangular region including a target pixel in the video frame, a pixel group arranged in a horizontal direction including the target pixel, or a pixel group arranged in a vertical direction including the target pixel, by multiplying the pixel group by a filter coefficient and performing an addition process; The imaging device according to any one of claims 1 to 3.

5. applying a filtering process to a video frame generated from the output signal of the image sensor; applying black level correction to the filtered video frames; When the high-sensitivity shooting function is turned on, outputting a high-brightness video frame generated using the plurality of video frames after the black level correction is applied; A video processing method in which a computer executes a process including the steps of:

6. a first black level correction unit that applies black level correction to the video frame that has been generated from the output signal of the image sensor and that has been subjected to the filtering process; a frame storage unit that outputs a high-brightness image frame generated using the plurality of image frames after the black level correction is applied when the high-sensitivity image capturing function is turned on; A video processing device comprising:

Images