Imaging control device, image processing device, imaging device, method for driving imaging device, and program

Abstract

To provide a technique capable of enabling a wide dynamic range imaging in which an exposure condition is set for each region while reducing the influence of flicker on imaging.SOLUTION: An imaging control device that causes an imaging element having a plurality of pixel blocks each including a plurality of pixels to perform imaging of a moving image, includes a control part that controls the imaging element so that one frame period in the moving image includes a first imaging period in which a charge accumulation time is controlled based on a first exposure condition determined for each of the plurality of pixel blocks, and a second imaging period including a charge accumulation time based on a second exposure condition set in advance.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to an imaging control device, an image processing device, an imaging device, a method for driving an imaging element, a device, and a program.

Background Art

[0002] In order to widen the dynamic range of an imaging device, a method of partially changing the exposure of an imaging sensor has been proposed. In Patent Document 1, it is disclosed that the entire light-receiving area of the imaging sensor is divided into a plurality of areas, and the exposure time, which is the charge accumulation time in the sensor, and the amplification gain for the analog signal from the sensor are set for each area. Patent Document 2 describes a technique of setting sensitivity and exposure for each area and lengthening the exposure time when imaging a flickering light source.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to address problems (LED flicker) caused by a periodically emitting light source, for example, a light-emitting diode (LED), an exposure time longer than the emission period of the LED is required. When the exposure time is lengthened, the sensitivity of the imaging sensor can be adjusted to prevent saturation of the imaging sensor. However, since there is a limit to the adjustment range of the sensitivity, the influence of LED flicker may occur. An object of the present disclosure is to provide a technique that can enable wide dynamic range imaging in which exposure conditions are set for each area while reducing the influence of flicker on imaging.

Means for Solving the Problems

[0005] An imaging control device according to an aspect of the present disclosure is an imaging control device that causes an image sensor having a plurality of pixel blocks each including a plurality of pixels to capture a moving image, wherein a charge accumulation time is controlled based on a first exposure condition determined for each of the plurality of pixel blocks during a one-frame period in the moving image in a first imaging period, and a second imaging period including a charge accumulation time based on a preset second exposure condition, and a control unit that controls the image sensor to include the second imaging period.

Advantages of the Invention

[0006] According to the present disclosure, it is possible to provide a technique capable of performing wide dynamic range imaging in which exposure conditions are set for each region while reducing the influence of flicker on imaging.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and duplicate explanations are omitted.

[0009] [Embodiment 1] FIG. 1 is a block diagram showing the connection between a schematic configuration example of an imaging device 100 to which the imaging control device according to the present embodiment is applied and an external controller 10. The imaging device 100 of the present embodiment has various configurations that a general imaging device has. However, for the sake of simplicity of illustration and description, FIG. 1 shows only the main constituent parts according to the present embodiment. In addition, each constituent part described below is an example, and the functions of the respective constituent parts described later may be appropriately combined into one or separated. Alternatively, one constituent part may also serve as another constituent part. Further, the constituent part can also be provided outside the imaging device 100, for example, in the controller 10.

[0010] The imaging device 100 according to the present embodiment may have a synchronization control unit 101, an imaging element 103, an A / D conversion unit 104, and an exposure correction unit 105. Further, the imaging device 100 may have a gradation conversion unit 106, an image output unit 108, an exposure time control unit 109, a gain control unit 110, an exposure condition calculation unit 111, and a control unit 150.

[0011] Each of the synchronization control unit 101, the exposure time control unit 109, and the gain control unit 110 can be said to be an imaging control device that controls imaging. The imaging control device may include at least any one of the synchronization control unit 101, the exposure time control unit 109, and the gain control unit 110. Each of the exposure correction unit 105 and the gradation conversion unit 106 functions as an image processing device that performs image processing on the exposure image 122. The image processing device may include at least either the exposure correction unit 105 or the gradation conversion unit 106. Further, the imaging device 100 may include a control unit 150 that controls the imaging device 100. The control unit 150 may be provided in each part of the imaging device 100, or may control a part or the whole of the imaging device 100.

[0012] An exposure time 112 and an analog gain value 113 can be set in the imaging device 100. The imaging element 103 is provided with a photoelectric conversion element that converts light into charges and accumulates them. Further, in order to reflect the settings given from the external controller 10, it has a serial input / output interface (SIO I / F) 141, a memory or a register 142. The imaging sensor 102 may include the imaging element 103 and the A / D conversion unit 104. In the present embodiment, the imaging device 100 can be connected to an external controller 10 by a serial communication line 11 and an output signal line 12.

[0013] Regarding the imaging device 100, an outline of each component will be described starting from the imaging element 103. The imaging element 103 has an imaging area (light receiving area). The imaging area is divided into a plurality of areas called pixel blocks. The imaging element 103 can be driven in units of pixel blocks (areas), and has a function of performing an exposure operation (charge accumulation) with different exposure times for each area. Here, the exposure time corresponds to the charge accumulation time during which the photoelectric conversion element accumulates charges. Further, the exposure operation corresponds to accumulating charges.

[0014] The pixel block will be described later with reference to FIG. 2. The pixel block includes a plurality of pixels. A plurality of pixel blocks are arranged in the image sensor 103. In the case of this embodiment, the charge accumulation time for accumulating charges in each region of each pixel block is set in the image sensor 103 by an exposure control signal 117 supplied from the exposure time control unit 109. The image sensor performs exposure with the charge accumulation time set for each of those regions. The exposure time control unit 109 controls the image sensor. Note that since this charge accumulation time is the time associated with imaging, it can be called the exposure time.

[0015] The exposure control signal 117 is a signal for setting the exposure time for each region in each region of the image sensor 103. Then, the image sensor 103 reads out the charges accumulated in each pixel with the exposure time set by the exposure control signal 117 for each region, and outputs them as pixel potentials 118 to the A / D conversion unit 104.

[0016] The A / D conversion unit 104 performs analog / digital conversion on the pixel potential 118 read from the image sensor 103 to convert it into a digital value. In the case of this embodiment, an analog gain 121 corresponding to each region is set in the A / D conversion unit 104 by the gain control unit 110. The analog gain 121 is an amplification gain value for the pixel potential 118. The A / D conversion unit 104 amplifies the pixel potential 118 output from the image sensor 103 with the analog gain 121 determined for each region, and then performs analog / digital conversion to convert it into a digital value. Hereinafter, an image composed of digital signals that have been amplified by the analog gain 121 for each region and then analog / digitally converted in the A / D conversion unit 104 is called an exposure image 122. The exposure image 122 output from the A / D conversion unit 104 is sent to the exposure condition calculation unit 111 and the exposure correction unit 105.

[0017] Based on the exposure image 122, the exposure condition calculation unit 111 calculates and updates the exposure time 112 and the analog gain value 113 for each region so that the imaging is under optimal conditions. For example, the exposure condition calculation unit 111 obtains a histogram of pixel values for each pixel block based on the luminance distribution of the exposure image 122. Then, if the pixel values are distributed on the bright side, the exposure condition calculation unit 111 changes (updates) the exposure time 112 and the analog gain value 113 of that pixel block (region) to the set values for darker imaging. Also, if the pixel values are distributed on the dark side, the exposure condition calculation unit 111 changes (updates) the exposure time 112 and the analog gain value 113 of that pixel block (region) to the set values for brighter imaging. And the value of the exposure time 112 for each region is sent to the exposure time control unit 109 and the exposure correction unit 105. The analog gain value 113 for each region is sent to the gain control unit 110 and the exposure correction unit 105.

[0018] The synchronization control unit 101 generates a synchronized exposure time output pulse 120 and a gain output pulse 114. The synchronization control unit 101 outputs the generated exposure time output pulse 120 to the exposure time control unit 109. The synchronization control unit 101 outputs the generated gain output pulse 114 to the gain control unit 110. Thus, the synchronization control unit 101 performs control to synchronize the processing of the exposure time control unit 109 and the processing of the gain control unit 110.

[0019] The exposure time output pulse 120 is a signal for controlling the timing at which the exposure time control unit 109 outputs the exposure control signal 117 to the imaging device 103. The exposure time control unit 109 outputs the exposure control signal 117 to the imaging device 103 based on the exposure time output pulse 120, thereby changing the exposure time for each arbitrary pixel block of the imaging device 103.

[0020] Also, the gain output pulse 114 is a signal for the gain control unit 110 to control the timing at which the analog gain 121 is output to the A / D conversion unit 104. Based on the gain output pulse 114, the gain control unit 110 outputs the analog gain 121 to the A / D conversion unit 104, thereby changing the gain for amplifying the pixel potential of the signal obtained from each pixel for each pixel block. Thus, in this embodiment, the synchronization control unit 101 synchronizes the exposure time control unit 109 and the gain control unit 110 to perform operation control, so that an exposure image 122 with appropriately changed exposure time and analog gain can be output for each pixel block of the image sensor 103.

[0021] The exposure time control unit 109 generates an exposure control signal 117 for each region based on the exposure time output pulse 120 and the value of the exposure time 112 for each region, and outputs it to the image sensor 103. Thereby, the exposure time corresponding to the exposure time 112 for each region is set for the image sensor 103 at an appropriate timing.

[0022] The gain control unit 110 outputs the analog gain value 113 for each region as the analog gain 121 for each region with respect to the pixel potential 118 for each region of the image sensor 103 to the A / D conversion unit 104 in accordance with the timing of the gain output pulse 114. Thereby, in the A / D conversion unit 104, the pixel potential 118 for each region is amplified by the analog gain 121 for each region corresponding to the region, and then analog / digital conversion is performed. The data after the analog / digital conversion is sent as the exposure image 122 for each region to the exposure correction unit 105 and the exposure condition calculation unit 111.

[0023] The exposure correction unit 105 accumulates the exposure images 122 taken with different exposure conditions within the same frame for the exposure image 122 for each area sent from the A / D conversion unit 104, performs necessary processing, and then conducts an addition process for each pixel data. Further, tone expansion processing based on the exposure time 112 and the analog gain value 113 is performed on the image after the addition process to generate a tone-expanded image 123. The exposure correction unit 105 generates a tone-expanded image 123 represented by 23 bits from the exposure image 122 for each area represented by, for example, 10 bits. The detailed operation of the exposure correction unit 105 will be described later. Then, the generated tone-expanded image 123 is sent to the tone conversion unit 106.

[0024] The tone conversion unit 106 performs tone conversion on the tone-expanded image 123 and outputs the tone-converted image 124 to the image output unit 108. In this embodiment, it is assumed that the tone conversion is a process of generating, for example, a 12-bit tone-converted image 124 by gamma conversion of the 23-bit tone-expanded image 123. Note that the tone conversion process in this embodiment is performed to suppress the data rate in the subsequent process. In this embodiment, the bit lengths of the exposure image 122 and the tone-converted image 124 are 10 bits and 12 bits respectively, but these bit lengths are merely examples and are not limited thereto.

[0025] The image output unit 108 outputs the tone-converted image 124 to the configuration at the subsequent stage of the imaging device 100 or to the outside. In this example, the controller 10 is connected as a processing module that receives image data from the imaging device 100. Here, an LVDS signal line having 16 data channels can be used as the output signal line 12 connecting the image output unit 108 and the controller 10. However, the type of this signal line and the data channel width are not limited by this example.

[0026] Further, the controller 10 is connected to the serial input / output (SIO) interface I / F 141 of the imaging device 100 via a serial communication line 11. The SIO I / F 141 is connected to a register 142, and the controller 10 can set necessary information in the register 142 inside the imaging device 100 via the SIO I / F 141. The information set in the register 142 is transmitted to the exposure condition calculation unit 111, enabling control of the calculation of exposure conditions.

[0027] FIG. 2 is a diagram for explaining a configuration example of the imaging element 103. The imaging area of the imaging element 103 has a plurality of pixel blocks 201. Further, the pixel block 201 has a plurality of pixels 202. In the present embodiment, it is assumed that the number of pixels in the width 206 direction (horizontal line direction) of the imaging area of the imaging element 103 is 2000 pixels, and the number of pixels in the height 205 direction is 1000 pixels (that is, the number of horizontal lines in the vertical direction is 1000 lines). Also, it is assumed that the number of pixels in the width 204 direction (horizontal line direction) of the pixel block 201 is 100 pixels, and the number of pixels in the height 203 direction is 100 pixels (for 100 lines of horizontal lines in the vertical direction). In this case, the number of pixel blocks 201 in the imaging area of the imaging element 103 is 20 in the horizontal direction and 10 in the vertical direction. Note that these numbers of pixels and lines are merely examples and are not limited thereto.

[0028] Also, the pixel blocks [0,0] to [19,9] described in each pixel block 201 in FIG. 2 represent the positions of the respective pixel blocks 201 in the imaging area, and the values within the brackets [ ] represent the horizontal and vertical indices of each pixel block in the imaging area. In FIG. 2, for example, in the case of the pixel block 201 located in the upper right of the imaging element 103, it is the pixel block [19,0]. Also, a set of pixel blocks represented by the same vertical index will be referred to as a block row.

[0029] That is, block row N consists of pixel blocks [0,N] to [19,N]. For example, block row 5 consists of pixel blocks [0,5] to [19,5]. Note that the sizes (number of pixels in the vertical and horizontal directions) of each of the imaging device 103 and the pixel block 201 are not limited to the above example. Also, the shape and aspect ratio of the pixel 202 are not limited, and for example, it may be rectangular instead of square. Furthermore, the pixel block 201 may be composed of only one pixel 202. And in this embodiment, in each pixel block 201, the exposure time and the analog gain are controllable.

[0030] Here, the exposure time corresponds to the time during which charges are accumulated in the pixels (light-receiving elements) of the imaging device 103 during imaging. Therefore, for example, assuming that the amount of incident light on the imaging device 103 is the same and the pixels do not saturate, the longer the exposure time, the higher the pixel potential 118, and the brighter the imaging can be. That is, when the amount of incident light is the same and pixel saturation is not considered, for example, comparing the case where the exposure time is (1 / 480) second and the case where it is (1 / 30) second, imaging can be brighter in the case of (1 / 30) second.

[0031] The analog gain represents the gain for amplifying the pixel potential 118 before digital conversion in the A / D conversion unit 104 during imaging. Therefore, the larger the value of the analog gain, the larger the input to the A / D conversion unit 104, and thus the larger the digital value output from the A / D conversion unit 104.

[0032] Returning to FIG. 1, the configuration and driving method of the imaging device 100 of this embodiment will be described. The imaging device 103 performs imaging with the exposure time controlled in units of pixel blocks 201 based on the exposure control signal 117. Then, the imaging device 103 outputs a pixel potential 118 corresponding to the charges accumulated for each pixel.

[0033] The A / D conversion unit 104 performs digital conversion on the pixel potential 118 output from the imaging device 103 after amplifying it with the analog gain 121 set corresponding to each pixel block of the imaging device 103, and outputs the exposure image 122. In this embodiment, it is assumed that the exposure image 122 is a 10-bit digital value. Also, the analog gain 121 can take four values, for example, as gain values, ×1 times, ×2 times, ×4 times, and ×8 times.

[0034] The exposure correction unit 105 performs tone expansion processing on the exposure image 122 for each region input from the A / D conversion unit 104 based on the exposure time 112 and the analog gain value 113, and outputs the tone-expanded image 123. The exposure correction unit 105 recognizes under what conditions the exposure image 122 for each region is captured by the exposure time 112 for each region and the analog gain value 113 for each region. Then, the exposure correction unit 105 corrects the exposure image 122 for each region based on the conditions under which the exposure image 122 for each region was captured.

[0035] The exposure correction unit 105 performs tone expansion processing based on the exposure time 112 and the analog gain value 113 on the exposure image 122 for each region sent from the A / D conversion unit 104 to generate the tone-expanded image 123. For example, the exposure correction unit 105 recognizes under what conditions the input exposure image 122 for each region is captured from the exposure time 112 for each region and the analog gain value 113 for each region, and corrects the exposure image 122 for each region according to the conditions.

[0036] Also, the exposure correction unit 105 generates the tone-expanded image 123 represented by 23 bits by performing tone expansion processing on the exposure image 122 for each region, for example, where the number of bits is represented by 10 bits. Then, the generated tone-expanded image 123 is sent to the tone conversion unit 106. For example, the exposure correction unit 105 recognizes under what conditions the input exposure image 122 for each region is captured from the exposure time 112 for each region and the analog gain value 113 for each region, and corrects the exposure image 122 for each region according to the conditions.

[0037] Next, the operation of the exposure correction unit 105 will be described. FIG. 3 is a block diagram showing a configuration example of the exposure correction unit 105. The exposure correction unit 105 includes a line buffer 301, an addition ratio calculation unit 302, an image addition unit 303, and a tone expansion unit 304. The line buffer 301 can be used to delay data and align the time when images are synthesized later.

[0038] How each component of the exposure correction unit shown in FIG. 3 operates will be described with reference to FIG. 4. FIG. 4 is a diagram showing the relationship between the control of the exposure time and the emission cycle of the LED, which is the light source of the imaging target, assuming a light-emitting diode (LED) as a periodic light source in the present embodiment.

[0039] In the present embodiment, the output frame rate of the image when the imaging device 100 captures a moving image is set to 30 frames / second. In this case, the frame period of the moving image is 1 / 30 second, and the period of one frame is approximately 33.3 ms (hereinafter referred to as 33.3 ms). Also, assume that the LED included in the imaging target emits light periodically with a period of 90 Hz and a duty of 50%. For the sake of explanation, FIG. 4 shows two temporally continuous frames, frame 1 and frame 2. The imaging period of frame 1 is divided into 402-1 and 401-1. 402-1 has a length of period 2, which is a predetermined period determined to be longer than the emission cycle of the LED. In this embodiment, the predetermined period is the same length as 1 / 90 second (about 11.1 ms, hereinafter referred to as 11.1 ms) of the emission cycle of the LED. 404-1 is the exposure time of the imaging element in period 2. In period 2, the exposure time of the imaging element is always set for the entire period 2.

[0040] As can be seen from FIG. 4, since period 2 is set to be equal to or longer than the light emission period of the LED, it includes the period during which the LED emits light (the convex portion of LED light emission in FIG. 4). Period 1 of 401-1 is the period of the imaging period of frame 1 excluding 402-1. Period 1 of 401-1 is the period during which area-by-area exposure control is performed according to the brightness of the object being imaged. 403-1 is the exposure time for 401-1 set by area-by-area exposure control, and in this embodiment, it is half the time of 401-1. The same control is also performed for frame 2 following frame 1. In frame 2, the exposure time 403-2 of period 1 is set to 1 / 8 of the time of 401-2 as a result of area-by-area exposure control. Therefore, during the period of 403-2, the light emission period of the LED is not captured by the imaging device. When the light emission period of the LED is not captured by the imaging device, flickering may occur in the captured video.

[0041] The control of the exposure time in period 1 and period 2 of this imaging device is implemented by the exposure time control unit 109 based on the exposure time 112 determined by the exposure condition calculation unit 111 in FIG. 1. The length of period 2 can be used by the exposure condition calculation unit 111 as the light emission period of the LED to be flicker-prevented, which is set in the register 142 from the external controller 10. Also, the exposure time in period 1 can be calculated based on the exposure image 122 in period 1 of the previous frame according to the exposure condition calculation algorithm for area-by-area exposure. Note that the length of period 2 can be determined according to the destination where this imaging device is used as a product, and can be set in advance as data read by the external controller 10. Also, it is possible to determine the destination using position information such as GPS, and at the time of startup of the imaging device, the external controller 10 refers to a table according to the position information and sets it in the register 142.

[0042] In FIG. 3, first, the exposure image 122 of period 2 of frame 1 in FIG. 4 is input to the exposure correction unit 105. Here, the image data of the exposure image 122 of period 2 of a certain frame at a certain pixel position is referred to as O2. After the elapse of period 2, the data of O2 is first stored in the line buffer 301. After the elapse of the time of period 1, the exposure image 122 of period 1 of frame 1 is input. The image data of the exposure image 122 of period 1 of the same frame 1 at this time is referred to as O1.

[0043] When O1 is supplied to the exposure correction unit 105 as the exposure image 122, the image addition unit 303 reads out the data of O2 from the line buffer 301 as the delayed image 306. The data holding period in the line buffer 301 at this time is the length of period 1. Then, based on the addition ratio k(305) calculated by the addition ratio calculation unit 302, the composite output O3, which is the composite image signal of O1 and O2, is calculated based on the following formula (1). Here, the addition ratio k is a value for weighted addition and correction of the image data O1 and the image data O2 based on their respective imaging conditions.

[0044] O3 = O1 + k × O2 ··· (Formula 1) Next, the calculation method of the addition ratio k of the addition ratio calculation unit 302 will be described with reference to FIG. 5. FIG. 5 is a table showing the relationship between the exposure conditions, exposure time, and analog gain in the area-by-area exposure control. This table is applicable when the frame rate is 30 fps and shows the relationship between the exposure time and analog gain for the entire one-frame period. In FIG. 5, the numbers existing at the intersection positions of the analog gain and exposure time in the horizontal direction of the table are referred to as EV values here. The EV value indicates, as a power-of-2 value, the ratio of the brightness of the imaging object based on the difference in exposure conditions when the signal levels (pixel values) of the pixels at the time of imaging are the same. When the brightness of the imaging object imaged in the case where the analog gain is ×8 and the exposure time is 1 / 30 second is set as L0, the brightness LE of the imaging object when imaged with the EV value E and the same pixel value is obtained can be expressed by the following formula 2.

[0045] LE = L0 × 2^E ··· (Formula 2) For example, when the analog gain is ×1 and the exposure time is 1 / 30 second, the EV value is 3. Similarly, when the analog gain is ×1 and the exposure time is 1 / 60 second, the EV value is 4. At this time, if the pixel values obtained by imaging are the same, it indicates that the imaging target captured under the EV value 4 condition has twice the brightness compared to the imaging target captured under the EV value 3 condition.

[0046] In addition, in the case of area-by-area exposure, as shown in the table in FIG. 5, the brightness of the imaging target in that area is estimated from the value of the pixel imaged in the previous frame and the exposure conditions used for imaging at that time, and the exposure conditions to be used for imaging in the next frame can be calculated.

[0047] Here, it is assumed that the exposure conditions used in period 402-1 of FIG. 4 are analog gain ×1 and the exposure time is fixed throughout period 2. Considering that this exposure condition is applied to the entire frame, the exposure time becomes 1 / 30 second for the entire frame, and the EV value in this case is 3 from FIG. 5. On the other hand, the exposure time used in 401-1 is half of period 1. Similarly, considering that this exposure condition is applied to the entire frame, the exposure time becomes 1 / 60 second, which is half of the entire frame. If the analog gain is ×4 at this time, the EV value at this time is 2 from FIG. 5.

[0048] This means that in the condition where it is originally judged that it is appropriate to use an EV value of 2 for area-by-area exposure, 402-1 was imaged with an EV value of 3. In the tone expansion unit 304 described later, the brightness of the pixel value is converted based on the EV value of 2 for the area-by-area exposure condition. Therefore, the value of O2 imaged with an EV value of 3 is converted to a value corresponding to when it is imaged with an EV value of 2 and then added to O1 to obtain the pixel value for the entire one-frame period. The coefficient for this conversion is k. k can be obtained by the following formula 3.

[0049] k = 2^(EV value of period 2) ÷ 2^(EV value of period 1) = 2^(EV value of period 2 - EV value of period 1) ··· (Formula 3) From formula 3, the conversion coefficient k1 of frame 1 in this embodiment is k1 = 2^(3 - 2) = 2^1 = 2 is obtained.

[0050] Similarly, consider Frame 2. The exposure time 403 - 2 of Frame 2 is 1 / 8 the length of Period 1. This corresponds to the exposure time 1 / 240 in FIG. 5. Assuming the analog gain at this time is ×1, the EV value of the exposure time 401 - 2 is 7. The EV value of 402 - 2 is 3, similar to 402 - 1, because the exposure conditions in Period 2 are set uniformly under the same conditions. The conversion coefficient k2 of Frame 2 at this time is k2 = 2^(3 - 7) = 2^(-4) = 1 / 16 is obtained.

[0051] In FIG. 3, the composite output O3 of O1 and O2 calculated based on Equation 1 by the image addition unit 303 is transmitted to the tone expansion unit 304. In the tone expansion unit 304, the EV value shown in FIG. 5 is obtained from the exposure time 112 and the analog gain value 113 of the corresponding pixel block. Assuming the EV value at that time is E, the pixel value after tone expansion can be obtained by Equation 2.

[0052] For example, in Period 1 of Frame 1 shown in FIG. 4, the exposure condition was that the exposure time corresponded to 1 / 60 second when the length of Period 1 was converted to the entire Frame 1, and the analog gain was ×4. Since the EV value at this time is 2, if the composite output after the addition process of Frame 1 is O 31 and the pixel value after tone expansion is O 41 then O 41 = O 31 ×2^2 = 4×O 31 is obtained.

[0053] Similarly, in Period 1 of Frame 2 shown in FIG. 4, the exposure condition was that the exposure time corresponded to 1 / 240 second when the length of Period 1 was converted to the entire Frame 2, and the analog gain was ×1. Since the EV value at this time is 7, if the composite output after the addition process of Frame 2 is O 32 and the pixel value after tone expansion is O 42 then O42 =O 32 ×2^7 = 128×O 32 This results in

[0054] At this time, assuming that 32 the bit width of the original exposure image of O is 10 bits, then O 42 becomes a 17-bit number. In the case of this embodiment, it can be seen from FIG. 5 that the EV value can take a maximum of 13. In that case, when tone expansion is performed by the exposure correction unit 105, the bit width of the data increases. In this embodiment, the tone-expanded image 123 may take a maximum of 23 bits. Thereafter, as described above, the tone conversion unit 106 generates, for example, a 12-bit tone-converted image 124 from the 23-bit tone-expanded image 123 by gamma conversion.

[0055] According to this embodiment, image processing is performed to appropriately combine the image data of period 2 and the image data of period 1 in one frame period. As a result, even if there is a light source with a blinking period such as an LED in the imaging range, it is possible to perform wide dynamic range (WDR) imaging that takes advantage of the characteristics of area-by-area exposure while suppressing flicker. For example, as in frame 2 of FIG. 3, even if the light emission period of the LED cannot be captured in period 1 during which area-by-area exposure is performed, the light emission of the LED is captured in period 2. Therefore, in the image of frame 2 obtained by synthesizing the outputs of period 1 and period 2, the light emission of the LED is captured, so it is possible to prevent the image from being such that the LED light source is turned off.

[0056] [Embodiment 2] As shown in FIG. 4, the control of the exposure time in one frame period in Embodiment 1 is such that period 2, which is set to be longer than the light emission period of the LED, is in the first half of the frame, and period 1, during which area-by-area exposure control is performed according to the brightness of the object being imaged, is in the second half of the frame. However, if period 1 comes after period 2 within one frame period, and period 2 is shorter than period 1 as in FIG. 4, the period for holding the data of period 2 in the line buffer 301 in FIG. 3 becomes longer, and the memory amount required for the line buffer 301 increases.

[0057] In Embodiment 2, when Period 2 is shorter than Period 1, a method for obtaining the same effect as in Embodiment 1 while reducing the memory amount that the line buffer 301 should have will be described with reference to FIG. 6. Note that, except for the points described below, it is the same as Embodiment 1 described above.

[0058] FIG. 6 is a diagram showing the relationship between the control of the exposure time and the light emission cycle of the LED of the imaging target in Embodiment 2. Different from FIG. 4, in FIG. 6, Period 2 defined as being equal to or longer than the light emission cycle of the LED is in the second half of the frame, and Period 1 in which area-by-area exposure control is performed according to the brightness of the imaging target starts from the beginning of the frame.

[0059] In FIG. 6, the imaging period of Frame 1 is divided into 601-1 and 602-1. 602-1 has the length of Period 2 defined as being equal to or longer than the light emission cycle of the LED. In this embodiment, it is assumed that Period 2 is 11.1 ms as in Embodiment 1. 604-1 is the exposure time of the imaging element in Period 2. In Period 2, the exposure time of the imaging element is always set for the entire Period 2. As can be seen from FIG. 6, since Period 2 is defined as being equal to or longer than the light emission cycle of the LED, it always includes the period during which the LED is emitting light (the convex portion of the LED light emission in FIG. 6). Period 1 of 601-1 is the period of the imaging period of Frame 1 excluding 602-1.

[0060] Period 1 of 601-1 is the period in which area-by-area exposure control is performed according to the brightness of the imaging target. 603-1 is the exposure time for 601-1 set by area-by-area exposure control, and in this embodiment, it is half the time of 601-1. As described above, in Embodiment 2, 601-1 of Period 1 exists in the first half of the frame, and 602-1 of Period 2 exists behind it. The same control is also performed for Frame 2. In this example, in Frame 2, the exposure time 603-2 of Period 1 is set to 1 / 8 of the time of 601-2 as a result of area-by-area exposure control. Therefore, during the period of 601-2, the light emission period of the LED is not captured by the imaging element.

[0061] Similar to Embodiment 1, the image data of the exposure image 122 in period 2 of a certain frame at a certain pixel position is referred to as O2. Also, the image data of the exposure image 122 in period 1 of the same frame is referred to as O1. In Embodiment 2, as can be seen from FIG. 6, as the exposure image 122 in FIG. 3, first O1 is input. Therefore, in FIG. 3, first the data O1 in period 1 is input to and held in the line buffer 301. The data of O1 is held in the line buffer during period 2, and when the data O2 in period 2 is input, the composite output O3 is calculated based on the aforementioned Equation 1. The following operations are the same as those in Embodiment 1.

[0062] According to Embodiment 2, the holding period of the data O1 in the line buffer 301 is the length of period 2. In Embodiment 1, the holding period was the length of period 1. When period 2 is shorter than period 1, the amount of data held in the line buffer 301 can be reduced by the difference in the lengths of period 2 and period 1, and it becomes possible to obtain the same effect as in Embodiment 1 with a smaller memory amount.

[0063] [Embodiment 3] In Embodiment 1, the output image signal was always the composite image signal of period 1 and period 2. However, since period 2 is determined regardless of the brightness of the imaging target, under conditions where it is determined to be bright in area-specific exposure, the output is likely to saturate. In Embodiment 3, in such a case, it is determined whether there is a light source that generates flicker in the imaging target, and an example of adaptively changing the composite ratio of period 1 and period 2 will be described with reference to FIGS. 7 to 10. Note that except for the points described below, it is the same as Embodiment 1 described above.

[0064] FIG. 7 is a block diagram showing a configuration example of the exposure correction unit 105 in Embodiment 3. The exposure correction unit 105 includes a line buffer 301, an addition ratio calculation unit 302, a flicker determination unit 701, an image addition unit 702, and a tone expansion unit 304. Also, as shown in FIG. 9, period information 704 is input from the register 142 and is used in the flicker determination unit 701 and the image addition unit 702. Components equivalent to those in Embodiment 1 are assigned the same numbers as in FIG. 3 of Embodiment 1.

[0065] Regarding how each component of the exposure correction unit 105 shown in FIG. 7 operates, an example of the imaging period shown in FIG. 4 will be referred to for explanation. Most of the operations of the exposure correction unit 105 in Embodiment 3 are the same as those in Embodiment 1, and here the parts with differences will be described.

[0066] In FIG. 7, first, the exposure image 122 of period 2 of frame 1 in FIG. 4 is input to the exposure correction unit 105. Also in this embodiment, the image data of the exposure image 122 of period 2 of a certain frame at a certain pixel position will be referred to as O2. The data of O2 is first stored in the line buffer 301. After the elapse of the time of period 1, the exposure image 122 of period 1 of frame 1 is input. The image data of the exposure image 122 of period 1 of the same frame at this time will be referred to as O1. When O1 is supplied as the exposure image 122, the flicker determination unit 701 reads the data of O2 from the line buffer 301 as the delayed image 306. Then, based on the addition ratio k(305) calculated by the addition ratio calculation unit 302, it is determined whether flicker occurs at the pixel, and the flicker determination result 703 is transmitted to the image addition unit 702. The operation of the flicker determination unit 701 will be described later.

[0067] In the image addition unit 702, the data of O2 read from the delayed image 306, the data of O1 supplied as the exposure image 122, and the addition ratio k are delayed by the time required for calculating the flicker determination result 703 in the flicker determination unit 701. Then, the composite output O3 is calculated using the flicker determination result 703. The operation of the image addition unit 702 will be described later.

[0068] FIG. 8 is a flowchart showing an example of the process of calculating the flicker determination result 703 in the flicker determination unit 701. In S801, the period ratio α, which is the ratio of period 1 to period 2, is calculated. The ratio of period 1 to period 2 is calculated based on the period information 704 input from the register 142. The period ratio α can be obtained by the following formula 4.

[0069] α = (Length of Period 1) ÷ (Length of Period 2) ··· (Equation 4) In Embodiment 3, as shown in FIG. 4, since the length of Period 1 is 22.2 ms and the length of Period 2 is 11.1 ms, the period ratio α = 2. The calculation of the period ratio α in S801 is first performed for each frame.

[0070] Using the period ratio α and the addition ratio k obtained as described above, the values of the image data O1 and the image data O2 are corrected respectively. Using the first correction signal obtained by correcting the image data O1 and the second correction signal obtained by correcting the image data O2, a flicker evaluation value D is calculated for each pixel in S802. The flicker evaluation value D can be obtained by the following Equation 5 using the image data O1, the image data O2, the addition ratio k calculated by the addition ratio calculation unit 302, and the period ratio α D = O1 ÷ k - α × O2 ··· (Equation 5)

[0071] According to Equation 5, the difference between the first correction signal obtained by correcting the image data O1 with the addition ratio k and the second correction signal obtained by correcting the image signal O2 with the period ratio α is obtained as the flicker evaluation value D. Next, it is determined in S803 whether flicker occurs for each pixel. Here, let the threshold for whether flicker occurs be β. If the absolute value of the flicker evaluation value D obtained in S802 is smaller than β, it is determined that no flicker occurs (Yes in S803) and the process proceeds to S804. Otherwise, it is determined that flicker occurs (No in S803) and the process proceeds to S805.

[0072] In Embodiment 3, β is a predetermined threshold value given in advance and is set inside the image addition unit 702. However, the setting method of β is not limited to this. For example, similar to the period information 704, it is also possible to set the value in the register 142 from the external controller 10 so that the value can be changed as needed and used. Compare D with the threshold β to determine whether D is smaller than the threshold β.

[0073] In S804, when the flicker determination result is F, F = 1 which means no flicker occurs is set. This value of F is transmitted to the image addition unit 702 as the flicker determination result 703. On the other hand, in S805, as the value of F indicating the flicker determination result, F = 0 which means flicker has occurred is set. Also in this case, the value of F is transmitted to the image addition unit 702 as the flicker determination result 703.

[0074] In S806, it is determined whether the calculation process of the flicker evaluation value D has been completed for the pixels of one frame. If not completed (No in S806), the processes after S802 are repeated. If completed (Yes in S806), this process is completed.

[0075] FIG. 10 is a flowchart showing an example of the image addition process in the image addition unit 702 according to the presence or absence of flicker. In S1001, the conversion ratio G used in S1003 is calculated. The conversion ratio G is a coefficient for converting the output value O1 obtained by the region-by-region exposure control into the pixel value of the frame when it is determined that there is no flicker. The conversion ratio G can be obtained by the following formula 6 using the lengths of period 1 and period 2 based on the period information 704 input from the register 142.

[0076] G = {(length of period 1)+(length of period 2)}÷(length of period 1) ··· (Formula 6) In the third embodiment, since the length of period 1 is 22.2 ms and the length of period 2 is 11.1 ms from FIG. 4, G = (22.2 + 11.1)÷22.2 = 1.5. The calculation of the conversion ratio G in this S1001 may be first performed for each frame.

[0077] Next, in S1002, referring to the flicker determination result F for each pixel, the calculation method of the pixel value O3 to be output is determined. If F = 1 and no flicker occurs (Yes in S1002), it proceeds to S1003. On the other hand, if F = 0 and flicker occurs in the pixel (No in S1002), it proceeds to S1004.

[0078] In S1003, when there is no flicker, the pixel value O3 to be output is calculated using the following Equation 7. Then, the process proceeds to S1005.

[0079] O3 = G × O1 ··· (Equation 7) In S1004, when there is flicker, the pixel value O3 to be output is calculated using Equation 1 shown in the first embodiment. Then, the process proceeds to S1005. In S1005, it is determined whether the calculation process of the pixel value O3 has been completed for the pixels of one frame. If not completed (No in S1005), the processes after S1002 are repeated. If completed (Yes in S1005), this process is completed.

[0080] According to the third embodiment, by determining whether there is a light source that causes flicker in the imaging target and adaptively changing the synthesis ratio between Period 1 and Period 2, it becomes possible to obtain a more appropriate image as a WDR image. Also, it goes without saying that the method presented in the third embodiment is applicable even when Period 2 is located in the second half of the frame as shown in the second embodiment.

[0081] [Embodiment 4] In the first to third embodiments, the synthesis of the outputs of Period 1 and Period 2 was performed inside the imaging device 100, but the synthesis location does not necessarily have to be limited to the inside of the imaging device 100. For example, it is also possible to perform it in the controller 10 connected to the imaging device 100. Such a case will be described with reference to FIGS. 11 and 12.

[0082] Figure 11 shows the relationship between the image output and the generated frame image in this embodiment. Here, assume that the image required by the user is a moving image at 30 fps, and the light emission cycle of the LED is 90 Hz. In this example, the imaging device 100 divides one frame period into three parts to generate three sub-frames and outputs them to the subsequent controller 10. In Figure 11, the sub-frames are represented as period 1, period 1', and period 2. The output of this sub-frame is operationally the same as performing imaging at 90 fps for the imaging device 100.

[0083] Figure 12 is a block diagram showing the connection between a schematic configuration example of the imaging device 100 to which the imaging control device according to this embodiment is applied and an external controller. The difference from Embodiment 1 is that there is no exposure correction unit 105 and gradation conversion unit 106, and instead, there is a data overlay unit 1201. The data overlay unit 1201 overlays the information of the exposure time 112 and the analog gain value 113 used for imaging each pixel block during the blanking period of the image data captured by the imaging sensor 102 and transmits it to the controller 10 via the LVDS signal line. Note that information communication may be performed between the controller 10 and the imaging device 100 by separately provided wired or wireless communication without overlaying information on the image data.

[0084] In this embodiment, the unit of region-by-region exposure control is preferably each sub-frame. Then, the exposure conditions used in the same sub-frame as the data of the exposure image are output to the controller 10. The controller 10 can correct the image with reference to the exposure conditions overlaid on the image of each sub-frame, and further synthesize them to generate a frame image at 30 fps.

[0085] Returning to FIG. 11, the three sub-frames that make up one frame in FIG. 11 are composed of sub-frame period 1 and sub-frame period 1' for performing exposure control by region, and sub-frame period 2 with the entire sub-frame period as the exposure time. The sub-frame period 1 of frame 1 is indicated as 1101-1, and the sub-frame period 1 of frame 2 is indicated as 1101-2. The sub-frame period 1' of frame 1 is indicated as 1101-1', and the sub-frame period 1' of frame 2 is indicated as 1101-2'. Similarly, the sub-frame period 2 of frame 1 is indicated as 1102-1, and the sub-frame period 2 of frame 2 is indicated as 1102-2. Here, it is assumed that the exposure times of sub-frame period 1 and sub-frame period 1' are equal for each frame. That is, period 1103-1 and period 1103-1' have the same length, and period 1103-2 and period 1103-2' have the same length.

[0086] The exposure times of sub-frame period 1 and sub-frame period 1' of frame 1 and frame 2 may be different because the exposure conditions are calculated for each frame. FIG. 11 according to this embodiment shows a case where the exposure times of frame 1 and frame 2 are different. On the other hand, regardless of the frame, the exposure times of period 2 (1104-1, 1104-2) are the same as the length of period 2. Based on such exposure conditions, each module inside the imaging device 100 performs control so that the charge exposed and accumulated in each sub-frame is output. The details are the same as those described in Embodiment 1, so the details are omitted. The controller 10 that receives the image data of the sub-frames with different exposure conditions divided into three for each frame adds the exposure data of period 1 and period 1' for each pixel. Then, by correcting as shown in Embodiments 1 to 3 above while referring to the data of the exposure conditions superimposed on each frame for the data of period 1 + period 1' and the data of period 2, it is possible to generate a suitable WDR image while preventing flicker. The details of the correction are omitted.

[0087] As described above, according to Embodiment 4, the synthesis of the outputs in Period 1 and Period 2 can be performed not inside the imaging device 100 but by an external controller or the like. Further, Period 1 can be divided into two sub-frame periods, and the exposure conditions can be controlled for each region for the two sub-frame periods.

[0088] <Application to Devices of Imaging Devices> Hereinafter, a device 1000 including a semiconductor device 1100 including a package 1020 on which a semiconductor chip 1110 including a semiconductor integrated circuit shown in FIG. 13 is mounted will be described. The semiconductor chip 1110 is housed in the package 1020 and mounted on the device 1000. In the configuration shown in FIG. 13, the semiconductor chip 1110 includes the imaging device according to the above-described embodiment. The semiconductor device 1100 can include a package 1020 including a base 1010 to which the semiconductor chip 1110 is fixed and a light-transmitting member 1030 such as glass facing the semiconductor chip 1110. The package 1020 can be provided with joining members such as wires and bumps that connect inner leads provided on the base 1010 and terminals such as pad electrodes provided on the semiconductor chip 1110.

[0089] The device 1000 can include at least any one of an optical device 1040, a control device 1050, a processing device 1060, a display device 1070, a storage device 1080, and a mechanical device 1090. The optical device 1040 is, for example, a lens, a shutter, or a mirror. The control device 1050 controls the semiconductor chip 1110. The control device 1050 is, for example, a semiconductor device such as an ASIC.

[0090] The processing device 1060 processes the output signal from the imaging device included in the semiconductor chip 1110. The processing device 1060 is a semiconductor device such as a CPU or an ASIC for constituting an AFE (Analog Front End) or a DFE (Digital Front End). For example, an image may be generated based on the event signal E. The display device 1070 is an EL display device or a liquid crystal display device that displays the information image obtained by the semiconductor chip 1110. The storage device 1080 is a magnetic device or a semiconductor device that stores the information image obtained by the semiconductor chip 1110. The storage device 1080 is a volatile memory such as SRAM or DRAM, or a non-volatile memory such as a flash memory or a hard disk drive.

[0091] The mechanical device 1090 has a movable part or a propulsion part such as a motor or an engine. In the device 1000, the signal output from the semiconductor chip 1110 is displayed on the display device 1070 or transmitted to the outside by a communication device (not shown) provided in the device 1000. For this purpose, the device 1000 may further include a storage device 1080 and a processing device 1060 separately from the storage circuit and the arithmetic circuit of the semiconductor chip 1110. The mechanical device 1090 may be controlled based on the signal output from the semiconductor chip 1110.

[0092] Also, the device 1000 is suitable for an information terminal having an imaging function, for example, a smartphone, a wearable terminal, or an electronic device such as a camera (for example, a lens interchangeable camera, a compact camera, a video camera, a surveillance camera, etc.). The mechanical device 1090 in the camera can drive the components of the optical device 1040 for zooming, focusing, and shutter operation. Alternatively, the mechanical device 1090 in the camera can move the optical device 1040 for anti-vibration operation.

[0093] Also, the device 1000 can be a transportation device such as a vehicle, a ship, or an aircraft. The mechanical device 1090 in the transportation device can be used as a moving device. The device 1000 as a transportation device is suitable for transporting semiconductor chips 1110 or for assisting and / or automating driving operations with an imaging function. The processing device 1060 for assisting and / or automating driving operations can perform processing for operating the mechanical device 1090 as a moving device based on the information obtained from the semiconductor chip 1110. Alternatively, the device 1000 may be a medical device such as an endoscope, a measuring device such as a distance measuring sensor, an analysis device such as an electron microscope, an office device such as a copier, or an industrial device such as a robot.

[0094] [Embodiment 5] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0095] (Other Embodiments) (Item 1) An imaging control device that causes an image sensor having a plurality of pixel blocks each including a plurality of pixels to capture a moving image, A control unit that controls the image sensor to include a first imaging period in which a charge accumulation time is controlled based on a first exposure condition determined for each of the plurality of pixel blocks during one frame period in the moving image, and a second imaging period including a charge accumulation time based on a preset second exposure condition. An imaging control device comprising: (Item 2) The imaging control device according to Item 1, wherein the second imaging period is sandwiched by the first imaging periods respectively included in temporally consecutive frames. (Item 3) The imaging control device according to Item 1 or 2, wherein the second imaging period is set after the first imaging period in the one frame period. (Item 4) The imaging control device according to any one of Items 1 to 3, wherein the second imaging period is set to be longer than the light emission period of a light source that emits light periodically and is included in the imaging target. (Item 5) The imaging control device according to any one of Items 1 to 4, wherein the first exposure condition and the second exposure condition include the charge accumulation time and a gain for amplifying an image signal obtained from the imaging element. (Item 6) The imaging control device according to any one of Items 1 to 5, wherein two first imaging periods are included in the one frame period. (Item 7) An image processing device that processes an image signal captured by the imaging element controlled by the imaging control device according to any one of Items 1 to 6, wherein the image processing device generates a composite image signal by weighted addition of a first image signal captured in the first imaging period and a second image signal captured in the second imaging period at a predetermined ratio for each pixel in the one frame period. (Item 8) The image processing device according to Item 7, wherein the predetermined ratio is based on the first exposure condition and the second exposure condition. (Item 9) The image processing device according to Item 7 or 8, wherein the predetermined ratio is determined by comparing the difference between a first correction signal obtained by correcting the first image signal of a pixel at a predetermined position by the first exposure condition and a second correction signal obtained by correcting the second image signal of the pixel at the predetermined position by the second exposure condition with a threshold value. (Item 10) The image processing device according to Item 9, wherein when the difference is smaller than the threshold value, the ratio of the second image signal in the weighted addition is set to 0. (Item 11) An imaging device including the imaging control device according to any one of Items 1 to 6, the image processing device according to any one of Items 7 to 10, and the imaging element. (Item 12) A method for driving an image sensor that drives an image sensor in which a plurality of pixel blocks each including a plurality of pixels are arranged to capture a moving image, a step of driving the imaging of the image sensor in a first imaging period including a charge accumulation time controlled based on exposure conditions for each of the plurality of pixel blocks during one frame period of the moving image; a step of driving the imaging of the image sensor in a second imaging period including a charge accumulation time preset in the one frame period, wherein the method for driving an image sensor is characterized by performing the steps. (Item 13) A program executed by a computer, the program for causing the computer to execute each step of the method for driving an image sensor according to Item 12. (Item 14) The imaging device according to Item 11, a processing device that processes an output signal from the imaging device, wherein the device is characterized by comprising the above.

[0096] The invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Signs

[0097] 105: Exposure correction unit, 112: Exposure time, 113: Analog gain value, 122: Exposure image, 123: Image after region expansion, 301: Line buffer, 302: Addition ratio calculation unit, 304: Tone expansion unit, 305: Addition ratio, 306: Delayed image

Claims

1. An imaging control device that causes an imaging device having a plurality of pixel blocks each including a plurality of pixels to capture a moving image, The imaging control device includes a control unit that controls the imaging device so as to include a first imaging period in which a charge accumulation time is controlled based on a first exposure condition determined for each of the plurality of pixel blocks during one frame period in the moving image, and a second imaging period including a charge accumulation time based on a preset second exposure condition.

2. The imaging control device according to claim 1, wherein the second imaging period is sandwiched by the first imaging periods respectively included in temporally consecutive frames.

3. The imaging control device according to claim 1, wherein the second imaging period is set after the first imaging period in the one frame period.

4. The imaging control device according to claim 1, wherein the second imaging period is set to be longer than a light emission period of a light source that emits light periodically and is included in an imaging target.

5. The imaging control device according to claim 1, wherein the first exposure condition and the second exposure condition include the charge accumulation time and a gain for amplifying an image signal obtained from the imaging device.

6. The imaging control device according to claim 1, wherein two first imaging periods are included in the one frame period.

7. An image processing device that processes an image signal captured by the imaging device controlled by the imaging control device according to claim 1, The image processing device generates a composite image signal by weighted addition of a first image signal captured in the first imaging period and a second image signal captured in the second imaging period at a predetermined ratio for each pixel in the one frame period.

8. The image processing device according to claim 7, wherein the predetermined ratio is based on the first exposure condition and the second exposure condition.

9. The image processing device according to claim 7, wherein the predetermined ratio is determined by comparing a difference between a first correction signal obtained by correcting the first image signal of pixels at a predetermined position according to the first exposure condition and a second correction signal obtained by correcting the second image signal of pixels at the predetermined position according to the second exposure condition with a threshold value.

10. The image processing device according to claim 9, wherein when the difference is smaller than the threshold value, the ratio of the second image signal in the weighted addition is set to 0.

11. An imaging device comprising the imaging control device according to any one of claims 1 to 6, the image processing device according to any one of claims 7 to 10, and the imaging element.

12. A method for driving an imaging element that drives the imaging element including a plurality of pixel blocks each including a plurality of pixels to perform imaging of a moving image, a step of driving the imaging of the imaging element in a first imaging period including a charge accumulation time in which one frame period of the moving image is controlled based on exposure conditions for each of the plurality of pixel blocks; a step of driving the imaging of the imaging element in a second imaging period including a charge accumulation time preset in the one frame period, the method for driving an imaging element being characterized by causing the steps to be performed.

13. A program executed by a computer, the program for causing the computer to execute each step included in the method for driving an imaging element according to claim 12.

14. An apparatus comprising the imaging device according to claim 11, and a processing device that processes an output signal from the imaging device. The apparatus is characterized by comprising the above.

Images