Imaging device, image processing system, and control method for imaging device

Abstract

To obtain an image having a high dynamic range with a little delay without using a large buffer when outputting an exposure image and exposure information for every area to the outside to perform correction processing on the image on the outside.SOLUTION: An imaging apparatus has means that divides an imaging area of an image pickup device into a plurality of areas and outputs, to the outside of the imaging apparatus, an exposure image picked up by the image pickup device with a control of an exposure condition for every area, and means that outputs, to the outside of the imaging apparatus, exposure information for every area indicating the exposure condition applied to every area when picking up the exposure image. The imaging apparatus ends output of the exposure information on an area in the exposure image on which exposure correction processing is performed before ending output of an image of the area.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device, an image processing system, and a control method for an imaging device.

Background Art

[0002] Generally, it is known that the dynamic range of an image sensor used in a digital camera or the like is smaller than the dynamic range of the natural world. For this reason, conventionally, methods for expanding the dynamic range of an image sensor have been studied. Patent Document 1 discloses a technique for expanding the dynamic range of an image sensor by determining the exposure time of each pixel from the information of a preliminary shot and performing the main shot.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, gain calculation is performed on the exposure image for each region based on the exposure amount map, and correction processing for the image is performed. When performing such exposure correction processing for an image outside the sensor device using the exposure information for each region generated within the sensor device, there are the following problems. If the exposure information for all regions is acquired after the image data of the exposure image has been acquired, the correction processing cannot be started until the exposure information is acquired, so all of the image data of the exposure image must be temporarily held, requiring a large buffer. Also, since the exposure information for each region is acquired after the exposure image, the correction processing of the image data is delayed by that delay, and a delay occurs until the desired image data can be obtained from the imaging result. This is not suitable for applications that emphasize real-time performance, such as for monitoring.

[0005] Therefore, an object of the present invention is to obtain a high-dynamic-range image with little delay without using a large buffer when an exposure image and exposure information for each region are output to the outside and exposure correction processing for the image is performed outside.

Means for Solving the Problems

[0006] The imaging device according to the present invention divides the imaging region of the imaging element into a plurality of regions, controls the exposure conditions for each region, and captures a first image captured by the imaging element 、and Exposure information for each region indicating the exposure conditions applied to each region when capturing the first image 、 Output to the outside of the imaging device coming out Power means a superimposing means for superimposing the exposure information for each region on the first image of the corresponding region when outputting the exposure information And before finishing outputting the image of the region to be subjected to exposure correction processing among the first images, finishing outputting the exposure information of the region.

Effects of the Invention

[0007] According to the present invention, when an exposure image and exposure information for each region are output to the outside and exposure correction processing for the image is performed outside, it is possible to obtain a high-dynamic-range image with little delay without using a large buffer.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations. The same reference numerals are used to describe the same configurations or processes.

[0010] FIG. 1 is a block diagram showing the schematic configuration of the imaging device 100 according to the first embodiment. The imaging device 100 of the present embodiment includes various configurations that a general imaging device has. However, for the sake of simplicity of illustration and description, FIG. 1 shows only the main constituent parts according to the present embodiment. The outline of each constituent part of the imaging device 100 will be described starting from the imaging element unit 103.

[0011] The imaging element unit 103 is configured to be drivable by dividing the imaging area into a plurality of areas called pixel blocks, and has a function of performing an exposure operation (charge accumulation) with different exposure times for each area. The pixel blocks will be described later. In the case of this embodiment, the imaging element unit 103 has the exposure time set for each area by an exposure control signal 117 supplied from an exposure time control unit 109 described later, and performs exposure with the exposure time set for each of those areas. The exposure control signal 117 is a signal for setting the exposure time for each area of the imaging element unit 103. Then, the imaging element unit 103 reads out the charge accumulated in each pixel with the exposure time controlled by the exposure control signal 117 for each area, outputs it as a pixel potential 118, and outputs it to an A / D conversion unit (analog / digital conversion unit) 104.

[0012] The A / D conversion unit 104 performs analog / digital conversion on the pixel potential 118 read from the imaging element unit 103 and converts it into a digital value. In the case of this embodiment, an analog gain 121 corresponding to each area is set for the A / D conversion unit 104 by a gain control unit 110. The A / D conversion unit 104 multiplies the pixel potential 118 from the imaging element unit 103 by the analog gain 121 for each area, and then performs analog / digital conversion to convert it into a digital value. Hereinafter, an image composed of digital signals that have been multiplied by the analog gain 121 for each area and subjected to analog / digital conversion by the A / D conversion unit 104 is referred to as an exposure image 122. The exposure image 122 output from the A / D conversion unit 104 is sent to an exposure condition calculation unit 111 and a data overlay unit 105.

[0013] Based on the exposure image 122, the exposure condition calculation unit 111 calculates and updates the exposure time 112 and the analog gain value 113 for each area so that the shooting conditions are optimal. Then, the value of the exposure time 112 for each area is sent to the exposure time control unit 109, and the analog gain value 113 for each area is sent to the gain control unit 110. Also, the exposure time 112 and the analog gain value 113 for each area are sent to the data overlay unit 105.

[0014] The synchronization control unit 101 generates a synchronized exposure time output pulse 120 and a gain output pulse 114, outputs the exposure time output pulse 120 to the exposure time control unit 109, and outputs the gain output pulse 114 to the gain control unit 110. Based on the exposure time output pulse 120 and the value of the exposure time 112 for each region, the exposure time control unit 109 generates an exposure control signal 117 for each region and outputs it to the imaging element unit 103. As a result, the exposure time for each region corresponding to the exposure time 112 for each region is set for the imaging element unit 103.

[0015] Based on the gain output pulse 114 and the analog gain value 113 for each region, the gain control unit 110 generates an analog gain 121 for each region for the pixel potential 118 for each region of the imaging element unit 103 and outputs it to the A / D conversion unit 104. As a result, in the A / D conversion unit 104, analog / digital conversion is performed after the analog gain 121 for each region corresponding to each pixel potential 118 for each region is applied. The data after analog / digital conversion is sent as an exposure image 122 for each region to the exposure condition calculation unit 111 and the data overlay unit 105.

[0016] The data overlay unit 105 receives the exposure time 112 and the analog gain value 113 for each region and performs packing as exposure information. Then, considering the order relationship with the exposure image 122, the data overlay unit 105 sequentially outputs the exposure information and the exposure image to the image output unit 108 in an appropriate order.

[0017] The image output unit 108 receives the exposure information and the exposure image 122 from the data superposition unit 105 and outputs them to the outside of the imaging device 100. In the example of the present embodiment, a controller 150 is connected as a processing module that receives image data from the imaging device 100. Here, it is assumed that the signal line connecting the image output unit 108 and the controller 150 of the imaging device 100 is an LVDS signal line having 16 data channels. However, the type of this signal line and the data channel width are not limited by this embodiment. In the present embodiment, the image output unit 108 is an example of the first output means and the second output means, and the controller 150 is an example of the processing device.

[0018] FIG. 2 is a diagram for explaining the configuration of the imaging element unit 103. The imaging area of the imaging element unit 103 is composed of a plurality of pixel blocks 201, and each pixel block 201 is further composed of a plurality of pixels 202. In the example of this embodiment, it is assumed that the number of pixels in the width 206 direction (horizontal line direction) of the imaging area of the imaging element unit 103 is 2000 pixels, and the number of pixels in the height 205 direction is 1000 pixels (that is, the number of horizontal lines in the vertical direction is 1000 lines). Also, it is assumed that the number of pixels in the width 204 direction (horizontal line direction) of the pixel block 201 is 100 pixels, and the number of pixels in the height 203 direction is 100 pixels (for 100 lines of horizontal lines in the vertical direction). In this case, the number of pixel blocks 201 in the imaging area of the imaging element unit 103 is 20 in the horizontal direction and 10 in the vertical direction. Also, the pixel blocks [0,0] to [19,9] described in each pixel block 201 in FIG. 2 represent the positions of the respective pixel blocks 201 in the imaging area, and the values within the brackets [ ] represent the horizontal and vertical indices of each pixel block in the imaging area. In FIG. 2, for example, in the case of the pixel block 201 located in the upper right of the imaging element unit 103, it is the pixel block [19,0]. Also, a set of pixel blocks represented by the same vertical index will be referred to as a block row. That is, block row N consists of pixel blocks [0,N] to [19,N]. For example, block row 5 consists of pixel blocks [0,5] to [19,5]. Note that the respective sizes (number of pixels in the vertical and horizontal directions) of the imaging element unit 103 and the pixel block 201 are not limited to the above example. Also, the shape and aspect ratio of the pixel 202 are not limited either, and for example, it may not be square but rectangular. Furthermore, the pixel block 201 may be composed of only one pixel 202.

[0019] In this embodiment, each pixel block 201 is a unit capable of controlling the exposure time and the analog gain. Here, the exposure time corresponds to the time during which charges are accumulated in the pixels (light-receiving elements) of the imaging element unit 103 during imaging. Therefore, for example, assuming that the amount of incident light on the imaging element unit 103 is the same and the pixels do not saturate, the pixel potential 118 becomes higher (the image can be taken brighter) as the exposure time becomes longer. That is, when the amount of incident light is the same and pixel saturation is not considered, for example, comparing the cases of an exposure time of 1 / 480 second and 1 / 30 second, it is possible to take a brighter image in the case of 1 / 30 second.

[0020] The analog gain is the gain applied to the pixel potential 118 in the A / D conversion unit 104 during imaging. Therefore, the larger the value of the analog gain, the larger the digital pixel value (the digital value obtained by analog / digital conversion after the gain is applied) output from the A / D conversion unit 104 becomes.

[0021] Returning to FIG. 1, the configuration and operation of the imaging device 100 of the present embodiment will be described. Based on the exposure control signal 117, the imaging element unit 103 performs imaging with the exposure time controlled in units of pixel blocks 201. Then, the imaging element unit 103 outputs a pixel potential 118 corresponding to the charges accumulated for each pixel.

[0022] The A / D conversion unit 104 performs digital conversion on each pixel potential 118 output from the imaging element unit 103 after multiplying it by the analog gain 121 set corresponding to each pixel block of the imaging element unit 103, and outputs an exposure image 122. In the present embodiment, it is assumed that the exposure image 122 is a 10-bit digital value. Also, the analog gain 121 can take four values, for example, as gain values, ×1 times, ×2 times, ×4 times, and ×8 times.

[0023] Next, the exposure time 112 and the analog gain value 113 will be described with reference to FIGS. 3, 4, and 5. Referring to FIG. 3, the exposure time 112 set for each pixel block 201 will be described. As shown in FIG. 3, the exposure time 112 includes an exposure time ID, a value of the exposure time (seconds), and an exposure correction coefficient. The exposure time ID is an index indicating the exposure time (seconds). The exposure time IDs [0,0] to [19,9] described in each pixel block 201 of FIG. 3 represent the respective exposure time IDs in each pixel block [0,0] to [19,9] shown in FIG. 2. In the case of this embodiment, the index value of the exposure time ID is set to be any one of 0 to 4. The example of FIG. 3 shows the case where the index value of the exposure time ID [19,0] in the pixel block [19,0] located in the upper right of the imaging region is 4. The actual exposure time (seconds) and the exposure correction coefficient corresponding to the exposure time ID will be described later with reference to FIG. 5.

[0024] Next, referring to FIG. 4, the analog gain value 113 will be described. As shown in FIG. 4, the analog gain value 113 includes a gain ID, a value of the analog gain, and a gain correction coefficient. The gain ID is an index indicating the analog gain. The gain IDs [0,0] to [19,9] described in each pixel block 201 of FIG. 4 represent the respective gain IDs in each pixel block [0,0] to [19,9] shown in FIG. 2. In the case of this embodiment, the index value of the gain ID is set to be any one of 0 to 3. The example of FIG. 4 shows the case where the index value of the gain ID [19,0] in the pixel block [19,0] located in the upper right of the imaging region is 2. The actual analog gain and the gain correction coefficient corresponding to the gain ID will be described later with reference to FIG. 5.

[0025] Next, with reference to FIG. 5, the exposure time ID and the corresponding exposure time and exposure correction factor will be described. As described above, the exposure time ID shall take values from 0 to 4 as index values. It is assumed that the index value 0 of the exposure time ID corresponds to 1 / 30 [seconds] of the exposure time [seconds]. Similarly hereinafter, the index value 1 of the exposure time ID corresponds to 1 / 60 [seconds] of the exposure time, the index value 2 corresponds to 1 / 120 [seconds], the index value 3 corresponds to 1 / 240 [seconds], and the index value 4 corresponds to 1 / 480 [seconds]. The exposure time is one of the parameters related to the shooting conditions. In this embodiment, the index value of the exposure time ID that enables shooting under the brightest conditions is set to 0. Assuming that the amount of incident light on the image sensor unit 103 is the same and there is no pixel saturation, when based on the brightest exposure time of 1 / 30 [seconds], the brightness during shooting with each exposure time (1 / 30 second to 1 / 480 second) is 1 to 1 / 16 times. For example, with respect to 1 / 30 [seconds] of the exposure time at the index value 0 of the exposure time ID, at 1 / 480 [seconds] of the exposure time at the index value 4 of the exposure time ID, the brightness during shooting is 1 / 16 times (= (1 / 480 second) ÷ (1 / 30 second)).

[0026] Also, the exposure correction factor is a correction factor for adjusting the levels of the pixel values when shooting is performed with the exposure time corresponding to the exposure time ID as described above. In the case of this embodiment, the exposure correction factor is a correction factor that adjusts the levels of the pixel values at each exposure time (1 / 30 second to 1 / 480 second) based on the level when shooting is performed with the exposure time of 1 / 30 [seconds] that enables shooting under the brightest conditions. Therefore, the reciprocal of the ratio of the brightness during shooting is used as the exposure correction factor. As described above, when based on the exposure time of 1 / 30 [seconds] that enables shooting under the brightest conditions, the brightness during shooting at the exposure times from 1 / 30 second to 1 / 480 second is 1 to 1 / 16 times. Therefore, as shown in FIG. 5, the exposure correction factor is 1 to 16 times, which is the reciprocal of those.

[0027] Next, with reference to FIG. 5, the gain ID and the corresponding analog gain and gain correction coefficient will be described. As described above, it is assumed that the gain ID takes values from 0 to 3 as index values. The index value 0 of the gain ID corresponds to an analog gain equivalent to 8 times, and similarly hereinafter, the index value 1 of the gain ID corresponds to 4 times, the index value 2 corresponds to 2 times, and the index value 3 corresponds to 1 time. The analog gain is one of the parameters related to the shooting conditions, like the exposure time described above. In this embodiment, the index value of the gain ID under the condition where the brightest image is obtained is set to 0.

[0028] The gain correction coefficient is a correction coefficient for adjusting the levels of the pixel values when the analog gain corresponding to each gain ID is multiplied, as described above. In the case of this embodiment, the gain correction coefficient is a correction coefficient that adjusts the levels of the pixel values at each analog gain (from 8 times to 1 time) based on the level of the pixel value when the analog gain that gives the brightest image is 8 times. For this reason, as shown in FIG. 5, the gain correction coefficient is set to 1 to 8 times, which is the inverse, for 8 times to 1 time of the analog gain.

[0029] Next, with reference to FIG. 5, the combination of the exposure time (seconds) corresponding to the exposure time ID and the exposure correction coefficient, and the analog gain and the gain correction coefficient corresponding to the gain ID will be described. The exposure time and the analog gain are each parameters related to the shooting conditions, as described above. In the case of this embodiment, the index values of the exposure time ID and the gain ID under the condition where shooting can be done most brightly are each set to 0. For this reason, for example, the combination of the index value of the exposure time ID being 0 (exposure time 1 / 30 second) and the index value of the gain ID being 0 (analog gain 8 times), which is indicated by A in FIG. 5, is the condition for shooting most brightly. Hereinafter, the setting of the shooting conditions by this combination will be denoted as shooting condition setting A.

[0030] On the other hand, the combination in which both the index values of the exposure time ID and the gain ID, indicated by C in FIG. 5, are the largest results in the condition of taking the darkest picture. Hereinafter, the setting of the shooting conditions by this combination is denoted as shooting condition setting C. The combination of these analog gains and exposure times is an example and is not limited to the above example. The combination example indicated by B in FIG. 5 (referred to as shooting condition setting B) will be described later.

[0031] Here, the image output unit 108 shown in FIG. 1 outputs the exposure image 122 for each region. Each region uses the combination of the exposure time ID and the gain ID described with reference to FIGS. 3 and 4 as the exposure information for each region, and the image is output with the brightness based on the exposure information for each region. Therefore, the controller 150 that receives the exposure image 122 needs to correct the exposure conditions for each region to generate the image required by the user. For example, when the user requests an image in which the brightness is smoothly connected over the entire imaging image region, it is necessary to perform exposure correction processing for each region using the exposure information for each region on the entire region of the imaging image. Hereinafter, how the controller 150 performs the exposure correction processing for each region in such a case will be described in detail with reference to FIGS. 6, 7, and 8, taking each of the shooting condition settings A, B, and C shown in FIG. 5 as an example.

[0032] The per-region exposure correction processing in the case of the setting for taking the brightest picture (shooting condition setting A in FIG. 5) will be described with reference to FIG. 6. FIG. 6 is a diagram showing each of the brightness (illuminance) of the subject, the pixel potential, the exposure image, the image after gain correction, the image after exposure correction, and the image after gradation expansion on each axis indicating the light and dark direction. FIG. 6 shows the process from when the subject is photographed until the image after gradation expansion is output. Note that the image after gain correction, the image after exposure correction, and the image after gradation expansion are obtained by processing on the side of the controller 150. Also, in the example of FIG. 6, the setting for photographing the subject with the highest brightness (the photographing condition setting A in FIG. 5) is used as the reference setting, and in this reference setting, the value of the lowest luminance (the value indicated by a ○ mark in the figure) and the value of the highest luminance (the value indicated by a △ mark in the figure) are used as the reference, and each axis is aligned. Note that the values of each axis of the subject, the pixel potential, and the exposure image are values with different units, but in order to make the explanation of the case where the setting is changed in FIGS. 7 and 8 described later easier to understand, in FIG. 6, the values corresponding to the lowest luminance and the highest luminance are drawn so as to be aligned horizontally.

[0033] Hereinafter, the transition of each value in the process from when the subject is photographed until the image after gradation expansion is output will be described. As described above, the photographing condition setting A shown in FIG. 5 is a combination for photographing with the highest brightness. In the case of this photographing condition setting A, the imaging element unit 103 images the subject and the like with an exposure time of 1 / 30 second, and the A / D conversion unit 104 performs A / D conversion by applying an analog gain of 8 times to the pixel potential from the imaging element unit 103. In the following description, the brightness that can be photographed with the photographing condition setting A for photographing with the highest brightness will be referred to as the "reference brightness". Also, it is assumed that the exposure image obtained by A / D converting the pixel potential is a 10-bit digital value as described above.

[0034] In the A / D conversion unit 104, A / D conversion is performed using an analog gain of 8 times according to the above-described shooting condition setting A. Since the gain correction coefficient when an analog gain of 8 times is applied is 1, the image after gain correction in the controller 150 is an image obtained by applying a gain correction coefficient (1 time) to the exposure image. Also, in the case of the example in FIG. 6, according to the shooting condition setting A, the imaging device unit 103 performs imaging with an exposure time of 1 / 30 second. Since the exposure correction coefficient when the exposure time is 1 / 30 second is 1, the image after exposure correction in the controller 150 is an image obtained by applying an exposure correction coefficient (1 time) to the image after gain correction.

[0035] Also, the exposure image is an image obtained by imaging with various combinations of shooting conditions as shown in FIG. 5 described above for each region of the imaging device unit 103. Therefore, the controller 150 adjusts the level of each region's image for the exposure image for each region. However, when considering adjusting the level for each region, in addition to the bit number (10 bits) of the exposure image, 4 bits corresponding to each of the above-described exposure times are further required, and 3 bits corresponding to each analog gain are further required. More specifically, as shown in FIG. 5, since the exposure time has a range from 1 / 30 second to 1 / 480 second, for example, in order to match the brightness when imaging with an exposure time of 1 / 480 second to the reference brightness when shooting with an exposure time of 1 / 30 second, it is necessary to multiply the pixel value by 16. This corresponds to +4 bits (16 = 2 4 ). Similarly, as shown in FIG. 5, since the analog gain has a range from 8 times to 1 time, for example, in order to match the brightness obtained with 1 time to the reference brightness of 8 times, it is necessary to multiply the pixel value by 8. This corresponds to +3 bits (8 = 2 3 ). From the above, the controller 150 generates a tone-expanded image of 17 bits (= 10 + 4 + 3) obtained by performing bit expansion processing while adjusting the level for each region for the image after exposure correction (10 bits).

[0036] In the setting for taking the brightest shot (shooting condition setting A) as in the example of Fig. 6, the process from subject shooting to output of the tone-expanded image maps the dark part side of the subject to the lower bit side of the tone-expanded image, indicating that it is suitable for shooting dark areas.

[0037] Next, an example in the case where the exposure time is 1 / 480 second and the analog gain is 2 times (i.e., shooting condition setting B in Fig. 5) will be described with reference to Fig. 7. Fig. 7 is also presented in the same manner as Fig. 6.

[0038] Here, 1 / 480 second of the exposure time is 1 / 16 of the reference exposure time of 1 / 30 second described above. Therefore, if the brightness (illuminance) of the subject is the same as when imaging with the reference exposure time (1 / 30 second), the pixel potential when the subject is imaged with an exposure time of 1 / 480 second is 1 / 16 times the pixel potential in the case of the reference exposure time of 1 / 30 second. Also, in shooting condition setting B, the analog gain is 2 times, which is 1 / 4 times the gain compared to 8 times the reference analog gain. Thus, the exposure image when the analog gain is 2 times is an image with a level 1 / 4 times that of the exposure image in the case of the reference analog gain (8 times). As a result, in the case of shooting condition setting B shown in Fig. 5, the values on the bright part side of the subject are mapped to 10 bits of the exposure image.

[0039] Next, the controller 150 adjusts the values of the exposure images for each region to the level in the case of the reference shooting conditions (exposure time 1 / 30 second, analog gain 8 times). In the case of shooting condition setting B (exposure time 1 / 480 second, analog gain 2 times), as shown in Fig. 5 described above, the gain correction coefficient is 4 (= 8 times ÷ 2 times), and the exposure correction coefficient is 16 (= (1 / 30 second) ÷ (1 / 480 second)). Therefore, when applying the gain correction coefficient of 4 and the exposure correction coefficient of 16, the exposure-corrected image is mapped 6 bits (4 × 16 = 2 6 ) to the upper bit side with respect to the exposure image. As a result, the exposure image is mapped to bits 6 to 15 out of 17 bits in the tone-expanded image.

[0040] That is, as in the example of FIG. 7, in the case of shooting condition setting B (exposure time 1 / 480 second, analog gain 2 times), the relatively bright part side of the subject is mapped to the post tone expansion image.

[0041] Next, with reference to FIG. 8, an example in the case where the exposure time is 1 / 480 second and the analog gain is 1 time (that is, shooting condition setting C where the darkest shooting is performed in FIG. 5) will be described. FIG. 8 is also a diagram presented in the same manner as FIGS. 6 and 7.

[0042] As described with reference to FIG. 7 above, since the exposure time of 1 / 480 second is 1 / 16 of the reference exposure time of 1 / 30 second, the pixel potential in the case of an exposure time of 1 / 480 second is 1 / 16 times the pixel potential in the case of the reference exposure time of 1 / 30 second. Further, in the case of shooting condition setting C, the analog gain is 1 time, which is 1 / 8 times the gain with respect to 8 times the reference analog gain. Therefore, the exposure image when the analog gain is 1 time is an image with a level 1 / 8 times that of the exposure image in the case of the reference analog gain (8 times).

[0043] Also, in the case of the example of FIG. 8 as well, the controller 150 adjusts the value of the exposure image for each region to the level in the case of the reference shooting conditions (exposure time 1 / 30 second, analog gain 8 times). In the case of shooting condition setting C (exposure time 1 / 480 second, analog gain 1 time), as shown in FIG. 5 above, the gain correction coefficient is 8 (= 8 times ÷ 1 time), and the exposure correction coefficient is 16 (= (1 / 30 second) ÷ (1 / 480 second)).

[0044] Therefore, as in the example of FIG. 8, in the case of shooting condition setting C (exposure time 1 / 480 second, analog gain 1 time), the bright part side of the subject is mapped to the most significant bit side (7 to 16 bits) of the post tone expansion image. As described above by taking FIGS. 6 to 8 as examples, the controller 150 in FIG. 1 performs a process of converting the exposure image 122 (10 bits) for each region into a post tone expansion image (17 bits). The controller 150 further performs tone conversion processing such as gamma conversion in accordance with the bit depth of the image to be utilized in the subsequent stage, but the details thereof are omitted here.

[0045] As described above with reference to FIGS. 6 to 8, when the user requests an image in which the brightness smoothly connects over the entire captured image area, it is necessary to perform exposure correction processing for each area using the exposure information for each area for the entire area of the captured image. At that time, it is necessary to appropriately apply the exposure information to the exposure image for each area. In order to perform the correction processing for each area, it is desirable that the exposure information applied to a certain area be sent to the controller 150 before the image data of the entire area to be corrected by the exposure information is sent to the controller 150. By doing so, it becomes possible to hold only the image data of the area to be corrected by the exposure information on the controller side. In addition, since the correction processing can be started every time the image data of the area to be corrected by the exposure information is complete, the delay of the image generated by the controller 150 can be reduced. Hereinafter, the transmission of the exposure information for each area from the imaging device 100 to the controller 150 in the present embodiment will be described with reference to FIGS. 9 to 12.

[0046] FIG. 9 is a diagram for explaining the order of image data when image data is output from the image output unit 108 of the imaging device 100 to the controller 150. The data overlapping unit 105 generates image data in the order shown in FIG. 9 and delivers it to the image output unit 108, and the image data is output from the image output unit 108 to the controller 150. The total transmission data for one frame sent in synchronization with the vertical synchronization signal of the image is shown as frame data 900. The frame data 900 includes pixel block data 901, OB area data 902, blanking area 903, synchronization code, and identification code 904 that constitute the exposure image for each area. In addition, the exposure information 905 corresponding to the exposure image for each area is inserted in the data overlapping unit 105 at the positions shown in the corresponding block rows. This frame data 900 is output from the image output unit 108 to the controller 150 in the raster scan order from left to right and from top to bottom. Hereinafter, each data will be described.

[0047] The pixel block data 901 obtained by exposure with an exposure time controlled for each area is shown in FIG. 9 for each area corresponding to the pixel block 201 in FIG. 2. For example, the pixel data captured in the area indicated by the pixel block [0,0] in FIG. 2 is output as the pixel block data [0,0] in FIG. 9.

[0048] The OB area data 902 is image data corresponding to the light-shielded OB area (optical black area) of the imaging element unit 103. The OB area data 902 is used, for example, to detect the dark current component of the pixels of the sensor and to detect the offset amount on the image.

[0049] The synchronization code and identification code 904 are special data sequences attached to represent the start of valid data and the position of the first row when transmitting data from the image output unit 108 to the controller 150 via the output signal line. The controller 150 analyzes the order of the data obtained from the output signal line and starts interpreting the subsequent data as significant data by detecting the data indicating the start of the valid data. Also, the controller 150 detects the identification code representing the position of the first row, detects the start of the frame data, and determines the subsequent operations.

[0050] For the blanking area 903, blank data indicating that it is a blank period containing no meaningful data is output. In the frame data 900 obtained by the image output unit 108, blank data is output for the portions of the data other than the pixel block data 901, the OB area data 902, the synchronization code, and the identification code 904.

[0051] The exposure information 905 is exposure information used by the controller 150 to correct the exposure image for each region. In the example shown in FIG. 9, the exposure information 905 is superimposed on the blank period (blanking region) before the pixel data for each row in the exposure image. Specifically, the exposure information 905 is superimposed on the blanking region 903 between the synchronization code, the identification code 904, and the OB region data 902. In FIG. 9, the exposure information 905 indicated by the numbers from 0 to 9 is the exposure information applied to the pixel blocks included in the corresponding block rows from block row 0 to block row 9. The exposure information 905 is not sent all at once for the entire frame of exposure information, but is arranged in a form corresponding to each block row and is superimposed over several pixel rows at the head position of each block row.

[0052] FIG. 10 shows an example of the exposure information 905. As described above, in this embodiment, the signal line connecting the image output unit 108 and the controller 150 is assumed to be an LVDS signal line having 16 data channels. Since the exposure image 122 for each region is assumed to be a 10-bit digital value as described above, image data for one pixel is transmitted and received using 10 cycles for each data channel. In each channel, data is sent in the order of bit 9 to bit 0 from the MSB toward the LSB in sequence. In the case of this embodiment, it means that image data for 16 pixels is sent in parallel through 16 data channels.

[0053] As shown in FIG. 10, the exposure information for each region is transmitted using each channel. In channels 1, 5, 9, and 13, the Valid flag 1001 indicating that the exposure information to be transmitted is valid, the horizontal direction ID 1002 of the pixel block, and the inverted value 1003 of the least significant bit in the horizontal direction ID 1002 are packed into 10 bits and transmitted. The Valid flag 1001 is set to 1 when the information to be transmitted is valid (there is valid exposure information).

[0054] On channels 2, 6, 10, and 14, the vertical ID 1004 of the pixel block and the inverted value 1005 of the least significant bit in the vertical ID 1004 are packed and transmitted. On channels 3, 7, 11, and 15, the 3-bit exposure time ID 1006, the 2-bit gain ID 1007, and the inverted value 1008 of the least significant bit in the gain ID 1007 are packed and transmitted. On channels 4, 8, 12, and 16, the 5-bit additional information 1009 and the inverted value 1010 of the least significant bit in the additional information 1009 are packed and transmitted. The additional information 1009 can include, for example, the value of the relationship of the information with the previous frame (such as whether it is the same data or how the value has changed). In FIG. 10, the other bits are dummy data and can be either 0 or 1. These packings are performed by the data overlay unit 105.

[0055] Note that the reason for including the inverted values 1003, 1005, 1008, and 1010 in each channel is that if all 10 bits become the value 0, it may become indistinguishable from the synchronization code or the like. Also, in this example, since the exposure image 122 for each region is a 10-bit digital value, it is packed in the form shown in FIG. 10, but the packing format such as the position of each ID data is not limited to the example described. Also, although the additional information is 5 bits, the number of bits is not limited to 5 bits.

[0056] Using FIG. 11, regarding the arrangement of the exposure information indicated by the exposure information 905 in FIG. 9, the exposure information related to block row 0 and the data around it will be described as an example. FIG. 11 shows the arrangement of the image data after the 0th row in block row 0 consisting of 100 pixel rows from the 0th row to the 99th row, starting from the leftmost data starting with the synchronization code 1 and up to the OB region data in order. The synchronization code 1 (1101), synchronization code 2 (1102), synchronization code 3 (1103), identification code 1104, and exposure information / blank code 1105 shown in FIG. 11 each correspond to the data transmitted and received in one cycle.

[0057] Synchronization code 1 (1101), synchronization code 2 (1102), and synchronization code 3 (1103) indicate that they are at the beginning of a data row using this arrangement of these three special data. Subsequently, identification code 1104 indicates, for example, that it is at the beginning of a frame row or a block row. Exposure information 1105 is arranged at the position following it. Starting from the 0th row of block row 0, exposure information for the number of pixel blocks included in this block row (block row 0) is packed in order from the top of the block row at this position.

[0058] Here, at the 0th row of block row 0, exposure information for pixel blocks with horizontal IDs of the block row ranging from 0 to 3 is arranged. That is, exposure information for pixel blocks [0,0], [1,0], [2,0], [3,0] is arranged. Here, exposure information for one pixel block is represented using data for 4 pixels. Therefore, for example, exposure information for pixel block [0,0] is arranged at positions corresponding to channels 1 to 4 among 16 data channels, and exposure information for pixel block [1,0] is arranged at positions corresponding to channels 5 to 8. Also, exposure information for pixel block [2,0] is arranged at positions corresponding to channels 9 to 12, and exposure information for pixel block [3,0] is arranged at positions corresponding to channels 13 to 16. Similarly, at the 1st row of block row 0, exposure information for pixel blocks with horizontal IDs of the block row ranging from 4 to 7 is arranged.

[0059] In this embodiment, since the number of pixel blocks included in one block row is set to 20 blocks, exposure information for each pixel block from the 0th row to the 4th row of the block row is included. Here, each exposure information is the exposure information shown in FIG. 10, and information up to channels 1 to 4 (CH1 to CH4) shown in FIG. 10 is respectively embedded at the position indicated by [0,0] in FIG. 11. Similarly, information up to channels 5 to 8 (CH5 to CH8) shown in FIG. 10 is respectively embedded at the position indicated by [1,0] in FIG. 11. Since there is no exposure information to be transmitted after the 5th row of the block row, blank codes are embedded. Since one block row consists of 100 pixel rows, blank codes are embedded from the 5th row to the 99th row of the block row instead of exposure information. Then, exposure information for the pixel blocks of the next block row is embedded at this position starting from the beginning (0th row) of the next block row. In the example of this embodiment, OB region data 1106 follows immediately after the exposure information, but there may be cases where a plurality of blank data are inserted between the exposure information and the OB data.

[0060] Next, with reference to FIG. 12, the timing at which the exposure information is generated will be described. FIG. 12 is a diagram for explaining the timing at which the exposure information is generated. FIG. 12 schematically shows the processing timing related to the image data of the Nth frame and the (N + 1)th frame. In FIG. 12, the horizontal direction indicates the flow of time. Also, the vertical direction corresponds to the vertical region in the image.

[0061] When the vertical synchronization signal 1206 is input, according to the pixel reading timing of the Nth frame indicated by the diagonal solid arrow 1201 in FIG. 12, pixel data is read out in order from the upper pixels (in the upper direction of the image) in the image sensor unit 103. In this embodiment, since the rolling shutter method is used, the vertical OB region (VOB) is read out first, and then pixel data for each pixel from the 0th row to the 99th row in the pixel blocks of block row 0 is sequentially read out.

[0062] At time 1202, the pixel data reading for the pixel block of block row 0 of the Nth frame is completed. Since all the pixel data in the pixel block of block row 0 has been read, the generation of the exposure information to be used for the (N + 1)th frame is started. The exposure information generated using the pixel data of the exposure image of the Nth frame, which is one frame before the (N + 1)th frame, becomes the exposure information applied to the shooting of the (N + 1)th frame. Therefore, time 1202 is the generation start timing of the exposure information using the exposure image of block row 0 of the Nth frame. In terms of FIG. 1, this means that all the pixel data of block row 0 in the exposure image 122 is input to the exposure condition calculation unit 111. The exposure condition calculation unit 111 generates the exposure information to be applied to the pixel block of block row 0 of the (N + 1)th frame from the exposure image of block row 0 of the Nth frame over the period 1204 shown in FIG. 12.

[0063] At time 1203 when the period 1204 has elapsed since time 1202, the exposure condition calculation unit 111 completes the generation of the exposure information using the exposure image of block row 0 of the Nth frame. At the timing of this time 1203, the exposure information 1205 to be applied to the pixel block of block row 0 of the next (N + 1)th frame is generated. The exposure information generated based on the exposure image of the Nth frame is schematically shown in FIG. 12 in a form where the block row numbers are surrounded by squares. The exposure information generated by the exposure condition calculation unit 111 has the exposure time 112 and the analog gain value 113 stored as information for each region every time the generation is completed.

[0064] As can be seen from FIG. 12, the exposure information 1205 applied to the pixel block of block row 0 of the (N + 1) - th frame is generated before the pixel read - out timing of the (N + 1) - th frame indicated by the diagonal solid - line arrow 1207. Therefore, before the exposure image of block row 0 of the (N + 1) - th frame reaches the A / D conversion unit 104, the data overlap unit 105 can pre - read the generated exposure information 1205 as shown by the dotted - line arc - shaped arrow in FIG. 12. By pre - reading the thus - generated exposure information 1205, as shown in FIGS. 9 to 12, it becomes possible to transmit the exposure information together with the exposure image 122 using the LVDS signal line for sending the image data.

[0065] For other block rows as well, the generation of the exposure information is performed by the exposure condition calculation unit 111 in the same manner as for block row 0, and the information is sequentially stored as the exposure time 112 and the analog gain value 113 for each region.

[0066] FIG. 13 is a block diagram showing a schematic configuration of the controller 150 in the first embodiment. The data output from the imaging device 100 and transmitted via the LVDS signal line is received by the image input I / F 1301. The image input I / F 1301 analyzes the synchronization code and the identification code included in the received signal, detects the start of the frame data, and extracts the subsequent input data. Then, when the image input I / F 1301 detects the exposure information, it sends the detected exposure information to the exposure information holding unit 1303. Also, the image input I / F 1301 sends the OB region data and the exposure image for each region to the block - row buffer 1302.

[0067] The block - row buffer 1302 holds the OB region data and the exposure image for each region input from the image input I / F 1301 for the number of pixel rows included in a block row. In this embodiment, since one block row is 100 rows, after holding 100 - row pixel data, the data is sent to the gain calculation unit 1304 in the order of input for performing the exposure correction process.

[0068] The exposure information holding unit 1303 holds the exposure information input from the image input I / F 1301. In the present embodiment, since the exposure information is superimposed on the 0th to 4th block rows as shown in FIG. 11, for example, when the vertical OB region is 72 rows, the exposure information of block row 0 is input to the exposure information holding unit 1303 by the subsequent 5 rows of image data. As described above, the input of the exposure image for each region to the gain calculation unit 1304 is delayed by 100 pixel rows. Therefore, when processing the leading pixel of each block row in the gain calculation unit 1304, the gain calculation unit 1304 can perform the gain calculation as shown in FIGS. 6 to 8 using the exposure information of the block row already held in the exposure information holding unit 1303. The image data subjected to the gain calculation is sent to the subsequent image processing unit 1305, and subsequent image processing is performed.

[0069] As described above, before finishing outputting the exposure image of each region photographed using the exposure conditions applied for each region, the imaging device 100 can output the exposure conditions as exposure information from the signal line for image output without waiting for the exposure conditions for one frame to be complete for each region. Further, before finishing outputting the image data of the block row that is the target of the exposure correction process among the exposure images, the imaging device 100 finishes outputting the exposure information of the pixel block of that block row. As a result, the controller 150 can perform the gain calculation of the exposure image for each received region and perform the exposure correction process of the image with little delay without having a buffer for holding the image data for one frame. As a result, in the image processing system including the imaging device 100 and the controller 150, it becomes possible to obtain a high-dynamic range image with little delay.

[0070] Also, in the present embodiment, since the image is output without performing the correction process on the imaging device 100 side, the controller 150 can select and perform the correction process only on the pixel blocks required in the subsequent process when performing the correction process on the image. At that time, it can be realized by selectively using a path 1306 that bypasses the image data without passing through the gain calculation unit 1304 according to the pixel block.

[0071] In the above description, as shown in FIG. 9, the exposure information 905 is superimposed on the blank period before the pixel data for each row in the exposure image, specifically, on the blanking area 903 between the synchronization code, the identification code 904, and the OB area data 902. However, the exposure information only needs to be received on the controller side before the image data of the block row to be corrected using the same is aligned on the controller 150 side, and is not limited to being superimposed as shown in FIG. 9.

[0072] For example, as shown in FIG. 14, the exposure information may be superimposed on the blank period (blanking area) after the pixel data for each row in the exposure image. In the example shown in FIG. 14, in the frame data 1400, the exposure information 1405 is superimposed over several rows starting from the first row of each block row, and its position is the blanking area 1403 behind the pixel block data 1401 which is the exposure image for each area. In FIG. 14, the exposure information 1405 marked with numbers from 0 to 9 is the exposure information applied to the pixel blocks included in the corresponding block rows from block row 0 to block row 9. Since it is not on the front side of the pixel block data 1401 as in FIG. 9, the data of the OB area data 1402 follows immediately after the synchronization code and the identification code 1404. The exposure information 1405 for block row 0 is arranged starting from the first row at the position following the pixel block data [19,0]. Similarly, hereinafter, the exposure information 1405 applied to each block row is arranged at the first row at the position following the pixel block data of each block row in order.

[0073] Hereinafter, the second embodiment will be described with reference to FIGS. 15 to 18. In the first embodiment, the exposure information was superimposed on the signal line for outputting the image data and sent to the controller 150, but it is also possible to transmit the exposure information to the controller using an interface (I / F) other than the signal line for outputting the image. Hereinafter, an example of such a case will be described.

[0074] FIG. 15 is a block diagram showing a schematic configuration of an imaging apparatus 1500 according to the second embodiment. The imaging apparatus 1500 of the present embodiment includes various configurations that a general imaging apparatus has. However, for the sake of simplicity of illustration and description, FIG. 15 shows only the main components according to the present embodiment. In FIG. 15, the same reference numerals are given to the components common to FIG. 1, and redundant descriptions are omitted.

[0075] In the imaging apparatus 1500 shown in FIG. 15, when the exposure condition calculation unit 111 calculates the exposure time 112 and the analog gain value 113 for each region, they are sent to the exposure time control unit 109 and the gain control unit 110, respectively. Also, the exposure information of the exposure time 112 value and the analog gain value 113 for each region is sent to the internal memory 1501. When the exposure condition calculation unit 111 stores the exposure information in the internal memory 1501, it issues an instruction to output an interrupt pulse to the interrupt interface (hereinafter referred to as interrupt I / F) 1503 every time the amount of the stored exposure information reaches a certain determined value. When the interrupt I / F 1503 receives an output instruction of an interrupt pulse from the exposure condition calculation unit 111, it outputs an interrupt pulse on the interrupt signal line 1505.

[0076] The serial IO interface (SIO I / F) 1502 transmits and receives various information and the like to and from the controller 1506 via the serial signal line 1504. For example, the SIO I / F 1502 transmits and receives exposure information and a read request thereof to and from the controller 1506 via the serial signal line 1504. In the present embodiment, the image output unit 108 is an example of the first output means, and the SIO I / F 1502 is an example of the second output means. Also, the interrupt I / F 1503 is an example of the notification means, and the controller 1506 is an example of the processing device.

[0077] The internal memory 1501 stores exposure information and associated information in a format as shown in an example in FIG. 17. FIG. 17 is a diagram showing an example of the storage format of exposure information and associated information in the internal memory 1501. As shown in FIG. 17, the information stored in the internal memory 1501 is substantially the same as the information shown in FIG. 10 in the first embodiment. In this embodiment, access to the internal memory 1501 is performed in 32-bit units. Therefore, exposure information and the like are packed into 32 bits and stored in the internal memory 1501.

[0078] The horizontal direction ID 1702 of the pixel block is stored in bits 14 to 10, and the vertical direction ID 1704 of the pixel block is stored in bits 9 to 5. Also, a 3-bit exposure time ID 1706 is stored in bits 4 to 2, and a 2-bit gain ID 1707 is stored in bits 1 to 0. Further, 5-bit additional information 1709 is stored in bits 20 to 16. An 8-bit frame number 1701 is stored in bits 31 to 24. The frame number 1701 is information not shown in FIG. 10 in the first embodiment, but is information for showing the correspondence with the frame. The value of the frame number 1701 is incremented by 1 each time data for one frame is sent, and the frame number is assigned in a manner that returns to 0 when the maximum value is reached. This is particularly effective when the exposure information is held in a double-buffer format on the internal memory 1501.

[0079] Regarding the generation of the exposure information and the timing of storing it in the internal memory 1501, etc., it will be described with reference to FIG. 16. FIG. 16 is a diagram for explaining the operation timing of the imaging device 1500 in the second embodiment. For parts common to the part shown in FIG. 12 in the first embodiment, the numbers attached in FIG. 12 are used, and only the different parts will be described.

[0080] The exposure condition calculation unit 111 generates, over the period 1204 shown in FIG. 16, the exposure information to be applied to the pixel block of block row 0 of the (N + 1)-th frame from the exposure image of block row 0 of the N-th frame. At time 1203 when the period 1204 has elapsed since time 1202, the exposure condition calculation unit 111 completes the generation of the exposure information using the exposure image of block row 0 of the N-th frame. At the timing of this time 1203, the exposure information 1205 to be applied to the pixel block of block row 0 of the next (N + 1)-th frame is generated. The exposure information generated based on the exposure image of the N-th frame is schematically shown in FIG. 16 in a form where the block row numbers are surrounded by squares. Each time the generation of the exposure information generated by the exposure condition calculation unit 111 is completed, the information is stored as the exposure time 112 and the analog gain value 113 for each region. Also, in the present embodiment, the exposure information is also stored in the internal memory 1501. Thereafter, each time a new block row exposure image is input, the exposure condition corresponding to the block row is sequentially generated by the exposure condition calculation unit 111, and the exposure information is accumulated in the internal memory 1501.

[0081] At time 1602, the exposure condition calculation unit 111 completes the generation of the exposure information to be applied to the pixel block of block row 6 of the (N + 1)-th frame, which is generated using the exposure image of block row 6 of the N-th frame. When the exposure information to be applied to the pixel block of block row 6 of the (N + 1)-th frame is written to the internal memory 1501 at the timing of time 1602, the exposure condition calculation unit 111 issues an instruction to the interrupt I / F 1503 to output an interrupt pulse. The interrupt I / F 1503 that has received the instruction outputs an interrupt pulse on the interrupt signal line 1505 as shown by the interrupt signal 1601 in FIG. 16.

[0082] Similarly, at time 1603, the exposure condition calculation unit 111 completes the generation of exposure information to be applied to the pixel block of block row 13 of the (N + 1)th frame, which is generated using the exposure image of block row 13 of the Nth frame. Further, at time 1604, the exposure condition calculation unit 111 completes the generation of exposure information to be applied to the pixel block of block row 19 of the (N + 1)th frame, which is generated using the exposure image of block row 19 of the Nth frame. Even at the timings of time 1603 and time 1604, the exposure condition calculation unit 111 issues an instruction to the interrupt I / F 1503 to output an interrupt pulse. Then, the interrupt I / F 1503 that has received the instruction outputs an interrupt pulse on the interrupt signal line 1505 as shown by the interrupt signal 1601. This is the same for subsequent (N + 1)th frames, and interrupt pulses are generated at the timings of time 1605, time 1606, and time 1607.

[0083] In this way, the interrupt I / F 1503 that has received an instruction from the exposure condition calculation unit 111 outputs an interrupt pulse to the interrupt signal line 1505 and notifies the controller 1506 that the exposure information has been generated and stored. When the controller 1506 detects an interrupt pulse input via the interrupt signal line 1505, it attempts to read the exposure information from the internal memory 1501 of the imaging device 1501 via the serial signal line 1504. The imaging device 1500 receives a read request from the controller 1506 at the SIO I / F 1502 and outputs the exposure information from the internal memory 1501. In the example of this embodiment, without waiting for all the exposure information applicable to one frame to be complete, it is output in three divisions.

[0084] FIG. 18 is a block diagram showing a schematic configuration of the controller 1506 in the second embodiment. In FIG. 18, the same reference numerals are given to the components common to those in FIG. 13, and the overlapping descriptions are omitted. When an interrupt pulse is input from the imaging device 1500 via the interrupt I / F 1802, the control CPU 1803 inside the controller 1506 instructs the SIO I / F 1801 to read out the exposure information from the imaging device 1500. The instructed SIO I / F 1801 outputs a read request for reading out the exposure information to the imaging device 1500. At this time, for the interrupt pulse input at the timing of time 1602 shown in FIG. 16, a read request is output so as to acquire the exposure information from block row 0 to block row 6. Similarly, for the interrupt pulse input at the timing of time 1603 shown in FIG. 16, a read request is output so as to acquire the exposure information from block row 7 to block row 13. Also, for the interrupt pulse input at the timing of time 1604 shown in FIG. 16, a read request is output so as to acquire the exposure information from block row 14 to block row 19.

[0085] It is assumed that the transfer speed of the serial IO (SIO) is set so that the time required for transferring the exposure information for one block row is shorter than the time required for transferring the image data for one block row from the imaging device 1500 to the controller 1506. Also, regarding which block row of exposure information is to be acquired for each interrupt pulse, the setting exchange between the controller 1506 and the imaging device 1500 is performed in advance and determined by parameter setting or the like. Note that the generation timing of the interrupt pulse and the number of exposure information to be exchanged accordingly in the example of this embodiment are just examples and are not restricted to the example described above.

[0086] In this way, the controller 1506 that has acquired the exposure information from the imaging device 1500 sequentially transfers the exposure information from the SIO I / F 1801 to the exposure information holding unit 1303. The gain calculation unit 1304 reads out the exposure information of the corresponding block row from the exposure information holding unit 1303 according to the position of the pixel data output from the block row buffer 1302, and performs gain calculation.

[0087] As described above, before finishing outputting the exposure images of each area photographed using the exposure conditions applied to each area, the imaging device 1500 can output the exposure conditions as exposure information without waiting for the exposure conditions for each area to be complete for one frame. Also, before finishing outputting the image data of the block row that is the target of the exposure correction process within the exposure image, the imaging device 1500 finishes outputting the exposure information of the pixel blocks of that block row. As a result, the controller 1506 can perform gain calculation on the received exposure images for each area and perform image correction processing with little delay without having a buffer for holding the image data for one frame. As a result, in the image processing system composed of the imaging device 1500 and the controller 1506, it becomes possible to obtain a high-dynamic-range image with little delay.

[0088] (Other Embodiments of the Present Invention) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0089] It should be noted that the above embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

Explanation of Reference Numerals

[0090] 100, 1500: Imaging device 103: Image sensor unit 104: A / D conversion unit 105: Data superimposing unit 108: Image output unit 109: Exposure time control unit 110: Gain control unit 111: Exposure condition calculation unit 112: Exposure time 113: Analog gain value 150, 1506: Controller 1501: Internal memory 1502, 1801: Serial I / O I / F (SIO I / F) 1503, 1802: Interrupt I / F 1803: Control CPU

Claims

1. An imaging device, which divides an imaging area of an imaging element into a plurality of areas, controls exposure conditions for each area, and images with the imaging element to obtain a first image, and outputs, to the outside of the imaging device, exposure information for each area indicating the exposure conditions applied to each area when imaging the first image; and has a superimposing means for superimposing the exposure information for each area on the first image of the corresponding area when outputting the exposure information, and finishes outputting the exposure information for a region before finishing outputting an image of the region that is a target of exposure correction processing within the first image. The imaging device is characterized by this.

2. The imaging device according to claim 1, wherein the exposure conditions include an exposure time corresponding to each area of the imaging element and an analog gain related to analog / digital conversion.

3. The imaging device according to claim 1 or 2, wherein the superimposing means superimposes the exposure information during a blank period when outputting the first image.

4. The imaging device according to claim 3, wherein the superimposing means arranges the exposure information during a blank period before data for each row in the first image.

5. The imaging device according to claim 3, wherein the superimposing means arranges the exposure information during a blank period after data for each row in the first image.

6. The imaging device according to any one of claims 1 to 5, further comprising exposure condition calculation means for calculating the exposure conditions applied to each area when imaging the first image based on a second image one frame before the first image.

7. An imaging device according to any one of claims 1 to 6, and An image processing system comprising a processing device that receives the first image and the exposure information output from the imaging device and performs exposure correction processing on the first image using the exposure information. **Claim 8** The image processing system according to claim 7, wherein the processing device performs the exposure correction processing using the exposure information on a selected area within the first image. **Claim 9** A control method for an imaging device, comprising: an output step of dividing an imaging area of an image sensor into a plurality of areas, controlling exposure conditions for each area, imaging a first image by the image sensor, and outputting, to the outside of the imaging device, exposure information for each area indicating the exposure conditions applied to each area when the first image is imaged; a superimposing step of superimposing the exposure information for each area on the first image of the corresponding area when the exposure information for each area is output; and a control method for an imaging device, characterized in that the output of the exposure information for a region is terminated before the output of the image of the region targeted for exposure correction processing within the first image is completed.

Images