Imaging device and solid-state imaging device

Abstract

The present invention is characterized by comprising a photoreceptor array having a plurality of photoreceptor groups, each having a plurality of photoreceptors, and a compression unit that generates a compressed signal for each photoreceptor group by compressing signals output from the plurality of photoreceptors in the photoreceptor group.

Description

Technical Field

[0001] The present invention relates to an imaging device including a light receiving element array and a solid-state imaging device.

Background Art

[0002] Conventionally, as a solid-state imaging device such as a CMOS used in an imaging device such as a digital camera, a solid-state imaging device described in Patent Document 1 (Japanese Patent Application Laid-Open No. 2022-164387) is known.

[0003] As shown in FIG. 11, this solid-state imaging device 500 has a light receiving element array 501 in which light receiving elements 502 including photoelectric conversion elements are two-dimensionally arranged in an array and are connected to pixel drive wirings 503 and vertical signal lines 504. As peripheral circuits, it has a column-parallel signal processing circuit 505, an output circuit 506, a timing control circuit 507, a horizontal scanning circuit 508, a vertical scanning circuit 509, and the like.

[0004] In this solid-state imaging device 500, when the light receiving element array 501 is exposed during imaging with an imaging device (such as a digital camera) including the solid-state imaging device 500, signals output from each of the plurality of light receiving elements 502 constituting the light receiving element array 501 are output to the outside through the vertical signal lines 504, the column-parallel signal processing circuit 505, the output circuit 506, and the like.

[0005] In recent years, power saving of the solid-state imaging device 500 and an imaging device including the solid-state imaging device 500 has been demanded. However, in this solid-state imaging device 500 and the imaging device, since all the signals output from each of the plurality of light receiving elements 502 included in the light receiving element array 501 are output to the outside (a so-called full amount readout configuration), the amount of signals output inside and outside the solid-state imaging device 500 is large, and in the solid-state imaging device 500, power consumption corresponding to the amount of signals is required. Therefore, power saving in the above solid-state imaging device 500 and an imaging device including this solid-state imaging device 500 has not been sufficient.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] Therefore, an object of the present invention is to provide an imaging device and a solid-state imaging device capable of suppressing power consumption.

Means for Solving the Problems

[0008] The imaging device according to the present invention includes: a light-receiving element array including a plurality of light-receiving element groups each having a plurality of light-receiving elements; a compression unit that generates a compressed signal obtained by compressing signals output from the plurality of light-receiving elements of each light-receiving element group, for each light-receiving element group.

[0009] Further, the imaging device may include: a solid-state imaging device having the light-receiving element array and the compression unit.

[0010] Further, in the imaging device, in the light-receiving element group, when the light-receiving element array is exposed, the exposure amount of at least one light-receiving element in the light-receiving element group may be different from the exposure amount of other light-receiving elements in the light-receiving element group.

[0011] Further, in the imaging device, in the light-receiving element group, the difference in the exposure amount may be caused by a difference in the exposure time of each light-receiving element.

[0012] Further, in the imaging device, the compressed signal may include a weighted average value or an average value of signal values included in signals output from each light-receiving element of the light-receiving element group.

[0013] Further, the imaging device may include a reconstruction processing unit that reconstructs the compression signal generated by the compression unit into a pixel signal of a pixel corresponding to the compression signal in the output image.

[0014] Also, in the imaging device, the reconstruction processing unit may reconstruct the compression signal into the pixel signal based on information before compression included in the compression signal.

[0015] Also, in the imaging device, each light-receiving element of the light-receiving element group has a photoelectric conversion element respectively, when there are light-receiving elements in the light-receiving element group that have overflowed and those that have not overflowed in the photoelectric conversion elements when the light-receiving element array is exposed, the reconstruction processing unit calculates an expected value of a signal value that can be obtained when the photoelectric conversion elements of the respective light-receiving elements have a capacity that does not overflow due to the exposure, and may reconstruct the compression signal into the pixel signal.

[0016] Also, in the imaging device, the plurality of light-receiving elements included in the light-receiving element group include a reference light-receiving element that is a reference light-receiving element and non-reference light-receiving elements other than the reference light-receiving element, the compression unit may generate the compression signal based on a difference between a reference signal value that is a signal value included in a signal output from the reference light-receiving element and a non-reference signal value that is a signal value included in a signal output from the non-reference light-receiving element.

[0017] Also, in the imaging device, in the non-reference signal value included in the signal output from the non-reference light-receiving element of the light-receiving element group, the compression unit when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, generates the compression signal including a weighted average value or an average value of signal values output from each light-receiving element of the light-receiving element group, When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated by deleting the signal output from each of the light-receiving elements that corresponds to the second signal value, and adding position information of the non-reference light-receiving element that output the first signal value or the second signal value relative to the reference light-receiving element to the signal output from the reference light-receiving element.

[0018] Moreover, in the imaging device, The compression unit is configured to: When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, the compressed signal is generated including a weighted average or an average of the signal values output from each light receiving element of the light receiving element group; When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element.

[0019] Further, the solid-state imaging device according to the present invention comprises: a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a functional block diagram of an imaging device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of a solid-state imaging element included in the imaging device. [Diagram 3] FIG. 3 is a diagram showing the configuration of a light receiving element group included in the solid-state imaging device. [Figure 4] FIG. 4 is a timing chart of the light receiving element group. [Figure 5] FIG. 5 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is no light receiving element in the light receiving element group where the photoelectric conversion element has overflowed. [Figure 6] FIG. 6 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is a light receiving element in a light receiving element group where the photoelectric conversion element has overflowed. [Figure 7] FIG. 7 is a diagram showing the configuration of a solid-state imaging device according to the second embodiment. [Figure 8] FIG. 8 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there are only non-reference light receiving elements in a light receiving element group whose pixel values differ from the pixel value of the reference light receiving element by less than a predetermined threshold. [Figure 9] FIG. 9 is a diagram for explaining the processing in the compression unit and the processing in the reconstruction processing unit when there is a non-reference light receiving element in a light receiving element group whose pixel value differs from the pixel value of the reference light receiving element by a predetermined threshold or more. [Figure 10] FIG. 10 is a diagram showing the processing in the compression unit and the processing in the reconstruction processing unit when an image is formed on a light receiving element array having a plurality of light receiving element groups. [Figure 11] FIG. 11 is a diagram showing the configuration of a conventional solid-state imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The imaging device according to this embodiment includes: a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression unit that generates a compressed signal for each light receiving element group by compressing signals output from the plurality of light receiving elements of the light receiving element group.

[0022] In this way, the compression unit generates a compressed signal for each light receiving element group and outputs each of these compressed signals (i.e., signals with reduced signal volume), thereby reducing power consumption in the configuration downstream of the compression unit. Note that signal compression in the compression unit according to the present invention is processing that results in a signal with a smaller signal volume than a signal generated by simple addition (pixel addition, etc.) of the signals output from each light receiving element.

[0023] The imaging device is The image sensor may include a solid-state image sensor having the light receiving element array and the compression section.

[0024] In addition, in the imaging device, In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group may be different from the amount of exposure of the other light receiving elements of the light receiving element group.

[0025] According to this configuration, it is possible to reduce the signal amount while including information on each signal output from light receiving elements with different exposure amounts.

[0026] in this case, In the light receiving element group, the difference in the amount of exposure may be caused by a difference in exposure time of each light receiving element.

[0027] In this way, by adjusting the exposure time of the light receiving elements and varying the amount of exposure of the light receiving elements within a light receiving element group, it becomes possible to change the amount of exposure of the light receiving elements by control.

[0028] In addition, in the imaging device, The compressed signal may include a weighted average or an average value of signal values included in the signals output from the light receiving elements of the light receiving element group.

[0029] Moreover, the imaging device A reconstruction processing unit may be provided that reconstructs the compression signal generated by the compression unit into a pixel signal of a pixel corresponding to the compression signal in the output image.

[0030] According to such a configuration, the pixel signal is reconstructed on the downstream side of the compression unit, and by outputting this pixel signal, the image quality of the obtained image is improved.

[0031] Also, in the imaging device, The reconstruction processing unit may reconstruct the compression signal into the pixel signal based on the information before compression included in the compression signal.

[0032] Also, in the imaging device, Each light-receiving element of the light-receiving element group has a photoelectric conversion element respectively, When there are light-receiving elements in the light-receiving element group where the photoelectric conversion elements overflowed and those that did not overflow when the light-receiving element array was exposed, the reconstruction processing unit calculates the expected value of the signal value that can be obtained when the photoelectric conversion elements of each light-receiving element have a capacity that does not overflow due to the exposure, and may reconstruct the compression signal into the pixel signal.

[0033] According to such a configuration, an image with a dynamic range larger than the dynamic range defined by each light-receiving element can be obtained, that is, imaging with a high dynamic range becomes possible.

[0034] Also, in the imaging device, The plurality of light-receiving elements included in the light-receiving element group have a reference light-receiving element that is a reference light-receiving element and non-reference light-receiving elements other than the reference light-receiving element. The compression unit may generate the compression signal based on the difference between the reference signal value that is the signal value included in the signal output from the reference light-receiving element and the non-reference signal value that is the signal value included in the signal output from the non-reference light-receiving element.

[0035] In this way, by compressing the signal based on the difference between the reference signal value and the non-reference signal value for each position of the light receiving element, the difference in the signal amount for each position can be reconstructed in the reconstruction processing unit, and thereby, the resolution can be ensured in the output image.

[0036] Specifically, In the non-reference signal value included in the signal output from the non-reference light receiving element of the light receiving element group, the compression unit when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, generates the compression signal including the weighted average value or the average value of the signal values output from each light receiving element of the light receiving element group, when there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold and a second signal value that is a non-reference signal value whose difference from the reference signal value is greater than or equal to the predetermined threshold, deletes the signal corresponding to the second signal value among the signals output from each light receiving element, and adds the position information of the non-reference light receiving element that output the first signal value or the second signal value with respect to the reference light receiving element to the signal output from the reference light receiving element, thereby generating the compression signal.

[0037] In this way, by configuring the compression signal with the signal output from the reference light receiving element and the position information added thereto, the signal amount can be suitably suppressed as compared with the case where all the signals output from each light receiving element of the light receiving element group are output to the downstream configuration.

[0038] Further, in the non-reference signal value included in the signal output from the non-reference light receiving element of the light receiving element group, the compression unit when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, generates the compression signal including the weighted average value or the average value of the signal values output from each light receiving element of the light receiving element group, When there are a first signal value, which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold value, and a second signal value, which is a non-reference signal value whose difference from the reference signal value is greater than or equal to the predetermined threshold value, a weighted average value or an average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that output the first signal value or the second signal value with respect to the reference light receiving element may be included to generate the compressed signal.

[0039] Also, the solid-state imaging device according to the present embodiment includes a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements, and a compression unit that generates a compressed signal obtained by compressing signals output from the plurality of light receiving elements of each light receiving element group for each light receiving element group.

[0040] In this way, by the compression unit generating a compressed signal for each light receiving element group and outputting each of these compressed signals (i.e., signals in a state where the signal amount is suppressed), power consumption can be suppressed in the configuration downstream of the compression unit.

[0041] Hereinafter, a first embodiment of the present invention will be described with reference to the accompanying drawings.

[0042] As shown in FIGS. 1 and 2, the imaging device 100 according to the first embodiment includes a light receiving element array 2 including a plurality of light receiving element groups 20 each having a plurality of light receiving elements 21, and at least one compression unit 10 that generates a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20 for each light receiving element group 20. The imaging device 100 of the present embodiment includes the same number of compression units 10 as the number of light receiving element groups 20, and each compression unit 10 generates a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 included in the corresponding light receiving element group 20 for each light receiving element group 20. The imaging device 100 can image an object such as a digital camera, a smartphone, or a tablet device, and the imaging device 100 of the present embodiment is, for example, a smartphone.

[0043] Specifically, as shown in FIG. 1, the imaging device 100 includes an imaging unit 101 and a control unit 102. The imaging device 100 of the present embodiment further includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

[0044] Under the control of the control unit 102, the imaging unit 101 converts the image of the object formed by the optical system of the imaging unit 101 into an electrical signal, then performs noise reduction processing, etc., and outputs the digital data as the image data of the output image. Specifically, the imaging unit 101 includes an optical system composed of at least one optical element such as a lens, and a solid-state imaging device 1 that converts the image of the object (imaging object) formed through the optical system into an electrical signal (i.e., captures an image).

[0045] The solid-state imaging device 1 is an element incorporated in, for example, a smartphone, a digital camera, etc., for imaging an object. The solid-state imaging device 1 of the present embodiment is, for example, a CMOS image sensor.

[0046] Specifically, as shown in FIG. 2, this solid-state imaging device 1 includes a light-receiving element array 2, a first circuit unit 3 and a second circuit unit 4 connected to the light-receiving element array 2, and a signal processing unit 5 connected to the second circuit unit 4. These light-receiving element array 2, first circuit unit 3, second circuit unit 4, and signal processing unit 5 are arranged on the same semiconductor substrate or on a plurality of electrically connected semiconductor substrates.

[0047] The light receiving element array 2 has a plurality of light receiving element groups 20 arranged two-dimensionally in a matrix. The light receiving element array 2 also has a plurality of row signal lines 25 arranged in each row and extending in the row direction (the left-right direction in FIG. 2 ) for the plurality of light receiving element groups 20 arranged in a matrix, and a plurality of column signal lines 26 arranged in each column and extending in the column direction (the up-down direction in FIG. 2 ). Each of the plurality of row signal lines 25 is connected to the first circuit unit 3, and each of the plurality of column signal lines 26 is connected to the second circuit unit 4.

[0048] As shown in FIG. 3 , each of the multiple light receiving element groups 20 includes multiple light receiving elements (pixels) 21. In the light receiving element group 20 of this embodiment, the multiple light receiving elements 21 are adjacent to one another. Specifically, the multiple light receiving elements 21 included in the light receiving element group 20 are arranged adjacent to one another in the row and column directions. In this embodiment, each light receiving element group 20 includes four light receiving elements 21 (a first light receiving element 21a, a second light receiving element 21b, a third light receiving element 21c, and a fourth light receiving element 21d) arranged in two rows and two columns (i.e., a 2×2 array). Each light receiving element group 20 also includes an amplifier 22 to which the light receiving elements 21a, 21b, 21c, and 21d are connected.

[0049] In FIG. 2, in order to make it easier to understand the configuration of the light receiving element array 2, the multiple light receiving element groups 20 are shown with a gap between adjacent light receiving element groups 20 in the row direction and between adjacent light receiving element groups 20 in the column direction, but in reality, in the light receiving element array 2, the multiple light receiving element groups 20 are arranged in a matrix with adjacent light receiving element groups 20 in the row direction adjacent to each other and adjacent light receiving element groups 20 in the column direction adjacent to each other.

[0050] In FIG. 3, for the sake of easy understanding of the configuration of the light-receiving element group 20, the light-receiving elements 21a, 21b, 21c, and 21d are shown with intervals provided between adjacent light-receiving elements 21 in the row direction and between adjacent light-receiving elements 21 in the column direction. However, actually, in the light-receiving element group 20, a plurality of light-receiving elements 21 are arranged in a matrix with adjacent light-receiving elements 21 adjacent to each other in the row direction and adjacent light-receiving elements 21 adjacent to each other in the column direction.

[0051] Each of the plurality of light-receiving elements 21 photoelectrically converts the input light (incident light) and accumulates an amount of charge corresponding to the input light amount inside, and outputs this accumulated charge as an electrical signal. The range (signal range) of the signal (pixel value) S1 that can be output from each light-receiving element 21 in this embodiment is 0 to 500. Also, each light-receiving element 21 has a photoelectric conversion element (in the example of this embodiment, a photodiode) 211.

[0052] Each of the plurality of row signal lines 25 transmits the reset signal RES, the first to fourth charge transfer signals TX1 to TX4, and the select signal SEL output from the first circuit unit 3 to each connected light-receiving element group 20.

[0053] Here, the reset signal RES is a signal for resetting the charges of the light-receiving elements 21a, 21b, 22c, and 21d. The first charge transfer signal TX1 is a signal for turning on and off the first light-receiving element 21a. The second charge transfer signal TX2 is a signal for turning on and off the second light-receiving element 21b. The third charge transfer signal TX3 is a signal for turning on and off the third light-receiving element 21c. The fourth charge transfer signal TX4 is a signal for turning on and off the fourth light-receiving element 21d.

[0054] Each of the plurality of column signal lines 26 is connected to each light-receiving element group 20 in the corresponding column and transmits the compression signal S2 output from each light-receiving element group 20 to the second circuit unit 4.

[0055] The first circuit section 3 outputs a reset signal RES, first to fourth charge transfer signals TX1 to TX4, a select signal SEL, etc. to each light receiving element group 20, thereby controlling the light receiving element array 2 and performing control for compressing signals S1a, S1b, S1c, S1d output from each light receiving element 21 for each light receiving element group 20. Incidentally, hereinafter, the signals S1a, S1b, S1c, S1d output from each light receiving element 21a, 21b, 22c, 21d of the light receiving element group 20 may be simply referred to as signal S1.

[0056] The second circuit section 4 performs signal processing such as reading out the compressed signal S2 from each light receiving element group 20, noise removal, and A / D (Analog / Digital) conversion of each read compressed signal S2.

[0057] The signal processing section 5 is a part that performs various processes on each compressed signal S2 output from the second circuit section 4. In this embodiment, the signal processing section 5 is an in-sensor ISP (Image Signal Processor).

[0058] In the solid-state imaging device 1 configured as described above, signals as shown in FIG. 4 are respectively transmitted from the first circuit section 3 to each light receiving element group 20, whereby signals S1a, S1b, S1c, S1d respectively output from each light receiving element 21a, 21b, 21c, 21d of the light receiving element group 20 are compressed to generate a compressed signal S2.

[0059] Specifically, in each light receiving element group 20, at time t1, the reset signal becomes ON and the select signal becomes OFF. At this time, the first to fourth charge transfer signals TX1 to TX4 become ON, and the charges of each light receiving element 21a, 21b, 21c, 21d are reset.

[0060] Subsequently, at time t2, the first charge transfer signal becomes OFF, and after a predetermined interval, the second charge transfer signal becomes OFF at time t3, the third charge transfer signal becomes OFF at time t4, and the third charge transfer signal becomes OFF at time t5.

[0061] Due to the difference in this OFF time (exposure time), when the light-receiving element array 2 is exposed during shooting or the like in the imaging device 100, the exposure amount of at least one light-receiving element 21 in the light-receiving element group 20 becomes different from the exposure amount of the other light-receiving elements 21 in the light-receiving element group 20. In the imaging device 100 of the present embodiment, the exposure amounts of the respective light-receiving elements 21a, 21b, 21c, and 21d in the light-receiving element group 20 are different from each other.

[0062] Subsequently, when the reset signal becomes ON and the select signal becomes OFF at time t6, and then the reset signal becomes OFF and the select signal becomes ON at time t9.

[0063] Then, at times t7 and t8 between time t6 and time t9, the first to fourth charge transfer signals TX1 to TX4 are simultaneously turned ON and OFF. As a result, while each of the light-receiving elements 21a, 21b, 21c, and 21d is OFF (specifically, the first light-receiving element 21a is during the first accumulation time T1 (from time t2 to time t7), the second light-receiving element 21b is during the second accumulation time T2 (from time t3 to time t7), the third light-receiving element 21c is during the third accumulation time T3 (from time t4 to time t7), and the fourth light-receiving element 21d is during the fourth accumulation time T4 (from time t5 to time t7)), the charges accumulated in each of the light-receiving elements 21a, 21b, 21c, and 21d are simultaneously output as electrical signals to the column signal line 26 connected to the light-receiving element group 20 including these light-receiving elements 21a, 21b, 21c, and 21d. Thereby, the signals S1a, S1b, S1c, and S1d respectively output from each of the light-receiving elements 21a, 21b, 21c, and 21d are compressed to become a compressed signal S2. Note that the relationship between the respective accumulation times T1 to T4 in the imaging device 100 of the present embodiment is T2 = (3 / 4)T1, T3 = (2 / 4)T1, and T4 = (1 / 4)T1.

[0064] The compressed signal S2 generated for each light-receiving element group 20 includes the average value of the pixel values (signal values) included in the signals S1 output from the respective light-receiving elements 21a, 21b, 21c, and 21d of the light-receiving element group 20.

[0065] For example, when focusing on one light receiving element group 20A, if the exposure amounts of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A are within the signal range, as shown in FIG. 5, when the pixel value included in the signal S1a output from the first light receiving element 21a is 400, the pixel value included in the signal S1b output from the second light receiving element 21b is 300, the pixel value included in the signal S1c output from the third light receiving element 21c is 200, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 100, the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is (400+300+200+100) / 4=250 In this embodiment, the "average value of the signal" refers to an unweighted sum of the signal.

[0066] Furthermore, when the exposure amount of some of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A exceeds the signal range, for example, as shown in FIG. 6, when the pixel value included in the signal S1a output from the first light receiving element 21a is 500, the pixel value included in the signal S1b output from the second light receiving element 21b is 500, the pixel value included in the signal S1c output from the third light receiving element 21c is 500, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 300, the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is (500+500+500+300) / 4=450 It becomes.

[0067] As described above, in the solid-state imaging device 1 of this embodiment, the above-mentioned compression unit 10 that generates, for each light-receiving element group 20, compressed signals S2 by compressing signals S1a, S1b, S1c, and S1d output from the plurality of light-receiving elements 21a, 21b, 21c, and 21d that each light-receiving element group 20, is made up of the light-receiving element group 20 and the first circuit unit 3 that controls the light-receiving element group 20. That is, the solid-state imaging device 1 of this embodiment includes a plurality of compression units 10 (the number of which corresponds to the number of light-receiving element groups 20).

[0068] The control unit 102 controls each part of the imaging device 100 according to the input signals and programs. This control unit 102 has a reconstruction processing unit 102A that reconstructs the compressed signal S2 output from the solid-state imaging device 1 into the pixel signal S3. Note that the configuration is not limited to the control unit 102 controlling the entire imaging device 100. A configuration in which a plurality of hardware share the processing to control the entire imaging device 100 may also be used.

[0069] The reconstruction processing unit 102A reconstructs the compressed signal (compressed signal corresponding to each light-receiving element group 20) S2 output from the solid-state imaging device 1 into the pixel signal S3 of the pixel corresponding to the compressed signal S2 in the output image based on the information before compression included in the compressed signal S2.

[0070] In the control unit 102 having this reconstruction processing unit 102A, each compressed signal S2 output from the solid-state imaging device 1 is reconstructed to generate a pixel signal S3 respectively, whereby an output image (image data) is generated. The control unit 102 outputs (displays, etc.) this output image to the outside through the display unit 106.

[0071] Specifically, when each photoelectric conversion element 211 of the light-receiving element group 20 did not overflow when the light-receiving element array 2 was exposed, the reconstruction processing unit 102A reconstructs the compressed signal S2 into the pixel signal S3 based on the pixel value included in the compressed signal S2 and the accumulation times T1, T2, T3, T4 of the respective light-receiving elements 21a, 21b, 21c, 21d. Further, when there are light-receiving elements 21 in which the photoelectric conversion element 211 overflowed and light-receiving elements 21 in which the photoelectric conversion element 211 did not overflow in the light-receiving element group 20 when the light-receiving element array 2 was exposed, the reconstruction processing unit 102A reconstructs the compressed signal S2 into the pixel signal S3 so as to include the predicted value of the pixel value (signal value) that can be obtained when the capacity of the photoelectric conversion element 211 of each light-receiving element 21 is such that it does not overflow due to exposure.

[0072] More specifically, as shown in FIG. 5 for example, when the pixel value included in the compression signal S2 output from the light-receiving element group 20A is 250, the reconstruction processing unit 102A performs the following calculation with the pixel value included in the pixel signal S3 being x, T4 = (1 / 4)T1, T3 = (2 / 4)T1, and T2 = (3 / 4)T1. {x+(3 / 4)x+(2 / 4)x+(1 / 4)x} / 4=250 x=400 Then, the reconstruction processing unit 102A reconstructs the pixel signal S3 from the compression signal S2 so that the pixel value x included in the pixel signal S3 becomes 400.

[0073] Further, as shown in FIG. 6 for example, when the pixel value included in the compression signal S2 output from the light-receiving element group 20A is 450 (i.e., when there are a light-receiving element 21 in which the photoelectric conversion element 211 has overflowed and a light-receiving element 21 in which the photoelectric conversion element 211 has not overflowed in the light-receiving element group 20), the reconstruction processing unit 102A performs the following calculation with the pixel value included in the signal S1 obtained from the light-receiving element 21d at the fourth accumulation time T4 being x1. (500+500+500+x1) / 4=450 x1=300 Since the pixel value x1 included in the signal S1d output from the light-receiving element 21d at the fourth accumulation time T4 is 300, the reconstruction processing unit 102A converts the signal S1d (pixel value 300) into the equivalent of the signal S1a output from the light-receiving element 21a at the first accumulation time T1. Specifically, since the first accumulation time T1 is four times the fourth accumulation time T4, it becomes 300×4 = 1200, and thereby the reconstruction processing unit 102A reconstructs the pixel signal S3 so that the pixel value (expected value) x2 included in the pixel signal S3 becomes 1200.

[0074] As described above, according to the reconstruction processing unit 102A of the image capturing device 100 of this embodiment, even if light of an amount such that each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 has a value (pixel value) greater than the signal range is incident (i.e., even if there is a light receiving element 21 in the light receiving element group 20 whose photoelectric conversion element 211 overflows), an output image is generated without whiteout. That is, the image capturing device 100 of this embodiment is capable of high dynamic range (HDR) image capturing with a signal range of 0 to 2000.

[0075] The nonvolatile memory 103 is an electrically erasable and recordable nonvolatile memory. In this embodiment, the nonvolatile memory 103 stores an OS (operating system), which is basic software executed by the control unit 102, and applications that cooperate with the OS to realize applied functions. The working memory 104 is used as an image display memory for the display unit 106, a working area for the control unit 102, etc.

[0076] The operation unit 105 is used by a user or the like to input instructions to the imaging device 100. The operation unit 105 of this embodiment has a power button for instructing ON / OFF of the power supply of the imaging device 100, a touch panel formed on the display unit 106, and the like.

[0077] The display unit 106 displays an output image (image data) and displays characters for operation.

[0078] The recording medium 107 records the image data output from the imaging unit 101 .

[0079] The connection unit 108 is an interface for connecting to an external device, and the imaging device 100 exchanges data with the external device via the connection unit 108.

[0080] The short-range wireless communication unit 109 is a communication unit for performing short-range wireless communication. The short-range wireless communication unit 109 is composed of an antenna for wireless communication, a modulation / demodulation circuit for processing wireless signals, and a communication controller.

[0081] The public network connection unit 110 is an interface for performing public wireless communication. Through this public network connection unit 110, the imaging device 100 conducts communication for calls with other devices. At this time, the control unit 102 realizes the above-mentioned call by inputting and outputting audio signals via the microphone 111 and the speaker 112. The public network connection unit 110 of the present embodiment is an antenna, and the control unit 102 connects to the public network via this antenna.

[0082] The above imaging device 100 includes a light-receiving element array 2 including a plurality of light-receiving element groups 20 each having a plurality of light-receiving elements 21, and a compression unit 10 that generates a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light-receiving elements 21 of the light-receiving element group 20 for each light-receiving element group 20. In this way, by generating the compressed signal S2 for each light-receiving element group 20 in the compression unit 10 and outputting these compressed signals S2 (that is, signals in a state where the signal amount is suppressed), the power consumption can be suppressed in the configuration downstream of the compression unit 10 (in the example of the present embodiment, the control unit 102, etc.).

[0083] Further, the imaging device 100 of the present embodiment includes a solid-state imaging device 1 having a light-receiving element array 2 and a compression unit 10. According to such a configuration, in the solid-state imaging device 1, the compression unit 10 generates the compressed signal S2 for each light-receiving element group 20, and by outputting these compressed signals S2 (that is, signals in a state where the signal amount is suppressed), the power consumption can be suppressed in the configuration downstream of the solid-state imaging device 1 (specifically, the compression unit 10).

[0084] Further, the imaging device 100 of the present embodiment includes a reconstruction processing unit 102A that reconstructs the compressed signal S2 generated by the compression unit 10 into a pixel signal S3 of a pixel corresponding to the compressed signal S2 in the output image based on the information before compression included in the compressed signal S2. Therefore, the pixel signal S3 is reconstructed downstream of the compression unit 10, and the image quality of the image (i.e., the image obtained from the display unit 106) displayed on the display unit 106 or the like is improved by outputting the pixel signal S3 to the display unit 106 or the like.

[0085] Also, in the imaging device 100 of the present embodiment, in the light receiving element group 20, when the light receiving element array 2 is exposed, the exposure amount of at least one light receiving element 21 in the light receiving element group 20 is different from the exposure amount of the other light receiving elements 21 in the light receiving element group 20. According to such a configuration, in each compressed signal S2, the signal amount can be suppressed while including the information of each signal S1a, S1b, S1c, S1d output from the light receiving elements 21a, 21b, 21c, 21d with different exposure amounts.

[0086] Also, in the imaging device 100 of the present embodiment, in the light receiving element group 20, the difference in the exposure amount for each of the light receiving elements 21a, 21b, 21c, 21d is caused by the difference in the exposure time (charge accumulation time) of each of the light receiving elements 21a, 21b, 21c, 21d. By adopting a configuration in which the exposure time for each of the light receiving elements 21a, 21b, 21c, 21d is adjusted to cause a difference in the exposure amount of the light receiving elements 21a, 21b, 21c, 21d within the light receiving element group 20, it becomes possible to change the exposure amount of the light receiving elements 21a, 21b, 21c, 21d by control (control by the first circuit unit 3 in the example of the present embodiment).

[0087] Also, in the imaging device 100 of the present embodiment, each compression unit 10 generates a compressed signal S2 including the average value of the pixel values (signal values) included in the signals S1a, S1b, S1c, S1d output from each of the light receiving elements 21a, 21b, 21c, 21d of the light receiving element group 20 for each corresponding light receiving element group 20. A weighted average value may be used instead of the average value.

[0088] In the imaging device 100 of the present embodiment, each light-receiving element 21 of the light-receiving element group 20 has a photoelectric conversion element 211. When the light-receiving element array 2 is exposed, if there are light-receiving elements 21 in which the photoelectric conversion element 211 has overflowed and those that have not overflowed in the light-receiving element group 20, the reconstruction processing unit 102A calculates the predicted value of the signal value that can be obtained when the photoelectric conversion element 211 of each light-receiving element 21 has a capacity that does not overflow due to exposure, and reconstructs the compressed signal S2 into the pixel signal S3. As a result, an output image (image data) with a dynamic range larger than the dynamic range defined by each light-receiving element 21 can be obtained, that is, imaging with a high dynamic range becomes possible.

[0089] Also, in the imaging device 100 of the present embodiment, even if the light-receiving element array 2 has N light-receiving elements 21, since there are m light-receiving element groups 20, the number (or frequency) of signals S2 read from the light-receiving element array 2 is suppressed to N / m. Note that N > m.

[0090] As a result, compared with a configuration in which signals of each light-receiving element are read from the light-receiving element array, the amount of data read from the light-receiving element array 2 is suppressed. Consequently, the amount of data transferred from the solid-state imaging device 1 to the control unit 102 and the like is suppressed, and it becomes possible to achieve both high image quality and low power consumption and high frame rate in the imaging device 100.

[0091] Also, in the imaging device 100 of the present embodiment, since operations such as reading pixels multiple times like dual conversion gain (DCG) can be omitted, it becomes possible to achieve both high image quality and low power consumption and high frame rate.

[0092] In the image capturing device 100 of this embodiment as described above, the signals S1a, S1b, S1c, and S1d output from the plurality of light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs the compressed signal S2, which contains information about the signals S1a, S1b, S1c, and S1d from the plurality of light receiving elements 21a, 21b, 21c, and 21d, into a pixel signal S3 corresponding to one pixel in the captured image, thereby generating a captured image. That is, the image capturing device 100 of this embodiment generates an output image with a number of pixels corresponding to the number of light receiving element groups 20.

[0093] Next, a second embodiment of the present invention will be described with reference to FIGS. 7 to 10. The same components as those in the first embodiment are designated by the same reference numerals, and components different from those in the first embodiment will be described in detail.

[0094] 7, an imaging device 100 according to the second embodiment includes a light receiving element array 2A including a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and at least one compression unit 10A that generates, for each light receiving element group 20, a compressed signal S2 by compressing a signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20. The imaging device 100 of this embodiment includes one compression unit 10A, and this one compression unit 10A generates, for each light receiving element group 20, a compressed signal S2 by compressing the signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20.

[0095] Specifically, the imaging device 100 includes an imaging unit 101 and a control unit 102. Similar to the first embodiment, the imaging device 100 of this embodiment also includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

[0096] The imaging unit 101 includes an optical system composed of at least one optical element, and a solid-state imaging device 1A that converts an image of an object (imaging object) formed through the optical system into an electrical signal (i.e., captures an image).

[0097] The solid-state imaging device 1A has a light-receiving element array 2A in which a plurality of light-receiving elements 21 are arranged in a matrix, a first circuit unit 3A and a second circuit unit 4A connected to the light-receiving element array 2A, and a signal processing unit 5A connected to the second circuit unit 4A.

[0098] In the light-receiving element array 2A of the present embodiment, a plurality of row signal lines 25A are arranged for each row with respect to the light-receiving elements 21 arranged in a matrix, and a plurality of column signal lines 26A are arranged for each column with respect to the light-receiving elements 21 arranged in a matrix.

[0099] The first circuit unit 3A controls the light-receiving element array 2, and the second circuit unit 4A performs signal processing such as reading out the signal S1 from each light-receiving element 21, noise removal, and A / D conversion of the read-out signals S1.

[0100] The signal processing unit 5A is a part where various processes are performed on the compressed signal S2 output from the second circuit unit 4, and also performs signal processing (compression processing) such as compressing the signal S1 output from each light-receiving element 21 for each light-receiving element group 20. That is, in the imaging device 100 of the present embodiment, at least a part of the signal processing unit 5A constitutes a compression unit 10A.

[0101] When the first light receiving element 21a is set as a reference light receiving element and the second to fourth light receiving elements 21b to 21d are set as non-reference light receiving elements among the multiple light receiving elements (in this embodiment, the first light receiving element 21a, the second light receiving element 21b, the third light receiving element 21c, and the fourth light receiving element 21d) included in the light receiving element group 20 (see FIG. 10), the signal processing unit 5A generates a compressed signal S2 based on the difference between a reference signal value Sv1a, which is a pixel value (signal value) included in the signal S1a output from the reference light receiving element 21a, and non-reference signal values Sv1b, Sv1c, Sv1d, which are pixel values (signal values) included in the signals S1b, S1c, S1d output from the non-reference light receiving elements 21b, 21c, 21d.

[0102] In the solid-state imaging device 1A of this embodiment, when one signal processing unit 5A (compression unit 10A) processes a signal S1 from each light receiving element 21 of the light receiving element array 2, it processes the signal S1 for each light receiving element group 20, treating a plurality of adjacent light receiving elements (in the example of this embodiment, light receiving elements arranged in a 2 × 2 matrix) 21a, 21b, 21c, and 21d as a light receiving element group 20, to generate a compressed signal S2, and outputs the compressed signal S2 generated for each light receiving element group 20 to a downstream configuration (such as the control unit 102).

[0103] Specifically, as shown in FIG. 8, the signal processing unit 5A (compression unit 10A) determines that, among the non-reference signal values (pixel values) Sv1b, Sv1c, Sv1d included in the signals S1b, S1c, S1d output from the non-reference light-receiving elements 21b, 21c, 21d of the light-receiving element group 20, when there are only non-reference signal values (first signal values) Sv1b, Sv1c, Sv1d for which the difference from the reference signal value (pixel value) Sv1a included in the signal S1a output from the reference light-receiving element 21a is less than a predetermined threshold value (in the example of this embodiment, when the signal range of each light-receiving element 21 is 0 to 500, the difference in pixel values is 25 to 75), it generates a compression signal S2 including the average value of the pixel values Sv1a, Sv1b, Sv1c, Sv1d included in the signals S1a, S1b, S1c, S1d output from each of the light-receiving elements 21a, 21b, 21c, 21d of the light-receiving element group 20 (first compression process). The predetermined threshold value is set based on approximately 10% of the maximum value of the signal range of the light-receiving element 21. Also, the first compression process in the signal processing unit 5A of this embodiment is so-called binning. Further, the compression signal S2 of this embodiment has the average value of the pixel values Sv1a, Sv1b, Sv1c, Sv1d and the position information 3b111. Note that the position information of the compression signal S2 generated by the first compression process in the signal processing unit 5A (compression unit 10) of this embodiment is a fixed value (3b111).

[0104] 9, when there is a non-reference signal value (first signal value) Sv1c whose difference from the reference signal value Sv1a is less than a predetermined threshold, and non-reference signal values (second signal values) Sv1b, Sv1d whose difference from the reference signal value Sv1a is equal to or greater than a predetermined threshold, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values Sv1b, Sv1d from the signals output from the light receiving elements 21a, 21b, 21c, 21d. At the same time, the pixel value Sv1a included in the signal S1a output by the reference light receiving element 21a is set to the average value of the pixel value Sv1a and the first signal value Sv1c (100 in the example shown in FIG. 9), and position information (symbol 3b010 of the compressed signal S2 in the example shown in FIG. 9) of the non-reference light receiving element 21c that output the signal S1c including the first signal value Sv1c relative to the reference light receiving element 21a is added to generate a compressed signal S2 (second compression process).

[0105] Note that the position information of the second compression process (symbol 3b010 of the compressed signal S2 in FIG. 9) is actually 3-bit data, and is "010" in the example shown in FIG. 9. In this three-digit number, the first digit (the leftmost digit) indicates the position of the second light receiving element 21b relative to the reference light receiving element (first light receiving element) 21a, the second digit indicates the position of the third light receiving element 21c relative to the reference light receiving element 21a, and the third digit indicates the position of the fourth light receiving element 21d relative to the reference light receiving element 21a. The pixel values included in the signals S1b, S1c, and S1d from the light receiving elements 21b, 21c, and 21d corresponding to each position are 1 when the pixel values are the first signal value and 0 when the pixel values are the second signal value.

[0106] The control unit 102 has a reconstruction processing unit 102B, which reconstructs, from the compressed signal S2 output from the solid-state imaging element 1A, pixel signals S3a, S3b, S3c, and S3d of the pixels corresponding to each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 corresponding to the compressed signal S2.

[0107] Specifically, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 4, position information 3b111: see FIG. 8) generated by the first compression processing as follows, for example.

[0108] For the compressed signal S2 output from the upper right light-receiving element group 20 in the light-receiving element array 2A of FIG. 10, as shown in FIG. 8, the reconstruction processing unit 102B sets the pixel value of the pixel signal S3a of the pixel corresponding to the reference light-receiving element 21a to be the same as the pixel value 4 of the compressed signal S2, and sets the pixel values of the pixel signals S3b, S3c, and S3d of the pixels corresponding to the non-reference light-receiving elements 21b, 21c, and 21d to be the same as the pixel value 4 of the pixel signal S3a of the pixel corresponding to the reference light-receiving element 21a, thereby reconstructing the compressed signal S2 into the respective pixel signals S3a, S3b, S3c, and S3d.

[0109] Also, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 100, position information 3b010: see FIG. 9) generated by the second compression processing as follows, for example.

[0110] For the compressed signal S2 output from the upper left light-receiving element group 20 in the light-receiving element array 2 of FIG. 10, as shown in FIG. 9, the reconstruction processing unit 102B sets the pixel value of the pixel signal S3a of the pixel corresponding to the reference light-receiving element 21a to be the same as the pixel value 100 of the compressed signal S2, and for the pixels corresponding to the non-reference light-receiving elements 21b, 21c, and 21d, sets the pixel value of the pixel signal S3c of the pixel that is 1 in the position information (3b010) to be the same as the pixel value 100 of the pixel corresponding to the reference light-receiving element 21a, and sets the pixel values of the pixel signals S3b and S3d of the pixels that are 0 in the position information (3b010) to be the same as the pixel value 4 of the pixel adjacent in the row direction (in the example shown in FIG. 10, the pixel to the right), thereby reconstructing the compressed signal S2 into the respective pixel signals S3a, S3b, S3c, and S3d. Note that the reconstruction processing unit 102B may set the respective pixel values of the pixel signals S3b and S3d of the pixels that are 0 in the position information (3b010) to be the weighted average value or the average value of the pixel value 100 of the compressed signal S2 and the pixel value 4 of the compressed signal S2 output from the light-receiving element group 20 adjacent in the row direction.

[0111] In the above imaging device 100, as shown in FIG. 10, when a vertical line image (a region that is white compared to other regions in FIG. 10) is formed at the left end of the light receiving element array 2 during exposure, the signal processing unit 5A (compression unit 10A) performs first compression processing on the light receiving element groups 20 arranged vertically on the right side of the light receiving element array 2, and performs second compression processing on the light receiving element groups 20 arranged vertically on the left side of the light receiving element array 2, respectively. In this way, the compression signals S2 generated for each light receiving element group 20 are output from the solid-state imaging device 1A, respectively.

[0112] Subsequently, the reconstruction processing unit 102B of the control unit 102 reconstructs the compression signal S2 generated by the first compression processing and the compression signal S2 generated by the second compression processing by different methods (see FIGS. 8 and 9), thereby generating a high-resolution image with less blur as image data.

[0113] The imaging device 100 according to the present embodiment includes a light receiving element array 2A including a plurality of light receiving element groups 20 each having a plurality of light receiving elements 21, and a signal processing unit 5A (compression unit 10A) that generates a compression signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 of the light receiving element group 20 for each light receiving element group 20. In this way, by generating the compression signal S2 for each light receiving element group 20 by the signal processing unit 5A and outputting these compressed compression signals (i.e., signals in a state where the signal amount is suppressed) S2, the power consumption is suppressed in the configuration downstream of the signal processing unit 5A.

[0114] In addition, in the imaging device 100 of the present embodiment, the plurality of light-receiving elements 21 included in the light-receiving element group 20 include a reference light-receiving element 21a that is a reference light-receiving element 21, and non-reference light-receiving elements 21b, 21c, 21d other than the reference light-receiving element 21a. Then, the signal processing unit 5A (compression unit 10A) generates a compressed signal S2 based on the difference between a reference signal value Sv1a that is a pixel value (signal value) included in the signal S1a output from the reference light-receiving element 21a and non-reference signal values Sv1b, Sv1c, Sv1d that are pixel values (signal values) included in the signals S1b, S1c, S1d output from the non-reference light-receiving elements 21b, 21c, 21d.

[0115] In this way, by compressing the signals S1a, S1b, S1c, S1d output from the light-receiving elements 21a, 21b, 21c, 21d based on the difference between the reference signal value (pixel value) Sv1a and the non-reference signal values (pixel values) Sv1b, Sv1c, Sv1d for each position of the non-reference light-receiving elements 21b, 21c, 21d with respect to the reference light-receiving element 21a, the difference in the signal amount for each position can be reconstructed in the reconstruction processing unit 102B, and thereby, the resolution can be ensured in the output image. That is, a high-resolution image with little blur can be obtained.

[0116] In addition, in the imaging device 100 of the present embodiment, when there are only non-reference signal values Sv1b, Sv1c, Sv1d in the non-reference signal values Sv1b, Sv1c, Sv1d included in the signals S1b, S1c, S1d output from the non-reference light-receiving elements 21b, 21c, 21d of the light-receiving element group 20 and the difference from the reference signal value Sv1a is less than a predetermined threshold, the signal processing unit 5A (compression unit 10A) generates a compressed signal S2 including the average value of the pixel values (signal values) Sv1a, Sv1b, Sv1c, Sv1d included in the signals S1a, S1b, S1c, S1d output from the respective light-receiving elements 21a, 21b, 21c, 21d of the light-receiving element group 20. Note that a weighted average value may be used instead of the average value.

[0117] Furthermore, when there is a first signal value Sv1c, which is a non-reference signal value whose difference from the reference signal value Sv1a is less than a predetermined threshold, and second signal values Sv1b, Sv1d, which are non-reference signal values whose difference from the reference signal value Sv1a is equal to or greater than a predetermined threshold, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values Sv1b, Sv1d from the signals S1a, S1b, S1c, S1d output from each of the light receiving elements 21a, 21b, 21c, 21d, and generates a compressed signal S2 by adding position information (3b010 in this embodiment) of the non-reference light receiving element 21c that output the first signal value Sv1c relative to the reference light receiving element 21a to the signal S1a output from the reference light receiving element 21a.

[0118] In this way, by constructing the compressed signal S2 using the signal S1a (more specifically, a signal based on the signal S1a) output from the reference light receiving element 21a and the position information (3b010, etc.) added thereto, the signal amount can be suitably reduced compared to when all of the signals S1a, S1b, S1c, and S1d output from each light receiving element 21a, 21b, 21c, and 21d in the light receiving element group 20 are output to a downstream configuration.

[0119] Furthermore, in the imaging device 100 of this embodiment, even if the light receiving element array 2 has N light receiving elements 21, there are m light receiving element groups 20, so the number (or frequency) of signals S2 output from the solid-state imaging element 1A is limited to N / m, where N>m.

[0120] This reduces the signal amount (data amount) of the signal S2 output from the solid-state imaging device 1A compared to a configuration in which signals read out from each light receiving element of the light receiving element array are output from the solid-state imaging element individually. Also, by reconstructing N pixels from information on m pixels + α in a configuration downstream of the solid-state imaging device 1A (such as the reconstruction processing unit 102B), the imaging device 100 can achieve both high image quality, low power, and a high frame rate.

[0121] In the imaging device 100 of the present embodiment as described above, signals S1a, S1b, S1c, and S1d output from a plurality of light-receiving elements 21a, 21b, 21c, and 21d in the light-receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs this compressed signal S2 into pixel signals S3a, S3b, S3c, and S3d of pixels corresponding to the respective light-receiving elements 21a, 21b, 21c, and 21d in the captured image, thereby generating the captured image. That is, the imaging device 100 of the present embodiment generates an output image having the number of pixels corresponding to the number of light-receiving elements 21.

[0122] Note that the imaging device 100 and the solid-state imaging devices 1 and 1A of the present invention are not limited to the above first and second embodiments, and it goes without saying that various changes can be made without departing from the gist of the present invention. For example, the configuration of another embodiment can be added to the configuration of a certain embodiment, and a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment. Further, a part of the configuration of a certain embodiment can be deleted.

[0123] The light-receiving element group 20 in the first and second embodiments described above has four light-receiving elements 21a, 21b, 21c, and 21d arranged in a 2×2 configuration, but is not limited to this configuration. The light-receiving element group 20 only needs to have a plurality (two or more) of light-receiving elements 21. In the light-receiving element group 20, it is preferable that the plurality of light-receiving elements 21 are arranged such that the number of light-receiving elements 21 arranged in the row direction and the number of light-receiving elements 21 arranged in the column direction are the same (that is, in a square matrix), such as in a 2×2 configuration, a 3×3 configuration, or a 4×4 configuration.

[0124] Also, in the light-receiving element group 20 of the second embodiment, the reference light-receiving element is arranged at the position of the first row and the first column (the upper left position in the examples of FIGS. 8 and 9) in the matrix arrangement (2×2 arrangement) shown in FIGS. 8 and 9, but it may be arranged at other positions.

[0125] In addition, in the imaging apparatuses 100 of the first and second embodiments, although the solid-state imaging devices 1 and 1A have the compression unit 10 and the reconstruction processing units 102A and 102B are arranged at positions (control unit 102) different from those of the solid-state imaging devices 1 and 1A, the present invention is not limited to this configuration. The solid-state imaging devices 1 and 1A may have both the compression unit 10 and the reconstruction processing units 102A and 102B.

[0126] In addition, in each of the imaging apparatuses 100 of the first and second embodiments, although the pixel values (average values) included in the compression signal S2 are values derived by so-called simple averaging, the present invention is not limited to this configuration. The average value (binning) may be a value derived by weighted averaging.

[0127] In addition, in the second embodiment, although the position information in the second compression process for generating the compression signal S2 is a signal indicating the positions of the light-receiving elements 21b, 21c, and 21d that output the first signal value with respect to the reference light-receiving element 21a, the present invention is not limited to this configuration. The position information in the second compression process may be a signal indicating the positions of the light-receiving elements 21b, 21c, and 21d that output the second signal value with respect to the reference light-receiving element 21a, and a configuration may be adopted in which the positions of the light-receiving elements 21b, 21c, and 21d that output the first signal value with respect to the reference light-receiving element 21a are specified from this position information.

[0128] Also, in the second embodiment, when there are a first signal value and a second signal value in the non-reference signal values included in the signals output from the non-reference light-receiving elements 21b, 21c, and 21d of the light-receiving element group 20, the signal processing unit (compression unit) 5A deletes the signal corresponding to the second signal value among the signals output from the light-receiving elements 21a, 21b, 21c, and 21d, and adds the position information of the non-reference light-receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value with respect to the reference light-receiving element 21a to the signal output from the reference light-receiving element 21a to generate the compressed signal S2. However, the present invention is not limited to this configuration. For example, when there are a first signal value and a second signal value in the non-reference signal values included in the signals output from the non-reference light-receiving elements 21b, 21c, and 21d of the light-receiving element group 20, the signal processing unit (compression unit) 5A may generate a compressed signal S2 including the weighted average value or the average value of the reference signal value and the first signal value, and the position information of the non-reference light-receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value with respect to the reference light-receiving element 21a, and the like.

[0129] In order to describe the present invention, the present invention has been appropriately and sufficiently described through embodiments with reference to the drawings above. However, it should be recognized that those skilled in the art can easily make changes and / or improvements to the above-described embodiments. Therefore, as long as the modified or improved forms implemented by those skilled in the art do not depart from the scope of the rights of the claims described in the claims, the modified form or the improved form is interpreted as being included in the scope of the rights of the claims.

Explanation of Reference Numerals

[0130] 1, 1A... Solid-state imaging device, 2, 2A... Light-receiving element array, 20... Light-receiving element group, 21... Light-receiving element, 21a... First light-receiving element, reference light-receiving element, 21b... Second light-receiving element, non-reference light-receiving element, 21c... Third light-receiving element, non-reference light-receiving element, 21d... Fourth light-receiving element, non-reference light-receiving element, 211... Photoelectric conversion element, 22... Amplifier, 25, 25A... Row signal lines, 26, 26A... Column signal lines, 3, 3A... First circuit section, 4, 4A... Second circuit section, 5... Signal processing section, 5A... Signal processing section (compression section), 10, 10A... Compression section, 100... Imaging device, 101... Imaging section, 102... Control section, 102A, 102B... Reconstruction processing section, 103... Non-volatile memory, 104... Working memory, 105... Operation section, 106... Display section, 107... Recording medium, 108... Connection section, 109... Short-range wireless communication section, 110... Public network connection section, 111... Microphone, 112... Speaker, 500... Solid-state imaging device, 501... Light-receiving element array, 502... Light-receiving element, 503... Pixel drive wiring, 504... Vertical signal line, 505... Column-parallel signal processing circuit, 506... Output circuit, 507... Timing control circuit, 508... Horizontal scanning circuit, 509... Vertical scanning circuit, RES... Reset signal, S1, S1a, S1b, S1c, S1d... Signals, S2... Compression signal, S3, S3a, S3b, S3c, S3d... Pixel signals, SEL... Select signal, Sv1a... Reference signal value, Sv1b, Sv1d... Non-reference signal values, second signal values, Sv1c... Non-reference signal value, first signal value, t1~t9... Times, T1... First accumulation time, T2... Second accumulation time, T3... Third accumulation time, T4... Fourth accumulation time, TX1... First charge transfer signal, TX2... Second charge transfer signal, TX3... Third charge transfer signal, TX4... Fourth charge transfer signal

Claims

1. A light-receiving element array including a plurality of light-receiving element groups each having a plurality of light-receiving elements; A compression unit that generates, for each light-receiving element group, a compressed signal obtained by compressing signals output from the plurality of light-receiving elements in the light-receiving element group; A reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into a pixel signal of a pixel corresponding to the compressed signal in an output image, the imaging device comprising: In the light-receiving element group, when the light-receiving element array is exposed, an exposure amount of at least one light-receiving element in the light-receiving element group is different from an exposure amount of another light-receiving element in the light-receiving element group; The compressed signal includes a weighted average value or an average value of signal values included in signals output from the respective light-receiving elements in the light-receiving element group; Each light-receiving element in the light-receiving element group has a photoelectric conversion element; When there are a light-receiving element in which the photoelectric conversion element has overflowed and a light-receiving element in which the photoelectric conversion element has not overflowed in the light-receiving element group when the light-receiving element array is exposed, the reconstruction processing unit calculates an expected value of a signal value that can be obtained when the photoelectric conversion element of each light-receiving element has a capacity that does not overflow due to the exposure, and reconstructs the compressed signal into the pixel signal, the imaging device.

2. The imaging device according to claim 1, comprising a solid-state imaging device having the light-receiving element array and the compression unit.

3. The imaging device according to claim 1 or 2, wherein, in the light-receiving element group, the difference in the exposure amount is caused by a difference in exposure time of each light-receiving element.

4. The imaging device according to claim 1 or 2, wherein the reconstruction processing unit reconstructs the compressed signal into the pixel signal based on information before compression included in the compressed signal.

5. The plurality of light-receiving elements included in the light-receiving element group include a reference light-receiving element that is a reference light-receiving element and a non-reference light-receiving element other than the reference light-receiving element; The compression unit generates the compressed signal based on a difference between a reference signal value that is a signal value included in a signal output from the reference light-receiving element and a non-reference signal value that is a signal value included in a signal output from the non-reference light-receiving element; The compression unit is in the non-reference signal value included in the signal output from the non-reference light-receiving element in the light-receiving element group, When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, a first compression process for generating the compressed signal including the weighted average value or the average value of the signal values output from each light receiving element of the light receiving element group is performed. When there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold and a second signal value that is a non-reference signal value whose difference from the reference signal value is greater than or equal to the predetermined threshold, a signal corresponding to the second signal value among the signals output from each light receiving element is deleted, and the position information of the non-reference light receiving element that output the first signal value or the second signal value with respect to the reference light receiving element is added to the signal output from the reference light receiving element, thereby performing a second compression process for generating the compressed signal. The imaging device according to claim 1.

6. The plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a reference light receiving element and non-reference light receiving elements other than the reference light receiving element. The compression unit generates the compressed signal based on the difference between a reference signal value that is a signal value included in the signal output from the reference light receiving element and a non-reference signal value that is a signal value included in the signal output from the non-reference light receiving element. The compression unit is configured such that, among the non-reference signal values included in the signals output from the non-reference light receiving elements of the light receiving element group, When there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold, a first compression process for generating the compressed signal including the weighted average value or the average value of the signal values output from each light receiving element of the light receiving element group is performed. When there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold and a second signal value that is a non-reference signal value whose difference from the reference signal value is greater than or equal to the predetermined threshold, a second compression process for generating the compressed signal including the weighted average value or the average value of the reference signal value and the first signal value and the position information of the non-reference light receiving element that output the first signal value or the second signal value with respect to the reference light receiving element is performed. The imaging device according to claim 1.

7. The predetermined threshold is set with reference to 10% of the maximum value of the signal range of the light receiving elements included in the light receiving element group. The imaging device according to claim 5 or 6.

8. The compressed signal generated by the first compression process further has position information that is a fixed value. The imaging device according to claim 5 or 6.

9. The reconstruction processing unit, for the compressed signal generated by the first compression processing, sets the pixel value of the pixel signal of the pixel corresponding to the reference light receiving element to be the same as the pixel value of the compressed signal, and sets the pixel values of the pixel signals of the pixels corresponding to the non-reference light receiving elements to be the same as the pixel value of the pixel signal of the pixel corresponding to the reference light receiving element, thereby reconstructing the compressed signal into the pixel signal. The imaging device according to claim 5 or 6.

10. The reconstruction processing unit, for the compressed signal generated by the second compression processing, sets the pixel value of the pixel signal of the pixel corresponding to the reference light receiving element to be the same as the pixel value of the compressed signal, and for each pixel corresponding to the non-reference light receiving element, sets the pixel value of the pixel signal of the pixel that is 1 in the position information to be the same as the pixel value of the pixel corresponding to the reference light receiving element, and sets the pixel value of the pixel signal of the pixel that is 0 in the position information to be the same as the pixel value of the pixel adjacent to the pixel in the row direction, thereby reconstructing the compressed signal into the pixel signal. The imaging device according to claim 5 or 6.

11. The reconstruction processing unit, for the compressed signal generated by the second compression processing, sets the pixel value of the pixel signal of the pixel corresponding to the reference light receiving element to be the same as the pixel value of the compressed signal, and for each pixel corresponding to the non-reference light receiving element, sets the pixel value of the pixel signal of the pixel that is 1 in the position information to be the same as the pixel value of the pixel corresponding to the reference light receiving element, and sets each pixel value of the pixel signal of the pixel that is 0 in the position information to be the weighted average value or the average value of the pixel value of the compressed signal and the pixel value of the compressed signal output from the light receiving element group adjacent to the pixel in the row direction, thereby reconstructing the compressed signal into the pixel signal. The imaging device according to claim 5 or 6.

12. A light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements, a compression unit that generates a compressed signal obtained by compressing signals output from the plurality of light receiving elements of each light receiving element group for each light receiving element group, a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into a pixel signal of a pixel corresponding to the compressed signal in an output image, and in each light receiving element group, when the light receiving element array is exposed, the exposure amount of at least one light receiving element in the light receiving element group is different from the exposure amount of other light receiving elements in the light receiving element group. the compressed signal includes a weighted average or an average of signal values included in the signals output from the light receiving elements of the light receiving element group, Each light receiving element of the light receiving element group has a photoelectric conversion element, The reconstruction processing unit, when there are photoreceptors in the photoreceptor group whose photoelectric conversion elements overflow and those whose photoelectric conversion elements do not overflow when the photoreceptor array is exposed to light, reconstructs the compressed signal into the pixel signal by calculating an expected value of the signal value that can be obtained if the photoelectric conversion element of each photoreceptor has a capacity that will not overflow due to the exposure.

Images