Imaging device and solid-state imaging element

Abstract

The present invention is characterized by comprising a photoreceptor array having a plurality of photoreceptor groups, each having a plurality of photoreceptors, and a compression unit that generates a compressed signal for each photoreceptor group by compressing signals output from the plurality of photoreceptors in the photoreceptor group.

Description

[Technical field]

[0001] The present invention relates to an imaging device including a light receiving element array and a solid-state imaging device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a solid-state imaging element such as a CMOS used in an imaging device such as a digital camera is known, for example, from Japanese Unexamined Patent Publication No. 2022-164387.

[0003] As shown in FIG. 11, this solid-state imaging element 500 has a light-receiving element array 501 in which light-receiving elements 502 including photoelectric conversion elements are arranged two-dimensionally in an array and connected to pixel drive wiring 503 and vertical signal lines 504, and also has peripheral circuits such as a column-parallel signal processing circuit 505, an output circuit 506, a timing control circuit 507, a horizontal scanning circuit 508, a vertical scanning circuit 509, etc.

[0004] In this solid-state imaging element 500, when a photodetector array 501 is exposed to light during imaging using an imaging device (digital camera, etc.) equipped with the solid-state imaging element 500, signals output from each of the multiple photodetectors 502 that make up the photodetector array 501 are output to the outside via a vertical signal line 504, a column-parallel signal processing circuit 505, an output circuit 506, etc.

[0005] In recent years, there has been a demand for power saving in the solid-state imaging element 500 and imaging devices equipped with the solid-state imaging element 500. However, in the solid-state imaging element 500 and imaging devices, all of the signals output from each of the multiple light receiving elements 502 included in the light receiving element array 501 are output to the outside (a so-called full readout configuration), and therefore the amount of signals output within the solid-state imaging element 500 and to the outside is large, and the solid-state imaging element 500 requires power consumption according to the amount of signals. As a result, power saving has not been sufficient in the solid-state imaging element 500 and imaging devices equipped with this solid-state imaging element 500. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-164387 Summary of the Invention [Problem to be solved by the invention]

[0007] SUMMARY OF THE PRESENT EMBODIMENTS In view of the above, an object of the present invention is to provide an imaging apparatus and a solid-state imaging element capable of suppressing power consumption. [Means for solving the problem]

[0008] The imaging device according to the present invention comprises: a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression section configured to generate compressed signals for each light receiving element group by compressing signals output from a plurality of light receiving elements of the light receiving element group.

[0009] Moreover, the imaging device The image sensor may further include a solid-state image sensor having the light receiving element array and the compression section.

[0010] In addition, in the imaging device, In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group may be different from the amount of exposure of other light receiving elements of the light receiving element group.

[0011] In addition, in the imaging device, In the light receiving element group, the difference in the amount of exposure may be caused by a difference in exposure time of each light receiving element.

[0012] In addition, in the imaging device, The compressed signal may include a weighted average or an average value of signal values ​​included in the signals output from each light receiving element of the light receiving element group.

[0013] Moreover, the imaging device The image forming apparatus may further include a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into a pixel signal of a pixel in an output image that corresponds to the compressed signal.

[0014] In addition, in the imaging device, The reconstruction processing unit may reconstruct the compressed signal into the pixel signal based on pre-compression information included in the compressed signal.

[0015] In addition, in the imaging device, Each light receiving element of the light receiving element group has a photoelectric conversion element, The reconstruction processing unit may reconstruct the compressed signal into the pixel signal by calculating a predicted value of the signal value that can be obtained when the photoelectric conversion element of each light receiving element has a capacity that will not overflow due to the exposure when there are light receiving elements in the light receiving element group whose photoelectric conversion elements overflow and light receiving elements whose photoelectric conversion elements do not overflow when the light receiving element array is exposed to light.

[0016] In addition, in the imaging device, the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element, The compression unit may generate the compressed signal based on the difference between a reference signal value, which is a signal value contained in the signal output from the reference light receiving element, and a non-reference signal value, which is a signal value contained in the signal output from the non-reference light receiving element.

[0017] In addition, in the imaging device, The compression unit is configured to: generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated by deleting the signal output from each of the light receiving elements which corresponds to the second signal value, and adding position information of the non-reference light receiving element which output the first signal value or the second signal value relative to the reference light receiving element to the signal output from the reference light receiving element.

[0018] In addition, in the imaging device, The compression unit is configured to: generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that outputs the first signal value or the second signal value relative to the reference light receiving element.

[0019] Further, the solid-state imaging device according to the present invention comprises: a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression section configured to generate compressed signals for each light receiving element group by compressing signals output from a plurality of light receiving elements of the light receiving element group. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a functional block diagram of the imaging device according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of a solid-state imaging element included in the imaging device. [Diagram 3] FIG. 3 is a diagram showing a configuration of a light receiving element group included in the solid-state imaging device. [Figure 4] FIG. 4 is a timing chart of the light receiving element group. [Diagram 5] FIG. 5 is a diagram for explaining the processing in the compression section and the processing in the reconstruction processing section when there is no light receiving element in which the photoelectric conversion element in the light receiving element group has overflowed. [Figure 6] FIG. 6 is a diagram for explaining the processing in the compression section and the processing in the reconstruction processing section when a light receiving element group includes a light receiving element in which a photoelectric conversion element has overflowed. [Figure 7] FIG. 7 is a diagram showing the configuration of a solid-state imaging device according to the second embodiment. [Figure 8] FIG. 8 is a diagram for explaining the processing in the compression section and the processing in the reconstruction processing section when there are only non-reference light receiving elements in a light receiving element group whose pixel values ​​differ from the pixel value of a reference light receiving element by less than a predetermined threshold value. [Figure 9] FIG. 9 is a diagram for explaining the processing in the compression section and the processing in the reconstruction processing section when there is a non-reference light receiving element in a light receiving element group whose pixel value differs from the pixel value of a reference light receiving element by a predetermined threshold or more. [Figure 10] FIG. 10 is a diagram showing the processing in the compression section and the processing in the reconstruction processing section when an image is formed on a light receiving element array having a plurality of light receiving element groups. [Figure 11] FIG. 11 is a diagram showing the configuration of a conventional solid-state imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The imaging device according to this embodiment includes: a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression section configured to generate compressed signals for each light receiving element group by compressing signals output from a plurality of light receiving elements of the light receiving element group.

[0022] In this way, the compression unit generates a compressed signal for each light receiving element group and outputs each of these compressed signals (i.e., signals with reduced signal amounts), thereby making it possible to reduce power consumption in the configuration downstream of the compression unit. Note that the compression of signals in the compression unit according to the present invention refers to a process that results in a signal with a smaller signal amount than a signal generated by simple addition (pixel addition, etc.) of the signals output from each light receiving element.

[0023] The imaging device includes: The image sensor may further include a solid-state image sensor having the light receiving element array and the compression section.

[0024] In addition, in the imaging device, In the light receiving element group, when the light receiving element array is exposed, the amount of exposure of at least one light receiving element of the light receiving element group may be different from the amount of exposure of other light receiving elements of the light receiving element group.

[0025] According to this configuration, it is possible to reduce the signal amount while including information on each signal output from light receiving elements with different exposure amounts.

[0026] in this case, In the light receiving element group, the difference in the amount of exposure may be caused by a difference in exposure time of each light receiving element.

[0027] In this manner, by adjusting the exposure time of the light receiving elements and varying the amount of exposure of the light receiving elements within a light receiving element group, it becomes possible to change the amount of exposure of the light receiving elements by control.

[0028] In addition, in the imaging device, The compressed signal may include a weighted average or an average value of signal values ​​included in the signals output from each light receiving element of the light receiving element group.

[0029] Moreover, the imaging device The image forming apparatus may further include a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into a pixel signal of a pixel in an output image that corresponds to the compressed signal.

[0030] According to this configuration, the pixel signal is reconstructed downstream of the compression unit, and this pixel signal is output, thereby improving the image quality of the resulting image.

[0031] In addition, in the imaging device, The reconstruction processing unit may reconstruct the compressed signal into the pixel signal based on pre-compression information included in the compressed signal.

[0032] In addition, in the imaging device, Each light receiving element of the light receiving element group has a photoelectric conversion element, The reconstruction processing unit may reconstruct the compressed signal into the pixel signal by calculating a predicted value of the signal value that can be obtained when the photoelectric conversion element of each light receiving element has a capacity that will not overflow due to the exposure when there are light receiving elements in the light receiving element group whose photoelectric conversion elements overflow and light receiving elements whose photoelectric conversion elements do not overflow when the light receiving element array is exposed to light.

[0033] According to this configuration, it is possible to obtain an image with a larger dynamic range than the dynamic range defined by each light receiving element, that is, imaging with a high dynamic range is possible.

[0034] In addition, in the imaging device, the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element, The compression unit may generate the compressed signal based on the difference between a reference signal value, which is a signal value contained in the signal output from the reference light receiving element, and a non-reference signal value, which is a signal value contained in the signal output from the non-reference light receiving element.

[0035] In this way, by compressing the signal based on the difference between the reference signal value and the non-reference signal value for each position of the light receiving element, the difference in signal amount for each position can be reconstructed in the reconstruction processing unit, thereby ensuring resolution in the output image.

[0036] in particular, The compression unit is configured to: generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated by deleting the signal output from each of the light receiving elements which corresponds to the second signal value, and adding position information of the non-reference light receiving element which output the first signal value or the second signal value relative to the reference light receiving element to the signal output from the reference light receiving element.

[0037] In this way, by constructing a compressed signal using the signal output from the reference light receiving element and the positional information added to it, the signal volume can be suitably reduced compared to when all signals output from each light receiving element in the light receiving element group are output to the downstream configuration.

[0038] Further, the compression unit is configured to, in the non-reference signal value included in the signal output from the non-reference light receiving element of the light receiving element group, generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; When there is a first signal value which is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value which is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal may be generated including a weighted average or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving element that outputs the first signal value or the second signal value relative to the reference light receiving element.

[0039] Moreover, the solid-state imaging device according to this embodiment has a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; The image forming apparatus further includes a compression section configured to generate compressed signals for each light receiving element group by compressing signals output from a plurality of light receiving elements of the light receiving element group.

[0040] In this way, the compression unit generates a compressed signal for each light receiving element group and outputs each of these compressed signals (i.e., signals with a reduced signal volume), thereby reducing power consumption in the configuration downstream of the compression unit.

[0041] Hereinafter, a first embodiment of the present invention will be described with reference to the accompanying drawings.

[0042] 1 and 2, an imaging device 100 according to the first embodiment includes a light receiving element array 2 including a plurality of light receiving element groups 20 each including a plurality of light receiving elements 21, and at least one compression unit 10 that generates a compressed signal S2 for each light receiving element group 20 by compressing a signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20. The imaging device 100 of this embodiment includes compression units 10 in a number corresponding to the number of light receiving element groups 20, and each compression unit 10 generates a compressed signal S2 for each light receiving element group 20 by compressing a signal S1 output from the plurality of light receiving elements 21 included in the corresponding light receiving element group 20. This imaging device 100 is capable of capturing an image of an object such as a digital camera, a smartphone, or a tablet device, and the imaging device 100 of this embodiment is, for example, a smartphone.

[0043] 1, the imaging device 100 includes an imaging unit 101 and a control unit 102. The imaging device 100 of this embodiment also includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

[0044] Under the control of the control unit 102, the imaging unit 101 converts the image of the object formed by the optical system of the imaging unit 101 into an electric signal, performs noise reduction processing, etc., and outputs the digital data as image data of an output image. Specifically, the imaging unit 101 has an optical system constituted by at least one optical element such as a lens, and a solid-state imaging element 1 that converts (i.e., captures) the image of the object (image-captured object) formed through the optical system into an electric signal.

[0045] The solid-state imaging device 1 is, for example, incorporated in a smartphone, a digital camera, or the like, and is an element for capturing an image of an object. The solid-state imaging device 1 of this embodiment is, for example, a CMOS image sensor.

[0046] 2, the solid-state imaging device 1 has a light receiving element array 2, a first circuit section 3 and a second circuit section 4 connected to the light receiving element array 2, and a signal processing section 5 to which the second circuit section 4 is connected. The light receiving element array 2, the first circuit section 3, the second circuit section 4, and the signal processing section 5 are arranged on the same semiconductor substrate or on multiple semiconductor substrates that are electrically connected to each other.

[0047] The light receiving element array 2 has a plurality of light receiving element groups 20 arranged two-dimensionally in a matrix. The light receiving element array 2 also has a plurality of row signal lines 25 arranged in each row and each extending in the row direction (left-right direction in FIG. 2) for the plurality of light receiving element groups 20 arranged in a matrix, and a plurality of column signal lines 26 arranged in each column and each extending in the column direction (up-down direction in FIG. 2). Each of the plurality of row signal lines 25 is connected to the first circuit unit 3, and each of the plurality of column signal lines 26 is connected to the second circuit unit 4.

[0048] As shown in FIG. 3, each of the light receiving element groups 20 has a plurality of light receiving elements (pixels) 21. In the light receiving element group 20 of this embodiment, the light receiving elements 21 are adjacent to each other. Specifically, the light receiving elements 21 included in the light receiving element group 20 are arranged adjacent to each other in the row and column direction. Each light receiving element group 20 of this embodiment has four light receiving elements 21 (a first light receiving element 21a, a second light receiving element 21b, a third light receiving element 21c, and a fourth light receiving element 21d) arranged in two rows and two columns (i.e., a 2×2 array). Each light receiving element group 20 also has an amplifier 22 to which the light receiving elements 21a, 21b, 21c, and 21d are connected.

[0049] In FIG. 2, in order to make it easier to understand the configuration of the light receiving element array 2, the multiple light receiving element groups 20 are shown with a gap between adjacent light receiving element groups 20 in the row direction and between adjacent light receiving element groups 20 in the column direction, but in reality, in the light receiving element array 2, the multiple light receiving element groups 20 are arranged in a matrix with adjacent light receiving element groups 20 in the row direction adjacent to each other and adjacent light receiving element groups 20 in the column direction adjacent to each other.

[0050] In addition, in Figure 3, in order to make it easier to understand the configuration of the light receiving element group 20, each light receiving element 21a, 21b, 21c, and 21d is shown with a gap between adjacent light receiving elements 21 in the row direction and between adjacent light receiving elements 21 in the column direction, but in reality, in the light receiving element group 20, the multiple light receiving elements 21 are arranged in a matrix with adjacent light receiving elements 21 in the row direction adjacent to each other and adjacent light receiving elements 21 in the column direction adjacent to each other.

[0051] Each of the multiple light receiving elements 21 photoelectrically converts input light (incident light) to accumulate an amount of charge corresponding to the amount of input light, and outputs the accumulated charge as an electrical signal. In this embodiment, the range (signal range) of the signal (pixel value) S1 that can be output from each light receiving element 21 is 0 to 500. Each light receiving element 21 has a photoelectric conversion element (photodiode in this embodiment) 211.

[0052] Each of the row signal lines 25 transmits the reset signal RES, the first to fourth charge transfer signals TX1 to TX4, and the select signal SEL output from the first circuit section 3 to each of the light receiving element groups 20 connected thereto.

[0053] Here, the reset signal RES is a signal for resetting the charges of each of the light receiving elements 21a, 21b, 22c, and 21d, the first charge transfer signal TX1 is a signal for turning the first light receiving element 21a ON / OFF, the second charge transfer signal TX2 is a signal for turning the second light receiving element 21b ON / OFF, the third charge transfer signal TX3 is a signal for turning the third light receiving element 21c ON / OFF, and the fourth charge transfer signal TX4 is a signal for turning the fourth light receiving element 21d ON / OFF.

[0054] Each of the plurality of column signal lines 26 is connected to each of the light receiving element groups 20 in the corresponding column, and transmits the compressed signal S2 output from each of the light receiving element groups 20 to the second circuit section 4.

[0055] The first circuit unit 3 outputs a reset signal RES, first to fourth charge transfer signals TX1 to TX4, a select signal SEL, and the like to each of the light receiving element groups 20, thereby controlling the light receiving element array 2 and controlling the compression of the signals S1a, S1b, S1c, and S1d output from each of the light receiving elements 21 for each of the light receiving element groups 20. Note that, hereinafter, the signals S1a, S1b, S1c, and S1d output from each of the light receiving elements 21a, 21b, 22c, and 21d of the light receiving element group 20 may also be simply referred to as signal S1.

[0056] The second circuit section 4 performs signal processing such as reading out the compressed signals S2 from each light receiving element group 20, removing noise, and A / D (Analog / Digital) conversion of each read out compressed signal S2.

[0057] The signal processing unit 5 is a section that performs various processes on each compressed signal S2 output from the second circuit unit 4, and the signal processing unit 5 in this embodiment is an ISP (Image Signal Processor) within the sensor.

[0058] In the solid-state imaging element 1 configured as described above, signals such as those shown in FIG. 4 are transmitted from the first circuit section 3 to each of the light-receiving element groups 20, and the signals S1a, S1b, S1c, and S1d output from each of the light-receiving elements 21a, 21b, 21c, and 21d in the light-receiving element group 20 are compressed to generate a compressed signal S2.

[0059] Specifically, at time t1, the reset signal turns ON and the select signal turns OFF in each light receiving element group 20. At this time, the first to fourth charge transfer signals TX1 to TX4 turn ON, and the charges in each of the light receiving elements 21a, 21b, 21c, and 21d are reset.

[0060] Next, at time t2, the first charge transfer signal is turned OFF, and after a predetermined interval at time t3, the second charge transfer signal is turned OFF, at time t4, the third charge transfer signal is turned OFF, and at time t5, the third charge transfer signal is turned OFF.

[0061] Due to this difference in the time (exposure time) when the light receiving element array 2 is exposed during shooting or the like in the imaging device 100, the amount of exposure of at least one light receiving element 21 in the light receiving element group 20 becomes different from the amount of exposure of the other light receiving elements 21 in the light receiving element group 20. In the imaging device 100 of this embodiment, the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 each have a different amount of exposure.

[0062] Subsequently, at time t6, the reset signal turns ON and the select signal turns OFF, and then at time t9, the reset signal turns OFF and the select signal turns ON.

[0063] Then, at times t7 and t8 between times t6 and t9, the first to fourth charge transfer signals TX1 to TX4 are simultaneously turned ON / OFF. As a result, while each of the light receiving elements 21a, 21b, 21c, and 21d is OFF (more specifically, while the first light receiving element 21a is in the first accumulation time T1 (time t2 to time t7), the second light receiving element 21b is in the second accumulation time T2 (time t3 to time t7), the third light receiving element 21c is in the third accumulation time T3 (time t4 to time t7), and the fourth light receiving element 21d is in the fourth accumulation time T4 (time t5 to time t7)), the charges accumulated therein are simultaneously output as electrical signals to the column signal lines 26 connected to the light receiving element groups 20 including each of the light receiving elements 21a, 21b, 21c, and 21d, and as a result, the signals S1a, S1b, S1c, and S1d output from each of the light receiving elements 21a, 21b, 21c, and 21d are compressed to become the compressed signal S2. The relationship between the accumulation times T1 to T4 in the imaging device 100 of this embodiment is T2=(3 / 4)T1, T3=(2 / 4)T1, and T4=(1 / 4)T1.

[0064] The compressed signal S2 generated for each light receiving element group 20 includes an average value of pixel values ​​(signal values) included in the signal S1 output from each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20.

[0065] For example, when focusing on one light receiving element group 20A, if the exposure amounts of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A are within the signal range, as shown in FIG. 5, when the pixel value included in the signal S1a output from the first light receiving element 21a is 400, the pixel value included in the signal S1b output from the second light receiving element 21b is 300, the pixel value included in the signal S1c output from the third light receiving element 21c is 200, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 100, then the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is (400+300+200+100) / 4=250 In this embodiment, the "average value of the signal" refers to an unweighted sum of the signal.

[0066] In addition, when the exposure amount of some of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20A exceeds the signal range, for example, as shown in FIG. 6, when the pixel value included in the signal S1a output from the first light receiving element 21a is 500, the pixel value included in the signal S1b output from the second light receiving element 21b is 500, the pixel value included in the signal S1c output from the third light receiving element 21c is 500, and the pixel value included in the signal S1d output from the fourth light receiving element 21d is 300, the pixel value (average value) included in the compressed signal S2 output from the light receiving element group 20A is (500+500+500+300) / 4=450 It becomes.

[0067] As described above, in the solid-state imaging device 1 of this embodiment, the above-mentioned compression section 10 that generates, for each light-receiving element group 20, compressed signals S2 by compressing signals S1a, S1b, S1c, and S1d output from the multiple light-receiving elements 21a, 21b, 21c, and 21d included in the light-receiving element group 20, is composed of the light-receiving element group 20 and a first circuit section 3 that controls the light-receiving element group 20. That is, the solid-state imaging device 1 of this embodiment has multiple compression sections 10 (the number of which corresponds to the number of light-receiving element groups 20).

[0068] The control unit 102 controls each unit of the imaging device 100 in accordance with input signals and programs. The control unit 102 has a reconstruction processing unit 102A that reconstructs the compressed signal S2 output from the solid-state imaging element 1 into a pixel signal S3. Note that the configuration is not limited to one in which the control unit 102 controls the entire imaging device 100. A configuration in which the entire imaging device 100 is controlled by a plurality of hardware devices sharing the processing may also be used.

[0069] The reconstruction processing unit 102A reconstructs the compressed signal S2 (compressed signal corresponding to each light receiving element group 20) output from the solid-state imaging element 1 into a pixel signal S3 of the pixel corresponding to the compressed signal S2 in the output image based on the pre-compression information contained in the compressed signal S2.

[0070] In the control unit 102 having this reconstruction processing unit 102A, each compressed signal S2 output from the solid-state imaging device 1 is reconstructed to generate a pixel signal S3, thereby generating an output image (image data). The control unit 102 outputs (displays, etc.) this output image to the outside via the display unit 106.

[0071] Specifically, when the photoelectric conversion elements 211 of the photoreceiving element group 20 do not overflow when the photoreceiving element array 2 is exposed, the reconstruction processing unit 102A reconstructs the compressed signal S2 into a pixel signal S3 based on the pixel values ​​included in the compressed signal S2 and the accumulation times T1, T2, T3, and T4 of the photoreceiving elements 21a, 21b, 21c, and 21d. In addition, when the photoreceiving element array 2 is exposed and there are light receiving elements 21 whose photoelectric conversion elements 211 have overflowed and light receiving elements 21 whose photoelectric conversion elements 211 have not overflowed in the light receiving element group 20, the reconstruction processing unit 102A reconstructs the compressed signal S2 into a pixel signal S3 so as to include predicted pixel values ​​(signal values) that can be obtained when the photoelectric conversion elements 211 of each light receiving element 21 have a capacity that does not overflow due to exposure.

[0072] More specifically, for example, as shown in FIG. 5, when the pixel value included in the compressed signal S2 output from the light receiving element group 20A is 250, the reconstruction processing unit 102A performs the following calculation, assuming that the pixel value included in the pixel signal S3 is x, based on T4=(1 / 4)T1, T3=(2 / 4)T1, and T2=(3 / 4)T1. {x+(3 / 4)x+(2 / 4)x+(1 / 4)x} / 4=250 x=400 Then, the reconstruction processing unit 102A reconstructs the pixel signal S3 from the compressed signal S2 so that the pixel value x included in the pixel signal S3 becomes 400.

[0073] Furthermore, for example, as shown in FIG. 6, when the pixel value included in the compressed signal S2 output from the light receiving element group 20A is 450 (i.e., when there are light receiving elements 21 in the light receiving element group 20 whose photoelectric conversion elements 211 have overflowed and light receiving elements 21 whose photoelectric conversion elements 211 have not overflowed), the reconstruction processing unit 102A performs the following calculation, assuming that the pixel value included in the signal S1 obtained from the light receiving element 21d at the fourth accumulation time T4 is x1. (500+500+500+x1) / 4=450 x1=300 Then, because the pixel value x1 included in the signal S1d output from the light receiving element 21d at the fourth accumulation time T4 is 300, the reconstruction processing unit 102A converts the signal S1d (pixel value 300) into a value equivalent to the signal S1a output from the light receiving element 21a at the first accumulation time T1. Specifically, because the first accumulation time T1 is four times the fourth accumulation time T4, 300×4=1200 is obtained. As a result, the reconstruction processing unit 102A reconstructs the pixel signal S3 so that the pixel value (predicted value) x2 included in the pixel signal S3 becomes 1200.

[0074] In this way, according to the reconstruction processing unit 102A of the imaging device 100 of this embodiment, even if light of an amount such that each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 has a value (pixel value) larger than the signal range is incident (i.e., even if there is a light receiving element 21 in the light receiving element group 20 whose photoelectric conversion element 211 overflows), an output image is generated without whiteout. That is, the imaging device 100 of this embodiment is capable of High Dynamic Range (HDR) imaging with a signal range of 0 to 2000.

[0075] The non-volatile memory 103 is an electrically erasable and recordable non-volatile memory. In this embodiment, the non-volatile memory 103 records an OS (operating system), which is basic software executed by the control unit 102, and applications that cooperate with the OS to realize applied functions. The working memory 104 is used as an image display memory for the display unit 106, a working area for the control unit 102, etc.

[0076] The operation unit 105 is used by a user or the like to input instructions to the imaging device 100. The operation unit 105 of this embodiment has a power button for instructing ON / OFF of the power supply of the imaging device 100, a touch panel formed on the display unit 106, and the like.

[0077] The display unit 106 displays an output image (image data) and displays characters for operation.

[0078] The recording medium 107 records the image data output from the imaging unit 101 .

[0079] The connection unit 108 is an interface for connecting to an external device, and the imaging device 100 exchanges data with the external device via the connection unit 108.

[0080] The short-distance wireless communication unit 109 is a communication unit for performing short-distance wireless communication. The short-distance wireless communication unit 109 is configured with an antenna for wireless communication, a modulation / demodulation circuit for processing wireless signals, and a communication controller.

[0081] The public network connection unit 110 is an interface for performing public wireless communication. The imaging device 100 communicates with other devices via this public network connection unit 110 to make a call. At this time, the control unit 102 realizes the call by inputting and outputting an audio signal via a microphone 111 and a speaker 112. The public network connection unit 110 in this embodiment is an antenna, and the control unit 102 connects to the public network via this antenna.

[0082] The imaging device 100 described above includes a light receiving element array 2 including a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and a compression unit 10 that generates a compressed signal S2 for each light receiving element group 20 by compressing a signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20. In this manner, the compression unit 10 generates a compressed signal S2 for each light receiving element group 20, and outputs each of these compressed signals (i.e., signals with a reduced signal amount) S2, thereby reducing power consumption in the configuration downstream of the compression unit 10 (such as the control unit 102 in this embodiment).

[0083] Moreover, the imaging device 100 of this embodiment includes a solid-state imaging element 1 having a light receiving element array 2 and a compression section 10. According to this configuration, the compression section 10 in the solid-state imaging element 1 generates a compressed signal S2 for each light receiving element group 20 and outputs each of these compressed signals (i.e., signals with a reduced signal amount) S2, thereby making it possible to reduce power consumption in the configuration downstream of the solid-state imaging element 1 (more specifically, the compression section 10).

[0084] The imaging device 100 of this embodiment also includes a reconstruction processing unit 102A that reconstructs the compressed signal S2 generated by the compression unit 10 into a pixel signal S3 of a pixel in an output image corresponding to the compressed signal S2, based on pre-compression information contained in the compressed signal S2. As a result, the pixel signal S3 is reconstructed downstream of the compression unit 10, and this pixel signal S3 is output to a display unit 106 or the like, thereby improving the image quality of the image displayed on the display unit 106 or the like (i.e., the image obtained from the display unit 106).

[0085] Furthermore, in the imaging device 100 of this embodiment, when the light receiving element array 2 is exposed, the amount of exposure of at least one light receiving element 21 in the light receiving element group 20 is different from the amount of exposure of the other light receiving elements 21 in the light receiving element group 20. With this configuration, it is possible to reduce the signal amount while including, in each compressed signal S2, information on each signal S1a, S1b, S1c, S1d output from the light receiving elements 21a, 21b, 21c, 21d having different amounts of exposure.

[0086] In the imaging device 100 of this embodiment, the difference in the amount of exposure for each of the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 is caused by the difference in the exposure time (charge accumulation time) of each of the light receiving elements 21a, 21b, 21c, and 21d. By adjusting the exposure time for each of the light receiving elements 21a, 21b, 21c, and 21d in this manner to cause a difference in the amount of exposure for the light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20, it is possible to change the amount of exposure for the light receiving elements 21a, 21b, 21c, and 21d by controlling (in the example of this embodiment, by the control of the first circuit unit 3).

[0087] Furthermore, in the imaging device 100 of this embodiment, each compression section 10 generates a compressed signal S2 including an average value of pixel values ​​(signal values) included in the signals S1a, S1b, S1c, and S1d output from each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 for each corresponding light receiving element group 20. A weighted average value may be used instead of the average value.

[0088] In the imaging device 100 of this embodiment, each light receiving element 21 in the light receiving element group 20 has a photoelectric conversion element 211. When the light receiving element array 2 is exposed to light and some light receiving elements 21 have overflowed photoelectric conversion elements 211 and some do not, the reconstruction processing unit 102A calculates a predicted value of the signal value that can be obtained when the photoelectric conversion element 211 of each light receiving element 21 has a capacity that does not overflow due to exposure, and reconstructs the compressed signal S2 into a pixel signal S3. This makes it possible to obtain an output image (image data) with a dynamic range larger than the dynamic range defined by each light receiving element 21, that is, imaging in a high dynamic range is possible.

[0089] Furthermore, in the imaging device 100 of this embodiment, even if the light receiving element array 2 has N light receiving elements 21, since there are m light receiving element groups 20, the number (or number of times) of signals S2 read from the light receiving element array 2 is suppressed to N / m, where N>m.

[0090] This reduces the amount of data read from the light receiving element array 2 compared to a configuration in which the signals of each light receiving element are read out from the light receiving element array. As a result, the amount of data transferred from the solid-state imaging element 1 to the control unit 102 and the like is reduced, and the imaging device 100 can achieve both high image quality, low power, and a high frame rate.

[0091] Furthermore, in the imaging device 100 of this embodiment, since it is possible to omit the operation of reading pixels multiple times, such as in dual conversion gain (DCG), it is possible to achieve both high image quality, low power, and a high frame rate.

[0092] In the imaging device 100 of this embodiment as described above, the signals S1a, S1b, S1c, and S1d output from the multiple light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs the compressed signal S2 containing information of the signals S1a, S1b, S1c, and S1d of the multiple light receiving elements 21a, 21b, 21c, and 21d into a pixel signal S3 corresponding to one pixel in the captured image, thereby generating a captured image. That is, the imaging device 100 of this embodiment generates an output image with a number of pixels corresponding to the number of light receiving element groups 20.

[0093] Next, a second embodiment of the present invention will be described with reference to Figs. 7 to 10. The same components as those in the first embodiment are designated by the same reference numerals, and components different from the first embodiment will be described in detail.

[0094] 7, an imaging device 100 according to the second embodiment includes a light receiving element array 2A including a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and at least one compression unit 10A that generates, for each light receiving element group 20, a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20. The imaging device 100 of the present embodiment includes one compression unit 10A, which generates, for each light receiving element group 20, a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 included in the light receiving element group 20.

[0095] Specifically, the imaging device 100 includes an imaging unit 101 and a control unit 102. Similarly to the first embodiment, the imaging device 100 of this embodiment also includes a non-volatile memory 103, a working memory 104, an operation unit 105, a display unit 106, a recording medium 107, a connection unit 108, a short-range wireless communication unit 109, a public network connection unit 110, a microphone 111, and a speaker 112.

[0096] The imaging unit 101 includes an optical system including at least one optical element, and a solid-state imaging element 1A that converts an image of an object (image capture object) formed through the optical system into an electrical signal (that is, captures an image).

[0097] The solid-state imaging element 1A has a light receiving element array 2A in which a plurality of light receiving elements 21 are arranged in a matrix, a first circuit section 3A and a second circuit section 4A connected to the light receiving element array 2A, and a signal processing section 5A to which the second circuit section 4A is connected.

[0098] In the photodetector array 2A of this embodiment, a plurality of row signal lines 25A are arranged for each row with respect to the photodetectors 21 arranged in a matrix, and a plurality of column signal lines 26A are arranged for each column with respect to the photodetectors 21 arranged in a matrix.

[0099] The first circuit unit 3A controls the light receiving element array 2, and the second circuit unit 4A performs signal processing such as reading out the signals S1 from each light receiving element 21, removing noise, and A / D converting each read out signal S1.

[0100] The signal processing unit 5A is a section where various processes are performed on the compressed signal S2 output from the second circuit unit 4, and also performs signal processing (compression processing) for compressing the signal S1 output from each light receiving element 21 for each light receiving element group 20. That is, in the imaging device 100 of this embodiment, at least a part of the signal processing unit 5A constitutes the compression unit 10A.

[0101] When the first light receiving element 21a is set as a reference light receiving element and the second to fourth light receiving elements 21b to 21d are set as non-reference light receiving elements among the multiple light receiving elements (in this embodiment, the first light receiving element 21a, the second light receiving element 21b, the third light receiving element 21c, and the fourth light receiving element 21d) included in the light receiving element group 20 (see FIG. 10), the signal processing unit 5A generates a compressed signal S2 based on the difference between a reference signal value Sv1a, which is a pixel value (signal value) included in the signal S1a output from the reference light receiving element 21a, and non-reference signal values ​​Sv1b, Sv1c, Sv1d, which are pixel values ​​(signal values) included in the signals S1b, S1c, S1d output from the non-reference light receiving elements 21b, 21c, 21d.

[0102] In the solid-state imaging element 1A of this embodiment, when one signal processing unit 5A (compression unit 10A) processes a signal S1 from each light receiving element 21 of the light receiving element array 2, it processes the signal S1 for each light receiving element group 20, treating a plurality of adjacent light receiving elements (in the example of this embodiment, light receiving elements arranged in a 2 × 2 matrix) 21a, 21b, 21c, and 21d as light receiving element groups 20, to generate a compressed signal S2, and outputs the compressed signal S2 generated for each light receiving element group 20 to a downstream configuration (such as the control unit 102).

[0103] In detail, as shown in FIG. 8, the signal processing unit 5A (compression unit 10A) calculates whether the difference between the non-reference signal values ​​(pixel values) Sv1b, Sv1c, Sv1d included in the signals S1b, S1c, S1d output from the non-reference light receiving elements 21b, 21c, 21d of the light receiving element group 20 and the reference signal value (pixel value) Sv1a included in the signal S1a output from the reference light receiving element 21a is a predetermined threshold value (in the example of this embodiment, each light receiving element 21a When the signal range of the first signal processing unit 5A is 0 to 500, and there are only non-reference signal values ​​(first signal values) Sv1b, Sv1c, and Sv1d whose pixel value difference is less than 25 to 75), a compressed signal S2 including the average value of the pixel values ​​Sv1a, Sv1b, Sv1c, and Sv1d included in the signals S1a, S1b, S1c, and S1d output from the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 is generated (first compression process). The predetermined threshold is set to approximately 10% of the maximum value of the signal range of the light receiving element 21. The first compression process in the signal processing unit 5A of this embodiment is so-called binning. The compressed signal S2 of this embodiment has the average value of the pixel values ​​Sv1a, Sv1b, Sv1c, and Sv1d and position information 3b111. The position information of the compressed signal S2 generated by the first compression process in the signal processing unit 5A (compression unit 10) of this embodiment is a fixed value (3b111).

[0104] Also, as shown in FIG. 9, when there is a non-reference signal value (first signal value) Sv1c whose difference from the reference signal value Sv1a is less than a predetermined threshold value, and non-reference signal values ​​(second signal values) Sv1b, Sv1d whose differences from the reference signal value Sv1a are equal to or greater than a predetermined threshold value, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values ​​Sv1b, Sv1d from the signals output from the light receiving elements 21a, 21b, 21c, 21d. At the same time, the pixel value Sv1a included in the signal S1a output by the reference light receiving element 21a is set to the average value of the pixel value Sv1a and the first signal value Sv1c (100 in the example shown in FIG. 9), and position information (symbol 3b010 of the compressed signal S2 in the example shown in FIG. 9) of the non-reference light receiving element 21c that output the signal S1c including the first signal value Sv1c with respect to the reference light receiving element 21a is added to generate a compressed signal S2 (second compression process).

[0105] In addition, the position information of the second compression process (symbol 3b010 of the compressed signal S2 in FIG. 9) is actually 3-bit data, and is "010" in the example shown in FIG. 9. In this three-digit number, the first number (the leftmost number) indicates the position of the second light receiving element 21b relative to the reference light receiving element (first light receiving element) 21a, the second number indicates the position of the third light receiving element 21c relative to the reference light receiving element 21a, and the third number indicates the position of the fourth light receiving element 21d relative to the reference light receiving element 21a. The pixel values ​​included in the signals S1b, S1c, and S1d from the light receiving elements 21b, 21c, and 21d corresponding to each position are 1 when the pixel value is the first signal value, and 0 when the pixel value is the second signal value.

[0106] The control unit 102 has a reconstruction processing unit 102B, which reconstructs, from the compressed signal S2 output from the solid-state imaging element 1A, pixel signals S3a, S3b, S3c, and S3d of pixels corresponding to each of the light receiving elements 21a, 21b, 21c, and 21d of the light receiving element group 20 corresponding to the compressed signal S2.

[0107] Specifically, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 4, position information 3b111: see FIG. 8) generated by the first compression process, for example, as follows.

[0108] For the compressed signal S2 output from the upper right photosensitive element group 20 in the photosensitive element array 2A in FIG. 10, the reconstruction processing unit 102B reconstructs the compressed signal S2 into each pixel signal S3a, S3b, S3c, S3d by setting the pixel value of the pixel signal S3a of the pixel corresponding to the reference photosensitive element 21a to the pixel value 4 of the compressed signal S2, and setting the pixel values ​​of the pixel signals S3b, S3c, S3d of the pixels corresponding to each of the non-reference photosensitive elements 21b, 21c, 21d to the pixel value 4 of the pixel signal S3a of the pixel corresponding to the reference photosensitive element 21a, as also shown in FIG. 8.

[0109] Furthermore, the reconstruction processing unit 102B reconstructs the compressed signal S2 (pixel value 100, position information 3b010: see FIG. 9) generated by the second compression process, for example, in the following manner.

[0110] 9, for the compressed signal S2 output from the upper left light receiving element group 20 in the light receiving element array 2 in FIG. 10, the reconstruction processing unit 102B sets the pixel value of the pixel signal S3a of the pixel corresponding to the reference light receiving element 21a to the pixel value 100 of the compressed signal S2, sets the pixel value of the pixel signal S3c of the pixel corresponding to the non-reference light receiving elements 21b, 21c, 21d, whose position information (3b010) is 1 to the pixel value 100 of the pixel corresponding to the reference light receiving element 21a, and sets the pixel values ​​of the pixel signals S3b, S3d of the pixel whose position information (3b010) is 0 to the pixel value 4 of the adjacent pixel in the row direction (in the example shown in FIG. 10, the adjacent pixel to the right), thereby reconstructing the compressed signal S2 into each pixel signal S3a, S3b, S3c, S3d. Furthermore, the reconstruction processing unit 102B may determine each pixel value of the pixel signals S3b, S3d of a pixel whose position information (3b010) is 0 to be the weighted average or average value of the pixel value 100 of the compressed signal S2 and the pixel value 4 of the compressed signal S2 output from adjacent light receiving element groups 20 in the row direction.

[0111] 10, in the above imaging device 100, when a vertical line-shaped image (a region that appears white compared to other regions in FIG. 10) is formed at the left end of the light receiving element array 2 during exposure, the signal processing unit 5A (compression unit 10A) performs a first compression process on the light receiving element groups 20 arranged vertically on the right side of the light receiving element array 2, and performs a second compression process row by row on the light receiving element groups 20 arranged vertically on the left side of the light receiving element array 2. The compressed signals S2 generated in this manner for each light receiving element group 20 are output from the solid-state imaging element 1A.

[0112] Next, the reconstruction processing unit 102B of the control unit 102 reconstructs the compressed signal S2 generated by the first compression process and the compressed signal S2 generated by the second compression process using different methods (see Figures 8 and 9), thereby generating a high-resolution image with little blur as image data.

[0113] The imaging device 100 according to this embodiment includes a light receiving element array 2A including a plurality of light receiving element groups 20, each having a plurality of light receiving elements 21, and a signal processing unit 5A (compression unit 10A) that generates, for each light receiving element group 20, a compressed signal S2 obtained by compressing a signal S1 output from the plurality of light receiving elements 21 in the light receiving element group 20. In this manner, the signal processing unit 5A generates a compressed signal S2 for each light receiving element group 20 and outputs each of these compressed signals (i.e., signals with a reduced signal amount) S2, thereby reducing power consumption in the configuration downstream of the signal processing unit 5A.

[0114] In the imaging device 100 of this embodiment, the multiple light receiving elements 21 included in the light receiving element group 20 include a reference light receiving element 21a which is the reference light receiving element 21, and non-reference light receiving elements 21b, 21c, and 21d other than the reference light receiving element 21a. The signal processing unit 5A (compression unit 10A) generates a compressed signal S2 based on the difference between a reference signal value Sv1a which is a pixel value (signal value) included in the signal S1a output from the reference light receiving element 21a, and non-reference signal values ​​Sv1b, Sv1c, and Sv1d which are pixel values ​​(signal values) included in the signals S1b, S1c, and S1d output from the non-reference light receiving elements 21b, 21c, and 21d.

[0115] In this way, the signals S1a, S1b, S1c, and S1d output from the light receiving elements 21a, 21b, 21c, and 21d are compressed based on the difference between the reference signal value (pixel value) Sv1a and the non-reference signal values ​​(pixel values) Sv1b, Sv1c, and Sv1d for each position of the non-reference light receiving elements 21b, 21c, and 21d relative to the reference light receiving element 21a, so that the signal amount difference for each position can be reconstructed in the reconstruction processing unit 102B, and thus the resolution can be ensured in the output image. That is, a high-resolution image with little blurring can be obtained.

[0116] Furthermore, in the imaging device 100 of this embodiment, when there are only non-reference signal values ​​Sv1b, Sv1c, Sv1d whose difference from the reference signal value Sv1a is less than a predetermined threshold value in the signals S1b, S1c, S1d output from the non-reference light receiving elements 21b, 21c, 21d of the light receiving element group 20, the signal processing unit 5A (compression unit 10A) generates a compressed signal S2 including an average value of pixel values ​​(signal values) Sv1a, Sv1b, Sv1c, Sv1d included in the signals S1a, S1b, S1c, S1d output from each light receiving element 21a, 21b, 21c, 21d of the light receiving element group 20. Note that a weighted average value may be used instead of the average value.

[0117] In addition, when there is a first signal value Sv1c, which is a non-reference signal value whose difference from the reference signal value Sv1a is less than a predetermined threshold, and second signal values ​​Sv1b, Sv1d, which are non-reference signal values ​​whose difference from the reference signal value Sv1a is equal to or greater than a predetermined threshold, the signal processing unit 5A (compression unit 10A) deletes the signals S1b, S1d corresponding to the second signal values ​​Sv1b, Sv1d from the signals S1a, S1b, S1c, S1d output from each light receiving element 21a, 21b, 21c, 21d, and generates a compressed signal S2 by adding position information (3b010 in this embodiment example) of the non-reference light receiving element 21c that output the first signal value Sv1c relative to the reference light receiving element 21a to the signal S1a output by the reference light receiving element 21a.

[0118] In this way, by constructing the compressed signal S2 using the signal S1a (more specifically, a signal based on the signal S1a) output from the reference light receiving element 21a and position information (3b010, etc.) added thereto, the signal amount is suitably reduced compared to the case where the signals S1a, S1b, S1c, and S1d output from each light receiving element 21a, 21b, 21c, and 21d of the light receiving element group 20 are all output to a downstream configuration.

[0119] Furthermore, in the imaging device 100 of this embodiment, even if the light receiving element array 2 has N light receiving elements 21, since there are m light receiving element groups 20, the number (or number of times) of signals S2 output from the solid-state imaging element 1A is suppressed to N / m, where N>m.

[0120] This reduces the signal amount (data amount) of the signal S2 output from the solid-state imaging device 1A compared to a configuration in which signals read out from each light receiving element of the light receiving element array are output from the solid-state imaging element. Also, by reconstructing N pixels from information of m pixels+α in a configuration downstream of the solid-state imaging device 1A (such as the reconstruction processing unit 102B), the imaging device 100 can achieve both high image quality, low power, and a high frame rate.

[0121] In the imaging device 100 of this embodiment as described above, the signals S1a, S1b, S1c, and S1d output from the multiple light receiving elements 21a, 21b, 21c, and 21d in the light receiving element group 20 are compressed to become a compressed signal S2, and the reconstruction processing unit 102A reconstructs this compressed signal S2 into pixel signals S3a, S3b, S3c, and S3d of pixels corresponding to each of the light receiving elements 21a, 21b, 21c, and 21d in the captured image, respectively, to generate a captured image. That is, the imaging device 100 of this embodiment generates an output image with a number of pixels corresponding to the number of light receiving elements 21.

[0122] The imaging device 100 and the solid-state imaging elements 1 and 1A of the present invention are not limited to the first and second embodiments described above, and various modifications can be made without departing from the spirit of the present invention. For example, the configuration of one embodiment can be added to the configuration of another embodiment, and part of the configuration of one embodiment can be replaced with the configuration of another embodiment. Furthermore, part of the configuration of one embodiment can be deleted.

[0123] The light receiving element group 20 in the first and second embodiments has four light receiving elements 21a, 21b, 21c, and 21d arranged in a 2×2 configuration, but is not limited to this configuration. The light receiving element group 20 may have a plurality (two or more) of light receiving elements 21. In the light receiving element group 20, the plurality of light receiving elements 21 are preferably arranged in a 2×2 arrangement, a 3×3 arrangement, a 4×4 arrangement, or the like, such that the number of light receiving elements 21 arranged in the row direction is the same as the number of light receiving elements 21 arranged in the column direction (i.e., in a square matrix).

[0124] In addition, in the light receiving element group 20 of the second embodiment described above, the reference light receiving element is arranged in the position of the first row and first column in the matrix arrangement (2 × 2 arrangement) shown in Figures 8 and 9 (the upper left position in the examples of Figures 8 and 9), but it may be arranged in another position.

[0125] In the imaging device 100 of the first and second embodiments, the solid-state imaging element 1, 1A has the compression unit 10, and the reconstruction processing units 102A, 102B are arranged in a position (control unit 102) different from the solid-state imaging element 1, 1A, but this configuration is not limited. The solid-state imaging element 1, 1A may have both the compression unit 10 and the reconstruction processing units 102A, 102B.

[0126] In addition, in each of the imaging devices 100 of the first and second embodiments, the pixel value (average value) included in the compressed signal S2 is a value derived by a so-called simple average, but is not limited to this configuration. The average value (binning) may be a value derived by a weighted average.

[0127] In the second embodiment, the position information in the second compression process that generates the compressed signal S2 is a signal that indicates the positions of the light receiving elements 21b, 21c, and 21d that output the first signal value relative to the reference light receiving element 21a, but is not limited to this configuration. The position information in the second compression process may be a signal that indicates the positions of the light receiving elements 21b, 21c, and 21d that output the second signal value relative to the reference light receiving element 21a, and the positions of the light receiving elements 21b, 21c, and 21d that output the first signal value relative to the reference light receiving element 21a may be specified from this position information.

[0128] In addition, in the above second embodiment, when there is a first signal value and a second signal value in the non-reference signal values ​​included in the signals output from the non-reference light-receiving elements 21b, 21c, and 21d of the light-receiving element group 20, the signal processing unit (compression unit) 5A deletes the signal corresponding to the second signal value from the signals output from each of the light-receiving elements 21a, 21b, 21c, and 21d, and generates the compressed signal S2 by adding position information of the non-reference light-receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value relative to the reference light-receiving element 21a to the signal output from the reference light-receiving element 21a, but is not limited to this configuration. For example, when a first signal value and a second signal value are present in the non-reference signal values ​​contained in the signals output from the non-reference light receiving elements 21b, 21c, and 21d of the light receiving element group 20, the signal processing unit (compression unit) 5A may be configured to generate a compressed signal S2 including a weighted average value or average value of the reference signal value and the first signal value, and position information of the non-reference light receiving elements 21b, 21c, and 21d that output the first signal value or the second signal value relative to the reference light receiving element 21a.

[0129] In order to express the present invention, the present invention has been described adequately and sufficiently through the embodiments with reference to the drawings in the above description, but it should be recognized that those skilled in the art can easily change and / or improve the above-mentioned embodiments. Therefore, unless the changes or improvements made by those skilled in the art are at a level that departs from the scope of the claims described in the claims, the changes or improvements are interpreted as being included in the scope of the claims. [Explanation of symbols]

[0130] 1, 1A... solid-state imaging element, 2, 2A... light receiving element array, 20... light receiving element group, 21... light receiving element, 21a... first light receiving element, reference light receiving element, 21b... second light receiving element, non-reference light receiving element, 21c ...third light receiving element, non-reference light receiving element, 21d...fourth light receiving element, non-reference light receiving element, 211...photoelectric conversion element, 22...amplifier, 25, 25A...row signal line, 26, 26A...column signal line, 3, 3A...first circuit section, 4, 4A...second circuit section, 5...signal processing section, 5A...signal processing section (compression section), 10, 10A...compression section, 100...imaging device, 101...imaging section, 102...control section, 102A, 102B...reconstruction processing section, 103...nonvolatile memory, 104...working memory, 105...operation section, 106...display section, 107...recording medium, 108...connection section, 109...near-field wireless communication section, 110...public network connection section, 111...microphone, 112...speaker, 500...solid-state imaging element, 501...light receiving element array, 502...light receiving element, 50 3...pixel drive wiring, 504...vertical signal line, 505...column parallel signal processing circuit, 506...output circuit, 507...timing control circuit, 508...horizontal scanning circuit, 509...vertical scanning circuit, RES...reset signal, S1, S1a, S1b, S1c, S1d...signal, S2...compression signal, S3, S3a, S3b, S3c, S3d...pixel signal, SEL...select signal, Sv1a...reference signal value, Sv1b, Sv1d...non-reference signal value, second signal value, Sv1c...non-reference signal value, first signal value, t1 to t9...time, T1...first accumulation time, T2...second accumulation time, T3...third accumulation time, T4...fourth accumulation time, TX1...first charge transfer signal, TX2...second charge transfer signal, TX3...third charge transfer signal, TX4...fourth charge transfer signal

Claims

1. a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; a compression unit configured to generate compressed signals for each light receiving element group by compressing signals output from a plurality of light receiving elements in the light receiving element group.

2. The imaging device according to claim 1 , further comprising a solid-state imaging element having the light receiving element array and the compression section.

3. 3. The imaging device according to claim 1, wherein, in the light receiving element group, an amount of exposure of at least one light receiving element in the light receiving element group differs from an amount of exposure of other light receiving elements in the light receiving element group when the light receiving element array is exposed.

4. The imaging device according to claim 3 , wherein the difference in the amount of exposure in the light receiving element group is caused by a difference in exposure time of each light receiving element.

5. 6. The imaging device according to claim 3, wherein the compressed signal includes a weighted average or an average value of signal values ​​included in the signals output from the light receiving elements of the light receiving element group.

6. 6. The imaging device according to claim 1, further comprising a reconstruction processing unit that reconstructs the compressed signal generated by the compression unit into a pixel signal of a pixel in an output image that corresponds to the compressed signal.

7. 6. The imaging device according to claim 1, further comprising a reconstruction processing unit that reconstructs a compressed signal into the pixel signal based on pre-compression information included in the compressed signal.

8. Each light receiving element of the light receiving element group has a photoelectric conversion element, 8. The imaging device according to claim 6, wherein when the photosensitive element array is exposed to light and there are photosensitive elements in the photosensitive element group whose photoelectric conversion elements overflow and those whose photoelectric conversion elements do not overflow, the reconstruction processing unit reconstructs the compressed signal into the pixel signal by calculating a predicted value of a signal value that can be obtained if the photoelectric conversion element of each of the photosensitive elements has a capacity that will not overflow due to the exposure.

9. the plurality of light receiving elements included in the light receiving element group include a reference light receiving element that is a light receiving element serving as a reference, and non-reference light receiving elements other than the reference light receiving element, 3. The imaging device according to claim 1, wherein the compression section generates the compressed signal based on a difference between a reference signal value, which is a signal value included in the signal output from the reference light receiving element, and a non-reference signal value, which is a signal value included in the signal output from the non-reference light receiving element.

10. The compression unit is configured to: generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; 9. The imaging device according to claim 8, wherein, when there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value that is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, a signal corresponding to the second signal value is deleted from the signals output from each of the light receiving elements, and position information of the non-reference light receiving element that output the first signal value or the second signal value relative to the reference light receiving element is added to the signal output from the reference light receiving element, thereby generating the compressed signal.

11. The compression unit is configured to: generating the compressed signal including a weighted average or an average of the signal values ​​output from each light receiving element of the light receiving element group when there is only a non-reference signal value whose difference from the reference signal value is less than a predetermined threshold value; 10. The imaging device of claim 9, wherein when there is a first signal value that is a non-reference signal value whose difference from the reference signal value is less than the predetermined threshold, and a second signal value that is a non-reference signal value whose difference from the reference signal value is equal to or greater than the predetermined threshold, the compressed signal is generated, the compressed signal including a weighted average or average value of the reference signal value and the first signal value, and position information of a non-reference light receiving element that outputs the first signal value or the second signal value relative to the reference light receiving element.

12. a light receiving element array including a plurality of light receiving element groups each having a plurality of light receiving elements; a compression section that generates, for each light-receiving element group, a compressed signal by compressing signals output from a plurality of light-receiving elements in the light-receiving element group.

Images