Camera device and camera method

Abstract

The present invention discloses an imaging apparatus and method capable of generating images for authentication purposes with low processing load. The imaging apparatus comprises: a pixel array unit comprising a plurality of pixels arranged in a matrix and each generating a pixel signal corresponding to light received by exposure, and acquiring image data of each pixel signal generated by the plurality of pixels; a compression unit that compresses the image data to generate compressed image data; a signature generation unit that generates signature data based on the compressed image data; and an output unit that outputs the image data and authentication data for which the signature data is attached to the compressed image data.

Description

technical field

[0001] The disclosure relates to an imaging device and an imaging method. prior art

[0002] In recent years, with the improvement of image processing technology, it is more and more important to prove the authenticity of the image (not tampered with). In order to prove the authenticity of the image, a method of adding a signature to the image captured by the sensor and outputting it to the outside of the sensor can be considered. For example, signature data is generated based on RAW data captured by a sensor, and the RAW data to which the signature data is attached is output to the outside of the sensor. RAW data output to the outside of the sensor is generally subjected to image processing such as contrast adjustment or compression coding processing, and is used as a processed image. Patent Document 1 describes an image sensor that correlates signature information with image information and outputs them.

[0003] The RAW data is certifiable based on the attached signed data. Therefore, outside the sensor, by comparing the processed image after image processing of the RAW data with the image of the RAW data, it can be determined whether the processed image has been tampered with. [Prior Art Literature] [Patent Document]

[0004] [Patent Document 1] Japanese Patent Laid-Open No. 2017-184198 Contents of the invention

[0005] [Problem to be solved by the invention]

[0006] In the above-mentioned existing authenticity proof method, the RAW data and signature for showing that the processed image has not been tampered with must be kept together with the image processed image. Therefore, in the existing authentication methods, the load on the memory in the camera equipped with the sensor, or the load on transmitting data from the camera is relatively large.

[0007] An object of this disclosure is to provide an imaging device and an imaging method capable of generating an image that requires less processing load for authenticity verification. [Technical means to solve the problem]

[0008] The imaging device of the present disclosure includes: a pixel array section including a plurality of pixels arranged in a matrix and each generating a pixel signal corresponding to light received by exposure, and obtaining each of the pixel signals generated by the plurality of pixels The image data; the compression part, which compresses the data amount of the image data to generate compressed image data; the signature generation part, which generates signature data based on the compressed image data; the output part, which outputs the image data and attaches a signature to the compressed image data Documents used to prove the authenticity of the documents. Brief description of the diagram

[0009] FIG. 1 is a schematic diagram showing an example of false image generation in the prior art. Fig. 2(a), (b) are schematic diagrams showing examples of false image generation in existing technologies. FIG. 3A is a schematic diagram showing a configuration example of a conventional camera using a sensor that adds signature data to image data and outputs it. FIG. 3B is a diagram illustrating an example of verifying the authenticity of output image data in the prior art. Fig. 4 is a block diagram schematically showing the configuration of an example of a camera applicable to each embodiment. FIG. 5 is a block diagram showing the configuration of an example of a pixel array unit applicable to each embodiment. Fig. 6 is a diagram showing an example of a generally used Bayer arrangement. FIG. 7A is a diagram showing an example in which the pixel array portion of each embodiment is formed by a two-layer structure of a laminated CIS. FIG. 7B is a diagram showing an example in which the pixel array portion of each embodiment is formed by a three-layered CIS. FIG. 8 is a schematic diagram for explaining the outline of each embodiment. Fig. 9 is a block diagram showing the configuration of an example of the sensor of the first embodiment. Fig. 10(a), (b) are schematic diagrams for explaining the first compression method applicable to the amount of data in the first embodiment. 11(a)~(c) are schematic diagrams for explaining the second compression method applicable to the amount of data in the first embodiment. Fig. 12(a), (b) are schematic diagrams for explaining the third compression method applicable to the amount of data in the first embodiment. Fig. 13A is a schematic diagram showing an example of changing the compression rate of compressed RAW data according to the object of authenticity certification applicable to the first embodiment. Fig. 13B is a schematic diagram showing an example of changing the compression rate of compressed RAW data according to the object of authenticity certification applicable to the first embodiment. Fig. 14 is a block diagram showing the configuration of an example of the sensor of the second embodiment. Fig. 15 is a sequence diagram showing an example of data output processing in the second embodiment. Fig. 16 is a block diagram showing the configuration of an example of the sensor of the third embodiment. Fig. 17 is a block diagram showing the configuration of an example of the sensor of the fourth embodiment. Fig. 18 is a circuit diagram showing the configuration of an example of a voltage holding type pixel applicable to the fifth embodiment. Fig. 19 is a block diagram showing the configuration of an example of the sensor of the fifth embodiment. Fig. 20 is a block diagram showing the configuration of an example of the sensor of the sixth embodiment. Fig. 21 is a schematic diagram for explaining an implementation example of authenticity proof of an image. Fig. 22 is a schematic diagram showing a system configuration for verifying the authenticity of an image according to the seventh embodiment. Fig. 23 is a diagram schematically illustrating authenticity verification of an image in the seventh embodiment. Fig. 24(a) and (b) are diagrams for explaining the first comparison method based on the edge (contour) of an object contained in an image, which can be applied to the seventh embodiment. 25( a ) to ( c ) are diagrams for explaining the second comparison method using color information of images applicable to the seventh embodiment. Implementation

[0010] Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. In addition, in the following embodiments, by attaching the same symbols to the same parts, overlapping descriptions are omitted.

[0011] Hereinafter, embodiments of the present disclosure will be described in the following order. 1. About Existing Technology 2. About technology applicable to each embodiment 3. Outline of each embodiment of this disclosure 4. The first embodiment of this disclosure 4-1. Configuration of the first embodiment 4-2. Data volume compression method applicable to the first embodiment 5. The second embodiment of this disclosure 6. The third embodiment of this disclosure 7. The fourth embodiment of this disclosure 8. The fifth embodiment of this disclosure 8-1. Example of pixel configuration 8-2. Configuration of the fifth embodiment 9. The sixth embodiment of this disclosure 10. The seventh embodiment of this disclosure 10-1. System configuration for judging the authenticity of images related to the seventh embodiment 10-2. About image comparison processing in the seventh embodiment

[0012] [1. Regarding existing technologies] First, prior to the description of the embodiments of the present disclosure, the existing technology related to the technology of the present disclosure will be described for easy understanding. In recent years, it has become a problem to verify the authenticity of images published via the Internet by reporting media or SNS (Social Networking Service: Social Networking Service). That is, in recent years, due to the development of fake (fabricated) image generation technology using image processing tools or AI (Artificial Intelligence), it has become difficult to prove the authenticity of images.

[0013] FIG. 1 and FIG. 2 are schematic diagrams showing examples of false image generation in the prior art. For example, as shown in FIG. 1 , there is known an image 301 that hides the lower half of the original image 300, and predicts the lower half based on the image 301 to generate an image 302a (false image a), or an image different from the image 302a. 302b (false image b) technique.

[0014] Also, even for dynamic images, there is also known a technology that can only delete the moving object 304 contained in the frame 303a from the frame 303a of the dynamic image shown in part (a) of FIG. 2 . In this case, the object 304 is deleted in the frame 303b shown in part (b) which is temporally close to the frame 303a, and the position where the object 304 should exist is replaced with the road surface.

[0015] In these examples, when not knowing the situation of the original image 300 or the frame 303a, it is judged whether the images 302a and 302b, or the frame 303b are fake images that have been tampered with, that is, for the images 302a and 302b, and the frame 303b Proof of authenticity is difficult.

[0016] As one of the methods for easily verifying the authenticity of such images, there is known a method of adding a signature to the captured image of the original image and encrypting the original image with the signature inside the sensor that acquires the captured image.

[0017] FIG. 3A is a schematic diagram showing a configuration example of a conventional camera using a sensor that adds signature data to image data and outputs it.

[0018] In FIG. 3A , the camera 2000 includes a sensor 2100 and an image processing unit 2120 . The sensor 2100 includes a plurality of pixels arranged in a matrix to generate pixel signals corresponding to received light, for example, it is composed of a semiconductor chip (or a plurality of bonded semiconductor chips). The sensor 2100 acquires RAW data in frame units based on pixel signals generated by each of a plurality of pixels according to exposure. RAW data is unprocessed data without performing demosaic processing on pixel data based on pixel signals.

[0019] The sensor 2100 generates signature data 2210 through a signature generation process 2111 based on the RAW data 2200 acquired by the pixel array unit 2110 . The sensor 2100 encrypts the data with the generated signature data 2210 added to the RAW data 2200 using, for example, a secret key 2112 of a public key encryption method. The encrypted data is output from the sensor 2100 and supplied to the image processing unit 2120 . Not limited thereto, the sensor 2100 may also add sensor information 2220 to the encrypted data and supply it to the image processing unit 2120 .

[0020] The image processing unit 2120 performs specific image processing such as demosaic processing, white balance processing, and high-quality processing on the RAW data included in the supplied data to generate visual image data, and performs specific processing on the image data, for example. The compression encoding process compresses the amount of data and outputs it as output image data 2230. In the example of FIG. 3A , the output image data 2230 is a JPEG data obtained by compressing and encoding the image data by means of JPEG (Joint Photographic Experts Group: Joint Photographic Experts Group).

[0021] Here, the image processing unit 2120 uses the public key 2121 corresponding to the secret key 2112 to decode the encryption of the raw data included in the supplied data when performing image processing. Based on the signature data 2210 included in the decoded data, it can be proved that the RAW data 2200 included in the data input to the image processing unit 2120 is the data obtained by the sensor 2100 .

[0022] However, if image processing is performed on the RAW data 2200 acquired by the sensor 2100 in the image processing unit 2120 to change the RAW data 2200, the signature data 2210 attached to the RAW data 2200 in the sensor 2100 becomes meaningless.

[0023] That is, the signature data attached to the sensor 2100 can only be used to prove the authenticity of the RAW data 2200 before the change. Therefore, in the existing technology, it is difficult to determine the authenticity of the falsification of the output image data 2230 output from the image processing unit 2120 .

[0024] In the existing technology, in order to verify the authenticity of the output image data 2230 output from the image processing unit 2120, as shown in FIG. The authenticity of the RAW data 2200 can be verified based on the signed data 2210 . In addition, the output image data 2230 can be authenticated by comparing with the image data of the RAW data 2200 .

[0025] However, generally speaking, compared with the corresponding output image data 2230, the RAW data 2200 has a larger amount of data, and the load of data transmission increases.

[0026] In this disclosure, the data volume of the RAW data 2200 is compressed, the signed data 2210 is added, and encryption processing is performed, and output together with the output image data 2230 . Thereby, the data transmission load when outputting the image data 2230 and the RAW data 2200 can be reduced.

[0027] [2. Technology applicable to each embodiment of the present disclosure] Next, technologies applicable to each embodiment of the present disclosure will be described. Fig. 4 is a block diagram schematically showing the configuration of an example of a camera applicable to each embodiment. In FIG. 4 , the camera 10 includes a sensor 100 , an overall control unit 110 , and an image processing unit 120 . The overall control unit 110 includes, for example, a CPU (Central Processing Unit: Central Processing Unit) and a memory, and controls the overall operation of the camera 10 according to a program stored in the memory.

[0028] The sensor 100 includes a pixel array unit 101 and a signature processing unit 1000 . The pixel array unit 101 includes a plurality of pixels arranged in a matrix to generate pixel signals corresponding to light received through exposure, and acquires image data of each of the pixel signals generated by the plurality of pixels. The image data obtained by the pixel array unit 101 is unprocessed RAW data.

[0029] The signature processing unit 1000 performs data volume compression processing on the RAW data acquired by the pixel array unit 101, and generates signature data based on the data volume compressed RAW data (called compressed RAW data). The signature processing unit 1000 encrypts the compressed RAW data and the signature data.

[0030] The RAW data acquired by the pixel array unit 101 is supplied to the image processing unit 120 as data for image processing. The image processing unit 120 performs specific image processing on the supplied RAW data to generate visual image data. The viewable image data generated by the image processing unit 120 is output from the camera 10 together with the encrypted compressed RAW data generated by the signature processing unit 1000 .

[0031] (Example of the configuration of the pixel array unit applicable to each embodiment) FIG. 5 is a block diagram showing the configuration of an example of the pixel array unit 101 applicable to each embodiment. In FIG. 5 , the pixel array section includes a pixel array 102 , a vertical scanning section 400 , a horizontal scanning / AD conversion section 402 , and a control section 401 .

[0032] The pixel array 102 includes a plurality of pixels 103 having imaging elements that respectively generate voltages corresponding to received light. As an imaging element, a photodiode can be used. In the pixel array 102, a plurality of pixels 103 are arranged in a matrix in the horizontal direction (column direction) and vertical direction (row direction). In the pixel array 102, the arrangement of the pixels 103 in the column direction is called a column. An image (image data) of 1 frame is formed based on pixel signals read from a specific number of columns in the pixel array 102 . For example, when an image of 1 frame is formed with 3000 pixels×2000 columns, the pixel array 102 includes at least 2000 columns including at least 3000 pixels 103 . In the pixel array 102, an area including the pixels 103 for forming an image of one frame is called an effective pixel area. Also, the image data formed in the pixel array 102 is RAW data.

[0033] In addition, in the pixel array 102, the columns and rows of the pixels 103 are connected to pixel signal lines HCTL for each column, and vertical signal lines VSL for each row.

[0034] An end of the pixel signal line HCTL not connected to the pixel array 102 is connected to the vertical scanning unit 400 . The vertical scanning unit 400 transmits, for example, a plurality of control signals such as drive pulses for reading pixel signals from the pixels 103 to the pixel array 102 through the pixel signal line HCTL according to the control signal supplied from the control unit 401 . An end of the vertical signal line VSL not connected to the pixel array 102 is connected to the horizontal scanning / AD conversion unit 402 .

[0035] The horizontal scan / AD conversion unit 402 includes an AD (Analog to Digital: analog to digital) conversion unit, an output unit, and a signal processing unit. The pixel signal read out from the pixel 103 is transmitted to the AD conversion section of the horizontal scanning / AD conversion section 402 through the vertical signal line VSL.

[0036] The outline of the readout and control of the pixel signal from the pixel 103 will be described. The readout of the pixel signal from the pixel 103 is carried out by transferring the charge accumulated in the imaging element by exposure to the floating diffusion layer (FD: Floating Diffusion), and converting the transferred charge into a voltage in the floating diffusion layer. . The voltage of the converted charge in the floating diffusion layer is output to the vertical signal line VSL through the amplifier.

[0037] More specifically, in the pixel 103, during the exposure period, the gap between the imaging element and the floating diffusion layer is set to a disconnected (open) state, and in the imaging device, charges generated by incident light are accumulated by photoelectric conversion. After the exposure is finished, the floating diffusion layer is connected to the vertical signal line VSL according to the selection signal supplied through the pixel signal line HCTL. Furthermore, according to the reset pulse supplied through the pixel signal line HCTL, the floating diffusion layer is connected to the supply line of the power supply voltage VDD or the black level voltage in a short time, and the floating diffusion layer is reset. The voltage of the reset level of the floating diffusion layer (set as voltage P) is output to the vertical signal line VSL. Thereafter, the charge accumulated in the imaging element is transferred to the floating diffusion layer by making the connection between the imaging element and the floating diffusion layer an on (closed) state by a transfer pulse supplied through the pixel signal line HCTL. A voltage corresponding to the charge amount of the floating diffusion layer (set as a voltage Q) is output to the vertical signal line VSL.

[0038] In the horizontal scanning / AD conversion section 402, the AD conversion section includes an AD converter provided for each vertical signal line VSL, and the pixel signal supplied from the pixel 103 via the vertical signal line VSL is subjected to AD conversion processing by the AD converter to generate 2 digital values ​​(values ​​corresponding to voltage P and voltage Q respectively) used for correlating double sampling (CDS: Correlated Double Sampling) processing for noise reduction.

[0039] The 2 digital values ​​generated by the AD converter are subjected to CDS processing by the signal processing unit to generate digital signal pixel signals (pixel data). The generated pixel data is output from the pixel array unit.

[0040] Under the control of the control unit 401, the horizontal scanning / AD conversion unit 402 selects the AD converters of each vertical signal line VSL in a specific order, thereby sequentially outputting the digital values ​​temporarily held by each AD converter to the signal processing department. The horizontal scanning / AD conversion unit 402 realizes this operation by including, for example, a shift register or an address decoder.

[0041] The control 401 performs driving control of the vertical scanning unit 400 , the horizontal scanning / AD conversion unit 402 , and the like in accordance with a control signal from the overall control unit 16 , for example. The control unit 401 generates various drive signals that serve as references for the operations of the vertical scanning unit 400 and the horizontal scanning / AD conversion unit 402 . The control unit 401 generates a control signal for the vertical scanning unit 400 to supply to each pixel 103 through the image signal line HCTL based on a vertical synchronization signal, an external trigger signal, and a horizontal synchronization signal supplied from the outside (for example, the control unit 401 ). The control unit 401 supplies the generated control signal to the vertical scanning unit 400 .

[0042] The vertical scanning unit 400 supplies various signals including drive pulses to each pixel 103 for each column to the pixel signal line HCTL of the selected pixel column of the pixel array 102 based on the control signal supplied from the control unit 401, and each pixel 103 The pixel signal is output to the vertical signal line VSL. The vertical scanning unit 400 is configured using, for example, a shift register or an address decoder.

[0043] The pixel array unit with such a configuration is a CMOS (Complementary Metal Oxide Semiconductor: Complementary Metal Oxide Semiconductor) image sensor in which an AD converter is arranged in each row.

[0044] (Outline of Color Filter Arrangement) Each pixel 103 can be configured with an optical filter that selectively transmits light of a specific wavelength bandwidth. When the wavelength bandwidth transmitted is the wavelength bandwidth of the visible light region, the optical filter is called a color filter. Hereinafter, for each pixel 103, a plurality of types of optical filters having different characteristics, specifically, color filters constituting the respective wavelength bandwidths of red (R), green (G) and blue (B) of the three primary colors are disposed. Not limited thereto, each pixel 103 may also be configured with color filters of various colors having a complementary color relationship, and may also be an optical filter that selectively transmits light of a wavelength bandwidth in the infrared region, or a filter that allows light in the visible light region to pass through. The light of the full wavelength bandwidth passes through the optical filter. Hereinafter, unless otherwise specified, these various optical filters will be described using color filters as a representative.

[0045] Fig. 6 is a diagram showing an example of a generally used Bayer arrangement. In FIG. 6, the Bayer arrangement includes two pixels 103G with a color filter of G color, one pixel 103R with a color filter of R color, and a pixel 103R with a color filter of B color. Pixel 103B. In the Bayer arrangement, the four pixels are arranged in a grid of 2 pixels×2 pixels so that the two pixels 103G are not adjacent to each other. In other words, the Bayer arrangement is an arrangement in which pixels 103 provided with color filters that transmit light of the same wavelength bandwidth are not adjacent to each other.

[0046] In addition, in the following, unless otherwise specified, the "pixel 103R in which the color filter of R color is arranged" is referred to as "the pixel 103R of R color", or simply referred to as "the pixel 103R". The same applies to the pixel 103G provided with the color filter of the G color and the pixel 103B provided with the color filter of the B color. In addition, when not paying particular attention to the color filter, each of the pixels 103R, 110G, and 110B is described as a representative pixel 103 .

[0047] (Example of the structure of the pixel array section applicable to each embodiment) Next, an example of the structure of the pixel array unit 101 applicable to each embodiment will be schematically described.

[0048] The pixel array unit 101 can be applied to a CMOS image sensor (CIS) in which each unit included in the pixel array unit 101 is integrally formed using CMOS (Complementary Metal Oxide Semiconductor). The pixel array unit 101 can be formed on one substrate. Not limited thereto, the pixel array unit 101 may also be a multilayer CIS in which a plurality of semiconductor chips are laminated and integrally formed. In addition, the pixel array unit 101 is not limited to this example, and may be other types of photosensors such as an infrared photosensor for imaging infrared light.

[0049] As an example, the pixel array unit 101 can be formed by a laminated CIS having a two-layer structure in which two semiconductor wafers are laminated. FIG. 7A is a diagram showing an example in which the pixel array section 101 of each embodiment is formed by a two-layered multilayer CIS. In the structure of FIG. 7A, the pixel portion 3020a is formed on the semiconductor wafer of the first layer, and the memory+logic portion 3020b is formed on the semiconductor wafer of the second layer.

[0050] The pixel unit 3020a includes at least the pixel array 102 in the pixel array unit 101 . The memory+logic unit 3020 b may include, for example, a vertical scanning unit 400 , a control unit 401 , a horizontal scanning / AD conversion unit 402 , and a signature processing unit 1000 . The memory + logic part 3020b may further include a memory for storing image data such as RAW data.

[0051] As shown on the right side of FIG. 7A , the pixel array unit 101 is constituted as one solid-state imaging device by electrically contacting and bonding the semiconductor wafer of the first layer and the semiconductor wafer of the second layer.

[0052] As another example, the pixel array unit 101 may be formed by a three-layer structure in which three layers of semiconductor chips are stacked. FIG. 7B is a diagram showing an example in which the pixel array section 101 of each embodiment is formed by a three-layered multilayer CIS. In the structure of FIG. 7B , the pixel portion 3020a is formed on the semiconductor wafer of the first layer, the memory portion 3020c is formed on the semiconductor wafer of the second layer, and the logic portion 3020d is formed on the semiconductor wafer of the third layer. In this case, the logic unit 3020d may include, for example, the vertical scanning unit 400 , the control unit 401 , the horizontal scanning / AD conversion unit 402 , and the signature processing unit 1000 . Also, the memory unit 3020c may include a memory for storing image data such as RAW data.

[0053] As shown on the right side of FIG. 7B , by electrically contacting and bonding the semiconductor wafer of the first layer, the semiconductor wafer of the second layer, and the semiconductor wafer of the third layer, the pixel array part 101 is regarded as a solid body. Composition of camera elements.

[0054] [3. Summary of Embodiments of the Present Disclosure] Next, an outline of each embodiment of the present disclosure will be described. FIG. 8 is a schematic diagram for explaining the outline of each embodiment. The configuration shown in FIG. 8 corresponds to the configuration described using FIG. 4 , and the camera 10 includes a sensor 100 and an image processing unit 120 as the imaging device of the present disclosure. Furthermore, the sensor 100 includes a pixel array unit 101 and a signature processing unit 1000 . In addition, in FIG. 8, the overall control unit 110 is omitted.

[0055] The pixel array section 101 acquires RAW data 200 of each of pixel signals generated by a plurality of pixels according to exposure. The RAW data 200 acquired by the pixel array section 101 is based on pixel signals generated from all the pixels included in the effective pixel area in all the pixels 103 of the pixel array 102 included in the pixel array section 101 .

[0056] The sensor 100 supplies the RAW data 200 acquired by the pixel array unit 101 to the image processing unit 120 as data for image processing. The image processing unit 120 performs demosaic processing on the supplied RAW data to generate visual image data. In addition, the image processing unit 120 performs image processing such as high-quality image processing on the image data generated based on the RAW data, performs compression coding processing on the image processed image data, and generates output image data 230. As the compression encoding method of the image processing unit 120, for example, the JPEG method can be applied.

[0057] The signature processing unit 1000 compresses the data volume of the RAW data acquired by the pixel array unit 101 . The signature processing unit 1000 generates the signature data 213 based on the compressed RAW data (compressed RAW data 212 ). For example, the signature processing unit 1000 generates a hash value based on the compressed RAW data 212, and uses the generated hash value as the signature data 213. The signature processing unit 1000 adds the generated signature data 213 to the compressed RAW data 212 to generate authenticity proof data for authenticity proof of the output image data 230 .

[0058] In addition, the output processing unit 1000 can also obtain the sensor information 211 from the sensor 100, add the obtained sensor information 211 to the signature data 213 and the compressed RAW data 212, and generate the authenticity proof data 210. The sensor information 211 may include, for example, identification information indicating the sensor 100 (manufacturing serial number, etc.), information on imaging conditions, and the like. Not limited to this, the signature processing unit 1000 may also generate the signature data 213 for the data obtained by combining the sensor information 211 and the compressed RAW data 212 , and generate the authenticity proof data 210 . In this way, the sensor information 211 is also included to generate the signature data 213, and tampering of the sensor data 211 can also be prevented.

[0059] The authenticity proof data 210 generated by the signature processing unit 1000 is associated with the output image data 230 generated by the image processing unit 120 and output to the outside of the camera 10, for example. Not limited to this, the authenticity proving data 210 may be associated with the output image data 230 and stored in a non-volatile storage medium such as a flash memory included in the camera 10 .

[0060] In this way, according to each embodiment of the present disclosure, the data volume of the RAW data 200 is compressed, and the compressed RAW data whose data volume is compressed is used as the data for authenticity proof and the compressed image data for viewing are associated and output. Therefore, it is possible to reduce the load on the memory in the camera equipped with the sensor, or the load on transmitting data from the camera.

[0061] Here, the image data used for image processing by the image processing unit 120 must be high-quality data. Therefore, the image processing section 120 is supplied with RAW data of pixel signals generated by all the pixels 103 included in the effective pixel area of ​​the pixel array 102 . On the other hand, the authenticity proving data 210 only needs to know that no objects are added or deleted to the image of the original RAW data. Therefore, as the authenticity proof data 210, the compressed RAW data 212 whose data volume is compressed can be used.

[0062] In addition, when verifying the authenticity of the output image data 230, the size of the compressed RAW data 212 is converted according to the size of the output image data 230, and the size of the converted compressed RAW data 212 is compared with the output image data 230. The details be described later.

[0063] [4. The first embodiment of this disclosure] Next, the first embodiment of the present disclosure will be described. The first embodiment is an example in which the sensor 100 has an output end for outputting RAW data 200 for image processing, and an output end for outputting authenticity proof data 210 including compressed RAW data 212 and signature data 213 .

[0064] (4-1. Configuration of the first embodiment) First, the configuration of the first embodiment will be described. Fig. 9 is a block diagram showing the configuration of an example of the sensor of the first embodiment. In FIG. 9 , a sensor 100 a includes a pixel array unit 101 , a signature processing unit 1000 a , an output interface (I / F) 104 , and a communication / sensor control unit 105 .

[0065] The communication / sensor control unit 105 communicates with an external device such as a host device via a terminal 130 . In addition, the communication / sensor control unit 105 includes, for example, a processor and a memory, and the processor operates according to a program stored in the memory to control the overall operation of the sensor 100a.

[0066] In the example of FIG. 9 , the pixel array unit 101 is displayed including a pixel array 102 , a pixel control unit 107 , and an ADC (Analog to Digital Converter: Analog to Digital Converter) 108 . Wherein, the pixel control unit 107 corresponds, for example, to the output unit of the vertical scanning unit 400 shown in FIG. 5 and the horizontal scanning / AD conversion unit 402 . Also, ADC 108 corresponds to the AD conversion section of horizontal scanning / AD conversion section 402 shown in FIG. 5 .

[0067] The signature processing unit 1000 a includes data processing units 1010 a and 1010 b , a compression processing unit 1020 , and a signature generation unit 1021 . The RAW data 200 output from the pixel array unit 101 are respectively input to the data processing units 1010a and 1010b. The data processing unit 1010a performs specific data processing for image processing in the subsequent image processing unit 120 (not shown) on the input RAW data 200 . The RAW data 200 processed by the data processing unit 1010 a is supplied to the output I / F 104 .

[0068] The data processing unit 1010b performs specific data processing for generating the authenticity proof data 210 on the supplied RAW data 200 .

[0069] The RAW data 200 after data processing by the data processing unit 1010 b is supplied to the compression processing unit 1020 . The compression processing unit 1020 performs compression processing for compressing the amount of data on the supplied RAW data 200 (a specific example will be described later), and generates the compressed RAW data 212 . The compression processing unit 1020 supplies the generated compressed RAW data 212 to the output I / F 104 and the signature generating unit 1021 .

[0070] The signature generation unit 1021 generates signature data 213 based on the supplied compressed RAW data 212 . For example, the signature generating unit 1021 generates a hash value from the supplied compressed RAW data 212 and uses the generated hash value as the signature data 213 . Not limited thereto, if the compressed RAW data 212 can be uniquely specified and the value is difficult to guess, the signature generation unit 1021 may use a value generated by other algorithms as the signature data 213 . The signature generating unit 1021 supplies the generated signature data 213 to the output I / F 104 .

[0071] In addition, the signature generation unit 1021 acquires the sensor information 211 from the communication / sensor control unit 105, for example, and supplies the acquired sensor information 211 to the output I / F 104.

[0072] The output I / F 104 includes two interfaces 131 and 132, and can output two data streams in parallel from these interfaces 131 and 132. As the interfaces 131 and 132, MIPI (Mobile Industry Processor Interface: Mobile Industry Processor Interface) can be applied. An output unit is constituted including an output I / F 104 and interfaces 131 and 132 .

[0073] Specifically, the output I / F 104 outputs the RAW data 200 supplied from the data processing unit 1010a from the interface 131 (first output end) in a state where it remains unchanged.

[0074] Also, the output I / F 104 adds the signature data 213 supplied from the signature generation unit 1021 to the compressed RAW data 212 supplied from the compression processing unit 1020 . The output I / F 104 outputs the sensor information 211, the compressed RAW data 212 and the signature data 213 as authenticity proof data 210 from the interface 132 (second output terminal). In addition, the authenticity proof material 210 may be encrypted with a secret key in the public key encryption method and output.

[0075] In this way, the sensor 100a of the first embodiment outputs the RAW data 200 and the authenticity verification data 210 through different interfaces 131 and 132, respectively. Therefore, it is possible to prevent the output of the RAW data 200 and the authenticity proof data 210 from being limited by the frame rate.

[0076] (4.2. Data volume compression method applicable to the first embodiment) Next, a method of compressing data volume applicable to the compression processing unit 1020 of the first embodiment will be described.

[0077] Fig. 10 is a schematic diagram for explaining the first compression method applicable to the data volume of the compression processing unit 1020 of the first embodiment. In the first compression method, the compression processing unit 1020 obtains the addition of pixel data for each specific number of pixels 103 that are provided with color filters of the same color and arranged in close positions in the arrangement of the pixels 103 of the pixel array 102 average. The compression processing unit 1020 calculates the average of the pixel data of the specific number of pixels 103, aggregates the specific number of pixels 103 into one pixel, and generates the compressed RAW data 212 that compresses the data amount of the RAW data 200.

[0078] In the example of FIG. 10 , the compression processing unit 1020 sets, for example, a red color filter in the Bayer arrangement shown in part (a), and obtains the pixels 103R 1 , 103R 2 , 103R 3 , and 103R 4 that are close to each other. The average of the pixel data of those. The compression processing unit 1020 generates the pixel 103R add having the pixel data of the pixel data of the pixels 103R 1 to 103R 4 averaged by addition. The compression processing unit 1020 also performs the same processing on the other pixels 103G and 103B to generate two pixels 103G add and 103B add. The compressed RAW data 212 is generated by arranging the pixels 103R add, 2 pixels 103G add, and the pixels 103B add in a Bayer arrangement. The compressed RAW data 212 is the data that compresses the data amount of the original RAW data 200 to 1 / 4.

[0079] According to the first compression method, by setting color filters of the same color, the addition average of the pixel data of pixels close to each other is obtained to generate compressed RAW data 212 pixels, which can compress the data amount of RAW data 200 and reduce noise .

[0080] Fig. 11 is a schematic diagram for explaining the second compression method applicable to the data volume of the compression processing unit 1020 of the first embodiment. In the second compression method, the compression processing unit 1020 generates compressed RAW data 212 by thinning out the RAW data 200 in units of pixels.

[0081] In the example of FIG. 11 , the compression processing unit 1020 thins out the RAW data 200 shown in the part (a) in units of columns as shown in the part (b). At this time, the compression processing unit 1020 thins out every two columns according to the repeating pattern of the Bayer array in the row direction. Part (c) of FIG. 11 shows an example in which the compressed RAW data 212 is constructed by combining the thinned rows closely together. In the case of this example, the compressed RAW data 212 is data that compresses the data amount of the original RAW data 200 to 1 / 2.

[0082] According to the second compression method, since no computation is required to compress the data amount of the RAW data 200, the load on the compression processing unit 1020 can be reduced. Also, according to the second compression method, the number of times of AD conversion in the ADC 108 can also be reduced.

[0083] Fig. 12 is a schematic diagram for explaining a third compression method applicable to the data volume of the compression processing unit 1020 of the first embodiment. In the third compression method, the amount of data is compressed for each pixel based on a specific compression algorithm. More specifically, the compression processing unit 1020 reduces the number of bits of the pixel data of each pixel of the RAW data 200 based on a specific compression algorithm to generate the compressed RAW data 212 . The compressed RAW data 212 is data obtained by compressing the data amount of the RAW data 200 by the number of bits of each pixel data of the RAW data 200 .

[0084] As an example, assume that the number of bits of each pixel data of the RAW data 200 is 12 bits. Part (a) of FIG. 12 shows an example of reducing the number of bits of each pixel data from 12 bits to 8 bits, and part (b) shows an example of reducing the number of bits from 12 bits to 6 bits. In the example of part (a), the compressed RAW data 212 is to compress the data amount of the original RAW data 200 to 2 / 3 of the data. Also, in the example of part (b), the compressed RAW data 212 is the data that compresses the data volume of the original RAW data 200 to 1 / 2.

[0085] As a method of reducing the number of bits of pixel data, several methods can be considered. As a first method, a method of reducing lower order bits of each pixel data of the RAW data 200 may be considered. In the example of part (a) of FIG. 12 , it is assumed that the lower 4 bits of the 12-bit pixel data of the RAW data 200 are deleted, so that the 4th bit of the pixel data becomes 8 bits of the 0th bit Element pixel data. In the example of part (b) of the figure, it is set to delete the lower 6 bits of the 12-bit pixel data of the RAW data 200, so that the 6th bit of the pixel data becomes the 6th bit of the 0th bit pixel data.

[0086] As a second method of reducing the number of bits of pixel data, after the grayscale compression process is performed on the RAW data 200, the lower-order bits are deleted in the same way as the above-mentioned first method. As grayscale compression processing, grayscale correction processing may be applied. In the above-mentioned first method, although the information of the part with a small contrast is lost, the loss of this information can be supplemented by performing gray scale compression processing.

[0087] As a third method of reducing the number of bits of pixel data, a method of utilizing the continuity of pixels can be considered. In the third method, for example, the number of bits is reduced by sequentially obtaining the difference between the pixel data of the adjacent pixels in accordance with the arrangement of the pixels based on the previous pixel (the first pixel of the column, etc.). As the third method, a compression algorithm specified in MIPI (Mobile Industry Processor Interface) can be applied.

[0088] The method of reducing the amount of data per pixel is not limited to the first to third methods above. In addition, the above-mentioned first method or second method, and the third method may be combined.

[0089] The compression rate of the compressed RAW data 212 can be changed according to the object of authenticity verification. 13A and 13B are schematic diagrams showing an example of changing the compression rate of the compressed RAW data 212 according to the object of authentication, which can be applied to the first embodiment.

[0090] FIG. 13A shows an example of judging whether or not to delete or add an object 34 included in an image 33 as an object of authentication. In this way, the compression rate of the compressed RAW data 212 can be increased when a large change of the image 33 is detected, for example. That is, in this case, the data size of the compressed RAW data 212 can be made smaller.

[0091] FIG. 13B shows an example of the face of a person 36 included in the judging image 35 as an object of authentication. For example, when it is desired to prove that the face of the person 36 is real (not tampered with), it can only be compressed to a size (resolution) that can prove that the face has not been tampered with. Therefore, in this case, and in the case of deleting or adding the above-mentioned object 34, the compression ratio of the compressed RAW data 212 is reduced.

[0092] A specific example of changing the compression rate of the compressed RAW data 212 according to the object of authentication will be described using the example of the third compression method with the amount of data described above. It can be considered to apply the example shown in part (b) of FIG. 12 to the situation of judging the face of a person 36 when judging whether there is an object 34 included in the image 33 as an object of authenticity proof. , apply the example shown in part (a) of Figure 12.

[0093] In addition, in the sensor 100a, the compression rate of the compressed RAW data 212 can also be set to be fixed, and when the authentication of the output image data 230 is performed externally based on the compressed RAW data 212, a tamper-determinable level is presented. The level of tampering is determined to be, for example, corresponding to the following conditions, that is, whether it is guaranteed that there is no major tampering such as deleting or adding objects to the image, and whether it is guaranteed to be the same face as the original RAW data 200 by face authentication. For example, in the authenticity proof data 210 , it may be considered that the sensor information 211 includes the compression rate of the compressed RAW data 212 .

[0094] [5. The second embodiment of this disclosure] Next, a second embodiment of the present disclosure will be described. In the first embodiment described above, the RAW data 200 and the authenticity proof data 210 are output from different interfaces 131 and 132 . On the other hand, in the second embodiment, the RAW data 200 and the authenticity proof data 210 are output from one interface. At this time, in the second embodiment, the RAW data 200 and the authenticity proving data 210 are output in column units in time division.

[0095] Fig. 14 is a block diagram showing the configuration of an example of the sensor of the second embodiment. In FIG. 14, the sensor 100b has an interface 131 for outputting the output data of the I / F104. Furthermore, the sensor 100b and the arbiter 1030 having the column memory 1031 are inserted between the signature processing unit 100b and the output I / F 104.

[0096] In FIG. 14, the columns of the pixel array 102 are shown as columns L#1, L#2, . . . respectively. In the data processing units 1010a and 1010b, the pixel array unit 101 outputs the data of each column L#1, L#2, . . . from the ADC 108 in column order. The data of each column L#1, L#2, .

[0097] The data processing unit 1010a performs specific data processing on the supplied column data and supplies them to the arbiter 1030 . The arbiter 1030 stores the column data supplied from the data processing unit 1010 a in the column memory 1031 .

[0098] The data processing unit 1010b performs specific data processing on the supplied row data, and supplies it to the compression processing unit 1020 . The compression processing unit 1020 performs a column-direction compression process on the supplied column data to compress the data amount, and supplies the compressed column data to the arbiter 1030 and the signature generation unit 1021 . The signature generation unit 1021 generates the signature data 213 based on the supplied data, and supplies the generated signature data to the arbiter 1030 .

[0099] The arbiter 1030 mediates between the row data of the RAW data supplied from the data processing unit 1010b and stored in the row memory 1031, the row data compressed in the row direction supplied from the compression processing unit 1020, and the signature data supplied from the signature generation unit 1021. output timing. The arbiter 1030 outputs the row data of the RAW data stored in the row memory 1031 in row order. Accompanying with this, the arbiter 1030 outputs the signature data and the compressed row data of a plurality of rows according to the output timing of each specific row of the row data of the RAW data.

[0100] Each data output from the arbiter 1030 is supplied to the output I / F 104 and output from the interface 131 .

[0101] Fig. 15 is a sequence diagram showing an example of data output processing in the second embodiment. In FIG. 15 , the column synchronous signal output from ADC108, the column data output from ADC108, the column synchronous signal output from arbiter 1030 (output I / F104), and the column synchronous signal output from sensor 100b (arbitration Each data output by device 1030). The synchronous period of the column synchronous signal output by ADC108 corresponds to the period of the horizontal synchronous signal Hsync of the pixel array part 101.

[0102] Column data L#1, L#2, . . . are sequentially output from ADC108 for each horizontal synchronization signal Hsync. The row data L#1, L#2, . For example, the column data L#1 output from the ADC 108 is supplied to the arbiter 1030 via the data processing unit 1010 a and stored in the column memory 1031 . If the following column data L#2 is output from the ADC108, the arbiter 1030 reads the column data L#1 from the column memory 1031 and supplies it to the output I / F 104 according to the column synchronous signal used for output, and stores the column data L#2 in column memory 1031.

[0103] If the following column data L#3 is output from the ADC108, the arbiter 1030 reads the column data L#2 from the column memory 1031 and supplies them to the output I / F104 according to the column synchronization signal for output.

[0104] The compression processing unit 1020 compresses the column data L#3 supplied from the ADC 108 and the column data L#1 already supplied from the ADC 108 in the horizontal direction, and supplies the compressed column data L#1 and L#3 to the arbiter 1030 Generate section 1021 with a signature. The signature generation unit 1021 generates the signature data 213 for each of the row data L#1 and L#3 based on the compressed and supplied row data L#1 and L#3. The arbiter 1030 uses the compressed column data L#1 and L#3 supplied from the compression processing unit 1020 and the signature data 213 generated based on the column data L#1 and L#3 supplied from the signature generation unit 1021 as a column. The data 210 for proving the authenticity of the data L#1 and L#3 is supplied to the output I / F104 according to the output synchronous signal.

[0105] Here, the column synchronous signal for output is set as follows: during the period of 2 column quantities of the column synchronous signal output by ADC108, output the column data of 2 columns of RAW data, and the signed data of 2 columns and the compressed column data. In this way, the sensor 100b of the second embodiment sets the column synchronous signal output by the ADC 108 and the column synchronous signal for output, and the arbiter 1030 mediates the column data of the two columns of RAW data and the signature of the two columns Output timing of data and compressed column data.

[0106] Thereby, the sensor 100b of the second embodiment can output RAW data and authenticity proof data from one interface 131 . In addition, the sensor 100b of the second embodiment can generate signature data from the same data as the output RAW data without using a frame memory.

[0107] In addition, in the configuration of the second embodiment, since the output control is complicated, there is a possibility that the operation of the sensor 100b limits the output bandwidth.

[0108] [6. The third embodiment of this disclosure] Next, a third embodiment of the present disclosure will be described. The third embodiment of this disclosure is an example in which a sensor has a frame memory, and the frame memory is used to output the RAW data 200 and the authenticity proof data 210 from one interface.

[0109] Fig. 16 is a block diagram showing the configuration of an example of the sensor of the third embodiment. In FIG. 16, the sensor 100c has an interface 131 for outputting the output data of the I / F104. In addition, the sensor 100c is inserted into the frame memory 1040 immediately after the compression processing unit 1020 in the signature processing unit 1000c, and the selector 1050 is provided at the output unit of the signature processing unit 1000c. In addition, the sensor 100c includes a data processing unit 1010 in the signature processing unit 1000c that performs specific data processing on the RAW data 200 supplied from the pixel array unit 101, and the data processing unit 1010 uses the RAW data for output. The data 200 is shared with the RAW data 200 used to generate the authenticity proof data 210 .

[0110] In FIG. 16 , the RAW data 200 output from the pixel array unit 101 is subjected to specific data processing in the data processing unit 1010 and supplied to the selector 1050 and the compression processing unit 1020 . The compression processing unit 1020 performs compression processing of the compressed data volume on the supplied RAW data 200 to generate the compressed RAW data 212 . The compression processing unit 1020 stores the generated compressed RAW data 212 in the frame memory 1040 . The frame memory 1040 only needs to have a capacity capable of storing at least one frame worth of compressed RAW data 212 .

[0111] The compressed RAW data 212 read from the frame memory 1040 is supplied to the selector 1050 and the signature generator 1021 . The signature generating unit 1021 generates signature data 213 based on the supplied compressed RAW data 212 , and supplies the generated signature data 213 to the selector 1050 .

[0112] The selector 1050 selects the RAW data 200 supplied from the data processing unit 1010, the compressed RAW data 212 supplied from the frame memory 1040, and the signature data supplied from the signature generation unit 1021 according to the control of the communication / sensor control unit 105. 213 selects the data to be output and supplies it to the output I / F104.

[0113] For example, the selector 1050 first selects the RAW data 200 supplied from the data processing unit 1010, and supplies one frame worth of the RAW data 200 to the output I / F 104. The output I / F 104 outputs the supplied RAW data 200 for 1 frame from the interface 131 .

[0114] Next, the selector 1050 selects, for example, the compressed RAW data 212 supplied from the frame memory 1040 and supplies it to the output I / F 104 , and then supplies the signature data 213 supplied from the signature generation unit 1021 to the output I / F 104 . The output I / F 104 collects the compressed RAW data 212 , signature data 213 , and sensor information 211 obtained from the communication / sensor control unit 105 as authenticity proof data 210 , and outputs it from the interface 131 .

[0115] In this way, the sensor 100c of the third embodiment must have the frame memory 1040, and on the other hand, the output control becomes simple.

[0116] [7. Fourth embodiment of this disclosure] Next, a fourth embodiment of the present disclosure will be described. The fourth embodiment of the present disclosure is an example in which a sensor has a frame memory, and the frame memory is used to output the RAW data 200 and the authenticity proof data 210 from one interface. The difference between the fourth embodiment and the above-mentioned third embodiment is that the frame memory is placed before the data processing unit 1010 .

[0117] Fig. 17 is a block diagram showing the configuration of an example of the sensor of the fourth embodiment. In FIG. 17, the sensor 100d is the same as the sensor 100c shown in FIG. 16, and has an interface 131 for outputting the output data of the I / F104. On the other hand, in the sensor 100d, a frame memory 1041 and a selector 1051 are provided on the input side of the data processing unit 1010 in the signature processing unit 1000d.

[0118] The RAW data 200 output from the pixel array unit 101 is supplied to the selector 1051 and stored in the frame memory 1041 . The selector 1051 selects the RAW data 200 to be output from the RAW data 200 output from the pixel array unit 101 and the RAW data 200 read from the frame memory 1041 in accordance with the control of the communication / sensor control unit 105, and supplies them to Data processing unit 1010 . The data processing unit 1010 performs specific data processing on the supplied RAW data 200 and supplies it to the selector 1050 and the compression processing unit 1020 .

[0119] The compression processing unit 1020 performs compression processing of the compressed data volume on the supplied RAW data 200 to generate the compressed RAW data 212 . The compression processing unit 1020 supplies the generated compressed RAW data 212 to the selector 1050 and the signature generating unit 1021 . The signature generating unit 1021 generates signature data 213 based on the supplied compressed RAW data 212 , and supplies the generated signature data 213 to the selector 1050 .

[0120] The selector 1050 selects the RAW data 200 supplied from the data processing unit 1010, the compressed RAW data 212 supplied from the frame memory 1040, and the signature data supplied from the signature generation unit 1021 according to the control of the communication / sensor control unit 105. 213 selects the data to be output and supplies it to the output I / F104.

[0121] For example, the selector 1050 first selects the RAW data 200 supplied from the data processing unit 1010, and supplies one frame worth of the RAW data 200 to the output I / F 104. The output I / F 104 outputs the supplied RAW data 200 for 1 frame from the interface 131 .

[0122] Next, the selector 1050 selects, for example, the compressed RAW data 212 supplied from the frame memory 1040 and supplies it to the output I / F 104 , and then supplies the signature data 213 supplied from the signature generation unit 1021 to the output I / F 104 . The output I / F 104 collects the compressed RAW data 212 , signature data 213 , and sensor information 211 obtained from the communication / sensor control unit 105 as authenticity proof data 210 , and outputs it from the interface 131 .

[0123] In the case of the generation process of the signature data 213 of the generation part 1021, the RAW data 200 read from the frame memory 1041 is selected according to the control of the communication / sensor control part 105. Also, corresponding to the selection of the selector 1051, select here, the selector 1051 performs the data amount compression processing of the compression processing unit 1020 and the signature selector 1050 selects the compression processing unit according to the control of the communication / sensor control unit 105 The output of 1020, and then select the output of the signature generating part 1021.

[0124] On the other hand, when the selector 1050 selects the output of the data processing unit 1010, the selector 1051 selects the output of the pixel array unit 101 under the control of the communication / sensor control unit 105.

[0125] In addition, in the example of FIG. 17, although the frame memory 1041 is provided in the signature processing part 1000d, it is not limited to this. For example, when a frame memory is provided in the pixel array unit 101 for other processing, the frame memory can also be used for the purpose of the frame memory 1041 . For example, in the configuration of the quasi-global shutter, there is a frame memory for storing each pixel data read out at high speed in the pixel array unit 101, and the pixel data is read out from the frame memory within one frame period, for example. situation. The frame memory of the quasi-global shutter can be used for the purpose of the frame memory 1041 .

[0126] [8. Fifth embodiment of this disclosure] Next, a fifth embodiment of the present disclosure will be described. In the fifth embodiment, the reading of the RAW data 200 for output from the pixel array unit 101 and the reading of the RAW data 200 for authenticating the data 210 are performed at different timings. In this case, in the fifth embodiment, a voltage holding type pixel is used as an example of each pixel 103 included in the pixel array unit 101 . In order to hold the pixel signal generated by exposure, the voltage hold type pixel can read out the pixel signal generated by one exposure multiple times.

[0127] (8-1. Example of Pixel Configuration) FIG. 18 is a circuit diagram showing the configuration of an example of a voltage holding type pixel 103VD applicable to the fifth embodiment. In FIG. 18 , the pixel 103VD includes a front-stage circuit 510 , capacitive elements 521 and 322 , a selection circuit 530 , a rear-stage reset transistor 541 , and a rear-stage circuit 550 .

[0128] The front-end circuit 510 includes a photoelectric conversion element 511 , a transmission transistor 512 , FD (Floating Diffusion, floating diffusion layer) reset transistors 513 , FD514 , a front-end amplification transistor 515 , and a current source transistor 516 .

[0129] The photoelectric conversion element 511 generates charges by photoelectric conversion. The transfer transistor 512 transfers charges from the photoelectric conversion element 511 to the FD 514 according to the transfer signal trg from the vertical scanning unit 400 .

[0130] The FD reset transistor 513 is initialized by extracting charges from the FD 514 according to the FD reset signal rst from the vertical scanning unit 400 . FD514 is one that accumulates charge and generates a voltage corresponding to the amount of charge. The front-stage amplifying transistor 515 amplifies the voltage level of the FD514 and outputs it to the front-stage node 520 .

[0131] In addition, drains of the FD reset transistor 513 and the front-stage amplifier transistor 515 are connected to the power supply voltage VDD. The current source transistor 516 is connected to the source of the front amplifier transistor 515 . The current source transistor 516 supplies the current id1 according to the control of the vertical scanning unit 400 .

[0132] One end of each of the capacitive elements 521 and 522 is commonly connected to the front node 520 , and the other end is connected to the selection circuit 530 .

[0133] The selection circuit 530 includes a selection transistor 531 and a selection transistor 532 . The selection transistor 531 switches the path between the capacitive element 521 and the subsequent node 540 according to the selection signal Φr from the vertical scanning unit 400 . The selection transistor 532 switches the path between the capacitive element 522 and the subsequent node 540 according to the selection signal Φ s from the vertical scanning unit 400 .

[0134] The rear reset transistor 541 initializes the level of the rear node 540 to a specific potential Vreg according to the rear reset signal rstb from the vertical scanning unit 400 . For the potential Vreg, a potential different from the power supply potential VDD (for example, a potential lower than VDD) is set.

[0135] The rear circuit 550 includes a rear amplification transistor 551 and a rear selection transistor 552 . The back-stage amplifying transistor 551 is for amplifying the level of the back-stage node 540 . The rear selection transistor 552 outputs the signal amplified by the rear amplification transistor 551 as a pixel signal to the vertical signal line VSL according to the rear selection signal selb from the vertical scanning unit 400 .

[0136] In addition, as various transistors (transfer transistor 512 etc.) in the pixel 103VD, for example, nMOS (n-channel Metal Oxide Semiconductor: n-channel Metal Oxide Semiconductor) is used.

[0137] The vertical scanning unit 400 supplies a high-level FD reset signal rst and a transfer signal trg to all pixels at the start of exposure. Thereby, the photoelectric conversion element 511 is initialized. Hereinafter, this control is referred to as "PD reset".

[0138] In addition, the vertical scanning unit 400 sets the subsequent reset signal rstb and the selection signal Φr to a high level for all pixels immediately before the end of the exposure, and supplies a high level FD reset signal rst during the traverse pulse period. Thereby, the FD514 is initialized, and the level corresponding to the level of the FD514 at this time is held in the capacitive element 521 . Hereinafter, this control is referred to as "FD reset".

[0139] Hereinafter, the level of FD514 when FD is reset, and the level corresponding to the level (the holding level of the capacitive element 521, or the level of the vertical signal line VSL) are collectively referred to as "P phase" or "reset phase". Level".

[0140] When the exposure ends, the vertical scanning unit 400 sets the rear reset signal rstb and the selection signal Φs to a high level for all pixels, and supplies a high-level transfer signal trg during the traversal pulse period. Thereby, the signal charge corresponding to the exposure amount is transmitted to the FD514, and the level corresponding to the level of the FD514 at this time is held in the capacitive element 522.

[0141] Hereinafter, the level of FD514 when transmitting the signal charge, and the level corresponding to the level (the holding level of the capacitive element 522, or the level of the vertical signal line VSL) are collectively referred to as "D phase" or "signal phase". Level".

[0142] Such exposure control that starts and ends exposure for all pixels at the same time is called the global shutter method. Through the exposure control, the front-end circuit 510 of all pixels sequentially generates reset levels and signal levels. The reset level is kept in the capacitive element 521 , and the signal level is kept in the capacitive element 522 .

[0143] After the exposure is finished, the vertical scanning unit 400 sequentially selects a row, and sequentially outputs the reset level and signal level of the row. When outputting the reset level, the vertical scanning unit 400 sets the FD reset signal rst of the selected column and the subsequent selection signal selb to a high level, and supplies a high level selection signal Φr throughout a specific period. Thereby, the capacitive element 521 is connected to the rear node 540, and the reset level is read out.

[0144] After reading the reset level, the vertical scanning unit 400 maintains the state of setting the FD reset signal rst and the subsequent segment selection signal selb of the selected row to a high level, and supplies a high level subsequent segment reset signal during the traverse pulse period. rstb. Thereby, the level of the subsequent node 540 is initialized. At this time, both the selection transistor 531 and the selection transistor 532 are in an on state, and the capacitive elements 521 and 522 are disconnected from the rear node 540 .

[0145] After the initialization of the rear node 540, the vertical scanning unit 400 keeps the FD reset signal rst and the rear selection signal selb of the selected column at a high level, and supplies a high level selection signal Φ s throughout a specific period. . Thereby, the capacitive element 522 is connected to the subsequent node 540, and the signal level is read out.

[0146] Through the above-mentioned readout control, the selection circuit 530 in the selected column sequentially performs the control of connecting the capacitive element 521 to the subsequent node 540, the control of disconnecting the capacitive elements 521 and 322 from the subsequent node 540, and the control of connecting the capacitive element 521 to the subsequent node 540. The element 522 is connected to the control of the downstream node 540 . Moreover, when the capacitive elements 521 and 322 are separated from the rear node 540 , the selected rear reset transistor 541 initializes the level of the rear node 540 . In addition, the rear circuit 550 of the selected row sequentially reads the reset level and the signal level from the capacitive elements 521 and 522 through the rear node 540, and outputs them to the vertical signal line VSL. Based on the difference between the signal levels and the reset level, pixel signals can be obtained.

[0147] Thus, in the voltage holding type signal 103VD, the reset level and the signal level are held in the capacitive elements 521 and 522 . Therefore, the pixel 103VD can read out the reset level and the signal level multiple times from the capacitive elements 521 and 522 to obtain pixel signals respectively. In addition, since the pixel signal is generated based on the reset level and the signal level held in the capacitive elements 521 and 522, it can be considered that the pixel signal is held in the pixel 103VD.

[0148] (8-2. Configuration of the fifth embodiment) Next, a configuration example of the sensor of the fifth embodiment will be described. Fig. 19 is a block diagram showing the configuration of an example of the sensor of the fifth embodiment. In FIG. 19, the sensor 100e includes a pixel array unit 101VD and a signature processing unit 1000e. The pixel array section 101VD includes a pixel array 102VD using the above-mentioned pixels 103VD, and a pixel control section 107VD for driving the pixels 103VD.

[0149] The RAW data 200 output from the pixel array unit 101VD is input to the signature processing unit 1000e, and supplied to the data processing unit 1010. The data processing unit 1010 performs specific data processing on the supplied RAW data 200 and supplies it to the selector 1050 and the compression processing unit 1020 .

[0150] The compression processing unit 1020 performs compression processing of the compressed data volume on the supplied RAW data 200 to generate the compressed RAW data 212 . The compression processing unit 1020 supplies the generated compressed RAW data 212 to the selector 1050 and the signature generating unit 1021 . The signature generating unit 1021 generates signature data 213 based on the supplied compressed RAW data 212 , and supplies the generated signature data 213 to the selector 1050 .

[0151] The selector 1050 selects the RAW data 200 supplied from the data processing unit 1010, the compressed RAW data 212 supplied from the frame memory 1040, and the signature data supplied from the signature generation unit 1021 according to the control of the communication / sensor control unit 105. 213 selects the data to be output and supplies it to the output I / F104. The output I / F 104 outputs the supplied data from the interface 131 .

[0152] Here, the pixel 103VD can read out the pixel signal generated by one exposure multiple times. On the other hand, the pixel signal read from the pixel 103VD is affected by switching noise every time it is read, and its quality is degraded. Also, although the quality of the authenticity proof data 210 is not important, for example, the compression processing unit 1020 can reduce noise by obtaining the average of pixel data together with the compression processing of the data volume.

[0153] Based on the characteristics of these pixels 103VD, the sensor 100e selects the RAW data 200 based on the pixel signal first read out for one exposure by the selector 1050, and outputs it from the interface 131 (1st). Next, the sensor 100e selects the compressed RAW data 212 and the signature data 213 by the selector 1050, and the additional sensor information 211 is output as the authenticity proof data 210 (2nd). The above-mentioned compressed RAW data 212 is compressed by the compression processing unit 1020 Based on the data amount of the RAW data 200 of the pixel signal read out again for the exposure, the signature data 213 is generated by the signature generating unit 1021 based on the compressed RAW data 212 .

[0154] In this way, in the fifth embodiment, the RAW data 200 based on the pixel signal first read from each pixel 103VD for one exposure is output from the interface 131 . Then, based on the exposed pixel signal read out again from each pixel 103VD, compressed RAW data 212 and signature data 213 are generated, and authenticity verification data 210 is output. Thereby, the RAW data 200 can be output with high image quality, and the authenticity proving data 210 based on the pixel signal reproduced by the same exposure as the RAW data 200 can be output.

[0155] Also, since the reset level and signal level of each pixel 103VD are held in the capacitive elements 521 and 522, the entire pixel array 102VD can be regarded as a frame memory, and no additional frame memory is required.

[0156] [9. The sixth embodiment of this disclosure] Next, a sixth embodiment of the present disclosure will be described. The sixth embodiment is an example in which two frames of imaging are continuously performed. In this case, for example, the RAW data 200 obtained by the first imaging is output from the interface 131 . Then, based on the RAW data 200 obtained by the second imaging, compressed RAW data 212 and signature data 213 are generated and output as authenticity proof data 210 . On the premise that the subject does not have a sudden scene change, the two images taken continuously can be regarded as the same, and the authenticity proof data 210 based on the second shooting can be used to execute the RAW data 200 based on the first shooting proof of authenticity.

[0157] Fig. 20 is a block diagram showing the configuration of an example of the sensor of the sixth embodiment. The sensor 100f shown in FIG. 20 has the same configuration as the sensor 100e shown in FIG. 19 except that the pixel 103 of the non-voltage holding type is used in the pixel array portion 101.

[0158] In the example of FIG. 20 , the sensor 100f outputs the RAW data 200 captured in the nth frame from the interface 131 (Frame#n). And, the sensor 100f uses the compressed RAW data 212 based on the RAW data 200 captured in the next (n+1)th frame, the signature data 213 generated based on the compressed RAW data 212, and the additional sensor information 211 as The authenticity proof data 210 is output from the interface 131 .

[0159] In addition, not limited to this example, the sensor 100f may also output the authenticity proving data 210 in the nth frame, and output the RAW data 200 in the next (n+1)th frame.

[0160] Thus, in the sixth embodiment, the sensor 100f separates the imaging for outputting the RAW data 200 from the imaging for generating and outputting the authenticity proof data 210 . Therefore, even if the RAW data 200 and the authenticity proof data 210 are output from one interface 131, there is no need to provide a frame memory.

[0161] [10. The seventh embodiment of the disclosure] Next, a seventh embodiment of the present disclosure will be described. The seventh embodiment is an example of authenticating the output image data 230 using the authenticity proving data 210 output from the camera 10 (sensors 100a to 100f) in the above-mentioned first to sixth embodiments.

[0162] (10-1. Regarding the system configuration for authenticating an image in the seventh embodiment) A schematic description will be given of the system configuration for authenticating an image according to the seventh embodiment. Fig. 21 is a schematic diagram for explaining an implementation example of authenticity proof of an image. The output image data 230 output from the camera 10 to which each embodiment is applied is finally authenticated by the person (organization) who posted the output image data 230 or the person (organization) who made evidence judgment based on the output image data 230 . . The authenticity proof is compared with the output image data 230 , and the image of the compressed RAW data 212 included in the authenticity proof data 210 output from the camera 10 and the image of the output image data 230 are performed.

[0163] The output image data 230 is published by the reporting media 50 such as newspapers, TV stations, and news websites, for example. The reporting media 50 or a third party obtains the output image data 230 and the authenticity proving data 210 corresponding to the output image data 230, and certifies the compressed RAW data 212 contained in the authenticity proving data 210 as authentic (not Tampered) output image data 230, for example, is published as a news image.

[0164] In another example, the output image data 230 is disclosed to the account holder through SNS (Social Networking Service). The SNS manager 51 or a third party obtains the output image data 230 and the authenticity proving data 210 corresponding to the output image data 230, and certifies the authenticity based on the compressed raw data 212 included in the authenticity proving data 210 ( Unaltered) output image data 230 "guaranteed to be authentic (unaltered)" is posted on SNS.

[0165] In another example, the output image data 230 is used by the public institution 52 for evidence images or certification images. As a public institution 52 in this case, a court or a passport issuing authority (Ministry of Foreign Affairs, etc.) may be considered. For example, the public agency 52 only recognizes the image of the output image data 230 that is associated with the signed data 213 and certified as a real thing (not tampered with) as a document of public certification.

[0166] Fig. 22 is a schematic diagram showing a system configuration for verifying the authenticity of an image according to the seventh embodiment.

[0167] In FIG. 22 , the camera 10 outputs the output image data 230 and the authenticity proof data 210 . The compressed RAW data 212 and signature data 213 contained in the authenticity proof data 210 are sent to the cloud computing service 60 via the Internet, for example, and stored in a server in the cloud computing service 60 . Reporting media 50, SNS managers 51 or public agencies 52 access cloud computing service 60, and can obtain compressed RAW data 212 and signed data 213 stored in the server.

[0168] On the other hand, the output image data 230 output from the camera 10 is attached with the signature data 213 corresponding to the output image data 230 and the related data 214 showing the association with the compressed RAW data 212 corresponding to the output image data 230, The document 240 for authenticity proof is handed over to the media 50, the SNS manager 51, or the public institution 52. At this time, the authenticity proving data 240 is linked in an unchangeable state using the authenticity proving data 240 corresponding to the other output image data 230 and a known technology such as a block chain.

[0169] The media 50, the SNS manager 51, or the public institution 52 acquire the desired authenticity-proof data 240 from the linked authenticity-proof data 240 . The reporting media 50, SNS manager 51 or public agency 52 obtains the information corresponding to the authenticity proving data 240 stored in the server in the cloud computing service 60 based on the related data 214 contained in the obtained authenticity proving data 240. RAW data 212 and signed data 213 are compressed.

[0170] The reporting media 50, SNS manager 51 or public agency 52 confirms that the compressed RAW data 212 obtained together with the signed data 213 has not been tampered with based on the signed data 213 obtained from the cloud computing service 60.

[0171] Reporting media 50, SNS managers 51 or public agencies 52 are roughly shown in Figure 23, comparing the compressed RAW data 212 confirmed to be unaltered based on the signature data 213, and the output image data included in the obtained authenticity proof data 240 230. Prove the authenticity of the output image data 230. For example, the reporting media 50, the SNS manager 51 or the public agency 52 obtain the degree of similarity between the compressed RAW data 212 and the output image data 230, and verify the authenticity of the output image data 230 based on the obtained similarity.

[0172] Reporting media 50, SNS managers 51, or public institutions 52 post the output image data 230 certified as authentic (not falsified) by the certificate of authenticity, or use them as evidence images or proof images.

[0173] (10-2. About image comparison processing in the seventh embodiment) Next, the comparison process between the output image data 230 and the compressed RAW data 212 in the seventh embodiment will be described more specifically. Various methods can be considered for image comparison, but here, as the image comparison method, the first comparison method based on the edge (outline) of the object included in the image, and the second comparison method using the color information of the image The method is explained.

[0174] (about the first comparison method) The first comparison method will be described. In the first comparison method, at the time of comparison, first, a process of matching the image size of the compressed RAW data 212 with the image size of the output image data 230 to be authenticated is performed.

[0175] For example, when compressing the RAW data 212 in the case of compressing the amount of data by using the first compression method explained in FIG. The size of each is enlarged using linear interpolation or the like. The size of the image of the compressed RAW data 212 is matched to the size of the image of the output image data 230 by the upscaling process in each of the vertical and horizontal directions.

[0176] Also, for example, when the compressed RAW data 212 compresses the amount of data by using the second compression method for row thinning described in FIG. . The size of the image of the compressed RAW data 212 is matched to the size of the image of the output image data 230 by the enlargement process in the vertical direction.

[0177] On the other hand, when the compressed RAW data 212 is compressed by using the third compression method for reducing the number of bits per pixel described in FIG. Size matching treatment.

[0178] Fig. 24 is a diagram for explaining the first comparison method based on the edges (contours) of objects included in images, which can be applied to the seventh embodiment.

[0179] In FIG. 24, part (a) schematically shows an example of an image of compressed RAW data 212 resized for size matching as described above. Also, part (b) shows an example of an image of the output image data 230 . Since the compressed RAW data 212resize does not perform demosaic processing, the visual appearance is different from the image of the output image data 230 shown in part (b).

[0180] As shown in parts (a) and (b) of FIG. 24 , the resized image of the compressed RAW data 212 and the image of the output image data 230 include the same object 70 .

[0181] As shown in part (a) of FIG. 24 , the edge 71a of the object 70 included in the image is detected by using a known edge detection method for the resized image of the compressed RAW data 212 . As an example of an edge detection method, a method using a first-order differential filter such as a Sobel filter is known. Similarly, as shown in part (b), for the image of the output image data 230, the edge 71b of the object 70 included in the image is detected.

[0182] A feature value is calculated for each of the edge 71 a detected from the image resized from the compressed RAW data 212 and the edge 71 b detected from the image of the output image data 230 . Based on the feature quantity of the edge 71a and the feature quantity of the edge 71b, for example, the difference between the two is extracted, so as to obtain the similarity between the edge 71a and the edge 71b. When the calculated similarity is greater than or equal to the threshold value, it is determined that the image of the output image data 230 is genuine (not tampered with).

[0183] (about the second comparison method) The second comparison method will be described. In the second comparison method, when performing the comparison, first, the compressed RAW data 212 is subjected to demosaic processing, and the color information of the image of the compressed RAW data 212 is restored. The color information of the image of the compressed RAW data 212 after restoring the color information is compared with the color information of the image of the output image data 230 .

[0184] Fig. 25 is a diagram for explaining a second comparison method using color information of images applicable to the seventh embodiment. In the second comparison method, demosaic processing is performed on the compressed RAW data 212 shown in part (a) of FIG. The size of the image is matched with the size of the image of the output image data 230 through enlarging processing and the like. Part (b) of FIG. 25 shows an example of an image of the compressed RAW data 212 dem after the demosaic processing and size matching processing are performed on the compressed RAW data 212 .

[0185] Compare the color information of the compressed RAW data 212dem with the color information of the output image data 230 shown in part (c) of FIG. 25 to obtain the similarity between the compressed RAW data 212dem and the output image data 230. For example, for each of the compressed RAW data 212dem and the output image data 230, feature quantities related to color information are calculated. Based on the feature quantity of the color information of the compressed RAW data 212dem and the feature quantity of the color information of the output image data 230, for example, the difference between the two can be obtained to obtain the similarity between the compressed RAW data 212dem and the output image data 230. When the calculated similarity is greater than or equal to the threshold value, it is determined that the image of the output image data 230 is genuine (not tampered with).

[0186] In addition, in the second comparison method, the comparison based on the color information and the comparison based on the edge of the object in the first comparison method may be executed in combination.

[0187] In addition, the above-mentioned first and second comparison methods are all executed by calculation using a processor such as a CPU (Central processing Unit). Not limited to this example, it is also possible to verify the authenticity of the output image data 230 by visually comparing the image of the compressed RAW data 212 and the image of the output image data 230 . In this case, it is preferable to use an image of the compressed RAW data 212 that at least performs demosaic processing on the compressed RAW data 212 and restores color information.

[0188] In addition, the effects described in this specification are only examples, not limitations, and other effects may also be possible.

[0189] In addition, the present technology may also adopt the following configurations. (1) A camera device comprising: A pixel array section comprising a plurality of pixels arranged in a matrix and each generating a pixel signal corresponding to light received through exposure, and obtaining image data of each of the pixel signals generated by the plurality of pixels; a compression unit, which compresses the data amount of the above-mentioned image data to generate compressed image data; a signature generation unit, which generates signature data based on the above-mentioned compressed image data; An output unit for outputting the above-mentioned image data and the data for authenticity verification in which the above-mentioned signature data is added to the above-mentioned compressed image data. (2) The imaging device as in (1) above, wherein The above output section contains: a first output terminal, which outputs the above-mentioned image data; and The second output terminal outputs the above-mentioned authenticity proof data. (3) The imaging device as in (1) above, wherein The above-mentioned output unit includes a row memory for storing the above-mentioned image data in row units in the row direction in the above-mentioned matrix arrangement, and The above-mentioned image data and the above-mentioned authenticity proof data are output from the output terminal in time-division by the above-mentioned row unit using the above-mentioned column memory. (4) As above-mentioned (1) imaging device, it further has: frame memory, which stores the above-mentioned compressed image data corresponding to the above-mentioned image data of at least one frame; and The above-mentioned signature generation unit generates the above-mentioned signature data based on the above-mentioned compressed image data stored in the above-mentioned frame memory, The above output output the above-mentioned image data of the above-mentioned 1 frame from the output terminal, and Output from the above-mentioned output terminal: attach the above-mentioned signature data generated based on the above-mentioned compressed image data stored in the above-mentioned frame memory to the above-mentioned authenticity proof data of the compressed image data. (5) As above-mentioned (1) imaging device, it further has: a frame memory, which stores at least one frame of the above-mentioned image data obtained by the above-mentioned pixel array unit; and The above-mentioned compression unit generates the above-mentioned compressed image data based on the above-mentioned image data stored in the above-mentioned frame memory, The output unit outputs the image data obtained by the pixel array unit from the output terminal, The above-mentioned authenticity proof data is output from the above-mentioned output terminal, the above-mentioned compression part uses the above-mentioned image data stored in the above-mentioned frame memory to generate the above-mentioned compressed image data, and the above-mentioned signature generation part is based on the above-mentioned The above-mentioned signature data is generated by compressing image data, and the above-mentioned signature data is attached to the compressed image data. (6) The imaging device as in (1) above, wherein Each of the plurality of pixels is a voltage-holding type pixel having a holding portion for holding the above-mentioned pixel signal corresponding to received light, The compression unit generates the compressed image data based on the image data of the pixel signal held in the holding unit of each of the plurality of pixels, The output section outputs the image data of the image signal read from the holding section of each of the plurality of pixels from an output terminal, Outputting the above-mentioned data for proving authenticity from the above-mentioned output terminal, the data for proving the authenticity is generated by the above-mentioned compressing part using the above-mentioned image data of the above-mentioned pixel signal read again from the above-mentioned holding part of each of the above-mentioned plural pixels. The above-mentioned compressed image data is generated by the above-mentioned signature generating unit based on the compressed image data, and the above-mentioned signature data is attached to the compressed image data. (7) The imaging device as in (1) above, wherein The output unit outputs the image data of the pixel signal generated by the first exposure from the output terminal, The image data of the pixel signal generated by the second exposure following the first exposure is output from the output terminal. (8) The imaging device according to any one of the above (1) to (7), wherein The above-mentioned pixel array unit provides a plurality of kinds of optical filters with different optical characteristics according to a specific pattern for each of the above-mentioned plural pixels, The above-mentioned compressing unit adds up the pixel signals for each of the specific number of pixels that are provided with the above-mentioned optical filter corresponding to the above-mentioned type and arranged in the above-mentioned array in the above-mentioned plurality of pixels, and sums up the above-mentioned specific number of pixels It is 1 pixel, thereby compressing the above-mentioned amount of data of the above-mentioned image data. (9) The imaging device according to any one of the above (1) to (8), wherein The compression unit compresses the data amount of the image data by deleting the lower order bits of the pixel data based on the pixel signals of each of the plurality of pixels included in the image data to reduce the number of bits of the pixel data. (10) The imaging device as in (9) above, wherein The compression unit deletes the lower-level bits of the pixel data that has undergone grayscale compression processing. (11) The imaging device according to any one of the above (1) to (10), wherein The compression unit compresses the data amount of the image data based on the continuity of the image. (12) The imaging device according to any one of the above (1) to (11), wherein The compression unit compresses the data amount of the image data by thinning out the image data in units of pixels. (13) The imaging device as in (12) above, wherein The compression unit thins out the image data in units of columns in a column direction in the matrix arrangement. (14) The imaging device as in (13) above, wherein The above-mentioned pixel array unit is provided with a plurality of types of optical filters with different optical characteristics according to a specific pattern for each of the plurality of pixels, The compressing unit thins out in units of columns according to the repetition of the specific pattern in the row direction in the matrix arrangement. (15) The imaging device according to any one of the above (1) to (14), wherein The pixel array unit, the compression unit, the signature generation unit, and the output unit are integrally formed. (16) The imaging device as in (15) above, which comprises the following components: The first wafer for disposing the above-mentioned pixel array unit; and The second chip is provided with the above-mentioned compression unit, the above-mentioned signature generation unit and the above-mentioned output unit, and is attached to the above-mentioned first chip. (17) A camera method, which is executed by a processor, and has the following steps: An obtaining step of obtaining an image of each of the above-mentioned image signals generated by the plurality of pixels obtained by a pixel array section including a plurality of pixels arranged in a matrix and each generating a pixel signal corresponding to received light material; A compression step, which compresses the data volume of the above-mentioned image data to generate compressed image data; A signature generation step, which generates signature data based on the above-mentioned compressed image data; and An output step of outputting the above-mentioned image data, and the data for authenticity proof in which the above-mentioned signature data is added to the above-mentioned compressed image data.

[0190] 10: camera 33: Image 34: object 35: image 36: people 50: Reporting the media 51: SNS manager 52:Public institutions 60: Cloud Computing Services 70: Object 71a: Edge 71b: Edge 100: sensor 100a~100f: sensor 101:Pixel array department 101VD: Pixel Array Department 102: pixel array 102VD: pixel array 103: pixel 103B: pixel 103B add: pixel 103G: pixel 103G add: pixel 103R: pixel 103R 1~103R 4: pixel 103R add: pixel 103VD: pixel 104: Output I / F 105:Communication / Sensor Control Unit 107:Pixel control department 107VD: pixel control unit 108: ADC 110: Overall Control Department 120: Image processing department 130: pixels 131: interface 132: interface 200: RAW data 210: Materials used to prove authenticity 211: Sensor information 212: Compress RAW data 212dem: Compress RAW data 212resize: Compress RAW data 213: Signature information 214: Related information 230: Output image data 240: Materials used to prove authenticity 300: original image 301: image 302a: Image 302b: Image 303a: Frame 303b: Frame 304: object 400: vertical scanning part 401: Control Department 402:Horizontal scan / AD conversion unit 510: front circuit 511: photoelectric conversion element 512: Transmit transistor 513: FD reset transistor 514:FD 515: front amplifier transistor 516: Current source transistor 520: front node 521: capacitive element 522: capacitive element 530: select circuit 531: select transistor 532: select transistor 540: back segment node 541: rear reset transistor 550: rear circuit 551: Rear amplification transistor 552: rear selection transistor 1000: signature processing department 1000a~1000e: signature processing department 1010: Data Processing Department 1010a: Data Processing Department 1010b: Data Processing Department 1020: Compression processing department 1021: Signature Generation Department 1030: Arbiter 1031: column memory 1040: frame memory 1041: frame memory 1050: selector 1051: selector 2000: camera 2100: sensor 2110: Pixel Array Department 2111: signature generation process 2112:Secret key 2120: Image Processing Department 2121: public key 2200: RAW data 2210: signed information 2220: Sensor information 2230: Output image data 3020a: Pixel Department 3020b: Memory + Logic 3020c: Memory Department 3020d: Logic Department HCTL: pixel signal line Hsync: horizontal synchronization signal id1: current L#1~L#7: column data rst: FD reset signal rstb: rear segment reset signal selb: rear segment selection signal trg: transmit signal VDD: power supply voltage VDD: power supply potential Vreg: Potential VSL: vertical signal line Φ r: selection signal Φ s: select signal

Claims

1. A camera device comprising: an image sensor including a plurality of pixels arranged in a matrix, each of the plurality of pixels generating a pixel signal corresponding to light received by exposure, the image sensor acquiring image data based on each of the pixel signals generated by the plurality of pixels, the image data including first frame data and second frame data, the first frame data and the second frame data representing consecutive frames; and one or more electronic processors configured to: compress a first amount of data of the first frame data to generate first compressed image data; compress a second amount of data of the second frame data to generate second compressed image data; generate signature data based on the second compressed image data but not based on the first compressed image data, the signature data serving as authentication data for both the first frame data and the second frame data; and output the image data and the authentication data obtained by attaching the signature data to the second compressed image data.

2. The camera device of claim 1, wherein the one or more electronic processors are configured to: output the image data via a first output terminal; and output the authenticity verification data via a second output terminal.

3. The camera device of claim 1 further comprises: a column memory configured to store the image data in column units along the column direction of the matrix arrangement, and the one or more electronic processors configured to output the image data and the authenticity verification data from the output terminal in a time-division multiplexing manner using the column memory in column units.

4. The camera device of claim 1 further comprises: a frame memory configured to store the second compressed image data; and the one or more electronic processors further configured to: generate the signature data based on the second compressed image data stored in the frame memory, output the image data from an output terminal, and output from the output terminal the authenticity verification data obtained by attaching the signature data generated based on the second compressed image data stored in the frame memory to the second compressed image data.

5. The camera device of claim 1 further comprises: a frame memory that stores the image data of at least one frame acquired by the image sensor; and the one or more electronic processors are further configured to: generate the second compressed image data based on the image data stored in the frame memory; output the image data acquired by the image sensor from an output terminal; and output the authenticity verification data from the output terminal, wherein the authenticity verification data is obtained by attaching the signature data to the second compressed image data, and the signature data is generated based on the second compressed image data generated using the image data stored in the frame memory.

6. The imaging apparatus of claim 1, wherein each of the plurality of pixels is a voltage-holding pixel, the voltage-holding pixel having a holding circuit configured to hold the pixel signal corresponding to the received light, and the one or more electronic processors are further configured to: generate the second compressed image data based on the image data of the pixel signals held by the holding circuits of each of the plurality of pixels, output the image data from an output terminal, the image data being: based on the pixel signals read from the holding circuits of each of the plurality of pixels, output the authenticity verification data from the output terminal, the authenticity verification data being: obtained by attaching the signature data to the second compressed image data, the signature data being generated based on the second compressed image data, the second compressed image data being generated using the image data, and the image data being: based on the pixel signals read again from the holding circuits of each of the plurality of pixels.

7. The imaging device of claim 1, wherein the one or more electronic processors are further configured to: output the image data based on the pixel signal generated by the first exposure from the output terminal, and output the image data based on the pixel signal generated by the second exposure following the first exposure from the output terminal.

8. The imaging apparatus of claim 1, wherein the image sensor comprises: a plurality of optical filters having different optical properties, each of the plurality of optical filters being arranged in a specific pattern for each of the plurality of pixels, and the one or more electronic processors being configured to: add pixel signals to each of the plurality of pixels having a specific number of the optical filters corresponding to the types disposed therein, the specific number of pixels being arranged close to each other in the arrangement, and combining the specific number of pixels into one pixel, thereby compressing the data amount of the image data.

9. The camera device of claim 1, wherein the one or more electronic processors are further configured to: delete lower-order bits of pixel data based on pixel signals of each of the plurality of pixels contained in the image data, reduce the number of bits of the pixel data, thereby compressing the second data amount of the second frame data.

10. The camera device of claim 9, wherein the one or more electronic processors are further configured to: delete the lower-order bits of the pixel data that has undergone grayscale compression processing.

11. The camera device of claim 1, wherein the one or more electronic processors are further configured to: compress the second data amount of the second frame data based on the continuity of the image.

12. The camera device of claim 1, wherein the one or more electronic processors are further configured to: compress the amount of the second data in the second frame data by sparsifying the second frame data in pixel units.

13. The camera device of claim 12, wherein the one or more electronic processors are further configured to: sparsify the second frame data in the column units of the matrix arrangement.

14. The imaging apparatus of claim 13, wherein the image sensor comprises: a plurality of optical filters having different optical properties, each of the plurality of optical filters being arranged in a specific pattern for each of the plurality of pixels, and the one or more electronic processors being configured to: sparsify in column units by repeating the row direction in the matrix arrangement of the specific pattern.

15. The camera device of claim 1, wherein the one or more electronic processors are integrally formed.

16. The camera device of claim 15, wherein the one or more electronic processors comprises: a first chip; and a second chip, wherein the second chip is attached to the first chip.

17. A camera method executed by a processor, comprising the following steps: an acquisition step, which acquires image data of each of the pixel signals generated by a plurality of pixels, the image data including first frame data and second frame data, the first frame data and the second frame data representing consecutive frames, the pixel signals being acquired by an image sensor including the plurality of pixels arranged in a matrix, each of the plurality of pixels generating a pixel signal corresponding to received light; a first compression step, which compresses a first amount of data of the first frame data to generate first compressed image data; and a second compression step, which compresses a second amount of data of the second frame data to generate second compressed image data. The signature generation step generates signature data based on the second compressed image data instead of the first compressed image data, and the signature data serves as authentication data for both the first frame data and the second frame data; and the output step outputs the image data and the authentication data obtained by attaching the signature data to the second compressed image data.

Images