Solid-state imaging device

Abstract

[The present invention suppresses image degradation in an image sensor.] This solid-state imaging device comprises: a pixel array in which pixels that perform photoelectric conversion of received light and output analog signals are arranged in a two-dimensional array; a conversion unit that converts the analog signals outputted from the pixels to digital data; a coding unit that generates one or a plurality of code bits for the digital data; a storage unit that saves the digital data and the code bit or bits; a decoding unit that decodes the code bit or bits for the digital data stored in the storage unit; a determination unit that, on the basis of the decoded code bit or bits, determines whether or not an error has occurred in the writing or reading of the digital data to or from the storage unit; and a signal processing unit that, on the basis of the output from the determination unit, processes the digital data read from the storage unit.

Description

[Technical field]

[0001] The present disclosure relates to a solid-state imaging device. [Background technology]

[0002] An image sensor has photodiodes (PD: Photo Diodes) arranged two-dimensionally on the sensor, and when capturing an image, the light received by these PDs is photoelectrically converted, and the charge is further converted into a digital value to output digital image data. In the column AD method, which has a dedicated ADC (Analog to Digital Converter) for multiple rows or columns of pixels, the output process of digital data is performed, for example, for each row, which is called a rolling shutter. In an image sensor that performs imaging processing using a rolling shutter, the readout time in the sensor differs for each row, so when a moving object is captured, the output image may be distorted due to rolling shutter distortion. In order to suppress this distortion, it is desirable to perform AD conversion at high speed.

[0003] Increasing the speed of AD conversion in an image sensor has the disadvantage of requiring a higher throughput in the circuit that performs the signal processing in the subsequent stage. Increasing the throughput of the signal processing circuit to address this creates new problems such as increased chip power consumption and circuit area. To solve these problems, a method has been implemented in which the output value is stored in the memory within the sensor after AD conversion in the image sensor. This allows for faster AD conversion and reduced throughput in the signal processing circuit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-085664 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the case of image sensors with built-in memory, image data is first stored in the memory before being read out, so if an error occurs in the timing of reading and writing from the memory, the output image may have an incorrect value. There is a method to reduce the impact of errors in the timing of reading and writing data by adding a correction code to make the sensor error-resistant, but there is a problem that the higher the correction strength, the higher the cost of the circuit becomes.

[0006] Therefore, the present disclosure provides a solid-state imaging device that suppresses image degradation in an image sensor with a built-in memory. [Means for solving the problem]

[0007] According to one embodiment, a solid-state imaging device includes a pixel array in which pixels are arranged in a two-dimensional array, the pixels performing photoelectric conversion on received light and outputting analog signals, a conversion unit that converts the analog signals output from the pixels into digital data, an encoding unit that generates one or more code bits for the digital data, a memory unit that stores the digital data and the code bits, a decoding unit that decodes the code bits for the digital data stored in the memory unit, a determination unit that determines whether an error has occurred in writing or reading the digital data to or from the memory unit based on the decoded code bits, and a signal processing unit that processes the digital data read from the memory unit based on the output of the determination unit.

[0008] The code bit may be an error detection code or an error correction code. By using an error correction code, it is possible to determine whether or not an error can be corrected by a simple process.

[0009] The determination unit may further determine whether or not an error is correctable in writing to or reading from the storage unit.

[0010] The signal processing unit may perform error correction based on the code bit when an error occurs and the determination unit determines that the error is correctable. The signal processing unit may perform the error correction. In this manner, specific signal processing other than detection may be performed by the signal processing unit.

[0011] The signal processing unit may execute detection of defects in the memory unit or pixels when an error occurs and the determination unit determines that the error is uncorrectable. In this manner, when the error cannot be corrected by the error correction code, the signal processing unit may determine that the error occurred at a timing other than when writing to, storing in, or reading from the memory.

[0012] When an error occurs and the determining unit determines that the error is uncorrectable, the signal processing unit may execute correction of the digital data in which the error is detected. In this manner, when the error is uncorrectable by the error correction code, the signal processing unit may execute an interpolation and defect correction process.

[0013] The determination section may add error information to the digital data and transmit the digital data to the signal processing section. In this manner, the determination section may output the digital data together with the error information.

[0014] The error information may comprise information to determine whether an error has occurred, for example, if the code bit is an error detection code, it may be information as to whether there is an error.

[0015] The error information may also include information for determining whether an error is correctable, for example, if the code bit is an error-correcting code, the information may be for determining whether an error is correctable, if there is an error.

[0016] When acquiring an image for a plurality of frames, the determination unit may add error information to the digital data between frames of the digital data for the image and transmit the digital data to the signal processing unit. In transmitting the data to the signal processing unit, the error information may be stored in the blank between the frames in this manner.

[0017] The determination unit may add error information to each packet constituting the digital data and transmit the data to the signal processing unit. In transmitting data to the signal processing unit, the error information may be stored in each packet data having a predetermined length in this manner.

[0018] The determination unit may add error information to each pixel value based on one or more pixels constituting the digital data and transmit the data to the signal processing unit. In this manner, when transmitting data to the signal processing unit, the error information may be stored for each predetermined number of pixels.

[0019] The decoding unit may perform error correction in the digital data when detecting a correctable error by decoding the code bits. In this way, when an error is detected in the error correction code rather than in the signal processing unit, the decoding unit may perform the correction and transmit the error-corrected data to the signal processing unit. At this time, the fact that the error has been corrected may be further added as error information.

[0020] According to an embodiment, a solid-state imaging device may include a first substrate on which at least a pixel array is arranged, and a second substrate on which at least a storage unit is arranged.

[0021] The second substrate may further include a conversion unit, an encoding unit, a decoding unit, and a judgment unit. In this manner, the conversion unit, the encoding unit, the storage unit, the decoding unit, and the judgment unit may be formed on the same chip. By forming them in this manner, each component can operate by communicating with the memory without complicating the interface.

[0022] The second substrate may further include a signal processing section, and the signal processing section may be formed on the same chip as the above chip.

[0023] The first substrate and the second substrate may be formed by stacking them together. That is, the solid-state imaging device may include a chip having an image sensor with a pixel array mounted thereon, and a chip performing signal processing and including a memory unit, etc., in a stacked structure.

[0024] The device may further include a third substrate on which at least a signal processing unit is disposed. In this manner, the signal processing unit may be provided on a substrate separate from the second substrate including the storage unit, etc. In this case, data may be transferred to the signal processing unit via the above-mentioned interface.

[0025] The first substrate, the second substrate, and the third substrate may be formed by stacking them together. That is, the solid-state imaging device may be formed by stacking a chip having an image sensor with a pixel array, a chip having a storage unit and the like for performing simple signal processing, and a chip having a signal processing unit for performing complex signal processing in a three-tiered configuration.

[0026] At least two substrates may be stacked in a CoC (Chip on Chip) manner.

[0027] At least two substrates may be stacked in a CoW (Chip on Wafer) manner.

[0028] At least two substrates may be stacked by a WoW (Wafer on Wafer) method. In this way, when multiple substrates are stacked, the stack may be formed by any method. Furthermore, the connection between the layers may be any method, such as vias, microbumps, micropads, etc. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic block diagram of a solid-state imaging device according to an embodiment. [Diagram 2] 5 is a flowchart showing a process of a solid-state imaging device according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a storage state of digital data in a serial interface according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating a storage state of digital data in a serial interface according to an embodiment. [Diagram 5] FIG. 2 is a diagram illustrating a storage state of digital data in a serial interface according to an embodiment. [Figure 6] FIG. 2 is a diagram illustrating a storage state of digital data in a serial interface according to an embodiment. [Figure 7] FIG. 2 is a diagram showing a substrate configuration of a solid-state imaging device according to an embodiment. [Figure 8] FIG. 2 is a diagram showing a substrate configuration of a solid-state imaging device according to an embodiment. [Figure 9] FIG. 2 is a diagram showing a substrate configuration of a solid-state imaging device according to an embodiment. [Figure 10] FIG. 2 is a diagram showing a substrate configuration of a solid-state imaging device according to an embodiment. [Figure 11] FIG. 2 is a diagram showing a substrate configuration of a solid-state imaging device according to an embodiment. [Figure 12] FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system. [Figure 13] FIG. 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit. [Figure 14] FIG. 1 is a diagram showing an example of a schematic configuration of an endoscopic surgery system. [Figure 15] FIG. 2 is a block diagram showing an example of the functional configuration of a camera head and a CCU. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, a description will be given of a solid-state imaging device according to some embodiments with reference to the drawings. Note that although the drawings and description do not refer to power supply voltages Vss, Vdd, etc., it is assumed that the power supply voltages are appropriately applied to the respective circuit elements, etc.

[0031] (Solid-state imaging device) 1 is a block diagram illustrating a solid-state imaging device 1 according to the present embodiment. The solid-state imaging device 1 includes an optical system 100, a pixel array 102, an analog circuit 104, an encoding unit 106, a storage unit 108, a decoding unit 110, a determination unit 112, and a signal processing unit 114.

[0032] The optical system 100 is provided, for example, above a pixel array 102 including pixels (light receiving elements), and is a system that corrects the optical path, aberration, and the like so that the pixel array 102 can sense light. The optical system 100 focuses light received from the outside on the pixels, and the like. For example, the optical system 100 includes a lens (including a virtual lens, etc.), and is installed so that the light is appropriately received by the pixel array 102.

[0033] The pixel array 102 is, for example, a two-dimensional array of pixels each including a PD. The pixel array 102 receives the intensity of light received through the optical system 100 at each pixel, converts the received light into an analog signal by photoelectric conversion, and outputs the analog signal.

[0034] The analog circuit 104 is a circuit that processes analog signals output from each pixel of the pixel array 102. For example, the analog circuit 104 may include a conversion circuit such as an ADC (Analog to Digital Converter) that converts an analog signal into digital data. In addition, the analog circuit 104 may include a DAC (Digital to Analog Converter) that outputs a signal required for AD conversion, a comparator that compares the voltage of the output from the DAC with that of the analog signal, a counter that counts the output from the comparator, and an amplifier that amplifies the output from the counter. As described above, the analog circuit 104 converts an analog signal into digital data in a conversion circuit, for example, and outputs the digital data.

[0035] The encoding unit 106 generates code bits for the digital data (image data) output from the conversion circuit of the analog circuit 104. The code bits may include an error detection code, such as a parity code or a checksum. The code bits may also include an error correction code, such as a Hamming code or a cyclic code. Specific examples of the code bits are not limited to those mentioned above, and the code bits may include an error detection code or an error correction code encoded by other methods. The encoding unit 106 performs these encoding operations on the digital data to generate code bits.

[0036] The storage unit 108 includes a memory for saving various data. This memory is provided, for example, on one of the boards provided in the solid-state imaging device 1. The digital data to which the coding bits are added by the coding unit 106 is temporarily stored in the storage unit 108, and is output after being subjected to appropriate signal processing.

[0037] The decoding unit 110 obtains the digital data to which the code bits have been added from the storage unit 108 and decodes the digital data. For example, the digital data output by the encoding unit 106 is stored in the storage unit 108, and is read and decoded by the decoding unit 110 based on the timing of executing signal processing. The decoding unit 110 performs error detection on the digital data based on the error detection code or error correction code. The decoding unit 110 decodes the code data of the digital data and outputs data related to the result. The data obtained from the storage unit 108 may also be output at the same timing.

[0038] The determination unit 112 determines whether or not an error has occurred during writing or reading of the digital data to or from the storage unit 108 based on the error detection result output from the decoding unit 110. Furthermore, if the code bit is an error correcting code, the determination unit 112 determines whether or not the error is correctable. The determination unit 112 outputs these determination results. If digital data is output from the decoding unit 110, the determination unit 112 may also output this digital data.

[0039] The signal processing unit 114 is a circuit that performs various processes on the digital data converted by the analog circuit 104. For example, the signal processing unit 114 converts the acquired digital data into an image format for output, or into data to be displayed on a display unit connected to the image sensor. The signal processing unit 114 may also perform various image processes such as statistical processing, neural network processing, filtering, etc., which perform clustering, classification, object detection, motion detection, etc.

[0040] The encoding unit 106, the storage unit 108, the decoding unit 110, the decision unit 112, and the signal processing unit 114 may be formed, for example, as a digital circuit having elements that perform various logical operations.

[0041] In Fig. 1, the data flow is shown as an example of a solid arrow, but is not limited thereto. For example, the signal processing unit 114 may obtain a result from the determination unit 112 and exchange data with the storage unit 108. That is, the data processed by the signal processing unit 114 may be stored again in the storage unit 108, or data may be obtained from the storage unit 108 at a timing required for signal processing. The decoding unit 110 may transmit error data to the determination unit 112 and transmit data related to the digital data directly to the signal processing unit 114. The storage unit 108 may also include a volatile cache memory or the like, and in this case, the cache memory or the like may be used to transmit and receive data at higher speeds with other elements or circuits as necessary.

[0042] Furthermore, although not shown, the solid-state imaging device 1 is equipped with an input / output interface for inputting and outputting from the outside, a selector for selecting a signal to be output to the outside via the input / output interface, and other circuits for appropriate imaging.

[0043] 2 is a flowchart showing the flow of processing of the solid-state imaging device 1 according to this embodiment with a focus on data. Note that this flowchart assumes that imaging has already started, and does not include operations such as starting imaging.

[0044] First, the pixel array 102 receives light from the outside via the optical system 100 (S100). Next, the pixel array 102 converts the received light into an analog signal by photoelectric conversion and outputs the analog signal (S102).

[0045] Next, the analog circuit 104 including a conversion circuit converts the analog signal output from the pixel array 102 into digital data (S104). In addition to the conversion, the analog circuit 104 may also perform other processes necessary for analog signal processing as appropriate.

[0046] Next, the encoding unit 106 encodes the digital data output from the analog circuit 104 (S106). Note that encoding is a concept that includes generating encoded bits and adding them to data. As another example, it may include obtaining error detection and error correction codes from the digital data itself and converting it into data that is easy to store in a memory.

[0047] Next, the encoding unit 106 stores the encoded digital data in the storage unit 108 (S108). The stored encoded digital data is read in accordance with the timing when signal processing is required. For example, when processing of signals acquired in the pixel array 102 is performed for each row of pixels, the data of that row is stored in the storage unit 108 until the row to be processed is reached, and is read out in accordance with the timing for processing the data of that row. Note that the unit of processing is not limited to each row, and processing may be performed in other units. In this case, the data is read out in accordance with the processing timing, similar to the case of processing each row.

[0048] Next, as described above, it is determined whether it is time for signal processing (S110). For example, when signal processing is performed for each row of pixels (S110: YES), it is determined that it is time for signal processing of the data of the row, and the decoding unit 110 reads and acquires the data of the row from the storage unit 108 (S112). The digital data to be read is encoded data, and for example, the data output by the analog circuit 104 and the encoded bits are read together. If it is not yet time for signal processing of the row (S110: NO), the standby state continues.

[0049] Next, the decoding unit 110 decodes the code bits of the read digital data (S114). If the digital data itself is encoded, the decoding unit 110 decodes the entire digital data. The decoding unit 110 outputs error information based on the decoded bits. If the digital data itself is encoded, the digital data may be transmitted together with the error information.

[0050] In the above, the error information may be, for example, 1-bit data indicating whether an error has not occurred or whether an error has occurred when an error detection code is used. As another example, the error information may be 1-bit data indicating whether an error has not occurred or whether the error is a correctable error, or an uncorrectable error when an error correction code is used. Also, the error information is not limited to 1 bit, and may be, for example, 2 bits indicating four states in the error correction code: no error has occurred, an error has occurred, an error is correctable, or an error is uncorrectable.

[0051] Next, the determination unit 112 determines whether an error has been detected based on the error information output by the decoding unit 110 (S116). If the code is an error correcting code, it may determine whether the error can be corrected. Then, these determination results are output. The determination unit 112 may output, as the determination result, both the error information for determining whether an error has occurred and the digital data in which the error has occurred. The determination unit 112 may output, as the determination result, both the error information for determining whether the error can be corrected and the digital data in which the error has occurred.

[0052] If the determination unit 112 determines that there is no error or that correction is possible from the error correction code (S116: YES), the signal processing unit 114 executes processing based on the code (S118). For example, if it is determined that no error was detected in writing or reading from the memory, the signal processing unit 114 executes necessary processing on the digital data acquired from the storage unit 108 (or the decoding unit 110 or the determination unit 112) (S122). If the determination unit 112 determines that correction is possible, the error is corrected in the acquired digital data based on the correction code (S118), and then other signal processing is executed (S122).

[0053] If an error correction code is added, the decoding unit 110 may execute error correction. Then, the signal processing unit 114 may receive a notification from the decoding unit 110 via the determination unit 112 that an error has occurred but has been corrected.

[0054] If S116: YES and it is determined that an error has occurred in the memory, the signal processing unit 114 may store the address of the memory. If errors occur frequently in the stored address, the signal processing unit 114 may feed back to the encoding unit 106, for example, so as not to write data to the address. By processing in this way, it is possible to suppress the occurrence of errors in the specific memory.

[0055] If the determination unit 112 determines that an error exists and is not correctable (S116: NO), the signal processing unit 114 determines that an uncorrectable error occurred during memory write or read, or that an error occurred in a path other than memory write or read. Based on this result, the signal processing unit 114 may, for example, correct or interpolate the defective data using an appropriate image processing method. Then, appropriate signal processing is performed (S122), and the processing for the data is terminated.

[0056] If S116: NO, for example, the storage area of ​​the pixel value in the memory unit 108 may not be fixed, thereby detecting a defect in a pixel in the pixel array 102. That is, if many errors occur in the output signal from the same pixel, there is a possibility that some defect occurs in the pixel. In such a case, for example, the signal processing unit 114 may grasp the position of the pixel and appropriately perform signal processing such as correction from surrounding pixels.

[0057] As another example, if a pixel is defective, the pixel value may be determined to be incorrect even if S116: YES (no error) is returned. In such a case, it is possible to detect the pixel defect separately from the occurrence of an error in the memory.

[0058] In this way, a simple configuration may be used to detect whether an error has occurred randomly due to bit inversion in memory caused by neutron radiation, or whether an error has occurred constantly in memory or pixels.

[0059] As described above, according to this embodiment, when performing signal processing on a signal from an image sensor, even if an uncorrectable error occurs in a memory, the signal processing unit 114 can appropriately perform processing to reduce the error, thereby suppressing deterioration of image quality. For example, as described above, by notifying the signal processing unit 114 of pixel data corrupted by an uncorrectable error, it is possible to perform processing to reduce image quality deterioration, such as correction processing from surrounding pixel values.

[0060] In addition, the signal processing unit is capable of dynamically detecting defective pixels on the sensor, and by obtaining the position of the defective pixels as described above, it is possible to avoid erroneous detection of errors that would degrade image quality, or corrections that would cause image quality degradation.

[0061] In addition, even when improving data errors in a memory system alone, the signal processing unit can process the data without increasing the number of ECC correctable bits or making the memory macro redundant, and the circuit scale can be reduced. Furthermore, even when a memory with a high error rate is incorporated into an imaging system such as a camera, the effects of image quality degradation and increased circuit scale can be reduced, leading to an expansion of the options for memories that can be installed in a camera, without being limited to ECC memories, for example.

[0062] (interface) Hereinafter, the embedding position of the code bit will be described with some examples. Data transmission from the determination unit 112 etc. to the signal processing unit 114 is performed, for example, in step S118 in Fig. 2 using the following interface. Note that these are merely some examples, and the aspects of the present disclosure are not limited to these aspects.

[0063] 3 is a schematic diagram showing the storage state of each frame in a serial interface according to an embodiment. Digital data is stored in the interface for each frame and transferred. As shown in FIG. 3, the digital data includes, for example, a blanking area or a dummy pixel area between frames.

[0064] The decoding unit 110 or the determination unit 112 may store the error information in this blanking area or the like, and transfer it to the signal processing unit 114. The signal processing unit 114 refers to the error information in the acquired blanking area or the like, and performs processing such as correction on the data in the frame if processing such as correction is necessary for the pixel data. The error information may be stored in this blanking area, for example, one frame's worth of error information. The error information of a certain frame data may be stored in the blanking area or the like immediately before or immediately after the frame data. Furthermore, the error information may be stored in a blanking area a predetermined number away, rather than immediately before or immediately after, based on the processing timing.

[0065] In this way, error information may be added to the data to be transferred by using an area provided between frames in the interface, and the data may be transmitted to the signal processing unit 114 .

[0066] Fig. 4 shows an example of data storage in a serial interface according to an embodiment. When storing in the blanking area or the like, each area may be provided with a control code in the physical layer, a packet header, and a packet footer, as shown in Fig. 4. Even with this structure, it is possible to store error information in the interface in the same way.

[0067] Fig. 5 is a diagram showing a typical storage state of data in a frame in a serial interface according to an embodiment. The interface is provided with a control code, a packet header, and a packet footer, as in Fig. 4. For example, error information may be incorporated in the packet header portion.

[0068] The diagram at the bottom of Fig. 5 shows one row of data extracted from the frame data. For example, a packet is configured in units of one row like this. That is, the data for each row of pixels in the pixel array 102 may be a packet unit. Also, the packet unit is not limited to this, and for example, data for any number of pixels may be a packet.

[0069] As another example of FIG. 5, the error information may be provided in the packet footer.

[0070] In this way, for example, error information may be added to each packet (one row's worth of pixel value data) of the pixel array 102 in the data to be transferred.

[0071] 6 is yet another example showing a schematic diagram of a data storage state in a serial interface according to an embodiment. For example, in many interfaces, data is managed in units of Bytes (8 bits). Therefore, it is desirable to store error information in units of Bytes.

[0072] Therefore, if digital data relating to pixels is expressed, for example, by 1 Byte of encoded data for one pixel, error information for eight pixels may be stored together in one Byte, as shown in Fig. 6. For example, encoded pixel data including codes for error correction and the like is stored in one Byte of data (such as DATA0).

[0073] The bottom row shows the error information bit string expanded horizontally. For example, the error information byte stores E_INFO0, which is the error information bit for DATA0, E_INFO1, which is the error information bit for DATA1, ..., E_INFO7, which is the error information bit for DATA7.

[0074] In this way, error information may be added to each pixel value based on one or more pixels in the pixel array 102. Also, if the error information is multiple bits, the number of bytes in which the error information is stored may be increased. As another example, if the error information is represented by 2 bits, one byte of error information may be added to four bytes of pixel data.

[0075] As described above, the interface for transmitting and receiving data can be freely selected. Data may be transmitted and received to and from the signal processing unit 114 via the interface described above. Furthermore, the signal processing unit 114 may discard this error information and transmit it to the outside after image correction, defect processing, and the like are completed. In this way, by storing error correction codes and the like, it becomes possible to appropriately perform signal processing for each line, etc., without increasing the circuit scale of the signal processing unit 114.

[0076] (Chip configuration) Hereinafter, examples of the configuration of the substrate of each configuration will be described. Note that these are merely examples, and the present disclosure is not limited to these embodiments.

[0077] 7 is a diagram showing an example of the configuration of a substrate (chip) of a solid-state imaging device 1 according to an embodiment. The solid-state imaging device 1 includes a first substrate 200, a second substrate 202, and a third substrate 204.

[0078] The first substrate 200 is provided as an independent substrate 20. The first substrate 200 includes an optical system 100 and a pixel array 102. The first substrate 200 outputs an analog signal output by the pixel array 102 to the second substrate 202. Any method may be used for this output. Here, the pixel array 102 may specifically mean only the PD light receiving unit constituting the pixel. In other words, the elements of the pixel other than the PD light receiving unit may be provided on the second substrate 202, etc., and the PD light receiving unit may be provided on the first substrate 200.

[0079] The second board 202 is provided as an independent board 22. The second board 202 includes an analog circuit 104, an encoding unit 106, a storage unit 108, a decoding unit 110, and a determination unit 112. The second board 202 acquires the analog signal output from the first board 200, executes the above-mentioned processing on the analog signal from the analog circuit 104 to the determination unit 112, and outputs image data and error information to the third board 204. This output is executed, for example, based on the above-mentioned interface, and is transmitted by any means.

[0080] The third board 204 is provided as an independent board 24. The third board 204 includes a signal processing unit 114. The third board 204 acquires the image data and error information output from the second board 202, executes the above-mentioned processing, and executes necessary processing such as outputting the data to the outside or storing the data in a storage unit 108 or the like.

[0081] In this manner, the solid-state imaging device 1 may include the first substrate 200, the second substrate 202, and the third substrate 204 as independent substrates.

[0082] FIG. 8 is a diagram showing an example of the configuration of a substrate of a solid-state imaging device 1 according to an embodiment.

[0083] The first substrate 200 and the second substrate 202 are provided as a laminated substrate 20 .

[0084] The first substrate 200 includes an optical system 100 and a pixel array 102. The first substrate 200 outputs an analog signal output by the pixel array 102 to an analog circuit 104 of the second substrate 202 connected via, for example, a via, a microbump, a micropad, or the like. The pixel array 102 may be configured such that the PD light receiving unit of the pixel array is provided on the first substrate 200, as in FIG. 7 .

[0085] The second board 202 includes an analog circuit 104, an encoding unit 106, a storage unit 108, a decoding unit 110, and a determination unit 112. The second board 202 acquires the analog signal output from the first board 200, executes the above-mentioned processing on the analog signal from the analog circuit 104 to the determination unit 112, and outputs image data and error information to the third board 204. This output is executed, for example, based on the above-mentioned interface, and is transmitted by any means.

[0086] The third substrate 204 is provided as an independent substrate 24. The third substrate 204 includes a signal processing unit 114.

[0087] In this manner, the solid-state imaging device 1 may include the first substrate 200 and the second substrate 202 formed as a laminated semiconductor, and the third substrate 204 that is independent of the laminated substrates.

[0088] FIG. 9 is a diagram showing an example of the configuration of a substrate of a solid-state imaging device 1 according to an embodiment.

[0089] The first substrate 200 is provided as an independent substrate 20. The first substrate 200 includes an optical system 100 and a pixel array 102. The pixel array 102 may have a PD light receiving unit of the pixel array provided on the first substrate 200, similar to FIG.

[0090] The second board 202 is provided as an independent board 22. The second board 202 includes an analog circuit 104, an encoding unit 106, a storage unit 108, a decoding unit 110, a determination unit 112, and a signal processing unit 114. The second board 202 acquires the analog signal output from the first board 200, and executes the above-mentioned processing on the analog signal from the analog circuit 104 to the signal processing unit 114.

[0091] In this manner, the solid-state imaging device 1 may include the first substrate 200 and the second substrate 202 as independent substrates.

[0092] FIG. 10 is a diagram showing an example of the configuration of a substrate of a solid-state imaging device 1 according to an embodiment.

[0093] A first substrate 200, a second substrate 202 and a third substrate 204 are provided as a stacked substrate 20.

[0094] The first substrate 200 includes an optical system 100 and a pixel array 102. The first substrate 200 outputs an analog signal output by the pixel array 102 to an analog circuit 104 of the second substrate 202 connected via, for example, a via, a microbump, a micropad, or the like. The pixel array 102 may be configured such that the PD light receiving unit of the pixel array is provided on the first substrate 200, as in FIG. 7 .

[0095] The second substrate 202 includes an analog circuit 104, an encoding unit 106, a storage unit 108, a decoding unit 110, and a determination unit 112. The second substrate 202 acquires the analog signal output from the first substrate 200, executes the above-mentioned processing on the analog signal from the analog circuit 104 to the determination unit 112, and outputs image data and error information to the third substrate 204 connected via, for example, vias, microbumps, micropads, etc. This output may be performed based on the above-mentioned interface, for example.

[0096] The third substrate 204 is provided as an independent substrate 24. The third substrate 204 includes a signal processing unit 114.

[0097] In this manner, the solid-state imaging device 1 may include the first substrate 200, the second substrate 202, and the third substrate 204 formed as stacked semiconductors. By forming them in this manner, a large area may be allocated to the signal processing unit 114.

[0098] 11 is a diagram showing an example of the configuration of a substrate of the solid-state imaging device 1 according to an embodiment. As in FIG. 10, the substrate 20 is formed by stacking three layers.

[0099] The configuration of the first substrate 200 is the same as that in FIG.

[0100] The second substrate 202 is formed between the first substrate 200 and the third substrate 204 and includes a memory unit 108 .

[0101] The third substrate 204 includes an analog circuit 104, an encoding unit 106, a decoding unit 110, a determination unit 112, and a signal processing unit 114. Data is written to or read from the memory unit 108 of the second substrate 202 at the timing when the third substrate 204 processes the signal.

[0102] In this manner, the solid-state imaging device 1 may include the first substrate 200, the second substrate 202, and the third substrate 204 formed as stacked semiconductors. By forming them in this manner, a large area may be allocated to the storage unit 108.

[0103] The substrates to be stacked may be laminated together by, for example, cutting out the substrates from a wafer and dividing them into individual pieces, and then laminating them one above the other, a so-called CoC (Chip on Chip) method. Alternatively, one of the substrates may be cut out from a wafer and divided into individual pieces, and then the divided substrate may be bonded to the previous substrate, a so-called CoW (Chip on Wafer) method. Alternatively, the substrates may be bonded together in the wafer state, a so-called WoW (Wafer on Wafer) method.

[0104] The substrates may be bonded together using various bonding methods, such as plasma bonding.

[0105] As described above, signals may be transferred between stacked layers using various connection methods, such as vias, microbumps, and micropads.

[0106] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, a robot, etc.

[0107] FIG. 12 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a moving object control system to which the technology according to the present disclosure can be applied.

[0108] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 12, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also, as functional configurations of the integrated control unit 12050, a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053 are illustrated.

[0109] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0110] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as head lamps, back lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves or signals of various switches transmitted from a portable device that replaces a key may be input to the body system control unit 12020. The body system control unit 12020 receives the input of these radio waves or signals and controls the door lock device, power window device, lamps, and the like of the vehicle.

[0111] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture an image outside the vehicle and receives the captured image. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for a person, a car, an obstacle, a sign, or characters on a road surface, based on the received image.

[0112] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received. The imaging unit 12031 can output the electrical signal as an image, or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0113] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing.

[0114] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, the steering mechanism, or the braking device based on the information inside and outside the vehicle acquired by the outside-of-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output a control command to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the inter-vehicle distance, vehicle speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc.

[0115] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0116] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information outside the vehicle acquired by the outside-vehicle information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside-vehicle information detection unit 12030, and perform cooperative control for the purpose of preventing glare, such as switching from high beams to low beams.

[0117] The audio / video output unit 12052 transmits at least one output signal of audio and video to an output device capable of visually or audibly notifying information to passengers in the vehicle or the outside of the vehicle. In the example of Fig. 12, an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as the output device. The display unit 12062 may include at least one of an on-board display and a head-up display, for example.

[0118] FIG. 13 is a diagram showing an example of the installation position of the imaging unit 12031.

[0119] In FIG. 13, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0120] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided at positions such as the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield in the vehicle interior of the vehicle 12100. The imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield in the vehicle interior mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided at the upper part of the windshield in the vehicle interior is mainly used to detect a preceding vehicle, a pedestrian, an obstacle, a traffic light, a traffic sign, a lane, or the like.

[0121] 13 shows an example of the imaging ranges of the imaging units 12101 to 12104. An imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and an imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.

[0122] At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera made up of a plurality of imaging elements, or may be an imaging element having pixels for detecting a phase difference.

[0123] For example, the microcomputer 12051 can extract, as a preceding vehicle, a three-dimensional object that is the closest three-dimensional object on the travel path of the vehicle 12100 and travels at a predetermined speed (for example, 0 km / h or more) in approximately the same direction as the vehicle 12100, by calculating the distance to each three-dimensional object in the imaging ranges 12111 to 12114 and the change over time of this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. Furthermore, the microcomputer 12051 can set a vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving that travels autonomously without relying on the driver's operation.

[0124] For example, the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0125] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured images of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured images of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the captured images of the imaging units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

[0126] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 12031 and the like among the configurations described above. By applying the technology according to the present disclosure to the imaging unit 12031, it is possible to form a circuit that can read out at high speed and suppresses the increase in the signal processing circuit.

[0127] The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be applied to an endoscopic surgery system.

[0128] FIG. 14 is a diagram showing an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure (the present technology) can be applied.

[0129] 14 shows a state in which an operator (doctor) 11131 is performing surgery on a patient 11132 on a patient bed 11133 using an endoscopic surgery system 11000. As shown in the figure, the endoscopic surgery system 11000 is composed of an endoscope 11100, other surgical tools 11110 such as an insufflation tube 11111 and an energy treatment tool 11112, a support arm device 11120 that supports the endoscope 11100, and a cart 11200 on which various devices for endoscopic surgery are mounted.

[0130] The endoscope 11100 is composed of a lens barrel 11101, a region of a predetermined length from the tip of which is inserted into a body cavity of a patient 11132, and a camera head 11102 connected to the base end of the lens barrel 11101. In the illustrated example, the endoscope 11100 is configured as a so-called rigid scope having a rigid lens barrel 11101, but the endoscope 11100 may be configured as a so-called flexible scope having a flexible lens barrel.

[0131] An opening into which an objective lens is fitted is provided at the tip of the lens barrel 11101. A light source device 11203 is connected to the endoscope 11100, and light generated by the light source device 11203 is guided to the tip of the lens barrel by a light guide extending inside the lens barrel 11101, and is irradiated via the objective lens toward an observation target in the body cavity of the patient 11132. The endoscope 11100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0132] An optical system and an image sensor are provided inside the camera head 11102, and reflected light (observation light) from an observation target is collected on the image sensor by the optical system. The observation light is photoelectrically converted by the image sensor to generate an electrical signal corresponding to the observation light, i.e., an image signal corresponding to an observation image. The image signal is transmitted to a camera control unit (CCU) 11201 as RAW data.

[0133] The CCU 11201 is configured with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and performs overall control of the operations of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), for displaying an image based on the image signal.

[0134] Under the control of the CCU 11201, the display device 11202 displays an image based on an image signal that has been subjected to image processing by the CCU 11201.

[0135] The light source device 11203 is composed of a light source such as an LED (light emitting diode), and supplies the endoscope 11100 with irradiation light when photographing an operation site or the like.

[0136] The input device 11204 is an input interface for the endoscopic surgery system 11000. A user can input various information and instructions to the endoscopic surgery system 11000 via the input device 11204. For example, the user inputs an instruction to change the imaging conditions (type of irradiated light, magnification, focal length, etc.) of the endoscope 11100.

[0137] The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 for cauterizing tissue, incising, sealing blood vessels, etc. The insufflation device 11206 sends gas into the body cavity of the patient 11132 via the insufflation tube 11111 to inflate the body cavity for the purpose of securing the field of view of the endoscope 11100 and securing the working space of the surgeon. The recorder 11207 is a device capable of recording various information related to the surgery. The printer 11208 is a device capable of printing various information related to the surgery in various formats such as text, image, or graph.

[0138] The light source device 11203 that supplies irradiation light to the endoscope 11100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these. When the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 11203. In this case, it is also possible to capture images corresponding to each of the RGB colors in a time-division manner by irradiating the observation object with laser light from each of the RGB laser light sources in a time-division manner and controlling the driving of the image sensor of the camera head 11102 in synchronization with the irradiation timing. According to this method, a color image can be obtained without providing a color filter to the image sensor.

[0139] The light source device 11203 may be controlled to change the intensity of the light it outputs at predetermined time intervals. The driving of the image sensor of the camera head 11102 may be controlled in synchronization with the timing of the change in the light intensity to obtain images in a time-division manner, and the images may be synthesized to generate an image with a high dynamic range that is free of so-called blackout and whiteout.

[0140] The light source device 11203 may be configured to supply light of a predetermined wavelength band corresponding to the special light observation. In the special light observation, for example, by utilizing the wavelength dependency of light absorption in body tissue, a narrow band light is irradiated compared to the irradiated light (i.e., white light) during normal observation, and a predetermined tissue such as blood vessels on the mucous membrane surface is photographed with high contrast, so-called narrow band imaging is performed. Alternatively, in the special light observation, a fluorescent observation may be performed in which an image is obtained by fluorescence generated by irradiating an excitation light. In the fluorescent observation, it is possible to irradiate an excitation light to a body tissue and observe the fluorescence from the body tissue (autofluorescence observation), or to locally inject a reagent such as indocyanine green (ICG) into the body tissue and irradiate the body tissue with excitation light corresponding to the fluorescent wavelength of the reagent to obtain a fluorescent image. The light source device 11203 may be configured to supply narrow band light and / or excitation light corresponding to such special light observation.

[0141] FIG. 15 is a block diagram showing an example of the functional configuration of the camera head 11102 and the CCU 11201 shown in FIG.

[0142] The camera head 11102 has a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 has a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are connected to each other by a transmission cable 11400 so as to be able to communicate with each other.

[0143] The lens unit 11401 is an optical system provided at the connection portion with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and enters the lens unit 11401. The lens unit 11401 is configured by combining a plurality of lenses including a zoom lens and a focus lens.

[0144] The imaging unit 11402 may include one imaging element (so-called single-plate type) or multiple imaging elements (so-called multiple-plate type). When the imaging unit 11402 is configured as a multiple-plate type, for example, each imaging element may generate an image signal corresponding to each of RGB, and a color image may be obtained by combining the image signals. Alternatively, the imaging unit 11402 may be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to a 3D (dimensional) display. By performing a 3D display, the surgeon 11131 can more accurately grasp the depth of the biological tissue in the surgical site. Note that when the imaging unit 11402 is configured as a multiple-plate type, the lens unit 11401 may also be provided in multiple systems corresponding to each imaging element.

[0145] Furthermore, the imaging unit 11402 does not necessarily have to be provided in the camera head 11102. For example, the imaging unit 11402 may be provided inside the lens barrel 11101, immediately behind the objective lens.

[0146] The driving unit 11403 is configured by an actuator, and moves the zoom lens and focus lens of the lens unit 11401 by a predetermined distance along the optical axis under the control of the camera head control unit 11405. This allows the magnification and focus of the image captured by the imaging unit 11402 to be appropriately adjusted.

[0147] The communication unit 11404 is configured by a communication device for transmitting and receiving various information to and from the CCU 11201. The communication unit 11404 transmits the image signal obtained from the imaging unit 11402 to the CCU 11201 via the transmission cable 11400 as RAW data.

[0148] Furthermore, the communication unit 11404 receives a control signal for controlling the driving of the camera head 11102 from the CCU 11201, and supplies the control signal to the camera head control unit 11405. The control signal includes information on the imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value at the time of capturing the image, and / or information specifying the magnification and focus of the captured image.

[0149] The image capturing conditions such as the frame rate, exposure value, magnification, and focus may be appropriately specified by a user, or may be automatically set by the control unit 11413 of the CCU 11201 based on an acquired image signal. In the latter case, the endoscope 11100 is equipped with a so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.

[0150] The camera head control unit 11405 controls the driving of the camera head 11102 based on a control signal received from the CCU 11201 via the communication unit 11404.

[0151] The communication unit 11411 is configured with a communication device for transmitting and receiving various information to and from the camera head 11102. The communication unit 11411 receives an image signal transmitted from the camera head 11102 via the transmission cable 11400.

[0152] Furthermore, the communication unit 11411 transmits, to the camera head 11102, a control signal for controlling the driving of the camera head 11102. The image signal and the control signal can be transmitted by electrical communication, optical communication, or the like.

[0153] The image processing unit 11412 performs various types of image processing on the image signal, which is RAW data sent from the camera head 11102 .

[0154] The control unit 11413 performs various controls related to imaging of the surgical site etc. by the endoscope 11100 and display of the captured image obtained by imaging the surgical site etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.

[0155] Further, the control unit 11413 causes the display device 11202 to display the captured image showing the surgical site, etc., based on the image signal that has been image-processed by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition techniques. For example, the control unit 11413 can recognize surgical tools such as forceps, specific living body parts, bleeding, mist when the energy treatment tool 11112 is used, etc., by detecting the shape and color of the edge of an object included in the captured image. When the control unit 11413 causes the display device 11202 to display the captured image, it may use the recognition result to superimpose various types of surgery support information on the image of the surgical site. By superimposing and presenting the surgery support information to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can proceed with the surgery reliably.

[0156] The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 is an electrical signal cable for communication of electrical signals, an optical fiber for optical communication, or a composite cable of these.

[0157] Here, in the illustrated example, communication is performed wired using a transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may also be performed wirelessly.

[0158] An example of an endoscopic surgery system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to the imaging unit 11402 of the camera head 11102 and the like among the configurations described above. By applying the technology according to the present disclosure to the imaging unit 11402, it is possible to suppress an increase in the circuit area while enabling high-speed readout in the same manner as described above.

[0159] Although an endoscopic surgery system has been described as an example here, the technology disclosed herein may also be applied to other systems, such as a microsurgery system.

[0160] The above-described embodiment may be modified as follows.

[0161] (1) a pixel array in which pixels that photoelectrically convert received light and output analog signals are arranged in a two-dimensional array; A conversion unit that converts the analog signal output from the pixel into digital data; an encoding unit for generating one or more code bits for the digital data; a storage unit that stores the digital data and the code bit; a decoding unit that decodes the code bits of the digital data stored in the storage unit; a determination unit that determines whether or not an error has occurred in writing or reading the digital data to or from the storage unit based on the decoded code bit; a signal processing unit that processes the digital data read from the storage unit based on an output of the determination unit; A solid-state imaging device comprising:

[0162] (2) The code bits are error correcting codes. A solid-state imaging device according to (1).

[0163] (3) The determination unit further determines whether an error is correctable in writing to or reading from the storage unit. A solid-state imaging device according to (2).

[0164] (4) the signal processing unit, when an error occurs and the determination unit determines that the error is correctable, performs error correction based on the code bit. A solid-state imaging device according to (3).

[0165] (5) the signal processing unit executes detection of defects in the memory unit or pixels when an error occurs and the determination unit determines that the error is uncorrectable. A solid-state imaging device according to (3) or (4).

[0166] (6) the signal processing unit, when an error occurs and the determination unit determines that the error is uncorrectable, corrects the digital data in which the error is detected. A solid-state imaging device according to any one of (3) to (5).

[0167] (7) The determination unit adds error information to the digital data and transmits the digital data to the signal processing unit. A solid-state imaging device according to any one of (3) to (6).

[0168] (8) The error information includes information for determining whether an error has occurred. A solid-state imaging device according to (7).

[0169] (9) the error information comprises information for determining whether the error is correctable; A solid-state imaging device according to (7).

[0170] (10) When acquiring an image for a plurality of frames, the determination unit adds the error information to the digital data between frames of the digital data for the image and transmits the digital data to the signal processing unit. A solid-state imaging device according to any one of (7) to (9).

[0171] (11) the determination unit assigns the error information to each packet constituting the digital data and transmits the packet to the signal processing unit. A solid-state imaging device according to any one of (7) to (10).

[0172] (12) The determination unit assigns the error information to each pixel value based on one or more of the pixels constituting the digital data and transmits the pixel value to the signal processing unit. A solid-state imaging device according to any one of (7) to (11).

[0173] (13) the decoding unit, when detecting a correctable error by decoding the code bits, corrects the error in the digital data. A solid-state imaging device according to any one of (1) to (12).

[0174] (14) a first substrate on which at least the pixel array is disposed; a second substrate on which at least the storage unit is disposed; The solid-state imaging device according to any one of (1) to (13), comprising:

[0175] (15) The second substrate further includes the conversion unit, the encoding unit, the decoding unit, and the determination unit. A solid-state imaging device according to (14).

[0176] (16) The second substrate further includes the signal processing unit. A solid-state imaging device according to (15).

[0177] (17) The first substrate and the second substrate are formed by stacking. A solid-state imaging device according to any one of (14) to (16).

[0178] (18) a third substrate on which at least the signal processing unit is disposed; The solid-state imaging device according to (14) or (15), further comprising:

[0179] (19) The first substrate, the second substrate, and the third substrate are formed by stacking. A solid-state imaging device according to (18).

[0180] (20) At least two substrates are stacked in a CoC (Chip on Chip) manner; A solid-state imaging device according to (17) or (19).

[0181] (twenty one) At least two substrates are stacked using the CoW (Chip on Wafer) method. A solid-state imaging device according to (17) or (19).

[0182] (twenty two) At least two substrates are stacked using the WoW (Wafer on Wafer) method. A solid-state imaging device according to (17) or (19).

[0183] The aspects of the present disclosure are not limited to the above-described embodiments, but include various conceivable modifications, and the effects of the present disclosure are not limited to the above-described contents. The components in each embodiment may be appropriately combined and applied. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and intent of the present disclosure derived from the contents defined in the claims and their equivalents. [Explanation of symbols]

[0184] 1: solid-state imaging device; 100:Optical system, 102: pixel array, 104: Analog circuit, 106: Encoding unit, 108: Memory section, 110: Decryption unit, 112: Judgment Department, 114: signal processing unit, 20, 22, 24: (laminated) substrate, 200: first substrate, 202: second substrate, 204: Third board

Claims

1. a pixel array in which pixels that perform photoelectric conversion on received light and output analog signals are arranged in a two-dimensional array; A conversion unit that converts the analog signal output from the pixel into digital data; an encoding unit for generating one or more code bits for the digital data; a storage unit that stores the digital data and the code bit; a decoding unit that decodes the code bits of the digital data stored in the storage unit; a determination unit that determines whether or not an error has occurred in writing or reading the digital data to or from the storage unit based on the decoded code bit; a signal processing unit that processes the digital data read from the storage unit based on an output of the determination unit; Equipped with The signal processing unit detects a pixel having a defect based on a determination result of the determination unit. Solid-state imaging device.

2. The code bits are error correcting codes. The solid-state imaging device according to claim 1 .

3. The determination unit further determines whether an error is correctable in writing to or reading from the storage unit. The solid-state imaging device according to claim 2 .

4. the signal processing unit, when an error occurs and the determination unit determines that the error is correctable, performs error correction based on the code bit. The solid-state imaging device according to claim 3 .

5. the signal processing unit executes detection of defects in pixels in the memory unit or the pixel array when an error occurs and the determination unit determines that the error is uncorrectable. The solid-state imaging device according to claim 3 .

6. the signal processing unit, when an error occurs and the determination unit determines that the error is uncorrectable, corrects the digital data in which the error is detected. The solid-state imaging device according to claim 3 .

7. The determination unit adds error information to the digital data and transmits the digital data to the signal processing unit. The solid-state imaging device according to claim 3 .

8. The error information includes information for determining whether an error has occurred. The solid-state imaging device according to claim 7 .

9. the error information comprises information for determining whether the error is correctable; The solid-state imaging device according to claim 7 .

10. When acquiring an image for a plurality of frames, the determination unit adds the error information to the digital data between frames of the digital data for the image and transmits the digital data to the signal processing unit. The solid-state imaging device according to claim 7 .

11. the determination unit assigns the error information to each packet constituting the digital data and transmits the packet to the signal processing unit. The solid-state imaging device according to claim 7 .

12. the determination unit assigns the error information to each pixel value based on the one or more pixels constituting the digital data and transmits the pixel value to the signal processing unit. The solid-state imaging device according to claim 7 .

13. the decoding unit, when detecting a correctable error by decoding the code bits, corrects the error in the digital data. The solid-state imaging device according to claim 1 .

14. a first substrate on which at least the pixel array is disposed; a second substrate on which at least the memory unit is disposed; The solid-state imaging device according to claim 1 .

15. The second substrate further includes the conversion unit, the encoding unit, the decoding unit, and the determination unit. The solid-state imaging device according to claim 14.

16. The second substrate further includes the signal processing unit. The solid-state imaging device according to claim 15.

17. The first substrate and the second substrate are formed by stacking them. The solid-state imaging device according to claim 14.

18. a third substrate on which at least the signal processing unit is disposed; The solid-state imaging device according to claim 14 , further comprising:

19. The first substrate, the second substrate, and the third substrate are formed by stacking. The solid-state imaging device according to claim 18.

Images