Information processing device and information processing method

Abstract

The present invention provides an information processing device and an information processing method that can more effectively correct output signals. [Solution] The system includes a correction amount acquisition unit that acquires a correction amount for a signal generated by photoelectric conversion of incident light in each of a plurality of pixels, and a correction unit that corrects the signal based on the correction amount, wherein one exposure period in each of the plurality of pixels is divided into a plurality of first periods, a first output value is generated for each of the plurality of pixels in each of the plurality of first periods, and the signal including a second output value which is the cumulative value of the first output value over the plurality of first periods is input to the correction amount acquisition unit for each pixel, the correction amount acquisition unit calculates a first probability that one of the possible values ​​of the first output value of the first pixel is generated, calculates a second probability that one of the possible values ​​of the first output value of the second pixel is generated, and calculates the correction amount based on the first probability and the second probability.

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus and an information processing method. [Background technology]

[0002] Some imaging devices, such as digital cameras, have multiple image sensors arranged in a grid, each converting incident light into electrical signals through photoelectric conversion. In imaging devices containing multiple image sensors, it is possible that some of the image sensors may output abnormally high signals.

[0003] Patent Document 1 proposes a method for correcting cluster-like defects that span multiple pixels, including a pixel on an image sensor that outputs such an abnormally high signal. Patent Document 2 proposes a photoelectric converter capable of performing various calculations on the counting result of a detection signal in a signal processing circuit that generates an output signal. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-118661 [Patent Document 2] Japanese Patent Publication No. 2021-044636 [Overview of the project] [Problems that the invention aims to solve]

[0005] There is room for further improvement in accuracy in methods for correcting the output signal of a photoelectric converter, such as those described in Patent Documents 1 and 2.

[0006] The present invention aims to provide an information processing device and an information processing method that can more effectively correct output signals. [Means for solving the problem]

[0007] According to one disclosure of this specification, an information processing device is provided, comprising: a correction amount acquisition unit that acquires a correction amount for a signal generated by photoelectric conversion of incident light in each of a plurality of pixels; and a correction unit that corrects the signal based on the correction amount, wherein one exposure period in each of the plurality of pixels is divided into a plurality of first periods, a first output value is generated for each of the plurality of pixels in each of the plurality of first periods, and the signal including a second output value which is the cumulative value of the first output value over the plurality of first periods is input to the correction amount acquisition unit for each pixel, the correction amount acquisition unit calculates a first probability that one of the possible values ​​of the first output value is generated in each of the plurality of first periods based on the second output value of the first pixel among the plurality of pixels, the correction amount acquisition unit calculates a second probability that one of the possible values ​​of the first output value is generated in each of the plurality of first periods based on the second output value of the second pixel among the plurality of pixels, and the correction amount acquisition unit calculates the correction amount based on the first probability and the second probability.

[0008] According to one disclosure of this specification, an information processing device is provided, comprising: a correction amount acquisition unit that acquires a correction amount for a signal generated by photoelectric conversion of incident light in each of a plurality of pixels; and a correction unit that corrects the signal based on the correction amount, wherein one exposure period in each of the plurality of pixels is divided into a plurality of first periods, a first output value is generated for each of the plurality of pixels in each of the plurality of first periods, and the signal including a second output value which is the cumulative value of the first output value over the plurality of first periods is input to the correction amount acquisition unit for each pixel, and the correction amount acquisition unit acquires the correction amount by referring to a table based on the second output value of the first pixel among the plurality of pixels and the second output value of the second pixel among the plurality of pixels, wherein the table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value in each of the plurality of first periods is generated in the first pixel, and a second probability that one of the possible values ​​of the first output value in each of the plurality of first periods is generated in the second pixel.

[0009] According to one disclosure of this specification, an information processing method is provided, comprising the steps of: obtaining a correction amount for a signal generated by photoelectric conversion of incident light in each of a plurality of pixels; and correcting the signal based on the correction amount, wherein one exposure period in each of the plurality of pixels is divided into a plurality of first periods, a first output value is generated for each of the plurality of pixels in each of the plurality of first periods, and a signal including a second output value which is the cumulative value of the first output value over the plurality of first periods is input to each pixel, and in the step of obtaining the correction amount, a first probability is calculated based on the second output value of a first pixel among the plurality of pixels that one of the possible values ​​of the first output value is generated in each of the plurality of first periods, a second probability is calculated based on the second output value of a second pixel among the plurality of pixels that one of the possible values ​​of the first output value is generated in each of the plurality of first periods, and the correction amount is calculated based on the first probability and the second probability.

[0010] According to one disclosure of this specification, an information processing method is provided, comprising the steps of: obtaining a correction amount for a signal generated by photoelectric conversion of incident light in each of a plurality of pixels; and correcting the signal based on the correction amount, wherein one exposure period in each of the plurality of pixels is divided into a plurality of first periods, a first output value is generated for each of the plurality of pixels in each of the plurality of first periods, and a signal including a second output value which is the cumulative value of the first output value over the plurality of first periods is input to each pixel, and in the step of obtaining the correction amount, the correction amount is obtained by referring to a table based on the second output value of a first pixel among the plurality of pixels and the second output value of a second pixel among the plurality of pixels, wherein the table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value in each of the plurality of first periods is generated in the first pixel, and a second probability that one of the possible values ​​of the first output value in each of the plurality of first periods is generated in the second pixel. [Effects of the Invention]

[0011] According to the present invention, an information processing device and an information processing method are provided that can more suitably correct an output signal. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the hardware configuration of the information processing device according to the first embodiment. [Figure 2] This is a schematic diagram showing the overall configuration of the photoelectric conversion device according to the first embodiment. [Figure 3] This is a schematic block diagram showing an example of the configuration of a sensor substrate according to the first embodiment. [Figure 4] This is a schematic block diagram showing an example of the configuration of a circuit board according to the first embodiment. [Figure 5] This is a schematic block diagram showing an example of the configuration of one pixel in the photoelectric conversion unit and pixel signal processing unit according to the first embodiment. [Figure 6] This is a diagram illustrating the operation of an avalanche photodiode according to the first embodiment. [Figure 7] This is a functional block diagram relating to the correction process of the information processing device according to the first embodiment. [Figure 8] This flowchart shows the correction process performed by the information processing device according to the first embodiment. [Figure 9] This diagram illustrates the concept of the emission crosstalk correction process according to the first embodiment. [Figure 10] This diagram illustrates the concept of the emission crosstalk correction process according to the first embodiment. [Figure 11] This figure illustrates the contribution rate of the emission crosstalk correction process according to the first embodiment. [Figure 12] This diagram illustrates the concept of the emission crosstalk correction process according to the second embodiment. [Figure 13] This figure shows an example of a lookup table according to the third embodiment. [Figure 14]This is a block diagram of the equipment according to the fourth embodiment. [Figure 15] This is a block diagram of the equipment according to the fifth embodiment. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described below with reference to the drawings. Elements identical or corresponding to each other across multiple drawings are denoted by the same reference numerals, and their descriptions may be omitted or simplified.

[0014] [First Embodiment] Figure 1 is a block diagram showing the hardware configuration of the information processing device 1 according to this embodiment. The information processing device 1 performs information processing such as correction on pixel data acquired by the photoelectric converter. Figure 1 shows an example where the information processing in the information processing device 1 is performed by a general computer, but the information processing functions of the information processing device 1 may be realized by other devices. For example, the information processing device 1 may be an image processing device specialized in image processing functions, or it may be an image processing unit incorporated into the photoelectric converter.

[0015] The information processing device 1 includes a data input unit 121, a data storage unit 122, a display unit 123, and an input unit 124. The information processing device 1 also includes a CPU (Central Processing Unit) 125, RAM (Ramdom Access Memory) 126, and ROM (Read Only Memory) 127. Furthermore, the information processing device 1 includes a communication unit 128 and an information processing unit 129. These units are interconnected via a bus. Figure 1 shows an example of the configuration of the information processing device 1. Some of the units shown in Figure 1 may be located in external devices, and other devices not shown in Figure 1 may be located within the information processing device 1.

[0016] The data input unit 121 includes a photoelectric converter such as an image sensor. The photoelectric converter includes a plurality of pixel circuits arranged in a plurality of rows and a plurality of columns. Each of the plurality of pixel circuits converts incident light into electrical signals. The electrical signals generated by each of the plurality of pixel circuits are converted into digital signals (pixel values). In this way, the data input unit 121 has the function of generating the pixel values ​​of each of the pixels arranged in a plurality of rows and a plurality of columns as image data and inputting it to the information processing device 1. If the photoelectric converter is located outside the information processing device, the data input unit 121 may be an interface for acquiring image data from the photoelectric converter.

[0017] The data storage unit 122 is a recording medium that holds data used for information processing, such as image data and parameters. The recording medium may be a computer-readable non-volatile recording medium such as a hard disk, SSD (Solid State Drive), or flexible disk. The recording medium may also be an optical disc such as a CD (Compact Disc)-ROM, CD-R (Recordable), DVD (Digital Versatile Disc), or Blu-ray (registered trademark). The recording medium may also be a semiconductor memory such as a memory card, CF (Compact Flash) card, SmartMedia, SD card, Memory Stick, xD-Picture Card, or USB (Universal Serial Bus) memory. The data storage unit 122 may store data other than programs and image data. Alternatively, a portion of the storage capacity of the RAM 126 may be used as the data storage unit 122. Alternatively, an external recording device connected to the information processing device 1 via the communication unit 128 may be used as the data storage unit 122.

[0018] The display unit 123 is a device that displays an image before image processing, an image after image processing, or an operating image such as a graphical user interface. The display unit 123 may be a CRT (Cathode-Ray Tube) display, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. Alternatively, the display unit 123 may be an external display provided outside the information processing device 1 and connected to it by a cable or the like.

[0019] The input unit 124 is a device for the user to input instructions or data. The input unit 124 includes a keyboard, a pointing device, etc. Examples of pointing devices include a mouse, trackball, trackpad, tablet, etc. Alternatively, if the information processing device 1 of this embodiment is applied to a device such as a digital camera or printer, the input unit 124 may be buttons, dials, etc. Furthermore, the input unit 124 may be a software keyboard displayed on the screen by software. In this case, the input unit 124 may be configured so that the user can input characters into the software keyboard by operating buttons, dials, or a pointing device.

[0020] Furthermore, a single device, such as a touchscreen device, may also function as both the display unit 123 and the input unit 124. In this case, information input to the information processing device 1 by a user operating the operation screen displayed on the touchscreen device is treated as input information from the input unit 124.

[0021] Furthermore, the input unit 124 may be configured to receive instructions from the user through gesture recognition processing. In this case, the input unit 124 includes an input device that inputs an image captured by visible light or infrared light, and a recognition device that recognizes the user's gestures from the image and converts them into commands. The data input unit 121 may also perform the function of the input device. The recognition device may be added as a dedicated gesture recognition circuit, or it may be implemented by the CPU 125 executing a gesture recognition program.

[0022] Furthermore, the input unit 124 may be configured to receive user instructions through speech recognition processing. In this case, the input unit 124 includes a microphone device and a recognition device that recognizes user speech from the voice data acquired by the microphone device and converts it into a command. The recognition device may be added as a dedicated speech recognition circuit, or it may be implemented by the CPU 125 executing a speech recognition program.

[0023] The gesture recognition and speech recognition processes described above may be performed by an external device to the information processing device 1. In this case, the information processing device 1 communicates with the external device or a server on the network via the communication unit 128 and transmits image data or audio data to the external device or server. The external device or server is configured to receive the image data or audio data according to a predetermined communication procedure, perform recognition processing, and transmit data indicating the recognition result to the information processing device 1.

[0024] The CPU 125 is a processor that controls and processes information for each part of the information processing device 1. The RAM 126 and ROM 127 provide the CPU 125 with programs, data, work areas, etc., necessary for control and information processing. If a program is stored in the data storage unit 122 or ROM 127, the program is first loaded into the RAM 126 and then executed by the CPU 125. The information processing device 1 may also be configured to receive programs from an external source via the communication unit 128. In that case, the program is either first stored in the data storage unit 122 and then loaded into the RAM 126, or directly loaded from the communication unit 128 into the RAM 126 and then executed by the CPU 125.

[0025] Although Figure 1 shows only one block representing the CPU 125, the number of CPUs 125 is not limited to one. In other words, the information processing device 1 may have multiple CPUs 125.

[0026] The communication unit 128 is an interface for communication between devices. The communication unit 128 may be based on a wired communication method such as a wired network, RS-232C, USB, IEEE1284, IEEE1394, or a telephone line. Alternatively, the communication unit 128 may be based on a wireless communication method such as infrared (IrDA), IEEE802.11a, IEEE802.11b, IEEE802.11g, IEEE802.11n, IEEE802.11ac, or IEEE802.11ax. Alternatively, the communication unit 128 may be based on another wireless communication method such as Bluetooth®, UWB (Ultra Wide Band), a wireless telephone line, or NFC (Near Field Communication). Alternatively, the communication unit 128 may be based on an inter-chip communication method such as I2C (Inter-Integrated Circuit) or SPI (Serial Peripheral Interface).

[0027] Furthermore, the communication unit 128 is not limited to supporting only one communication method, but may support multiple methods. For example, the communication unit 128 may be configured to support two or more of the various communication methods described above. Alternatively, the information processing device 1 may have multiple communication units 128 that support different communication methods. In this embodiment, however, the multiple communication units 128 will be collectively referred to as the communication unit 128.

[0028] The information processing unit 129 is a signal processing circuit including a DSP (Digital Signal Processor), logic circuits, etc. Alternatively, the information processing unit 129 may be a GPU (Graphics Processing Unit). The information processing unit 129 performs arithmetic processing on image data input from the data input unit 121 or image data held in the RAM 126, data storage unit 122, etc. The processing results in the information processing unit 129 may be output to the RAM 126, data storage unit 122, display unit 123, etc., or may be output to an external device of the information processing device 1 via the communication unit 128. Note that if the arithmetic processing load is small (when high processing speed is not required, or when the amount of data to be processed is small), the CPU 125 may also perform the role of the information processing unit 129.

[0029] Although not shown in Figure 1, register circuits may be added as needed. The register circuits hold the operating parameters of the CPU 125 or the information processing unit 129. The values ​​held in the register circuits may be set by the CPU 125 or the information processing unit 129, or by an external device via the communication unit 128.

[0030] If the information processing device 1 is a camera device, the display unit 123 may have a function to display a preview image of the subject and a function to display the captured image. However, these images may be displayed on another device (for example, a smartphone) connected via the communication unit 128. In that case, the display unit 123 may be omitted. Similarly, the information processing device 1 may perform an action in response to a command received by another device connected via the communication unit 128 from a user. In this case, the CPU 125 or the information processing unit 129 may perform the process of identifying the corresponding action from the command.

[0031] Alternatively, if software processing and control are not required, the CPU 125, ROM 127, etc., may be omitted. An example of such a case is when the information processing unit 129 contains logic circuits that implement the necessary processing and control.

[0032] Alternatively, the information processing device 1 may be a stacked sensor in which a substrate on which a photoelectric conversion element is arranged and a substrate on which a signal processing circuit is arranged are stacked. In this case, logic circuits, memory, CPU 125, etc. can be arranged inside the stacked sensor. In such a configuration, the data input unit 121 may be configured to include the photoelectric conversion element and its peripheral circuits. In addition, the data storage unit 122, CPU 125, RAM 126, ROM 127, communication unit 128, and information processing unit 129 may be arranged inside the stacked sensor. In this case, the display unit 123 and input unit 124 can be omitted.

[0033] Next, a specific example of the configuration of a photoelectric conversion device including an avalanche photodiode that can be applied to the data input unit 121 of the information processing device 1 shown in Figure 1 will be described. The configuration example of this embodiment is just one example, and the photoelectric conversion device applicable to the data input unit 121 is not limited to this.

[0034] Figure 2 is a schematic diagram showing the overall configuration of the photoelectric converter 100 according to this embodiment. The photoelectric converter 100 has a sensor substrate 11 (first substrate) and a circuit board 21 (second substrate) stacked on top of each other. The sensor substrate 11 and the circuit board 21 are electrically interconnected. The sensor substrate 11 has a pixel region 12 on which a plurality of pixel circuits 101 are arranged in a plurality of rows and a plurality of columns. The circuit board 21 has a first circuit region 22 on which a plurality of pixel signal processing units 103 are arranged in a plurality of rows and a plurality of columns, and a second circuit region 23 arranged on the outer periphery of the first circuit region 22. The second circuit region 23 may include circuits for controlling the plurality of pixel signal processing units 103. The sensor substrate 11 has a light incident surface that receives incident light and a connection surface that faces the light incident surface. The sensor substrate 11 is connected to the circuit board 21 on the connection surface side. In other words, the photoelectric converter 100 is a so-called back-illuminated type.

[0035] In this specification, "plan view" refers to viewing from a direction perpendicular to the surface opposite to the light incidence surface. Similarly, "cross-section" refers to the surface of the sensor substrate 11 perpendicular to the surface opposite to the light incidence surface. While the light incidence surface may appear rough at a microscopic level, in such cases, the plan view is defined based on the light incidence surface as viewed macroscopically.

[0036] In the following description, the sensor substrate 11 and the circuit board 21 are assumed to be diced chips, but the sensor substrate 11 and the circuit board 21 are not limited to chips. For example, the sensor substrate 11 and the circuit board 21 may be wafers. Furthermore, if the sensor substrate 11 and the circuit board 21 are diced chips, the photoelectric converter 100 may be manufactured by stacking wafers and then dicing them, or by stacking wafers after dicing them.

[0037] Figure 3 is a schematic block diagram showing an example of the arrangement of the sensor substrate 11. Multiple pixel circuits 101 are arranged in multiple rows and multiple columns in the pixel region 12. Each of the multiple pixel circuits 101 has a photoelectric conversion unit 102 on the substrate, which includes an avalanche photodiode (hereinafter referred to as APD) as a photoelectric conversion element.

[0038] In an APD (Automated Precipitator), the conductivity type of the charge used as the signal charge is called the first conductivity type. The first conductivity type refers to a conductivity type in which the majority carriers are charges of the same polarity as the signal charge. Conversely, the conductivity type opposite to the first conductivity type, i.e., a conductivity type in which the majority carriers are charges of a different polarity than the signal charge, is called the second conductivity type. In the APD described below, the anode of the APD is at a fixed potential, and the signal is extracted from the cathode of the APD. Therefore, the semiconductor region of the first conductivity type is the N-type semiconductor region, and the semiconductor region of the second conductivity type is the P-type semiconductor region. Alternatively, the cathode of the APD may be at a fixed potential, and the signal may be extracted from the anode of the APD. In this case, the semiconductor region of the first conductivity type is the P-type semiconductor region, and the semiconductor region of the second conductivity type is the N-type semiconductor region. Furthermore, the following description focuses on the case where one node of the APD is at a fixed potential, but a configuration in which the potentials of both nodes fluctuate is also possible.

[0039] Figure 4 is a schematic block diagram showing an example of the configuration of the circuit board 21. The circuit board 21 has a first circuit region 22 in which multiple pixel signal processing units 103 are arranged in multiple rows and multiple columns.

[0040] Furthermore, the circuit board 21 includes a vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, a pixel output signal line 113, an output circuit 114, and a control signal generation unit 115. The multiple photoelectric conversion units 102 shown in Figure 3 and the multiple pixel signal processing units 103 shown in Figure 4 are electrically connected via connecting wiring provided for each pixel circuit 101.

[0041] The control signal generation unit 115 is a control circuit that generates and supplies control signals to drive the vertical scanning circuit 110, the horizontal scanning circuit 111, and the readout circuit 112. In this way, the control signal generation unit 115 controls the drive timing and other aspects of each component.

[0042] The vertical scanning circuit 110 supplies control signals to each of the multiple pixel signal processing units 103 based on the control signals supplied from the control signal generation unit 115. The vertical scanning circuit 110 supplies control signals to each pixel signal processing unit 103 row by row via drive lines provided for each row of the first circuit region 22. As will be described later, there may be multiple drive lines for each row. Logic circuits such as shift registers and address decoders may be used in the vertical scanning circuit 110. This allows the vertical scanning circuit 110 to select the row from which the pixel signal processing unit 103 will output a signal.

[0043] The signal output from the photoelectric conversion unit 102 of the pixel circuit 101 is processed by the pixel signal processing unit 103. The pixel signal processing unit 103 acquires and stores a digital signal having multiple bits by counting the number of pulses output from the APD included in the photoelectric conversion unit 102.

[0044] The horizontal scanning circuit 111 supplies control signals to the readout circuit 112 based on control signals supplied from the control signal generation unit 115. The pixel signal processing unit 103 is connected to the readout circuit 112 via pixel output signal lines 113, which are provided for each column of the first circuit region 22. The pixel output signal line 113 of one column is shared by multiple pixel signal processing units 103 of the corresponding column. The pixel output signal line 113 includes multiple wires and has at least the function of outputting a digital signal from each pixel signal processing unit 103 to the readout circuit 112, and the function of supplying a control signal to the pixel signal processing unit 103 for selecting the column from which to output a signal. The readout circuit 112 outputs a signal to an external storage unit or signal processing unit of the photoelectric converter 100 via the output circuit 114 based on the control signals supplied from the control signal generation unit 115.

[0045] The photoelectric conversion units 102 in the pixel region 12 may be arranged in a one-dimensional manner. Furthermore, the pixel signal processing unit 103 does not necessarily have to be provided for every pixel circuit 101. For example, one pixel signal processing unit 103 may be shared by multiple pixel circuits 101. In this case, the pixel signal processing unit 103 provides signal processing functionality to each pixel circuit 101 by sequentially processing the signals output from each photoelectric conversion unit 102.

[0046] As shown in Figures 3 and 4, a first circuit region 22, in which multiple pixel signal processing units 103 are arranged, is located in the region that overlaps with the pixel region 12 in a plan view. A vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generation unit 115 are arranged so as to overlap between the edge of the sensor substrate 11 and the edge of the pixel region 12 in a plan view. In other words, the sensor substrate 11 has a pixel region 12 and a non-pixel region arranged around the pixel region 12. A second circuit region 23 is located in the circuit substrate 21, in which the vertical scanning circuit 110, a horizontal scanning circuit 111, a readout circuit 112, an output circuit 114, and a control signal generation unit 115 are arranged in the region that overlaps with the non-pixel region in a plan view.

[0047] Note that the arrangement of the pixel output signal lines 113, the readout circuit 112, and the output circuit 114 is not limited to those shown in Figure 4. For example, the pixel output signal lines 113 may be arranged to extend in the row direction and be shared by multiple pixel signal processing units 103 in the corresponding row. The readout circuit 112 may be arranged so that the pixel output signal lines 113 of each row are connected.

[0048] Figure 5 is a schematic block diagram showing an example of the configuration of one pixel of the photoelectric conversion unit 102 and the pixel signal processing unit 103 according to this embodiment. Figure 5 schematically shows a more specific configuration example, including the connection relationship between the photoelectric conversion unit 102 arranged on the sensor substrate 11 and the pixel signal processing unit 103 arranged on the circuit board 21. In Figure 5, the drive lines between the vertical scanning circuit 110 and the pixel signal processing unit 103 in Figure 4 are shown as drive lines 213 and 214.

[0049] The photoelectric conversion unit 102 has an APD 201. The pixel signal processing unit 103 has a quench element 202, a waveform shaping unit 210, a counter circuit 211, and a selection circuit 212. Note that the pixel signal processing unit 103 only needs to have at least one of the waveform shaping unit 210, the counter circuit 211, and the selection circuit 212.

[0050] The APD201 generates charge pairs corresponding to incident light through photoelectric conversion. A voltage VL (first voltage) is supplied to the anode of the APD201. The cathode of the APD201 is connected to the first terminal of the quench element 202 and the input terminal of the waveform shaping unit 210. A voltage VH (second voltage), which is higher than the voltage VL supplied to the anode, is supplied to the cathode of the APD201. As a result, a reverse bias voltage is supplied to the anode and cathode of the APD201, causing the APD201 to perform avalanche multiplication. When a charge is generated by incident light in the APD201 with the reverse bias voltage supplied, this charge undergoes avalanche multiplication, generating an avalanche current.

[0051] When a reverse bias voltage is supplied to the APD201, there are two operating modes: Geiger mode and linear mode. Geiger mode is a mode in which the anode and cathode potential difference is greater than the breakdown voltage, while linear mode is a mode in which the anode and cathode potential difference is near or below the breakdown voltage.

[0052] An APD operating in Geiger mode is called a SPAD (Single Photon Avalanche Diode). In this case, for example, the voltage VL (first voltage) is -30V and the voltage VH (second voltage) is 1V. The APD201 may operate in linear mode or Geiger mode. In the case of a SPAD, the potential difference is larger compared to a linear-mode APD, and the avalanche multiplication effect is more pronounced, so it is preferable to use a SPAD.

[0053] The quench element 202 functions as a load circuit (quench circuit) during signal multiplication by avalanche multiplication. The quench element 202 suppresses the voltage supplied to the APD201, thereby suppressing avalanche multiplication (quench operation). The quench element 202 also restores the voltage supplied to the APD201 to voltage VH by flowing a current corresponding to the voltage drop caused by the quench operation (recharge operation). The quench element 202 may be, for example, a resistor.

[0054] The waveform shaping unit 210 shapes the cathode potential change of the APD201 obtained during photon detection and outputs a pulse signal. For example, an inverter circuit can be used as the waveform shaping unit 210. Figure 5 shows an example in which one inverter is used as the waveform shaping unit 210, but the waveform shaping unit 210 may also be a circuit in which multiple inverters are connected in series, or it may be any other circuit that has a waveform shaping effect.

[0055] The counter circuit 211 counts the pulse signals output from the waveform shaping unit 210 and holds a digital signal indicating the count value. When a control signal is supplied from the vertical scanning circuit 110 via the drive line 213, the counter circuit 211 resets the signal it is holding.

[0056] The selection circuit 212 receives a control signal from the vertical scanning circuit 110 shown in Figure 4 via the drive line 214 shown in Figure 5. In response to this control signal, the selection circuit 212 switches between electrically connecting and disconnecting the counter circuit 211 and the pixel output signal line 113. The selection circuit 212 includes, for example, a buffer circuit for outputting a signal corresponding to the value held in the counter circuit 211.

[0057] In the example shown in Figure 5, the selection circuit 212 switches between the electrical connection and disconnection of the counter circuit 211 and the pixel output signal line 113. However, the method for controlling the signal output to the pixel output signal line 113 is not limited to this. For example, switches such as transistors may be placed at nodes such as between the quench element 202 and the APD 201, or between the photoelectric conversion unit 102 and the pixel signal processing unit 103, to switch between electrical connection and disconnection and control the signal output to the pixel output signal line 113. Alternatively, the signal output to the pixel output signal line 113 may be controlled by changing the value of the voltage VH or voltage VL supplied to the photoelectric conversion unit 102 using a switch such as a transistor.

[0058] Figure 5 shows an example configuration using the counter circuit 211. However, instead of the counter circuit 211, a Time-to-Digital Converter (TDC) and memory may be used to obtain the timing for detecting pulses. In this case, the generation timing of the pulse signal output from the waveform shaping unit 210 is converted into a digital signal by the TDC. In this case, a control signal (reference signal) can be supplied to the TDC from the vertical scanning circuit 110 in Figure 4 via a drive line. The TDC obtains a signal indicating the relative time of the input timing of the pulse with respect to the control signal as a digital signal.

[0059] Figures 6(a), 6(b), and 6(c) illustrate the operation of the APD201 according to this embodiment. Figure 6(a) is a diagram showing the APD201, quench element 202, and waveform shaping unit 210 extracted from Figure 5. As shown in Figure 6(a), the connection node of the input terminals of the APD201, quench element 202, and waveform shaping unit 210 is designated as nodeA. Also, as shown in Figure 6(a), the output side of the waveform shaping unit 210 is designated as nodeB.

[0060] Figure 6(b) is a graph showing the time evolution of the potential of node A in Figure 6(a). Figure 6(c) is a graph showing the time evolution of the potential of node B in Figure 6(a). During the period from time t0 to time t1, a voltage of VH-VL is applied to APD201 in Figure 6(a). When a photon is incident on APD201 at time t1, avalanche multiplication occurs in APD201. This causes an avalanche current to flow in the quench element 202, and the potential of node A drops. Subsequently, the amount of potential drop increases further, and the voltage applied to APD201 gradually decreases. Then, at time t2, avalanche multiplication in APD201 stops. As a result, the voltage level of node A no longer drops below a certain value. Subsequently, during the period from time t2 to time t3, a current flows from the node with voltage VH to node A to compensate for the voltage drop, and at time t3, node A settles back to its original potential.

[0061] In the process described above, the potential of nodeB becomes high during the period when the potential of nodeA is below a certain threshold. In this way, the waveform of the potential drop at nodeA caused by the photon incidence is shaped by the waveform shaping unit 210 and output as a pulse to nodeB.

[0062] Figure 7 is a functional block diagram relating to the correction process of the information processing device according to this embodiment. The information processing device 1 includes an image acquisition unit 131, a reading unit 132, a data holding unit 133, a contribution rate holding unit 134, a correction amount acquisition unit 135, and a correction unit 136.

[0063] The functions of the image acquisition unit 131 and the readout unit 132 are realized, for example, by the data input unit 121 in Figure 1. More specifically, the functions of the image acquisition unit 131 and the readout unit 132 are realized, for example, by the photoelectric conversion unit 102, the pixel signal processing unit 103, the vertical scanning circuit 110, the horizontal scanning circuit 111, the readout circuit 112, and the output circuit 114, as shown in Figures 3 to 5.

[0064] The functions of the data storage unit 133 and the contribution rate storage unit 134 are realized, for example, by the data storage unit 122 or the information processing unit 129 in Figure 1. The functions of the correction amount acquisition unit 135 and the correction unit 136 are realized, for example, by the information processing unit 129 in Figure 1. The functions of the correction amount acquisition unit 135 and the correction unit 136 may also be realized by the CPU 125 executing a correction processing program. This correction processing program may be stored in advance in the data storage unit 122 or the ROM 127, or it may be acquired from another device via the communication unit 128. The operation of each of these units will be described later.

[0065] Figure 8 is a flowchart showing the correction process performed by the information processing device according to this embodiment. The correction process method of this embodiment will be explained in accordance with the flowchart in Figure 8. In the explanation of Figure 8, it is assumed that the image data acquired by the image acquisition unit 131 and the reading unit 132 is already stored in the data holding unit 133.

[0066] In step S11, the correction amount acquisition unit 135 acquires data from the image data held in the data holding unit 133 that includes the pixel to be corrected (the pixel of interest) and at least one pixel other than the pixel of interest. Hereinafter, one of the pixels other than the pixel of interest may be called pixel A (second pixel), and the pixel of interest may be called pixel B (first pixel). The data acquired at this time is a cumulative output value (second output value) obtained by accumulating a digital value (first output value) indicating whether or not avalanche multiplication occurred in one of the multiple periods (first period) in which one exposure period is divided, over N periods. Here, N is an integer of 2 or more.

[0067] In step S12, the correction amount acquisition unit 135 calculates a correction amount based on the cumulative output value of pixel A, the cumulative output value of pixel B, and the contribution rate. Here, the contribution rate is a parameter that indicates the probability that the output value of pixel A is affected by pixel B. In step S13, the correction unit 136 corrects the cumulative output value of pixel B by subtracting the correction amount from the cumulative output value of pixel B.

[0068] Here, the target of correction in this embodiment will be described. In this embodiment, the noise component exerted by pixel A on pixel B is corrected. An example of such a noise component is crosstalk (hereinafter referred to as emission crosstalk) caused by the emission phenomenon when charges generated by avalanche multiplication recombine, when the pixel is an APD. Noise due to emission crosstalk may occur in nearby pixels, for example, starting from a defective pixel that outputs a signal with an abnormally high output value, in which case cluster-like defects appear in the image. However, emission crosstalk can also occur in pixels other than defective pixels. Therefore, the target of correction for noise caused by pixel A may be the output value of a specific set of pixels, or it may be the output value of all pixels. Furthermore, the noise source considered in calculating the correction amount is not limited to a single pixel A, but may be, for example, a group of pixels including multiple pixels in the vicinity of pixel B.

[0069] Figure 9 is a diagram illustrating the concept of the light emission crosstalk correction process according to this embodiment. The correction in this embodiment will be described in more detail with reference to Figure 9. In the following description, an example of a process for correcting noise components due to light emission crosstalk that may occur in the case of Figure 9 will be described. However, Figure 9 and its description are for the purpose of helping to understand the embodiment and do not limit the technical scope of the present invention.

[0070] Figure 9 schematically illustrates a process that determines multiple times whether or not avalanche multiplication occurred in pixels A and B during the exposure period T1. The exposure period T1 is divided into N periods T11 through T1N. In each of the periods T11 through T1N, it is determined whether or not avalanche multiplication occurred in pixels A and B. In each period, if avalanche multiplication occurred, the output value is "1" (second value), and if avalanche multiplication did not occur, the output value is "0" (first value). From pixel A, N output values ​​VA1 through VAN are output from period T11 through T1N. From pixel B, N output values ​​VB1 through VBN are output from period T11 through T1N. In Figure 9, the boxes representing the output values ​​VA1 through VAN and VB1 through VBN contain values ​​of "0" or "1". Thus, in the determination method of this embodiment, the exposure period T1 is divided into N periods T11 to T1N, allowing the avalanche multiplication determination to be performed N times. As a result, the number of avalanche multiplications can be obtained in the range of 0 to N. Note that the output values ​​shown in Figure 9 are values ​​when there is no effect of emission crosstalk.

[0071] Here, the conditional probability that emission crosstalk occurs from pixel A to pixel B when avalanche multiplication occurs at pixel A is given by the contribution rate K. AB Let's assume that the contribution rate K AB This indicates the probability that the output value of pixel B is affected when the output value of pixel A is "1". In this case, the presence or absence of influence due to emission crosstalk is divided into the following cases depending on whether or not there is avalanche multiplication between pixel A and pixel B.

[0072] As shown in Figure 9 during period T11, if avalanche multiplication does not occur in either pixel A or pixel B, then emission crosstalk caused by avalanche multiplication in pixel A does not occur. Similarly, as shown in Figure 9 during period T13, if avalanche multiplication does not occur in pixel A but does occur in pixel B, then emission crosstalk caused by avalanche multiplication in pixel A does not occur. Therefore, in these cases, the effect of emission crosstalk does not occur.

[0073] As shown in Figure 9 during period T12, if avalanche multiplication occurs in pixel A and not in pixel B, the emission crosstalk due to avalanche multiplication in pixel A is due to a contribution of K. AB This occurs with a certain probability. In this situation, if emission crosstalk does not occur, the output value of pixel B is "0", and if emission crosstalk occurs, the output value of pixel B is "1". Therefore, in these cases, the output value of pixel B may change due to the effect of emission crosstalk.

[0074] As shown in Figure 9 during period T14, even when avalanche multiplication occurs in both pixel A and pixel B, the emission crosstalk caused by avalanche multiplication in pixel A has a contribution rate of K. AB This occurs with a certain probability. However, in this case, since the output value of pixel B is originally "1", emission crosstalk does not contribute to the output value of pixel B. Therefore, in this case, there is no effect of emission crosstalk.

[0075] Therefore, of the four cases described above, the only case in which the effects of emission crosstalk occur is when avalanche multiplication occurs in pixel A and not in pixel B (period T12 in Figure 9). Thus, by calculating a correction amount so as to compensate for the effects of emission crosstalk that occur in this case, the effect of emission crosstalk on the output value of pixel B can be appropriately corrected.

[0076] Here, it may be difficult to individually store each output value of "0" or "1" for each period from period T11 to period T1N in the data holding unit 133 due to constraints on memory resources or data transfer bandwidth. Therefore, in this embodiment, the data holding unit 133 is configured not to store each output value from period T11 to period T1N, but to store a cumulative value (cumulative output value) obtained by accumulating the output values over periods T11 to T1N. Specifically, the data holding unit 133 is configured to hold the cumulative output value of pixel A obtained by accumulating output values VA1 to VAN, and the cumulative output value of pixel B obtained by accumulating output values VB1 to VBN. In this case, it is difficult to infer each output value from period T11 to period T1N from the cumulative output value. Therefore, in this embodiment, a correction amount is calculated by calculating an expected value of the number of cases where avalanche multiplication occurs in pixel A and does not occur in pixel B using the probability that avalanche multiplication has occurred. [[ID=...]]

[0077] Let the probability that the output value of pixel A is "1" be "p A1 ", and the probability that the output value of pixel A is "0" be "p A0 ". Also, let the probability that the output value of pixel B is "1" be "p B1 ", and the probability that the output value of pixel B is "0" be "p B0 ". At this time, since the cumulative output value of pixel A is the number of periods in which the output value of pixel A is "1" among N periods, the probability p A1 is obtained by (cumulative output value of pixel A)÷N. Also, since the probability p A0 is the probability of the complementary event of the event that the output value of pixel A is "1", it is (1 - p A1 ). Similarly, the probability p B1 is (cumulative output value of pixel B)÷N, and the probability p B0 is (1 - p B1 ). Therefore, "p A1 ", "p A0 ", "p B1 ", and "p B0 " can be calculated based on the cumulative output value of pixel A, the cumulative output value of pixel B, and the number of periods.

[0078] As described above, the effect of emission crosstalk occurs when avalanche multiplication occurs in pixel A and not in pixel B. Therefore, of these four probabilities, the probability p is used to calculate the noise component due to emission crosstalk. A1 (Second probability) and probability p B0 The (first probability) is used. By calculating the expected value from these probabilities, the noise component due to emission crosstalk can be estimated. More specifically, the correction amount corresponding to the noise component that affects the cumulative output value of pixel B from the emission crosstalk generated from pixel A to pixel B can be obtained by the following equation (1).

number

[0079] Furthermore, it may be possible to obtain VAN from output value VA1 and VBN from output value VB1 individually from pixel A and pixel B. In such cases, it is possible to directly count the number of cases in which avalanche multiplication occurs in pixel A and does not occur in pixel B, without calculating the expected value using probability. In this case, the number of cases and the contribution rate K are as described above. AB The correction amount may be calculated based on this.

[0080] A modified example of the emission crosstalk correction process described with reference to Figure 9 will be described, which is extended to a case where there are multiple pixels that cause emission crosstalk. Figure 10 is a diagram illustrating the concept of the emission crosstalk correction process according to this embodiment. Figure 10 shows an example in the example of Figure 9 where there are two pixels that cause emission crosstalk with respect to pixel B. Similar to Figure 9, Figure 10 and its description are for the purpose of helping to understand the embodiment and do not limit the technical scope of the present invention.

[0081] In the example in Figure 10, it is assumed that emission crosstalk occurs from pixels A and C to pixel B. Pixels A and B are the same as in Figure 9. From pixel C, N output values ​​VC1 to VCN are output from period T11 to period T1N. The conditional probability that emission crosstalk occurs from pixel C to pixel B when avalanche multiplication occurs at pixel C is given by the contribution rate K. CB As explained using Figure 9, the case in which the effect of emission crosstalk occurs is when avalanche multiplication occurs in pixel C and avalanche multiplication does not occur in pixel B. The probability that the output value of pixel C is "1" is "p C1 The correction amount corresponding to the noise component that affects the cumulative output value of pixel B among the emission crosstalk generated from pixel A and pixel C to pixel B can be determined by the inclusion-exclusion principle using the following equation (2).

number

[0082] Furthermore, even when there are three or more pixels that experience emission crosstalk with respect to pixel B, the formula for calculating the correction amount can be similarly derived using the inclusion-exclusion principle. Therefore, this embodiment is applicable regardless of the number of pixels that experience emission crosstalk.

[0083] The contribution rate K AB and contribution rate K CB When the coefficient of contribution is sufficiently small, the formula for calculating the correction amount may be approximated to ignore terms in which the coefficient of contribution is multiplied multiple times. For example, (K) in equation (2) AB ×K CB The following equation (3), which is an approximation obtained by ignoring the term containing ), may be used as the formula for calculating the correction amount.

number

[0084] Figures 11(a) and 11(b) illustrate the contribution rates of the emission crosstalk correction process according to this embodiment. An example of a method for referencing the contribution rates according to this embodiment will be explained with reference to Figures 11(a) and 11(b).

[0085] Figure 11(a) schematically shows a part of the pixel array. The pixel of interest P22, located in the center of Figure 11(a), is the target of correction. In correcting the pixel of interest P22, the range of 3 rows and 3 columns including the pixel of interest P22 is referenced. That is, the pixel of interest P22 and its neighboring pixels P11, P12, P13, P21, P23, P31, P32, and P33 are referenced.

[0086] Figure 11(b) is a table showing the contribution rates in matrix form. The second row and second column of Figure 11(b) correspond to the position of the pixel of interest P22. Here, the contribution rate from a given pixel to the pixel of interest is determined according to the relative position of that pixel to the pixel of interest within the pixel array. The contribution rate values ​​in Figure 11(b) are pre-stored in the contribution rate storage unit 134 in the form of array data or the like. The contribution rate values ​​can be statistically obtained in advance, for example, by investigating the correlation between the pixel position and the probability of emission crosstalk occurring from measured data.

[0087] In other words, the nine numbers arranged in the third row and third column of the table in Figure 11(b) indicate the contribution rate of the pixel at the corresponding position in Figure 11(a). For example, the contribution rate of pixel P12, located in the first row and second column, to the pixel of interest P22 is 0.02. Note that the contribution rate from pixels located outside the third row and third column of the table in Figure 11(b) is assumed to be zero. For example, if pixel P33 were the pixel of interest, the contribution rates from pixels P11, P12, P13, P21, and P31 would be zero.

[0088] The contribution rate setting methods shown in Figures 11(a) and 11(b) are examples only and are not limited to them. For example, the range in which the contribution rate is set centered on the pixel of interest is not limited to rows 3 and columns 3. Also, the value of the contribution rate is not limited to that shown in Figure 11(b). Furthermore, the position of the pixel of interest is not limited to the center of the contribution rate matrix.

[0089] As described above, the information processing device 1 of this embodiment can perform correction based on the cumulative output values ​​of multiple pixels, taking into account the probability that one pixel will become a noise source for other pixels. Therefore, this embodiment provides an information processing device and an information processing method that can more suitably correct the output signal. Furthermore, as described above, the correction method of this embodiment is more suitable for correcting light emission crosstalk that may occur between APDs when the photoelectric conversion elements included in the pixels are APDs.

[0090] [Second Embodiment] This embodiment describes a modified version of the correction process of the first embodiment. In this embodiment, elements common to the first embodiment may be omitted or simplified in their explanation.

[0091] Figure 12 is a diagram illustrating the concept of the light emission crosstalk correction process according to this embodiment. The correction in this embodiment will be described in more detail with reference to Figure 12. Hereafter, an example of a process for correcting noise components due to light emission crosstalk that may occur in the case of Figure 12 will be described. However, Figure 12 and its description are for the purpose of helping to understand the embodiment and do not limit the technical scope of the present invention.

[0092] Figure 12 schematically shows a process for determining multiple times whether or not avalanche multiplication occurred in pixels A and B during the exposure period T1. The exposure period T1 is divided into N periods T11 to T1N. In each of the periods T11 to T1N, it is determined whether or not avalanche multiplication occurred in pixels A and B. In each period, if avalanche multiplication occurred, an output value of "1", "2", or "3" is output depending on the timing of the avalanche multiplication; if avalanche multiplication did not occur, an output value of "0" is output. In other words, in this embodiment, if avalanche multiplication occurs, a variable value that is an integer of 1 or more is output depending on the timing of the avalanche multiplication. In Figure 12, the boxes indicating the output values ​​VA1 to VAN and VB1 to VBN contain the values ​​"0", "1", "2", or "3". Note that the output values ​​shown in Figure 12 are values ​​when there is no effect of emission crosstalk.

[0093] Furthermore, in this embodiment, each of the N periods T11 through T1N is further divided into three subperiods (second periods). Figure 12 illustrates, as an example, three subperiods T111, T112, and T113 included in period T11. Subperiod T111 is the first subperiod within period T11, and subperiod T112 is the subperiod following T111. Subperiod T113 is the period following subperiod T112 and is the last subperiod within period T11. Each of the periods T12 through T1N similarly contains three subperiods.

[0094] If avalanche multiplication occurs in sub-period T111, the output value for period T11 is "3". If avalanche multiplication occurs in sub-period T112, the output value for period T11 is "2". If avalanche multiplication occurs in sub-period T113, the output value for period T11 is "1". Thus, in this embodiment, the output value is adjusted so that the earlier the avalanche multiplication occurs within a single period, the larger the output value is output. If avalanche multiplication does not occur in any of sub-periods T111 to T113, the output value for period T11 is "0". Figure 12 shows an example where avalanche multiplication occurs in sub-period T113. Therefore, the output value VB1 for period T11 is "1".

[0095] Furthermore, if avalanche multiplication occurs in two or more of the subperiods T111, T112, and T113, the output value corresponding to the earliest subperiod in which avalanche multiplication occurred will be output. For example, if avalanche multiplication occurs in both subperiod T112 and subperiod T113, the output value will be "2".

[0096] In this embodiment, if avalanche multiplication occurs in a pixel during a certain sub-period, noise due to emission crosstalk may occur in the output value of another pixel during the same sub-period. That is, in a single period, if avalanche multiplication occurs in pixel A but not in pixel B, emission crosstalk caused by the avalanche multiplication in pixel A may affect the output value of pixel B. Furthermore, even in a single period, if the sub-period in which avalanche multiplication occurs in pixel A is earlier than the sub-period in which avalanche multiplication occurs in pixel B, the effects of emission crosstalk may still occur.

[0097] The probabilities that the output value of pixel A is "0", "1", "2", and "3" are given by "p A0 "p A1 "pA2 "p A3 Let's define the probability that the output value of pixel B is "0", "1", "2", and "3", respectively as "p B0 "p B1 "p B2 "p B3 As described in the first embodiment, these probabilities are used to calculate the noise component due to light crosstalk. Therefore, first, "p A0 "p A1 "p A2 "p A3 "p B0 "p B1 "p B2 "p B3 This explains how to calculate "[...]".

[0098] Assume that the lengths of the three subperiods are the same. In this case, the expected value of the cumulative output when P photons are incident on a certain pixel within N periods is C. exp This can be calculated from equations (4) to (8) below.

number

number

number

number

number

[0099] Equations (4) through (8) establish a correspondence between the cumulative output value and the number of incident photons. The cumulative output value of pixel A over N periods is C. A Therefore, using the inverse transform table based on equations (4) to (8), the expected number of incident photons to pixel A, P, can be calculated. A We can calculate this. And the expected value P A Based on this, equations (9) to (12) below show "p A0 "p A1 "p A2 "p A3 The second probability group can be calculated.

number

number

number

number

[0100] "p B0 "p B1 "p B2 "p B3 The same method can be used to calculate the (first probability group). Therefore, the cumulative output value C of pixel A is calculated in the same way. A and the cumulative output value C of pixel B B Given, "p A0 "pA1 "p A2 "p A3 "p B0 "p B1 "p B2 "p B3 It is possible to calculate the cumulative output value C of pixel B among the emission crosstalk generated from pixel A to pixel B. B The amount of correction corresponding to the noise component that affects the result can be calculated using the following equation (13).

number

[0101] Equation (13) is the sum of the noise components of multiple cases in which the effects of emission crosstalk occur among the possible combinations of output values ​​of pixel A and pixel B. As described above, in this embodiment, it is assumed that if avalanche multiplication occurs in two or more sub-periods, the output value corresponding to the earliest sub-period is output. Therefore, in equation (13), the sigma symbol is set so as not to include the case in which the timing of avalanche multiplication of pixel A is later than the timing of avalanche multiplication of pixel B. However, if a different assumption is adopted, equation (13) may be modified accordingly. Also, if there are multiple pixels that cause emission crosstalk with respect to pixel B, the formula for calculating the correction amount can be extended using the same approach as in the first embodiment.

[0102] In this embodiment, the number of subperiods is set to three as an example, but it is not limited to this. The number of subperiods may be any integer of two or more, in which case equations (4) to (13) above can be modified as appropriate. Also, in this embodiment, the lengths of the multiple subperiods are assumed to be the same, but it is not limited to this. The lengths of the multiple subperiods may be different from each other, in which case equations (4) to (13) above can be modified as appropriate. In the modification of equations (4) to (13), the probability of an avalanche multiplication occurring within a subperiod can be calculated by the following equation (14), using the number of periods N, the number of incident photons P, the length of one period L, and the length of the subperiod from the start time of one period to a certain time l (l ≤ L).

number

[0103] The probability calculation method in equation (14) is applicable to any method of setting subperiods, and therefore can be applied even when the lengths of multiple subperiods are different. For example, the first subperiod may be from the start time of one period to one-eighth of the time, the second subperiod may be from one-eighth of the time of one period to four-eighths of the time, and the last subperiod may be from four-eighths of the time of one period to the end time of one period.

[0104] Another variation is that if an avalanche multiplication occurs in an odd-numbered subperiod among several subperiods, the output value may be "2", and if an avalanche multiplication occurs in an even-numbered subperiod, the output value may be "1". In this way, the same output value may be associated with two of the multiple subperiods.

[0105] As described above, in this embodiment, one period is divided into multiple sub-periods, and different output values ​​can be generated depending on the sub-period in which avalanche multiplication occurs. By using the correction amount calculation formula of this embodiment, it is possible to correct the noise component in the same way as in the first embodiment, even with this configuration. Therefore, according to this embodiment, an information processing device and an information processing method are provided that can more suitably correct the output signal.

[0106] [Third Embodiment] This embodiment describes a modified version of the correction process of the second embodiment. In this embodiment, elements common to the second embodiment may be omitted or simplified in their explanation.

[0107] In the calculation of equation (13) in the second embodiment, the number of terms to be calculated can become enormous depending on conditions such as the number of sub-periods. In this case, the calculation time of the correction amount acquisition unit 135 can become a bottleneck in the processing of the information processing device 1. Therefore, in this embodiment, a method for shortening the calculation time by simplifying the calculation of equation (13) is described.

[0108] A second embodiment will be described as an example. In the second embodiment, the calculation may be performed in the following manner. First, the cumulative output value C of pixel A is calculated. A and the cumulative output value C of pixel B B The cumulative output value C is given. A , C B Using the input, the expected number of incident photons to pixel A, P, is obtained by the inverse transformation table based on equations (4) to (8). A and the expected number of incident photons to pixel B P B We can calculate this. And the expected value P A , P B Using as input, the "p" in equations (9) to (12) A0 "p A1 "p A2 "p A3 "p B0 "p B1 "p B2 "p B3can be calculated. Then, by inputting these probabilities into Equation (13), the correction amount can be calculated. According to the above calculation flow, the cumulative output value C A , C B is input, and the correction amount is uniquely determined.

[0109] Therefore, for all possible combinations of the cumulative output value C A of pixel A and the cumulative output value C B of pixel B, by calculating the correction amount in advance and holding it in the correction amount acquisition unit 135 or the like, the calculation time can be shortened. The correction amount calculated in advance can be held, for example, in the form of a lookup table in which the correction amount L is associated with the cumulative output value C A of pixel A and the cumulative output value C B of pixel B.

[0110] FIG. 13 is a diagram showing an example of the lookup table according to the present embodiment. As shown in FIG. 13, the cumulative output value C A of pixel A and the cumulative output value C B of pixel B are integers of 0 or more. By constructing a lookup table such that one correction amount L ij is associated with each combination of these possible values, the arithmetic processing from Equation (4) to Equation (13) can be replaced with the reference processing of the lookup table. Thereby, the calculation time can be shortened.

[0111] Note that it is not essential for the lookup table to hold the correction amount L A corresponding to all possible combinations of the cumulative output value C B of pixel A and the cumulative output value C ij of pixel B. The lookup table may hold the correction amount L A corresponding to only a part of the possible values of the cumulative output value C B of pixel A and the cumulative output value C ij of pixel B. In this case, for combinations not included in the lookup table, the cumulative output value C A of pixel A and the cumulative output value CB If this is input, the interpolation process will correct the amount L ij You may calculate this. This can reduce the size of the lookup tables that need to be kept.

[0112] Therefore, according to this embodiment, an information processing device and an information processing method are provided that can more favorably correct the output signal, similar to the first and second embodiments. Furthermore, according to this embodiment, the calculation time can be reduced.

[0113] [Fourth Embodiment] The information processing device 1 in the above-described embodiment is applicable to various devices. Examples of such devices include digital still cameras, digital camcorders, camera heads, photocopiers, fax machines, mobile phones, in-vehicle cameras, observation satellites, and surveillance cameras. Figure 14 shows a block diagram of a digital still camera as an example of such a device. Figure 14 shows an example of applying the information processing device 1 shown in Figure 1 to a digital still camera.

[0114] The device 70 shown in Figure 14 includes a barrier 706, a lens 702, an aperture 704, and an imaging device 700 (an example of a photoelectric converter). The device 70 further includes a signal processing unit (processing unit) 708, a timing generation unit 720, an overall control / calculation unit 718 (control device), a memory unit 710 (storage device), a recording medium control I / F unit 716, a recording medium 714, and an external I / F unit 712. At least one of the barrier 706, lens 702, and aperture 704 is an optical device corresponding to the device. The barrier 706 protects the lens 702, and the lens 702 forms an optical image of the subject on the imaging device 700. The aperture 704 varies the amount of light passing through the lens 702. The imaging device 700 converts the optical image formed by the lens 702 into image data (image signal). The signal processing unit 708 performs various corrections, data compression, etc., on the imaging data output from the imaging device 700. The timing generation unit 720 outputs various timing signals to the imaging device 700 and the signal processing unit 708. The overall control / calculation unit 718 controls the entire digital still camera, and the memory unit 710 temporarily stores image data. The recording medium control I / F unit 716 is an interface for recording or reading image data to or from the recording medium 714, which is a removable recording medium such as a semiconductor memory for recording or reading imaging data. The external I / F unit 712 is an interface for communicating with an external computer or the like. Timing signals and the like may be input from outside the device. Furthermore, the device 70 may also include a display device (monitor, electronic viewfinder, etc.) that displays information obtained from the photoelectric converter. The device includes at least a photoelectric converter. Furthermore, the device 70 includes at least one of an optical device, a control device, a processing device, a display device, a storage device, and a mechanical device that operates based on information obtained from the photoelectric converter. The mechanical device is a movable part (for example, a robot arm) that operates in response to signals from the photoelectric converter.

[0115] Each pixel may include multiple photoelectric conversion units (a first photoelectric conversion unit and a second photoelectric conversion unit). The signal processing unit 708 may be configured to process a pixel signal based on the charge generated by the first photoelectric conversion unit and a pixel signal based on the charge generated by the second photoelectric conversion unit to acquire distance information from the imaging device 700 to the subject.

[0116] [Fifth Embodiment] Figures 15(a) and 15(b) are block diagrams of the equipment related to the in-vehicle camera in this embodiment. Figures 15(a) and 15(b) show an example of applying the information processing device 1 shown in Figure 1 to a moving object such as a vehicle. The device 80 includes an imaging device 800 (an example of a photoelectric converter) and a signal processing device (processing device) that processes signals from the imaging device 800. The device 80 includes an image processing unit 801 that performs image processing on a plurality of image data acquired by the imaging device 800, and a parallax calculation unit 802 that calculates parallax (phase difference of parallax images) from a plurality of image data acquired by the device 80. The device 80 also includes a distance measurement unit 803 that calculates the distance to an object based on the calculated parallax, and a collision determination unit 804 that determines whether or not there is a possibility of collision based on the calculated distance. Here, the parallax calculation unit 802 and the distance measurement unit 803 are examples of distance information acquisition means that acquire distance information to an object. That is, distance information is information related to parallax, defocus amount, distance to an object, etc. The collision determination unit 804 may use any of this distance information to determine the possibility of a collision. The distance information acquisition means may be implemented by specially designed hardware or by a software module. It may also be implemented by an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a combination thereof.

[0117] Device 80 is connected to a vehicle information acquisition device 810 and can acquire vehicle information such as vehicle speed, yaw rate, and steering angle. Device 80 is also connected to a control ECU 820, which is a control device that outputs a control signal to generate braking force on the vehicle based on the collision determination result of the collision determination unit 804. Furthermore, device 80 is connected to a warning device 830 that issues a warning to the driver based on the collision determination result of the collision determination unit 804. For example, if the collision determination result of the collision determination unit 804 indicates a high probability of collision, the control ECU 820 performs vehicle control to avoid a collision or mitigate damage by applying the brakes, releasing the accelerator, or suppressing engine output. The warning device 830 warns the user by sounding an alarm, displaying warning information on a screen such as a car navigation system, or vibrating the seatbelt or steering wheel. As described above, device 80 functions as a control means that controls the actions that control the vehicle.

[0118] In this embodiment, the equipment 80 images the area around the vehicle, for example, in front of or behind it. Figure 15(b) shows the equipment when imaging the area in front of the vehicle (imaging range 850). The vehicle information acquisition device 810, acting as an imaging control means, sends instructions to the equipment 80 or imaging device 800 to perform the imaging operation. This configuration allows for further improvement of the accuracy of distance measurement.

[0119] The above example described controlling a vehicle to avoid collisions with other vehicles, but it can also be applied to control systems that automatically follow other vehicles, or control systems that automatically stay within their lane. Furthermore, the equipment is not limited to vehicles such as automobiles, but can be applied to mobile objects (mobile devices) such as ships, aircraft, satellites, industrial robots, and consumer robots. In addition, it can be applied not only to mobile objects, but also to a wide range of devices that utilize object recognition or biometric recognition, such as intelligent transportation systems (ITS) and surveillance systems.

[0120] [Modified Embodiment] The present invention is not limited to the embodiments described above and can be modified in various ways. For example, an example in which a part of the configuration of one embodiment is added to another embodiment, or an example in which a part of the configuration of one embodiment is replaced with a part of the configuration of another embodiment, is also an embodiment of the present invention.

[0121] The disclosures in this specification include the complements of the concepts described herein. That is, if this specification contains a statement such as "A is B" (A=B), the specification shall be deemed to disclose or imply "A is not B" (A≠B) even if a statement such as "A is not B" is omitted. This is because the statement "A is B" presupposes that the case where "A is not B" is being considered.

[0122] The disclosures in this specification include the following configurations or methods: (Composition 1) A correction amount acquisition unit that acquires the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A correction unit that corrects the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to the correction amount acquisition unit for each pixel. The correction amount acquisition unit calculates a first probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, based on the second output value of the first pixel among the plurality of pixels. The correction amount acquisition unit calculates a second probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, based on the second output value of the second pixel among the plurality of pixels. The correction amount acquisition unit calculates the correction amount based on the first probability and the second probability. An information processing device characterized by the following: (Configuration 2) The aforementioned probability 1 is the probability that the first value is generated, The second probability is the probability that a second value different from the first value is generated. The information processing device according to configuration 1, characterized by the above. (Composition 3) The first value is 0. The information processing apparatus according to configuration 2, characterized in that... (Composition 4) The second value mentioned above is an integer greater than or equal to 1. An information processing apparatus according to configuration 2 or 3, characterized by the above. (Composition 5) The correction amount acquisition unit calculates the correction amount based on the product of the first probability and the second probability. An information processing device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The correction amount acquisition unit calculates the correction amount based on the product of the first probability, the second probability, and the number of the multiple first periods in one exposure period. An information processing apparatus according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The correction amount acquisition unit calculates the correction amount based on a contribution rate that includes the probability that the first output value of the first pixel is affected by the second pixel when the first output value of the second pixel is a predetermined value. An information processing device according to any one of configurations 1 to 6. (Composition 8) The correction amount acquisition unit calculates the correction amount based on the product of the first probability, the second probability, the number of the multiple first periods in one exposure period, and the contribution rate. The information processing apparatus according to configuration 7, characterized by the features described above. (Composition 9) The contribution rate has a value corresponding to the relative position of the second pixel to the first pixel in the array of the plurality of pixels. The information processing apparatus according to configuration 7 or 8, characterized by the above. (Composition 10) The first output value is based on the incidence of photons onto the avalanche photodiode. An information processing apparatus according to any one of configurations 1 to 9, characterized by the above. (Composition 11) The first probability is the probability that, in one first period, avalanche multiplication does not occur due to the incidence of photons onto the avalanche photodiode. The information processing apparatus according to configuration 10, characterized by the above. (Composition 12) The second probability is the probability that avalanche multiplication occurs in one first period due to the incidence of photons onto the avalanche photodiode. An information processing apparatus according to configuration 10 or 11, characterized by the above. (Composition 13) The aforementioned correction amount indicates the amount of crosstalk caused by light resulting from the recombination of charges generated by avalanche multiplication in the avalanche photodiode. An information processing apparatus according to any one of configurations 10 to 12, characterized in that (Composition 14) Each of the aforementioned multiple first periods is further divided into multiple second periods, If no photon incidence is detected in any of the aforementioned second periods, the first output value is the first value. If photon incidence is detected in any of the plurality of second periods, the first output value is a variable value corresponding to the second period in which photon incidence was detected. An information processing device according to any one of configurations 1 to 13, characterized by the above. (Composition 15) If photon incidence is detected in two or more of the aforementioned multiple second periods, the first output value is the value corresponding to the earliest second period in which photon incidence was detected. The information processing apparatus according to configuration 14, characterized by the features described herein. (Composition 16) The lengths of two of the aforementioned multiple second periods are different from each other. The information processing apparatus according to configuration 14 or 15, characterized by the above. (Composition 17) When the incidence of a photon is detected in any of the aforementioned second periods, the first output value is larger the earlier the second period in which the incidence of the photon is detected. An information processing device according to any one of the configurations 14 to 16, characterized by the above. (Composition 18) The correction amount acquisition unit generates a first probability group by calculating, based on the second output value of the first pixel, the probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, for each possible value of the first output value. The correction amount acquisition unit generates a second probability group by calculating, based on the second output value of the second pixel, the probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, for each possible value of the first output value. The correction amount acquisition unit calculates the correction amount based on the first probability group and the second probability group. An information processing device according to any one of configurations 1 to 17, characterized by the above. (Composition 19) The correction unit corrects the signal by subtracting the correction amount from the second output value of the first pixel. An information processing apparatus according to any one of configurations 1 to 18, characterized by the above. (Composition 20) A correction amount acquisition unit that acquires the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A correction unit that corrects the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to the correction amount acquisition unit for each pixel. The correction amount acquisition unit acquires the correction amount by referring to a table based on the second output value of the first pixel among the plurality of pixels and the second output value of the second pixel among the plurality of pixels. The table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value is generated at the first pixel in each of the plurality of first periods, and a second probability that one of the possible values ​​of the first output value is generated at the second pixel in each of the plurality of first periods. An information processing device characterized by the following: (Composition 21) The plurality of pixels, An information processing device according to any one of configurations 1 to 20, to which signals output from the plurality of pixels are input, A photoelectric converter including a photoelectric converter. (Composition 22) The photoelectric conversion device described in configuration 21, Optical device corresponding to the aforementioned photoelectric converter, A control device for controlling the aforementioned photoelectric converter, A processing device that processes the signal output from the aforementioned photoelectric converter, A display device that displays information obtained by the aforementioned photoelectric converter. A storage device for storing information obtained by the aforementioned photoelectric converter, and A device characterized by comprising at least one of the following: a mechanical device that operates based on information obtained from the photoelectric converter. (Composition 23) The apparatus according to configuration 22, characterized in that the processing device acquires distance information from the photoelectric converter to the subject. (Method 24) A step of obtaining the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A step of correcting the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to each pixel. In the step of obtaining the correction amount, Based on the second output value of the first pixel among the plurality of pixels, a first probability is calculated that one of the possible values ​​of the first output value is generated in each of the plurality of first periods. Based on the second output value of the second pixel among the plurality of pixels, a second probability is calculated that one of the possible values ​​of the first output value is generated in each of the plurality of first periods. The correction amount is calculated based on the first and second probabilities. An information processing method characterized by the following: (Method 25) A step of obtaining the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A step of correcting the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to each pixel. In the step of obtaining the correction amount, the correction amount is obtained by referring to a table based on the second output value of the first pixel among the plurality of pixels and the second output value of the second pixel among the plurality of pixels. The table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value is generated at the first pixel in each of the plurality of first periods, and a second probability that one of the possible values ​​of the first output value is generated at the second pixel in each of the plurality of first periods. An information processing method characterized by the following: (Composition 26) A program for causing a computer to execute the information processing method described in Method 24 or 25. (Composition 27) A recording medium containing a program for causing a computer to execute the information processing method described in Method 24 or 25.

[0123] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by a process in which one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0124] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features. [Explanation of Symbols]

[0125] 131 Image acquisition unit 132 Reading section 133 Data storage unit 134 Contribution rate holding unit 135 Correction amount acquisition unit 136 Correction section

Claims

1. A correction amount acquisition unit that acquires the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A correction unit that corrects the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to the correction amount acquisition unit for each pixel. The correction amount acquisition unit calculates a first probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, based on the second output value of the first pixel among the plurality of pixels. The correction amount acquisition unit calculates a second probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, based on the second output value of the second pixel among the plurality of pixels. The correction amount acquisition unit calculates the correction amount based on the first probability and the second probability. An information processing device characterized by the following:

2. The aforementioned first probability is the probability that the first value is generated, The second probability is the probability that a second value different from the first value is generated. The information processing apparatus according to feature 1.

3. The first value is 0. The information processing apparatus according to feature 2.

4. The second value is an integer greater than or equal to 1. The information processing apparatus according to feature 2.

5. The correction amount acquisition unit calculates the correction amount based on the product of the first probability and the second probability. The information processing apparatus according to feature 1.

6. The correction amount acquisition unit calculates the correction amount based on the product of the first probability, the second probability, and the number of the plurality of first periods in one exposure period. The information processing apparatus according to feature 1.

7. The correction amount acquisition unit calculates the correction amount based on a contribution rate that includes the probability that the first output value of the first pixel is affected by the second pixel when the first output value of the second pixel is a predetermined value. The information processing apparatus according to feature 1.

8. The correction amount acquisition unit calculates the correction amount based on the product of the first probability, the second probability, the number of the multiple first periods in one exposure period, and the contribution rate. The information processing apparatus according to feature 7.

9. The contribution rate has a value corresponding to the relative position of the second pixel with respect to the first pixel in the array of the plurality of pixels. The information processing apparatus according to feature 7.

10. The first output value is based on the incidence of photons onto the avalanche photodiode. The information processing apparatus according to feature 1.

11. The first probability is the probability that, in one first period, avalanche multiplication based on the incidence of photons onto the avalanche photodiode does not occur. The information processing apparatus according to feature 10.

12. The second probability is the probability that avalanche multiplication occurs in one first period due to the incidence of photons onto the avalanche photodiode. The information processing apparatus according to feature 10.

13. The aforementioned correction amount indicates the amount of crosstalk caused by light resulting from the recombination of charges generated by avalanche multiplication in the avalanche photodiode. The information processing apparatus according to feature 10.

14. Each of the aforementioned multiple first periods is further divided into multiple second periods, If no photon incidence is detected in any of the aforementioned second periods, the first output value is the first value. If photon incidence is detected in any of the plurality of second periods, the first output value is a variable value corresponding to the second period in which photon incidence was detected. The information processing apparatus according to feature 1.

15. If photon incidence is detected in two or more of the aforementioned multiple second periods, the first output value is the value corresponding to the earliest second period in which photon incidence was detected. The information processing apparatus according to feature 14.

16. The lengths of two of the aforementioned plurality of second periods are different from each other. The information processing apparatus according to feature 14.

17. When the incidence of a photon is detected in any of the aforementioned multiple second periods, the first output value is larger the earlier the second period in which the incidence of the photon is detected. The information processing apparatus according to feature 14.

18. The correction amount acquisition unit generates a first probability group by calculating, based on the second output value of the first pixel, the probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, for each possible value of the first output value. The correction amount acquisition unit generates a second probability group by calculating, for each of the possible values ​​of the first output value, the probability that one of the possible values ​​of the first output value will be generated in each of the plurality of first periods, based on the second output value of the second pixel. The correction amount acquisition unit calculates the correction amount based on the first probability group and the second probability group. The information processing apparatus according to feature 1.

19. The correction unit corrects the signal by subtracting the correction amount from the second output value of the first pixel. The information processing apparatus according to feature 1.

20. A correction amount acquisition unit that acquires the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A correction unit that corrects the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to the correction amount acquisition unit for each pixel. The correction amount acquisition unit acquires the correction amount by referring to a table based on the second output value of the first pixel among the plurality of pixels and the second output value of the second pixel among the plurality of pixels. The table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value is generated at the first pixel in each of the plurality of first periods, and a second probability that one of the possible values ​​of the first output value is generated at the second pixel in each of the plurality of first periods. An information processing device characterized by the following:

21. The plurality of pixels, An information processing device according to any one of claims 1 to 20, to which signals output from the plurality of pixels are input, A photoelectric converter including a photoelectric converter.

22. The photoelectric conversion device according to claim 21, Optical device corresponding to the aforementioned photoelectric converter, A control device for controlling the aforementioned photoelectric converter, A processing device that processes the signal output from the aforementioned photoelectric converter, A display device that displays information obtained by the aforementioned photoelectric converter. A storage device for storing information obtained by the aforementioned photoelectric converter, and A device characterized by comprising at least one of the following: a mechanical device that operates based on information obtained from the photoelectric converter.

23. The apparatus according to claim 22, characterized in that the processing device acquires distance information from the photoelectric converter to the subject.

24. A step of obtaining the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A step of correcting the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to each pixel. In the step of obtaining the correction amount, Based on the second output value of the first pixel among the plurality of pixels, a first probability is calculated that one of the possible values ​​of the first output value is generated in each of the plurality of first periods. Based on the second output value of the second pixel among the plurality of pixels, a second probability is calculated that one of the possible values ​​of the first output value is generated in each of the plurality of first periods. The correction amount is calculated based on the first and second probabilities. An information processing method characterized by the following:

25. A step of obtaining the correction amount of the signal generated by photoelectric conversion of incident light at each of multiple pixels, A step of correcting the signal based on the correction amount, It has, Each of the aforementioned plurality of pixels has an exposure period that is divided into a plurality of first periods. In each of the plurality of first periods, a first output value is generated for each of the plurality of pixels. The signal, which includes a second output value that is the cumulative value of the first output value over the plurality of first periods, is input to each pixel. In the step of obtaining the correction amount, the correction amount is obtained by referring to a table based on the second output value of the first pixel among the plurality of pixels and the second output value of the second pixel among the plurality of pixels. The table includes a correction amount calculated in advance based on a first probability that one of the possible values ​​of the first output value is generated at the first pixel in each of the plurality of first periods, and a second probability that one of the possible values ​​of the first output value is generated at the second pixel in each of the plurality of first periods. An information processing method characterized by the following:

26. A program for causing a computer to execute the information processing method described in claim 24 or 25.

27. A recording medium storing a program for causing a computer to execute the information processing method described in claim 24 or 25.

Images