Imaging system and imaging device

Abstract

This imaging device comprises a light source device and an imaging device including a plurality of pixels arranged in a two-dimensional array, each of the plurality of pixels including an analog memory that stores a signal corresponding to the exposure light amount. When the light source device is off, the imaging device exposes pixels in one group of rows of the array, which is either the odd-numbered rows or the even-numbered rows, and stores, in the analog memory of each pixel in the one group of rows, a signal corresponding to the exposure light amount of the pixel. When the light source device is on, the imaging device exposes pixels in the other group of rows, which is either the odd-numbered rows or the even-numbered rows, and stores, in the analog memory of each pixel in the other group of rows, a signal corresponding to the exposure light amount of the pixel. The imaging device then generates a differential image indicating the difference between when the light source device is off and when the light source device is on, on the basis of the signals stored in the analog memory of each pixel in the one group of rows and the signals stored in the analog memory of each pixel in the other group of rows.

Description

Imaging system and imaging device

[0001] The present disclosure relates to an imaging system and an imaging device.

[0002] Various techniques for generating difference images are known.

[0003] JP 2018-42139 A JP 2022-123810 A JP 2020-21987 A

[0004] To generate a difference image, two frames of exposure are required. Depending on the application, the light irradiation on the subject may be switched between the first and second frame exposures. If control takes a long time, the time required to complete the two frame exposures will be longer, and the subject may move during that time, making it impossible to obtain an appropriate difference image.

[0005] One aspect of the present disclosure is to shorten the time required to complete exposure of two frames.

[0006] An imaging system according to one aspect of the present disclosure includes a light source device and an imaging device including a plurality of pixels arranged in a two-dimensional array, each of the plurality of pixels including an analog memory that stores a signal corresponding to the amount of exposure light, and when the light source device is off, the imaging device exposes the pixels of one of the odd and even rows of the array and stores a signal corresponding to the amount of exposure light for each pixel of the one row in the analog memory of each pixel, and when the light source device is on, exposes the pixels of the other of the odd and even rows and stores a signal corresponding to the amount of exposure light for each pixel of the other row in the analog memory of each pixel, and generates a differential image showing the difference between when the light source device is off and when the light source device is on, based on the signal stored in the analog memory of each pixel of the one row and the signal stored in the analog memory of each pixel of the other row.

[0007] An imaging device according to one aspect of the present disclosure is an imaging device used together with a light source device, and includes a plurality of pixels arranged in a two-dimensional array, each including an analog memory that stores a signal corresponding to the amount of exposure light; when the light source device is off, the pixels in one of the odd and even rows of the array are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the one row is stored in the analog memory of each pixel; when the light source device is on, the pixels in the other of the odd and even rows are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the other row is stored in the analog memory of each pixel; and a differential image showing the difference between when the light source device is off and when the light source device is on is generated based on the signal stored in the analog memory of each pixel in the one row and the signal stored in the analog memory of each pixel in the other row.

[0008] 1 is a diagram illustrating an example of a schematic configuration of an imaging system 1 according to an embodiment; FIG. 2 is a diagram illustrating an example of a schematic configuration of an imaging device 2; FIG. 3 is a diagram illustrating an example of a pixel circuit; FIG. 4 is a diagram illustrating an example of a pixel circuit; FIG. 5 is a diagram illustrating an example of a row of a pixel array unit 3; FIG. 6 is a diagram illustrating an example of generation of a difference image 10; FIG. 7 is a diagram illustrating an example of calculation of a difference value D3; FIG. 8 is a diagram illustrating an example of a plurality of difference images 10; FIG. 9 is a timing chart illustrating an example of synchronization control; FIG. 10 is a flowchart illustrating an example of processing (imaging method, image generation method) executed in the imaging system 1; FIG. 11 is a diagram illustrating a first subtraction technique; FIG. 12 is a diagram illustrating a second subtraction technique; FIG. 13 is a diagram illustrating a third subtraction technique; FIG. 14 is a diagram illustrating a third subtraction technique; FIG. 15 is a diagram illustrating a third subtraction technique;

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same elements are designated by the same reference numerals, and redundant description will be omitted.

[0010] The present disclosure will be described in the following order: 0. Introduction 1. Embodiments 2. Conclusion

[0011] 0. Introduction Background subtraction is known as one technique for obtaining a difference image. A difference image is an image in which only the target object is extracted from an image within the imaging range. For example, the signal values ​​of an image containing only the background (background image) are first obtained, and then the signal values ​​of the background image are subtracted from the signal values ​​of an image containing both the background and the target object. A difference image is generated based on the signal values ​​after subtraction. This allows for subsequent image processing to be performed effectively, as it is possible to exclude external disturbances such as light.

[0012] One application of background subtraction is light section measurement, and it is used, for example, to measure the cross-sectional shape of a tunnel. A difference image is generated that shows the difference between when a tunnel is irradiated with laser light and when it is not. The image of the tunnel corresponds to the background image mentioned above, and the pattern of laser light projected onto the tunnel (projection pattern) corresponds to the target of interest mentioned above. The generated difference image shows the two-dimensional shape of the projection pattern, i.e., the cross-sectional shape of the tunnel at that position. Multiple difference images are generated by irradiating (scanning) different positions on the tunnel with laser light throughout the tunnel. These difference images can also be combined to obtain a three-dimensional image of the tunnel.

[0013] To generate a single difference image, simply subtracting the signal values ​​of two frames of images requires a frame memory to hold the signal values ​​of one frame of image. For example, Patent Document 1 avoids this problem by compressing one frame of image to a low resolution using a unique compression technology. However, because signal readout occurs between the exposure of the first frame and the exposure of the second frame, it takes a long time to complete the exposure of two frames.

[0014] Furthermore, when light irradiation is also performed, such as in light section measurement, synchronized control of the on / off switching of the light source device and the exposure is required. This manual work is burdensome and time-consuming. While it is possible to shorten the time to some extent through mechanical automation, it still requires several seconds. It is difficult to reduce the time even further, for example, to less than one second, on the order of milliseconds or microseconds.

[0015] For example, when the frame rate of imaging is 60 fps, the imaging time for one frame is approximately 16.7 milliseconds. The imaging time for two frames is at least approximately 33.4 milliseconds. If a moving object such as a person in the background image moves at a speed of 1.5 meters per second, a deviation of approximately 5 centimeters will occur. For example, if an object moves at high speed, such as the blades of a ventilation fan, significant noise may appear in the difference image depending on the timing of exposure. Increasing the speed of control can be an important technical challenge.

[0016] The global shutter method, which exposes all pixels simultaneously and stores the signal in analog memory, can be an effective technology because it takes a short time to complete the exposure of one frame. However, as mentioned above, the signal must be read out before the exposure of the next frame, and the readout time becomes a bottleneck.

[0017] The disclosed technology addresses at least some of the above-mentioned problems. As will be described in detail later, signals obtained by two frame exposures are stored in an analog memory, and then the signals are read out. Since there is no signal readout between the exposure of the first frame and the exposure of the second frame, the time required to complete two frame exposures can be significantly shortened (ultra-high speed shutter). Image quality comparable to that of a global shutter system is achieved. A frame memory is not required.

[0018] 1 is a diagram showing an example of a schematic configuration of an imaging system 1 according to an embodiment. The imaging system 1 includes an imaging device 2, a light source device 7, and a control device 8.

[0019] The imaging device 2 detects light from within the imaging range. An object or the like located within the imaging range of the imaging device 2 is referred to as a subject 9 and is illustrated schematically. The imaging device 2 generates an image showing the difference between when the light source device 7 is off and when the light source device 7 is on. This image is referred to as a difference image 10. The term "image" may be interpreted to include video, and "imaging" may be interpreted to include photography. To the extent that there is no contradiction, the terms "image" and "imaging" may be interpreted as appropriate interchangeably. Further details of the imaging device 2 will be described later.

[0020] The light source device 7 includes a light source and is configured to output light from the light source. Various known light sources may be used. The state in which the light source device 7 outputs light and the state in which it does not output light are also referred to as the on and off states of the light source device 7.

[0021] The light source device 7 is configured to be switchable between on and off. The light output by the light source device 7 may be pulsed light. For example, high-speed switching control (pulse control) on the order of microseconds is possible. More specifically, the off and on cycle of the light source device 7 can be several tens of microseconds or shorter.

[0022] An example of the light source is a laser light source such as a semiconductor laser. In this case, the light output from the light source device 7 is laser light. By directly modulating the on / off state of the current of the semiconductor laser, the light source device 7 can be switched on and off at high speed.

[0023] The light source device 7 may output shaped laser light. For example, the laser light may be shaped and output so that the cross section of the laser light has a line shape. Optical elements such as lenses and slits required for this purpose may also be included in the light source device 7. The laser light may also be called slit light or the like.

[0024] The control device 8 controls each element of the imaging system 1, thereby controlling the entire imaging system 1. For example, the control device 8 controls imaging by the imaging device 2 and controls the on / off of the light source device 7. The control device 8 may be realized by hardware design or software design. Note that the functions of the control device 8 may be incorporated into one or both of the imaging device 2 and the light source device 7, in which case the control device 8 may not be necessary.

[0025] The imaging device 2 will be further described with reference to FIGS.

[0026] 2 is a diagram showing an example of a schematic configuration of the imaging device 2. The imaging device 2 is a solid-state imaging device such as a CMOS image sensor, and each element is provided on a semiconductor substrate. The imaging device 2 includes a pixel array unit 3, as well as peripheral circuits, signal lines, etc.

[0027] The peripheral circuits are indicated by reference numerals as a control circuit 61, a vertical drive circuit 62, a DAC circuit 63, a column signal processing circuit 64, a horizontal drive circuit 65, and an output circuit 66. The wirings are indicated by reference numerals as a signal line 67, a signal line 68, and a signal line 69.

[0028] The pixel array unit 3 includes a plurality of pixels 4. The plurality of pixels 4 are arranged in a two-dimensional array. FIG. 1 also shows an XYZ coordinate system for the pixel array unit 3. The X-axis direction corresponds to the row direction of the array, and corresponds, for example, to the lateral direction (horizontal direction) of the imaging device 2. The Y-axis direction corresponds to the column direction of the array, and corresponds, for example, to the longitudinal direction (vertical direction) of the imaging device 2. The Z-axis direction corresponds, for example, to the front-to-rear direction of the imaging device 2.

[0029] The pixel 4 includes a photoelectric conversion unit. The photoelectric conversion unit generates an electric charge according to the amount of incident light. An example of the photoelectric conversion unit is a photodiode (PD). A circuit is also provided that generates and outputs a voltage signal (pixel signal) according to the amount of electric charge generated in the photoelectric conversion unit. The pixel 4 may be interpreted as including such a circuit, and may also be called a pixel circuit.

[0030] The control circuit 61 receives data instructing an input clock, an operation mode, etc., and outputs data such as internal information of the photodetector device 100. The control circuit 61 generates clock signals and control signals that serve as references for the operations of the vertical drive circuit 62, the DAC circuit 63, the column signal processing circuit 64, the horizontal drive circuit 65, etc., based on a vertical synchronization signal, a horizontal synchronization signal, and a master clock. The control circuit 61 supplies these generated signals to the vertical drive circuit 62, the DAC circuit 63, the column signal processing circuit 64, the horizontal drive circuit 65, etc.

[0031] The vertical drive circuit 62 includes, for example, a shift register. The vertical drive circuit 62 is connected to the pixel array unit 3 via a plurality of signal lines 67 (horizontal signal lines) extending in the row direction of the pixels 4. Each signal line 67 extends, for example, for each pixel row, and each signal line 67 may include a plurality of signal lines. The vertical drive circuit 62 supplies a drive signal (for example, a pulse signal) for driving the pixels 4 to a selected signal line 67.

[0032] Driving the pixels 4 by the vertical drive circuit 62 includes driving transistors ( FIGS. 3 and 4 ) described below. The transistors are driven to output voltage signals (pixel signals) corresponding to the amount of charge generated in the photoelectric conversion units in the pixels 4 to corresponding signal lines 68 among a plurality of signal lines 68 (vertical signal lines) extending in the column direction of the pixels 4.

[0033] The DAC circuit 63 generates a sawtooth wave RAMP signal by DA (Digital to Analog) conversion and supplies the generated RAMP signal to the column signal processing circuit 64.

[0034] The column signal processing circuits 64 are connected to the pixel array unit 3 via signal lines 68. Each signal line 68 may include a plurality of signal lines. The column signal processing circuits 64 are arranged, for example, for each pixel column, and perform signal processing such as noise removal for each pixel column on pixel signals from one row of pixels 4. The column signal processing circuits 64 perform signal processing such as CDS (Correlated Double Sampling) for removing fixed pattern noise specific to the pixels 4, signal amplification, and AD (Analog to Digital) conversion.

[0035] The horizontal drive circuit 65 includes, for example, a shift register. The horizontal drive circuit 65 sequentially outputs horizontal scanning pulses to select each of the column signal processing circuits 64 in turn, and causes each of the column signal processing circuits 64 to output a pixel signal to a signal line 69.

[0036] The output circuit 66 performs signal processing on the pixel signals sequentially supplied from each of the column signal processing circuits 64 through a signal line 69, and outputs the processed signals. For example, buffering, black level adjustment, column variation correction, various digital signal processing, etc. are performed.

[0037] The imaging device 2 is capable of capturing images using a global shutter method. That is, all pixels 4 are simultaneously exposed and the signals are held. The signals are then read out row by row from the array. Two types of circuit configurations are described below as examples of pixel circuit configurations.

[0038] 3 and 4 are diagrams showing examples of pixel circuits. The photoelectric conversion unit included in the pixel 4 is shown as a photoelectric conversion unit 40. In this example, the photoelectric conversion unit 40 is a photodiode with an anode connected to GND (ground).

[0039] The pixel 4 includes an analog memory 5. The analog memory 5 stores a signal corresponding to the amount of exposure light of the pixel 4. Two types of analog memories 5 are exemplified. The analog memory 5 shown in FIG. 3 stores a charge corresponding to the amount of exposure light (charge domain system). The analog memory 5 shown in FIG. 4 stores a voltage corresponding to the amount of exposure light (voltage domain system). These will be described in order.

[0040] <Charge Domain Method> In the pixel circuit of FIG. 3, the pixel 4 includes a charge storage unit FD1 and several transistors (pixel transistors) in addition to the photoelectric conversion unit 40 and the analog memory 5.

[0041] As described above, the photoelectric conversion unit 40 generates charges according to the amount of incident light. In this example, the analog memory 5 is a capacitor 50. The charge storage unit FD1 is a floating diffusion region.

[0042] The transistors are designated by reference numerals as transistor 41, transistor 42, transistor 43, transistor 47, and transistor 48. When no particular distinction is required, they are simply referred to as transistors. In the following description, when a transistor is connected between two elements, this means that one of the source and drain of the transistor is connected to one element, and the other of the source and drain is connected to the other element.

[0043] The transistor 41 is provided between the photoelectric conversion unit 40 and the analog memory 5, and transfers the charges in the photoelectric conversion unit 40 to the analog memory 5. The gate electrode of the transistor 41 is connected to a corresponding signal line 67. A control signal from the signal line 67 controls conduction and non-conduction (on and off) between the drain and source of the transistor 41. When the transistor 41 is on, the charges in the photoelectric conversion unit 40 are transferred to the analog memory 5 via the transistor 41.

[0044] The analog memory 5 holds the charge transferred from the photoelectric conversion unit 40 by the transistor 41. As described above, in this example, the analog memory 5 is a capacitor 50.

[0045] The transistor 42 is provided between the analog memory 5 and the charge storage unit FD1, and transfers the charges in the analog memory 5 to the charge storage unit FD1. The gate electrode of the transistor 42 is connected to a corresponding signal line 67. The transistor 42 is turned on and off by a control signal from the signal line 67. When the transistor 42 is on, the charges in the analog memory 5 are transferred to the charge storage unit FD1 via the transistor 42.

[0046] The charge accumulation unit FD1 accumulates the charge transferred from the analog memory 5 by the transistor 42 and generates a voltage corresponding to the accumulated charge. It can also be said that the charge accumulated in the photoelectric conversion unit 40 is transferred to the charge accumulation unit FD1 via the transistors 41 and 42. The transistors 41 and 42 can also be called a first transfer transistor and a second transfer transistor.

[0047] The transistor 43 is connected between the charge storage unit FD1 and the power supply node Vreg, and discharges the charge stored in the charge storage unit FD1 to the power supply node Vreg. The transistor 43 can also be called a reset transistor. The gate electrode of the transistor 43 is connected to a corresponding signal line 67. The on / off of the transistor 43 is controlled by a control signal from the signal line 67. When the transistor 43 is on, the charge in the charge storage unit FD1 is discharged to the power supply node Vreg via the transistor 43. Note that when the transistor 42 is on, the charge in the analog memory 5 is also discharged, and when the transistor 41 is also on, the charge in the photoelectric conversion unit 40 connected to the charge storage unit FD1 of the photoelectric conversion unit 40 is also discharged.

[0048] The transistor 47 is connected between the power supply node VDD and the transistor 48. The gate of the transistor 47 is connected to the charge storage unit FD1. The transistor 47 amplifies and outputs the voltage generated in the charge storage unit FD1. The transistor 47 can also be called an amplifying transistor.

[0049] The voltage of the power supply node Vreg may be lower than the voltage of the power supply node VDD, for example.

[0050] The transistor 48 is connected between the transistor 47 and a signal line 68. The transistor 48 selectively outputs the output voltage of the transistor 47 to the signal line 68. The transistor 48 can also be called a selection transistor. The gate electrode of the transistor 48 is connected to the corresponding signal line 67. The on / off of the transistor 48 is controlled by a control signal from the signal line 67. When the transistor 48 is on, the output voltage of the transistor 47 is output to the signal line 68 via the transistor 48. This voltage can become a pixel signal.

[0051] The analog memory 5 may be configured to be able to hold two or more charge signals. For example, the analog memory 5 may include two capacitors 50, and each capacitor 50 may be selectively connectable to another element such as the photoelectric conversion unit 40 via a switch (transistor or the like) not shown.

[0052] <Voltage Domain Method> The pixel circuit of FIG. 4 differs from the pixel circuit of FIG. 3 described above in that the configuration of the analog memory 5 is different and that the pixel 4 does not include the transistor 41 but includes the transistors 44, 45, and 46.

[0053] The transistor 42 is provided between the photoelectric conversion unit 40 and the charge accumulation unit FD1, and transfers the charge in the photoelectric conversion unit 40 to the charge accumulation unit FD1.

[0054] The transistor 44 is connected between the power supply node VDD and the transistor 45. The gate of the transistor 44 is connected to the charge storage unit FD1. The transistor 44 amplifies and outputs the voltage generated in the charge storage unit FD1.

[0055] The transistor 45 is connected between the transistor 44 and the analog memory 5 and GND. The transistor 45 opens and closes the connection path between the transistor 44 and GND. The gate electrode of the transistor 45 is connected to a corresponding signal line 67. The on / off state of the transistor 45 is controlled by a control signal from the signal line 67. When the transistor 45 is off, the output voltage of the transistor 44 is output to the analog memory 5. The transistor 45 can also be called a precharge transistor.

[0056] The analog memory 5 holds the voltage output by the transistor 44. In this example, the analog memory 5 is a sample-and-hold circuit, and includes a capacitor 51, a transistor 52, a capacitor 53, and a transistor .

[0057] A series connection of a capacitor 51 and a transistor 52 and a series connection of a capacitor 53 and a transistor 54 are connected in parallel to each other. The gates of the transistors 52 and 54 are connected to corresponding signal lines 67. The on and off states of the transistors 52 and 54 are controlled by control signals from the signal lines 67. The operation of the sample-and-hold circuit itself is well known, so further detailed description will be omitted. See, for example, Patent Document 3.

[0058] The transistor 46 is connected between the analog memory 5 and the power supply node Vreg and initializes the voltage of the analog memory 5. The transistor 46 can also be called a reset transistor. The gate of the transistor 46 is connected to a corresponding signal line 67. The on / off of the transistor 46 is controlled by a control signal from the signal line 67. When the transistor 46 is on (and further, when the transistors 52 and 54 are also on), the voltage of the analog memory 5 is reset to the voltage of the power supply node Vreg.

[0059] The gate of the transistor 47 is connected to the analog memory 5. The transistor 47 amplifies the voltage of the analog memory 5 and outputs it.

[0060] 2, pixels 4 each including an analog memory 5 such as those shown in FIG. 3 or 4 are arranged in a two-dimensional array to form a pixel array unit 3. The light from the subject 9 is detected by the pixel array unit 3 by obtaining a signal from each pixel 4 exposed to light from the subject 9.

[0061] As described above, the imaging device 2 generates a difference image 10 that shows the difference between when the light source device 7 is off and when the light source device 7 is on. The image that would be obtained when the light source device 7 is off is also referred to as image 10a. The image that would be obtained when the light source device 7 is on is also referred to as image 10b. The signal value of the difference image 10 is calculated by subtracting the signal value of one of the images 10a and 10b from the signal value of the other (calculating the difference). The difference image 10 is generated based on these signal values.

[0062] Simply preparing signal values ​​for two frames (two images), image 10a and image 10b, and subtracting them would require a frame memory for storing signal values ​​for at least one frame, which increases memory capacity. The larger the memory capacity, the longer the time required for memory access, and the longer the processing time.

[0063] In the imaging system 1 according to the embodiment, the increase in required memory capacity is suppressed by using the analog memory 5 of each pixel 4 described above. Specifically, to obtain signals for the image 10a, the pixels 4 in one of the odd-numbered and even-numbered rows in the pixel array unit 3 are used. To obtain signals for the image 10b, the pixels 4 in the other of the odd-numbered and even-numbered rows in the pixel array unit 3 are used. In the following, it is assumed that the pixels 4 in the odd-numbered rows are used to obtain signals for the image 10a, and the pixels 4 in the even-numbered rows are used to obtain signals for the image 10b.

[0064] 5 is a diagram showing an example of rows in the pixel array unit 3. The pixels 4 and the analog memories 5 therein are shown schematically. The number of rows in the pixel array unit 3 is set to N. In the entire pixel array unit 3, there are rows numbered from 1 to N.

[0065] The odd-numbered rows are referred to as odd-numbered rows L1. There are 1st to N / 2nd odd-numbered rows L1 in the entire pixel array section 3. The odd-numbered rows L1 are referred to as odd-numbered row L1-1, odd-numbered row L1-2, odd-numbered row L1-(N / 2), etc. in the drawings.

[0066] The even-numbered rows are referred to as even-numbered rows L2. There are 1st to N / 2nd even-numbered rows L2 in the entire pixel array section 3. The even-numbered rows L2 are referred to as even-numbered row L2-1, even-numbered row L2-2, even-numbered row L2-(N / 2), etc. in the drawings.

[0067] When multiple pixels across multiple rows share a part of the pixel circuit (including the analog memory 5), the multiple rows may be treated as one odd row L1 or one even row L2. For example, when four pixels 4 across two rows and two columns share a part of the pixel circuit, the odd row L1 and the even row L2 are swapped every two rows.

[0068] Each pixel 4 in the odd-numbered row L1 and each pixel 4 in the even-numbered row L2 are exposed individually (independently of each other). Explaining this using the circuits of Figures 3 and 4 described above, turning off a transistor (transistor 41 in Figure 3, transistor 42 in Figure 4) connected to the photoelectric conversion unit 40 to generate and accumulate electric charge in the photoelectric conversion unit 40 can correspond to exposing the pixel 4.

[0069] In each pixel 4, an electric charge corresponding to the amount of light incident on that pixel 4, more specifically, the amount of light (amount of exposure light) incident on the photoelectric conversion unit 40 of that imaging device 2, is generated in the photoelectric conversion unit 40. A signal corresponding to the amount of exposure light is held in the analog memory 5. Since the signal line 67 that controls the transistor extends for each row, exposure control for each row is possible.

[0070] When the light source device 7 is off, the imaging device 2 exposes the pixels 4 in the odd-numbered rows L1 and stores a signal corresponding to the amount of exposure light for each pixel 4 in the odd-numbered rows L1 in the analog memory 5 of each pixel 4. When the light source device 7 is on, the imaging device 2 exposes the pixels 4 in the even-numbered rows L2 and stores a signal corresponding to the amount of exposure light for each pixel 4 in the even-numbered rows L2 in the analog memory 5 of each pixel 4. Note that these operations of the imaging device 2 in accordance with the on / off of the light source device 7 are performed under the control of the imaging device 2 and the light source device 7 by the control device 8.

[0071] The imaging device 2 generates a difference image 10 based on the signals held in the analog memories 5 of the pixels 4 in the odd-numbered rows L1 and the signals held in the analog memories 5 of the pixels 4 in the even-numbered rows L2. The description will also be made with reference to FIGS. 6 and 7.

[0072] 6 is a diagram showing an example of generating a difference image 10. In this example, the subject 9 is a tunnel, and the above-mentioned light section measurement is possible, for example.

[0073] Image 10a is an image that would be obtained based on the signals held in the analog memories 5 of each pixel 4 in the odd-numbered row L1, and includes the subject 9 when the light source device 7 is off. Image 10b is an image that would be obtained based on the signals held in the analog memories 5 of each pixel 4 in the even-numbered row L2, and includes the subject 9 when the light source device 7 is on. Note that images 10a and 10b do not need to be actually generated.

[0074] A pattern of light projected onto the subject 9 is also observed in the image 10b. This light pattern is referred to as a projection pattern 70 and is illustrated. In this example, the projection pattern 70 has a line shape that extends on the surface of the tunnel wall in a direction that intersects (e.g., substantially perpendicular to) the direction in which the tunnel extends. The shape of the projection pattern 70 may correspond to the shape of the tunnel opening at the projection position (illumination position).

[0075] The difference image 10 is an image showing the difference between the image 10a and the image 10b, and more specifically, it is an image showing only the parts of the image 10b that are different from the image 10a. The signal value of the difference image 10 is obtained by subtracting the signal value of the image 10a from the signal value of the image 10b. In the example shown in FIG. 6 , the difference between the image 10b and the image 10a is the projection pattern 70. That is, the difference image 10 includes only the projection pattern 70. The difference image 10 can also be said to be an image obtained by removing the object 9 from the image 10b.

[0076] Specifically, to generate the difference image 10, a difference value between the signal held in the analog memory 5 of each pixel 4 in the odd-numbered row L1 and the signal held in the analog memory 5 of each pixel 4 in the even-numbered row L2 is calculated. This difference value is referred to as a difference value D3. An example of the main body of the calculation is the column signal processing circuit 64, the details of which will be described later.

[0077] The calculation of the difference value D3 will be described with reference to Fig. 7. Note that, hereinafter, the signals held in the analog memories 5 of the pixels 4 in the odd-numbered rows L1 will also be simply referred to as the signals of the odd-numbered rows L1, etc. Similarly, the signals held in the analog memories 5 of the pixels 4 in the even-numbered rows L2 will also be simply referred to as the signals of the even-numbered rows L2, etc.

[0078] FIG. 7 is a diagram showing an example of calculation of the difference value D3. For every two adjacent rows, the odd-numbered row L1 and the even-numbered row L2, the signals of each row are read out, and a difference value D3 is calculated by subtraction processing. Specifically, the signals of the odd-numbered row L1-1 and the even-numbered row L2-1 are read out, and a difference value D3 therebetween is calculated. This difference value D3 becomes the signal value of the first row of the difference image 10. The signals of the odd-numbered row L1-2 and the even-numbered row L2-2 are read out, and a difference value D3 therebetween is calculated. This difference value D3 becomes the signal value of the second row of the difference image 10. The same processing is repeated, and finally, the signals of the odd-numbered row L1-N / 2 and the even-numbered row L2-N / 2 are read out, and a difference value D3 therebetween is calculated. This difference value D3 becomes the signal value of the N / 2th row of the difference image 10. A difference image 10 is generated using all the calculated difference values ​​D3 as signal values.

[0079] Note that if the difference image 10 is generated using only the difference values ​​D3 for N / 2 rows, the number of rows in the difference image 10 will be N / 2, resulting in a decrease in resolution. To prevent this, interpolation processing or the like may be performed to make the number of rows in the difference image 10 N. For example, rows having the same signal value as the signal values ​​of the preceding and following rows or an average signal value thereof may be added to the difference image 10.

[0080] One difference image 10 corresponds to one frame of image. The projection pattern 70 in the difference image 10 can correspond to the shape of the object 9 at that position. The difference image 10 including the projection pattern 70 shows the two-dimensional shape of the object 9 at the position of the projection pattern 70.

[0081] By generating the difference images 10 while gradually changing (scanning) the position of the projection pattern 70, a plurality of difference images 10 are obtained, each showing the two-dimensional shape of the subject 9 at each position. These difference images 10 show the entire three-dimensional shape of the subject 9. This will be described with reference to FIG. 8 as well.

[0082] 8 is a diagram showing an example of a plurality of difference images 10. By arranging a plurality of difference images 10 along the longitudinal direction of the tunnel, the shape of the tunnel along the longitudinal direction (opening shape) can be grasped. For example, by combining a plurality of difference images 10, it is possible to generate a three-dimensional image or a three-dimensional model of the tunnel.

[0083] 1 , the control device 8 will be further described. The control device 8 controls the imaging device 2 and the light source device 7 so that the pixels 4 in the odd-numbered rows L1 of the pixel array unit 3 of the imaging device 2 are exposed when the light source device 7 is off, and the pixels 4 in the even-numbered rows L2 of the pixel array unit 3 of the imaging device 2 are exposed when the light source device 7 is on. This control can also be called synchronous control of the on / off of the light source device 7 and the exposure of the pixels 4 in the odd-numbered rows L1 and the exposure of the pixels 4 in the even-numbered rows L2.

[0084] The specific method of synchronous control is not particularly limited, but for example, the control device 8 transmits a control signal to each of the image capture device 2 and the light source device 7, thereby synchronously controlling the image capture device 2 and the light source device 7. An example of the control signal is a signal for register setting. The control signal from the control device 8 to the light source device 7 may include a signal for controlling the on / off of the light source device 7. The control signal from the control device 8 to the image capture device 2 may include a signal for controlling the exposure of the pixels 4 in the odd-numbered rows L1 and the exposure of the pixels 4 in the even-numbered rows L2 of the pixel array unit 3 of the image capture device 2.

[0085] 9 is a timing chart showing an example of synchronous control. Currently, the light source device 7 is off. From time t1 to time t2, the pixels 4 in the odd-numbered row L1 are exposed. At time t3, the light source device 7 is turned on. From time t4 to time t5, the pixels 4 in the even-numbered row L2 are exposed. At time t6, the light source device 7 is turned off. For example, by using such synchronous control, the pixels 4 in the odd-numbered row L1 can be exposed when the light source device 7 is on, and the pixels 4 in the even-numbered row L2 can be exposed when the light source device 7 is off.

[0086] 10 is a flowchart showing an example of processing (imaging method, image generation method) executed in the imaging system 1. Description of content that overlaps with the above will be omitted where appropriate. It is assumed that the light source device 7 is initially off.

[0087] In step S1, the imaging conditions are adjusted. Examples of the imaging conditions include the position and direction (imaging position, imaging direction) of the imaging device 2, the position and direction (illumination position, illumination direction) of the light source device 7, etc. The adjustment of the imaging conditions may be performed manually by a user of the imaging system 1, or may be automated. In the latter case, for example, a mechanical device may be used to adjust the positions, directions, etc. of the imaging device 2 and the light source device 7.

[0088] In step S2, the odd-numbered rows L1 are exposed. The control device 8 controls the imaging device 2 so that the pixels 4 in the odd-numbered rows L1 are exposed and a signal corresponding to the amount of exposure light for each pixel 4 is stored in the analog memory 5 of each pixel 4. As a result, a signal when the light source device 7 is off (corresponding to the signal of the image 10a in FIG. 6 ) is stored in the analog memory 5.

[0089] In step S3, the control device 8 controls the light source device 7 so that the light source device 7 is turned on.

[0090] In step S4, the imaging device 2 exposes the pixels 4 in the even-numbered rows L2. The control device 8 controls the light source device 7 so that the pixels 4 in the even-numbered rows L2 are exposed and a signal corresponding to the amount of exposure light for each pixel 4 is stored in the analog memory 5 of the pixel 4. As a result, a signal when the light source device 7 is off (corresponding to the signal of the image 10b in FIG. 6 ) is stored in the analog memory 5.

[0091] In step S5, the control device 8 controls the light source device 7 so that the light source device 7 is turned off.

[0092] In step S6, the imaging device 2 calculates a difference value D3. The calculated difference value D3 corresponds to the signal value of one row of the difference image 10. For example, in the first loop of the flowchart, the difference value D3 between the signal of the odd-numbered row L1-1 and the signal of the even-numbered row L2-1 is calculated, and the signal value of the first row of the difference image 10 is obtained. The specific calculation method will be described later with reference to FIGS. 11 to 18.

[0093] In step S7, the data is transferred, which corresponds to the difference value D3 for one row of the difference image 10, and this data is transferred to another location.

[0094] In step S8, it is determined whether the transfer of data for one difference image 10 has been completed. For example, if the difference value D3 calculated in the previous step S6 is the difference value D3 between the signal for the last odd-numbered row L1 and the signal for the even-numbered row L2, and the data transferred in step S7 corresponds to the difference value D3 data for the last row of the difference image 10, it is determined that the data transfer has been completed. If the data transfer has been completed (step S8: Yes), the process proceeds to step S9. If the data transfer has not been completed (step S8: No), the process returns to step S6, and the difference value D3 between the signal for the next odd-numbered row L1 and the signal for the even-numbered row L2 is calculated.

[0095] In step S9, it is determined whether or not to perform imaging to generate the next difference image 10. If imaging is to be performed (step S9: Yes), the process returns to step S1. If imaging is not to be performed (step S9: No), the process of the flowchart ends.

[0096] For example, the difference image 10 can be generated as described above. As described above, in the imaging device 2, when the light source device 7 is off, the pixels 4 in the odd-numbered row L1 are exposed, and a signal corresponding to the amount of exposure light (the signal for image 10a) is stored in the analog memory 5 of each pixel 4. Furthermore, when the light source device 7 is on, the pixels 4 in the even-numbered row L2 are exposed, and a signal corresponding to the amount of exposure light (the signal for image 10b) is stored in the analog memory 5 of each pixel 4. The signals stored in the analog memories 5 are then read out, and a difference value D3 is calculated. A difference image 10 is generated using the calculated difference value D3 as its signal value. Because the signals are read out after two frames of exposure, the time required to complete two frames of exposure can be shorter than, for example, when signals are read out between the first frame exposure (exposure for obtaining the signal for image 10a) and the second frame exposure (exposure for obtaining the signal for image 10b). A frame memory is also not required.

[0097] An example of the entity that calculates the difference value D3 is the column signal processing circuit 64 (FIG. 2). The difference value D3 is calculated mainly by utilizing the AD conversion process (including CDS process) in the column signal processing circuit 64. The following description will be made with reference to FIGS. 11 to 18.

[0098] 11 and 12 are diagrams illustrating a first subtraction method. In this method, a line memory 64m is used. The line memory 64m may be provided inside or outside the column signal processing circuit 64.

[0099] 11, first, a reset level signal is read out from (the analog memory 5 of each pixel 4) in the odd-numbered row L1. The reset level signal is a signal obtained when the charge in the photoelectric conversion unit 40 (FIGS. 3 and 4) of the pixel 4 has been discharged. This signal can also be called a noise signal, etc.

[0100] More specifically, auto-zero adjustment (which can also be called offset adjustment) is performed based on the reset level signal. The signal at this time is referred to as signal P1. Note that auto-zero adjustment is also called AZ adjustment.

[0101] Furthermore, a signal corresponding to the amount of exposure light is read out from (the analog memory 5 of each pixel 4 of) the odd-numbered row L1. This signal is referred to as signal D1.

[0102] CDS processing is performed based on the signal P1 and the signal D1. Specifically, a signal value is obtained by subtracting the signal P1 from the signal D1. For simplicity of explanation, this signal value will be referred to as the signal value D1, using the same D1. The signal value D1 is stored in the line memory 64m.

[0103] Similarly, AZ adjustment is performed on the reset level signals from the pixels in the even-numbered row L2. The signal at this time is referred to as signal P2. A signal corresponding to the amount of exposure light is also read out. This signal is referred to as signal D2.

[0104] CDS processing is performed based on the signal P2 and the signal D2. Specifically, a signal value is obtained by subtracting the signal P2 from the signal D2. For simplicity of explanation, this signal value will be referred to as the signal value D2, using the same D2.

[0105] A difference value between the signal value D2 and the signal value D1 stored in the line memory 64m is calculated. Specifically, the signal value D1 is subtracted from the signal value D2. The signal value obtained by the subtraction becomes a difference value D3.

[0106] A timing chart is shown in Figure 12. The signal of the odd-numbered row L1-1 is read out, and after AZ adjustment, a signal P1 and a signal D1 are obtained. After CDS processing, a signal value D1 after subtraction is obtained. The data of the signal value D1 is transferred and stored in the line memory 64m.

[0107] The signal of the even-numbered row L2-1 is read out, and after AZ adjustment, the signal P2 and the signal D2 are obtained. After CDS processing, the signal value D2 after subtraction is obtained. The data of the signal value D2 is transferred and subtracted by the signal value D11 stored in the line memory 64m. The obtained value is output as the difference value D3.

[0108] The same process is repeated until all the difference values ​​D3 for N / 2 rows are output.

[0109] 12, the period during which the analog memory 5 holds the signal is shown as a holding period Td. The holding period Td lasts until a signal value D2 based on the signal of the last row, i.e., the even-numbered row L2-(N / 2), is obtained.

[0110] <Second Subtraction Method> The second subtraction method is different from the first subtraction method in that it does not use the line memory 64m. Effective use of CDS also speeds up processing.

[0111] 13 and 14 are diagrams illustrating the second subtraction technique.

[0112] Referring to FIG. 13, first, a reset level signal is read out from the odd-numbered row L1, and AZ adjustment is performed based on this signal.

[0113] Next, a signal corresponding to the amount of exposure light is read out from the odd-numbered row L1, and a signal D1 is obtained.

[0114] Similarly, AZ adjustment is performed based on the reset level signal of the even-numbered row L2. A signal corresponding to the amount of exposure light is read out, and a signal D2 is obtained.

[0115] CDS processing is performed on the signals D1 and D2. A signal value is obtained by subtracting the signal D1 from the signal D2. This signal value becomes the difference value D3.

[0116] An example of a timing chart is shown in Figure 14. The signal of the odd-numbered row L1 is read out, and after AZ adjustment, a signal D1 is obtained. The signal of the even-numbered row L2 is read out, and after AZ adjustment, a signal D2 is obtained. After CDS processing, a signal value D2 after subtraction is obtained. This signal value D2 is transferred (output) as a difference value D3.

[0117] Similar processing is repeatedly executed, and all difference values ​​D3 for N / 2 rows are output. Compared to the first method (FIG. 12) described above, the number of processes required for AD conversion is reduced, and processing such as data transfer to the line memory 64m is also unnecessary. This increases the processing speed accordingly. The signal retention period Td by the analog memory 5 is also shortened. As the retention period Td is shortened, for example, degradation of the signal (deterioration of data) retained in the analog memory 5 can be suppressed.

[0118] <Third Subtraction Method> The third subtraction method enables even faster processing.

[0119] 15 to 18 are diagrams illustrating the third subtraction technique.

[0120] 15 shows an example of a schematic configuration of the column signal processing circuit 64. Of the circuits included in the column signal processing circuit 64, a comparator 641, a feedback circuit 642, and a count circuit 643 are indicated by reference numerals. In addition, elements included in the column signal processing circuit 64 include a capacitor C1, a capacitor C2, a switch S Ｄ２ , switch S ＡＺ０ , switch S ＡＺ１ , switch S ＡＺ２ , switch S ＦＢ１ and switch S ＦＢ２ are also indicated by symbols.

[0121] The RAMP signal and the signal of the odd-numbered row L1 or the even-numbered row L2 are input to the comparator 641. The RAMP signal is generated by the DAC circuit 63 (FIG. 2) and supplied to the column signal processing circuit 64, as described above.

[0122] 15, the signal for the odd-numbered row L1 and the signal for the even-numbered row L2 are supplied via different paths. The point where the signal for the odd-numbered row L1 is supplied is shown as a node N1, and the point where the signal for the even-numbered row L2 is supplied is shown as a node N2.

[0123] A capacitor C1 is connected in series between the node N1 and one input terminal of the comparator 641. The connection point between the capacitor C1 and the comparator 641 is shown as a node N3.

[0124] A switch S is connected between the node N2 and the other input terminal of the comparator 641. Ｄ２ and capacitor C2 are connected in series. Ｄ２ The connection point between the capacitor C2 and the comparator 641 is shown as a node N5.

[0125] Another node shown is node N6, which is connected to the auto-zero voltage V ＡＺ In this example, the auto-zero voltage V ＡＺ is obtained from the comparator 641.

[0126] Switch S Ｄ２ is connected between the node N2 and the node N4. ＡＺ０ is connected between the node N1 and the node N4. ＡＺ１ is connected between the node N3 and the node N6. ＡＺ２ is connected between node N5 and node N6.

[0127] Switch S ＦＢ１ is disposed in the comparator 641 and connected to one input terminal of the comparator 641. ＦＢ２ is disposed in the comparator 641 and is connected to the other input terminal of the comparator 641. Specifically, the switch S ＦＢ１ is connected to node N3. ＦＢ２ is connected to node N5. ＦＢ１ or switch S ＦＢ２The signal at the node N3 or the signal at the node N5 is input to the comparator 641 via

[0128] The feedback circuit 642 includes a switch S ＦＢ１ and switch S ＦＢ２ The output of the comparator 641 is fed back to the count circuit 643 for on / off control. The count circuit 643 performs counting based on the output of the comparator 641. The count value corresponds to a signal value indicating the magnitude of the signal input to the comparator 641.

[0129] An example of a timing chart is shown in FIG. 16. It is assumed that the signal for the odd-numbered row L1 and the signal for the even-numbered row L2 have already been obtained. First, the switch S ＡＺ０ , switch S ＡＺ１ and switch S ＡＺ２ Then, the switch S Ｄ２ is turned on and the AD operation starts. ＦＢ１ is on and switch S ＦＢ２ is off.

[0130] Specifically, at time t11, the voltage of the RAMP signal becomes equal to the auto-zero voltage V ＡＺ The count value of the count circuit 643 at this time may correspond to the value of the signal D1 described above.

[0131] At time t12, the output of the comparator 641 is inverted. The count circuit 643 starts counting. The output of the comparator 641 is fed back, and the switch S ＦＢ１ is turned off and switch S ＦＢ２ At this time, AD conversion cannot be performed.

[0132] At time t13, the output of the comparator 641 returns to its original state (is inverted), and AD conversion becomes possible again. The period up to time t13 during which AD conversion cannot be performed is referred to as a dead band period T1.

[0133] At time t14, the signal of the even-numbered row L2 is AD converted to obtain a signal value D2. This AD conversion is performed after the AZ adjustment based on the signal of the odd-numbered row L1 as described above. The signal value D2 obtained by this AD conversion is equal to the signal value after the signal D1 has been subtracted. In other words, this signal value D2 becomes the difference value D3 as it is.

[0134] At time t15, the output of the comparator 641 is inverted, and the count circuit 643 stops counting.

[0135] The time t14 at which the signal value D2 is obtained must be later than the time t13 at which the dead band period T1 ends. This condition is more likely to be met as the difference in light intensity between when the projection pattern 70 is on and when it is off increases. As mentioned above, this is effective when the light output from the light source device 7 is a line-shaped laser beam.

[0136] 17, first, a signal corresponding to the amount of exposure light is read from the odd-numbered row L1, and AZ adjustment is performed based on this signal. Next, a signal corresponding to the amount of exposure light is read from the even-numbered row L2. As previously described with reference to FIGS. 15 and 16, a signal value D2 and, therefore, a difference value D3 are obtained.

[0137] 18 shows an example of a timing chart. The signal of the odd-numbered row L1 is read out and AZ adjustment is performed. The signal of the even-numbered row L2 is read out and a signal value D2 is obtained. The obtained signal value D2 is transferred (output) as a difference value D3.

[0138] The same process is repeated until all the difference values ​​D3 for the N / 2 rows are output. Compared to the second method (FIG. 12), the number of processes required for AD conversion is further reduced. This further speeds up the process, and the signal retention period Td by the analog memory 5 is also further shortened.

[0139] For example, the difference value D3 can be calculated using the first to third subtraction methods described above.

[0140] 2. Summary The techniques described above can be specified, for example, as follows. One of the disclosed techniques is an imaging system 1. As described with reference to FIGS. 1 to 7 and 9 to 18 , the imaging system 1 includes an imaging device 2 including a plurality of pixels 4 arranged in a two-dimensional array, and a light source device 7. Each of the plurality of pixels 4 includes an analog memory 5 that stores a signal corresponding to an amount of exposure light. When the light source device 7 is off, the imaging device 2 exposes the pixels 4 in one row (e.g., odd row L1) of the odd rows L1 and even rows L2 of the array, and stores a signal corresponding to the amount of exposure light for each pixel 4 in the one row in the analog memory 5 for each pixel 4. When the light source device 7 is on, the imaging device 2 exposes the pixels 4 in the other row (e.g., even row L2) of the odd rows L1 and even rows L2, and stores a signal corresponding to the amount of exposure light for each pixel 4 in the other row in the analog memory 5 for each pixel 4. Then, the imaging device 2 generates a differential image 10 showing the difference between when the light source device 7 is off and when the light source device 7 is on, based on the signals held in the analog memories 5 of each pixel 4 in one row and the signals held in the analog memories 5 of each pixel 4 in the other row.

[0141] According to the imaging system 1 described above, after two frames of exposure, that is, exposure of the pixels 4 in the odd-numbered rows L1 and exposure of the pixels 4 in the even-numbered rows L2, are completed, the signals held in the analog memories 5 of the pixels 4 are read out. For example, the time required to complete two frames of exposure can be made shorter than when signals are read out between the exposure of the first frame and the exposure of the second frame.

[0142] 1 and other figures, the light output by the light source device 7 is pulsed light, and the on / off switching of the light source device 7 may be controlled on the order of microseconds. This allows the time required to complete exposure for two frames to be shortened to the order of microseconds. For example, even if the subject 9 captured by the imaging device 2 is an object moving at high speed, it is more likely that noise that may occur due to a timing difference between the exposure of the first frame and the exposure of the second frame can be suppressed.

[0143] 2 to 7 and 9 to 18, the imaging device 2 may calculate a difference value D3 between the signal held in the analog memory 5 of each pixel 4 in one row (for example, odd-numbered row L1) and the signal held in the analog memory 5 of each pixel 4 in the other row (for example, even-numbered row L2), and generate a difference image 10 based on the calculated difference value D3. In this way, for example, a difference image 10 indicating the difference between when the light source device 7 is off and when the light source device 7 is on can be generated.

[0144] 11 and 12 , the imaging device 2 may store, in the line memory 64 m, a signal value D1 based on the signals held in the analog memory 5 of each pixel 4 in one row (for example, the odd-numbered row L1), and calculate a difference value D3 between the signal value D2 based on the signals held in the analog memory 5 of each pixel 4 in the other row (for example, the even-numbered row L2) and the signal value D1 stored in the line memory 64 m. For example, the difference value D3 can be calculated by performing a subtraction process using the line memory 64 m in this way.

[0145] As described with reference to Figures 13 and 14, the imaging device 2 may calculate the difference value D3 by performing CDS processing on the signals held in the analog memories 5 of each pixel 4 in one row (e.g., odd-numbered row L1) and the signals held in the analog memories 5 of each pixel 4 in the other row (e.g., even-numbered row L2). This reduces the number of processes required for AD conversion and eliminates the need to use the line memory 64m, thereby speeding up the processing. The signal holding period Td by the analog memory 5 is also shortened, which increases the likelihood of avoiding signal degradation, etc.

[0146] 15 to 18 , the imaging device 2 may calculate the difference value D3 by performing auto-zero adjustment based on the signals held in the analog memories 5 of the pixels 4 in one row (e.g., odd-numbered row L1), and then AD converting the signals held in the analog memories 5 of the pixels 4 in the other row (e.g., even-numbered row L2). This further reduces the number of processes required for AD conversion, contributing to even faster processing. The signal holding period Td by the analog memories 5 is also further shortened.

[0147] 1 and 6, the light output from the light source device 7 may be laser light. The cross-sectional shape of the laser light may be a line. For example, the laser light may be used for light section measurement.

[0148] As described with reference to Fig. 3 etc., the analog memory 5 may hold a charge corresponding to the amount of exposure light. Alternatively, as described with reference to Fig. 4 etc., the analog memory 5 may hold a voltage corresponding to the amount of exposure light. For example, the difference image 10 can be generated by applying such a charge domain or voltage domain global shutter method.

[0149] The imaging device 2 described with reference to FIGS. 1 to 7 and 9 to 18 is also one of the disclosed technologies. The imaging device 2 is used together with a light source device 7. The imaging device 2 includes a plurality of pixels 4 arranged in a two-dimensional array, each including an analog memory 5 that stores a signal corresponding to the amount of exposure light. When the light source device 7 is off, the imaging device 2 exposes the pixels 4 in one row (e.g., the odd row L1) of the odd rows L1 and the even rows L2 of the array, and stores a signal corresponding to the amount of exposure light for each pixel 4 in the one row in the analog memory 5 of each pixel 4. When the light source device 7 is on, the imaging device 2 exposes the pixels 4 in the other row (e.g., the odd row L1) of the odd rows L1 and the even rows L2, and stores a signal corresponding to the amount of exposure light for each pixel 4 in the other row in the analog memory 5 of each pixel 4. The imaging device 2 then generates a difference image 10 that indicates the difference between when the light source device 7 is off and when the light source device 7 is on, based on the signals held in the analog memories 5 of the pixels 4 in one row and the signals held in the analog memories 5 of the pixels 4 in the other row. As described above, this imaging device 2 also shortens the time required to complete exposure for two frames.

[0150] The effects described in this disclosure are merely examples and are not limited to the disclosed contents. Other effects may also be obtained.

[0151] Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, components of different embodiments and modifications may be combined as appropriate.

[0152] Note that the present technology can also be configured as follows: (1) An imaging system comprising: an imaging device including a plurality of pixels arranged in a two-dimensional array; and a light source device, wherein each of the plurality of pixels includes an analog memory that stores a signal corresponding to an amount of exposure light, wherein the imaging device, when the light source device is off, exposes pixels in one of odd and even rows of the array and stores a signal corresponding to the amount of exposure light for each pixel in the one row in the analog memory of each pixel, when the light source device is on, exposes pixels in the other of odd and even rows and stores a signal corresponding to the amount of exposure light for each pixel in the other row in the analog memory of each pixel, and generates a differential image indicating a difference between when the light source device is off and when the light source device is on, based on the signal stored in the analog memory of each pixel in the one row and the signal stored in the analog memory of each pixel in the other row. (2) The imaging system according to (1), wherein the light output by the light source device is pulsed light, and the on / off switching of the light source device is controlled on the order of microseconds. (3) The imaging system according to (1) or (2), wherein the imaging device calculates a difference value between a signal held in an analog memory of each pixel in the one row and a signal held in an analog memory of each pixel in the other row, and generates the difference image based on the calculated difference value. (4) The imaging system according to (3), wherein the imaging device stores a signal value based on the signal held in the analog memory of each pixel in the one row in a line memory, and calculates a difference value between the signal value based on the signal held in the analog memory of each pixel in the other row and the signal value stored in the line memory. (5) The imaging system according to (3), wherein the imaging device calculates the difference value by performing CDS processing on the signals held in the analog memory of each pixel in the one row and the signals held in the analog memory of each pixel in the other row. (6) The imaging system according to (3), wherein the imaging device calculates the difference value by performing auto-zero adjustment based on the signal held in the analog memory of each pixel in the one row, and then AD-converting the signal held in the analog memory of each pixel in the other row.(7) The imaging system according to any one of (1) to (6), wherein the light output from the light source device is laser light. (8) The imaging system according to (7), wherein a cross-sectional shape of the laser light is a line. (9) The imaging system according to any one of (1) to (8), wherein the analog memory holds a charge according to the amount of exposure light. (10) The imaging system according to any one of (1) to (8), wherein the analog memory holds a voltage according to the amount of exposure light. (11) An imaging device used together with a light source device, comprising a plurality of pixels arranged in a two-dimensional array, each including an analog memory that stores a signal corresponding to an amount of exposure light; when the light source device is off, the pixels in one of the odd and even rows of the array are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the one row is stored in the analog memory of each pixel; when the light source device is on, the pixels in the other of the odd and even rows are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the other row is stored in the analog memory of each pixel; and a differential image showing a difference between when the light source device is off and when the light source device is on is generated based on the signal stored in the analog memory of each pixel in the one row and the signal stored in the analog memory of each pixel in the other row.

[0153] REFERENCE SIGNS LIST 1 Imaging system 10 Difference image 10a Image 10b Image 2 Imaging device 3 Pixel array section 4 Pixel 40 Photoelectric conversion section 41 Transistor 42 Transistor 43 Transistor 44 Transistor 45 Transistor 46 Transistor 47 Transistor 48 Transistor 5 Analog memory 50 Capacitor 51 Capacitor 52 Transistor 53 Capacitor 54 Transistor 61 Control circuit 62 Vertical drive circuit 63 DAC circuit 64 Column signal processing circuit 641 Comparator 642 Feedback circuit 643 Count circuit 64m Line memory 65 Horizontal drive circuit 66 Output circuit 67 Signal line 68 Signal line 69 Signal line 7 Light source device 70 Projection pattern 8 Control device 9 Object C1 Capacitor C2 Capacitor FD1 Charge storage section L1 Odd row L2 Even row N1 Node N2 Node N3 Node N4 Node N5 Node N6 Node VDD Power supply node Vreg Power supply node

Claims

1. An imaging system comprising: an imaging device including a plurality of pixels arranged in a two-dimensional array; and a light source device, each of the plurality of pixels including an analog memory that stores a signal corresponding to the amount of exposure light; wherein when the light source device is off, the imaging device exposes the pixels of one of the odd and even rows of the array and stores a signal corresponding to the amount of exposure light for each pixel of the one row in the analog memory of each pixel; when the light source device is on, the imaging device exposes the pixels of the other of the odd and even rows and stores a signal corresponding to the amount of exposure light for each pixel of the other row in the analog memory of each pixel; and generates a differential image showing the difference between when the light source device is off and when the light source device is on, based on the signals stored in the analog memory of each pixel of the one row and the analog memory of each pixel of the other row.

2. The imaging system according to claim 1, wherein the light output from the light source device is pulsed light, and the on / off switching of the light source device is controlled on the order of microseconds.

3. The imaging system according to claim 1, wherein the imaging device calculates a difference value between a signal held in an analog memory of each pixel in one row and a signal held in an analog memory of each pixel in the other row, and generates the difference image based on the calculated difference value.

4. The imaging system according to claim 3, wherein the imaging device stores signal values ​​based on signals held in the analog memories of each pixel in one of the rows in a line memory, and calculates a difference between the signal values ​​based on signals held in the analog memories of each pixel in the other row and the signal values ​​stored in the line memory.

5. The imaging system according to claim 3, wherein the imaging device calculates the difference value by performing CDS processing on the signals held in the analog memories of the pixels in one row and the signals held in the analog memories of the pixels in the other row.

6. The imaging system according to claim 3, wherein the imaging device calculates the difference value by performing auto-zero adjustment based on the signal held in the analog memory of each pixel in the one row, and then AD converting the signal held in the analog memory of each pixel in the other row.

7. The imaging system according to claim 1, wherein the light output from the light source device is laser light.

8. The imaging system according to claim 7, wherein the cross-sectional shape of the laser light is a line shape.

9. The imaging system according to claim 1, wherein the analog memory holds an electric charge corresponding to the amount of exposure light.

10. The imaging system according to claim 1, wherein the analog memory holds a voltage corresponding to the amount of exposure light.

11. An imaging device used with a light source device, comprising a plurality of pixels arranged in a two-dimensional array, each including an analog memory that stores a signal corresponding to the amount of exposure light; when the light source device is off, the pixels in one of the odd and even rows of the array are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the one row is stored in the analog memory of each pixel; when the light source device is on, the pixels in the other of the odd and even rows are exposed to light, and a signal corresponding to the amount of exposure light for each pixel in the other row is stored in the analog memory of each pixel; and a differential image showing the difference between when the light source device is off and when the light source device is on is generated based on the signal stored in the analog memory of each pixel in the one row and the signal stored in the analog memory of each pixel in the other row.

Images