Sensor system

Abstract

This sensor system comprises a plurality of sensors that each detect light from a subject, wherein: the plurality of sensors include a Z sensor that includes a plurality of imaging surface phase difference pixels arranged in a matrix, and an image sensor that includes a plurality of imaging pixels arranged in a matrix; each time one row of imaging surface phase difference pixel signals is read out, the Z sensor transmits, to the image sensor, a transmission signal generated on the basis of the read imaging surface phase difference pixel signals; and the image sensor processes imaging pixel signals specified by the transmission signal from the Z sensor, among one row of imaging pixel signals.

Description

Sensor system

[0001] The present disclosure relates to a sensor system.

[0002] For example, as disclosed in Patent Document 1 and Patent Document 2, various sensors that can be used for imaging are known.

[0003] Japanese Patent Application Laid-Open No. 2021-40294 International Publication No. 2020 / 241108

[0004] There is an application such as obtaining only an image of a certain region within the imaging range. When an image of the entire image is obtained and then an image of a region therein is extracted, processing of the entire image becomes necessary, and the power consumption increases accordingly.

[0005] One aspect of the present disclosure is to suppress an increase in power consumption.

[0006] A sensor system according to one aspect of the present disclosure includes a plurality of sensors each of which detects light from a subject. The plurality of sensors include a Z sensor including a plurality of imaging surface phase difference pixels arranged in a matrix, and an image sensor including a plurality of imaging pixels arranged in a matrix. The Z sensor transmits a transmission signal generated based on the read imaging surface phase difference pixel signal to the image sensor each time it reads out an imaging surface phase difference pixel signal for one row. The image sensor processes the imaging pixel signal specified by the transmission signal from the Z sensor among the imaging pixel signals for one row.

[0007] This figure shows an example of the schematic configuration of the sensor system 1 according to the embodiment. This figure shows an example of the subject 9. This figure shows an example of the schematic configuration of the sensor 4. This figure shows an example of the detection result of the Z sensor 4-1. This figure shows an example of the processing of the image pixel signal Sm by the image sensor 4-2. This figure shows an example of the processing (detection method) performed in the sensor system 1. This figure shows an example of the processing (detection method) performed in the sensor system 1. This figure shows an example of the generation and transmission of the transmission signal Tx by the logic operation unit 48. This figure shows an example of the generation and transmission of the transmission signal Tx by the logic operation unit 48. This figure shows an example of the generation and transmission of the transmission signal Tx by the logic operation unit 48. This figure shows an example of the generation and transmission of the transmission signal Tx by the logic operation unit 48. This figure shows an example of the generation and transmission of the transmission signal Tx by the logic operation unit 48. This figure shows an example of the generation and transmission of the transmission signal Tx by the column signal processing circuit 45. This figure shows an example of the generation and transmission of the transmission signal Tx by the column signal processing circuit 45. This figure shows an example of the generation and transmission of the transmission signal Tx by the column signal processing circuit 45. This figure shows an example of the generation and transmission of the transmission signal Tx by the column signal processing circuit 45. This figure shows an example of generating and transmitting a transmission signal Tx by the column signal processing circuit 45. This figure shows an example of the schematic configuration of the sensor system 1. This figure shows an example of the operation (detection method) of the image sensor 4-2 and DVS 4-3. This figure shows an example of the processing (detection method) performed in the sensor system 1.

[0008] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same elements will be denoted by the same reference numerals to avoid redundant descriptions.

[0009] This disclosure will be described in the following order of items: 0. Introduction 1. Embodiments 2. Modifications 3. Conclusion

[0010] 0. Introduction In the field of imaging technology, for example, to synthesize images, only images of a region of interest (ROI) within the imaging range are sometimes used. There is still room for improvement in techniques for identifying regions of interest and efficiently acquiring only their images.

[0011] According to the disclosed technology, a Z-sensor, which is a sensor for image plane phase difference detection, identifies the location of a region of interest, and the image sensor captures the region of interest. The Z-sensor directly transmits a signal to the image sensor to identify the region of interest. During this process, no processing such as analysis of the Z-sensor's detection results is involved, thereby reducing processing load and power consumption. Signal transmission from the Z-sensor to the image sensor occurs on the order of pixel row drive time. For example, significantly lower latency is achieved compared to general image plane phase difference detection. Even when the region of interest moves, it can be accurately captured.

[0012] 1. Figure 1 of the embodiment shows an example of the schematic configuration of the sensor system 1 according to the embodiment. The sensor system 1 detects (senss) objects, etc. within its sensing range. The objects, etc. to be detected are collectively referred to as subject 9 and are shown in the figure. The sensor system 1 detects a region of interest in particular within the subject 9. The region of interest is referred to as region of interest 90 and is shown in the figure. An example of subject 9 will be explained with reference to Figure 2 as well.

[0013] Figure 2 shows an example of subject 9. In this example, subject 9 is part of a studio set and includes various objects such as the floor, ceiling, windows, and walls. The region of interest 90 is the moderator standing near the main table. Two commentators are also seated near a table separate from the main table. Of these three people, only the moderator is extracted as the region of interest 90, and its image is acquired.

[0014] Returning to Figure 1, the sensor system 1 includes an optical system 2, an optical system 3, a plurality of sensors 4, an information processing device 5, and a focus adjustment device 6. Note that the information may be understood to include data, and these may be reinterpreted as appropriate within a range that does not contradict each other.

[0015] Optical systems 2 and 3 guide light from the subject 9 to each of the multiple sensors 4. More specifically, optical system 2 captures light from the subject 9 and guides it to optical system 3. Optical system 2 may include a lens, in which case it focuses the light from the subject 9 and guides it to optical system 3. Note that the light after passing through optical system 2 may also continue to be referred to as light from the subject 9.

[0016] The optical system 3 guides light from the subject 9 to each of the multiple sensors 4. The specific configuration of the optical system 3 is not particularly limited, and various known optical elements such as prisms and mirrors may be used. In the example shown in Figure 1, there are two sensors 4 as the multiple sensors 4. A portion of the light from the subject 9 enters one of the sensors 4. The remaining portion of the light from the subject 9 enters the other sensor 4. The optical distance from the optical system 2 through the optical system 3 to each of the multiple sensors 4 may be the same for all of them.

[0017] Each of the multiple sensors 4 is a light detection sensor that detects light from the subject 9. The multiple sensors 4 include a Z sensor 4-1 and an image sensor 4-2. Unless otherwise specified, they are simply referred to as sensors 4 or each of the sensors 4.

[0018] The Z sensor 4-1 is a phase difference detection sensor. Based on the light from the subject 9, the Z sensor 4-1 detects the amount of focus shift (defocus amount) at each position of the subject 9. Phase difference is also called image plane phase difference, etc.

[0019] The image sensor 4-2 is an imaging sensor (for example, an RGB sensor). The image sensor 4-2 captures an image of the subject 9 based on the light from the subject 9.

[0020] The Z sensor 4-1 and the image sensor 4-2 are electrically connected to each other so that at least the Z sensor 4-1 can transmit a signal to the image sensor 4-2. The signal that the Z sensor 4-1 transmits to the image sensor 4-2 is referred to as the transmission signal Tx. For example, the Z sensor 4-1 and the image sensor 4-2 are mounted on a substrate (not shown), and the transmission signal Tx is transmitted via wiring provided on that substrate.

[0021] Of the information processing device 5 and the focus adjustment device 6, the focus adjustment device 6 will be described first. The focus adjustment device 6 adjusts the focus of the sensor 4. The method of focus adjustment is not particularly limited. For example, the focus may be adjusted by driving the optical system 2, Z sensor 4-1, image sensor 4-2, etc., to move them along the optical axis. In this embodiment, the focus adjustment device 6 is used in particular to focus the Z sensor 4-1 and image sensor 4-2 on the region of interest 90 of the subject 9.

[0022] The information processing device 5 processes the data obtained from the sensor 4 based on its detection results and controls the focus adjustment device 6. For example, the information processing device 5 identifies a region of interest 90 in the subject 9 based on the detection results of the image sensor 4-2, more specifically, the image (data) of the subject 9. Various image recognition processing techniques may be used. The information processing device 5 also controls the focus adjustment device 6 based on the detection results of the Z sensor 4-1. More specifically, the information processing device 5 controls the focus adjustment device 6 so that the Z sensor 4-1 and the image sensor 4-2 focus on the region of interest 90 based on the amount of defocus at each position of the subject 9, in particular the amount of defocus of the region of interest 90 identified as described above.

[0023] Many of the sensors 4 share common configurations. An example of a basic configuration will be explained with reference to Figure 3.

[0024] Figure 3 shows an example of the schematic configuration of sensor 4. Sensor 4 is a photodetector based on, for example, CMOS (Complementary Metal Oxide Semiconductor) technology, and each element is provided on a semiconductor substrate. One sensor 4 may be a single semiconductor chip.

[0025] Sensor 4 includes a pixel array section 40 and also includes peripheral circuits and signal lines. Peripheral circuits include a control circuit 42, a vertical drive circuit 43, a DAC circuit 44, a column signal processing circuit 45, a horizontal drive circuit 46, and an output circuit 47, all of which are indicated by reference numerals. Signal lines HL, VL, and L are indicated by reference numerals.

[0026] The pixel array section 40 includes a plurality of pixels 41. The plurality of pixels 41 are arranged in a matrix in a two-dimensional array. In Figure 3, the directions in the pixel array section 40 are shown as the V direction and the H direction. The V direction corresponds to the column direction of the array, for example, the vertical direction of the sensor 4. The H direction corresponds to the row direction of the array, for example, the horizontal direction of the sensor 4.

[0027] Pixel 41 includes a photoelectric conversion unit. The photoelectric conversion unit generates an electric charge corresponding to the amount of incident light. An example of a photoelectric conversion unit is a photodiode (PD). A circuit is also provided that generates and outputs a voltage signal corresponding to the amount of charge generated by the photoelectric conversion unit. Pixel 41 can be understood to include such a circuit and can also be called a pixel circuit. The voltage signal output by the pixel circuit is called the pixel signal S.

[0028] Within the bounds of consistency, the pixel array section 40 and the multiple pixels 41 may be appropriately reinterpreted.

[0029] The control circuit 42 receives data that commands the input clock, operating mode, etc., and outputs data such as internal information of the sensor 4. Based on the vertical synchronization signal, horizontal synchronization signal, and master clock, the control circuit 42 generates clock signals and control signals that serve as the reference for the operation of the vertical drive circuit 43, DAC circuit 44, column signal processing circuit 45, and horizontal drive circuit 46, etc. The control circuit 42 supplies these generated signals to the vertical drive circuit 43, DAC circuit 44, column signal processing circuit 45, and horizontal drive circuit 46, etc.

[0030] The vertical drive circuit 43 is configured to include, for example, a shift register. The vertical drive circuit 43 is connected to the pixel array section 40 via a plurality of signal lines HL (horizontal signal lines) that extend in the pixel row direction (H direction). Each signal line HL extends, for example, for each pixel row, and each signal line HL may include multiple signal lines. The vertical drive circuit 43 supplies a drive signal (for example, a pulse signal) for driving the pixels 41 to a selected signal line HL.

[0031] The driving of the pixels 41 by the vertical drive circuit 43 includes driving a pixel transistor (not shown). The pixel transistor is driven to output a voltage signal, i.e., a pixel signal S, corresponding to the amount of charge generated in the photoelectric conversion unit within the pixel 41, to the corresponding signal line VL (vertical signal line) among a plurality of signal lines VL (vertical signal lines) extending in the pixel column direction (V direction).

[0032] The DAC circuit 44 generates a sawtooth waveform RAMP signal through DA (Digital to Analog) conversion. The DAC circuit 44 supplies the generated RAMP signal to the column signal processing circuit 45.

[0033] The column signal processing circuit 45 processes the pixel signal S for each pixel column. The column signal processing circuit 45 is connected to the pixel array section 40 via signal lines VL. Each signal line VL may contain multiple signal lines. The column signal processing circuit 45 is arranged, for example, for each pixel column, and performs signal processing such as noise reduction on the pixel signal S from one row of pixels 41 for each pixel column. The column signal processing circuit 45 performs signal processing such as CDS (Correlated Double Sampling), signal amplification, and AD (Analog to Digital) conversion to remove fixed pattern noise specific to the pixels 41.

[0034] The horizontal drive circuit 46 is configured to include, for example, a shift register. The horizontal drive circuit 46 sequentially outputs horizontal scanning pulses, thereby sequentially selecting each of the column signal processing circuits 45 and causing each of the column signal processing circuits 45 to output a pixel signal S to the signal line L.

[0035] The output circuit 47 processes the pixel signals S that are sequentially supplied from each of the column signal processing circuits 45 through the signal lines L and outputs the result. For example, buffering, black level adjustment, column variation correction, and various digital signal processing may be performed.

[0036] The sensor 4 further includes a logic unit 48. The logic unit 48 is located after the column signal processing circuit 45, in this example after the output circuit 47, and performs various logic operations. The logic unit 48 may be implemented by a processor or the like.

[0037] For example, the above configuration can be a common configuration for each sensor 4, namely the Z sensor 4-1 and the image sensor 4-2. To distinguish between the Z sensor 4-1 and the image sensor 4-2, the pixels 41 and pixel signals S of the Z sensor 4-1 will also be referred to as the image sensor phase-difference pixels 41z and image sensor phase-difference pixel signals Sz (see Figure 9 below). The pixels 41 and pixel signals S of the image sensor 4-2 will also be referred to as the image pixels 41m and image sensor pixel signals Sm (see Figure 10 below).

[0038] Let's further describe the Z sensor 4-1. In the Z sensor 4-1, each of the multiple image plane phase difference pixels 41z includes a pair of image plane phase difference pixels 41z in which different regions are shielded from light.

[0039] For example, in a pair of image plane phase-difference pixels 41z, one image plane phase-difference pixel 41z has its left half region, that is, the half region on the negative H direction side of that image plane phase-difference pixel 41z, shielded from light. This image plane phase-difference pixel 41z is referred to as image plane phase-difference pixel 41z-L. The other image plane phase-difference pixel 41z has its right half region, that is, the half region on the positive H direction side of that image plane phase-difference pixel 41z, shielded from light. This image plane phase-difference pixel 41z is referred to as image plane phase-difference pixel 41z-R.

[0040] A pair of image plane phase-difference pixels 41z as described above are arranged, for example, throughout the entire pixel array section 40. When no particular distinction is made between image plane phase-difference pixels 41z-L and image plane phase-difference pixels 41z-R, they are simply referred to as image plane phase-difference pixels 41z or a pair of image plane phase-difference pixels 41z.

[0041] In the Z sensor 4-1, the difference value of the respective image plane phase difference pixel signals Sz of a pair of image plane phase difference pixels 41z is calculated. This difference value is referred to as the difference value d. This difference value d indicates the amount of defocus. The calculation of the difference value d is performed, for example, by the column signal processing circuit 45 or the logic operation unit 48, and the details will be described later.

[0042] The difference value d calculated for each phase-difference pixel 41z on the imaging plane can become the detection result of the Z sensor 4-1. See also Figure 4 for further explanation.

[0043] FIG. 4 is a diagram showing an example of the detection result of the Z sensor 4-1. The pixel array unit 40 and the subject 9 are shown in association with each other. A portion where the magnitude of the difference value d is small is shown in black.

[0044] Here, it is assumed that the Z sensor 4-1 is focused on the region of interest 90 in the subject 9. The magnitude of the difference value d at the position corresponding to the region of interest 90 in the pixel array unit 40, that is, the position of the imaging surface phase difference pixel 41z that detects light from the region of interest 90, becomes small. The magnitude of the difference value d at other positions becomes large. Based on the magnitude of the difference value d at each position in the pixel array unit 40, that is, based on the detection result of the Z sensor 4-1, the position, range, etc. of the region of interest 90 in the pixel array unit 40 can be specified.

[0045] In one embodiment, the transmission signal Tx may be a signal that specifies a pixel column in which the magnitude (absolute value) of the difference value d is within a threshold range. The threshold is referred to as a threshold dth (FIG. 11 etc. described later). In the threshold determination, for example, it is determined whether the magnitude of the difference value d is within the range of the threshold dth, more specifically, whether it is less than or equal to the threshold dth (|d| ≤ dth?). |d| indicates the absolute value of the difference value d. The transmission signal Tx specifies a pixel column in which the magnitude of the difference value d is less than or equal to the threshold dth.

[0046] In the present embodiment, every time the Z sensor 4-1 reads out the imaging surface phase difference pixel signal Sz for one row, the transmission signal Tx generated based on the read imaging surface phase difference pixel signal Sz is transmitted to the image sensor 4-2. The transmission signal Tx is a signal for specifying the pixel column corresponding to the region of interest 90 among each pixel column of that row. The image sensor 4-2 processes the imaging pixel signal Sm based on the transmission signal Tx from the Z sensor 4-1. This will also be described with reference to FIG. 5.

[0047] FIG. 5 is a diagram showing an example of processing of the imaging pixel signal Sm in the image sensor 4-2. The image sensor 4-2 processes the imaging pixel signal Sm specified by the transmission signal Tx from the Z sensor 4-1 among the imaging pixel signals Sm for one row. For example, the image sensor 4-2 processes the imaging pixel signals Sm of the pixel columns in which the magnitude of the difference value d is within the range of the threshold value dth among the imaging pixel signals Sm for one row. In this example, only the imaging pixel signals Sm of the pixel columns corresponding to the region of interest 90 are processed. The other imaging pixel signals Sm are not processed. Only the region of interest 90 in the subject 9 is imaged, and its image is obtained.

[0048] FIG. 6 is a diagram showing an example of processing (detection method) executed in the sensor system 1. The row selection signal XHS schematically shows a drive signal (pulse signal) for selecting a pixel row. STANDBY of the sensor 4 and the focus adjustment device 6 indicates that they are in the standby state. The standby sensor 4 does not read out or process pixel signals. The standby focus adjustment device 6 does not perform focus adjustment.

[0049] <Initial Focus Adjustment> Focus adjustment for focusing the Z sensor 4-1 and the image sensor 4-2 on the region of interest 90 in the subject 9 is performed at times t1 to t3.

[0050] At times t1 to t2, the Z sensor 4-1 and the focus adjustment device 6 are in the standby state. The image sensor 4-2 images the subject 9. That is, the image sensor 4-2 reads out and processes the imaging pixel signals Sm of all the imaging pixels 41m to generate an image of the subject 9. Based on the generated image, the information processing device 5 identifies the region of interest 90 in the subject 9.

[0051] Between times t2 and t3, the Z sensor 4-1 detects a phase difference. The Z sensor 4-1 reads and processes the image plane phase difference pixel signals Sz from all image plane phase difference pixels 41z and calculates the difference value d for each position of the subject 9. Based on the calculated difference value d, the information processing device 5 controls the focus adjustment device 6 so that the Z sensor 4-1 and the image sensor 4-2 focus on the region of interest 90 in the subject 9. Focus adjustment is performed by the focus adjustment device 6.

[0052] <Pixel row-by-pixel operation> After the focus adjustment device 6 focuses the Z sensor 4-1 and the image sensor 4-2 on the region of interest 90, the pixel row-by-pixel operation described above is performed. The focus adjustment device 6 may be in a standby state.

[0053] The Z sensor 4-1 reads and processes the image plane phase difference pixel signal Sz for each pixel row and transmits a transmission signal Tx. Each time a row of image plane phase difference pixel signals Sz is read, the transmission signal Tx generated based on that image plane phase difference pixel signal Sz is transmitted from the Z sensor 4-1 to the image sensor 4-2.

[0054] The image sensor 4-2 processes the image pixel signal Sm for each pixel row based on the transmission signal Tx. Of the image pixel signals Sm for one row, the image pixel signal Sm identified by the transmission signal Tx from the image sensor 4-2 is processed. An image of one row of the region of interest 90 in the subject 9 is generated.

[0055] The above pixel row operations by the Z sensor 4-1 and the image sensor 4-2 are repeatedly performed, thereby efficiently generating an image of the region of interest 90 in the subject 9.

[0056] The information processing device 5 processes the image data generated by the image sensor 4-2. Various image processing operations are performed on the region of interest 90 in the subject 9.

[0057] For example, as described above, after the Z sensor 4-1 and image sensor 4-2 focus on the region of interest 90, the Z sensor 4-1 can identify (track) the region of interest 90 while the image sensor 4-2 efficiently images the region of interest 90. During this time, the information processing device 5 does not need to process the detection results of the Z sensor 4-1, thereby reducing the processing load and power consumption of the information processing device 5. In addition, the Z sensor 4-1 transmits a transmission signal Tx to the image sensor 4-2 each time it reads out one row of image plane phase difference pixel signals Sz. The time required from reading out to transmitting and receiving the transmission signal Tx is shorter compared to general image plane phase difference sensors. Low latency is achieved. Even when the region of interest 90 moves, it can be accurately imaged.

[0058] The focus may be readjusted at any time after the aforementioned time t3. This will be explained with reference to Figure 7.

[0059] Figure 7 shows an example of the processing (detection method) performed in the sensor system 1. In this example, the focus is readjusted at times t31 to t33 and t34 to t36, after time t3. The operation is the same as that described earlier in Figure 6 from time t1 to t3, so the explanation will not be repeated.

[0060] By readjusting the focus at any time after time t3, even if the focus has shifted from the initial focus adjustment state (times t1 to t3 in Figure 6), the Z sensor 4-1 and image sensor 4-2 can be refocused on the region of interest 90. For example, this can address focus shifts that may occur due to changes in the distance from the sensor system 1 (e.g., optical system 2) to the region of interest 90.

[0061] <Example of generation of transmission signal Tx and transmission entity> As mentioned above, the column signal processing circuit 45 or logic unit 48 of the Z sensor 4-1 can calculate the difference value d. Therefore, the column signal processing circuit 45 or logic unit 48 may generate a transmission signal Tx based on the difference value d and transmit it to the image sensor 4-2. The logic unit 48 and the column signal processing circuit 45 will be described in that order.

[0062] <Example of generation and transmission of transmission signal Tx by the logic unit 48> Figures 8 to 12 show an example of generation and transmission of transmission signal Tx by the logic unit 48. As shown in Figure 8, the transmission signal Tx is transmitted from the logic unit 48 of the Z sensor 4-1 to the image sensor 4-2. The logic unit 48 of the Z sensor 4-1 calculates the difference value d, performs a threshold determination on the difference value d, and transmits the transmission signal Tx generated based on the determination result to the image sensor 4-2.

[0063] The logic unit 48 may be designed to perform various logic operations, thereby enabling the generation and transmission of transmission signals Tx in various forms. For example, it can also identify the column address (address value) of a pixel sequence corresponding to the region of interest 90. In one embodiment, the transmission signal Tx transmitted by the logic unit 48 may describe the column address of a pixel sequence whose difference value d is within the range of a threshold dth. The value of the column address is referred to as the address value H.

[0064] Figure 9 illustrates eight of the two rows and eight columns of image-plane phase-difference pixels 41z in the pixel array section 40 of the Z-sensor 4-1, with each pixel labeled accordingly. The column address values ​​of each pixel column are referred to as address values ​​H0 to H7. When no particular distinction is made between them, they are simply referred to as address value H or each address value H.

[0065] The image plane phase difference pixel signals Sz of the image plane phase difference pixels 41z with address values ​​H0 to H7 are referred to as image plane phase difference pixel signals Sz0 to Sz7. When not specifically distinguished, they are simply called image plane phase difference pixel signals Sz or each image plane phase difference pixel signal Sz.

[0066] Figure 9 schematically shows four pairs of image plane phase-difference pixels 41z. Specifically, the image plane phase-difference pixel 41z-L with address value H0 and the image plane phase-difference pixel 41z-R with address value H2 constitute a pair of image plane phase-difference pixels 41z. The image plane phase-difference pixel 41z-L with address value H1 and the image plane phase-difference pixel 41z-R with address value H3 constitute a pair of image plane phase-difference pixels 41z. The image plane phase-difference pixel 41z-L with address value H4 and the image plane phase-difference pixel 41z-R with address value H6 constitute a pair of image plane phase-difference pixels 41z. The image plane phase-difference pixel 41z-L with address value H5 and the image plane phase-difference pixel 41z-R with address value H7 constitute a pair of image plane phase-difference pixels 41z.

[0067] The column signal processing circuit 45 includes a signal processing circuit C for each column. The signal processing circuits C corresponding to address values ​​H0 to H7 are referred to as signal processing circuits C0 to C7 and are shown in the diagram. Unless otherwise specified, they are simply called signal processing circuit C or each signal processing circuit C.

[0068] The image plane phase difference pixel signals Sz from each image plane phase difference pixel 41z are processed by the corresponding signal processing circuit C and supplied to the logic operation unit 48 via the output circuit 47 (Figure 3).

[0069] The logic unit 48 calculates the difference value d of the image plane phase difference pixel signals Sz of a pair of image plane phase difference pixels 41z. The difference value d of image plane phase difference pixel signals Sz0 and Sz2 is referred to as difference value d02. The difference value d of image plane phase difference pixel signals Sz1 and Sz3 is referred to as difference value d13. The difference value d of image plane phase difference pixel signals Sz4 and Sz6 is referred to as difference value d46. The difference value d of image plane phase difference pixel signals Sz5 and Sz7 is referred to as difference value d57. When these are not specifically distinguished, they are simply called difference value d or each difference value d.

[0070] The logical operation unit 48 performs a threshold check on the calculated difference value d. Specifically, it determines whether the magnitude of the difference value d is within the range of the threshold dth (|d| ≤ dth). In this example, the magnitude of the difference value d02 is within the range of the threshold dth (|d02| ≤ dth). The magnitude of the difference value d13 is within the range of the threshold dth (|d13| ≤ dth). The magnitude of the difference value d46 is outside the range of the threshold dth (|d46| > dth). The magnitude of the difference value d57 is outside the range of the threshold dth (|d57| > dth).

[0071] The logic unit 48 generates a transmission signal Tx that identifies a sequence of pixels whose difference value d is within the threshold dth range. In this example, the transmission signal Tx is generated to identify the sequence of pixels from address value H0 to address value H3. For example, the transmission signal Tx may be a signal that describes whether or not the difference value d is within the threshold dth range for each address value H. Alternatively, the transmission signal Tx may be a signal that describes only the address values ​​H (in this example, address values ​​H0 to address values ​​H3) whose difference value d is within the threshold dth range. The logic unit 48 transmits the generated transmission signal Tx to the image sensor 4-2.

[0072] Figure 10 illustrates eight imaging pixels 41m of the 2 rows and 8 columns of imaging pixels 41m in the pixel array section 40 of the image sensor 4-2, with each pixel being labeled with an index. The imaging pixel signals Sm of the imaging pixels 41m with address values ​​H0 to H7 are referred to as imaging pixel signals Sm0 to Sm7. When not specifically distinguished, they are referred to as imaging pixel signals Sm or each imaging pixel signal Sm.

[0073] The imaging pixel signals Sm from each imaging pixel 41m are processed by the corresponding signal processing circuit C of the column signal processing circuit 45 and supplied to the logic operation unit 48 via the output circuit 47 (Figure 3).

[0074] The logic unit 48 processes the imaging pixel signals Sm of the imaging pixels 41m of the pixel sequence identified by the transmitted signal Tx. In this example, imaging pixel signals Sm0 to Sm3 are read out and processed by the logic unit 48. An image based on imaging pixel signals Sm0 to Sm3, i.e., image data of the region of interest 90, is obtained.

[0075] Figure 11 shows an example of the processing (sensing method) performed in sensor 4. Processing is carried out based on the clock signal CLK for the pixel row selected by the row selection signal XHS.

[0076] The logic unit 48 of the Z sensor 4-1 calculates the difference value d and determines the threshold. As mentioned earlier, in this case, the magnitudes of difference values ​​d02 and d13 are within the range of the threshold dth, while the magnitudes of difference values ​​d46 and d57 are outside the range of the threshold dth. The logic unit 48 generates a transmission signal Tx that identifies these values ​​and transmits it to the image sensor 4-2.

[0077] The logic unit 48 of the image sensor 4-2 receives a transmission signal Tx from the Z sensor 4-1 and processes the image pixel signals Sm of the pixel sequence identified by the transmission signal Tx. In this example, the pixel sequence from address value H0 to address value H3 is selected (pixel sequence selection: Yes), and image pixel signals Sm0 to image pixel signals Sm3 are processed. The pixel sequence from address value H4 to address value H7 is not selected (pixel sequence selection: No), and image pixel signals Sm4 to image pixel signals Sm7 (Figure 10) are not processed.

[0078] Figure 12 schematically shows the operation of the Z sensor 4-1. The row address value is referred to as the address value V. Some of the address values ​​V from row 0 to row m are shown as address value V0, address value V1, address value Vi, address value Vi+1, address value Vi+2, address value Vi+3, address value Vi+4, address value Vi+5, address value Vm-1, and address value Vm.

[0079] In each pixel row, the magnitude of the difference value d of the pixel sequence corresponding to the region of interest 90 falls within the threshold dth range. In this example, in the pixel rows from address value Vi to address value Vi+5, there are pixel sequences whose difference value d is within the threshold dth range. A signal identifying the address value H of those pixel sequences is transmitted as a transmission signal Tx from the Z sensor 4-1 to the image sensor 4-2.

[0080] In one embodiment, the transmitted signal Tx may identify whether the pixel row corresponds to the region of interest 90. For example, the transmitted signal Tx may also describe the row address of a pixel row in which there are pixel sequences in which the magnitude of the difference value d is within the threshold dth range. In this case, the logic unit 48 of the Z sensor 4-1 determines, for each pixel row, whether the number of pixel sequences in which the magnitude of the difference value d is within the threshold dth range is greater than or equal to a predetermined number. Pixel rows in which the number of such pixel sequences is greater than or equal to a predetermined number are identified by the transmitted signal Tx. The predetermined number may be any integer greater than or equal to 1. Setting the predetermined number to 2 or a number greater than that increases the likelihood of avoiding uncertainty in pixel row determination that may arise due to noise, etc.

[0081] For example, as described above, the logic unit 48 of the Z sensor 4-1 can transmit a transmission signal Tx to the image sensor 4-2.

[0082] <Example of generation and transmission of transmission signal Tx by column signal processing circuit 45> Figures 13 to 17 show an example of generation and transmission of transmission signal Tx by column signal processing circuit 45. As shown in Figure 13, the transmission signal Tx is transmitted from the column signal processing circuit 45 of the Z sensor 4-1 to the image sensor 4-2. The column signal processing circuit 45 of the Z sensor 4-1 calculates the difference value d, performs a threshold determination on the difference value d, and transmits the transmission signal Tx generated based on the determination result to the image sensor 4-2.

[0083] The transmission signal Tx transmitted by the column signal processing circuit 45 may include a plurality of determination signals J. The determination signal J indicates whether the magnitude of the difference value d of the corresponding pixel sequence is within the range of the threshold dth. The determination signal J may be a 1-bit signal, in which case the determination signal J is an H-level signal with an H (high) level voltage or an L-level signal with an L (low) level voltage.

[0084] In the following explanation, if the magnitude of the difference value d is within the range of the threshold dth, the decision signal J will be assumed to be an H-level signal. If the magnitude of the difference value d is outside the range of the threshold dth, the decision signal J will be assumed to be an L-level signal.

[0085] As shown in Figure 14, the column signal processing circuit 45 of the Z sensor 4-1 includes a signal processing circuit C and a transmission circuit 450.

[0086] Each signal processing circuit C of the column signal processing circuit 45 outputs the image plane phase difference pixel signal Sz of the image plane phase difference pixel 41z of its pixel column, and also generates and outputs the judgment signal J described above.

[0087] The judgment signal J indicates the result of a threshold judgment on the difference value d, and can therefore be called a signal based on the difference value d. The judgment signal J based on the difference value d02 between the image plane phase difference pixel signals Sz0 and Sz2 is called judgment signal J02. The judgment signal J based on the difference value d13 between the image plane phase difference pixel signals Sz1 and Sz3 is called judgment signal J13. The judgment signal J based on the difference value d46 between the image plane phase difference pixel signals Sz4 and Sz6 is called judgment signal J46. The judgment signal J based on the difference value d57 between the image plane phase difference pixel signals Sz5 and Sz7 is called judgment signal J57. When these are not specifically distinguished, they are simply called judgment signal J or each judgment signal J.

[0088] Signal processing circuit C0 outputs the image plane phase difference pixel signal Sz0 and the determination signal J02. Signal processing circuit C1 outputs the image plane phase difference pixel signal Sz1 and the determination signal J13. Signal processing circuit C2 outputs the image plane phase difference pixel signal Sz2 and the determination signal J02. Signal processing circuit C3 outputs the image plane phase difference pixel signal Sz3 and the determination signal J13. Signal processing circuit C4 outputs the image plane phase difference pixel signal Sz4 and the determination signal J46. Signal processing circuit C5 outputs the image plane phase difference pixel signal Sz5 and the determination signal J57. Signal processing circuit C6 outputs the image plane phase difference pixel signal Sz6 and the determination signal J46. Signal processing circuit C7 outputs the image plane phase difference pixel signal Sz7 and the determination signal J57. A more specific configuration of signal processing circuit C will be explained later with reference to Figure 16.

[0089] The phase-difference pixel signal Sz and the determination signal J, both output by the signal processing circuit C, are supplied to the transmission circuit 450. Multiple determination signals J, equal in number to the number of pixel rows, are supplied from the signal processing circuit C to the transmission circuit 450.

[0090] The transmission circuit 450 transmits a transmission signal Tx containing multiple judgment signals J to the image sensor 4-2. For example, the multiple judgment signals J are transmitted one by one in sequence (serial transmission). An example of the configuration of the transmission circuit 450 will be explained later with reference to Figure 17.

[0091] As shown in Figure 15, the column signal processing circuit 45 of the image sensor 4-2 includes a receiving circuit 455 and a signal processing circuit C.

[0092] The receiving circuit 455 receives the transmission signal Tx from the transmission circuit 450 (Figure 14) of the Z sensor 4-1. In this example, the receiving circuit 455 is a 2-input, 1-output AND gate. The transmission signal Tx, i.e., the determination signal J, is input to one input terminal of the receiving circuit 455. The row selection signal XHS is input to the other input terminal of the receiving circuit 455. The receiving circuit 455 outputs the logical AND signal of these two input signals. When row selection is performed, the receiving circuit 455 outputs the same signal as the determination signal J (L-level signal or H-level signal).

[0093] The output signal of the receiving circuit 455 is supplied to the corresponding signal processing circuit C. The determination signal J02 in the transmitted signal Tx is supplied to signal processing circuits C0 and C2. The determination signal J13 is supplied to signal processing circuits C1 and C3. The determination signal J46 is supplied to signal processing circuits C4 and C6. The determination signal J57 is supplied to signal processing circuits C5 and C7.

[0094] Of the signal processing circuits C, the signal processing circuit C supplied with the high-level signal, which is the determination signal J, processes the corresponding image pixel signal Sm of the image pixel 41m. Image pixel signals Sm0 to Sm3 are processed by signal processing circuits C0 to C3. An image based on image pixel signals Sm0 to Sm3, i.e., an image of the region of interest 90 in the subject 9, is obtained.

[0095] Figure 16 shows an example of the schematic configuration of the column signal processing circuit 45 of the Z sensor 4-1. Signal processing circuit C0 and signal processing circuit C2 are exemplified as a pair of signal processing circuits C corresponding to a pair of image plane phase difference pixels 41z. Each of signal processing circuit C0 and signal processing circuit C2 includes a full adder 72, a temporary latch 73, and a column IF latch 74.

[0096] The full adder 72 receives bit data (a signal of bit data) indicating the level of the image plane phase difference pixel signal Sz2 of the corresponding image plane phase difference pixel 41z. Before this, processing such as AD conversion may be performed. The bit data indicating the level of the image plane phase difference pixel signal Sz is also simply called the bit data of the image plane phase difference pixel signal Sz.

[0097] The full adder 72 can also receive bit data from the temporary latch 73. When only the bit data of the image plane phase difference pixel signal Sz is input to the full adder 72, that bit data is output directly from the full adder 72. When both the bit data of the image plane phase difference pixel signal Sz and the bit data from the temporary latch 73 are input to the full adder 72, bit data indicating the result of subtracting them is output from the full adder 72.

[0098] A temporary latch 73 and a column IF latch 74 are provided downstream of the full adder 72. The temporary latch 73 and the column IF latch 74 hold the bit data output by the full adder 72. The temporary latch 73 can output the bit data it holds to the full adder 72. In this example, the temporary latch 73 can hold 11 bits of bit data. The column IF latch 74 can hold 12 bits of bit data.

[0099] First, the bit data of the pixel signal from the non-exposed phase-difference pixel 41z on the image sensor is input to the full adder 72. This bit data is output directly from the full adder 72 and held by the temporary latch 73. Next, the bit data of the image sensor phase-difference pixel signal Sz from the exposed image sensor phase-difference pixel 41z is input to the full adder 72, and the bit data held by the temporary latch 73 is also input to the full adder 72. Bit data showing the result of subtracting these is output from the full adder 72 and held by the column IF latch 74. This bit data is output as the image sensor phase-difference pixel signal Sz after processing by the signal processing circuit C. This processing can correspond to the CDS described earlier.

[0100] The difference value d02 between the image plane phase difference pixel signal Sz0 processed by the signal processing circuit C0 and the image plane phase difference pixel signal Sz2 processed by the signal processing circuit C2 is calculated, and a threshold determination is also performed on this value. The selector 71 and the determination circuit 75 are shown in the figure with reference numerals as components for this purpose.

[0101] In this example, the selector 71 is located before the full adder 72 of the signal processing circuit C0. In addition to the bit data of the image plane phase difference pixel signal Sz0, the selector 71 is also supplied with the bit data held by the column IF latches 74 of the signal processing circuits C0 and C2, respectively. The necessary wiring for this purpose is provided and connected between the signal processing circuits C0 and C2.

[0102] The selector 71 selectively supplies the input bit data to the full adder 72 according to a given selection signal. Specifically, when calculating the difference value d02, the selector 71 supplies the bit data from each column IF latch 74 of signal processing circuit C0 and signal processing circuit C2 to the full adder 72. The bit data held by one column IF latch 74 of signal processing circuit C0 and signal processing circuit C2 is subtracted from the bit data held by the other column IF latch 74. Bit data indicating the subtraction result is output from the full adder 72 and overwrites the column IF latch 74 of signal processing circuit C0. This bit data corresponds to the difference value d02.

[0103] The determination circuit 75 performs a threshold determination on the difference value d and generates and outputs a determination signal J indicating the determination result. In this example, the determination circuit 75 determines whether the magnitude of the difference value d02 is within the range of the threshold dth and generates and outputs a determination signal J02 indicating the result. This determination signal J02 becomes the output of signal processing circuit C0 and signal processing circuit C2, respectively.

[0104] The same applies to other pairs of signal processing circuits C, such as signal processing circuits C1 and C3, C4 and C6, and C5 and C7.

[0105] Figure 17 shows an example of the schematic configuration of the transmission circuit 450. In this example, the transmission circuit 450 is a shift register circuit that sequentially transfers multiple decision signals J. Multiple flip-flop circuits 451 are shown as components of the shift register circuit, indicated by reference numerals.

[0106] The same number of flip-flop circuits 451 as the number of signal processing circuits C included in the column signal processing circuit 45 are connected in a cascode configuration. Each flip-flop circuit 451 is selectively input either a decision signal J from the corresponding signal processing circuit C or a signal from the preceding flip-flop circuit 451.

[0107] Note that the first-stage flip-flop circuit 451 does not receive any signal input from the preceding flip-flop circuit 451. The output signal of the final-stage flip-flop circuit 451 is supplied to the receiving circuit 455 (Figure 15) of the column signal processing circuit 45 of the image sensor 4-2, as described earlier. The transmission signal Tx is transmitted by sequentially transferring the respective judgment signals J from each signal processing circuit C.

[0108] For example, as described above, the column signal processing circuit 45 of the Z sensor 4-1 can transmit a transmission signal Tx to the image sensor 4-2. This allows for even lower latency than when the transmission signal Tx is transmitted from the logic unit 48 located after the column signal processing circuit 45. The time required for transmitting and receiving the transmission signal Tx is approximately the number of clock cycles equal to the number of pixels in the column direction when using a shift register configuration as shown in Figure 17, which is extremely short compared to typical image plane phase difference.

[0109] 2. Modifications In the above embodiment, a Z sensor 4-1 and an image sensor 4-2 were described as examples of multiple sensors 4. Other sensors may be used. These will be explained with reference to Figures 18 to 20.

[0110] Figure 18 shows an example of the schematic configuration of the sensor system 1. Sensor 4 further includes DVS4-3. DVS4-3 is a dynamic vision sensor (DVS) that detects address events for each pixel and is used to detect changes in the subject 9. DVS is also called an event-based vision sensor (EVS).

[0111] The basic configuration of DVS4-3 is the same as that of Figure 3 described earlier. In DVS4-3, changes in the brightness of each pixel 41 are detected, and the detection results, such as the address value H and address value V of the pixel 41 in which the change was detected, are output.

[0112] The optical system 3 is configured to guide light from the subject 9 to the Z sensor 4-1, the image sensor 4-2, and the DVS 4-3, respectively. The specific configuration is not particularly limited, and various known optical elements such as prisms and mirrors may be used.

[0113] When the DVS 4-3 detects a change in the subject 9, the information processing device 5 identifies the region of interest 90 within the subject 9 based on the detection result of the image sensor 4-2, i.e., the image of the subject 9. As explained earlier with reference to Figure 7, it is not necessary for the image sensor 4-2 to image the subject 9 at regular intervals. This reduces the processing load on the image sensor 4-2. Also, since the DVS 4-3 outputs data in amounts corresponding to the amount of change, the amount of data is small, and the processing burden such as calculations is not very large.

[0114] Figure 19 shows an example of the operation (detection method) of the image sensor 4-2 and DVS 4-3. ACTIVE indicates that the DVS 4-3 is in an active state, more specifically, in a state where it detects changes when they occur in the subject 9. The image sensor 4-2 is in a standby state when the DVS 4-3 does not detect changes in the subject 9, and captures images of the subject 9 when the DVS 4-3 detects changes in the subject 9. The imaging of the subject 9 referred to here corresponds, for example, to the imaging at times t1, t31, and t34 in Figures 6 and 7 described earlier.

[0115] Figure 20 shows an example of the processing (detection method) performed in the sensor system 1. Initially, the DVS 4-3 is in the active state. The Z sensor 4-1, image sensor 4-2, information processing device 5, and focus adjustment device 6 are all in the standby state. At time t11, the DVS 4-3 detects a change in the subject 9. In response, the image sensor 4-2 takes an image of the subject 9. This imaging is started, for example, under the control of the information processing device 5. The operations from time t11 to time t13 and thereafter are the same as the operations from time t1 to time t3 and thereafter in Figure 6 described earlier, so the explanation is omitted.

[0116] 3. Summary The technology described above can be identified, for example, as follows: One of the disclosed technologies is a sensor system 1. As described with reference to Figures 1 to 17, the sensor system 1 comprises a plurality of sensors 4, each detecting light from a subject 9. The plurality of sensors 4 include a Z sensor 4-1 including a plurality of matrix-arranged imaging plane phase difference pixels 41z, and an image sensor 4-2 including a plurality of matrix-arranged imaging pixels 41m. Each time the Z sensor 4-1 reads out one row of imaging plane phase difference pixel signals Sz, it transmits a transmission signal Tx generated based on the read imaging plane phase difference pixel signals Sz to the image sensor 4-2. The image sensor 4-2 processes the imaging pixel signals Sm identified by the transmission signal Tx from the Z sensor 4-1 from among the imaging pixel signals Sm of one row.

[0117] According to the sensor system 1 described above, the transmission signal Tx is directly transmitted from the Z sensor 4-1 to the image sensor 4-2, and the image pixel signal Sm identified by the transmission signal Tx is processed. For example, since the detection result of the Z sensor 4-1 does not need to be processed by an external device such as the information processing device 5, power consumption can be reduced accordingly. In addition, the Z sensor 4-1 transmits the transmission signal Tx to the image sensor 4-2 each time it reads out one row of image plane phase difference pixel signals Sz. For example, compared to a typical image plane phase difference sensor, a significant reduction in latency can be achieved.

[0118] As explained with reference to Figures 3, 4 and 8 to 17, the plurality of image plane phase difference pixels 41z include a pair of image plane phase difference pixels 41z-L and 41z-R whose different regions are shielded from light. The transmitted signal Tx identifies a pixel row in which the magnitude of the difference value d of each image plane phase difference pixel signal Sz of the pair of image plane phase difference pixels 41z-L and 41z-R is within the threshold dth range. The image sensor 4-2 may process the image pixel signals Sm of the pixel row in which the magnitude of the difference value d is within the threshold dth range from one row of image pixel signals Sm. In this way, an image of the region that the sensor 4 is focused on can be efficiently generated.

[0119] As explained with reference to Figures 3 and 8 to 12, the Z sensor 4-1 includes a column signal processing circuit 45 that processes an imaging plane phase difference pixel signal Sz for each pixel column, and a logic operation unit 48 located after the column signal processing circuit 45. The logic operation unit 48 of the Z sensor 4-1 calculates a difference value d, performs a threshold determination on the difference value d, and transmits a transmission signal Tx to the image sensor 4-2. For example, in this way, the transmission signal Tx can be transmitted from the Z sensor 4-1 to the image sensor 4-2. By utilizing the logic operations performed by the logic operation unit 48, various types of transmission signals Tx can be generated and transmitted. For example, the transmission signal Tx may include a signal describing the column address (address value H) of a pixel column in which the magnitude of the difference value d is within the threshold dth range. The transmission signal Tx may also include a signal describing the row address (address value V) of a pixel row in which a pixel column in which the magnitude of the difference value d is within the threshold dth range exists.

[0120] As explained with reference to Figures 3 and 13 to 17, the Z sensor 4-1 includes a column signal processing circuit 45 that processes the image plane phase difference pixel signal Sz for each pixel row. The column signal processing circuit 45 of the Z sensor 4-1 calculates a difference value d, performs a threshold determination on the difference value d, and transmits a transmission signal Tx to the image sensor 4-2. For example, in this way, the transmission signal Tx can be transmitted from the Z sensor 4-1 to the image sensor 4-2. By devising the configuration and signal processing of the column signal processing circuit 45 in the Z sensor 4-1, it becomes possible to generate and transmit the transmission signal Tx. For example, the transmission signal Tx may include a plurality of determination signals J that indicate whether the magnitude of the difference value d of the corresponding pixel row is within the range of a threshold dth. The determination signals J may be 1-bit signals. The column signal processing circuit 45 of the Z sensor 4-1 includes a pair of signal processing circuits C (e.g., signal processing circuit C0 and signal processing circuit C2, etc.) corresponding to the pixel rows of a pair of image plane phase difference pixels 41z-L and image plane phase difference pixels 41z-R, and each of the pair of signal processing circuits C includes a column IF latch 74 that holds bit data indicating the level of the image plane phase difference pixel signal Sz, and the column signal processing circuit 45 of the Z sensor 4-1 may calculate a difference value d by subtracting the bit data held by the column IF latch 74 of the other signal processing circuit C from the bit data held by the column IF latch 74 of one of the pair of signal processing circuits C. The column signal processing circuit 45 of the Z sensor 4-1 may include a determination circuit 75 that performs threshold determination on the difference value d. The column signal processing circuit 45 of the Z sensor 4-1 may include a transmission circuit 450 that transmits a transmission signal Tx to the image sensor 4-2. The transmission circuit 450 may be a shift register circuit that sequentially transfers multiple judgment signals J. For example, with such a configuration, the column signal processing circuit 45 of the Z sensor 4-1 can calculate the difference value d, perform threshold determination, and then transmit the transmission signal Tx to the image sensor 4-2.

[0121] As explained with reference to Figures 1, 6, and 7, the sensor system 1 includes a focus adjustment device 6 that focuses the Z sensor 4-1 and the image sensor 4-2 on the region of interest 90 in the subject 9. After the focus adjustment device 6 focuses the Z sensor 4-1 and the image sensor 4-2 on the region of interest 90, each time the Z sensor 4-1 reads out one row of image plane phase difference pixel signals Sz, it transmits a transmission signal Tx generated based on the read image plane phase difference pixel signals Sz to the image sensor 4-2, and the image sensor 4-2 may process the image pixel signals Sm identified by the transmission signal Tx from the Z sensor 4-1 from the image pixel signals Sm of one row. The transmission signal Tx is a signal for identifying the pixel row corresponding to the region of interest 90, and the image sensor 4-2 may process the image pixel signals Sm of the pixel row corresponding to the region of interest 90 from the image pixel signals Sm of one row. This makes it possible to efficiently image the region of interest 90 with the image sensor 4-2 while the Z sensor 4-1 identifies (tracks) the region of interest 90.

[0122] As explained with reference to Figures 1, 6, and 7, the sensor system 1 includes an information processing device 5 that controls the focus adjustment device 6. The information processing device 5 may control the focus adjustment device 6 based on the detection result of the Z sensor 4-1 so that the Z sensor 4-1 and the image sensor 4-2 focus on the region of interest 90. The information processing device 5 may identify the region of interest 90 in the subject 9 based on the detection result of the image sensor 4-2. For example, in this way, the Z sensor 4-1 and the image sensor 4-2 can be made to focus on the region of interest 90.

[0123] As explained with reference to Figures 18 to 20, the multiple sensors 4 further include a DVS 4-3 that detects changes in the subject 9, and the information processing device 5 may identify a region of interest 90 in the subject 9 based on the detection result of the image sensor 4-2 when the DVS 4-3 detects a change in the subject 9. By using the DVS 4-3, the processing on the image sensor 4-2 can be reduced.

[0124] The effects described in this disclosure are merely illustrative and not limited to those disclosed. Other effects may also occur.

[0125] While embodiments of this disclosure have been described above, the technical scope of this disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of this disclosure. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0126] Furthermore, this technology can also take the following configurations: (1) A sensor system comprising a plurality of sensors, each detecting light from a subject, wherein the plurality of sensors include a Z sensor including a plurality of image plane phase difference pixels arranged in a matrix, and an image sensor including a plurality of image pixels arranged in a matrix, wherein each time the Z sensor reads out a row of image plane phase difference pixel signals, it transmits a transmission signal generated based on the read-out image plane phase difference pixel signals to the image sensor, and the image sensor processes the image pixel signals identified by the transmission signal from the Z sensor from the row of image pixel signals. (2) The sensor system according to (1), wherein the plurality of image plane phase difference pixels include a pair of image plane phase difference pixels in which different regions are shielded from light, the transmission signal identifies a pixel row in which the magnitude of the difference value of each image plane phase difference pixel signal of the pair of image plane phase difference pixels is within a threshold range, and the image sensor processes the image pixel signals of the pixel row in which the magnitude of the difference value is within a threshold range from the row of image pixel signals. (3) The sensor system according to (2), wherein the Z sensor includes a column signal processing circuit that processes the image plane phase difference pixel signals for each pixel row, and a logic operation unit provided after the column signal processing circuit, wherein the logic operation unit of the Z sensor calculates the difference value, performs a threshold determination on the difference value, and transmits the transmission signal to the image sensor. (4) The sensor system according to (3), wherein the transmission signal includes a signal describing the column address of the pixel row in which the magnitude of the difference value is within the threshold range. (5) The sensor system according to (4), wherein the transmission signal includes a signal describing the row address of the pixel row in which the pixel row in which the magnitude of the difference value is within the threshold range exists. (6) The sensor system according to (2), wherein the Z sensor includes a column signal processing circuit that processes the image plane phase difference pixel signals for each pixel row, wherein the column signal processing circuit of the Z sensor calculates the difference value, performs a threshold determination on the difference value, and transmits the transmission signal to the image sensor. (7) The sensor system according to (6), wherein the transmitted signal includes a plurality of determination signals indicating whether the magnitude of the difference value of the corresponding pixel sequence is within a threshold range.(8) The sensor system according to (7), wherein the determination signal is a 1-bit signal. (9) The sensor system according to any one of (6) to (8), wherein the column signal processing circuit of the Z sensor includes a pair of signal processing circuits corresponding to the pixel rows of the pair of image plane phase difference pixels, each of the pair of signal processing circuits includes a column IF latch that holds bit data indicating the level of the image plane phase difference pixel signal, and the column signal processing circuit of the Z sensor calculates the difference value by subtracting the bit data held by the column IF latch of the other signal processing circuit from the bit data held by the column IF latch of one of the pair of signal processing circuits. (10) The sensor system according to any one of (6) to (9), wherein the column signal processing circuit of the Z sensor includes a determination circuit that performs threshold determination on the difference value. (11) The sensor system according to any one of (6) to (10), wherein the column signal processing circuit of the Z sensor includes a transmission circuit that transmits the transmission signal to the image sensor. (12) The sensor system according to (11), wherein the transmission signal includes a plurality of determination signals indicating whether the magnitude of the difference value of each corresponding pixel row is within a threshold range, the determination signal is a 1-bit signal, and the transmission circuit is a shift register circuit that sequentially transfers the plurality of determination signals. (13) The sensor system according to any one of (1) to (12), further comprising a focus adjustment device that focuses the Z sensor and the image sensor on a region of interest in the subject, wherein, after the focus adjustment device has focused the Z sensor and the image sensor on the region of interest, the Z sensor reads out one row of image plane phase difference pixel signals and transmits a transmission signal generated based on the read image plane phase difference pixel signals to the image sensor, and the image sensor processes the image pixel signals identified by the transmission signal from the Z sensor among the row of image pixel signals. (14) The sensor system according to (13), wherein the transmission signal is a signal for identifying a pixel sequence corresponding to the region of interest, and the image sensor processes the image pixel signals of the pixel sequence corresponding to the region of interest from the image pixel signals of one row.(15) The sensor system according to (13) or (14), comprising an information processing device for controlling the focus adjustment device, wherein the information processing device controls the focus adjustment device so that the Z sensor and the image sensor focus on the region of interest based on the detection result of the Z sensor. (16) The sensor system according to (15), wherein the information processing device identifies the region of interest in the subject based on the detection result of the image sensor. (17) The sensor system according to (16), wherein the plurality of sensors further include a DVS for detecting changes in the subject, wherein the information processing device identifies the region of interest in the subject based on the detection result of the image sensor when the DVS detects a change in the subject.

[0127] 1 Sensor system 2 Optical system 3 Optical system 4 Sensor 40 Pixel array section 41 Pixel 41m Imaging pixel 41z Image plane phase difference pixel 41z-L Image plane phase difference pixel 41z-R Image plane phase difference pixel 42 Control circuit 43 Vertical drive circuit 44 DAC circuit 45 Column signal processing circuit 450 Transmitting circuit 451 Flip-flop circuit 455 Receiving circuit 46 Horizontal drive circuit 47 Output circuit 48 Logic operation section 4-1 Z sensor 4-2 Image sensor 4-3 DVS 5 Information processing device 6 Focus adjustment device 71 Selector 72 Full adder 73 Temporary latch 74 Column IF latch 75 Judgment circuit 9 Subject 90 Region of interest C Signal processing circuit S Pixel signal Sm Imaging pixel signal Sz Image plane phase difference pixel signal Tx Transmitted signal

Claims

1. A sensor system comprising a plurality of sensors, each detecting light from a subject, wherein the plurality of sensors include a Z sensor including a plurality of image plane phase difference pixels arranged in a matrix, and an image sensor including a plurality of image pixels arranged in a matrix, wherein the Z sensor transmits a transmission signal generated based on the read image plane phase difference pixel signals to the image sensor each time it reads out a row of image plane phase difference pixel signals, and the image sensor processes the image pixel signals identified by the transmission signal from the Z sensor from a row of image pixel signals.

2. The sensor system according to claim 1, wherein the plurality of image plane phase difference pixels include a pair of image plane phase difference pixels in which different regions are shielded from light, the transmission signal identifies a pixel row in which the magnitude of the difference value of each image plane phase difference pixel signal of the pair of image plane phase difference pixels is within a threshold range, and the image sensor processes the image pixel signals of the pixel row in which the magnitude of the difference value is within a threshold range from the image pixel signals of one row.

3. The sensor system according to claim 2, wherein the Z sensor includes a column signal processing circuit that processes the image plane phase difference pixel signal for each pixel row, and a logic operation unit provided downstream of the column signal processing circuit, the logic operation unit of the Z sensor calculates the difference value, performs a threshold determination on the difference value, and transmits the transmission signal to the image sensor.

4. The sensor system according to claim 3, wherein the transmitted signal includes a signal describing the column address of a pixel sequence in which the magnitude of the difference value is within a threshold range.

5. The sensor system according to claim 4, wherein the transmitted signal includes a signal describing the row address of a pixel row in which a pixel sequence whose magnitude is within a threshold range exists.

6. The sensor system according to claim 2, wherein the Z sensor includes a column signal processing circuit that processes the image plane phase difference pixel signal for each pixel row, and the column signal processing circuit of the Z sensor calculates the difference value, performs a threshold determination on the difference value, and transmits the transmission signal to the image sensor.

7. The sensor system according to claim 6, wherein the transmitted signal includes a plurality of determination signals indicating whether the magnitude of the difference value of the corresponding pixel sequence is within a threshold range.

8. The sensor system according to claim 7, wherein the determination signal is a 1-bit signal.

9. The sensor system according to claim 6, wherein the column signal processing circuit of the Z sensor includes a pair of signal processing circuits corresponding to the pixel rows of the pair of image plane phase difference pixels, each of the pair of signal processing circuits includes a column IF latch that holds bit data indicating the level of the image plane phase difference pixel signal, and the column signal processing circuit of the Z sensor calculates the difference value by subtracting the bit data held by the column IF latch of the other signal processing circuit from the bit data held by the column IF latch of one of the pair of signal processing circuits.

10. The sensor system according to claim 6, wherein the column signal processing circuit of the Z sensor includes a determination circuit that performs threshold determination on the difference value.

11. The sensor system according to claim 6, wherein the column signal processing circuit of the Z sensor includes a transmission circuit for transmitting the transmission signal to the image sensor.

12. The sensor system according to claim 11, wherein the transmission signal includes a plurality of determination signals indicating whether the magnitude of the difference value of each corresponding pixel sequence is within a threshold range, the determination signals are 1-bit signals, and the transmission circuit is a shift register circuit that sequentially transfers the plurality of determination signals.

13. The sensor system according to claim 1, comprising a focus adjustment device for focusing the Z sensor and the image sensor on a region of interest in the subject, wherein, after the focus adjustment device has focused the Z sensor and the image sensor on the region of interest, the Z sensor reads out one row of image plane phase difference pixel signals and transmits a transmission signal generated based on the read out image plane phase difference pixel signals to the image sensor, and the image sensor processes the image pixel signals identified by the transmission signal from the Z sensor from one row of image pixel signals.

14. The sensor system according to claim 13, wherein the transmission signal is a signal for identifying a pixel sequence corresponding to the region of interest, and the image sensor processes the image pixel signals of the pixel sequence corresponding to the region of interest from the image pixel signals of one row.

15. The sensor system according to claim 13, comprising an information processing device for controlling the focus adjustment device, wherein the information processing device controls the focus adjustment device based on the detection result of the Z sensor so that the Z sensor and the image sensor focus on the region of interest.

16. The sensor system according to claim 15, wherein the information processing device identifies the region of interest in the subject based on the detection result of the image sensor.

17. The sensor system according to claim 16, wherein the plurality of sensors further include a DVS for detecting changes in the subject, and the information processing device identifies the region of interest in the subject based on the detection result of the image sensor when the DVS detects a change in the subject.

Images