Sensor device

Abstract

[Problem] To appropriately arrange and connect dummy pixels and event detection pixels. [Solution] This sensor device is provided with a pixel array. In the pixel array, the virtual pixels do not receive incident light; the gray pixels are provided with sub-gray pixels used for receiving incident light and acquiring gray information; and the event detection pixels have sub-event detection pixels and sub-grayscale pixels for receiving incident light and detecting changes in acquired grayscale information, and the dummy pixels, the grayscale pixels, and the event detection pixels are all arranged in an array in a row direction and a column direction. Sub-grayscale pixels and dummy pixels belonging to a row different from the row to which the sub-grayscale pixels belong are accessed at the same time, and a signal is acquired.

Description

Technical Field

[0001] The present disclosure relates to sensor devices. Background Art

[0002] Image sensors are used in a wide range of fields. Pixels in such image sensors are configured to acquire grayscale information. Now, there are some configurations in which both pixels for acquiring grayscale information and pixels for acquiring event detection information are provided. In addition, sensors using a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) perform processing such as noise reduction based on output from optical black corresponding to pixels that do not receive light.

[0003] As a method of reducing horizontal stripes in an image by reflecting the output from optical black in the column direction from the output of the effective pixel, there is a method of subtracting the output value of the optical black from the output value of the effective pixel to obtain the pixel value using an analog-to-digital converter (ADC) arranged separately in the column direction. In such a configuration in which both the event detection pixel and the grayscale pixel are set, there is a possibility that the voltage fluctuates greatly when an event is detected in the event detection pixel, and there is a possibility that a signal deviation of the grayscale signal occurs due to signal interference between the event detection pixel that has detected the event and the signal line dedicated to the optical black.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2020-161992

[0007] Patent Document 2: Japanese Patent Application Publication No. 2011-040807 Summary of the invention

[0008] Problems to be solved by the present invention

[0009] Therefore, an object of the present disclosure is to provide a sensor device in which virtual pixels, grayscale pixels, and event detection pixels are appropriately arranged and connected.

[0010] Solution to the problem

[0011] According to an embodiment, a sensor device includes a pixel array. The pixel array includes: virtual pixels that do not receive incident light; grayscale pixels that include sub-grayscale pixels that receive incident light to obtain grayscale information; and event detection pixels that include sub-grayscale pixels and sub-event detection pixels, the sub-event detection pixels detecting changes in grayscale information obtained by receiving incident light, the virtual pixels, grayscale pixels, and event detection pixels are arranged in an array in row and column directions; and signals are obtained by simultaneously accessing sub-grayscale pixels and virtual pixels belonging to rows different from the rows to which the sub-grayscale pixels belong.

[0012] The pixel array may include: a first virtual area in which the virtual pixels are arranged; and a light receiving area in which the grayscale pixels and the event detection pixels are arranged.

[0013] The virtual pixel may include a light-shielded pixel corresponding to a grayscale pixel whose light receiving surface has been shielded from light or an event detection pixel whose light receiving surface has been shielded from light.

[0014] The first dummy area may be disposed on an edge of the pixel array at least along a row direction.

[0015] The dummy pixels arranged in the first dummy area and the event detection sub-pixels belonging to the light receiving area may perform output for each column through the same signal line.

[0016] The dummy pixel may include a pixel arranged in the first dummy area in an arrangement configuration different from an arrangement configuration of the grayscale pixels and the event detection pixels in the light receiving area, and the pixel is shielded from light.

[0017] The dummy pixel does not need to include a sub-pixel corresponding to the light-shielded sub-event detection pixel of the event detection pixel.

[0018] The pixel array may include multiple grayscale pixels and multiple event detection pixels, which sub-pixel of each pixel acquires and outputs a signal is switched in response to a trigger signal, and in response to the trigger signal, a connection to a virtual pixel corresponding to the shielded sub-event detection pixel is not established.

[0019] A plurality of column signal lines may be further included, through which signals output from pixels belonging to one of the columns are propagated, and one of the plurality of column signal lines is exclusively connected to the dummy pixel.

[0020] It may also include multiple column signal lines. In a unit pixel group including two pixels arranged continuously in a row direction and multiple pixels arranged continuously in a column direction, signals output from the unit pixel group belonging to the column direction are transmitted through the column signal lines, and pixels belonging to the same row and at the same position in the unit pixel group are connected to the column signal lines set at the same relative position among the multiple column signal lines.

[0021] It may also include multiple column signal lines. In a unit pixel group including two pixels arranged continuously in a row direction and multiple pixels arranged continuously in a column direction, signals output from the unit pixel group belonging to the column direction are transmitted through the column signal lines, and pixels belonging to the same row and at the same position in the unit pixel group are connected to column signal lines arranged at different relative positions among the multiple column signal lines.

[0022] The unit pixel group may include eight pixels including: two pixels arranged continuously in a row direction; and four pixels arranged continuously in a column direction.

[0023] It may also include a second virtual area, which is in a row different from the row of the first virtual area on the edge of the pixel array, and the second virtual area includes virtual pixels corresponding to pixels that have the same configuration as the row in the light receiving area and are shielded from light, and a reference value is calculated based on the signal value output by the virtual pixel belonging to the second virtual area, and the reference value is used to correct the signal value output by the pixel belonging to the light receiving area.

[0024] The signal value output from the first virtual area may be compared with a reference value, and a correction area may be acquired, the correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected.

[0025] Grayscale information in the correction area may be corrected based on a signal value output from the first virtual area.

[0026] It may also include a third virtual area, in which virtual pixels are arranged on the edge of the pixel array along the column direction so as not to overlap with the first virtual area, and a reference value is calculated based on the signal value output by the virtual pixel belonging to the third virtual area, and the reference value is used to correct the signal value output by the pixel belonging to the light receiving area.

[0027] The signal value output from the first virtual area may be compared with a reference value, and a correction area may be acquired, the correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected.

[0028] Grayscale information in the correction area may be corrected based on a signal value output from the first virtual area.

[0029] The dummy pixel may include an analog dummy pixel that outputs a predetermined analog voltage. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a block diagram schematically showing a sensor device according to an embodiment.

[0031] Figure 2is a block diagram schematically showing an example of a first signal processing circuit according to an embodiment.

[0032] Figure 3 is a diagram schematically showing a configuration example of a pixel according to an embodiment.

[0033] Figure 4 is a diagram schematically showing a configuration example of a pixel according to an embodiment.

[0034] Figure 5 is a diagram schematically showing a configuration example of a pixel according to an embodiment.

[0035] Figure 6 is a diagram schematically showing a configuration example of a pixel according to an embodiment.

[0036] Figure 7 is a diagram schematically showing a configuration example of a pixel according to an embodiment.

[0037] Figure 8 is a diagram schematically showing a configuration example of a pixel array according to an embodiment.

[0038] Fig. 9 is a diagram schematically showing a configuration example of a pixel array according to an embodiment.

[0039] Fig.10 is a diagram schematically showing a connection example between pixels and signal lines according to an embodiment.

[0040] Fig.11 is a flowchart illustrating a correction process according to an embodiment.

[0041] Fig.12 is a diagram schematically showing a connection example between pixels and signal lines according to an embodiment.

[0042] Fig.13 is a diagram schematically showing a connection example between pixels and signal lines according to an embodiment.

[0043] Fig.14 is a diagram schematically showing an example of a light-shielded pixel according to an embodiment.

[0044] Fig.15 is a diagram schematically showing an example of a light-shielded pixel according to an embodiment.

[0045] Fig.16 is a diagram schematically showing an example of a light-shielded pixel according to an embodiment.

[0046] Fig.17 This is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0047] Fig.18 It is a diagram for assisting in explaining an example of the installation positions of the vehicle exterior information detection unit and the imaging unit. DETAILED DESCRIPTION

[0048] Embodiments of the present disclosure are described below with reference to the accompanying drawings. The accompanying drawings are for description, and the shape and size of each component in an actual device, the ratio of the size to other components, etc. are not necessarily as shown in the drawings. In addition, because the accompanying drawings are shown in a simplified manner, it should be understood that components necessary for implementation other than those shown in the accompanying drawings are appropriately provided.

[0049] Figure 1 1 is a block diagram schematically showing an example of a sensor device according to an embodiment, which is not limited thereto. The sensor device 1 includes a pixel array 10, a timing control circuit 12, an access control circuit 14, a first reading circuit 16, a first signal processing circuit 18, a second reading circuit 20, a second signal processing circuit 22, a time stamp generation circuit 24, and an output interface (hereinafter, referred to as output I / F 26). The sensor device 1 is provided in, for example, an electronic device such as a solid-state imaging device.

[0050] The pixel array 10 includes a plurality of pixels 100. The pixels 100 are arranged in at least a plurality of columns (row direction). It is desirable that the pixels 100 are also arranged in at least a plurality of lines (column direction), and thus arranged in a two-dimensional array.

[0051] The pixel array 10 includes at least a path for outputting a signal from each pixel 100 to a reading circuit, a path for outputting the detection of an event to the access control circuit 14 when an event is detected in the pixel 100, or a path for inputting a signal from the access control circuit 14 indicating which pixel 100 to read information from.

[0052] Each pixel 100 includes a light receiving element capable of acquiring grayscale information. Each pixel 100 includes a pixel circuit that drives the light receiving element to appropriately acquire an output from the light receiving element. The pixel circuit may include, for example, a circuit that converts a signal acquired from the light receiving element into a signal indicating grayscale information and outputs the resulting signal, and a circuit that detects a change in a signal acquired from the light receiving element and converts the change into a signal indicating event detection information and outputs the resulting signal, or both.

[0053] As an example, the pixel 100 may be activated as a circuit that outputs a signal indicating event detection information when the difference in grayscale value from the previous frame exceeds a predetermined value, and as another example, the pixel 100 may be activated when the contrast exceeds a threshold. Here, the activation indicates that an event is detected in the pixel 100.

[0054] It should be noted that a detailed configuration of the pixel 100 in the present disclosure will be described later.

[0055] The timing control circuit 12 and the access control circuit 14 constitute a control circuit that controls the timing of access to the pixel 100, the timing of reading a signal from the pixel 100, and the timing of signal processing of the read signal. In addition, the control circuit can control the output timing of a signal subjected to signal processing.

[0056] For example, the timing control circuit 12 outputs a frame synchronization signal and a horizontal synchronization signal to the access control circuit 14 based on the input clock signal. In addition, the timing control circuit 12 can generate a timing for performing signal processing based on a signal corresponding to the excitation state of the pixel 100 received from the access control circuit 14 and output the timing to the signal processing circuit.

[0057] The access control circuit 14 outputs an operation signal for selecting the pixel 100 to be accessed based on the horizontal synchronization signal obtained from the timing control circuit 12, so that the pixel 100 outputs event information to the second reading circuit 20. That is, based on the frame information output from the timing control circuit 12, the event detection in the present disclosure is performed by operating the pixel 100 for each frame.

[0058] The first reading circuit 16 converts the analog signal representing the grayscale information output from the pixel 100 into a digital signal. The first reading circuit 16 outputs the digital signal obtained by the conversion to the first signal processing circuit 18. The first reading circuit 16 may include an analog-to-digital converter (ADC) that converts the analog signal into a digital signal. The ADC may be a column ADC provided for each column or a pixel ADC provided for each pixel.

[0059] The first signal processing circuit 18 is a circuit that performs signal processing to convert the acquired grayscale information into appropriate image information. The first signal processing circuit 18 can perform, for example, at least one of linear matrix processing, filtering processing, image processing, or machine learning processing, and convert the input digital signal into an image signal and output the image signal. In addition, the first signal processing circuit 18 performs processing to correct the grayscale information based on a reference value. The reference value will be described in detail later. The first signal processing circuit 18 outputs the processed signal as image data to the outside, such as a processor of an electronic device set outside, via the output I / F 26.

[0060] The second reading circuit 20 appropriately converts the information acquired from the pixel 100 that detects the event information, and outputs the obtained information to the second signal processing circuit 22. The second reading circuit 20 can operate as, for example, an analog front end (AFE). The second reading circuit 20 can include a latch that temporarily stores the event detection information output from each pixel 100 (for example, for each column).

[0061] The second signal processing circuit 22 converts the event information output from the second reading circuit 20 based on the access information about the pixel 100 controlled by the access control circuit 14 acquired via the timing control circuit 12, and outputs the resulting event information as event data to the outside, such as a processor of an electronic device provided outside, via the output I / F 26. The second signal processing circuit 22 can rearrange the acquired event information or format the event information as needed and output the resulting event information. In addition, as described above, the second signal processing circuit 22 can perform signal processing in synchronization with the timing generated by the timing control circuit 12 based on the output of the access control circuit 14.

[0062] The time stamp generation circuit 24 outputs time stamp information, for example, simply referred to as time information, to the first signal processing circuit 18 and the second signal processing circuit 22. The first signal processing circuit 18 and the second signal processing circuit 22 add appropriate time stamps to the data and output the resulting data. As described above, by adding appropriate time stamps, an external processor or the like can obtain an appropriate order of time stamps of output data or the like and perform signal processing or the like.

[0063] The output I / F 26 is an interface that outputs grayscale information and event detection information acquired and converted by the sensor device 1 to the outside. The output I / F 26 may be an interface such as MIPI (registered trademark), for example. The sensor device 1 outputs the acquired event information to the outside via the output I / F 26.

[0064] With this configuration, access in the horizontal direction and the timing of the horizontal synchronization signal can be appropriately controlled according to the amount of data to be acquired.

[0065] This configuration allows the same synchronization signal to be used, in particular, as a synchronization signal for access control in the reading circuit and signal processing control in the signal processing circuit of the pixel array 10. Therefore, in the case where the data output speed of the sensor device 1 is limited by the data bus, the throughput can be improved.

[0066] Note that the timing control circuit 12 is not an essential component. For example, in the case where either the timing of accessing and reading the pixel 100 or the timing of data transmission from the reading circuit to the signal processing circuit is constant, the synchronization signal may be fixed, thereby enabling the sensor device 1 to operate without the timing control circuit 12.

[0067] Furthermore, the sensor device 1 may include a frame memory (not shown). The frame memory is a memory capable of temporarily storing grayscale information and event detection information for each frame.

[0068] Next, how to correct the pixel value in the first signal processing circuit 18 will be described. Figure 2 is a block diagram schematically showing an example of the first signal processing circuit 18 .

[0069] The first signal processing circuit 18 can perform signal processing based on the grayscale data, light-shielding pixel data, and reference value data acquired from the first reading circuit 16. The grayscale data is data acquired by the ADC. The light-shielding pixel data is data output from the light-shielding pixel (optical black) set in the pixel array 10. The reference value data is data for calculating the reference value. The light-shielding pixel data and the reference value data will be described in detail later.

[0070] The first signal processing circuit 18 performs preprocessing on the input data. The preprocessing is a process of converting each piece of data into appropriate data. The preprocessing may include, for example, a process of performing linear matrix processing on grayscale data to calculate image data that appropriately indicates the grayscale value of each pixel.

[0071] The first signal processing circuit 18 calculates the reference value based on the reference value data. Together with this processing, the first signal processing circuit 18 determines the area to be corrected based on the light-shielded pixel data. The first signal processing circuit 18 acquires the level at which the output of the correction pixel 100 is located from the information about the reference value and the correction area, and corrects the grayscale data in the correction area of ​​the image data using the correction level based on the information.

[0072] After this processing, the first signal processing circuit 18 performs various filter processing and the like on the corrected image data and outputs the resulting image data. The sensor device 1 can output image data subjected to appropriate gradation processing by performing such processing.

[0073] Next, the arrangement of pixels 100 in the pixel array 10 and the arrangement of signal lines through which signals are output from the pixels 100 will be described. In the present disclosure, the arrangement of pixels 100 in the pixel array 10 and the connection of signal lines provided for each column in the pixel arrangement will be described in detail.

[0074] The pixel array 10 includes virtual pixels, grayscale pixels, and event detection pixels arranged in an array in the row direction and the column direction, the virtual pixels are configured to receive incident light, the grayscale pixels are configured to receive incident light and output grayscale information, and the event detection pixels are configured to receive incident light and output grayscale information or detect changes in grayscale information and output event detection information. Such a pixel includes a plurality of sub-pixels.

[0075] Figure 3 1 is a diagram schematically showing a non-limiting example of a pixel configuration. In a light receiving area of ​​a pixel array 10, a plurality of pixels 100 are arranged in an array. Each pixel 100 represented by a dotted line includes a plurality of sub-pixels. The sub-pixels include, for example, a sub-grayscale pixel 1020 and a sub-event detection pixel 1040.

[0076] The sub-grayscale pixel 1020 is a sub-pixel that outputs an analog signal obtained as a result of photoelectric conversion in a light receiving element as grayscale information via a pixel circuit. The pixel circuit that outputs an analog signal obtained as a result of photoelectric conversion as grayscale information may be any circuit that outputs grayscale information as an analog signal. R, G, and B shown in the sub-grayscale pixel 1020 each represent the color of light to be received. The sub-grayscale pixel 1020 represented by R, G, and B includes elements such as photodiodes that appropriately receive light in a red wavelength region, light in a green wavelength region, and light in a blue wavelength region, respectively.

[0077] The sub-event detection pixel 1040 is a sub-pixel represented by Ev. The sub-event detection pixel 1040 is a sub-pixel that outputs a change in grayscale information of an analog signal obtained as a result of photoelectric conversion in a light receiving element via a pixel circuit. The pixel circuit that outputs event detection information from an analog signal obtained as a result of photoelectric conversion may be any circuit as long as the circuit outputs event detection information as an analog signal.

[0078] The sub-grayscale pixel 1020 and the sub-event detection pixel 1040 form the grayscale pixel 102 and the event detection pixel 104 .

[0079] The grayscale pixel 102 may have all its sub-pixels as sub-grayscale pixels 1020. With this configuration, the grayscale pixel 102 outputs grayscale information at any timing.

[0080] The event detection pixel 104 may have some of its sub-pixels, for example, sub-pixels belonging to a half region of all its sub-pixels as sub-event detection pixels 1040, and have other sub-pixels as sub-grayscale pixels 1020. With this configuration, whether the event detection pixel 104 outputs grayscale information or event detection information is controlled according to timing.

[0081] The pixel 100 sequentially connects a plurality of sub-pixels to output grayscale information or event detection information at appropriate timing. As an example, pixels 100 belonging to the same row of the pixel array 10 may each access sub-pixels located at the same relative position in the pixel at the same timing.

[0082] For example, in Figure 3 In the figure, the sub-grayscale pixel 1020 on the left side of the green grayscale pixel 102, the sub-event detection pixel 104 on the left side of the blue event detection pixel 104, and the sub-grayscale pixel 1020 on the left side of the red event detection pixel 104 can be driven at the same timing to obtain their respective grayscale signals or event detection signals.

[0083] According to the above-described access example, appropriate grayscale information can be acquired and highly accurate event detection can be achieved.

[0084] As an example, the unit pixel group is arranged as shown by the dotted line in the figure. A suitable path can be established for each unit pixel group via a signal line extending in the column direction. For example, the unit pixel group can be a set of 2×4 pixels in two pixels arranged continuously in the row direction and four pixels arranged continuously in the column direction. The connection will be described later by a specific example.

[0085] Note that Figure 3 , the color arrangement of the grayscale pixels 102 and the event detection pixels 104 is, but not limited to, a Bayer arrangement. For example, complementary colors such as cyan, yellow, or magenta may be included as at least some of the pixels, or pixels that receive white light may be included. In addition, a pixel that receives infrared light, another multispectral pixel, a pixel that includes a plasma filter, and the like may be provided. In addition, although a structure in which the G pixel 100 does not include the sub-event detection pixel 1040 is shown, the pixel structure is not limited to this structure, and the G pixel 100 may be configured as an event detection pixel 104 that includes the sub-event detection pixel 1040.

[0086] Furthermore, a configuration may be employed in which, instead of event detection pixels, for example, an avalanche photodiode (APD) or a single photon avalanche diode (SPAD) is provided to acquire time-of-flight (ToF) information or depth information instead of event detection.

[0087] Figure 4 is a diagram showing another configuration example of the pixel 100. Figure 4 As shown, each pixel 100 may include four sub-pixels. Also with this configuration, in the grayscale pixel 102 and the event detection pixel 104, sub-pixels located at the same relative position can be accessed at the same timing.

[0088] Figure 5 is a diagram showing yet another configuration example of the pixel 100. Figure 5 As shown, each pixel 100 may include eight sub-pixels. For example, the grayscale pixel 102 includes eight sub-grayscale pixels 1020. The event detection pixel 104 includes, for example, four sub-event detection pixels 1040 and four sub-grayscale pixels 1020. Although not shown in the figure, the unit pixel group may be defined as follows: Figure 3 and Figure 4 A collection of eight pixels in .

[0089] Even in the case where the pixel 100 includes four sub-pixels or eight sub-pixels, the sub-pixels accessed at the same timing may be controlled based on their respective relative positions in the pixel 100 .

[0090] Figure 61 is a diagram showing a configuration example of a virtual pixel (light shielding pixel). As an example, the light shielding pixel 106 includes Figure 3 , Figure 4 or Figure 5 The number of segmentations shown is the same as the number of sub-pixels. Figure 6 1 is a top view of an example of the pixel array 10 when viewed from the direction in which light is incident. A pixel 100 including two sub-pixels, a pixel 100 including four sub-pixels, and a pixel 100 including eight sub-pixels are sequentially shown from the top.

[0091] The sub-pixels indicated as diagonal lines extending from the lower left to the upper right and included in the light-shielded pixel 106 are sub-light-shielded pixels corresponding to the sub-grayscale pixels 1020 having light-shielded light-receiving surfaces. Figure 6 As shown in , the light-shielded pixel may include a sub-light-shielded pixel corresponding to the sub-grayscale pixel 1020 having a light-shielded light receiving surface.

[0092] Figure 7 106 is a diagram showing another configuration example of a light-shielded pixel. The sub-pixel represented by the diagonal line extending from the lower right to the upper left included in the light-shielded pixel 106 is a sub-light-shielded pixel corresponding to the sub-event detection pixel 1040 whose light receiving surface is shielded from light. As described above, in a manner similar to the arrangement of other pixels in the pixel array 10, the light-shielded pixel 106 may include a sub-light-shielded pixel corresponding to the sub-grayscale pixel 1020 having a light-shielded light receiving surface and the sub-event detection pixel 1040 having a light-shielded light receiving surface.

[0093] Figure 8 1 is a diagram showing a light receiving pixel area and a virtual area in a pixel array 10. For example, a virtual area domain in which a light shielding pixel 106 is arranged on a line on an edge of the pixel array 10. For example, each virtual area domain includes a plurality of lines, and the pixels in the virtual area domain are formed as light shielding pixels 106 having a light receiving surface that is shielded from light. For example, the pixel array 10 includes a first virtual area 120, a second virtual area 122, and a light receiving area as other areas.

[0094] The first virtual area 120 is an area including the light-shielded pixels 106 for calculating the influence of signal interference between pixels such as signal lines and an image formed based on a signal acquired in the light receiving area of ​​the pixel array 10. For example, Figure 7 As shown, in the first dummy area 120, light shielding pixels 106 having a configuration different from that of pixels in the light receiving area may be provided.

[0095] The second virtual area 122 is an area including the light-shielded pixels 106 that output data for calculating a reference value for correcting the grayscale value of a signal acquired in the light receiving area of ​​the pixel array 10 based on the dark portion signal. Figure 6 As shown in FIG. 1 , the light-shielding pixels 106 having the same configuration as that of the pixels in the light-receiving region may be provided. The light-shielding pixels 106 provided in the second dummy region 122 operate in a manner similar to so-called optical black in a general sense.

[0096] Fig. 9 1 is a diagram showing another example of a light receiving pixel region and a virtual region domain of the pixel array 10. For example, the virtual region domain including the light shielding pixel 106 may include a first virtual region domain 120 and a second virtual region domain 122 disposed on a row, and a third virtual region domain 124 disposed on a column on an edge of the pixel array 10.

[0097] The third dummy area 124 is an area where the light shielding pixels 106 are arranged on the columns on the edge of the pixel array 10. Fig. 9 124 is disposed on the left and right sides, but the third virtual area 124 may be disposed on either the left edge or the right edge. In addition, for example, the arrangement configuration of the light shielding pixels 106 disposed in the third virtual area 124 may be the same as that in Figure 6 The arrangement configuration of the pixels 100 in the light receiving area shown in FIG. 1 is similar.

[0098] As shown in the figure, the third virtual area 124 is arranged to overlap neither the first virtual area 120 nor the second virtual area 124. As another example, the third virtual area 124 may be defined as an area including all light-shielding pixels 106 belonging to the column direction in the pixel array 10, and the first virtual area 120 and the second virtual area 122 may be defined as an area extending along a line and not overlapping the third virtual area 124.

[0099] The third virtual area 124 may output data for calculating the reference value, instead of the second virtual area 122 .

[0100] With the above configuration, the grayscale pixel 102 is accessed at the same timing as the light-shielded pixel 106 belonging to any virtual area domain, and then the signal is acquired. The grayscale pixel 102 and the light-shielded pixel 106 may be pixels belonging to the same column. That is, in the sensor device 1, for example, the grayscale pixel 102 and the light-shielded pixel 106 belonging to the same column but belonging to different rows can be driven at the same timing to output a signal.

[0101] The connection between pixels and signal lines will be described.

[0102] Fig.101 is a diagram schematically showing an example of connection between pixels and signal lines according to an embodiment. Signal lines 140, 142, 144, and 146 are signal lines shared by unit pixel groups belonging to the same column. The signal output from each pixel (more specifically, each sub-pixel) is transmitted to the reading circuit through any signal line.

[0103] For example, at the timing when the lower right sub-pixel of the use pixel 100 outputs a signal, a signal from the sub-pixel indicated by an arrow is output through any one of the signal lines 140 , 142 , 144 , and 146 .

[0104] As an example, signals are output from two pixels belonging to unit pixel groups in cyclic adjacent rows at the same time. With this configuration, two signal lines can be used to output signals for the unit pixel group. Adjacent lines are connected to different signal lines. Unit pixel groups that are continuous along a column can have sub-pixels output signals using signal lines different from the above two signal lines.

[0105] For example, the pixels 100 of the upper unit pixel group in the figure are connected to any one of the signal lines 144 and 146 , and the pixels 100 of the lower unit pixel group in the figure are connected to any one of the signal lines 140 and 142 .

[0106] Note that for the shading pixels 106 belonging to the second virtual area 122 and the third virtual area 124, a signal line dedicated to the shading pixels 106 may be provided separately through a general connection, or the signal value of the shading pixels may be output using a signal line 140 or the like through any general method.

[0107] Fig.10 Two columns and four unit pixel groups 110A, 110B, 110C, and 110D are shown. At a specific timing, grayscale pixels 102A and 102B of the unit pixel group 110A, event detection pixels 104A and 104B of the unit pixel group 110B, grayscale pixels 102C and 102D of the unit pixel group 110A, and event detection pixels 104C and 104D of the unit pixel group 110D are selected, and in addition, each pixel is controlled to output a signal from a sub-pixel located at the lower left.

[0108] For each unit pixel group, the shading pixels 106A, 106B, 106C and 106D belonging to the virtual area are arranged in the same column on the edge of the pixel array 10. Similarly, the lower left sub-shading pixel of the shading pixel 106 belonging to any row is connected to any signal line belonging to the column. The connection between the shading pixel 106 and the signal line can be switchable. In other words, for example, the shading pixel 106B can be controlled to be selectively connected to any one of the signal lines 140, 142, 144 and 146 according to the timing.

[0109] With the above configuration, a connection example of the signal line will be described. As an example, for the selected grayscale pixels 102A and 102B, signals are output from the sub-pixels located at the lower left through signal lines 146 and 144, respectively. Similarly, for the selected event detection pixels 104A and 104B, signals are output from the sub-pixels located at the lower left through signal lines 140 and 142, respectively.

[0110] Grayscale pixels 102A and 102B and event detection pixel 104A output grayscale signals, and event detection pixel 104B outputs event detection information. With this configuration, light-shielded pixel 106B can be connected to signal line 142, and event detection information is output through signal line 142. Through this connection, the influence of interference between the event detection pixel and the signal line can be output, that is, the influence of interference of the signal value of each column mixed with the output of the light-shielded pixel 106 and the grayscale signal can be prevented.

[0111] Generally, the period for acquiring event detection information is much shorter than the period for acquiring grayscale information. Therefore, even at the moment when event information is not acquired, there is no particular problem in using the same signal line as the pixel for acquiring event detection information for the data of the light-shielded pixel 106. In addition, in the case where an event is detected, by outputting the event detection information using the same signal line as the output of the light-shielded pixel 106, information about the position to which the signal line that outputs the signal value of the light-shielded pixel 106 belongs and which may be disturbed by the signal value due to event detection can be obtained.

[0112] Figure 2 The first signal processing circuit 18 shown is based on Fig.11 The signal value output from each sub-grayscale pixel 1020 is corrected according to the flowchart shown.

[0113] First, the first signal processing circuit 18 acquires reference value data (S100). The reference value data is, for example, data for calculating a reference value for removing thermal noise or the like for correction. For example, the reference value data can be acquired based on the output from the light-shielded pixel 106 belonging to the second virtual area 122. As another example, the reference value data can be acquired based on the output from the light-shielded pixel 106 belonging to the third virtual area 124. By making the pixels in the second virtual area domain 122 and the third virtual area domain 124 have the same structure as the pixels in the light-receiving area, it is possible to acquire reference value data that is closer to the characteristics of the light-receiving area.

[0114] Next, the first signal processing circuit 18 calculates the reference value from the reference value data (S102). For example, the first signal processing circuit 18 calculates the reference value by calculating the average value of the output data of the light-shielded pixels 106 in the row acquired in S100 or the average value of the output data of the light-shielded pixels 106 belonging to the second virtual area 122 in the column. The processing in this step is similar to the processing of the configuration of the general optical black in the row direction and the column direction, so that any desired method for acquiring the correction value (corresponding to the reference value) from the optical black can be used.

[0115] Next, the first signal processing circuit 18 acquires interference information ( S104 ). The interference information may be light-shielded pixel data output from the light-shielded pixels 106 belonging to the first virtual area 120 .

[0116] Next, the first signal processing circuit 18 extracts the column area to be corrected (S106). In S104, the first signal processing circuit 18 extracts the correction area based on the light-shielded pixel data acquired from the first virtual area 120. For example, the first signal processing circuit 18 compares the reference value calculated in S102 with the light-shielded pixel data acquired in S104 for the line being scanned, and determines and extracts the area where the light-shielded pixel data exceeds the reference value as the area where interference occurs.

[0117] Next, the first signal processing circuit 18 calculates the correction level of the correction area acquired in S106 (S108). For example, the first signal processing circuit 18 may use the difference between the light-shielded pixel data acquired for each column and the reference value as the correction value. As another example, the first signal processing circuit 18 may use the difference between the average value of the light-shielded pixel data in the correction area and the reference value as the correction value.

[0118] Next, the first signal processing circuit 18 corrects the grayscale value of each pixel in the area acquired in S106 using the correction value acquired in S108 (S110). It should be noted that, in a manner similar to this correction, the first signal processing circuit 18 can also perform correction of thermal noise and the like in the correction area and other areas based on the data acquired from the second virtual area domain and / or the third virtual area domain 124.

[0119] Note that the above-mentioned processing can be replaced as needed within the range that does not affect the calculation results.

[0120] As described above, the sensor device according to the present embodiment can appropriately extract the state of interference with the signal line caused by the output of the event detection pixel by using a shading pixel having an output path identical to the output path of the event detection pixel, and appropriately correct the grayscale value of the area where interference has occurred.

[0121] Hereinafter, the connection relationship between pixels and signal lines will be described through some examples.

[0122] Fig.12 is a diagram showing an example of connection between pixels and signal lines according to an embodiment. Fig.10 In the embodiment, the unit pixel groups arranged along the rows have the same relative connection relationship between the plurality of signal lines and the pixels. However, aspects of the present disclosure are not limited thereto.

[0123] like Fig.12 As shown, the connection states between the pixels and the signal lines in the unit pixel groups 110A and 110B and the unit pixel groups 110C and 110D may be different.

[0124] Specifically, the grayscale pixel 102A of the unit pixel group 110A and the grayscale pixel 102C of the unit pixel section 110C belonging to the same relative position can be connected to signal lines arranged at different relative positions. For example, the grayscale pixel 102A is connected to the third signal line 144A from the left side of the column signal line to which the unit pixel group 110A belongs. On the other hand, the grayscale pixel 102C can be connected to the fourth signal line 146B from the left side of the column signal line to which the unit pixel group 110C belongs.

[0125] As for the column direction, maintain Fig.10 Needless to say, there is a possibility that the relative positions of the signal lines connected to the sub-event detection pixels of the event detection pixels 104B and 104C responsible for event detection also change.

[0126] In this case, if Fig.12 As shown, the connection state of the light-shielding pixel 106 belonging to the first virtual area 120 is also appropriately changed. For example, the light-shielding pixel 106B is connected to the same signal line 142A as the event detection pixel 104B responsible for event detection of the unit pixel group belonging to the same column, and the light-shielding pixel 106D is connected to the same signal line 144B as the event detection pixel 104D responsible for event detection of the unit pixel group belonging to the same column, so that the relative positional relationship between the signal lines in the column becomes different.

[0127] according to Fig.12 According to the aspect shown in , in a case where the amount of interference differs in a manner depending on the position of the wiring of the signal line, more accurate extraction of the buffer region can be achieved.

[0128] Fig.13 1 is a diagram showing a connection example between pixels and signal lines according to an embodiment. The sensor device 1 may include a signal line 148 dedicated to the light-shielded pixel 106 of the first dummy area 120 in a column belonging to each unit pixel group.

[0129] In the pixel array 10, when the density of sub-event detection pixels is sparse and R, G, and B signal values ​​can be acquired at the same timing, the connection of the signal lines in the above form has the possibility that the signal line representing any grayscale value and the shading pixel 106 belonging to the first virtual area 120 are connected to the same signal line, which prevents the acquisition of appropriate grayscale values.

[0130] In this case, if Fig.13 As shown, by providing the signal line 148 connected only to the light shielding area 106 of the first dummy area 120, grayscale information with higher accuracy can be obtained.

[0131] Fig.14 1 is a diagram showing an example of grayscale pixels 102, event detection pixels 104, and light-shielded pixels 106 in a first virtual area 120 according to an embodiment. As shown in the figure, for example, the light-shielded pixel 106 may correspond to a light-receiving element that is similar in arrangement to the grayscale pixel 102 and the event detection pixel 104 and has a light-shielded light-receiving surface. Oblique lines indicate that a sub-pixel is light-shielded.

[0132] In this case, in both the grayscale pixel 102 and the event detection pixel 104 , the trigger signals TRG0 to TRG7 are appropriately controlled to drive sub-pixels located at the same relative position in the pixel.

[0133] For example, when a sub event detection pixel indicated by an arrow enters an event detection standby state, a corresponding trigger signal TRGO and a sub light shielding pixel indicated by an arrow are driven, and an interference region may be acquired using an output from the sub light shielding pixel.

[0134] On the other hand, in order to appropriately select the sub-shielding pixels connected to the sub-event detection pixels, the sub-shielding pixels may be set in the shielding pixels 106 to be exclusively connected to the trigger signals TRG2, TRG3, TRG6, and TRG7. With this configuration, in the case where the sub-event detection pixels are selected by the trigger signals, the sub-shielding pixels and the sub-event detection pixels of the shielding pixels 106 in the first virtual area 120 may be appropriately connected.

[0135] Fig.15 1 is a diagram illustrating another example. In the figure, the light-shielding pixel 106 in the first virtual area 120 may correspond to the grayscale pixel 102 having a light-shielded light-receiving surface. In this case, the sub-event detection pixel and the sub-light-shielding pixel may be appropriately connected by driving the sub-light-shielding pixel corresponding to each trigger signal.

[0136] Fig.16 1 is a diagram illustrating another example. In the first dummy area 120, an analog voltage source 169 may be used instead of the light shielding pixel.

[0137] The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure can also be implemented as a device installed on any kind of mobile body such as a car, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, a robot, a construction machine, an agricultural machine (tractor), etc.

[0138] Fig.17 7000 is a block diagram showing a schematic configuration example of a vehicle control system 7000 as an example of a mobile body control system to which the technology of the present disclosure is applied. The vehicle control system 7000 includes a plurality of electronic control units connected to each other via a communication network 7010. Fig.17 In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting the plurality of control units may be, for example, an in-vehicle communication network conforming to any standard such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), or FlexRay (registered trademark).

[0139] Each control unit includes a microcomputer that performs arithmetic processing according to various programs, a storage unit that stores programs executed by the microcomputer, parameters for various arithmetic operations, etc., and a drive circuit that drives various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via the communication network 7010, and includes a communication I / F for communicating with devices, sensors, etc. inside and outside the vehicle by wired or wireless communication. Fig.17 The functional configuration of the integrated control unit 7600 shown includes a microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning section 7640, a beacon receiving section 7650, an in-vehicle device I / F 7660, a sound / image output section 7670, an in-vehicle network I / F 7680, and a storage section 7690. Other control units similarly include a microcomputer, a communication I / F, a storage section, and the like.

[0140] The drive system control unit 7100 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 is used as a control device for a drive force generating device (such as an internal combustion engine or a drive motor) for generating a drive force of the vehicle, a drive force transmitting mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, a braking device for generating a braking force of the vehicle, etc. The drive system control unit 7100 may have a function as a control device such as an anti-lock braking system (ABS) or an electronic stability control (ESC).

[0141] The drive system control unit 7100 is connected to the vehicle state detection unit 7110. The vehicle state detection unit 7110 includes, for example, a gyro sensor for detecting the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor for detecting the acceleration of the vehicle, and at least one of sensors for detecting the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, the engine speed, or the speed of the wheels. The drive system control unit 7100 performs calculation processing using the signal input from the vehicle state detection unit 7110 to control the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0142] The body system control unit 7200 controls the operation of various devices provided for the vehicle body according to various programs. For example, the body system control unit 7200 is used as a control device for a keyless entry system, a smart key system, a power window device, or various lights such as a headlight, a backup light, a brake light, a turn signal, a fog light, etc. In this case, a radio wave emitted from a mobile device may be input to the body system control unit 7200 as a substitute for a signal of a key or various switches. The body system control unit 7200 receives these input radio waves or signals and controls a door lock device, a power window device, lights, etc. of the vehicle.

[0143] The battery control unit 7300 controls the secondary battery 7310 as a power source for driving the motor according to various programs. For example, information about the battery temperature, the battery output voltage, the charge state of the battery, etc. is supplied to the battery control unit 7300 from the battery device including the secondary battery 7310. The battery control unit 7300 performs arithmetic processing using these signals to adjust the temperature of the secondary battery 7310 or control a cooling device provided for the battery device, etc.

[0144] The vehicle exterior information detection unit 7400 detects information outside the vehicle including the vehicle control system 7000. For example, the vehicle exterior information detection unit 7400 is connected to at least one of the imaging unit 7410 and the vehicle exterior information detection unit 7420. The imaging unit 7410 includes at least one of a time-of-flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, or another camera. For example, the vehicle exterior information detection unit 7420 includes at least one of an environmental sensor for detecting current weather conditions or meteorological conditions or a surrounding information detection sensor for detecting another vehicle, an obstacle, a pedestrian, etc. around the vehicle including the vehicle control system 7000.

[0145] The environmental sensor may include, for example, at least one of a raindrop sensor for detecting rain, a fog sensor for detecting fog, a sunlight sensor for detecting sunlight levels, or a snow sensor for detecting snowfall. The surrounding information detection sensor may include at least one of an ultrasonic sensor, a radar device, or a LIDAR device (a light detection and ranging device, or a laser imaging detection and ranging device). The imaging unit 7410 and the vehicle exterior information detection unit 7420 may be provided as independent sensors or devices, or may be provided as devices in which a plurality of sensors or devices are integrated.

[0146] Here, Fig.18 An example of the setting position of the imaging unit 7410 and the vehicle exterior information detection unit 7420 is shown. The imaging units 7910, 7912, 7914, 7916 and 7918 are set, for example, at least one of the positions including the front nose, side mirrors, rear bumper and rear door of the vehicle 7900, and the upper part of the windshield in the vehicle. In the vehicle, the camera unit 7910 set at the front nose and the camera unit 7918 set at the upper part of the windshield mainly capture the image in front of the vehicle 7900. The imaging units 7912 and 7914 set at the side mirrors mainly capture the image of the side of the vehicle 7900. The camera unit 7916 set at the rear bumper or the rear door mainly captures the rear of the vehicle 7900. The imaging unit 7918 set at the upper part of the windshield in the vehicle is mainly used to detect the front vehicle, pedestrians, obstacles, signals, traffic signs, lanes, etc.

[0147] Notice, Fig.18 An example of the imaging range of each of the imaging units 7910, 7912, 7914, and 7916 is shown. Imaging range a represents the imaging range of the imaging unit 7910 disposed at the front nose, imaging ranges b and c represent the imaging ranges of the imaging units 7912 and 7914 disposed at the side mirrors, respectively, and imaging range d represents the imaging range of the imaging unit 7916 disposed at the rear bumper or rear door. For example, by superimposing image data captured by the imaging units 7910, 7912, 7914, and 7916, a bird's-eye view image of the vehicle 7900 viewed from above can be obtained.

[0148] The vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, 7930 provided at the front, rear, side, corners and upper portion of the windshield in the vehicle 7900 may include, for example, ultrasonic sensors or radar devices. For example, the vehicle exterior information detection units 7920, 7926 and 7930 provided at the front nose, rear bumper and rear door of the vehicle 7900 and upper portion of the windshield of the vehicle may each include a LIDAR device. These vehicle exterior information detection units 7920 to 7930 are mainly used to detect preceding vehicles, pedestrians, obstacles, etc.

[0149] Return to see Fig.17 , will continue to describe. The vehicle exterior information detection unit 7400 enables the imaging unit 7410 to capture images outside the vehicle and receive captured image data. In addition, the vehicle exterior information detection unit 7400 receives detection information from the vehicle exterior information detection unit 7420 connected to the vehicle exterior information detection unit 7400. In the case where the vehicle exterior information detection unit 7420 includes an ultrasonic sensor, a radar device, a LIDAR device, the vehicle exterior information detection unit 7400 sends ultrasonic waves or electromagnetic waves, etc., and receives information related to the received reflected waves. In addition, the vehicle exterior information detection unit 7400 may also perform processing for detecting objects such as people, vehicles, obstacles, signs, and characters on the road surface based on the received information, or detect processing for detecting the distance to the object. In addition, the vehicle exterior information detection unit 7400 may also perform environmental recognition processing for identifying rainfall, fog, road conditions, etc. based on the received information. The vehicle exterior information detection unit 7400 may calculate the distance to objects outside the vehicle based on the received information.

[0150] In addition, the vehicle exterior information detection unit 7400 may also perform image recognition processing for identifying objects such as people, vehicles, obstacles, signs, text, etc. on the road, or detect the distance to the object based on the received image data. The vehicle exterior information detection unit 7400 may perform processing such as distortion correction, alignment, etc. on the received image data, and combine the image data captured by different imaging units 7410 to generate a bird's-eye view image or a panoramic image. The vehicle exterior information detection unit 7400 may perform viewpoint conversion processing using the image data captured by different imaging units 7410.

[0151] The in-vehicle information detection unit 7500 detects information about the interior of the vehicle. For example, the in-vehicle information detection unit 7500 is connected to a driver state detection unit 7510 that detects the state of the driver. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, a microphone that collects sounds in the vehicle, and the like. For example, the biosensor is provided on a seat surface, a steering wheel, and the like, and detects the biometric information of a passenger sitting on the seat or a driver holding the steering wheel. The in-vehicle information detection unit 7500 may calculate the driver's fatigue or the driver's concentration based on the detection information input from the driver state detection unit 7510, or may determine whether the driver is dozing off. The in-vehicle information detection unit 7500 may subject the collected sound signals to processing such as noise elimination processing, and the like.

[0152] The integrated control unit 7600 controls the general operation in the vehicle control system 7000 according to various programs. The integrated control unit 7600 is connected to the input unit 7800. The input unit 7800 is implemented by a device such as a touch panel, a button, a microphone, a switch, a rod, etc. that can be operated by the occupant. The integrated control unit 7600 can be supplied with data obtained by voice recognition of the voice input through the microphone. The input unit 7800 may include, for example, a remote control device using infrared or other radio waves, or an external connection device such as a mobile phone, a personal digital assistant (PDA), etc. compatible with the vehicle control system 7000. The input unit 7800 may include, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by the occupant may be input. In addition, the input unit 7800 may include, for example, an input control circuit that generates an input signal based on the information input by the occupant using the above-mentioned input unit 7800 and outputs the generated input signal to the integrated control unit 7600. The occupants and the like input various data or instruct processing operations to the vehicle control system 7000 by operating the input unit 7800 .

[0153] The storage unit 7690 may include a read-only memory (ROM) storing various programs executed by the microcomputer and a random access memory (RAM) storing various parameters, operation results, sensor values, etc. In addition, the storage unit 7690 may be implemented by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, etc.

[0154] The general communication I / F 7620 is a general communication I / F that mediates communication with various devices existing in the external environment 7750. The general communication I / F 7620 may implement therein a cellular communication protocol, for example, Global System for Mobile Communications (GSM (registered trademark)), Worldwide Interoperability for Microwave Access (WiMAX (registered trademark)), Long Term Evolution (LTE (registered trademark)), LTE-Advanced (LTE-A), etc., or another wireless communication protocol, for example, a wireless LAN (also known as Wireless Fidelity (Wi-Fi (registered trademark)), Bluetooth (registered trademark), etc. For example, the general communication I / F 7620 may be connected to a device (for example, an application server or a control server) existing on an external network (for example, the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, for example, the general communication I / F 7620 may be connected to a terminal existing near the vehicle (the terminal is, for example, a driver's terminal, a pedestrian or a store, or a machine type communication (MTC) terminal) using a peer-to-peer (P2P) technology.

[0155] The dedicated communication I / F 7630 is a communication I / F that supports the development of a communication protocol for use in a vehicle. For example, the dedicated communication I / F 7630 may implement a standard protocol therein, such as Wireless Access in Vehicular Environments (WAVE) (which is a combination of IEEE 802.11p as a lower layer and IEEE 17609 as a higher layer), Dedicated Short Range Communication (DSRC), or a cellular communication protocol. The dedicated communication I / F 7630 generally performs V2X communication, which is a concept including one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0156] The positioning unit 7640 receives a global navigation satellite system (GNSS) signal (e.g., a global positioning system (GPS) signal from a GPS satellite) from a GNSS satellite to perform positioning, and generates position information, for example, including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may identify the current position by exchanging signals with a wireless access point, or may obtain position information from a terminal such as a mobile phone, a personal handyphone system (PHS) terminal, or a smart phone having a positioning function.

[0157] For example, the beacon receiving unit 7650 receives radio waves or electromagnetic waves transmitted from a wireless station installed on a road or the like to obtain information about the current position, congestion, closed roads, travel time, etc. Note that the function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

[0158] The vehicle-mounted device I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various vehicle-mounted devices 7760 present in the vehicle. The vehicle-mounted device I / F 7660 can establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless universal serial bus (WUSB). In addition, the vehicle-mounted device I / F 7660 can establish a wired connection such as a universal serial bus (USB), a high-definition multimedia interface (HDMI (registered trademark)), a mobile high-definition link (MHL), etc. via a connection terminal not shown (and a cable when necessary). The vehicle-mounted device 7760 may include, for example, at least one of a mobile device or a wearable device possessed by an occupant, or an information device carried in a vehicle or attached to the vehicle. In addition, the vehicle-mounted device 7760 may include a navigation device that searches for a route to any desired destination. The vehicle-mounted device I / F 7660 exchanges control signals or data signals with these vehicle-mounted devices 7760.

[0159] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like according to a predetermined protocol supported by the communication network 7010.

[0160] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the vehicle-mounted device I / F 7660, or the vehicle-mounted network I / F 7680. For example, the microcomputer 7610 can calculate the control target value of the driving force generation device, the steering mechanism, or the braking device based on the information obtained about the inside and outside of the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control intended to realize the functions of the advanced driver assistance system (ADAS), which include vehicle collision prevention or shock absorption, follow-up driving based on the distance between vehicles, adaptive cruise control, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 can perform cooperative control intended for automatic driving, which makes the vehicle automatically travel without depending on the operation of the driver, etc., by controlling the driving force generation device, the steering mechanism, the braking device, etc. based on the information obtained about the surrounding environment of the vehicle.

[0161] The microcomputer 7610 may generate three-dimensional distance information between the vehicle and objects such as surrounding structures, people, etc., and generate local map information including information about the surroundings of the current position of the vehicle based on information obtained via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the vehicle-mounted device I / F 7660, or the vehicle network I / F 7680. In addition, the microcomputer 7610 may predict dangers such as collision of vehicles, approach of pedestrians, etc., entering a closed road, etc. based on the obtained information, and generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or lighting a warning lamp.

[0162] The sound / image output unit 7670 transmits an output signal of at least one of sound or image to an output device, which can provide information visually or auditorily to the occupants of the vehicle or the outside of the vehicle. Fig.17In the example of , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are illustrated as output devices. The display unit 7720 may include, for example, at least one of an onboard display or a head-up display. The display unit 7720 may have an augmented reality (AR) display function. In addition to the above-mentioned devices, the output device may include another device such as headphones, a wearable device such as a glasses-type display worn by an occupant, a projector, or a lamp. In the case where the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from another control unit in various forms (such as text, images, tables, or graphics). In addition, in the case where the output device is a sound output device, the sound output device converts an audio signal including reproduced audio data, sound data, etc. into an analog signal, and outputs the analog signal audibly.

[0163] Note that Fig.17 In the example shown, at least two control units connected by the communication network 7010 can be integrated into one control unit. Alternatively, each individual control unit may include multiple control units. In addition, the vehicle control system 7000 may include another control unit (not shown). In addition, some or all of the functions performed by any control unit described above may be assigned to another control unit. In other words, as long as information is sent and received via the communication network 7010, predetermined arithmetic processing may be performed by any one of the control units. Similarly, a sensor or device connected to any one of the control units may be connected to another control unit, and multiple control units may send and receive detection information to each other via the communication network 7010.

[0164] The above-mentioned embodiment can have the following modes.

[0165] (1) A sensor device comprising a pixel array, wherein:

[0166] The pixel array includes:

[0167] Virtual pixels, do not receive incident light;

[0168] Grayscale pixels, including sub-grayscale pixels that receive incident light to obtain grayscale information; and event detection pixels, including sub-grayscale pixels and sub-event detection pixels, the sub-event detection pixels detecting changes in grayscale information obtained by receiving incident light, the virtual pixels, the grayscale pixels, and the event detection pixels being arranged in an array in row and column directions; and

[0169] A signal is acquired by simultaneously accessing a sub-grayscale pixel and a dummy pixel belonging to a row different from a row to which the sub-grayscale pixel belongs.

[0170] (2) The sensor device according to (1), wherein:

[0171] The pixel array includes:

[0172] a first virtual area in which the virtual pixels are arranged; and

[0173] A light receiving area, grayscale pixels, and event detection pixels are arranged in the light receiving area.

[0174] (3) The sensor device according to (2), wherein:

[0175] The virtual pixels include light-shielded pixels corresponding to grayscale pixels whose light-receiving surfaces have been shielded from light or event detection pixels whose light-receiving surfaces have been shielded from light.

[0176] (4) The sensor device according to (2) or (3), wherein:

[0177] The first dummy area is arranged at least along a row direction on an edge of the pixel array.

[0178] (5) The sensor device according to (4), wherein:

[0179] The dummy pixels arranged in the first dummy area and the event detection sub-pixels belonging to the light receiving area perform output for each column through the same signal line.

[0180] (6) The sensor device according to (5), wherein:

[0181] The dummy pixels include pixels arranged in the first dummy area in an arrangement configuration different from an arrangement configuration of grayscale pixels and event detection pixels in the light receiving area, and the pixels are shielded from light.

[0182] (7) The sensor device according to (6), wherein:

[0183] The dummy pixel does not include a sub-pixel corresponding to a sub-event detection pixel of the event detection pixel that is shielded from light.

[0184] (8) The sensor device according to (6), wherein:

[0185] The pixel array includes a plurality of grayscale pixels and a plurality of event detection pixels, and switches which sub-pixel in each pixel acquires and outputs a signal in response to a trigger signal, and

[0186] In response to the trigger signal, a connection to a virtual pixel corresponding to the shielded sub-event detection pixel is not established.

[0187] (9) The sensor device according to any one of (3) to (8), further comprising: a plurality of column signal lines through which signals output from pixels belonging to one of the columns are propagated, wherein:

[0188] One column signal line among the plurality of column signal lines is exclusively connected to the dummy pixel.

[0189] (10) The sensor device according to any one of (3) to (8), further comprising: a plurality of column signal lines, through which signals output from a unit pixel group belonging to the column direction are transmitted in a unit pixel group including two pixels arranged continuously in the row direction and a plurality of pixels arranged continuously in the column direction, wherein:

[0190] Pixels belonging to the same row and having the same position in the unit pixel group are connected to column signal lines provided at the same relative position among a plurality of column signal lines.

[0191] (11) The sensor device according to any one of (3) to (8), further comprising: a plurality of column signal lines, through which signals output from a unit pixel group belonging to the column direction are transmitted in a unit pixel group including two pixels arranged continuously in the row direction and a plurality of pixels arranged continuously in the column direction, wherein:

[0192] Pixels that belong to the same row and are located at the same position in the unit pixel group are connected to column signal lines provided at different relative positions among a plurality of column signal lines.

[0193] (12) The sensor device according to (10) or (11), wherein:

[0194] The unit pixel group includes eight pixels, and the eight pixels include:

[0195] two pixels arranged consecutively in a row direction; and

[0196] Four pixels are arranged consecutively in the column direction.

[0197] (13) The sensor device according to (4), further comprising: a second virtual area, the second virtual area being in a row different from the row of the first virtual area on an edge of the pixel array, the second virtual area including virtual pixels corresponding to pixels having the same configuration as the row in the light receiving area and being shielded from light, wherein:

[0198] A reference value is calculated based on the signal value output from the virtual pixel belonging to the second virtual area, and the reference value is used to correct the signal value output from the pixel belonging to the light receiving area.

[0199] (14) The sensor device according to (13), wherein:

[0200] The signal value output from the first virtual area is compared with a reference value, and a correction area is acquired, the correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected.

[0201] (15) The sensor device according to (14), wherein:

[0202] Grayscale information in the correction area is corrected based on the signal value output from the first virtual area.

[0203] (16) The sensor device according to (4) or (13), further comprising: a third virtual area, in which the virtual pixels are arranged on an edge of the pixel array along the column direction so as not to overlap with the first virtual area, wherein:

[0204] A reference value is calculated based on the signal value output from the virtual pixel belonging to the third virtual area, and the reference value is used to correct the signal value output from the pixel belonging to the light receiving area.

[0205] (17) The sensor device according to (16), wherein:

[0206] The signal value output from the first virtual area is compared with a reference value, and a correction area is acquired, the correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected.

[0207] (18) The sensor device according to (17), wherein:

[0208] Grayscale information in the correction area is corrected based on the signal value output from the first virtual area.

[0209] (19) The sensor device according to (2), wherein:

[0210] The dummy pixels include analog dummy pixels, and the analog dummy pixels output predetermined analog voltages.

[0211] The aspects of the present disclosure are not limited to the above-described embodiments, and include various conceivable variations. The effects of the present disclosure are not limited to the above-described contents. The components in each embodiment may be appropriately combined and applied. That is, various additions, modifications, and partial deletions may be made without departing from the conceptual concept and gist of the present disclosure obtained from the contents defined in the claims and their equivalents, etc.

[0212] Reference Symbols List

[0213] 1 Sensor device

[0214] 10-Pixel Array

[0215] 100 pixels

[0216] 1020 sub-grayscale pixels

[0217] 1040 sub-event detection pixels

[0218] 102 grayscale pixels

[0219] 104 event detection pixels

[0220] 106 shading pixels

[0221] 110 unit pixel group

[0222] 120 First Virtual Area

[0223] 122 Second Virtual Area

[0224] 124 Third Virtual Area

[0225] 140,142,144,146,148 signal lines

[0226] 160 Voltage Source

[0227] 12 Timing control circuit

[0228] 14 Access control circuit

[0229] 16. First reading circuit

[0230] 18. First signal processing circuit

[0231] 20 Second reading circuit

[0232] 22. Second signal processing circuit

[0233] 24 Timestamp generation circuit

[0234] 26 output I / F.

Claims

1. A sensor device comprising a pixel array, wherein: The pixel array comprises: Virtual pixels, do not receive incident light; Grayscale pixels, including sub-grayscale pixels that receive incident light to obtain grayscale information; and an event detection pixel, comprising the sub-grayscale pixel and the sub-event detection pixel, the sub-event detection pixel detecting a change in the grayscale information obtained by receiving incident light, the virtual pixel, the grayscale pixel and the event detection pixel being arranged in an array in a row direction and a column direction; and A signal is acquired by simultaneously accessing the sub-grayscale pixel and the dummy pixel belonging to a row different from a row to which the sub-grayscale pixel belongs.

2. The sensor device according to claim 1, wherein: The pixel array comprises: a first virtual area, the virtual pixels being arranged in the first virtual area; and A light receiving area is provided in which the grayscale pixels and the event detection pixels are arranged.

3. The sensor device according to claim 2, wherein: The virtual pixels include light-shielded pixels corresponding to grayscale pixels whose light-receiving surfaces have been shielded from light or event detection pixels whose light-receiving surfaces have been shielded from light.

4. The sensor device according to claim 2, wherein: The first virtual area is arranged at least along the row direction on an edge of the pixel array.

5. The sensor device according to claim 4, wherein: The dummy pixels arranged in the first dummy area and the event detection sub-pixels belonging to the light receiving area perform output for each column through the same signal line.

6. The sensor device according to claim 5, wherein: The dummy pixels include pixels arranged in the first dummy area in an arrangement configuration different from an arrangement configuration of the grayscale pixels and the event detection pixels in the light receiving area, and the pixels are shielded from light.

7. The sensor device according to claim 6, wherein: The virtual pixel does not include a sub-pixel corresponding to the sub-event detection pixel of the event detection pixel that is shielded from light.

8. The sensor device according to claim 6, wherein: The pixel array includes a plurality of the grayscale pixels and a plurality of the event detection pixels, and switches which sub-pixel in each pixel acquires and outputs a signal in response to a trigger signal, and In response to the trigger signal, a connection to the virtual pixel corresponding to the sub-event detection pixel that is shielded from light is not established.

9. The sensor device according to claim 3, further comprising: a plurality of column signal lines through which signals output from pixels belonging to one of the columns are propagated, wherein One column signal line among the plurality of column signal lines is exclusively connected to the dummy pixel.

10. The sensor device according to claim 3, further comprising: a plurality of column signal lines, in a unit pixel group including two pixels arranged continuously in the row direction and a plurality of pixels arranged continuously in the column direction, through which signals output from the unit pixel group belonging to the column direction are propagated, wherein Pixels belonging to the same row and having the same position in the unit pixel group are connected to column signal lines provided at the same relative position among a plurality of column signal lines.

11. The sensor device according to claim 3, further comprising: a plurality of column signal lines, in a unit pixel group including two pixels arranged continuously in the row direction and a plurality of pixels arranged continuously in the column direction, through which signals output from the unit pixel group belonging to the column direction are propagated, wherein Pixels that belong to the same row and are located at the same position in the unit pixel group are connected to column signal lines provided at different relative positions among a plurality of column signal lines.

12. The sensor device according to claim 10, wherein: The unit pixel group includes eight pixels, and the eight pixels include: two pixels arranged consecutively in a row direction; and Four pixels are arranged consecutively in the column direction.

13. The sensor device according to claim 4, further comprising: a second virtual area in a row different from the row of the first virtual area on the edge of the pixel array, the second virtual area including the virtual pixels corresponding to pixels having the same configuration as the row in the light receiving area and being shielded from light, wherein, A reference value is calculated based on a signal value output from the virtual pixel belonging to the second virtual area, the reference value being used to correct a signal value output from the pixel belonging to the light receiving area.

14. The sensor device according to claim 13, wherein: The signal value output from the first virtual area is compared with the reference value, and a correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected is acquired.

15. The sensor device according to claim 14, wherein: Grayscale information in the correction area is corrected based on the signal value output from the first virtual area.

16. The sensor device according to claim 4, further comprising: A third virtual area, in which the virtual pixels are arranged on the edge of the pixel array along the column direction so as not to overlap with the first virtual area, wherein: A reference value is calculated based on a signal value output from the virtual pixel belonging to the third virtual area, the reference value being used to correct a signal value output from the pixel belonging to the light receiving area.

17. The sensor device according to claim 16, wherein: The signal value output from the first virtual area is compared with the reference value, and a correction area corresponding to a pixel area after the signal value output from the light receiving area is corrected is acquired.

18. The sensor device according to claim 17, wherein: Grayscale information in the correction area is corrected based on the signal value output from the first virtual area.

19. The sensor device according to claim 2, wherein: The virtual pixel includes an analog virtual pixel, and the analog virtual pixel outputs a predetermined analog voltage.

Images