Visual sensor

Abstract

The present disclosure provides a visual sensor. The visual sensor comprises: a target silicon base provided with a first pixel array layer and a second pixel array layer. The first pixel array layer is disposed on a front face of the target silicon base and is used for extracting an optical signal of a first wave band from incident light on the visual sensor. The second pixel array layer is disposed below the first pixel array layer and is used for extracting an optical signal of a second wave band which is transmitted by incident light onto the second pixel array layer. The optical signal of the first pixel array layer located in the first wave band penetrates through the depth range of the target silicon base. The optical signal of the second pixel array layer located in the second wave band penetrates through the depth range of the target silicon base. The penetration depth of the optical signal of the second wave band in the target silicon base is greater than the penetration depth of the optical signal of the first wave band in the target silicon base.

Description

A visual sensor Technical Field

[0001] The present disclosure relates to the field of image sensing technology, and in particular to a visual sensor. Background Art

[0002] With the continuous development of image sensing technology, visual sensors have been widely used in various fields. Visual sensors are instruments that use optical components and imaging devices to obtain image information of the external environment. However, visual sensors in related technologies generally only capture image information in a single wavelength band. For example, ordinary visual sensors can only capture image information in the visible light band, and infrared visual sensors often only capture image information in the infrared band. Summary of the Invention

[0003] The present disclosure provides a visual sensor, comprising a target silicon substrate provided with a first pixel array layer and a second pixel array layer;

[0004] The first pixel array layer is provided on the front surface of the target silicon substrate and is used to extract a light signal of a first wavelength band from the incident light of the visual sensor;

[0005] The second pixel array layer is provided below the first pixel array layer and is used to extract a light signal of a second wavelength band transmitted through the second pixel array layer by the incident light;

[0006] The first pixel array layer is located in the depth range of the light signal of the first wavelength band penetrating the target silicon substrate; the second pixel array layer is located in the depth range of the light signal of the second wavelength band penetrating the target silicon substrate; and the penetration depth of the light signal of the second wavelength band in the target silicon substrate is greater than the penetration depth of the light signal of the first wavelength band in the target silicon substrate.

[0007] In some embodiments, the first wavelength band is a visible light band, and the first pixel array layer is used to capture an image of a target object based on an optical signal in the visible light band;

[0008] The second wavelength band is an infrared light wavelength band, and the second pixel array layer is used for depth detection of a target object based on the flight time of a light signal in the infrared light wavelength band.

[0009] In some embodiments, the visual sensor further comprises a circuit silicon base; the circuit silicon base comprises: a pixel array circuit coupled to the first pixel array layer, and a pixel array circuit coupled to the second pixel array layer.

[0010] In some embodiments, an isolation layer is disposed inside the target silicon substrate; and the first pixel array layer and the second pixel array layer are respectively located on both sides of the isolation layer.

[0011] In some embodiments, the first pixel array layer is located on an upper portion of the target silicon substrate front side, and the second pixel array layer is located on a lower portion of the target silicon substrate front side.

[0012] In some embodiments, the second pixel array layer is located on the back side of the target silicon substrate.

[0013] In some embodiments, the isolation layer comprises an oxygen-implanted isolation layer.

[0014] In some embodiments, the first pixel array layer and the second pixel array layer have different doping concentrations.

[0015] In some embodiments, the first pixel array layer includes an array of first pixel sensing units; the first pixel sensing units are used to determine at least one of the light intensity and the light intensity variation of the light signal in the first wavelength band;

[0016] The second pixel array layer includes an array of second pixel sensing units; the second pixel sensing units are used to determine the light intensity of the light signal in the second wavelength band; the first pixel sensing units and the second pixel sensing units correspond one to one.

[0017] In some embodiments, the visual sensor further includes a light source for emitting a light signal of the second band; the second pixel sensing unit includes a SPAD pixel for receiving the light signal of the second band emitted by the light source; the light source and the second pixel array layer are used to perform distance measurement based on the flight time of the light signal of the second band.

[0018] In some embodiments, the first pixel sensing unit includes: at least one of a first pixel sensing subunit and a second pixel sensing subunit;

[0019] The first pixel sensing subunit is used to determine the light intensity of the light signal in the first wavelength band; and the second pixel sensing subunit is used to determine the change in the light intensity of the light signal in the first wavelength band.

[0020] In some embodiments, the second pixel sensing subunit includes an excitatory photosensitive subunit and an inhibitory photosensitive subunit;

[0021] The excitatory photosensitive subunit and the inhibitory photosensitive subunit are both used to extract the light signal of the first wavelength band and convert the light signal of the first wavelength band into an electrical signal;

[0022] The difference between the electrical signals converted by the excitatory photosensitive unit and the inhibitory photosensitive unit is used as an electrical signal to represent the intensity change of the light signal in the first wavelength band.

[0023] In some embodiments, the first pixel sensing sub-unit includes a red photosensitive sub-unit, a green photosensitive sub-unit, and a blue photosensitive sub-unit.

[0024] The visual sensor provided by the embodiment of the present disclosure is capable of receiving light signals of a first band in the incident light through the first pixel array layer, and receiving light signals of a second band in the incident light that passes through the first pixel array layer through the second pixel array layer, thereby enabling the visual sensor to simultaneously receive light signals of different bands, which helps to perceive different information in the incident light of the visual sensor through the first pixel array layer and the second pixel array layer, thereby improving the performance of the visual sensor and broadening the application scenarios of the visual sensor.

[0025] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings. In the accompanying drawings:

[0027] FIG1 is a schematic structural diagram of a visual sensor provided by an embodiment of the present disclosure;

[0028] FIG2 is a schematic diagram of the penetration distance of light in a silicon substrate according to an embodiment of the present disclosure;

[0029] FIG3 is a schematic diagram of the structure of another visual sensor provided by an embodiment of the present disclosure;

[0030] FIG4 is a schematic structural diagram of the second pixel array layer provided in an embodiment of the present disclosure;

[0031] FIG5 is a diagram comparing the effects of depth maps acquired by a visual sensor provided by an embodiment of the present disclosure and a traditional camera;

[0032] FIG6 is a schematic structural diagram of the first pixel array layer provided in an embodiment of the present disclosure.

[0033] In the figure, a, light signal of the first band; b, light signal of the second band; 100, target silicon base; 200, first pixel array layer; 210, first pixel sensing subunit; 211, red photosensitive subunit; 212, green photosensitive subunit; 213, blue photosensitive subunit; 220, second pixel sensing subunit; 221, excitatory photosensitive subunit; 222, inhibitory photosensitive subunit; 300, second pixel array layer; 310, second pixel sensing unit; 400, isolation layer; 500, circuit silicon base. DETAILED DESCRIPTION

[0034] To enable those skilled in the art to better understand the technical solutions of the present disclosure, exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0035] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0036] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0037] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. Similar words such as "connected" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined as such herein.

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0040] As described in the background technology, visual sensors can often only obtain image information in one wavelength band. For example, ordinary visual sensors can only obtain image information in the visible light band. SPAD (Single photon avalanche diodes) and ToF (Time of flight) depth sensors often measure the depth information of objects in the field of view (i.e., the distance between the object and the depth sensor) based on the flight time of infrared light. After obtaining image information in different wavelength bands through different visual sensors, a complex sensor calibration process is often required for data fusion. In addition, the calibration accuracy is also affected by factors such as lens distortion, calibration method, and calibration object.

[0041] To address the above-mentioned issues, an embodiment of the present disclosure provides a visual sensor capable of acquiring image information in different wavelength bands. FIG1 is a schematic structural diagram of a visual sensor provided by an embodiment of the present disclosure. As shown in FIG1 , the visual sensor may include a target silicon substrate 100 provided with a first pixel array layer 200 and a second pixel array layer 300; the first pixel array layer 200 is provided on the front surface of the target silicon substrate 100, and is used to extract a light signal a of a first wavelength band from the incident light of the visual sensor; the second pixel array layer 300 is provided below the first pixel array layer 200, and is used to extract a light signal b of a second wavelength band transmitted through the incident light into the second pixel array layer 300; wherein the first pixel array layer 200 is located in the depth range in which the light signal a of the first wavelength band penetrates the target silicon substrate 100; the second pixel array layer 300 is located in the depth range in which the light signal b of the second wavelength band penetrates the target silicon substrate 100; and the penetration depth of the light signal b of the second wavelength band in the target silicon substrate 100 is greater than the penetration depth of the light signal a of the first wavelength band in the target silicon substrate 100.

[0042] The visual sensor provided in the embodiments of the present disclosure can be used to shoot target objects to acquire image signals or video signals, wherein the target objects can be static people, dynamic people, static scenes or dynamic scenes, etc., or other forms of objects, which are not limited by the embodiments of the present disclosure.

[0043] In the embodiment of the present disclosure, the first pixel array layer 200 can be used to receive a light signal a in a first wavelength band of incident light from the visual sensor. For example, when the first pixel array layer 200 includes a photosensitive unit, the light signal a in the first wavelength band of the incident light can be received by the photosensitive unit and converted into a corresponding electrical signal. The light signal a in the first wavelength band can be a light signal in at least a portion of the visible light band, the infrared light band, and the ultraviolet light band.

[0044] Similarly, the second pixel array layer 300 is configured to receive a light signal b in a second wavelength band from the incident light of the vision sensor. For example, when the second pixel array layer 300 includes a photosensitive unit, the photosensitive unit can directly extract the light signal b in the second wavelength band from the incident light of the vision sensor and convert the light signal b in the second wavelength band into a corresponding electrical signal. The light signal b in the second wavelength band can be a light signal in at least a portion of the visible light, infrared light, and ultraviolet light bands.

[0045] FIG2 is a schematic diagram of the penetration distance of light in a silicon substrate according to an embodiment of the present disclosure; as shown in FIG2 , light signals of different wavelengths have different penetration distances in the target silicon substrate 100. In general, the penetration distance of visible light photons in the target silicon substrate 100 is between 10 4 In the range of nm, the penetration distance of infrared photons in the target silicon substrate 100 is within 10 4 In addition, the penetration distance of visible light photons is greater than that of most ultraviolet light bands (such as UVA, UVB, and UVC).

[0046] In the embodiment of the present disclosure, both the first pixel array layer 200 and the second pixel array layer 300 are disposed on the target silicon substrate 100. The first pixel array layer 200 is disposed on the front surface of the target silicon substrate 100, and the second pixel array layer 300 is disposed below the first pixel array layer 200. The penetration depth of the optical signal a in the first wavelength band into the target silicon substrate 100 is less than the penetration depth of the optical signal b in the second wavelength band. The first pixel array layer 200 is disposed within the depth range within which the optical signal a in the first wavelength band penetrates the target silicon substrate 100; the second pixel array layer 300 is disposed within the depth range within which the optical signal b in the second wavelength band penetrates the target silicon substrate 100. When the incident light includes the optical signal a in the first wavelength band and the optical signal b in the second wavelength band, the optical signal a in the first wavelength band of the incident light is completely received by the first pixel array layer 200, while the second pixel array layer 300 receives only the optical signal b in the second wavelength band of the incident light, excluding the optical signal a in the first wavelength band of the incident light.

[0047] For example, when the first wavelength is a visible light wavelength and the second wavelength is an infrared wavelength, the thickness of the first pixel array layer 200 may be greater than or equal to the penetration distance of visible light photons in the target silicon substrate 100 in 10 4 nm, so that the second pixel array layer 300 receives only the optical signal in the infrared light band of the incident light, without being interfered by the visible light band of the incident light.

[0048] In the embodiment of the present disclosure, the first pixel array layer 200 and the second pixel array layer 300 are both disposed on the target silicon substrate 100, with the first pixel array layer 200 disposed on the front surface of the target silicon substrate 100 and the second pixel array layer 300 disposed below the first pixel array layer 200. The first pixel array layer 200 receives a light signal a in a first wavelength band of the incident light, while the second pixel array layer 300 receives a light signal b in a second wavelength band of the incident light that passes through the first pixel array layer 200. This enables the visual sensor to simultaneously receive light signals of different wavelength bands, facilitating perception of different information in the incident light of the visual sensor through the first pixel array layer 200 and the second pixel array layer 300, thereby improving the performance of the visual sensor and broadening its application scenarios.

[0049] In some embodiments, the first wavelength band is a visible light band, and the first pixel array layer is used to capture an image of a target object based on an optical signal in the visible light band;

[0050] The second wavelength band is an infrared light wavelength band, and the second pixel array layer is used for depth detection of a target object based on the flight time of a light signal in the infrared light wavelength band.

[0051] In the embodiment of the present disclosure, the target object includes an object in the field of view of the visual sensor.

[0052] In the embodiment of the present disclosure, the first pixel array layer 200 is disposed on the front surface of the target silicon substrate 100 and is used to extract optical signals in the visible light band from the incident light to obtain a color image or a black and white image of an object in the field of view; the second pixel array layer 300 is disposed below the first pixel array layer 200 and is used to extract optical signals b in the second wavelength band of the incident light transmitted through the second pixel array layer 300 to obtain a depth image of the object in the field of view. The first pixel array layer 200 is located within the depth range within which optical signals in the infrared light band penetrate the target silicon substrate 100; the second pixel array layer 300 is located within the depth range within which optical signals in the infrared light band penetrate the target silicon substrate 100; and the penetration depth of optical signals in the infrared light band into the target silicon substrate 100 is greater than the penetration depth of optical signals in the visible light band into the target silicon substrate 100.

[0053] The visual sensor provided by the embodiments of the present disclosure can receive visible light bands to capture images of objects in the field of view, and can also receive light signals in the infrared light band to perform depth detection on objects in the field of view. This helps the visual sensor perceive the information carried by different bands in the incident light, thereby improving the performance of the visual sensor and broadening the application scenarios of the visual sensor.

[0054] In some embodiments, the visual sensor further comprises a circuit silicon base; the circuit silicon base comprises: a pixel array circuit coupled to the first pixel array layer, and a pixel array circuit coupled to the second pixel array layer.

[0055] In embodiments of the present disclosure, the pixel array circuit can be used to directly communicate with the first pixel array layer or the second pixel array layer. Specifically, the pixel array circuit can include circuits such as sense amplifiers, row and column address decoders, etc. In some cases, the pixel array circuit can also be used to provide a clock signal.

[0056] In some embodiments, an isolation layer 400 is disposed inside the target silicon substrate 100 ; the first pixel array layer 200 and the second pixel array layer 300 are respectively located on both sides of the isolation layer 400 .

[0057] As shown in FIG. 1 , the upper portion of the target silicon substrate 100 may be set as the first pixel array layer 200 , the middle portion may be set as the isolation layer 400 , and the lower portion may be set as the second pixel array layer 300 .

[0058] In the embodiment of the present disclosure, by providing an isolation layer 400 between the first pixel array layer 200 and the second pixel array layer 300 , the electronic crosstalk effect in the visual sensor is improved, which helps to improve the imaging quality of the visual sensor.

[0059] In some embodiments, the first pixel array layer 200 is located on the upper portion of the front surface of the target silicon substrate 100 , and the second pixel array layer 300 is located on the lower portion of the front surface of the target silicon substrate 100 .

[0060] In the embodiment of the present disclosure, the front side of the target silicon substrate 100 refers to the side of the target silicon substrate 100 that receives incident light.

[0061] As shown in FIG. 1 , the first pixel array layer 200 is located on the upper portion of the front surface of the target silicon substrate 100 , and the second pixel array layer 300 is located on a side of the target silicon substrate 100 close to the isolation layer 400 .

[0062] In some embodiments, the second pixel array layer 300 is located on the back side of the target silicon substrate 100 .

[0063] In the embodiment of the present disclosure, the back side of the target silicon substrate 100 refers to the other side of the target silicon substrate 100 opposite to the front side.

[0064] Figure 3 is a structural schematic diagram of another visual sensor provided by an embodiment of the present disclosure; as shown in Figure 3, the first pixel array layer 200 is arranged on the front side of the target silicon base 100, and the second pixel array layer 300 is located on the back side of the target silicon base 100.

[0065] In the embodiment of the present disclosure, by locating the second pixel array layer 300 on the back side of the target silicon base 100, compared with locating the second pixel array layer 300 on the side of the target silicon base 100 close to the isolation layer 400 in the previous embodiment, the doping concentration of the upper and lower parts of the target silicon base 100 can be conveniently adjusted, which helps to simplify the processing technology of the visual sensor and reduce the cost of the visual sensor.

[0066] In some embodiments, the isolation layer 400 includes an oxygen-implanted isolation layer.

[0067] In the embodiment of the present disclosure, the isolation layer 400 is generated by oxygen injection isolation technology, that is, by injecting oxygen ions into the target silicon substrate 100 , thereby forming an oxide layer in the middle of the target silicon substrate 100 .

[0068] In the embodiment of the present disclosure, an oxygen-implanted isolation layer is provided in the middle of the target silicon substrate 100 , thereby improving the electronic crosstalk effect in the visual sensor, which helps to improve the imaging quality of the visual sensor.

[0069] In some embodiments, the first pixel array layer 200 and the second pixel array layer 300 have different doping concentrations.

[0070] In the embodiment of the present disclosure, the first pixel array layer 200 and the second pixel array layer 300 have different doping concentrations, thereby improving the spectral sensitivity of the first pixel array layer 200 and the second pixel array layer 300 to a specified wavelength, thereby helping to improve the imaging quality of the visual sensor.

[0071] In some embodiments, the first pixel array layer 200 includes an array of first pixel sensing units; the first pixel sensing units are used to determine at least one of the light intensity and the light intensity variation of the light signal a in the first wavelength band;

[0072] The second pixel array layer 300 includes an array of second pixel sensing units 310 ; the second pixel sensing units 310 are used to determine the light intensity of the light signal b in the second wavelength band; the first pixel sensing units and the second pixel sensing units 310 correspond one to one.

[0073] In the disclosed embodiment, the first pixel sensing unit is configured to determine the light intensity of the light signal a in the first wavelength band, and / or the first pixel sensing unit is configured to determine the intensity variation of the light signal a in the first wavelength band. That is, the first pixel sensing unit may include at least one of a normal pixel that senses light intensity and an event pixel that senses the intensity variation of light intensity.

[0074] In an embodiment of the present disclosure, if the first pixel sensing unit is used to determine the intensity of a light signal in the visible light band, the visual sensor may further include a filter layer corresponding to the first pixel sensing unit. The first pixel sensing unit is used to determine the intensity of different color components in the visible light band, that is, the visual sensor obtains a color image of an object in the field of view based on the first pixel sensing unit. Furthermore, the visual sensor may not be provided with a filter layer. The first pixel sensing unit is used to determine the intensity of light in the visible light band, that is, the visual sensor obtains a grayscale image of an object in the field of view based on the first pixel sensing unit.

[0075] In the embodiment of the present disclosure, the second pixel sensing unit 310 can be used to sense light signals in the infrared light band.

[0076] In the disclosed embodiment, the first pixel sensing units in the first pixel array layer 200 correspond to the second pixel sensing units 310 in the second pixel array layer 300. The visual sensor can be used to acquire image information in both the visible light band and the infrared light band, without requiring a complex registration process before use. Furthermore, the correspondence between the first pixel sensing units and the second pixel sensing units 310 facilitates the fusion processing of image information in the visible light band and the infrared light band, thereby improving image quality in both the visible light band and the infrared light band.

[0077] In some embodiments, the visual sensor further includes a light source for emitting the light signal b of the second band; the second pixel sensing unit 310 includes a SPAD pixel for receiving the light signal b of the second band emitted by the light source; the light source and the second pixel array layer 300 are used to perform ranging based on the flight time of the light signal b of the second band.

[0078] In the embodiment of the present disclosure, the visual sensor may further include a light source for emitting infrared light signals.

[0079] FIG4 is a structural diagram of the second pixel array layer 300 provided in an embodiment of the present disclosure; As shown in FIG4 , the second pixel sensing unit 310 includes a SPAD (Single photon avalanche diodes, single photon avalanche diode) pixel. After the light source of the visual sensor emits the light signal b of the second wavelength band, the SPAD pixel can receive the light signal b of the second wavelength band reflected by the object in the field of view, triggering an avalanche effect (avalanche): Due to the high reverse bias electric field inside the SPAD pixel, a small amount of electrons generated by photon conversion causes the SPAD to produce an avalanche state. At this time, the photoelectric conversion gain is theoretically infinite, generating a digital signal that can be captured by the TDC circuit (Time-to-Digital Converter, time digital conversion circuit), and obtaining the reception time of the light signal b of the second wavelength band. Then, based on the emission time and reception time of the light signal b of the second wavelength band, the flight time of the light signal b of the second wavelength band is determined, and the distance between the object and the visual sensor (i.e., depth information) is obtained, and finally a depth map of the object in the field of view is obtained.

[0080] Figure 5 compares the depth maps captured by the visual sensor provided by an embodiment of the present disclosure and a traditional camera. Figure 5(a) shows a color image of an object in the field of view, Figure 5(b) shows a depth map captured by a TOF depth camera, and Figure 5(c) shows a depth map captured by the visual sensor provided by an embodiment of the present disclosure. The colors in Figures 5(b) and 5(c) represent the depth information of objects in the field of view.

[0081] It is not difficult to see from Figure 5 that the depth map collected by the TOF depth camera has the disadvantages of low resolution and blurred outlines, which is not suitable for close-range perception and is not suitable for high-precision scenes. The depth map collected by the visual sensor provided by the embodiment of the present disclosure has higher resolution, clearer outlines, and more accurate close-range perception. This is because, in the visual sensor provided by the embodiment of the present disclosure, the first pixel sensing unit in the first pixel array layer 200 and the second pixel sensing unit 310 in the second pixel array layer 300 have a clear and definite correspondence; compared with the independent color camera and depth sensor in the related art, the complicated alignment process is eliminated; at the same time, the higher-precision correspondence between the two helps to optimize the depth image based on the color image, thereby obtaining a better quality depth map.

[0082] In some embodiments, the first pixel sensing unit includes: at least one of a first pixel sensing subunit 210 and a second pixel sensing subunit 220;

[0083] The first pixel sensing subunit 210 is used to determine the light intensity of the light signal a in the first wavelength band; the second pixel sensing subunit 220 is used to determine the light intensity change of the light signal a in the first wavelength band.

[0084] FIG6 is a schematic structural diagram of the first pixel array layer 200 provided in an embodiment of the present disclosure. As shown in FIG6 , the first pixel sensing unit includes a first pixel sensing sub-unit 210 and a second pixel sensing sub-unit 220 .

[0085] In the embodiment of the present disclosure, the first pixel sensing unit may only include a first pixel sensing sub-unit 210, and the first pixel sensing sub-unit 210 may be used to obtain a grayscale image of an object in the field of view; the visual sensor may also include a filter layer corresponding to the first pixel sensing unit, and the first pixel sensing unit is used to determine the light intensity of different color components in the visible light band, that is, the visual sensor obtains a color image of the object in the field of view based on the first pixel sensing unit.

[0086] In the embodiment of the present disclosure, the first pixel sensing unit may only include the second pixel sensing sub-unit 220, that is, the first pixel sensing unit can sense the light intensity change of the light signal a in the first wavelength band, and output a current signal representing the light intensity change of the light signal a in the first wavelength band using an asynchronous event address representation method. The output signal is specifically in the form of (X, Y, P, T). Where "X, Y" is the event address (pixel coordinates), "P" is a 4-value event output (including the first sign bit), for example, the P value can represent the light intensity change, and "T" is the time when the event occurs, such as the shooting time.

[0087] In some embodiments, the second pixel sensing sub-unit 220 includes an excitatory photosensitive sub-unit 221 and an inhibitory photosensitive sub-unit 222;

[0088] The excitatory photosensitive subunit 221 and the inhibitory photosensitive subunit 222 are both used to extract the light signal a in the first wavelength band and convert the light signal a in the first wavelength band into an electrical signal;

[0089] The difference between the electrical signals converted by the excitatory photosensitive unit 221 and the inhibitory photosensitive unit 222 is an electrical signal used to represent the light intensity change of the light signal a in the first wavelength band.

[0090] In the disclosed embodiment, the excitatory photosensitive unit 221 is used to simulate an excitatory rod cell, and the inhibitory photosensitive unit 222 is used to simulate an inhibitory rod cell. Both the excitatory photosensitive unit 221 and the inhibitory photosensitive unit 222 are used to extract the light signal a in the first wavelength band and convert the light signal a in the first wavelength band into an electrical signal.

[0091] In the embodiment of the present disclosure, the visual sensor can perform a differential operation on the numerical value corresponding to the electrical signal converted by the excitatory photosensitive unit 221 and the electrical signal converted by the adjacent inhibitory photosensitive unit 222 to obtain a differential signal, thereby simulating the excitatory rod cells and inhibitory rod cells of the human eye to obtain an electrical signal representing the intensity change of the light in the pixel area, thereby simulating the rod cells to obtain light intensity gradient information.

[0092] Specifically, the visual sensor can directly subtract the numerical value corresponding to the electrical signal converted by the excitatory photosensitive unit 221 from the numerical value corresponding to the electrical signal converted by the inhibitory photosensitive unit 222 to obtain a differential signal. The visual sensor can also subtract the numerical value corresponding to the electrical signal converted by the excitatory photosensitive unit 221 from the average value corresponding to the electrical signal converted by each adjacent inhibitory photosensitive unit 222 to obtain a differential signal.

[0093] As shown in FIG6 , the first pixel sensing subunits 210 and the second pixel sensing subunits 220 in the first pixel array layer 200 are arranged alternately in an array. A row of the array includes two arrangements: one row is composed entirely of alternating excitatory photosensitive subunits 221 and first pixel sensing subunits 210, or one row is composed entirely of alternating inhibitory photosensitive subunits 222 and first pixel sensing subunits 210, with rows of pixels having excitatory photosensitive subunits 221 alternating with rows of pixels having inhibitory photosensitive subunits 222 in the pixel array. A column of the array includes two arrangements: one row is composed entirely of alternating excitatory photosensitive subunits 221 and first pixel sensing subunits 210, or one column is composed entirely of alternating inhibitory photosensitive subunits 222 and first pixel sensing subunits 210, with columns of pixels having excitatory photosensitive subunits 221 alternating with columns of pixels having inhibitory photosensitive subunits 222 in the pixel array. The excitatory photosensitive subunit 221 and the inhibitory photosensitive subunit 222 are located in different rows and columns of the pixel array, which can enhance the infrared bionic vision sensor's ability to perceive grayscale changes in light signals.

[0094] In some embodiments, the first pixel sensing sub-unit 210 includes a red photosensitive sub-unit 211 , a green photosensitive sub-unit 212 , and a blue photosensitive sub-unit 213 .

[0095] In the disclosed embodiment, the visual sensor may further include a filter layer corresponding to the first pixel sensing subunit 210. The filter layer is provided with filter devices corresponding to the red photosensitive subunit 211, the green photosensitive subunit 212, and the blue photosensitive subunit 213. The filter devices allow not only visible light of the corresponding colors to pass through, but also optical signals in the infrared band to pass through.

[0096] As shown in FIG6 , the red photosensitive subunit 211, green photosensitive subunit 212, and blue photosensitive subunit 213 in the first pixel sensor subunit 210 are arranged in a Bayer pattern. The visual sensor can also convert the light signal a in the first wavelength band into an electrical signal representing its color intensity information through the first pixel sensor subunit 210, thereby simulating cone cells to obtain color intensity information.

[0097] In the embodiment of the present disclosure, the pixel array circuit connected to the first pixel array layer may further include an excitatory control circuit and an inhibitory control circuit connected to the excitatory control circuit; the excitatory control circuit is connected to the excitatory photosensitive unit; the inhibitory control circuit is connected to the inhibitory photosensitive unit; the inhibitory control circuit is used to transmit the current signal converted by the inhibitory photosensitive unit to the excitatory control circuit connected to the inhibitory control circuit. The excitatory control circuit is used to control itself and the inhibitory control circuit to be turned on or off according to the received control signal, and output an electrical signal for characterizing the light intensity change of the light signal of the first wavelength band according to the difference between the current signals converted by the excitatory photosensitive unit and the inhibitory photosensitive unit.

[0098] Based on the same technical concept, the present disclosure also provides another visual sensor, including:

[0099] A target silicon substrate 100 is provided with a first pixel array layer 200, a second pixel array layer 300 and an oxygen implantation isolation layer;

[0100] A circuit silicon substrate 500 provided with a digital circuit;

[0101] An infrared light source used to transmit light signals in the infrared light band.

[0102] The first pixel array layer 200 is disposed on the upper portion of the target silicon substrate 100 and is used to extract optical signals in the visible light band from the incident light of the vision sensor; the second pixel array layer 300 is disposed on the lower portion of the target silicon substrate 100 and is used to extract optical signals in the infrared light band from the incident light that is transmitted through the second pixel array layer 300; and the oxygen-implanted isolation layer is disposed between the second pixel array layers 300. The first pixel array layer 200 is located within the depth range within which optical signals in the first wavelength band penetrate the target silicon substrate 100; the second pixel array layer 300 is located within the depth range within which optical signals in the second wavelength band penetrate the target silicon substrate 100; and the penetration depth of optical signals in the second wavelength band into the target silicon substrate 100 is greater than the penetration depth of optical signals in the first wavelength band into the target silicon substrate 100.

[0103] In the embodiment of the present disclosure, the first pixel array layer 200 includes an array of first pixel sensing units, which include a first pixel sensing sub-unit 210 and a second pixel sensing sub-unit 220. The first pixel sensing sub-unit 210 includes a red photosensitive sub-unit 211, a green photosensitive sub-unit 212, and a blue photosensitive sub-unit 213.

[0104] In the embodiment of the present disclosure, the second pixel array layer 300 includes an array composed of SPAD pixels; the second pixel array layer 300 includes an array composed of SPAD pixels; the SPAD pixels are used to determine the light intensity of the light signal in the infrared light band.

[0105] In the embodiment of the present disclosure, the first pixel sensing sub-units 210 and the second pixel sensing sub-units 220 in the first pixel array layer 200 are arranged alternately in an array. A row of the array includes two arrangements: one row is composed entirely of alternating excitatory photosensitive sub-units 221 and first pixel sensing sub-units 210, or one row is composed entirely of alternating inhibitory photosensitive sub-units 222 and first pixel sensing sub-units 210, and the rows of pixels having excitatory photosensitive sub-units 221 are alternately arranged with the rows of pixels having inhibitory photosensitive sub-units 222 in the pixel array. A column of the array includes two arrangements: one row is composed entirely of alternating excitatory photosensitive sub-units 221 and first pixel sensing sub-units 210, or one column is composed entirely of alternating inhibitory photosensitive sub-units 222 and first pixel sensing sub-units 210, and the columns of pixels having excitatory photosensitive sub-units 221 are alternately arranged with the columns of pixels having inhibitory photosensitive sub-units 222 in the pixel array. In the first pixel array layer 200 , the sum of the photosensitive areas of the first pixel sensing sub-units 210 is substantially the same as the sum of the photosensitive areas of the second pixel sensing sub-units 220 .

[0106] In the disclosed embodiment, the SPAD pixels on the second pixel array layer 300 are arranged in an array; the rows on the second pixel array layer 300 correspond one-to-one with the rows on the first pixel array layer 200. One column on the second pixel array layer 300 corresponds to two columns on the first pixel array layer 200. The two columns on the first pixel array layer 200 corresponding to one column on the second pixel array layer 300 include: a pixel column with excitatory photosensitive subunits 221, and a pixel column with inhibitory photosensitive subunits 222. The first pixel sensing units correspond one-to-one with the SPAD pixels.

[0107] The photosensitive areas of the first pixel sensor subunit 210, the excitatory photosensitive subunit 221, and the inhibitory photosensitive subunit 222 are substantially equal. The photosensitive area of ​​the SPAD pixel is equal to the sum of the photosensitive areas of the first pixel sensor subunit 210 and the excitatory photosensitive subunit 221, or the photosensitive area of ​​the SPAD pixel is equal to the sum of the photosensitive areas of the first pixel sensor subunit 210 and the inhibitory photosensitive subunit 222. The photosensitive region of the SPAD pixel corresponds to the photosensitive regions of the first pixel sensor subunit 210 and the excitatory photosensitive subunit 221, or the photosensitive region of the SPAD pixel corresponds to the photosensitive regions of the first pixel sensor subunit 210 and the inhibitory photosensitive subunit 222.

[0108] In the disclosed embodiment, the visual sensor may further include a filter layer corresponding to the first pixel sensing subunit 210. The filter layer is provided with filter devices corresponding to the red photosensitive subunit 211, the green photosensitive subunit 212, and the blue photosensitive subunit 213. The filter devices allow not only visible light of the corresponding colors to pass through, but also optical signals in the infrared band to pass through.

[0109] In the disclosed embodiment, the visual sensor receives incident light that includes both visible light signals reflected by objects in the field of view and infrared light signals reflected by objects in the field of view. After passing through the filter layer, the visible light signals in the incident light are received by the first pixel sensor subunit 210 and the second pixel sensor subunit 220 on the first pixel array layer 200. The red, green, and blue photosensitive subunits 211, 212, and 213 in the first pixel sensor subunit 210 convert the light signal a in the first wavelength band into an electrical signal representing its color intensity information, thereby simulating cone cells to obtain color intensity information. The excitatory photosensitive subunit 221 and the inhibitory photosensitive subunit 222 in the second pixel sensor subunit 220 are both used to extract light signals in the visible light band and convert the light signals in the visible light band into electrical signals; the difference between the electrical signals converted by the excitatory photosensitive subunit 221 and the inhibitory photosensitive subunit 222 is used to characterize the electrical signal of the light intensity change of the light signal in the visible light band, thereby simulating the excitatory rod cells and inhibitory rod cells of the human eye to obtain the electrical signal of the light intensity change of the light in the pixel area, so as to simulate the rod cells to obtain light intensity gradient information.

[0110] In the disclosed embodiment, after the optical signal in the visible light band of the incident light is absorbed by the first pixel array layer 200, the optical signal in the infrared light band of the incident light is received by the second pixel array layer 300. After receiving the optical signal in the infrared light band, the SPAD pixels on the second pixel array layer 300 trigger an avalanche effect, obtaining the reception time of the optical signal b in the second waveband. The time of flight of the optical signal b in the second waveband is then determined based on the emission time and reception time of the optical signal b in the second waveband, thereby measuring the distance between the object and the visual sensor, and ultimately obtaining a depth map of the object in the field of view.

[0111] In the disclosed embodiments, the visual sensor can be used to acquire color image data, depth image data, and event data of objects in the field of view. The clear correspondence between pixel positions in the color image data, depth image data, and event data helps simplify the fusion of different types of data and optimizes one type of data using other data to improve image quality. This ultimately improves the performance of the visual sensor and broadens its application scenarios.

[0112] It is understood that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

[0113] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly indicated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, it will be understood by those skilled in the art that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A visual sensor, wherein: comprising a target silicon substrate provided with a first pixel array layer and a second pixel array layer; The first pixel array layer is provided on the front surface of the target silicon substrate and is used to extract a light signal of a first wavelength band from the incident light of the visual sensor; The second pixel array layer is provided below the first pixel array layer and is used to extract a light signal of a second wavelength band transmitted through the second pixel array layer by the incident light; The first pixel array layer is located in the depth range of the light signal of the first wavelength band penetrating the target silicon substrate; the second pixel array layer is located in the depth range of the light signal of the second wavelength band penetrating the target silicon substrate; and the penetration depth of the light signal of the second wavelength band in the target silicon substrate is greater than the penetration depth of the light signal of the first wavelength band in the target silicon substrate.

2. The visual sensor according to claim 1, wherein: The first wavelength band is a visible light band, and the first pixel array layer is used to capture an image of a target object based on an optical signal in the visible light band; The second wavelength band is an infrared light wavelength band, and the second pixel array layer is used for depth detection of a target object based on the flight time of a light signal in the infrared light wavelength band.

3. The visual sensor according to claim 1, wherein: The visual sensor further includes a circuit silicon base; the circuit silicon base includes: a pixel array circuit connected to the first pixel array layer, and a pixel array circuit connected to the second pixel array layer.

4. The visual sensor according to claim 1, wherein: An isolation layer is provided inside the target silicon substrate; the first pixel array layer and the second pixel array layer are respectively located on both sides of the isolation layer.

5. The visual sensor according to claim 4, wherein: The first pixel array layer is located on the upper portion of the target silicon substrate front surface, and the second pixel array layer is located on the lower portion of the target silicon substrate front surface.

6. The visual sensor according to claim 4, wherein: The second pixel array layer is located on the back side of the target silicon substrate.

7. The visual sensor according to claim 4, wherein: The isolation layer includes an oxygen-implanted isolation layer.

8. The visual sensor according to claim 4, wherein: The first pixel array layer and the second pixel array layer have different doping concentrations.

9. The visual sensor according to claim 1, wherein: The first pixel array layer includes an array of first pixel sensing units; the first pixel sensing units are used to determine at least one of the light intensity and the light intensity variation of the light signal in the first wavelength band; The second pixel array layer includes an array of second pixel sensing units; the second pixel sensing units are used to determine the light intensity of the light signal in the second wavelength band; The first pixel sensing units and the second pixel sensing units correspond one to one.

10. The visual sensor according to claim 9, wherein: The visual sensor also includes a light source for emitting light signals in the second band; the second pixel sensing unit includes a SPAD pixel for receiving light signals in the second band emitted by the light source; the light source and the second pixel array layer are used to measure distance based on the flight time of the light signals in the second band.

11. The visual sensor according to claim 1, wherein: The first pixel sensing unit includes: at least one of a first pixel sensing subunit and a second pixel sensing subunit; The first pixel sensing subunit is used to determine the light intensity of the light signal in the first wavelength band; the second pixel sensing subunit is used to determine the light intensity change of the light signal in the first wavelength band.

12. The visual sensor according to claim 11, wherein: The second pixel sensing subunit includes an excitatory photosensitive subunit and an inhibitory photosensitive subunit; The excitatory photosensitive subunit and the inhibitory photosensitive subunit are both used to extract the light signal of the first wavelength band and convert the light signal of the first wavelength band into an electrical signal; The difference between the electrical signals converted by the excitatory photosensitive unit and the inhibitory photosensitive unit is used as an electrical signal to represent the intensity change of the light signal in the first wavelength band.

Images