Display device with detection function

Abstract

The display device having a detection function according to the present disclosure includes: a display unit having a transmission region through which light passes; and a sensor unit having a light emitting unit and a light receiving unit arranged on a rear surface side of the display unit, the sensor unit measuring a flight time at which light emitted by the light emitting unit passes through the transmission region, is reflected by an object within a specific distance range from the display unit, passes through the transmission region, and is received by the light receiving unit, and thereby measuring the distance to the object within the specific distance range.

Description

Technical Field

[0001] The present disclosure relates to a display device with a detection function. Background Art

[0002] There is a technology that includes a plurality of photodetectors and a plurality of illumination sources on the rear surface side of a display screen, and uses the plurality of photodetectors to detect light output from the plurality of illumination sources, passing through the display screen, and reflected by a target object (see Patent Document 1). According to this technology, ranging and recognition are performed on a target object such as a finger by a stereo method.

[0003] Reference List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Unexamined Patent Application Publication (published Japanese translation of PCT application) No. JP2015-529372. Summary of the invention

[0006] When using the above-mentioned technology based on the stereo method, sometimes unnecessary light reflected by the display material or the display protective film is superimposed on the detection result, and this makes it difficult to identify the feature points. Unless the feature points are correctly identified, distance measurement cannot be achieved, and failure to correctly identify the feature points also deteriorates the recognition accuracy of the target object.

[0007] Therefore, it is desirable to provide a display device having a detection function capable of improving the detection accuracy of a target object within a specific distance range.

[0008] According to an embodiment of the present disclosure, a first display device with a detection function includes: a display unit having a light transmission area; and a sensor unit including a light emitting unit and a light receiving unit arranged on the rear surface side of the display unit. The sensor unit performs distance measurement of a target object within a specific distance range by measuring the flight time of light, the light is output from the light emitting unit, passes through the transmission area, is reflected by a target object within a specific distance range from the display unit, passes through the transmission area, and enters the light receiving unit.

[0009] According to an embodiment of the present disclosure, a second display device with a detection function includes: a display unit having a light transmission area; and a sensor unit including a light emitting unit and a light receiving unit arranged on the rear surface side of the display unit. The sensor unit performs ranging of a target object within a specific distance range by measuring the flight time of light, and the light is output from the light emitting unit, passes through the transmission area, is reflected by a target object within a specific distance range from the display unit, passes through the transmission area, and enters the light receiving unit. The light receiving unit includes: a light receiving element; and a switch element, which activates the light receiving element when receiving reflected light from a target object within a specific distance range based on a timing signal indicating an effective period of ranging.

[0010] When using the first display device or the second display device with a detection function according to an embodiment of the present disclosure, the sensor unit includes a light emitting unit and a light receiving unit arranged on the rear surface side of the display unit, and the sensor unit performs ranging of a target object within a specific distance range by measuring the flight time of light. Light is output from the light emitting unit, reflected by the target object within the specific distance range, and enters the light receiving unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a cross-sectional view schematically showing a configuration example of a display device with a detection function according to an embodiment of the present disclosure.

[0012] Figure 2 is a plan view schematically showing an example of a pixel structure of a display device with a detection function according to an embodiment.

[0013] Figure 3 is a cross-sectional view schematically showing a modification of the pixel structure according to the embodiment.

[0014] Figure 4 is a plan view schematically showing a modification of the pixel structure of the display device with a detection function according to the embodiment.

[0015] Figure 5 is a cross-sectional view schematically showing a modification of the display device with a detection function according to the embodiment.

[0016] Figure 6 is a cross-sectional view schematically showing a modification of the light receiving section of the display device with a detection function according to the embodiment.

[0017] Figure 7 is a plan view schematically showing an example of a module structure of a display device with a detection function according to an embodiment.

[0018] Figure 8 is a block diagram schematically showing a first configuration example of a circuit configuration of a display device with a detection function according to an embodiment.

[0019] Fig. 9 is a timing chart showing an example of a detection process performed by the display device with a detection function according to the embodiment.

[0020] Fig.10 is a block diagram schematically showing a second configuration example of the circuit configuration of the display device with a detection function according to the embodiment.

[0021] Fig.11 is a schematic diagram showing the Fig.100 is a circuit diagram of a configuration example of one of the pixels of the light receiving section of the second configuration example shown in FIG.

[0022] Fig.12 is a schematic diagram showing the Fig.10 2 is a circuit diagram of an operation example of a pixel of a light receiving portion of a second configuration example shown in FIG.

[0023] Fig.13 is a flowchart showing a first example of identification / authentication processing performed by the display device with a detection function according to the embodiment.

[0024] Fig.14 is a flowchart showing a second example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0025] Fig.15 is a flowchart showing a third example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0026] Fig.16 is a flowchart showing a fourth example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0027] Fig.17 is a block diagram schematically showing a first example of a chip configuration of a display device with a detection function according to an embodiment.

[0028] Fig.18 is a block diagram schematically showing a second example of a chip configuration of a display device with a detection function according to an embodiment.

[0029] Fig.19 is a block diagram schematically showing a third example of a chip configuration of the display device with a detection function according to the embodiment.

[0030] Fig. 20 is a block diagram schematically showing a fourth example of a chip configuration of the display device with a detection function according to the embodiment.

[0031] Fig.21 is a block diagram schematically showing a fifth example of a chip configuration of the display device with a detection function according to the embodiment. DETAILED DESCRIPTION

[0032] Next, with reference to the drawings, details of an embodiment of the present disclosure will be described. It should be noted that the description will be given in the following order.

[0033] 1. Implementation Method

[0034] 1.1. Configuration

[0035] 1.1.1. Overall configuration example ( Figures 1 to 7 )

[0036] 1.1.2. Circuit Configuration Example ( Figures 8 to 12 )

[0037] 1.2. Identification / Authentication Processing ( Figures 13 to 16 )

[0038] 1.3.Chip Configuration Example ( Figures 17 to 21 )

[0039] 1.4. Effect

[0040] 2. Other Implementation Methods

[0041] <1. Implementation Method>

[0042] [1.1. Configuration]

[0043] (1.1.1. Overall configuration example)

[0044] Figure 1 A configuration example of a display device with a detection function according to an embodiment of the present disclosure is schematically shown. Figure 2 An example of a pixel structure of a display device with a detection function according to an embodiment is schematically shown.

[0045] The display device with a detection function according to the embodiment includes a display section 1 and a sensor section 2. The sensor section 2 includes a light emitting section 20 and a light receiving section 30 arranged on the rear surface side of the display section 1.

[0046] The display section 1 includes a pixel section 10. The pixel section 10 has a front surface provided with a protective glass 11. The protective glass 11 has a front surface on which a protective film 12 may be provided.

[0047] The pixel section 10 includes an image display array in which a plurality of display pixels are arranged two-dimensionally. The plurality of display pixels may be light emitters that emit light of a visible wavelength, such as an organic light emitting diode (OLED). For example, the plurality of display pixels include an R (red) pixel 10R, a G (green) pixel 10G, and a B (blue) pixel 1B.

[0048] The display section 1 has a light transmission area that transmits light between its front surface and rear surface. As the transmission area, the display section 1 includes a first transmission area 70 and a second transmission area 80. The first transmission area 70 is provided in the optical path through which the output light L1 from the light emitting section 20 passes. The second transmission area 80 is provided in the optical path through which the reflected light L2 from the target object 200 within the specific distance range Da passes before the reflected light L2 enters the light receiving section 30.

[0049] The area density of the plurality of display pixels of the display section 1 may be uniform over the entire surface of the display section. In the case where the area density is uniform over the entire surface of the display section, no particular structure is required, and this makes it possible to implement the display section 1 at a low cost. In addition, the display section 1 may have a structure in which the area density of the plurality of display pixels in at least one of the first transmission area 70 and the second transmission area 80 is lower than the area density of the plurality of display pixels in the area other than the transmission area. Preferably, it is better if the area density of the plurality of display pixels in at least the second transmission area 80 is lower than the area density of the plurality of display pixels in the area other than the second transmission area 80.

[0050] Figure 1 and Figure 2 A configuration example is shown in which the display section 1 has a structure in which the area density of the plurality of display pixels in the first transmission region 70 is lower than the area density of the plurality of display pixels in the region other than the first transmission region 70. This configuration example makes it possible to improve the transmittance of the output light L1 from the light emitting section 20 in the display section 1, so that the output light L1 can be effectively emitted to the target object 200, and the ranging and recognition accuracy can be improved.

[0051] Figure 3 and Figure 4 A modified example of the pixel structure of the display device with a detection function according to the embodiment is schematically shown. Figure 3 and Figure 4 A configuration example is shown in which the area density of the plurality of display pixels in the second transmission region 80 is lower than the area density of the plurality of display pixels in the region other than the second transmission region 80. This configuration example makes it possible to increase the transmittance of the reflected light L2 toward the light receiving section 30 in the display section 1, makes it possible to increase the amount of light received by the light receiving section 30, and improves the ranging and recognition accuracy.

[0052] The light emitting section 20 and the light receiving section 30 are arranged in the housing 3. The housing 3 includes a condenser lens 40 between the light emitting section 20 and the display section 1. The housing 3 also includes a filter 50 and a condenser lens 60 between the light receiving section 30 and the display section 1.

[0053] The structures of the condenser lens 40 and the condenser lens 60 are not particularly limited. However, if a Fresnel lens or a metasurface is used, an effect of making the housing 3 thinner is expected. The optical filter 50 selectively transmits light of a wavelength band including the output light L1 from the light emitting section 20 .

[0054] For example, the sensor section 2 makes it possible to measure the distance to the target object 200 by a direct time-of-flight (dToF) method. The dToF method is a method of emitting light to the target object 200, receiving reflected light L2 from the target object 200, and measuring the flight time of the light to measure the distance to the target object 200. In order to measure the flight time, a time digital converter (TDC) 91 ( Figure 8 The TDC 91 is configured to convert the elapsed time into a digital signal. The TDC 91 sequentially generates a time code in a time series from the start of emitting light to the target object 200, and acquires the time code generated when the reflected light L2 is received. By outputting a digital signal corresponding to the elapsed time of the acquired time code, the flight time of the light can be detected.

[0055] The sensor section 2 performs ranging of the target object 200 within the specific distance range Da by measuring the flight time of light, which is output from the light emitting section 20, passes through the first transmission area 70, is reflected by the target object 200 within the specific distance range Da, passes through the second transmission area 80, and enters the light receiving section 30. Based on a timing signal (effective ranging period identification signal) indicating an effective period of ranging (described later) (see Fig. 9 In (A)), the sensor unit 2 excludes the reflected light L2 from the position deviating from the specific distance range Da from the target of distance measurement.

[0056] The display device with a detection function can generate at least one of a reflected light intensity map (reflected light intensity image) and a depth map (depth image) based on the result of the distance measurement performed by the sensor unit 2. Next, at least one of a recognition process and an authentication process can be performed on the target object 200 based on at least one of the reflected light intensity map and the depth map.

[0057] The light emitting section 20 includes one or more light emitting sources 21 that emit light of wavelengths other than visible wavelengths. For example, as output light L1, the light emitting source 21 may be a laser diode that emits pulsed near-infrared light. In the case where the light emitting section 20 includes only one light emitting source 21, this makes the structure of the light emitting source 21 and the control of the driver that drives the light emitting source 21 simpler at low cost. In addition, compared with the case where the light emitting section 20 includes a plurality of light emitting sources 21, this case eliminates the influence of variations in light emission intensity between the plurality of light emitting sources 21, and achieves a uniform amount of light suitable for authentication.

[0058] Figure 5A modified example of the light emitting unit 20 is schematically shown. The light emitting unit 20 may include a plurality of light emitting sources 21. The plurality of light emitting sources 21 may be arranged two-dimensionally. In the case where the plurality of light emitting sources 21 are arranged two-dimensionally, a region of interest (ROI) or a target region may be specified according to the position and size of the target object 200. For example, after specifying the position and size of the target object 200, power may be suppressed by emitting light only from the light emitting source 21 in the region corresponding to the target object 200.

[0059] The light receiving unit 30 includes a plurality of light receiving pixels. For example, the plurality of light receiving pixels may be a light receiving element 31 including a single photon avalanche diode (SPAD). SPAD is an avalanche photodiode (APD) that improves the charge multiplication effect by applying a reverse bias voltage exceeding the breakdown voltage. In SPAD, the charge generated by photoelectric conversion increases rapidly due to the high multiplication effect. The increased charge causes the flow of a rapidly rising current in the SPAD. By detecting the current and generating a pulse signal, the incidence of a single photon can be detected. This mode of operation is called Geiger mode.

[0060] The plurality of light receiving pixels (light receiving elements) 31 may be SPAD pixels arranged two-dimensionally. In the case where the plurality of light receiving elements 31 are arranged two-dimensionally, for example, power can be suppressed by activating only the light receiving pixels in the area corresponding to the target object 200 after specifying the position and size of the target object 200. Unnecessary light L3 from other than the target object 200 can also be suppressed.

[0061] Regarding the light receiving unit 30, a TDC 91 ( Figure 8 The circuits of the plurality of light receiving elements 31 and the plurality of light receiving elements 31 can be formed on the same substrate. In this case, the sensor section 2 having a thinner thickness can be realized, and this improves the degree of freedom of product design.

[0062] Figure 6 The modified example of the light receiving unit 30 is schematically shown. Figure 8 A circuit (etc.) (described later) may be formed on a substrate 32 different from the plurality of light receiving elements 31. For example, a substrate on which a plurality of light receiving elements 31 are formed may be stacked above a substrate 32 on which a circuit including a TDC 91 is formed. This realizes a stacked sensor structure that electrically connects the plurality of light receiving elements 31 to the TDC 91. This makes it possible to narrow the pixel pitch between the plurality of light receiving elements. Therefore, the spatial resolution required for authentication can be achieved by using a small chip, and this reduces costs. In addition, in the case of a structure in which the area density of display pixels corresponding to the light receiving portion 30 is reduced and the transmittance in the second transmission region 80 is increased in the display portion 1 ( Figure 3and Figure 4 ), which makes it possible to reduce the area of ​​the second transmission region 80, and to alleviate the degradation of the image quality of the display section 1 (the difference in image quality between the second transmission region 80 and the region other than the second transmission region 80).

[0063] Figure 7 An example of a module structure of a display device with a detection function according to an embodiment is schematically shown.

[0064] For example, a product (such as a smartphone) provided with a display device having a detection function may include an RGB sensor 4 that acquires an RGB image as another sensor. In addition, the display device having a detection function may perform at least one of a recognition process and an authentication process by combining the RGB image acquired by the RGB sensor 4 with a depth image or a reflected light intensity image acquired based on the detection result performed by the sensor unit 2. In this case, the RGB sensor 4 and the sensor unit 2 are preferably arranged at positions close to each other. As the distance (baseline length) between the two sensors including the RGB sensor 4 and the sensor unit 2 becomes shorter, the correction operation cost is reduced. In addition, the shorter inter-sensor distance makes it possible to increase the spatial resolution. It should be noted that there is no specific limitation on the structure of the RGB sensor 4. In a manner similar to the sensor unit 2, the RGB sensor 4 can be arranged on the rear surface side of the display unit 1.

[0065] (1.1.2. Circuit Configuration Example)

[0066] Figure 8 A first configuration example of a circuit configuration of a display device with a detection function according to an embodiment is shown. Figure 8 The circuit configuration of a portion related to distance measurement, recognition processing, and authentication processing in a display device with a detection function is shown. Fig. 9 is a timing chart showing an example of a detection process performed by the display device with a detection function according to the embodiment. Fig. 9 shows a light emission timing signal ( Fig. 9 (A) in), time code signal ( Fig. 9 (B) in the effective ranging period identification signal ( Fig. 9 (C) in), SPAD trigger timing signal ( Fig. 9 (D) in the figure and the output from TDC 91 ( Fig. 9 (E) in the timing diagram.

[0067] The display device with a detection function includes a light emitting section 80 , a light receiving section 30 , a timing generating section 90 , a TDC 91 , a histogram counter 92 , a trigger number counter 93 , a distance calculating / image processing section 94 , and an authentication / recognition calculating section 95 .

[0068] The timing generating section 90 generates a light emission timing signal ( Fig. 9 (A) in FIG. 1 ), and outputs the generated signal to the light emitting section 20. In addition, the timing generating section 90 generates a time code signal ( Fig. 9 (B) in the figure), and outputs the generated signal to TDC91. In addition, the timing generation unit 90 generates an effective ranging period identification signal ( Fig. 9 (C) in FIG. 1 ), and outputs the generated signal to TDC 91. The timing generation section 90 may also output the effective ranging period identification signal to the trigger number counter 93.

[0069] Based on the SPAD trigger timing signal ( Fig. 9 (D) in the figure) and the time code signal and the effective distance measurement period identification signal from the timing generation unit 90, the TDC 91 generates a digital signal ( Fig. 9 In (E)), light is output from the light emitting portion 20 , reflected by the target object 200 , and enters the light receiving portion 30 .

[0070] The effective ranging period identification signal is a timing signal indicating the effective period and invalid period of ranging. The effective period of ranging is a period corresponding to the distance range in which ranging is effective, and is a period including the time when the light (output light L1) output from the light emitting section 20 is reflected by the target object 200 within the specific distance range Da and enters the light receiving section 30 as the reflected light L2. The effective period of ranging is set in such a way that the effective period of ranging does not include the time when the unnecessary light L3 enters the light receiving section 30. In other words, the invalid period of ranging is a period including the time when the unnecessary light L3 enters the light receiving section 30. In this way, the effective distance range of ranging can be controlled by using the effective ranging period identification signal. For example, the unnecessary light L3 is the light generated when the output light from the light receiving section 30 is reflected by the protective glass 11, the protective film 12, etc. Therefore, the SPAD trigger timing signal based on the reflected light (unnecessary light L3) from the position deviating from the specific distance range Da can be excluded from the target of ranging, and the SPAD trigger timing signal based only on the reflected light L2 from the specific distance range Da can be included in the target of ranging.

[0071] The histogram counter 92 generates a histogram of the flight time based on the digital signal indicating the flight time output from the TDC 91. The generated histogram is output to the distance calculation / image processing section 94.

[0072] The trigger number counter 93 is based on the effective distance measurement period identification signal and the SPAD trigger timing signal ( Fig. 9 (D) in the figure counts the number of triggers of the SPAD pixel. By counting the number of triggers only in the effective period of ranging based on the effective ranging period identification signal, the trigger number counter 93 can count the number of triggers based only on the reflected light L2 from the specific distance range Da.

[0073] The distance calculation / image processing section 94 calculates the distance to the target object 200 based on the counter value from the trigger number counter 93 and the histogram from the histogram counter 92 , and generates at least one of a reflected light intensity map and a depth map based on the calculation result of the distance.

[0074] The authentication / recognition calculation unit 95 is a calculation unit that performs at least one of recognition processing and authentication processing on the target object 200 based on at least one of the reflected light intensity map and the depth map. The authentication / recognition calculation unit 95 can receive the RGB sensor 4 ( Figure 7 The recognition / authentication calculation unit 95 can perform at least one of recognition processing and authentication processing by combining the RGB image acquired by the RGB sensor 4 and at least one of the reflected light intensity map and the depth map.

[0075] According to the first configuration example, by providing a valid ranging period identification signal to TDC 91 and trigger number counter 93, the result of TDC 91 can be invalidated during the invalid period of ranging, and trigger number counter 93 can stop counting the number of triggers. The first configuration example realizes control of ranging by using a pure logic circuit.

[0076] Fig.10 A second configuration example of the circuit configuration of the display device with a detection function according to the embodiment is shown.

[0077] According to the second configuration example, a valid ranging period identification signal is provided to the light receiving section 30. Therefore, by deactivating the SPAD pixel during the invalid period of ranging, the SPAD trigger timing signal based on the reflected light (unnecessary light L3) from a position deviating from the specific distance range Da can be excluded from the target of ranging, and the SPAD trigger timing signal based only on the reflected light L2 from the specific distance range Da can be included in the target of ranging. It should be noted that when the reverse bias voltage (breakdown voltage) for turning on the Geiger mode is not applied to the light receiving element 31, the SPAD pixel is deactivated.

[0078] Fig.11 The schematic diagram shows the Fig.10The configuration example of one of the SPAD pixels of the light receiving section 30 of the second configuration example shown in FIG. Fig.11 A configuration example in which the light receiving element 31 is used as a cathode readout element is shown. Fig.12 Schematically shows the Fig.10 An operation example of the SPAD pixel of the light receiving section 30 of the second configuration example shown in FIG.

[0079] Per one SPAD pixel, the light receiving section 30 includes a light receiving element 31 , a constant current source 102 , a buffer amplifier 103 , an n-type metal oxide semiconductor (NMOS) transistor 104 , and an inverter 105 .

[0080] like Fig.12 As shown, by applying a reverse bias voltage exceeding the breakdown voltage as the reverse bias voltage, the light receiving element 31 is activated and enters the Geiger mode. In the Geiger mode, the incidence of photons can be detected. The light receiving element 31 has an anode to which a negative anode voltage is applied. The light receiving element 31 has a cathode coupled to a constant current source 102, a buffer amplifier 103 and an NMOS transistor 104. The constant current source 102 provides a power supply voltage VDD.

[0081] The buffer amplifier 103 amplifies the signal generated by the light receiving element 31 , and outputs the amplified signal to the TDC 91 as a light receiving signal.

[0082] The inverter 105 receives an input of the effective ranging period identification signal. The NMOS transistor 104 is a switching element that activates the light receiving element 31 when receiving the reflected light L2 from the target object 200 within the specific distance range Da based on the effective ranging period identification signal input via the inverter 105. The NMOS transistor 104 is coupled to the cathode of the light receiving element 31, the constant current source 102 (predetermined voltage line), and the buffer amplifier 103.

[0083] The NMOS transistor 104 controls the voltage to be applied based on the effective ranging period identification signal so that the voltage between the cathode and the anode becomes less than or equal to the breakdown voltage during the invalid period of ranging. It should be noted that in the case of anode readout, the anode of the light receiving element 31 is coupled to the PMOS transistor as a switching element to control the voltage to be applied. This makes it possible to control the effective distance range of ranging by the switching operation of the NMOS transistor 104 or the PMOS transistor used as a switching element.

[0084] According to the first configuration example, even during the invalid period of ranging, power is consumed with the triggering of the SPAD pixel of the light receiving unit 30. In addition, if the dead time is long after the SPAD pixel is triggered, it will be difficult to detect the reflected light L2 from the target object 200 in a close range. In contrast, according to the second configuration example, the SPAD pixel is deactivated during the invalid period of ranging, and this makes it possible to suppress power consumption. The detection accuracy of the reflected light L2 from the target object 200 in a close range can also be improved.

[0085] [1.2. Identification / Authentication Process]

[0086] Fig.13 is a flowchart showing a first example of identification / authentication processing performed by the display device with a detection function according to the embodiment.

[0087] For example, when the user starts authentication by starting an application for authentication, the distance calculation / image processing section 94 first acquires a depth map (step S101). In addition, the distance calculation / image processing section 94 detects the presence or absence of the target object 200. In addition, the distance calculation / image processing section 94 can determine the authenticity of the target object 200. Here, in the case where the target object 200 is not detected, the distance calculation / image processing section 94 repeats the processing in step S101. On the contrary, in the case where it is determined that the target object 200 is detected, next, the distance calculation / image processing section 94 acquires a reflected light intensity map (step S102). It should be noted that in the case where the distance calculation / image processing section 94 has determined that the target object 200 is detected, the sensor section 2 can limit the target area (ROI) for performing the distance measurement for the next and subsequent identification / authentication processing (at least once) based on at least one of the reflected light intensity map and the depth map. For example, when the identification / authentication processing is performed again on the same target object 200, the distance calculation / image processing section 94 can generate a reflected light intensity map and a depth map based on the result of the distance measurement using the limited target area. For example, the sensor section 2 may set the light receiving section 30 to an ROI corresponding to the position and size of the target object 200 determined based on at least one of the reflected light intensity map and the depth map. This may activate only the light receiving pixels within the region corresponding to the target object 200. The same applies to Figures 14 to 16 The second example to the fourth example shown in (described later).

[0088] Next, the authentication / recognition calculation unit 95 performs at least one of the recognition process and the authentication process by using the reflected light intensity map (step S103). Note that the authentication / recognition calculation unit 95 can also perform at least one of the recognition process and the authentication process by adding a signal from a sensor such as an RGB sensor 4 ( Figure 7 ) and performs at least one of recognition processing and authentication processing.

[0089] As described above, according to the first example, the reflected light intensity map and the depth map are acquired at different times. In addition, the reflected light intensity map is used alone for identification processing or authentication processing.

[0090] Fig.14 is a flowchart showing a second example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0091] In the second example, the processing in step S101 and step S102 is similar to that in the above-mentioned first example ( Fig.13 According to the second example, the authentication / recognition calculation unit 95 performs at least one of the recognition process and the authentication process by using the reflected light intensity map and the depth map (step S203). Note that the authentication / recognition calculation unit 95 can also perform at least one of the recognition process and the authentication process by adding a signal from a sensor such as an RGB sensor 4 ( Figure 7 ) and performs at least one of recognition processing and authentication processing on an image acquired by another sensor.

[0092] As described above, according to the second example, the reflected light intensity map and the depth map are acquired at different times. In addition, the reflected light intensity map and the depth map are used for identification processing or authentication processing.

[0093] Fig.15 is a flowchart showing a third example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0094] For example, when the user starts authentication by starting an application for authentication, the distance calculation / image processing unit 94 first acquires a reflected light intensity map and a depth map (step S301). In addition, the distance calculation / image processing unit 94 detects the presence or absence of the target object 200. In addition, the distance calculation / image processing unit 94 can determine the authenticity of the target object 200. Here, in the case where the target object 200 is not detected, the distance calculation / image processing unit 94 repeats the processing in step S301. On the contrary, in the case where it is determined that the target object 200 is detected, next, the authentication / recognition calculation unit 95 performs at least one of the recognition processing and the authentication processing by using the reflected light intensity map (step S302). Note that the authentication / recognition calculation unit 95 can also be further configured by adding a signal from a sensor such as an RGB sensor 4 ( Figure 7 ) and performs at least one of recognition processing and authentication processing on an image acquired by another sensor.

[0095] As described above, according to the third example, the reflected light intensity map and the depth map are acquired at the same time. In addition, the reflected light intensity map is used alone for the identification process or the authentication process.

[0096] Fig.16is a flowchart showing a fourth example of the identification / authentication process performed by the display device with a detection function according to the embodiment.

[0097] In the fourth example, the processing in step S301 is similar to that in the above-mentioned third example ( Fig.15 According to the fourth example, the authentication / recognition calculation unit 95 performs at least one of the recognition process and the authentication process by using the reflected light intensity map and the depth map (step S402). Note that the authentication / recognition calculation unit 95 can also perform at least one of the recognition process and the authentication process by adding a signal from a sensor such as an RGB sensor 4 ( Figure 7 ) and performs at least one of recognition processing and authentication processing.

[0098] As described above, according to the fourth example, the reflected light intensity map and the depth map are acquired at the same time. In addition, the reflected light intensity map and the depth map are used for identification processing or authentication processing.

[0099] [1.3.Chip configuration example]

[0100] Fig.17 A first example of a chip configuration of a display device with a detection function according to an embodiment is schematically shown.

[0101] like Fig.17 As shown, the circuits of the parts related to ranging, identification processing and authentication processing in the display device with detection function can be configured as a single chip 111 as a whole. In this case, a single sensor chip can implement the process from ranging to identification processing or authentication processing. This allows a product (such as a smart phone) provided with a display device with detection function to perform identification processing or authentication processing while also using its computing resources for other purposes. A single sensor chip can also autonomously implement identification processing or authentication processing. This makes it possible to immediately perform identification processing or authentication processing again after the failure of identification processing or authentication processing.

[0102] Fig.18 A second example of the chip configuration of the display device with a detection function according to the embodiment is schematically shown.

[0103] like Fig.18As shown, with respect to the circuit of the part related to ranging, identification processing and authentication processing in the display device with detection function, the part of the circuit other than the light emitting unit 20 can be configured as a single chip 111, and the light emitting unit 20 can be configured as another chip 112. In this case, a single sensor chip can implement the process from ranging to identification processing or authentication processing. This allows a product (such as a smart phone) provided with a display device with a detection function to perform identification processing or authentication processing while also using its computing resources for other purposes. A single sensor chip can also autonomously implement identification processing or authentication processing. This makes it possible to immediately perform identification processing or authentication processing again after the failure of identification processing or authentication processing. In addition, because the light emitting unit 20 is configured as another chip 112, the degree of freedom of selection from the light emission source 21 can be increased. This configuration is effective if the light emission source 21 requires a high voltage that is difficult to handle during the manufacturing process of the chip 111.

[0104] Fig.19 A third example of the chip configuration of the display device with a detection function according to the embodiment is schematically shown.

[0105] like Fig.19 As shown, regarding the circuit of the portion related to the distance measurement, identification processing, and authentication processing in the display device with a detection function, the portion of the circuit other than the light emitting section 20, the distance calculation / image processing section 94, and the authentication / identification calculation section 95 may be configured as a single chip 111, the light emitting section 20 may be configured as another chip 112, and the distance calculation / image processing section 94 and the authentication / identification calculation section 95 may be configured as yet another chip 113. In this case, since the light emitting section 20 is configured as another chip 112, the degree of freedom in selecting from the light emission source 21 can be increased. This configuration is effective if the light emission source 21 requires a high voltage that is difficult to handle during the manufacturing process of the chip 111.

[0106] In addition, since the sensor chips (chips 111 and 112) are arranged on the rear surface side of the display unit 1, there may be a restriction on the area or mounting position of the sensor chips (chips 111 and 112). When the distance calculation / image processing unit 94 and the authentication / recognition calculation unit 95 serving as the operation section are implemented by a chip 113 (such as an application processor) different from the sensor chip, the amount of memory that can be used for the operation section can be increased, and the recognition processing and the authentication processing can be performed with high accuracy.

[0107] Fig. 20 A fourth example of the chip configuration of the display device with a detection function according to the embodiment is schematically shown. Fig.21A fifth example of the chip configuration of the display device with a detection function according to the embodiment is schematically shown.

[0108] In addition, if Fig. 20 As shown, regarding the circuit of the portion related to the distance measurement, identification processing, and authentication processing in the display device with a detection function, the portion of the circuit other than the authentication / identification calculation unit 95 can be configured as a single chip 111, and the authentication / identification calculation unit 95 can be configured as another chip 114. In addition, as Fig.21 As shown, regarding the circuit of the portion related to the distance measurement, identification processing, and authentication processing in the display device with a detection function, the portion of the circuit other than the light emitting section 20 and the authentication / identification calculation section 95 can be configured as a single chip 111, the light emitting section 20 can be configured as another chip 112, and the authentication / identification calculation section 95 can be configured as yet another chip 114. When the authentication / identification calculation section 95 serving as the operation section is implemented by a chip 114 (such as an application processor) different from the sensor chip, the amount of memory that can be used for the operation section can be increased, and the identification processing and the authentication processing can be performed with high accuracy.

[0109] [1.4. Effect] As described above, when the display device with a detection function according to the embodiment is used, the sensor section 2 includes the light emitting section 20 and the light receiving section 30 arranged on the rear surface side of the display section 1, and the sensor section 2 performs distance measurement of the target object 200 within the specific distance range Da by measuring the flight time of light, which is output from the light emitting section 20, reflected by the target object 200 within the specific distance range Da, and enters the light receiving section 30. This makes it possible to improve the detection accuracy of the target object 200 within the specific distance range Da.

[0110] In addition, the display device with a detection function according to the embodiment uses a dedicated dToF technology, which makes it possible to detect only a specific distance. Therefore, it becomes feasible to obtain a reflected light image and a depth image, thereby eliminating the influence of interference light (unnecessary light L3) and the like of the protective glass 11 and the protective film 13 of the display unit 1 without the need for image processing. This makes it possible to significantly reduce the image processing for performing authenticity determination, identification processing, and authentication processing of the target object 200. Therefore, the occupation of computing resources can be avoided and the power used for image processing can be reduced. On the contrary, when a stereo method, an indirect ToF (iToF) method, or a structured light method is used, it is difficult to achieve selective exposure based on distance, and this makes it difficult to avoid interference light during ranging.

[0111] It should be noted that the effects described herein are for illustrative purposes only and other effects may exist. The same applies to the effects according to other embodiments described below.

[0112] (Application Example)

[0113] For example, the display device with a detection function according to the embodiment may be applied to an identification process and an authentication process for the following products.

[0114] Authentication processing is performed on a smartphone, laptop, tablet, etc. to perform payment or unlocking operations. For example, authentication processing is performed on a smart watch to display personal information (email, schedule, etc.) only if a specific user is authenticated.

[0115] In the case where the display device with a detection function according to the embodiment is applied to a car navigation display or a vehicle-mounted digital interior mirror, a recognition process or an authentication process is performed by a child presence detection system or a cabin monitoring system.

[0116] An authentication process is performed to automatically set or cancel the age limit of the viewer of the TV. An identification process is performed to recognize that there is no one and switch the TV to a power saving mode.

[0117] An identification process is performed to understand user attributes regarding the digital signage.

[0118] <2. Other Implementation Methods>

[0119] The technology according to the present disclosure is not limited to the above-described embodiments, and various modifications are possible.

[0120] For example, the present technology can be configured as follows.

[0121] According to the present technology having the following configuration, the sensor section includes a light emitting section and a light receiving section arranged on the rear surface side of the display section, and the sensor section performs distance measurement of a target object within a specific distance range by measuring the flight time of light, light is output from the light emitting section, is reflected by the target object within the specific distance range, and enters the light receiving section. This makes it possible to improve the detection accuracy of the target object within the specific distance range. (1)

[0123] A display device with a detection function, the display device comprising:

[0124] A display portion having a light-transmitting area; and

[0125] a sensor section including a light emitting section and a light receiving section arranged on the rear surface side of the display section,

[0126] Among them, the sensor part performs ranging of a target object within a specific distance range by measuring the flight time of light, and the light is output from the light emitting part, passes through the transmission area, is reflected by the target object within a specific distance range from the display part, passes through the transmission area, and enters the light receiving part. (2)

[0128] The display device with a detection function according to (1), wherein the sensor section excludes reflected light from a position deviating from a specific distance range from a target of distance measurement based on a timing signal indicating a valid period of distance measurement. (3)

[0130] A display device with a detection function according to (1) or (2), wherein the transmission area includes a first transmission area and a second transmission area, the first transmission area is an area where the output light from the light emitting unit passes through the display unit, and the second transmission area is an area where the reflected light from the target object within a specific distance range passes through the display unit before the reflected light enters the light receiving unit. (4)

[0132] The display device with detection function according to (3), wherein:

[0133] The display unit includes a plurality of display pixels, and

[0134] The display portion has a structure in which the area density of the plurality of display pixels in at least one of the first transmission area and the second transmission area is lower than the area density of the plurality of display pixels in areas other than the transmission area. (5)

[0136] The display device with a detection function according to any one of (1) to (4), wherein:

[0137] The light emitting unit includes one or more light emitting sources, and

[0138] The light receiving section includes a plurality of single photon avalanche diode (SPAD) pixels. (6)

[0140] The display device with a detection function according to (5), wherein the light emission source emits light of a wavelength other than a visible wavelength. (7)

[0142] The display device with a detection function according to any one of (1) to (6), wherein:

[0143] The light receiving portion includes a plurality of SPAD pixels, and

[0144] SPAD pixels include:

[0145] a light receiving element having an anode and a cathode, and

[0146] The switch element is coupled between the predetermined voltage line and the anode or cathode of the light receiving element. (8)

[0148] The display device with a detection function according to (7), wherein the switching operation of the switch element controls an effective distance range of distance measurement. (9)

[0150] The display device with a detection function according to any one of (1) to (8), wherein:

[0151] The sensor section further includes a time-to-digital converter that generates a digital signal indicating a time of flight, the light receiving section includes a plurality of SPAD pixels, and

[0152] Multiple SPAD pixels and time-to-digital converters are mounted on the same substrate. (10)

[0154] The display device with a detection function according to any one of (1) to (8), wherein:

[0155] The sensor section further includes a time-to-digital converter that generates a digital signal indicating a time of flight, the light receiving section includes a plurality of SPAD pixels, and

[0156] Multiple SPAD pixels and time-to-digital converters are mounted on different substrates. (11)

[0158] The display device with detection function according to any one of (1) to (10), further comprising:

[0159] The image processing unit generates at least one of a reflected light intensity map and a depth map based on a result of the distance measurement performed by the sensor unit. (12)

[0161] The display device with detection function according to (11) further includes:

[0162] The calculation unit performs at least one of a recognition process and an authentication process on the target object based on at least one of the reflected light intensity map and the depth map. (13)

[0164] The display device with a detection function according to (11) or (12), wherein the sensor section limits the distance measurement target area based on at least one of the reflected light intensity map and the depth map. (14)

[0166] The display device with a detection function according to (12), wherein the calculation unit performs at least one of the recognition process and the authentication process by combining the RGB image acquired by the RGB sensor with at least one of the reflected light intensity map and the depth map. (15)

[0168] A display device with a detection function, the display device comprising:

[0169] A display portion having a light-transmitting area; and

[0170] a sensor section including a light emitting section and a light receiving section arranged on the rear surface side of the display section,

[0171] wherein the sensor portion performs distance measurement of a target object within a specific distance range by measuring the flight time of light, light is output from the light emitting portion, passes through the transmission area, is reflected by a target object within a specific distance range from the display portion, passes through the transmission area, and enters the light receiving portion, and

[0172] The light receiving unit includes:

[0173] a light receiving element, and

[0174] The switch element activates the light receiving element upon receiving reflected light from a target object within a specific distance range based on a timing signal indicating a valid period of distance measurement.

[0175] This application claims the benefit of Japanese Priority Patent Application JP2022-164785 filed in the Japan Patent Office on October 13, 2022, the entire contents of which are incorporated herein by reference.

[0176] It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.

Claims

1. A display device with a detection function, the display device comprising: A display portion having a light transmission area; as well as a sensor section including a light emitting section and a light receiving section arranged on the rear surface side of the display section, The sensor unit performs distance measurement of a target object within a specific distance range by measuring the flight time of light, wherein the light is output from the light emitting unit, passes through the transmission area, is reflected by the target object within the specific distance range from the display unit, passes through the transmission area, and enters the light receiving unit.

2. The display device with detection function according to claim 1, wherein: The sensor section excludes reflected light from a position deviating from the specific distance range from a target of the distance measurement based on a timing signal indicating a valid period of the distance measurement.

3. The display device with detection function according to claim 1, wherein: The transmission area includes a first transmission area and a second transmission area, the first transmission area is an area where the output light from the light emitting unit passes through the display unit, and the second transmission area is an area where the reflected light from the target object within the specific distance range passes through the display unit before entering the light receiving unit.

4. The display device with detection function according to claim 3, wherein: The display unit includes a plurality of display pixels, and The display section has a structure in which an area density of a plurality of display pixels in at least one of the first transmission area and the second transmission area is lower than an area density of a plurality of display pixels in an area other than the transmission area.

5. The display device with detection function according to claim 1, wherein: The light emitting section includes one or more light emitting sources, and The light receiving portion includes a plurality of single photon avalanche diode (SPAD) pixels.

6. The display device with detection function according to claim 5, wherein: The light emission source emits light of wavelengths other than visible wavelengths.

7. The display device with detection function according to claim 1, wherein: The light receiving portion includes a plurality of SPAD pixels, and The SPAD pixel comprises: a light receiving element having an anode and a cathode, and A switch element is coupled between a predetermined voltage line and the anode or the cathode of the light receiving element.

8. The display device with detection function according to claim 7, wherein: The switching operation of the switching element controls the effective distance range of the ranging.

9. The display device with detection function according to claim 1, wherein: The sensor portion further includes a time-to-digital converter that generates a digital signal indicative of the time of flight, The light receiving portion includes a plurality of SPAD pixels, and The plurality of SPAD pixels and the time-to-digital converter are mounted on a same substrate.

10. The display device with detection function according to claim 1, wherein: The sensor portion further includes a time-to-digital converter that generates a digital signal indicative of the time of flight, The light receiving portion includes a plurality of SPAD pixels, and The plurality of SPAD pixels and the time-to-digital converter are mounted on different substrates.

11. The display device with detection function according to claim 1, further comprising: The image processing unit generates at least one of a reflected light intensity map and a depth map based on a result of the distance measurement performed by the sensor unit.

12. The display device with detection function according to claim 11, further comprising: A calculation unit performs at least one of a recognition process and an authentication process on the target object based on the at least one of the reflected light intensity map and the depth map.

13. The display device with detection function according to claim 11, wherein: The sensor section limits a distance measurement target area based on the at least one of the reflected light intensity map and the depth map.

14. The display device with detection function according to claim 12, wherein: The calculation section performs the at least one of the recognition process and the authentication process by combining an RGB image acquired by an RGB sensor with the at least one of the reflected light intensity map and the depth map.

15. A display device with a detection function, the display device comprising: A display portion having a light transmission area; as well as a sensor section including a light emitting section and a light receiving section arranged on the rear surface side of the display section, wherein the sensor portion performs distance measurement of a target object within a specific distance range by measuring a flight time of light, the light being output from the light emitting portion, passing through the transmission area, being reflected by the target object within the specific distance range from the display portion, passing through the transmission area, and entering the light receiving portion, and The light receiving unit comprises: a light receiving element, and The switch element activates the light receiving element upon receiving reflected light from the target object within the specific distance range based on a timing signal indicating a valid period of the distance measurement.

Images