Light source for camera

By integrating an infrared light source and sensor configuration within the camera assembly, the issue of visual obstructions in camera-display integration is resolved, enabling bezel-less designs and improved imaging capabilities.

JP7583786B2Active Publication Date: 2024-11-14SYNAPTICS INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2022505340
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-08-19
Filing Date
2020-06-02
Publication Date
2024-11-14
Estimated Expiration
2040-06-02

AI Technical Summary

Technical Problem

Cameras integrated with displays in electronic devices often require unsightly black borders, notches, or cutouts due to obstructed views, which can be addressed by incorporating infrared light sources and sensors to enable imaging through display surfaces.

Method used

Incorporating a camera assembly with an infrared light source and sensor configuration that emits and detects infrared light through the display surface, utilizing reflectors, light guides, and filters to minimize occlusion and enhance imaging capabilities.

Benefits of technology

This configuration allows for bezel-less displays by enabling effective depth sensing and imaging through display pixels, reducing visual obstructions and enhancing screen-to-body ratios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007583786000001
    Figure 0007583786000001
  • Figure 0007583786000002
    Figure 0007583786000002
  • Figure 0007583786000003
    Figure 0007583786000003
Patent Text Reader

Abstract

The device includes a first camera including one or more light sensors configured to detect visible light transmitted through a first surface in a first direction, a light source configured to emit infrared (IR) light through the first surface in a second direction opposite the first direction, and a second camera configured to detect reflection of the IR light.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present embodiments relate generally to light sources for cameras. [Background technology]

[0002] Some electronic devices, such as smartphones, tablets, laptops, and other computing systems, include a camera and a display. The camera is often located in an area of ​​the device that does not obstruct the view of the environment. Thus, when integrated into the same surface as the display, the camera may be located adjacent to the display, within a large and unattractive black border around the display, or within a notch or cutout. Summary of the Invention

[0003] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0004] One innovative aspect of the present disclosure may be implemented as a device that includes a first camera with one or more light sensors configured to detect visible light transmitted through a first surface in a first direction. The device may further include a light source configured to emit infrared (IR) light through the first surface in a second direction opposite the first direction. The device may further include a second camera configured to detect reflection of the IR light.

[0005] Another innovative aspect of the present disclosure may be implemented as a camera assembly that may include a lens and one or more light sensors configured to detect visible light transmitted through the lens in a first direction. The camera assembly may further include a light source configured to emit light through the lens in a second direction opposite the first direction.

[0006] Another innovative aspect of the present disclosure may be implemented as a camera assembly. The camera assembly may include one or more light sensors configured to detect visible or infrared (IR) light transmitted through a first surface of the camera assembly in a first direction. The camera assembly may further include a light source configured to emit light through the first surface in a second direction opposite the first direction. The camera assembly may further include a light guide configured to direct light to the first surface, where the first surface comprises at least a portion of the light guide. [Brief description of the drawings]

[0007] The present embodiments are illustrated by way of example and are not intended to be limited by the figures of the accompanying drawings.

[0008] [Figure 1] FIG. 1 illustrates an exemplary electronic system according to some embodiments.

[0009] [Diagram 2] FIG. 2 is a diagram illustrating an exemplary camera assembly according to some embodiments.

[0010] [Figure 3A] FIG. 3A illustrates an exemplary camera assembly according to some embodiments.

[0011] [Figure 3B] FIG. 3B illustrates an exemplary camera assembly according to some embodiments.

[0012] [Figure 3C] FIG. 3C illustrates an exemplary camera assembly according to some embodiments.

[0013] [Figure 3D]FIG. 3D illustrates an exemplary camera assembly according to some embodiments.

[0014] [Figure 3E] FIG. 3E illustrates an exemplary camera assembly according to some embodiments.

[0015] [Figure 3F] FIG. 3F illustrates an exemplary camera assembly according to some embodiments.

[0016] [Figure 3G] FIG. 3G illustrates an exemplary camera assembly according to some embodiments.

[0017] [Figure 4A] FIG. 4A illustrates an exemplary electronic system according to some embodiments. [Figure 4B] FIG. 4B illustrates an exemplary electronic system according to some embodiments. [Figure 4C] FIG. 4C illustrates an exemplary electronic system according to some embodiments. [Figure 4D] FIG. 4D illustrates an exemplary electronic system according to some embodiments. [Figure 4E] FIG. 4E illustrates an exemplary electronic system according to some embodiments. [Figure 4F] FIG. 4F illustrates an exemplary electronic system according to some embodiments. [Figure 4G] FIG. 4G illustrates an exemplary electronic system according to some embodiments. [Figure 4H] FIG. 4H illustrates an exemplary electronic system according to some embodiments. [Figure 4I] FIG. 4I illustrates an exemplary electronic system according to some embodiments. [Figure 4J]FIG. 4J illustrates an exemplary electronic system according to some embodiments. [Figure 4K] FIG. 4K illustrates an exemplary electronic system according to some embodiments.

[0018] [Diagram 5] FIG. 5 is a cross-sectional view of an exemplary electronic system according to some embodiments.

[0019] [Figure 6] FIG. 6 is a cross-sectional view of an exemplary electronic system according to some embodiments.

[0020] [Figure 7] FIG. 7 is a cross-sectional view of an exemplary electronic system according to some embodiments.

[0021] [Figure 8] FIG. 8 is a cross-sectional view of an exemplary electronic system according to some embodiments.

[0022] [Figure 9] FIG. 9 illustrates an exemplary image captured by a camera positioned below a display, according to some embodiments.

[0023] [Figure 10] FIG. 10 is a block diagram of an exemplary image processing system according to some embodiments.

[0024] [Figure 11A] FIG. 11A illustrates an example image before filtering.

[0025] [Figure 11B] FIG. 11B illustrates an example image after filtering. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] In the following description, numerous specific details are set forth, such as examples of specific components, circuits, and processes, to provide a thorough understanding of the present disclosure. The term "coupled" as used herein means directly connected or connected through one or more intervening components or circuits. Furthermore, in the following description, and for purposes of explanation, specific nomenclature is used to provide a thorough understanding of aspects of the present disclosure. However, it will be apparent to those skilled in the art that these specific details may not be required to practice the exemplary embodiments. In other instances, well-known circuits and devices are shown in block diagram form to avoid obscuring the disclosure. Some portions of the following detailed description are presented in the form of processes, logic blocks, processes, and other symbolic representations of operations on data bits within a computer memory. Interconnections between circuit elements or software blocks may be depicted as buses or single signal lines. Each of the buses may alternatively be a single signal line, and each of the single signal lines may alternatively be a bus, and a single signal line or bus may represent any one or more of a myriad of physical or logical mechanisms for communication between components.

[0027] As will become apparent from the discussion below, unless specifically stated otherwise, discussions made throughout this application using expressions such as "accessing," "receiving," "transmitting," "using," "selecting," "determining," "normalizing," "multiplying," "averaging," "monitoring," "comparing," "applying," "updating," "measuring," "deriving," and the like, will be deemed to refer to the actions and processing of a computer system (or similar electronic computing device) that manipulates and converts data, represented as physical (electronic) quantities in the computer system's registers and memory, into other data, which are similarly represented as physical quantities in the computer system's memory or registers, or other such information storage, transmitter, or display device.

[0028] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any features described as modules or components may be implemented together in an integrated logic device or as separate but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable non-transitory storage medium that includes instructions that, when executed, implement one or more of the methods described above. The computer-readable non-transitory storage medium may form part of a computer program product. In addition, the computer program product may include packaging materials.

[0029] The non-transitory processor-readable storage medium may comprise random access memory (RAM), such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, other known storage media, etc. The present technology may additionally or alternatively be implemented, at least in part, by a processor-readable communications medium that conveys or communicates code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer or other processor.

[0030] The various example logic blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors. As used herein, the term "processor" may refer to any general purpose processor, conventional processor, controller, microcontroller, and / or state machine capable of executing scripts or instructions of one or more software programs stored in memory.

[0031] 1 illustrates an exemplary electronic system 100 according to some embodiments. The electronic system 100 may include a display 106, a camera 102, and a camera 104.

[0032] The display 106 may include a porous display, such as an organic light-emitting diode (OLED) display or a micro light-emitting diode (micro-LED) display, that includes holes or gaps between display pixels and / or display sub-pixels.

[0033] Each of the cameras 102 and 104 may comprise an array of active pixel sensors or photosensors (e.g., photodiodes, complementary metal oxide-semiconductor (CMOS) image sensor arrays, charge coupled device (CCD) arrays), and / or any other sensors capable of detecting wavelengths of light in the visible spectrum, the IR spectrum (NIR or shortwave IR), or the ultraviolet spectrum). In some embodiments, each of the cameras 102 and 104 may be configured to detect NIR light. In other embodiments, one of the two cameras 102 and 104 may be configured to detect visible light (e.g., red, green, and blue (RGB) components of light) (an "RGB camera") and the other camera may be configured to detect NIR light. Additionally, in some embodiments, each of the cameras 102 and 104 may be configured to support depth sensing. For example, camera 102 and camera 104 may each be configured to capture images of an object or scene within the camera's field-of-view (FOV). The images may be combined and depth information of the object or scene may be extracted from the combined images. The resolution of the depth sensing may depend on the separation distance between camera 102 and camera 104. In other words, the greater the separation distance, the greater the amount of depth information that can be derived from the images captured by camera 102 and camera 104.

[0034] In some embodiments, one or both of the camera 102 and the camera 104 may be located behind the display 106. For example, when the camera 102 is located behind the display 106, the camera's FOV may be partially occluded by display pixels and / or display sub-pixels of the display 106. As a result, as light passes through the display 106, some of the light may be occluded by the pixels and / or sub-pixels while the remaining light may be detected by the camera 102. The camera 102 may capture this remaining light in an image, which may include noise, distortion, or interference due to the occlusion. In some embodiments, a neural network model may be used to filter out this noise or interference so that the image is suitable for depth sensing and / or other image processing. Among other advantages, placing the camera 102 behind the display 106 may eliminate the need for unsightly black borders, cutouts, or notches in the bezel of the display 106. Similarly, the screen-to-body ratio of the electronic system 100 may be increased, and the electronic system 100 may support a bezel-less display.

[0035] In some embodiments, the under-display camera may require a light source to capture an image of an object or scene. Aspects of the present disclosure provide a camera assembly (not shown in FIG. 1 ) configured to emit light for a camera such as an under-display camera or a camera built into the camera assembly. As described further below, the camera assembly may comprise a lens, a camera, and a light source. Additionally, the camera assembly may be disposed adjacent to the display 106. In some embodiments, the camera assembly may further comprise a reflector configured to reflect light emitted from the light source through the lens. In other embodiments, the camera assembly may further comprise a light guide configured to direct light emitted by the light source through a first surface of the camera assembly.

[0036] 2 illustrates an exemplary camera assembly 200 according to some embodiments. The camera assembly 200 may include a light source 210, a reflector 216, a lens 212, a camera 214, a filter 218, a proximity sensor 254, and a housing 208.

[0037] A light source 210 is disposed within the housing 208 to the side of the lens 212. The light source 210 may comprise an LED, a vertical-cavity surface-emitting laser (VCSEL), a laser diode, or other light source capable of emitting wavelengths of light in the IR spectrum, including, but not limited to, the NIR or short-wave infrared (SWIR).

[0038] The reflector 216 is configured to reflect IR light emitted by the light source 210 away from the camera 214 and transmit visible light from the lens 212 to the camera 214. In some embodiments, the reflector 216 may include a reflector, a dichroic reflector, and / or a dichroic mirror. In other embodiments, the reflector 216 may comprise a beam splitter.

[0039] The lens 212 is an optical lens or assembly of lenses configured to transmit visible and / or IR light. More specifically, the lens 212 may be configured to transmit visible light 222 in an inward direction (or a first direction) from outside the camera assembly 200 to the camera 214. In some aspects, the lens 212 may be configured to focus the visible light 222 onto the camera 214. The lens 212 may also be configured to transmit IR light 220 emitted by the light source 210 and reflected by the reflector 216. As shown in FIG. 2, the IR light 220 may propagate in a direction opposite to the direction of the visible light 222, or in an outward direction (or a second direction) away from the camera assembly 200.

[0040] The camera 214 may comprise an array of active pixel sensors or light sensors (e.g., photodiodes, CMOS image sensor arrays, CCD arrays, and / or any other sensors capable of detecting wavelengths of light in the visible, IR, or UV spectrum). In some embodiments, the camera 214 may be an RGB camera. Furthermore, the camera 214 may be configured to detect visible light 222 while the light source 210 emits IR light 220. Furthermore, the camera 214 may be configured to perform wide range imaging where an object or scene is imaged either near or far.

[0041] The filter 218 is disposed between the camera 214 and the lens 212. The filter 218 may include an IR cut filter. The filter 218 may be configured to reflect and / or block IR wavelengths of light (e.g., mid-infrared wavelengths, NIR, or SWIR) so that the IR light does not interfere with the camera 214.

[0042] The proximity sensor 254 is disposed within the housing 208 and proximate to the light source 210. The proximity sensor 254 may comprise an array of active pixel sensors or light sensors (e.g., photodiodes, a CMOS image sensor array, a CCD array, and / or any other sensor capable of detecting wavelengths of light in the visible, IR, or UV spectrum). Additionally, the proximity sensor 254 is configured to detect reflections of the IR light 220 emitted by the light source 210.

[0043] The housing 208 may comprise a structure that surrounds the lens 212 and encloses the light source 210, the reflector 216, the filter 218, and the camera 214. As shown in FIG. 2, a first surface 258 is formed by a top of the housing 208, a top of the lens 212, and a portion of a side of the lens 212. Further, a length L1 is the length of the housing 208 measured from the top of the housing 208 at the first surface 258 to the base of the housing 208 near the camera 214. In some embodiments, it may be advantageous to minimize the length L1 of the housing 208.

[0044] In some embodiments, during operation, the light source 210 may emit IR (e.g., NIR or SWIR) light 220. The IR light 220 may reflect off the reflector 216 and transmit through the lens 212 in an outward direction (or second direction) away from the camera assembly 200. The IR light 220 may illuminate a scene in front of the camera assembly 200 and the camera 214. Objects in the scene may reflect at least a portion of the IR light 220 back to the camera assembly 200 and a display disposed proximate to the camera assembly 200. Some of the reflected IR light 220 is blocked by pixels and / or subpixels in the display, while the remainder of the reflected IR light 220 passes through holes between the display pixels and / or subpixels. The reflected IR light 220 that passes through the holes may be detected by a camera ("IR camera") configured to detect the reflected IR light 220. Each IR camera may capture an image of the detected IR light 220. The captured images may be suitable for depth sensing.

[0045] In other embodiments, during operation, the light source 210 may emit IR (e.g., NIR or SWIR) light 220. The IR light 220 may reflect off the reflector 216 and transmit through the lens 212 in an outward direction away from the camera assembly 200. The IR light 220 may illuminate a scene in front of the camera assembly 200 and the camera 214. Objects within the scene may reflect at least a portion of the IR light 220 back toward the camera assembly 200. The reflected IR light 220 may transmit through the lens 212, reflect off the reflector 216, and be detected by the proximity sensor 254. In this manner, the camera assembly 200 may be capable of emitting and sensing IR light, for example, to detect the presence of a user.

[0046] 3A illustrates an exemplary camera assembly 300A according to some embodiments. The camera assembly 300A may include a light source 310, a light guide 324, a housing 308, a lens 312, a filter 318, and a camera 314.

[0047] The light source 310 may be coupled to the light guide 324 and configured to emit IR light 320. The light source 310 may include an LED, VCSEL, laser diode, or other light source capable of emitting wavelengths of light in the visible spectrum and / or the IR spectrum, including, but not limited to, the NIR or SWIR.

[0048] The light guide 324 may be configured to direct the IR light 320 emitted by the light source 310 in an outward direction (or a second direction) away from the camera assembly 300A. In some embodiments, the light guide 324 may include a transparent material, such as a clear polycarbonate. Additionally, the light guide 324 may be configured to reflect light. For example, in some embodiments, the light guide 324 may be configured to reflect light where an inner surface of the light guide 324 (which may include an outer surface of the housing 308) has a refractive index greater than an inner surface of the housing 308. As another example, the light guide 324 may be configured to reflect light where an inner surface of the light guide 324 (which may include an outer surface of the housing 308) includes a reflective surface and an inner surface of the housing 308 includes a darkened surface or a surface capable of absorbing visible or IR (e.g., NIR and / or SWIR) light. In some embodiments, the light guide 324 may include optical fibers, while in other embodiments, the light guide 324 may include a mirror and / or reflector. Additionally, the light guide 324 may be coupled to the housing 308 (eg, via a molding).

[0049] The lens 312 is an optical lens or assembly of lenses configured to transmit visible light 322 in an inward direction (or first direction) from outside the camera assembly 300A to the camera 314. In some aspects, the lens 312 may be configured to focus the visible light 322 onto the camera 314. In some embodiments, the lens 312 may comprise acrylic.

[0050] The camera 314 may comprise an array of active pixel sensors or light sensors (e.g., photodiodes, a CMOS image sensor array, a CCD array, and / or any other sensor capable of detecting light at wavelengths in the visible spectrum, the IR spectrum (e.g., NIR or SWIR), or the ultraviolet spectrum). In some embodiments, the camera 314 may include an RGB camera. Additionally, the camera 314 may be configured to detect visible light 322 while the light source 310 emits IR light 320. Additionally, the camera 314 may be configured to perform wide range imaging where an object or scene is imaged either close or far.

[0051] Filter 318 is disposed between camera 314 and lens 312. Filter 318 may include an IR cut filter. Filter 318 may be configured to reflect and / or block IR wavelengths of light (e.g., mid-infrared wavelengths, NIR, or SWIR light) so that the IR light does not interfere with camera 314.

[0052] The housing 308 may surround the lens 312 and may enclose the filter 318 and the camera 314. As described above, the interior surface of the housing 308 (e.g., the wall facing the camera 314) may include a darkened surface or a surface capable of absorbing visible or IR (e.g., NIR and / or SWIR) light. As shown in FIG. 3A, a first surface 358 is formed by the top of the housing 308, the top of the lens 312, and the side of the light guide 324. Furthermore, the length L1 is the length of the housing 308 measured from the top of the housing 308 at the first surface 358 to the base of the housing 308 near the camera 314. In some embodiments, it may be advantageous to minimize the length L1 of the housing 308.

[0053] In operation, the light source 310 may emit IR (e.g., NIR or SWIR) light 320 that is directed through the light guide 324 and the first surface 358. The IR light 320 may illuminate a scene in front of the camera assembly 300A and the camera 314. Objects in the scene may reflect at least a portion of the IR light 320 back to the camera assembly 300A and a display disposed proximate to the camera assembly 300A. Some of the reflected IR light 320 may pass through holes between the display pixels and / or subpixels and be detected by an IR camera disposed behind the display. Each IR camera may capture the detected IR light into an image. The captured image may be suitable for depth sensing.

[0054] 3A, in some embodiments, the light source 310 may be configured to emit IR (e.g., NIR or SWIR) light 320, and the camera 314 may be configured to detect the IR light 320 and / or visible light 322. Further, in some embodiments, the light source 310 may be configured to emit IR light 320 while the camera 314 detects the IR light 320 and / or visible light 322. For example, during imaging, the light source 310 may emit IR light 320 that is guided through the light guide 324 and the first surface 358 to the front of the camera assembly 300A. A portion of the IR light 320 may reflect off an object or feature in front of the camera assembly 300A and pass through the lens 312 to the camera 314. In this manner, the camera assembly 300A may be capable of emitting and sensing IR light for imaging.

[0055] In other embodiments, the light source 310 may be configured to emit visible light. During imaging, the light source 310 may emit visible light that is guided through the light guide 324 and the first surface 358 to the front of the camera assembly 300A. A portion of the visible light may reflect off an object or feature in front of the camera assembly 300A and pass through the lens 312 to the camera 314. In this manner, the camera assembly 300A may be capable of emitting and sensing visible light for imaging.

[0056] FIG. 3B illustrates an exemplary camera assembly 300B, according to some embodiments. The camera assembly 300B may be an embodiment of the camera assembly 300A described above with reference to FIG. 3A. As shown in FIG. 3B, the camera assembly 300B includes two light sources 310, although in some embodiments, the camera assembly 300B may include more than two light sources. Each light source 310 may be coupled to a light guide 324. Additionally, each light source 310 may include an LED, VCSEL, laser diode, or other light source capable of emitting wavelengths of light in the IR spectrum, including but not limited to NIR or SWIR. In operation, each light source 310 may emit IR light 320 that propagates through the light guide 324 toward the front surface of the camera assembly 300B. As described above with reference to FIG. 2 and FIG. 3A, the IR (e.g., NIR or SWIR) light 320 may reflect off an object or feature in front of the camera assembly 300B and be detected by an under-display IR camera for depth sensing and image processing.

[0057] FIG 3C illustrates an exemplary camera assembly 300C according to some embodiments. The camera assembly 300C may be an embodiment of the camera assembly 300A and / or the camera assembly 300B described above with reference to FIG 3A and / or FIG 3B. As shown in FIG 3C, the camera assembly 300C includes a camera 314 capable of detecting wavelengths of light in the visible or IR (e.g., NIR or SWIR) spectrum. Additionally, the camera assembly 300C includes a light source 310 capable of emitting wavelengths of light in the visible or IR (e.g., NIR or SWIR) spectrum.

[0058] As shown in FIG. 3C, the camera assembly 300C also includes a light guide 324. The light guide 324 includes a diffusing surface 326. The diffusing surface 326 may be configured to diffuse, spread, or scatter the light 320 (e.g., visible light, NIR light, or SWIR light) emitted by the light source 310. The diffusing surface 326 may include a portion of the first surface 358 that forms a ring around the housing 308. Furthermore, the diffusing surface 326 may include a portion of the light guide 324. For example, the diffusing surface 326 may be integrated with the light guide 324 via molding. In some embodiments, the diffusing surface 326 may include a diffraction grating having a certain roughness. More specifically, the diffraction grating may include a number of fine grooves, slits, or ridges. In some embodiments, the grooves, slits, and / or ridges may be randomly arranged. Alternatively, they may be arranged in a parallel or uniform manner to form a pattern. In this manner, the diffusing surface 326 may be configured to provide a uniform illumination profile or tailored illumination pattern of the light 320 .

[0059] FIG. 3D illustrates an exemplary camera assembly 300D according to some embodiments. The camera assembly 300D may be an embodiment of the camera assembly 300A, the camera assembly 300B, and / or the camera assembly 300C described above with reference to FIG. 3A, FIG. 3B, and / or FIG. 3C. As shown in FIG. 3D, the light guide 324 includes a lens and / or a microlens 328 ("microlens 328"). In some embodiments, the microlenses 328 may be integrated (e.g., molded) into a portion of the first surface 358 that forms a ring around the housing 308. The microlenses 328 may be configured to diffuse and / or diffract the IR light 320 emitted by the light source 310.

[0060] FIG. 3E illustrates an exemplary camera assembly 300E according to some embodiments. The camera assembly 300E may be an embodiment of the camera assembly 300A, the camera assembly 300B, the camera assembly 300C, and / or the camera assembly 300D described above with reference to FIGS. 3A-3D. As shown in FIG. 3E, the light guide 324 includes two microlenses 330 disposed on either side of the housing 308. In some embodiments, more than two microlenses 330 may be disposed along a surface of the light guide 324 on each side of the housing 308 forming a ring around the housing 308. The microlenses 330 may be configured to diffuse and / or diffract the light emitted by the light source 310.

[0061] FIG. 3F illustrates an exemplary camera assembly 300F according to some embodiments. The camera assembly 300F may be an embodiment of the camera assembly 300A, the camera assembly 300B, the camera assembly 300C, the camera assembly 300D, and / or the camera assembly 300E described above with reference to FIGS. 3A-3E. In some embodiments, the camera assembly 300F may include an ambient light sensor 352. The ambient light sensor 352 may be configured to detect an amount of ambient light in front of the camera assembly 300F. During sensing, the ambient light 350 in front of the camera assembly 300F may be guided through the light guide 324 to the ambient light sensor 352. Thus, the camera assembly 300F may be able to measure the ambient light, for example, to adjust the brightness of a display screen coupled to the camera assembly 300F.

[0062] 3G illustrates an exemplary camera assembly 300G, which may be an embodiment of camera assembly 300A, camera assembly 300B, camera assembly 300C, camera assembly 300D, camera assembly 300E, and / or camera assembly 300F described above with reference to FIGS.

[0063] 3G, the camera assembly 300G includes a light guide 324 that may be configured to reflect light. In some embodiments, the light guide 324 may be configured to reflect light where an inner surface of the light guide 324 has a refractive index greater than the refractive index of the inner surface of the housing 308. Additionally, in some embodiments, the inner surface of the light guide 324 may be configured to reflect light where the inner surface of the housing 308 includes a darkened surface or a surface capable of absorbing visible or IR light (e.g., NIR and / or SWIR light).

[0064] In some embodiments, the camera assembly 300G may include a proximity sensor 354. The proximity sensor 354 may be configured to detect IR (e.g., NIR or SWIR) light 320 emitted by the light source 310. During sensing, the light source 310 may emit IR light 320 that may be guided to a front surface of the camera assembly 300G through a light guide 324 (for simplicity, the emitted IR light 320 is not shown). A portion of the IR light 320 may reflect off an object or feature in front of the camera assembly 300G and propagate back through the light guide 324 to the proximity sensor 354. Thus, the camera assembly 300G may be capable of emitting and sensing IR light, for example, to detect the presence of a user.

[0065] 4A-4K illustrate exemplary electronic systems 400A-400K according to some embodiments. Each of the electronic systems 400A-400K includes a camera assembly 432, a camera 402 and / or a camera 404, and a display 406. The camera assembly 432 may be an embodiment of the camera assembly 200 and / or the camera assemblies 300A-300G of FIGS. 2 and 3A-3G. The camera 402 and the camera 404 may be an embodiment of the camera 102 and / or the camera 104 of FIG. 1. Additionally, each of the cameras 402 and 404 may be disposed below the display 406 and configured to capture images for depth sensing.

[0066] 4A, camera assembly 432 is positioned near a corner of electronic system 400A, and camera 402 and camera 404 are positioned along the longitudinal axis near an end of electronic system 400A.

[0067] In operation, IR (e.g., NIR or SWIR) light may be emitted by a light source in camera assembly 432 and illuminate a scene in front of camera assembly 432. Objects in the scene may reflect at least a portion of the IR light back toward camera assembly 432 and display 406. Some of the reflected IR light may pass through holes between pixels and / or subpixels of display 406 and be detected by IR camera 402 and IR camera 404 disposed behind display 406. Each of camera 402 and camera 404 may capture an image of the detected IR light, which may be suitable for depth sensing.

[0068] In FIG. 4B, the camera assembly 432 is located near a corner of the electronic system 400B. And the cameras 402 and 404 are located along the long axis near a central region of the electronic system 400B. As discussed above with reference to FIG. 1, the distance between the two cameras affects the resolution of the depth sensing. In other words, the greater the distance between the cameras, the greater the amount of depth information that can be obtained from the images captured by the cameras. Compared to FIG. 4A, the cameras 402 and 404 in FIG. 4B are separated by a shorter distance. As a result, the cameras 402 and 404 in FIG. 4A may support higher resolution depth sensing than the cameras 402 and 404 in FIG. 4B.

[0069] Figure 4C shows camera assembly 432 positioned near a corner of electronic system 400C and camera 402 and camera 404 positioned near a central region of electronic system 400C along the minor axis. Figure 4D shows camera assembly 432 positioned near a corner of electronic system 400D along the minor axis and camera 402 and camera 404 positioned on top of electronic system 400D.

[0070] FIG. 4E shows a camera assembly 432 disposed near one end of the electronic system 400E. The cameras 402 and 404 are aligned along the long axis with the camera assembly 432. In FIG. 4F, the camera assembly 432 is disposed near one end of the electronic system 400F. The cameras 402 and 404 are aligned along the long axis with the camera assembly 432. Unlike FIG. 4E, the cameras 402 and 404 in FIG. 4F are separated by a shorter distance. As a result, the cameras 402 and 404 in FIG. 4E may support higher resolution depth sensing than the cameras 402 and 404 in FIG. 4F.

[0071] 4G shows camera assembly 432 positioned near one end of electronic system 400G. Camera 402 and camera 404 are positioned near a central region of electronic system 400G along the minor axis.

[0072] 4H shows a camera assembly 432 disposed near one end of electronic system 400H and a camera 402 disposed near the other end of electronic system 400H. In some embodiments, both camera assembly 432 and camera 402 may be configured to capture images for depth sensing.

[0073] 4I shows a camera assembly 432 disposed near one end of the electronic system 400I and a camera 402 disposed near a central region of the electronic system 400I. In some embodiments, both the camera assembly 432 and the camera 402 may be configured to capture images for depth sensing.

[0074] 4J illustrates a camera assembly 432 disposed near one corner of electronic system 400J and a camera 402 disposed near an end of electronic system 400J. In some embodiments, both camera assembly 432 and camera 402 may be configured to capture images for depth sensing.

[0075] 4K illustrates a camera assembly 432 disposed near one corner of electronic system 400K and a camera 402 disposed near a central region of electronic system 400K. In some embodiments, both camera assembly 432 and camera 402 may be configured to capture images for depth sensing.

[0076] FIG. 5 illustrates a cross-section of an exemplary electronic system 500, according to some embodiments. Electronic system 500 may be an embodiment of electronic system 100 and / or electronic system 400A-400G of FIGS. 1 and 4A-4G. As shown in FIG. 5, electronic system 500 includes camera assembly 532, display layer 506, and module 540. Camera assembly 532 may be an embodiment of camera assembly 200 and / or camera assembly 432 of FIGS. 2 and 4A-4G. Module 540 includes camera 502 and camera 504. Camera 502 and camera 504 may be an embodiment of camera 102, camera 104, camera 402, and / or camera 404 of FIGS. 1 and 4A-4G, respectively.

[0077] The camera assembly 532 includes a light source 510, a reflector 516, a lens 512, a camera 514, a filter 518, and a housing 508. Additionally, a length L1 is the length of the housing 508 measured from the top of the housing 508 at the first surface 558 to the base of the housing 508 near the camera 514. In some embodiments, it may be advantageous to minimize the length L1 of the housing 508. Although the length L1 appears to be shorter than the length L2 (spanning the length of the display layer 506 and the module 540), L1 may be approximately any length less than, equal to, or greater than the length L2. Additionally, although the camera assembly 532 is shown adjacent to the display layer 506 and the module 540, in an actual implementation, components (not shown) may separate the camera assembly 532 from the display layer 506 and the module 540.

[0078] The display layer 506 may include multiple layers of a porous display, such as an OLED display or a micro LED display. As shown in Figure 5, the display layer 506 may include a display surface 536 and an emissive layer 538. In some embodiments, the emissive layer 538 may include a plurality of display pixels and / or sub-pixels with holes, gaps, or empty spaces between each display pixel and / or sub-pixel.

[0079] The module 540 is disposed below the display layer 506 and may include cameras 502 and 504 and corresponding lenses 542. The cameras 502 and 504 may include IR CMOS sensors configured to perform wide-range imaging, where objects or scenes are imaged either near or far. The cameras 502 and 504 may be further configured to rapidly sense objects such as fingers, hands, and heads. Furthermore, the cameras 502 and 504 may each be configured to capture images for depth sensing. The resolution of the depth sensing may depend on the separation D, which is the distance between the cameras 502 and 504. In other words, the greater the distance D, the greater the amount of depth information that can be derived from the images captured by the cameras 502 and 504.

[0080] In operation, light source 510 may emit IR (e.g., NIR or SWIR) light 520. The IR light 520 may reflect off reflector 516 and transmit through lens 512. The IR light 520 may illuminate a scene in front of camera assembly 532 and cameras 502 and 504. Objects in the scene may reflect at least a portion of the IR light 520 back toward display layer 506, e.g., as reflected light 520(R)(1) and reflected light 520(R)(2). A portion of reflected light 520(R)(1) and reflected light 520(R)(2) may be blocked by display pixels and / or subpixels in emissive layer 538, while the remainder of reflected light 520(R)(1) and reflected light 520(R)(2) may pass through holes between display pixels and / or subpixels. Reflected light 520(R)(1) and reflected light 520(R)(2) that pass through hole and lens 542 may be detected by camera 502 and camera 504, respectively. Each of camera 502 and camera 504 may capture separate images of the reflected IR light. These captured images may be combined for depth sensing.

[0081] FIG. 6 illustrates a cross-section of an exemplary electronic system 600, according to some embodiments. Electronic system 600 may be an embodiment of any of electronic systems 400H-400K of FIGS. 4H-4K. As shown in FIG. 6, electronic system 600 includes camera assembly 632, display layer 606, and module 640. Camera assembly 632 may be an embodiment of camera assembly 200, camera assembly 432, and / or camera assembly 532 of FIGS. 2, 4A-4K, and 5. Display layer 606 may be an embodiment of display layer 506 of FIG. 5. As shown in FIG. 6, module 640 includes camera 604. Camera 604 may be an embodiment of camera 102, camera 104, camera 402, camera 404, camera 502, and / or camera 504 of FIGS. 1, 4A-4K, and 5.

[0082] During operation, the light source 610 may emit IR (e.g., NIR or SWIR) light 620. The IR light 620 may reflect off the reflector 616 and be transmitted through the lens 612. The IR light 620 may illuminate a scene in front of the camera assembly 632 and the camera 604. Objects in the scene may reflect at least a portion of the IR light 620 back toward the display layer 606, e.g., as reflected light 620(R). A portion of the reflected light 620(R) may be transmitted through the display layer 606 and the lens 642 and detected by the camera 604 and captured as an image. In some embodiments, substantially simultaneously as the light source 610 emits the IR light 620, the camera 614 may detect visible light 622 (e.g., from the sun or other light source) reflecting off objects in the scene and capture the visible light 622 into an image. Images captured by camera 614 and camera 604 may be combined for depth sensing.

[0083] FIG. 7 illustrates a cross-section of an exemplary electronic system 700, according to some embodiments. Electronic system 700 may be an embodiment of electronic system 100 and / or electronic systems 400A-400G of FIGS. 1 and 4A-4G. As shown in FIG. 7, electronic system 700 includes a camera assembly 732, a display layer 706, and a module 740. Camera assembly 732 may be an embodiment of camera assembly 300A and / or any of camera assemblies 300B-300G in FIGS. 3A-3G. Display layer 706 may be an embodiment of display layer 506 and / or display layer 606 of FIGS. 5 and 6. Furthermore, module 740 may be an embodiment of module 540 of FIG. 5.

[0084] The camera assembly 732 comprises a light source 710, a light guide 724, a lens 712, a camera 714, a filter 718, and a housing 708. In some embodiments, the camera assembly 732 may further comprise additional light sources, scattering surfaces, microlenses, ambient light sensors, and / or proximity sensors, as shown in Figures 3B-3G.

[0085] Length L1 is the length of the housing 708 measured from the top of the housing 708 at the first surface 758 to the base of the housing 708 near the camera 714. In some embodiments, it may be advantageous to minimize length L1 of the housing 708. Furthermore, although length L1 appears to be shorter than length L2 (spanning the length of the display layer 706 and module 740), L1 may be nearly any length less than, equal to, or greater than length L2. Additionally, while camera assembly 732 is shown adjacent to display layer 706 and module 740, in an actual implementation, components (not shown) may separate camera assembly 732 from display layer 706 and module 740.

[0086] In operation, the light source 710 may emit IR (e.g., NIR or SWIR) light 720. The IR light 720 may be directed through the light guide 724 and the first surface 758. The IR light 720 may illuminate a scene in front of the camera 702, the camera 704, and the camera 714. Objects in the scene may reflect at least a portion of the IR light 720 back toward the display layer 706, e.g., as reflected IR light 720(R)(1) and IR light 720(R)(2). Portions of the reflected IR light 720(R)(1) and IR light 720(R)(2) may pass through the display layer 706 and the lens 742 and be detected by the camera 702 and the camera 704, respectively. The camera 702 and the camera 704 may capture the IR light as separate images, which may be combined for depth sensing.

[0087] FIG. 8 illustrates a cross-section of an exemplary electronic system 800, according to some embodiments. Electronic system 800 may be an embodiment of any of electronic systems 400H-400K of FIGS. 4H-4K. As shown in FIG. 8, electronic system 800 includes camera assembly 832, display layer 806, and module 840. Camera assembly 832 may be an embodiment of camera assembly 300A and / or camera assembly 300B-300G of FIGS. 3A-3G. Display layer 806 may be an embodiment of display layer 506, display layer 606, and / or display layer 706 of FIGS. 5-7. Additionally, module 840 includes camera 804. Camera 804 may be an embodiment of camera 102, camera 104, camera 402, camera 404, camera 502, camera 504, camera 604, camera 702, and / or camera 704 of Figures 1, 4A-4K, and 5-7. In some embodiments, camera 804 may be configured to detect IR (e.g., NIR or SWIR) light, and filter 818 may shield camera 804 from visible light.

[0088] During operation, the light source 810 may emit IR (e.g., NIR or SWIR) light 820 that is directed through the light guide 824 and the first surface 858. The IR light 820 may illuminate a scene in front of the cameras 804 and 814. Objects in the scene may reflect at least a portion of the IR light 820 back toward the camera assembly 800 and the display layer 806, e.g., as reflected IR light 820(R)(1) and IR light 820(R)(2). A portion of the reflected light 820(R)(1) may pass through the display layer 806 and the lens 842 and be detected and captured as an image by the camera 804. Additionally, a portion of the reflected light 820(R)(2) may pass through the lens 812 and the filter 818 toward the camera 814, which may capture the reflected light 820(R)(2) as an image. The images captured by camera 804 and camera 814 may be combined for depth sensing.

[0089] In some embodiments, the camera 814 may be configured to detect visible light, and the filter 818 may shield the camera 814 from IR light. In operation, the light source 810 may emit IR (e.g., NIR or SWIR) light 820 that is directed through the light guide 824 and the first surface 858. The IR light 820 may illuminate the camera 804 and a scene in front of the camera 814. Objects in the scene may reflect at least a portion of the IR light 820(R)(1) back toward the display layer 806, e.g., as reflected light 820(R)(1). A portion of the reflected light 820(R)(1) may pass through the display layer 806 and the lens 842 and be detected by the camera 804 and captured as an image. In some embodiments, at substantially the same time that light source 810 emits IR light 820, camera 814 may detect visible light 822 (e.g., from the sun or other light source) reflecting off objects in the scene and capture the visible light 822 into an image. The images captured by camera 814 and camera 804 may be combined for depth sensing.

[0090] FIG. 9 illustrates an exemplary image 900 captured by a camera positioned below a display, according to some embodiments. Image 900 may be captured by a camera positioned behind or below a porous display, such as, for example, camera 102, camera 104, camera 402, camera 404, camera 502, camera 504, camera 604, camera 702, camera 704, or camera 804 of FIGS. 1, 4A-4K, 5-8. As shown in FIG. 9, image 900 includes a grid of dark lines and light spots, or a "screen door effect." The dark lines correspond to a pattern of display pixels and / or display subpixels. The dark lines are created by the display pixels and / or display subpixels blocking some of the light from passing through the display. Meanwhile, the light spots correspond to gaps or holes between the display pixels and / or display subpixels. The light spots are created by light passing through holes or gaps in the display toward the camera below.

[0091] In some embodiments, the image captured by the under-display camera may include a grid, or screen door effect, that appears as if the under-display camera is simultaneously imaging through multiple layers of screen door (or stacks of screen doors), where each layer of screen door has a different pitch (e.g., hole size or density). In such embodiments, light may pass through holes or gaps between display pixels and / or display sub-pixels in the screen door layers. Furthermore, each point of light is spread to different positions horizontally and vertically (e.g., as a point spread function). Also, each point of IR light is spread in a similar manner across the image, which may result in significant distortion.

[0092] 9, portions of the image 900 are obscured by display pixels and / or display sub-pixels of the display. In other words, the display pixels and / or display sub-pixels may reduce the amount of light available for sensing, effectively blocking the FOV of the under-display camera. As such, in some embodiments, the captured image may be filtered or calibrated to remove screen door effects, distortions, or other artifacts.

[0093] FIG. 10 is a block diagram of an image processing system 1000 according to some embodiments. The image processing system 1000 includes a device interface 1010, a processor 1020, and a memory 1030. For purposes of discussion herein, the processor 1020 of FIG. 10 is shown as being coupled to the device interface 1010 and the memory 1030. In an actual embodiment, the device interface 1010, the processor 1020, and / or the memory 1030 may be coupled together using one or more buses (not shown for simplicity). In some embodiments, the image processing system 1000 may be an application specific integrated circuit (ASIC) or other integrated circuit (IC) of the electronic system 100, electronic systems 400A-400K, and / or electronic systems 500-800 described with reference to FIGS. 1, 4A-4K, and 5-8.

[0094] The device interface 1010 may send signals to and receive signals from input devices, including, but not limited to, one or more cameras, camera assemblies, and / or displays. In some embodiments, the device interface 1010 may include a display interface 1012. The display interface 1012 may be used to communicate with a display and / or provide a visual interface to a user of the image processing system 1000. In some embodiments, the device interface 1010 may further include an RGB camera interface 1014, an IR camera interface 1016, and an IR camera interface 1018. The RGB camera interface 1014, the IR camera interface 1016, and the IR camera interface 1018 may each be used to communicate with a different camera. For example, the RGB camera interface 1014 may send an activation signal to and receive sensor information from the camera 514 or the camera 714 to capture an image of an object or scene. Additionally, IR camera interface 1016 and IR camera interface 1018 may send activation signals to and receive sensor information from, for example, camera 502, camera 504, camera 702, or camera 704 to capture IR images of objects and / or scenes.

[0095] Memory 1030 may include an image buffer 1032 for storing images received via camera interface 1014, camera interface 1016, and / or camera interface 1018. Memory 1030 may also include a non-transitory computer readable medium (e.g., one or more non-volatile memory elements such as EPROM, EEPROM, flash memory, hard drive, etc.) that may store at least the following SW modules: An image filtering SW module 1034 that filters images received via the IR camera interface 1016 and / or the IR camera interface 1018 (or other interface in communication with the under-display camera), the image filtering SW module 1034 further comprising: A neural network model 1035 that filters, reduces, or removes noise from images received via the IR camera interface 1016 and / or the IR camera interface 1018; and ●A depth map SW module (or depth map generator) 1036, which combines images received via (i) both IR camera interface 1016 and IR camera interface 1018, or (ii) RGB camera interface 1014 and either IR camera interface 1016 or IR camera interface 1018, and extracts depth information from the images, for example using stereoscopic vision.

[0096] Each software module contains instructions that, when executed by processor 1020, cause image processing system 1000 to perform a corresponding function.

[0097] For example, the processor 1020 may execute the image filtering SW module 1034 to filter images received via the IR camera interface 1016 and / or the IR camera interface 1018. In executing the image filtering SW module 1034, the processor 1020 may use the neural network model 1035 to filter, reduce, or remove noise (such as a screen door effect) from images received via the IR camera interface 1016 and / or the IR camera interface 1018. The processor 1020 may further execute the depth map SW module 1036 to determine depth information of objects or features in the filtered images. For example, the depth map generator may combine the filtered images using stereoscopic vision and extract depth information from the combined filtered images. The depth map generator may further generate a depth map of the object or feature based at least in part on the extracted depth information. The processor 1020 may further recognize the object, object, or feature in the image based at least in part on the depth information.

[0098] As discussed above with reference to Figures 9 and 10, images captured by an under-display camera may include a screen door effect or other noise, distortion, or interference. Aspects of the present disclosure recognize that a screen door effect occurs when light (e.g., IR, NIR, or SWIR reflected light) passes through a display and some of the light is blocked by pixels and / or sub-pixels of the display, while the remaining light passes through holes between the pixels and / or sub-pixels and is captured in an image by the under-display camera. To facilitate depth sensing or other applications, an image filter may use a neural network model to filter out the screen door effect from the image.

[0099] FIG. 11A shows an example image 1110 captured by an under-display camera before filtering. Referring to FIG. 10, the image 1110 may be sent from the under-display IR camera to, for example, the IR camera interface 1016. The IR camera interface 1016 may send the example image 1110 to the image buffer 1032 for storage. As shown in FIG. 11A, the image 1110 is obscured by a screen door effect. To support depth sensing or other applications, the image filtering SW module 1034 may filter the example image 1110 to remove distortion.

[0100] FIG 11B shows an example image 1120 after filtering. The example image 1120 may be a filtered version of the example image 1110 of FIG 11A. Referring to FIG 10, the image buffer 1032 may send the example image 1110 to the image filtering SW module 1034. The image filtering SW module 1034 may use a neural network model 1035 to filter out the screen door effect from the example image 1110. As shown in FIG 11B, the filtered image 1120 more clearly depicts a person and may be suitable for depth sensing or other applications.

[0101] In some embodiments, the neural network model 1035 may include a CNN trained to filter out screen door effects or other noise from captured images. In other words, the CNN may be trained to infer an image of an original object or scene, or an image of a clear object or scene, from an image of an obscured object or scene. In some embodiments, the CNN may be trained by receiving and processing at least one image of the original object or scene and at least one image of the object or scene that is obscured. In some embodiments, instead of or in addition to the CNN, an inverse point spread function transform may be used to filter the image. Thus, the image filtering SW module 1034 may use the neural network model 1035 and / or the inverse point spread function transform to filter the image from the under-display IR camera.

[0102] Some captured images (e.g., obscured or filtered images) and image content (e.g., depth information, depth maps, or identified people, objects, or scenes) may be stored in a secure repository (e.g., trusted environment) on the electronic system 100, electronic systems 400A-400K, and / or electronic systems 500-800 of FIGS. 1, 4A-4K, and 5-8. The secure repository may be virtually and / or physically separated from other parts of the electronic system such that only applications and / or hardware within the trusted environment may access the data stored in the secure repository. In some aspects, the secure repository may be formed, at least in part, within the memory 1030. Thus, certain captured images and image content may be stored in the secure repository of the memory 1030 (e.g., within the image buffer 1032).

[0103] In some embodiments, the neural network model 1035 may reside, at least in part, within a trusted environment within the electronic system 100, electronic systems 400A-400K, and / or electronic systems 500-800 of Figures 1, 4A-4K, and 5-8. Placing the neural network model 1035 within a trusted environment enables the neural network model 1035 to perform machine learning (e.g., using CNN) with protected captured images and image content.

[0104] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0105] Moreover, those skilled in the art will appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in connection with the aspects disclosed herein may be implemented as electrical hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints required for the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0106] The methods, procedures, or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in software modules executed by a processor, or in a combination of both. The software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor.

[0107] In the foregoing specification, embodiments have been described with reference to specific examples thereof. It will be apparent, however, that various modifications and changes may be made thereto without departing from the broader scope of the disclosure, as set forth in the appended claims. The specification and drawings are therefore to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A display and a lens separate from the display; a first camera comprising one or more light sensors configured to detect visible light transmitted through the lens in a first direction; a light source configured to emit infrared (IR) light that is not transmitted through the display and that passes through the lens in a second direction opposite the first direction; a second camera disposed behind the display and configured to detect reflections of the IR light passing through the display in the first direction; Equipped with device.

2. the lens is configured to focus the visible light onto the one or more photosensors. The device of claim 1.

3. A display; a first camera comprising one or more light sensors configured to detect visible light transmitted in a first direction through a first surface separated from the display; a light source configured to emit infrared (IR) light that is transmitted through the first surface in a second direction opposite the first direction such that the IR light is not transmitted through the display; a second camera disposed behind the display and configured to detect reflections of the IR light passing through the display in the first direction; a dichroic reflector configured to reflect the IR light away from the one or more photosensors while transmitting the visible light to the one or more photosensors; Equipped with device.

4. A display; a first camera comprising one or more light sensors configured to detect visible light transmitted in a first direction through a first surface separated from the display; a light source configured to emit infrared (IR) light that is transmitted through the first surface in a second direction opposite the first direction such that the IR light is not transmitted through the display; a second camera disposed behind the display and configured to detect reflections of the IR light passing through the display in the first direction; a light guide configured to direct the IR light to the first surface; Equipped with the first surface comprises at least a portion of the light guide; device.

5. the portion of the light guide comprises a diffusing surface or a plurality of microlenses configured to diffuse the IR light emitted from the first surface. The device of claim 4.

6. Further comprising an ambient light sensor, the light guide is further configured to direct ambient light transmitted through the first surface to the ambient light sensor. A device according to claim 4 or 5.

7. Further comprising a proximity sensor, the light guide is further configured to direct reflection of the IR light transmitted through the first surface to the proximity sensor. A device according to any one of claims 4 to 6.

8. A display and a lens separate from the display; one or more light sensors configured to detect visible light transmitted through the lens in a first direction; a light source configured to emit infrared (IR) light through the lens in a second direction opposite the first direction such that the IR light is not transmitted through the display in the second direction; a reflector configured to transmit the visible light to the one or more photosensors while reflecting the IR light away from the one or more photosensors; Equipped with device.

9. A display and one or more light sensors configured to detect visible or infrared (IR) light transmitted in a first direction through a first surface separated from the display; a light source configured to emit light through the first surface in a second direction opposite the first direction such that the light is not transmitted through the display in the second direction; a light guide configured to direct the emitted light towards the first surface; Equipped with the first surface comprises at least a portion of the light guide; device.

10. the light emitted by the light source comprises the visible light; the portion of the light guide is configured to diffuse the visible light emitted from the first surface. The device of claim 9.

Citation Information

Patent Citations

  • Monitor camera device serving as optical unit

    JP2004094050A

  • Imaging apparatus

    JP2007235872A

  • Display device with sensor and electronic apparatus

    JP2012070356A

  • Camera lens structures and display structures for electronic devices

    JP2012098726A

  • Imaging display device

    JP2014238442A