Display-in-camera activation

Abstract

Methods, systems, and devices for display-in-camera activation are described. The method includes receiving a request to activate a camera of a device, identifying a start of a frame marker associated with activating the camera in response to the request, and transmitting a signal to a display of the device in response to the identified start of the frame marker, where the transmitted signal triggers the display to transition a set of liquid crystal elements of the display from a display mode to a camera mode for camera operation.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims priority to non-provisional application No. 17 / 160,007 entitled “IN-DISPLAYCAMERAACTIVATION”, filed on January 27, 2021, which has been assigned to the assignee and is hereby expressly incorporated by reference. Background Technology

[0003] The following pertains to in-display camera activation, including in-display camera activation.

[0004] Multimedia systems are widely deployed to provide various types of multimedia communication content, such as voice, video, packet data, messaging, and broadcasting. These multimedia systems are capable of processing, storing, generating, manipulating, and reproducing multimedia information. Examples of multimedia systems include entertainment systems, information systems, virtual reality systems, modeling and simulation systems, and more. These systems can employ a combination of hardware and software technologies to support the processing, storage, generation, manipulation, and reproduction of multimedia information, such as capture devices, storage devices, communication networks, computer systems, and display devices.

[0005] Image capture systems are typically incorporated into a wide variety of devices (e.g., digital cameras, smartphone cameras, cellular cameras, satellite radio cameras, video cameras, tablet cameras, laptop cameras, webcams, etc.). An image capture system includes a device configured to capture one or more images (e.g., photographic images, video images, sequences of images, digital images). Some image capture systems may include in-display cameras. Improvements may be required in devices with in-display cameras. Summary of the Invention

[0006] The described technology relates to improved methods, systems, devices, and apparatuses supporting in-display camera activation. Generally, the described technology provides an image capture device configured with a camera sensor, a display, and one or more processors (e.g., an image processor, a central processing unit (CPU), a mobile station modem, etc.). The camera sensor can be configured to capture images. In some cases, the camera sensor can be configured to receive light through at least a portion of the display. The image capture device, combined with one or more processors, can be configured to perform one or more operations. In some cases, the image capture device can receive a request to activate the camera of the image capture device. In some cases, the image capture device can recognize the start of a frame marker associated with camera activation in response to the request. In some cases, the image capture device can transmit a signal to the display of the image capture device in response to the recognized start of a frame marker. In some cases, the transmitted signal triggers the display to switch the display's liquid crystal element set from a display mode to a camera mode for camera operation. In some cases, the transmitted signal can include a general-purpose input / output signal, or an interrupt signal, or both.

[0007] A method for activating an in-display camera by a device is described. The method may include: receiving a request to activate the camera of the device; recognizing the start of a frame marker associated with camera activation in response to the request; and transmitting a signal to the display of the device in response to the recognized start of the frame marker, wherein the transmitted signal triggers the display to switch the display's liquid crystal element set from a display mode to a camera mode for camera operation.

[0008] An apparatus for activating an in-display camera by means of a device is described. The apparatus may include a processor, memory coupled to the processor, and instructions stored in the memory. The instructions may be executed by the processor to cause the apparatus to: receive a request to activate the camera of the device; in response to the request, identify the start of a frame marker associated with camera activation; and in response to the identified start of the frame marker, transmit a signal to the display of the device, wherein the transmitted signal triggers the display to switch its liquid crystal element array from a display mode to a camera mode for camera operation.

[0009] Another apparatus for activating an in-display camera by a device is described. The apparatus may include components for: receiving a request to activate the camera of the device; recognizing the start of a frame marker associated with camera activation in response to the request; and transmitting a signal to the display of the device in response to the recognized start of the frame marker, wherein the transmitted signal triggers the display to switch the display's liquid crystal element array from a display mode to a camera mode for camera operation.

[0010] A non-transitory computer-readable medium is described, storing code for activating an in-display camera by a device. The code may include processor-executable instructions to: receive a request to activate the camera of the device; recognize the start of a frame marker associated with camera activation in response to the request; and transmit a signal to the display of the device in response to the recognized start of the frame marker, wherein the transmitted signal triggers the display to switch its liquid crystal element array from a display mode to a camera mode for camera operation.

[0011] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, the signal may include an operation, feature, method or instruction for determining a first DC voltage level based on receiving a request to activate the camera of the device, and for the camera sensor of the device to emit a GPIO signal at the first DC voltage level to trigger a transition of the liquid crystal element assembly.

[0012] In some examples of the methods, apparatuses, and non-transitory computer-readable media described herein, the electrical connection of the GPIO signal is connected between the camera sensor and the switch of the controller or device's display.

[0013] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, when a controller or switch detects a GPIO signal at a first DC voltage level, the controller or switch of the display emits a trigger voltage, wherein the trigger voltage triggers a transition of the liquid crystal element assembly.

[0014] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, transforming the liquid crystal element set may include operations, features, methods or instructions for aligning the liquid crystal element set in a first orientation when a GPIO signal can be transmitted at a first DC voltage level, thereby increasing the transmission of light through the set of liquid crystal elements to a camera sensor in the first orientation.

[0015] In some examples of the methods, apparatuses and nontransitory computer-readable media described herein, when a GPIO signal can be transmitted at a second level different from a first DC voltage level, the liquid crystal element assembly is aligned in a second orientation, which reduces the transmission of light through the liquid crystal element assembly to the camera sensor.

[0016] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, transmitting signals may include operations, features, methods or instructions for generating an interrupt by the device's processor in response to the processor recognizing the start of a frame marker.

[0017] Some examples of the methods, apparatuses and non-transitory computer-readable media described herein may also include operations, features, methods or instructions for registering a generated interrupt to the camera driver during camera driver initialization, wherein the camera driver may be initialized after receiving a request to activate the camera and before the camera can be activated.

[0018] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, transmitting a signal to the display of a device may include operations, features, methods or instructions for sending a registered interrupt from a camera driver to the display.

[0019] In some examples of the methods, apparatuses, and non-transitory computer-readable media described herein, it is shown that the display driver receives a registered interrupt, triggering a transition in the liquid crystal element assembly.

[0020] In some examples of the methods, apparatuses and non-transitory computer-readable media described herein, a camera may be positioned behind an assembly of liquid crystal elements, and the field of view of the camera may be aligned with the assembly of liquid crystal elements. Attached Figure Description

[0021] Figure 1 An example of a system supporting in-display camera activation according to various aspects of this disclosure is shown.

[0022] Figure 2 An example of an image capture system supporting in-display camera activation according to various aspects of this disclosure is shown.

[0023] Figure 3 An example of a display system supporting in-display camera activation according to various aspects of this disclosure is shown.

[0024] Figure 4 An example block diagram is shown that supports in-display camera activation according to various aspects of this disclosure.

[0025] Figure 5 An example block diagram is shown that supports in-display camera activation according to various aspects of this disclosure.

[0026] Figure 6 and Figure 7 A block diagram of a device supporting in-display camera activation according to various aspects of this disclosure is shown.

[0027] Figure 8 A block diagram of a sensor configuration manager supporting in-display camera activation according to various aspects of this disclosure is shown.

[0028] Figure 9A schematic diagram of a system including a device supporting in-display camera activation according to various aspects of this disclosure is shown.

[0029] Figures 10 to 12 A flowchart illustrating a method for supporting in-display camera activation according to various aspects of this disclosure is shown. Detailed Implementation

[0030] This technology includes in-display camera activation. This technology provides improvements to the operation associated with an in-display camera system. Some devices may include an in-display camera, wherein a camera sensor is positioned under a portion of the device's display (e.g., a portion of the display's total pixels). When the display shows content, this portion of the display above the camera sensor can be used to display that content. When the device's in-display camera is activated, the in-display camera can detect light signals passing through this portion of the display above the camera sensor.

[0031] When initializing a camera positioned beneath this portion of the display, it may be desirable to transition that portion of the display above the camera sensor from display mode to camera mode, or vice versa. In some cases, transitioning this portion of the display to camera mode may involve physically transitioning at least some pixels of that portion of the display above the camera sensor to improve light transmission across the display to the camera sensor. However, some transition processes may include relatively significant delays (e.g., due to processor scheduling issues).

[0032] In some cases, the sequence of camera stack operations related to camera power-on / activation operations may include one or more operations based on the mobile computing API. In some cases, the camera operation may include a camera power-on operation (e.g., a request for access to the camera initiated by a process), followed by a camera power-on operation (e.g., opening the camera when access is authorized), followed by a display pixel-on operation (e.g., a display driver call to activate the display of pixels captured by the camera), followed by a camera streaming on operation (e.g., streaming pixels from the camera to the display). In some cases, the camera power-on operation and the display pixel-on operation may be configured to execute within the shortest possible time interval (e.g., at the same time or relatively at the same time) to reduce camera startup latency, although CPU scheduling latency can increase overall camera latency. In the software stack, the camera power-on operation and the display pixel-on operation can be called from the user mobile application to the user-mode driver and from the user-mode driver to the kernel-mode driver, which may take 60 to 70 milliseconds (ms) on relatively high-end devices or 100 ms on relatively low-end devices, depending on processor scheduling latency. A call transition time of 60 to 100 ms can cause delays in camera convergence and preview output.

[0033] In some cases, this technology provides a hardware mechanism to activate display pixels when the camera is powered on, reducing the 60 to 100 ms latency from software stack calls. Therefore, this technology includes replacing software mechanisms with hardware mechanisms. In some cases, this technology may include a device generating a signal to trigger a transition in that portion of the display above the camera sensor. In some cases, that portion of the display above the camera sensor may include a subset of the display's liquid crystal elements. In some cases, this technology includes a device generating a signal to trigger a transition (e.g., alignment in a given orientation) in the liquid crystal elements of the display to allow additional light to pass through the display to the camera sensor below the transitioned liquid crystal elements. In some cases, the generated signal may be triggered by the start of a frame marker generated by the device's image signal processor. In some cases, the generated signal may include a dedicated general-purpose input / output (GPIO) signal or an interrupt signal (e.g., a CPU interrupt signal), or both. In some cases, the GPIO signal may be emitted from the camera sensor to the display (e.g., to the display's logic circuitry, to the display driver, to the display processor, etc.).

[0034] The aspects of the subject matter described herein can be implemented to achieve one or more advantages. The described techniques can support improvements in system efficiency associated with devices that switch between display modes and camera modes on the camera sensor. Additionally, the described techniques reduce invocation switching time, resulting in faster camera convergence time and faster preview output time. Therefore, the techniques enable reduced system latency and improved user experience.

[0035] The aspects of this disclosure were initially described in the context of multimedia systems. These aspects are further illustrated and described with reference to in-display camera systems, display systems, and block diagrams. The aspects of this disclosure are further illustrated and described with reference to schematic diagrams of apparatus, systems, and flowcharts relating to in-display camera activation.

[0036] Figure 1 A multimedia system 100 supporting in-display camera activation is illustrated according to various aspects of this disclosure. Multimedia system 100 may include device 105, server 110, and database 115. Although multimedia system 100 illustrates two devices 105, one server 110, one database 115, and a network 120, this disclosure is applicable to any multimedia system architecture having one or more devices 105, server 110, database 115, and network 120. Device 105, server 110, and database 115 can communicate with each other and exchange information supporting in-display camera activation, such as multimedia packets, multimedia data, or multimedia control information, via network 120 using communication link 125. In some cases, some or all of the techniques for supporting in-display camera activation described herein may be implemented by device 105 or server 110, or both.

[0037] Device 105 may be a cellular phone, smartphone, personal digital assistant (PDA), wireless communication device, handheld device, tablet computer, laptop computer, cordless phone, display device (e.g., monitor), and / or device supporting various types of communication and multimedia-related functional features (e.g., sending, receiving, broadcasting, streaming, sinking, capturing, storing, and recording multimedia data). Additionally or alternatively, device 105 may be referred to by those skilled in the art as user equipment (UE), smartphone, Bluetooth device, Wi-Fi device, mobile station, subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, access terminal, mobile terminal, wireless terminal, remote terminal, mobile phone, user agent, mobile client, client, and / or some other suitable terms. In some cases, device 105 may also be directly able to communicate with another device (e.g., using point-to-point (P2P) or device-to-device (D2D) protocols). For example, device 105 may be able to receive or send various information, such as instructions or commands (e.g., multimedia-related information), to another device 105.

[0038] Device 105 may include application 130 and sensor configuration manager 135. Although the multimedia system 100 shows device 105 including both application 130 and sensor configuration manager 135, application 130 and sensor configuration manager 135 may be optional features of device 105. In some cases, application 130 may be a multimedia-based application that can receive (e.g., download, stream, broadcast) multimedia data from server 110, database 115, or another device 105, or send (e.g., upload) multimedia data to server 110, database 115, or another device 105 via communication link 125.

[0039] The sensor configuration manager 135 may be a general-purpose processor, a digital signal processor (DSP), an image signal processor (ISP), a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a part of a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof, or other programmable logic device, discrete gate or transistor logic, discrete hardware component, or any combination designed to perform the functions described in this disclosure. For example, the sensor configuration manager 135 may process multimedia (e.g., image data, video data, audio data) from the local memory or database 115 of device 105 and / or write multimedia data to the local memory or database 115 of device 105.

[0040] The sensor configuration manager 135 can also be configured to provide functions such as multimedia enhancement, multimedia restoration, multimedia analysis, multimedia compression, multimedia streaming, and multimedia synthesis. For example, the sensor configuration manager 135 can perform white balance, cropping, scaling (e.g., multimedia compression), resolution adjustment, multimedia stitching, color processing, multimedia filtering, spatial multimedia filtering, artifact removal, frame rate adjustment, multimedia encoding, multimedia decoding, and multimedia filtering. As a further example, the sensor configuration manager 135 can process multimedia data according to the techniques described herein to support the activation of an in-display camera.

[0041] Server 110 may be a data server, cloud server, server associated with a multimedia subscription provider, proxy server, network server, application server, communication server, home server, mobile server, or any combination thereof. In some cases, server 110 may include multimedia distribution platform 140. Multimedia distribution platform 140 may allow device 105 to discover, browse, share, and download multimedia via network 120 using communication link 125, and thus provide digital distribution of multimedia from multimedia distribution platform 140. In this way, digital distribution can be a form of delivering media content (such as audio, video, and images) without using physical media, but via online delivery media (such as the Internet). For example, device 105 may upload or download multimedia-related applications for streaming, downloading, uploading, processing, enhancing, etc., of multimedia (e.g., images, audio, and video). Server 110 may also send various information, such as instructions or commands (e.g., multimedia-related information), to device 105 to download multimedia-related applications on device 105.

[0042] Database 115 can store various types of information, such as instructions or commands (e.g., multimedia-related information). For example, database 115 can store multimedia 145. The device can support activation of an in-display camera associated with multimedia 145. Device 105 can retrieve stored data from database 115 via network 120 using communication link 125. In some examples, database 115 can be a relational database (e.g., a relational database management system (RDBMS) or a structured query language (SQL) database), a non-relational database, a web database, an object-oriented database, or other types of database that stores various types of information, such as instructions or commands (e.g., multimedia-related information).

[0043] Network 120 may provide encryption, access authorization, tracking, Internet Protocol (IP) connectivity, and other access, computation, modification, and / or functions. Examples of network 120 may include any combination of cloud computing networks, local area networks (LANs), wide area networks (WANs), virtual private networks (VPNs), wireless networks (e.g., using 802.11), cellular networks (using third-generation (3G), fourth-generation (4G), Long Term Evolution (LTE), or New Radio (NR) systems (e.g., fifth-generation (5G)), etc. Network 120 may include the Internet.

[0044] The communication link 125 shown in the multimedia system 100 may include uplink transmissions from device 105 to server 110 and database 115, and / or downlink transmissions from server 110 and database 115 to device 105. Communication link 125 may transmit bidirectional and / or unidirectional communication. In some examples, communication link 125 may be a wired or wireless connection, or both. For example, communication link 125 may include one or more connections, including but not limited to Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), cellular, Z-WAVE, 802.11, peer-to-peer, local area network, wireless local area network (WLAN), Ethernet, FireWire, fiber optic, and / or other connection types associated with wireless communication systems.

[0045] In some examples, the display of device 105 may include a first set of liquid crystal elements and a second set of liquid crystal elements. In some cases, the first set of liquid crystal elements may be fixed in place (e.g., fixed in place with a set orientation), while the second set of liquid crystal elements may be configured to change (e.g., configured to physically move or physically rotate in place). In some cases, the camera of device 105 may be positioned aligned with the second set of liquid crystal elements. In some cases, device 105, in conjunction with sensor configuration manager 135, may switch the display of device 105 from a first mode to a second mode (e.g., from display mode to camera mode). In some cases, the switching of the display may increase the amount of light incident on the sensor of the camera of device 105.

[0046] The techniques described herein can provide improvements to the operation of device 105 and sensor configuration manager 135, thereby increasing system efficiency and improving user experience when device 105 switches between display mode and camera mode.

[0047] Figure 2 Example 200 of an image capture system supporting the activation of an in-display camera according to various aspects of this disclosure is shown. In some examples, the image capture system 200 may implement various aspects of the multimedia system 100.

[0048] In the illustrated example, image capture system 200 includes or is part of device 205. Examples of device 205 may include a smartphone, personal digital assistant, camera device, tablet computer, laptop computer, desktop computer, handheld audio recording device, computer monitor, or any combination thereof. As shown, device 205 may include image sensor 215 (e.g., camera sensor), image processor 220 (e.g., image signal processor) connected to image sensor 215, sensor configuration manager 225 connected to image processor 220, and display 230 (e.g., display panel displaying images captured by image sensor 215, etc.). Sensor configuration manager 225 may be... Figure 1 An example of a sensor configuration manager 135. Although a single image sensor (image sensor 215) is shown, in some cases, a front-facing camera may be associated with a first image sensor (e.g., image sensor 215), and a rear-facing camera may be associated with a second image sensor, and the first and second image sensors may be connected to an image processor 220.

[0049] In some examples, display 230 may include liquid crystal elements. In some cases, the liquid crystal elements of display 230 may be arranged in a grid of liquid crystal elements. In some cases, display 230 may include a transition aperture 210.

[0050] In the illustrated example, image sensor 215 may be positioned relative to transition aperture 210. In some cases, image sensor 215 may be positioned under at least a portion of transition aperture 210. In some cases, transition aperture 210 may include an array of liquid crystal elements configured to switch between a display mode (e.g., a mode for displaying graphics on display 230) and a camera mode (e.g., a mode for capturing images via image sensor 215). In some cases, the array of liquid crystal elements in transition aperture 210 may be a subset of the total liquid crystal elements of display 230.

[0051] In some examples, the sensor configuration manager 225, in conjunction with the image sensor 215 and / or the image processor 220, can perform one or more of the operations described herein. For example, the sensor configuration manager 225 can perform one or more operations associated with switching the collection of liquid crystal elements of the display 230 from a first mode to a second mode (e.g., from a display mode to a camera mode).

[0052] In some examples, display 230 may include a first set of liquid crystal elements and a second set of liquid crystal elements. In some cases, the first set of liquid crystal elements may be fixed in place (e.g., fixed in place with a set orientation), while the second set of liquid crystal elements may be configured to change (e.g., configured to physically move or physically rotate in place). In some cases, the transition aperture 210 may include the second set of liquid crystal elements configured to make the transition. Thus, when device 205, in conjunction with sensor configuration manager 225, instructs display 230 to change from a first mode to a second mode, the second set of liquid crystal elements may change from a first orientation associated with the first mode (e.g., physically move) to a second orientation associated with the second mode, while the first set of liquid crystal elements remains fixed in place. For example, when device 205, in conjunction with sensor configuration manager 225, instructs display 230 to change from a display mode to a camera mode, the second set of liquid crystal elements may change from a first orientation associated with the display mode (e.g., physically move) to a second orientation associated with the camera mode, while the first set of liquid crystal elements remains fixed in place.

[0053] The operation of the sensor configuration manager 225, in conjunction with the image sensor 215 or the image processor 220, or both, increases system efficiency and improves user experience when the device 205 switches between display mode and camera mode by changing the aperture 210.

[0054] Figure 3 An example of a display system 300 supporting the activation of an in-display camera according to various aspects of this disclosure is shown. In some examples, the display system 300 may implement various aspects of the multimedia system 100. As shown, the display system 300 may include a display 305 of a device (e.g., device 205), an external screen 310 (e.g., the outer surface of the display 305), an image sensor 315, and a switching aperture 325-a. As shown, the display 305 may include a liquid crystal element assembly 320. In some cases, the display 305 may be... Figure 2 Example of a display 230. In some cases, the image sensor 315 may be... Figure 2 An example of an image sensor 215. In some cases, the variable aperture 325-a can be... Figure 2 An example of a 210-degree aperture conversion.

[0055] In the example shown, the transition aperture 325-a may include a subset of the liquid crystal elements of the liquid crystal element assembly 320. As shown, the image sensor 315 may be positioned relative to the transition aperture 325-a. In some cases, the image sensor 315 may be positioned such that the field of view of the image sensor 315 is aligned with at least one liquid crystal element of the subset of liquid crystal elements of the transition aperture 325-a.

[0056] In the example shown, a subset of the liquid crystal elements with changing aperture 325-a can be configured to switch between display mode and camera mode. In some cases, the subset of liquid crystal elements with changing aperture 325-a can switch between display mode and camera mode based on a signal. In some cases, this signal can be generated based on the start marker of a frame.

[0057] In some examples, the signal may include a general purpose input / output (GPIO) signal. In some cases, the GPIO signal may be emitted by the image sensor 315. In some cases, the GPIO signal may be emitted by the image sensor in conjunction with an image processor (e.g., image processor 220, sensor configuration manager 225). In some cases, the GPIO signal may be received by a display controller (e.g., display processor, display driver, logic integrated circuit, etc. of display 305).

[0058] In some examples, the signal may include an interrupt signal. In some cases, the interrupt signal may be emitted by the processor of the associated device (e.g., the device's image processor 220, sensor configuration manager 225, the device's CPU, mobile station modem, etc.). In some cases, the interrupt may be received by the display controller (e.g., the display processor, display driver, logic integrated circuit, etc. of the display 305).

[0059] In some examples, transition aperture 325-a depicts a subset of liquid crystal elements in display mode, while transition aperture 325-b depicts a subset of liquid crystal elements in camera mode. In some cases, when display 305 receives a signal, the subset of liquid crystal elements in transition aperture 325-a in display mode can transition to the subset of liquid crystal elements in transition aperture 325-b in camera mode. In some cases, display 305 can transition from camera mode to display mode. In some cases, when display 305 receives a first signal (e.g., a camera trigger signal), the subset of liquid crystal elements in transition aperture 325-a in display mode can transition to the subset of liquid crystal elements in transition aperture 325-b in camera mode. When display 305 receives a second signal (e.g., a display trigger signal), the subset of liquid crystal elements in transition aperture 325-b in camera mode can transition to the subset of liquid crystal elements in transition aperture 325-a in display mode. In some cases, the first or second signal may include a GPIO signal, an interrupt signal, or both.

[0060] As shown in the figure, when a subset of liquid crystal elements is in display mode (e.g., with aperture 325-a), the subset of liquid crystal elements is aligned with the other liquid crystal elements in the liquid crystal element set 320. When a subset of liquid crystal elements is in camera mode (e.g., with aperture 325-b), the orientation of the subset of liquid crystal elements is perpendicular to the other liquid crystal elements in the liquid crystal element set 320.

[0061] This technology reduces call transition time and frees up one or more devices (e.g., battery-operated devices) by emitting a signal that triggers the display to switch between display mode and camera mode. Figure 2 This technology improves the user experience of one or more devices by reducing the processing cycle of devices such as device 205, thereby improving display / camera switching time, extending battery life, and improving service quality. This technology enables faster camera convergence time and faster preview output time. Therefore, the described technology reduces system latency and improves user experience.

[0062] Figure 4 Example 400 is shown as a block diagram supporting the activation of an in-display camera according to various aspects of this disclosure. In some examples, block diagram 400 may implement various aspects of multimedia system 100. In the example shown, block diagram 400 includes an image sensor 405, display control 415, and display 420. In some cases, image sensor 405 may be... Figure 2 Image sensor 215 or Figure 3 An example of an image sensor 315. In some cases, the display 420 may be... Figure 2 Monitor 230 or Figure 3 An example of display 305. In some cases, display control 415 may include the display processor, or display driver, or logic integrated circuit, or any combination thereof of display 420. In some cases, display 420 may include at least a portion of display control 415.

[0063] In some examples, block diagram 400 illustrates an example of a GPIO synchronization mechanism to switch display pixels (e.g., switch a subset of liquid crystal elements with varying apertures) when the camera is powered on. As shown, GPIO connection 410 can be connected between image sensor 405 and display control 415. In some cases, image sensor 405 can generate GPIO signals and transmit them to display control 415 via GPIO connection 410. In some cases, the transmitted GPIO signals trigger a transition between display mode and camera mode. In some examples, image sensor 405 can generate GPIO signals in conjunction with one or more processors or logic integrated circuits (e.g., image processor 220, sensor configuration manager 225, CPU of associated devices, etc.).

[0064] In some examples, the transmitted GPIO signal can control the transition between display mode and camera mode based on the voltage level of the GPIO signal. When the image sensor 405 is powered on and begins streaming captured pixels (e.g., including the start of a frame marker), the GPIO voltage level can be set to a camera mode voltage level (e.g., a high voltage level or a low voltage level, etc.). When the image sensor 405 is powered off and stops streaming captured pixels, the GPIO voltage level can be set to a display mode voltage level (e.g., a low voltage level or a high voltage level, or a voltage level opposite to the camera mode voltage level, etc.). In some cases, a zero voltage level or a voltage level relatively close to zero can indicate a low voltage level, while a voltage level above a given voltage threshold (e.g., a voltage level of 4.3 volts or 5 volts, or a voltage level relatively close to that) can indicate a high voltage level.

[0065] In some examples, a transition between camera mode and display mode (e.g., from camera mode to display mode, or from display mode to camera mode) can be triggered when the voltage level of the GPIO between the cathode and anode associated with display 420 is higher than a voltage threshold. In other cases, a transition between camera mode and display mode (e.g., from display mode to camera mode, or from camera mode to display mode) can be triggered when the voltage level of the GPIO between the cathode and anode associated with display 420 is lower than a voltage threshold.

[0066] In some examples, display 420 may receive or process GPIO signals via display control 415. In some cases, the processor associated with image sensor 405 may detect the start of a frame marker (e.g., the start of a frame marker from a camera serial interface decoder of image sensor 405). The processor associated with image sensor 405 may include an image processor (e.g., image processor 220), or the CPU of the device, or the mobile station modem of the device, or a logic integrated circuit integrated into image sensor 405, or any combination thereof.

[0067] As shown, image sensor 405 (e.g., in conjunction with a processor associated with image sensor 405) can generate GPIO signals and send the generated GPIO signals to display control 415 via GPIO connection 410. In some cases, GPIO connection 410 can be dedicated to triggering transitions between display mode and camera mode.

[0068] In some examples, the display control 415 may include a multiplexer. In some cases, when the display control 415 receives a GPIO at or near a high voltage level, it may set the output voltage (Vout) to a first voltage (V1), and when it receives a GPIO at or near a low voltage level, it may set Vout to a second voltage (V2). In the example shown, V1 is a voltage level above a certain voltage threshold (V1>Vth), while V2 is a voltage level below the voltage threshold (V2<Vth> ... <Vth)。

[0069] In the example shown, a high-voltage GPIO signal can trigger a transition from display mode to camera mode. Therefore, when the image sensor 405 sends a high-voltage GPIO signal, the display control 415 can determine that the received GPIO signal is at a high voltage level. When the display control 415 determines that the received GPIO signal is at a high voltage level, it can set Vout to V1 and output Vout to the display 420.

[0070] This technology improves one or more devices (e.g., battery-operated devices) by transmitting a GPIO signal on GPIO connection 410 to trigger the display 420 to switch between display mode and camera mode, thereby improving display / camera switching time, extending battery life, and improving service quality. Figure 2 The technology reduces the call transition time and release processing cycle of devices (such as device 205). This results in faster camera convergence time and faster preview output time. Therefore, the described technology reduces system latency and improves user experience.

[0071] Figure 5 Example 500 is shown as a block diagram supporting the activation of an in-display camera according to various aspects of this disclosure. In some examples, block diagram 500 may implement various aspects of multimedia system 100. In the example shown, block diagram 500 includes an image sensor 505, display control 515, and display 520. In some cases, image sensor 505 may be... Figure 3 Image sensor 315 or Figure 4 An example of an image sensor 405. In some cases, the display 520 may be... Figure 3 Monitor 330 or Figure 4 Examples of display 420. In some cases, display control 515 may include a display processor, or display driver, or logic integrated circuit, or any combination thereof of display 520. In some cases, display 520 may include at least a portion of display control 515.

[0072] In some examples, block diagram 500 illustrates an example of a GPIO synchronization mechanism to switch display pixels (e.g., switch a subset of liquid crystal elements with varying apertures) when the camera is powered on. As shown, GPIO connection 510 can be connected between image sensor 505 and display control 515. In some cases, image sensor 505 can generate GPIO signals and transmit them to display control 515 via GPIO connection 510. In some cases, the transmitted GPIO signals trigger a transition between display mode and camera mode. In some examples, image sensor 505 can generate GPIO signals in conjunction with one or more processors or logic integrated circuits (e.g., image processor 220, sensor configuration manager 225, CPU of associated devices, etc.).

[0073] In some examples, the transmitted GPIO signal can control the transition between display mode and camera mode based on the voltage level of the GPIO signal. When the image sensor 505 is powered on and begins streaming captured pixels (e.g., including the start of a framer marker), the GPIO voltage level can be set to a camera mode voltage level (e.g., a high voltage level or a low voltage level, etc.). When the image sensor 505 is powered off and stops streaming captured pixels, the GPIO voltage level can be set to a display mode voltage level (e.g., a low voltage level or a high voltage level, or a voltage level opposite to the camera mode voltage level, etc.). In some cases, a zero voltage level or a voltage level relatively close to zero can indicate a low voltage level, while a voltage level above a given voltage threshold (e.g., a voltage level of 5.3 volts or 5 volts, or a voltage level relatively close to that) can indicate a high voltage level.

[0074] In some examples, a transition between camera mode and display mode (e.g., from camera mode to display mode, or from display mode to camera mode) can be triggered when the voltage level of the GPIO between the cathode and anode associated with display 520 is higher than a voltage threshold. In other cases, a transition between camera mode and display mode (e.g., from display mode to camera mode, or from camera mode to display mode) can be triggered when the voltage level of the GPIO between the cathode and anode associated with display 520 is lower than a voltage threshold.

[0075] In some examples, display 520 may receive or process GPIO signals via display control 515. In some cases, the processor associated with image sensor 505 may detect the start of a frame marker (e.g., the start of a frame marker from a camera serial interface decoder of image sensor 505). The processor associated with image sensor 405 may include an image processor (e.g., image processor 220), or the CPU of the device, or the mobile station modem of the device, or a logic integrated circuit integrated into image sensor 505, or any combination thereof.

[0076] As shown in the figure, image sensor 505 (e.g., in conjunction with a processor associated with image sensor 505) can generate GPIO signals and send the generated GPIO signals to display control 515 via GPIO connection 510. In some cases, GPIO connection 510 can be dedicated to triggering transitions between display mode and camera mode.

[0077] In some examples, the display control 515 may include a multiplexer. In some cases, when the display control 515 receives a GPIO at or near a high voltage level, it may set the output voltage (Vout) to a first voltage (V1), and when it receives a GPIO at or near a low voltage level, it may set Vout to a second voltage (V2). In the example shown, V1 is a voltage level above a certain voltage threshold (V1>Vth), while V2 is a voltage level below the voltage threshold (V2<Vth> ... <Vth)。

[0078] In the example shown, a high-voltage GPIO signal can trigger a transition from display mode to camera mode. Therefore, when the image sensor 405 sends a high-voltage GPIO signal, the display control 515 can determine that the received GPIO signal is at a high voltage level. When the display control 515 determines that the received GPIO signal is at a high voltage level, the display control 515 can set Vout to V1 and output Vout to the display 520.

[0079] This technology improves one or more devices (e.g., battery-operated devices) by transmitting a GPIO signal on GPIO connection 510 to trigger the display 520 to switch between display mode and camera mode, thereby improving display / camera switching time, extending battery life, and improving service quality. Figure 1 Equipment 105 Figure 2 The technology reduces the call transition time and release processing cycle of devices (such as device 205). This results in faster camera convergence time and faster preview output time. Therefore, the described technology reduces system latency and improves user experience.

[0080] Figure 6 A block diagram 600 shows a device 605 supporting the activation of an in-display camera according to various aspects of this disclosure. Device 605 may be an example of an aspect of the camera device described herein. Device 605 may include a sensor 610, a sensor configuration manager 615, and a display 620. Device 605 may also include a processor. Each of these components may communicate with each other (e.g., via one or more buses).

[0081] One or more sensors 610 (e.g., image sensors, cameras, etc.) can receive information (e.g., light, such as visible and / or invisible light), which can be transmitted to other components 605 of the device. In some cases, sensor 610 can be a reference sensor. Figure 9 Example of an aspect of the described I / O controller 915. Sensor 610 may utilize one or more photosensitive elements sensitive to the electromagnetic radiation spectrum to receive information (e.g., sensor 610 may be configured or adjusted to receive pixel intensity values, pixel red-green-blue (RGB) values, infrared (IR) light values, near-IR light values, ultraviolet (UV) light values, etc.). This information can then be passed to other components 605 of the device.

[0082] Sensor configuration manager 615 can receive a request to activate the camera of the device, identify the start of a frame marker associated with camera activation in response to the request, and transmit a signal to the device's display in response to the identified start of the frame marker, wherein the transmitted signal triggers the display to switch the display's liquid crystal element set from display mode to camera mode for camera operation. Sensor configuration manager 615 can be an example of an aspect of sensor configuration manager 910.

[0083] The sensor configuration manager 615 or its sub-components may be implemented in hardware, processor-executable code (e.g., software or firmware), or any combination thereof. If implemented in processor-executable code, the functionality of the sensor configuration manager 615 or its sub-components may be performed by a general-purpose processor, DSP, application-specific integrated circuit (ASIC), FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of devices designed to perform the functionality described in this disclosure.

[0084] The sensor configuration manager 615 or its sub-components may be physically located in various locations, including being distributed such that some functions are implemented by one or more physical components at different physical locations. In some examples, the sensor configuration manager 615 or its sub-components may be separate and distinct components according to various aspects of this disclosure. In some examples, the sensor configuration manager 615 or its sub-components may be combined with one or more other hardware components, including but not limited to input / output (I / O) components, transceivers, network servers, other computing devices, one or more other components described in this disclosure, or combinations thereof according to various aspects of this disclosure.

[0085] Display 620 can display content generated by other components of the device. Display 620 can be a reference. Figure 6 An example of the described display 20. In some examples, the display 620 may be connected to a display buffer that stores rendered data until the image is ready to be displayed (e.g., as shown in reference). Figure 6 (As described). Display 620 may emit light based on signals or information generated by other components of device 605. For example, display 620 may receive display information (e.g., pixel mapping, display adjustment) from sensor 610 and may emit light accordingly. Display 620 may represent a unit capable of displaying video, images, text, or any other type of data for a viewer to consume. Display 620 may include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic LED (OLED), an active-matrix OLED (AMOLED), etc. In some cases, display 620 and I / O controller (e.g., I / O controller 915) may be or represent aspects of the same component (e.g., a touchscreen) of device 605. Display 620 may be any suitable display or screen that allows user interaction and / or allows the presentation of information (such as captured images and videos) for user viewing. In some aspects, display 620 may be a touch-sensitive display. In some cases, display 620 may display images captured by sensors, wherein the displayed images captured by sensors may depend on the configuration of the light source and active sensors by sensor configuration manager 615.

[0086] Figure 7 This is a block diagram 700 showing a device 705 supporting in-display camera activation according to various aspects of this disclosure. Device 705 may be an example of an aspect of device 605 or an aspect of device 105 described herein. Device 705 may include a sensor 710, a sensor configuration manager 715, and a display 735. Device 705 may also include a processor. Each of these components may communicate with each other (e.g., via one or more buses).

[0087] One or more sensors 710 (e.g., image sensors, cameras, etc.) can receive information (e.g., light, such as visible and / or invisible light), which can be transmitted to other components 705 of the device. In some cases, sensor 710 can be a reference sensor. Figure 9 Example of an aspect of the described I / O controller 915. Sensor 710 may utilize one or more photosensitive elements sensitive to the electromagnetic radiation spectrum to receive information (e.g., sensor 710 may be configured or adjusted to receive pixel intensity values, pixel red-green-blue (RGB) values, infrared (IR) light values, near-IR light values, ultraviolet (UV) light values, etc.). This information can then be passed to other components 705 of the device.

[0088] Sensor configuration manager 715 may be an example of an aspect of sensor configuration manager 615 described herein. Sensor configuration manager 715 may include mode manager 720, transition manager 725, and signal manager 730. Sensor configuration manager 715 may be an example of an aspect of sensor configuration manager 910 described herein. Mode manager 720 may receive a request to activate the device's camera. Transition manager 725 may, in response to the request, identify the start of a frame marker associated with camera activation.

[0089] The signal manager 730 can transmit a signal to the device's display in response to the start of a recognized frame marker, wherein the transmitted signal triggers the display to switch the display's liquid crystal element set from display mode to camera mode for camera operation.

[0090] The display 735 can display content output by other components of the device. The display 735 can be a reference. Figure 7 An example of a display 735 is described. In some examples, the display 735 may be connected to a display buffer that stores rendered data until the image is ready to be displayed (e.g., as shown in the reference). Figure 7(As described). Display 735 may emit light based on signals or information generated by other components of device 705. For example, display 735 may receive display information (e.g., pixel mapping, display adjustment) from sensor 710 and may emit light accordingly. Display 735 may represent a unit capable of displaying video, images, text, or any other type of data for a viewer to consume. Display 735 may include liquid crystal display (LCD), light-emitting diode (LED) display, organic LED (OLED), active-matrix OLED (AMOLED), etc. In some cases, display 735 and I / O controller (e.g., I / O controller 915) may be or represent aspects of the same component (e.g., touchscreen) of device 705. Display 735 may be any suitable display or screen that allows user interaction and / or allows the presentation of information (such as captured images and videos) for user viewing. In some aspects, display 735 may be a touch-sensitive display. In some cases, display 735 may display images captured by sensors, wherein the displayed images captured by sensors may depend on the configuration of the light source and active sensors by sensor configuration manager 715.

[0091] Figure 8 This is a block diagram 800 showing a sensor configuration manager 805 supporting the activation of an in-display camera according to various aspects of this disclosure. The sensor configuration manager 805 may be an example of aspects of the sensor configuration manager 615, sensor configuration manager 715, or sensor configuration manager 910 described herein. The sensor configuration manager 805 may include a mode manager 810, a transition manager 815, a signal manager 820, and an interrupt manager 825. Each of these modules may communicate directly or indirectly with each other (e.g., via one or more buses).

[0092] The mode manager 810 can receive a request to activate the camera of the device. The transition manager 815 can, in response to the request, identify the start of a frame marker associated with camera activation. The signal manager 820 can, in response to the identified start of a frame marker, transmit a signal to the device's display, wherein the transmitted signal triggers the display to switch its liquid crystal element set from display mode to camera mode for camera operation. In some cases, the camera is positioned behind the liquid crystal element set, and the camera's field of view is aligned with the liquid crystal element set.

[0093] In some examples, signal manager 820 may determine a first DC voltage level based on a received request to activate the camera of the device. In some examples, signal manager 820 may trigger a transition of the liquid crystal element assembly by having the camera sensor of the device emit a GPIO signal at the first DC voltage level. In some examples, when emitting a GPIO signal at the first DC voltage level, signal manager 820 may align the liquid crystal element assembly in a first orientation, increasing the transmission of light through the liquid crystal element assembly to the camera sensor in the first orientation.

[0094] In some cases, the electrical connection of the GPIO signal is established between the camera sensor and the switch of the controller or device's display. In some cases, when the controller or switch detects a GPIO signal at a first DC voltage level, the display's controller or transmitter emits a trigger voltage, which triggers a transition in the liquid crystal element assembly. In some cases, when the GPIO signal is emitted at a second level different from the first DC voltage level, the liquid crystal element assembly is aligned in a second orientation, which reduces the transmission of light through the liquid crystal element assembly to the camera sensor.

[0095] Interrupt manager 825 can generate an interrupt in response to the start of a processor-recognized frame marker in the device. In some examples, interrupt manager 825 can register the generated interrupt with the camera driver during camera driver initialization, where the camera driver is initialized after a request to activate the camera is received and before the camera is activated.

[0096] In some examples, the interrupt manager 825 can send registered interrupts from the camera driver to the display driver of the monitor. In some cases, the display driver receiving the registered interrupt triggers a transition in the liquid crystal element assembly.

[0097] Figure 9 This is a schematic diagram of a system 900 including a device 905 supporting the activation of an in-display camera according to various aspects of this disclosure. Device 905 may be, or include, examples of components of the device 605, device 705, or camera device described herein. Device 905 may include components for bidirectional voice and data communication, including components for transmitting and receiving communications, including a sensor configuration manager 910, an I / O controller 915, a memory 920, a processor 925, and a light source 935. These components may be in electrical communication via one or more buses (e.g., a bus).

[0098] The sensor configuration manager 910 can receive a request to activate the camera of the device, identify the start of a frame marker associated with the activation of the camera in response to the request, and transmit a signal to the display of the device in response to the identified start of the frame marker, wherein the transmitted signal triggers the display to switch the display's liquid crystal element set from display mode to camera mode for camera operation.

[0099] The I / O controller 915 can manage the input and output signals of the device 905. The I / O controller 915 can also manage peripheral devices not integrated into the device 905. In some cases, the I / O controller 915 can represent a physical connection or port to an external peripheral device. In some cases, the I / O controller 915 can utilize an operating system, such as... Or other known operating systems. In other cases, the I / O controller 915 may represent or interact with a modem, keyboard, mouse, touchscreen, or similar device. In some cases, the I / O controller 915 may be implemented as part of a processor. In some cases, a user may interact with the device 905 via the I / O controller 915 or via hardware components controlled by the I / O controller 915.

[0100] Memory 920 may include RAM and ROM. Memory 920 may store computer-readable, computer-executable code 930, including instructions that, when executed, cause the processor to perform the various functions described herein. In some cases, among others, memory 920 may contain a BIOS that controls basic hardware or software operations, such as interaction with peripheral components or devices.

[0101] Processor 925 may include intelligent hardware devices (e.g., general-purpose processors, DSPs, CPUs, microcontrollers, ASICs, FPGAs, programmable logic devices, discrete gate or transistor logic components, discrete hardware components, or any combination thereof). In some cases, processor 925 may be configured to use a memory controller to operate a memory array. In other cases, the memory controller may be integrated into processor 925. Processor 925 may be configured to execute computer-readable instructions stored in memory (e.g., memory 920) to cause device 905 to perform various functions (e.g., functions or tasks supporting in-display camera activation).

[0102] Software 930 may include instructions for implementing aspects of this disclosure, including instructions for supporting in-display camera activation. Software 930 may be stored in a non-transitory computer-readable medium, such as system memory or other types of memory. In some cases, software 930 may not be directly executed by a processor 925, but may enable a computer (e.g., when compiled and executed) to perform the functions described herein.

[0103] One or more light sources 935 may include light sources capable of emitting visible and / or invisible light. In the example, light source 935 may include visible light sources and active invisible light sources (e.g., IR light sources, near-IR light sources, UV light sources). In some cases, light source 935 may be a reference. Figure 9 Examples of aspects of the described light source 935.

[0104] Figure 10 A flowchart illustrating a method 1000 for activating an in-display camera according to various aspects of this disclosure is shown. Operation of method 1000 can be performed by a camera device described herein (e.g., Figure 1 The device 105) or its components are implemented. For example, the operation of method 1000 can be carried out by reference. Figures 6 to 9 The described sensor configuration manager is executed. In some examples, the camera device may execute a set of instructions to control the functional elements of the camera device to perform the functions described below. Additionally, or alternatively, the camera device may use dedicated hardware to perform aspects of the functions described below.

[0105] At point 1005, the camera device can receive a request to activate the camera on the device. The operation at point 1005 can be performed according to the method described herein. In some examples, aspects of the operation at point 1005 can be derived from [reference needed]. Figures 6 to 9 The described pattern manager is executed.

[0106] At point 1010, the camera device can, in response to the request, recognize the start of a frame marker associated with camera activation. The operation of point 1010 can be performed according to the methods described herein. In some examples, aspects of the operation of point 1010 can be derived from references... Figures 6 to 9 The described transformation manager is executed.

[0107] At point 1015, the camera device can transmit a signal to the device's display in response to the start of a recognized frame marker, wherein the transmitted signal triggers the display to switch its liquid crystal element array from display mode to camera mode for camera operation. The operation of point 1015 can be performed according to the method described herein. In some examples, aspects of the operation of point 1015 can be referenced... Figures 6 to 9The described signal manager is executed.

[0108] Figure 11 A flowchart illustrating an activation method 1100 for an in-display camera according to various aspects of this disclosure is shown. Operation of method 1100 can be implemented by a camera device or its components described herein. For example, operation of method 1100 can be performed by referring to... Figures 6 to 9 The sensor configuration manager described herein is executed. In some examples, the camera device may execute a set of instructions to control the functional elements of the camera device to perform the functions described below. Additionally, or alternatively, the camera device may use dedicated hardware to perform aspects of the functions described below.

[0109] At point 1105, the camera device can receive a request to activate the camera on the device. The operation at point 1105 can be performed according to the methods described herein. In some examples, aspects of the operation at point 1105 can be derived from [reference needed]. Figures 6 to 9 The described pattern manager is executed.

[0110] At 1110, the camera device can, in response to the request, recognize the start of a frame marker associated with camera activation. The operation of 1110 can be performed according to the methods described herein. In some examples, aspects of the operation of 1110 can be derived from references... Figures 6 to 9 The described transformation manager is executed.

[0111] At 1115, the camera device can transmit a signal to the device's display in response to the start of a recognized frame marker, wherein the transmitted signal triggers the display to switch its liquid crystal element array from display mode to camera mode for camera operation. The operation of 1115 can be performed according to the method described herein. In some examples, aspects of the operation of 1115 can be referenced... Figures 6 to 9 The described signal manager is executed.

[0112] At 1120, the camera device can determine a first DC voltage level based on a received request to activate the camera. The operation of 1120 can be performed according to the method described herein. In some examples, aspects of the operation of 1120 can be determined by reference to... Figures 6 to 9 The described signal manager is executed.

[0113] At 1125, the camera device can trigger a transition of the liquid crystal element assembly by transmitting a GPIO signal at a first DC voltage level via its camera sensor. The operation of 1125 can be performed according to the method described herein. In some examples, aspects of the operation of 1125 can be referenced... Figures 6 to 9 The described signal manager is executed.

[0114] Figure 12 A flowchart illustrating an activation method 1200 for an in-display camera according to various aspects of this disclosure is shown. Operation of method 1200 can be implemented by a camera device or its components described herein. For example, operation of method 1200 can be performed by referring to... Figures 6 to 9 The sensor configuration manager described herein is executed. In some examples, the camera device may execute a set of instructions to control the functional elements of the camera device to perform the functions described below. Additionally, or alternatively, the camera device may use dedicated hardware to perform aspects of the functions described below.

[0115] At point 1205, the camera device can receive a request to activate the camera on the device. The operation of point 1205 can be performed according to the methods described herein. In some examples, aspects of the operation of point 1205 can be derived from references... Figures 6 to 9 The described pattern manager is executed.

[0116] At 1210, the camera device can, in response to the request, recognize the start of a frame marker associated with camera activation. The operation of 1210 can be performed according to the methods described herein. In some examples, aspects of the operation of 1210 can be derived from references... Figures 6 to 9 The described transformation manager is executed.

[0117] At 1215, the camera device can transmit a signal to the device's display in response to the start of a recognized frame marker, wherein the transmitted signal triggers the display to switch its liquid crystal element array from display mode to camera mode for camera operation. The operation of 1215 can be performed according to the method described herein. In some examples, aspects of the operation of 1215 can be referenced... Figures 6 to 9 The described signal manager is executed.

[0118] At 1220, the camera device may generate an interrupt in response to the device's processor recognizing the start of a frame marker. The operation of 1220 can be performed according to the methods described herein. In some examples, aspects of the operation of 1220 can be referenced... Figures 6 to 9 The described interrupt manager is executed.

[0119] It should be noted that the methods described in this paper describe possible implementations, the operations and steps can be rearranged or otherwise modified, and other implementations are possible. Furthermore, aspects from two or more methods can be combined.

[0120] The following provides an overview of the various aspects of this disclosure:

[0121] Aspect 1: A method for activating an in-display camera by a device, comprising: receiving a request to activate the camera of the device; recognizing, in response to the request, the start of a frame marker associated with the activation of the camera; and transmitting a signal to the display of the device in response to the recognized start of the frame marker, wherein the transmitted signal triggers the display to switch the liquid crystal element set of the display from a display mode to a camera mode for camera operation.

[0122] Aspect 2: According to the method of aspect 1, wherein the signal includes a general purpose input / output (GPIO) signal, and transmitting the signal includes: determining a first DC voltage level based at least in part on receiving a request to activate the camera of the device; and transmitting a GPIO signal at the first DC voltage level by the camera sensor of the device to trigger a transition of the liquid crystal element set.

[0123] Aspect 3: According to the method of aspect 2, the electrical connection of the GPIO signal is connected between the camera sensor and the controller or switch of the device's display.

[0124] Aspect 4: According to the method of aspect 3, when the controller or switch detects that the GPIO signal is at a first DC voltage level, the controller or switch of the display emits a trigger voltage, which triggers the transition of the liquid crystal element set.

[0125] Aspect 5: The method according to any one of Aspects 2 to 4, wherein transforming the liquid crystal element assembly includes: aligning the liquid crystal element assembly in a first orientation when transmitting a GPIO signal at a first DC voltage level, and increasing the transmission of light through the liquid crystal element assembly to the camera sensor in the first orientation.

[0126] Aspect 6: According to the method of aspect 5, when a GPIO signal is transmitted at a second level different from the first DC voltage level, the liquid crystal element assembly is aligned in a second orientation, thereby reducing the transmission of light through the liquid crystal element assembly to the camera sensor in the second orientation.

[0127] Aspect 7: The method according to any one of Aspects 1 to 6, wherein transmitting the signal includes: generating an interrupt by the processor of the device in response to the processor recognizing the start of a frame marker.

[0128] Aspect 8: The method according to aspect 7 includes: registering a generated interrupt to the camera driver of the camera during the initialization of the camera driver, wherein the camera driver is initialized after receiving a request to activate the camera and before the camera is activated.

[0129] Aspect 9: According to the method of aspect 8, transmitting a signal to the display of the device includes: sending a registered interrupt from the camera driver to the display driver of the display.

[0130] Aspect 10: According to the method of aspect 9, the display driver receives a registered interrupt, triggering a transition of the liquid crystal element set.

[0131] Aspect 11: The method according to any one of Aspects 1 to 10, wherein the camera is positioned behind the liquid crystal element assembly and the field of view of the camera is aligned with the liquid crystal element assembly.

[0132] Aspect 12: A camera for a device for activating a camera within a display, comprising: a processor, a memory coupled to the processor; and instructions stored in the memory and executable by the processor to cause the camera to: receive a request to activate the camera; recognize the start of a frame marker associated with camera activation in response to the request; and transmit a signal to the display of the device in response to the recognized start of the frame marker, wherein the transmitted signal triggers the display to switch the liquid crystal element set of the display from a display mode to a camera mode for camera operation.

[0133] Aspect 13: According to the camera of aspect 12, wherein the signal includes a general purpose input / output (GPIO) signal, and the transmitted signal includes instructions stored in memory and executable by a processor, to cause the camera to: determine a first DC voltage level based at least in part on receiving the request for activation of the camera; and transmit a GPIO signal at the first DC voltage level by the camera sensor of the camera to trigger a transition of the liquid crystal element assembly.

[0134] Aspect 14: An apparatus for activating an in-display camera by a device, comprising a processor; a memory coupled to the processor; and instructions stored in the memory and executable by the processor to cause the apparatus to perform any one of aspects 1 to 11.

[0135] Aspect 15: An apparatus for activating an in-display camera by a device, comprising at least one component for performing the method of any one of Aspects 1 to 11.

[0136] Aspect 16: A non-transitory computer-readable medium storing code for activating an in-display camera by a device, the code including instructions executable by a processor to perform the method of any one of Aspects 1 to 11.

[0137] Aspect 17: An apparatus comprising a processor; a memory coupled to the processor; and instructions stored in the memory and executable by the processor to cause the apparatus to perform any one of aspects 12 to 13.

[0138] Aspect 18: An apparatus comprising at least one component for performing the method of any one of Aspects 12 to 13.

[0139] Aspect 19: A non-transitory computer-readable medium storing code including instructions executable by a processor to perform the methods of any one of Aspects 12 to 13.

[0140] The information and signals described herein can be represented using a variety of different techniques and processes. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the description can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof.

[0141] The various illustrative blocks and modules disclosed herein may be implemented or performed using a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination thereof, designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration).

[0142] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, these functions can be stored or transmitted as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope of this disclosure and the appended claims. For example, due to the nature of software, the functions described herein can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in different locations, including being distributed such that portions of the functions are implemented in different physical locations.

[0143] Computer-readable media include non-transitory computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. A non-transitory storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. By way of example, and not limitation, a non-transitory computer-readable medium can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), flash memory, optical disc (CD) ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Additionally, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are all included in the definition of medium. The disks and optical discs used in this article include CDs, laser discs, optical discs, digital multifunction discs (DVDs), floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while optical discs copy data optically using lasers. Combinations of these are also included within the scope of computer-readable media.

[0144] As used herein, including in the claims, the word "or" used in a list of items (e.g., preceded by phrases such as "at least one" or "one or more") indicates an inclusive list, such as a list of at least one of A, B, or C meaning A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Additionally, as used herein, the phrase "based on" should not be construed as referring to a closed set of conditions. For example, an exemplary step described as "based on condition A" may be based on both condition A and condition B without departing from the scope of this disclosure. In other words, as used herein, the phrase "based on" should be interpreted in the same manner as the phrase "at least partially based on".

[0145] In the accompanying drawings, similar components or features may have the same reference label. Furthermore, various components of the same type can be distinguished by adding a dash after the reference label and a second label to differentiate them. If only the first reference label is used in the specification, the description applies to any similar component having the same first reference label, regardless of the second or subsequent reference labels.

[0146] The description herein, taken in conjunction with the accompanying drawings, illustrates exemplary configurations and does not represent all examples that can be implemented or that are within the scope of the claims. The term "exemplary" as used herein means "as an example, instance, or illustration," and not "preferred" or "advantageous over other examples." The detailed description includes specific details intended to provide an understanding of the described techniques. However, these techniques can also be practiced without these specific details. In some cases, well-known structures and devices are shown in block diagram form to avoid obscuring the concept of the described examples.

[0147] The description herein is intended to enable those skilled in the art to make or use the contents of this disclosure. Various modifications to the contents of this disclosure will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but should be given the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for activating an in-display camera by a device, comprising: Receive a request to activate the camera of the device; In response to the request, identify the start of a frame marker associated with activating the camera; as well as In response to the start of the identified frame marker, a signal is transmitted to the display of the device, wherein the transmitted signal triggers the display to switch the display's element set from display mode to camera mode for camera operation.

2. The method according to claim 1, wherein, The signal includes general purpose input / output (GPIO) signals, and transmitting the signal includes: The first DC voltage level is determined at least in part based on the received request to activate the camera of the device; and The camera sensor of the device emits the GPIO signal at the first DC voltage level to trigger the transition of the component set.

3. The method according to claim 2, wherein, The GPIO signal is electrically connected between the camera sensor of the device and the controller or switch of the display.

4. The method according to claim 3, wherein, When the controller or the switch detects the GPIO signal at a first DC voltage level, the controller or the switch of the display emits a trigger voltage, wherein the trigger voltage triggers the transition of the component set.

5. The method according to claim 2, wherein, Transforming the set of elements includes: When the GPIO signal is transmitted at the first DC voltage level, the component assembly is aligned in a first orientation, increasing the transmission of light through the component assembly to the camera sensor in the first orientation.

6. The method according to claim 5, wherein, When the GPIO signal is transmitted at a second level different from the first DC voltage level, the component assembly is aligned in a second orientation, which reduces the transmission of light through the component assembly to the camera sensor.

7. The method according to claim 1, wherein, Transmitting the signal includes: An interrupt is generated by the processor of the device in response to the processor recognizing the start of the frame marker.

8. The method of claim 7, comprising: During the initialization of the camera driver, a generated interrupt is registered with the camera driver of the camera, wherein the camera driver is initialized after receiving the request to activate the camera and before the camera is activated.

9. The method according to claim 8, wherein, Transmitting the signal to the display of the device includes: The registered interrupt is sent from the camera driver to the display driver of the monitor.

10. The method according to claim 9, wherein, The display driver receives a registered interrupt, triggering the transition of the component set.

11. The method according to claim 1, wherein, The camera is positioned behind the component assembly, and the field of view of the camera is aligned with the component assembly.

12. The method according to claim 1, wherein, The set of components includes a set of liquid crystal components.

13. An apparatus for activating an in-display camera, comprising: processor, Memory coupled to the processor; as well as Instructions stored in the memory and executable by the processor to cause the device to perform the following operations: Receive a request to activate the camera of the device; In response to the request, identify the start of a frame marker associated with activating the camera; as well as A signal is transmitted to the display of the device in response to the start of the identified frame marker, wherein The transmitted signal triggers the display to switch the display's component assembly from display mode to camera mode for camera operation.

14. The apparatus according to claim 13, wherein, The signal includes general purpose input / output (GPIO) signals, and transmitting the signal includes instructions stored in the memory and executable by the processor to cause the device to perform the following operations: The first DC voltage level is determined at least in part based on the received request to activate the camera of the device; as well as The camera sensor of the device emits the GPIO signal at the first DC voltage level to trigger the transition of the component set.

15. The apparatus according to claim 14, wherein, The GPIO signal is electrically connected between the camera sensor of the device and the controller or switch of the display.

16. The apparatus according to claim 15, wherein, When the controller or the switch detects the GPIO signal at the first DC voltage level, the controller or the switch of the display emits a trigger voltage, wherein the trigger voltage triggers the transition of the component set.

17. The apparatus according to claim 14, wherein, The instructions for transforming the set of elements can be executed by the processor to cause the device to: When the GPIO signal is transmitted at a first DC voltage level, the component assembly is aligned in a first orientation, increasing the transmission of light through the component assembly to the camera sensor in the first orientation.

18. The apparatus according to claim 17, wherein, When the GPIO signal is transmitted at a second level different from the first DC voltage level, the component assembly is aligned in a second orientation, which reduces the transmission of light through the component assembly to the camera sensor.

19. The apparatus according to claim 13, wherein, The instructions for transmitting the signal can be executed by the processor to cause the device to: An interrupt is generated in response to the processor recognizing the start of the frame marker.

20. The apparatus according to claim 13, wherein, The set of components includes a set of liquid crystal components.

21. A device for activating an in-display camera, comprising: Camera; A display, comprising a collection of components; At least one processor; Memory coupled to the at least one processor; as well as Instructions stored in the memory and executable by the at least one processor to cause the camera to perform the following operations: Receive a request to activate the camera; In response to the request, identify the start of a frame marker associated with activating the camera; as well as In response to the start of the identified frame marker, a signal is transmitted to the display of the device, wherein the transmitted signal triggers the display to switch the display's element set from display mode to camera mode for camera operation.

22. The device according to claim 21, wherein, The signal includes general purpose input / output (GPIO) signals, and transmitting the signal includes instructions stored in the memory and executable by the at least one processor to cause the camera to perform the following operations: The first DC voltage level is determined at least in part based on the received request to activate the camera of the device; as well as The camera sensor of the camera emits the GPIO signal at the first DC voltage level to trigger the transition of the component set.

23. The device according to claim 21, wherein, The set of components includes a set of liquid crystal components.

24. An apparatus for activating an in-display camera, comprising components for performing the method according to any one of claims 1 to 12.

25. A computer-readable medium having program code recorded thereon, wherein the program code is executable by one or more processors to cause the processors to perform the method according to any one of claims 1 to 12.

Images