Generate a dynamic virtual mask layer for the cutout area of ​​the display panel

Abstract

Certain aspects of the present disclosure provide methods and apparatus for generating a virtual mask layer around a cutout area for a front camera in a display panel. The front camera can be located below the display panel and receive light through the cutout area. The mask layer can hide defects in the cutout area that does not have functional pixels. The disclosed technology can improve the aesthetic appearance of a display with a punch hole camera having a reduced hole size and an improved production quality pass rate.

Description

Technical Field

[0001] Generally speaking, the present disclosure relates to display panels, and more specifically, the present disclosure relates to one or more techniques for display or graphics processing of display panels. Background Art

[0002] Computing devices often utilize a graphics processing unit (GPU) to accelerate the rendering of graphical data for display. Such computing devices may include, for example, computer workstations, mobile phones such as so-called smartphones, embedded systems, personal computers, tablet computers, and video game consoles. The GPU executes a graphics processing pipeline comprising one or more processing stages that operate together to execute graphics processing commands and output frames. A central processing unit (CPU) can control the operation of the GPU by issuing one or more graphics processing commands to the GPU. Modern CPUs are typically capable of executing multiple applications simultaneously, each of which may require the use of a GPU during execution.

[0003] Portable computing devices including smartphones, tablets, and wearable devices can present graphical content on a display. Often, these devices include a front-facing camera (e.g., a camera facing the user) integrated with the display for video conferencing, video chatting, selfies, and other activities involving the user. Recent market demands have prompted manufacturers to adopt displays with minimal borders to achieve aesthetics. Therefore, the display is usually spanned from edge to edge on the device. Therefore, the front-facing camera must usually be included on the periphery of the display.

[0004] In some cases, the display includes a cutout region that allows the integrated front camera to receive light. The cutout region is typically created using a physical removal process that removes existing material of the display. Due to random errors that occur during the manufacturing process, such a physical removal process does not always produce uniform, consistent, or precise results and has a negative impact on the aesthetics of the display. It should be noted that such a cutout region may be included in the display for purposes other than the front camera, and the display may have more than one cutout region. Summary of the invention

[0005] A brief summary of one or more aspects is given below to provide a basic understanding of such aspects. This summary is not an exhaustive review of all expected aspects, and is neither intended to identify the key elements of all aspects, nor to describe the scope of any or all aspects. Its only purpose is to present some concepts of one or more aspects in a simple form as a prelude to the detailed description below.

[0006] Certain aspects of the present disclosure provide a computing device including a display. The display includes an area without functional pixels. For example, the area can be physically cut out from the display. The computing device also includes a non-volatile memory and a processor. The processor is configured to obtain area information indicating the size, shape, geometry and / or location of the area on the display from the non-volatile memory. The processor can generate a mask layer having a size, shape and location based on the area information, and synthesize an image based on the mask layer and one or more layers. The mask layer can prevent any part of the one or more layers that overlaps with the mask layer from being displayed. The processor can output the synthesized image on the display. Therefore, the mask layer with the area can present the appearance of a perfect cutout area that hides the defects of the physical removal of the area.

[0007] Certain aspects of the present disclosure provide a computing device including a unit for display. The computing device also includes: a unit for obtaining area information indicating the size, geometry and position of the area without functional pixels. The computing device also includes: a unit for generating a first mask layer having a size, shape and position based on the obtained area information. The computing device includes: a unit for synthesizing a first image based on the mask layer and one or more layers. The computing device may include: a unit for generating a second mask layer based on the area information; and a unit for synthesizing a second image based on at least the second mask layer. The computing device may also include: a unit for outputting the first image and the second image on the unit for display.

[0008] Certain aspects of the present disclosure provide a method for operating a computing device as discussed herein. Certain aspects of the present disclosure provide a non-transitory computer-readable medium storing instructions that, when executed by a computing device as discussed herein, cause the computing device to operate as discussed herein.

[0009] The details of one or more examples of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the disclosure will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] A more specific description briefly outlined above may be provided by reference to some of the various aspects shown in the accompanying drawings so that the manner in which the above-mentioned features of the present disclosure are understood in detail. However, it should be noted that the accompanying drawings illustrate only certain typical aspects of the present disclosure and should not be considered to limit the scope of protection of the present invention, as the description recognizes other equally effective aspects.

[0011] Figure 1 is a block diagram illustrating an example computing device configured to dynamically generate a virtual mask layer for a cutout region in accordance with one or more techniques of this disclosure.

[0012] Figure 2 One or more techniques in accordance with the present disclosure illustrate an example display panel implementing techniques for generating a virtual mask layer.

[0013] Figure 3A and Figure 3B Another example display panel implementing techniques for generating a virtual mask layer is shown in accordance with one or more techniques of the present disclosure.

[0014] Figure 4 Example operations for a computing device to generate a virtual mask layer for a cutout area of ​​a display panel are shown according to certain aspects of the present disclosure.

[0015] Figure 5 An example operation of a production system is shown, which provides acquired region information for use in computing device generation. Figure 4 Virtual mask layer.

[0016] Like reference numerals refer to like elements. DETAILED DESCRIPTION

[0017] Generally, aspects disclosed herein provide techniques for improving the visual quality of displays including cutout areas, such as for integrating cameras or other purposes. For example, the disclosed techniques dynamically adapt the display panel to areas without functional pixels and form a mask layer to hide defects in the area.

[0018] Using conventional techniques, display panels typically include a physical cutout area (e.g., a punctured hole), for example, to allow a front-facing camera to receive light. However, such a cutout area typically includes manufacturing errors in shape, size, and / or position. In order to hide such errors, conventional techniques further apply a physical mask layer (e.g., a painted black circle having a larger diameter than the cutout area) so that the cutout area and the mask layer together appear to have a perfectly circular aperture for the front-facing camera. However, even with a larger diameter, the physical mask layer may still not be able to completely cover or hide manufacturing errors, resulting in a less attractive display effect. And such conventional masking techniques produce a black circle that is much larger than the camera lens. The various aspects discussed herein overcome these shortcomings and provide a virtual mask layer having a geometry and / or position that can be adjusted based on each cutout area specific to each display panel to achieve an improved aesthetic appearance.

[0019] In some cases, it is desirable to include an edge-to-edge display panel in a device with the smallest possible impact resulting from including a front-facing camera. Therefore, the perceived aperture size for the front-facing camera is a key performance indicator (KPI) and selling point. Many times, in order to avoid blocking light from passing through the display to the front-facing camera, an aperture (e.g., punching) is created in the display panel using mechanical methods for removing existing and properly functioning pixels. However, in other embodiments, the removal may be limited to certain opaque or translucent backings (e.g., backlights or reflective layers) of the display panel, and clear or transparent pixels may be retained (but still considered non-functional pixels).

[0020] During the production process, the quality of punching holes (or otherwise removing functional pixels from a display panel) in a final releasable product is often limited by three conflicting factors or goals: (1) the conformance rate of the display panel punching holes; (2) the hole diameter and size; and (3) the conformance rate of the display panel assembly and the front-facing camera in the complete device. As the hole diameter decreases (i.e., the accuracy requirement increases), the conformance rate typically decreases (i.e., the rejection rate increases). Certain aspects of the present invention resolve these conflicts and can achieve these goals simultaneously.

[0021] Certain aspects provide a method for dynamically adjusting a virtual mask layer (generally referred to herein as a "mask layer") for each display panel, comprising: creating a slightly larger virtual aperture to hide defects in physically removed portions or areas of the display panel. The disclosed mask layer can reduce device failure rates by accepting manufacturing variability while still achieving aesthetic goals. In some embodiments, the method includes: offline measurement of the physical cutout area of ​​each display panel. The measured cutout area information can then be used to generate a mask layer displayed on the display panel to simulate a traditional physical mask layer. For example, the method can include: applying a virtual mask layer when the device is started, so that the mask layer displays black pixels to display an aesthetically pleasing (and relatively small) shape around the front camera.

[0022] In some examples, certain aspects may use an electron microscope in a production line to measure the shape and position of the cutout area for each display panel. The measurement results may be saved for each display panel, and the measurement results may be used to generate a corresponding mask layer. For example, the mask layer may include information to control the display panel, for example, "1" for the measured cutout area and "0" for the original non-cutout area. As disclosed, the shape, diameter, and position of the mask layer may be adaptively adjusted based on the actual measured cutout area customized for each display panel.

[0023] In some examples, the mask layer applied to the display is a type of frame layer. In some examples, the display processor can perform synthesis (or mixing) of pixel data from different frame layers to generate a display image for presentation via the display. As used herein, a frame layer (sometimes referred to as a "layer") refers to a layer that is combined with one or more other frame layers into a single layer (e.g., a display image). The mask layer can be a transparent layer (e.g., without changing the existing display image), has an opaque mask area and displays the darkest color of the display panel (e.g., black). The mask layer is overlaid on one or more other frame layers so that when combined into a single layer with one or more other frame layers, the mask area appears to be the aperture of the front camera, and presents a virtual cutout shape for hiding manufacturing defects.

[0024] Certain aspects of the present invention, including certain devices and systems, therefore enjoy several technical advantages over conventional masking methods. First, the disclosed techniques enable manufacturers to continually pursue reduced-size cutout areas without sacrificing production yields. Second, the disclosed techniques reduce the rate of device failures caused by manufacturing errors by adaptively adjusting for such errors and achieving similar aesthetic effects. Third, when more complex cutout shapes, sizes, or locations may be introduced in the future for multiple cameras, sensors, or other reasons, the disclosed techniques provide additional flexibility for various display panel layout designs with cutout areas.

[0025] The following is a more comprehensive description of various aspects of the system, device, computer program product and method with reference to the accompanying drawings. However, the present disclosure can be implemented in a variety of different forms and should not be interpreted as being limited to any specific structure or function given throughout the present disclosure. Instead, these aspects are provided to make the present disclosure comprehensive and complete, and the protection scope of the present disclosure will be fully conveyed to those of ordinary skill in the art. Based on the teachings of this article, it should be recognized by those of ordinary skill in the art that the protection scope of the present disclosure is intended to cover any aspect of the system, device, computer program product and method disclosed herein, whether it is implemented independently or in combination with other aspects of the present disclosure. For example, the device can be implemented or the method can be implemented using any number of aspects set forth herein. In addition, the protection scope of the present disclosure is intended to cover such a device or method, which can be practiced by using other structures and functions other than the various aspects of the present disclosure set forth herein, or other structures and functions different from the various aspects of the present disclosure set forth herein. Any aspect disclosed herein can be embodied by one or more elements of the claims.

[0026] Although various aspects are described herein, many variations and arrangements of these aspects also fall within the scope of protection of the present disclosure. Although some potential benefits and advantages of various aspects of the present disclosure are mentioned, the scope of protection of the present disclosure is not intended to be limited to specific benefits, uses or goals. Instead, aspects of the present disclosure are intended to be widely applicable to different wireless technologies, system configurations, networks and transmission protocols, some of which are illustrated by way of example in the drawings and the description below. The specific embodiments and drawings are merely illustrative of the present disclosure and are not intended to be limiting. The scope of protection of the present disclosure is defined by the appended claims and their equivalents.

[0027] Several aspects are now presented with reference to various devices and methods. These devices and methods will be described in the detailed description below and depicted in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). Such elements may be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0028] For example, an element or any part of an element or any combination of elements can be implemented as a "processing system" including one or more processors (which can also be referred to as a processing unit). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on chips (SOCs), baseband processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gate logic, discrete hardware circuits, and other suitable hardware configured to perform various functions described throughout the present disclosure. One or more processors in a processing system can execute software. Software should be broadly interpreted as meaning instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, processes, functions, etc., whether they are referred to as software, firmware, middleware, microcode, hardware description languages, or other terms. The term application can refer to software. As described herein, one or more technologies may refer to an application, i.e., software, configured to perform one or more functions. In such an example, the application may be stored in a memory (e.g., an on-chip memory of a processor, a system memory, or any other memory). The hardware described herein (e.g., a processor) may be configured to execute an application. For example, an application may be described as including code, and when the code is executed by hardware, the hardware is caused to execute one or more technologies described herein. For example, the hardware may access code from a memory, and execute code accessed from the memory to execute one or more technologies described herein. In some examples, components are identified in the present disclosure. In such an example, a component may be hardware, software, or a combination thereof. A component may be a separate component or a subcomponent of a single component.

[0029] Therefore, in one or more examples described herein, the functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, the functions can be stored or encoded as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any available media that a computer can access. By way of example and not limitation, such computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, a combination of computer-readable media of the aforementioned types, or any other media that can be used to store computer executable code in the form of instructions or data structures and can be accessed by a computer.

[0030] In general, examples disclosed herein provide techniques for generating a virtual mask layer (e.g., for a front-facing camera) around a cutout area. For example, a front-facing camera can be located below a display and receive light through the cutout area. The mask layer can hide defects in areas without functional pixels. The disclosed techniques can improve the aesthetic appearance of displays with reduced-size cutout areas, as well as improve production quality pass rates. For example, certain aspects provide techniques for performing graphics and / or display processing in any device that utilizes a display having an area without functional pixels. Other example benefits are described throughout this disclosure.

[0031] As used herein, instances of the term "content" may refer to "graphics content," "images," and vice versa. This is true regardless of whether these terms are used as adjectives, nouns, or other parts of speech. In some examples, as used herein, the term "graphics content" may refer to content produced by one or more processes of a graphics processing pipeline. In some examples, as used herein, the term "graphics content" may refer to content produced by a processing unit configured to perform graphics processing. In some examples, as used herein, the term "graphics content" may refer to content produced by a graphics processing unit.

[0032] In some examples, as used herein, the term "display content" may refer to content generated by a processing unit configured to perform display processing. In some examples, as used herein, the term "display content" may refer to content generated by a display processing unit. Graphics content may be processed to become display content. For example, a graphics processing unit may output graphics content such as a frame to a buffer (which may be referred to as a frame buffer). The display processing unit may read graphics content (e.g., one or more frames) from the buffer, and perform one or more display processing techniques thereon to generate display content. For example, the display processing unit may be configured to perform synthesis on one or more rendering layers to generate a frame. For another example, the display processing unit may be configured to synthesize, fuse, or otherwise combine two or more layers into a single frame. The display processing unit may be configured to perform scaling on a frame, such as zooming in or out. In some examples, a frame may refer to a layer. In other examples, a frame may refer to two or more layers that have been fused together to form a frame, that is, a frame includes two or more layers, and a frame including two or more layers may be subsequently fused.

[0033] Figure 11 is a block diagram of an example system 100 including an example computing device 104 configured to dynamically generate a virtual mask layer for a cutout region according to one or more techniques of the present disclosure. The computing device 104 may include one or more components or circuits for performing the various functions described herein. In some examples, one or more components of the computing device 104 may be components of a SOC. The computing device 104 may include one or more components configured to perform one or more techniques of the present disclosure. In the example shown, the computing device 104 may include a processor 120 and a system memory 124.

[0034] In some examples, computing device 104 may include multiple additional or alternative components (e.g., communication interface 126, transceiver 132, receiver 128, transmitter 130, display processor 127, and display client 131). References to display client 131 may refer to one or more displays. For example, display client 131 may include a single display or multiple displays. Display client 131 may include a first display and a second display, or a foldable or detachable display. In other examples, the results of graphics processing may not be displayed on the device, for example, the first display and the second display may not receive any frames presented thereon. Instead, the frames or graphics processing results may be transmitted to another device. In some aspects, this may be referred to as split rendering.

[0035] The processor 120 may include an internal memory 121. The processor 120 may be configured to perform graphics processing, for example, in the graphics processing pipeline 107. In some examples, the computing device 104 may include a display processor or display processing unit (e.g., a display processor 127) to perform one or more display processing techniques on one or more frames generated by the processor 120 before the display client 131 presents the frames. The display processor 127 may be configured to perform display processing. For example, the display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by the processor 120. The display processor 127 may output image data to the display client 131 according to an interface protocol such as, for example, MIPI DSI (Mobile Industry Processor Interface, Display Serial Interface).

[0036] Display client 131 may be configured to display or otherwise present frames processed by display processor 127. In some examples, display client 131 may include one or more of a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection display device, an augmented reality display device, a virtual reality display device, a head mounted display, or any other type of display device.

[0037] exist Figure 1 In the example shown, the display client 131 includes a display controller 132, a buffer 134, and a display 136. The example display 136 includes a plurality of pixel elements for displaying image data. The display controller 132 can receive image data from the display processor 127 and store the received image data in the buffer 134. In some examples, the display controller 132 can output the image data stored in the buffer 134 to the display 136. Therefore, the buffer 134 can represent a local memory of the display client 131. In some examples, the display controller 132 can output the image data received from the display processor 127 to the display 136.

[0038] In some aspects, display 136 includes an area without functional pixels. For example, the area of ​​display 136 can be physically cut out of display 136. Figure 2 , where a display panel 240 includes a cutout area 205 that allows light to reach the front camera 210. The display panel 240 belongs to an example smartphone 215 of the computing system 200. In some cases, for example Figure 2 As shown in , when the cutout area 205 is within the design tolerance (e.g., within a three pixel variation from the design specification (e.g., in terms of size, shape, and position)), the display panel 240 can have a mask layer that completely overlaps the cutout area 205. However, in other cases, the cutout area is different from the design specification, and the disclosed mask layer is adaptively adjusted for such variation, as described below in Figure 3A and Figure 3B Further described in .

[0039] In addition, as described above, the display client 131 can be configured according to the MIPI DSI standard. The MIPI DSI standard supports a video mode and a command mode. In an example where the display client 131 operates in a video mode, the display processor 127 can continuously refresh the graphics content of the display client 131. For example, the entire graphics content can be refreshed every refresh cycle (e.g., line by line).

[0040] In examples where display client 131 operates in command mode, display processor 127 may write graphics content of a frame to buffer 134. In some such examples, display processor 127 may not continuously refresh the graphics content of display client 131. Instead, display processor 127 may use vertical synchronization (Vsync) pulses to coordinate the rendering and consumption of graphics content at buffer 134. For example, when a Vsync pulse is generated, display processor 127 may output new graphics content to buffer 134. Thus, the generation of a Vsync pulse may indicate when the current graphics content at buffer 134 has been rendered.

[0041] Memory external to processor 120 (e.g., system memory 124) is accessible to processor 120. For example, processor 120 may be configured to read from, and / or write to, external memory (e.g., system memory 124). Processor 120 may be communicatively coupled to system memory 124 via a bus. In some examples, processor 120 and system memory 124 may be communicatively coupled to each other via a bus or different connections.

[0042] It should be appreciated that in some examples, computing device 104 may include a content encoder / decoder configured to receive graphics and / or display content from any source (e.g., system memory 124 and / or communication interface 126). System memory 124 may be configured to store the received encoded or decoded content. In some examples, the content encoder / decoder may be configured to receive the encoded or decoded content in the form of encoded pixel data, for example, from system memory 124 and / or communication interface 126. In some examples, the content encoder / decoder may be configured to encode or decode any content.

[0043] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, SRAM, DRAM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic data media or optical storage media, or any other type of memory. In some aspects, internal memory 121 may also include region information 140 indicating the size of a region without functional pixels and the location of the region on the display.

[0044] According to some examples, the internal memory 121 or the system memory 124 may be a non-transitory storage medium. The term "non-transitory" may refer to a storage medium that is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be interpreted to mean that the internal memory 121 or the system memory 124 is non-removable or that its contents are static. As an example, the system memory 124 may be removed from the computing device 104 and moved to another device. As another example, the system memory 124 may not be removable from the computing device 104.

[0045] Processor 120 may be a central processing unit (CPU), a graphics processing unit (GPU), a general purpose GPU (GPGPU), or any other processing unit that may be configured to perform graphics processing. In some examples, processor 120 may be integrated into the motherboard of computing device 104. In some examples, processor 120 may be present on a graphics card installed in a port of the motherboard of computing device 104, or may be otherwise incorporated in a peripheral device configured to interoperate with computing device 104. Processor 120 may include one or more processors, for example, one or more microprocessors, GPUs, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), discrete logic, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, processor 120 may store instructions for the software in an appropriate, non-transitory computer-readable storage medium (e.g., internal memory 121), and may use one or more processors to execute instructions in hardware to perform the technology of the present disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. In some aspects, processor 120 may include display processor 127 or may be integrated with display processor 127.

[0046] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any receiving functions described herein with respect to the computing device 104. In addition, the receiver 128 may be configured to receive information (e.g., eye or head position information, rendering commands, or position information) from another device. The transmitter 130 may be configured to perform any transmitting functions described herein with respect to the computing device 104. For example, the transmitter 130 may be configured to send information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined into a transceiver 132. In such an example, the transceiver 132 may be configured to perform any receiving functions and / or transmitting functions described herein with respect to the computing device 104.

[0047] In some examples, the graphical content from processor 120 for display via display client 131 is not static and may be changing. Therefore, display processor 127 may periodically refresh the graphical content displayed via display client 131. For example, display processor 127 may periodically retrieve graphical content from system memory 124, where the graphical content may have been updated by the execution of an application (and / or processor 120) that outputs the graphical content to system memory 124.

[0048] It should be recognized that although Figure 1 1 and 127. Although shown as a separate component in FIG. 1 , in some examples, display client 131 (sometimes referred to as a “display panel”) may include display processor 127.

[0049] like Figure 1 As shown in , in some aspects, the display processor 127 (also referred to as a display processing unit (DPU)) can be configured to operate the functions of the display client 131. For example, in some aspects, the display processor 127 is configured to output a plurality of codewords (e.g., corresponding to frames) to the display controller 132. Each codeword can be represented by a binary number in the digital domain. Each codeword can correspond to a pixel of the display 136 (e.g., red, green, blue, white, etc.).

[0050] The display controller 132 can be configured to convert the codewords received from the display processor 127 into analog signals for driving the pixels of the display 136. In some aspects, for each codeword corresponding to a pixel, the display controller 132 is configured to convert the codeword into an analog signal that drives the pixel to a particular brightness level. Thus, in some aspects, the codeword and / or analog signal corresponds to the brightness level of the pixel.

[0051] In some aspects, the processor 120, the display processor 127, or both can be configured to generate a mask layer on the display 136 to hide manufacturing defects. The following description uses the processor 120 as an example, and similar techniques, methods, or processes performed by the processor 120 can also be performed in the display processor 127, or in combination therewith.

[0052] In some aspects, processor 120 is configured to retrieve region information 140 from a non-volatile memory, such as internal memory 121 or system memory 124. Region information 140 may indicate a cutout region in display 136 (such as, for example, Figure 2 The cutout area 205 of the display panel 240 or Figure 3A and Figure 3B In some aspects, the computing device 104 may also include a camera located below the area in the display 136 (e.g., Figure 2 Camera 210 or Figure 3A and Figure 3B In some aspects, the processor 120 is configured to receive the region information 140 from the second computing device and store the region information 140 in the non-volatile internal memory 121. In some other aspects, the region information 140 can be specific to the computing device 104. For example, the region information 140 can describe the physical removal of material in the display 136 and the front-facing camera ( Figure 1 The positioning of the device is not shown).

[0053] The region information may indicate the size and location of the region on the display 136, or may include other information that fully describes the geometry or condition of the region on the display 136 (e.g., burrs, cracks, etc.). For example, the region information may include the size of the region and the location of the region. In some aspects, the region information may include a mask layer based on the size of the region and the location of the region.

[0054] The processor 120 is configured to generate a mask layer having a size, shape, and position based on the retrieved regional information. For example, the processor 120 may be configured to generate a mask layer surrounding an area without functional pixels. In some cases, the area may be a circle meant to be punched to accommodate a front camera, and may not have a circular edge due to tool wear. The generated mask layer may have the same circle, but a slightly larger diameter to hide edge defects in the area. In other cases, the area may deviate from the direction due to a slight misalignment between the display panel and the processing tool (e.g., a punch, a milling machine, a laser engraver, etc.). The generated mask layer can be adjusted to the same offset direction, as well as visually hide the misalignment. Other settings for generating a mask layer may be provided. For example, the mask layer may have a shape different from the shape of the area (e.g., a square that is resized to completely cover an area of ​​a circular shape).

[0055] In some aspects, the processor 120 can synthesize an image based on a mask layer and one or more layers. The mask layer is configured to prevent any portion of the one or more layers from being displayed that overlaps with the mask layer. The one or more layers can be rendered by a graphics processing unit (e.g., graphics processing pipeline 107) to generate a frame. The processor 120 can then output the image on the display 136.

[0056] In some aspects, the processor 120 may be configured to generate a second mask layer based on the region information. The processor 120 may synthesize a second image based at least on the second mask layer. The processor 120 may output the second image on the display 136.

[0057] In some aspects, the mask layer generated by processor 120 can be dynamically adjusted for the cutout area that is physically removed from display 136. Figure 2 As shown in , in an ideal situation, region 205 of display panel 240 can have the exact shape, size, and position as desired, and thus overlap with the mask layer. For example, when region 205 is within a difference of three pixels (or any other specified error tolerance) compared to the target design specification, processor 120 can determine that a default mask layer that overlaps region 205 without adjustment can be used. In a more practical situation, for example, Figure 3A and Figure 3B As described in the example above, various manufacturing errors may occur that result in defects in the shape, size, and / or position of region 305, which is intended to share a center with camera 310, have a circular shape, and be slightly larger than camera 310. For example, during the manufacturing process, the size and shape of region 305 may change as the tool wears. The position of region 305 may be affected by random errors that occur when placing each display panel under the tool.

[0058] like Figure 3A and 3B As shown in , the region 305 has different positions and sizes. The processor 120 can receive the region information 140 indicating the size and position of the region 305. For example, when the display panel 240 turns on all functional pixels 325, a microscope (not shown) can be used to measure the region 305. The microscope can record the non-functional pixels of the region 305 and determine the key geometric parameters of the region 305. The key geometric parameters may include one or more of the shape, size, deviation, position, center and / or other values ​​or information required by the processor 120 to generate the mask layer 315. In some aspects, these key geometric parameters, together with any post-processing results (e.g., area, size, etc.), can form the region information 140. In some aspects, the region information 140 can be directly stored in the internal memory 121 when measuring, or subsequently transferred from the microscope (or other measurement device) to the internal memory 121. In some aspects, the key geometric parameters can be used by another device to generate a mask layer (e.g., mask layer 315), and the region information 140 can include the mask layer itself.

[0059] The processor 120 or another appropriate device may be configured to generate a mask layer 315 based on the measured area information of the area 305. For example, the mask layer 315 may be required to have the smallest possible area to completely cover the area 305 to achieve the smallest overall size on the display panel 240. In other cases, the mask layer 315 may be required to be concentric with the camera 310, and the position of the camera 310 may be measured or assumed to be a constant position relative to the edge of the display panel. In some other cases, the mask layer 315 may be required to have a shape that is symmetrical around the center of the camera 310. In some other cases, the mask layer 315 may be required to have smooth edges as defined by a rounding radius threshold. In some other cases, the mask layer 315 may be required to adjust its size, shape, and position based on user or application specific criteria and based on the area information 140.

[0060] like Figure 3A As shown in FIG. 1 , mask layer 315 is concentric with camera 310, completely surrounds region 305, and forms a circle with smooth edges. Functional pixels of region 320 within region 305 can be powered off during operation to display black or a dark color, thereby simulating a conventional mask layer. Processor 120 is configured to synthesize an image based on mask layer 315 and other layers associated with the operation of computing device 104, such that mask layer 315 prevents pixels in region 320 from displaying other layers. Processor 120 is configured to perform a specific operation on a specific region 320. Figure 3AThe mask layer 315 of the display panel 240 is used to synthesize the image, and the image is saved in the computing device 104.

[0061] Similarly, if Figure 3B As shown in FIG. , although region 305 has Figure 3A The mask layer 315 may be a different size and position than the region 305 of FIG. 1 , but the mask layer 315 that meets the same set of requirements regarding size, shape, and position still completely covers the region 305. Both the black space in the region 305 and the unpowered pixels in the region 320 appear black in appearance. Thus, the mask layer 315 successfully hides errors and defects in the manufacturing process. In some aspects, the processor 120 is configured to perform a specific Figure 3B The mask layer 315 of the display panel 240 synthesizes different images, but achieves the same Figure 3A In some embodiments not shown, region 305 may require adjustment of the size of mask layer 315, such as when region 305 is further offset from camera 310 and an increased mask layer size is required to completely cover region 305. Similarly, processor 120 may generate mask layer 315 that appears to always be in contact with region 305 to minimize the size of mask layer 315.

[0062] As described herein, a device such as computing device 104 may refer to any device, apparatus, or system configured to perform one or more of the techniques described herein. For example, a device may be a server, a base station, a user device, a client device, a station, an access point, a computer (e.g., a personal computer, a desktop computer, a laptop computer, a tablet computer, a computer workstation, or a mainframe computer), a terminal product, an apparatus, a phone, a smart phone, a server, a video game platform or console, a handheld device (e.g., a portable video game device or a personal digital assistant (PDA)), a wearable computing device (e.g., a smart watch, an augmented reality device, or a virtual reality device), a non-wearable device, a display or display device, a television, a television set-top box, an intermediate network device, a digital media player, a video streaming device, a content streaming device, an onboard computer, any mobile device, any device configured to generate graphic content, or any device configured to perform one or more of the techniques described herein. The process herein may be described as being performed by a specific component (e.g., a GPU), but in another embodiment, other components (e.g., a CPU) consistent with the disclosed embodiments may be used to perform.

[0063] Figure 4According to certain aspects of the present disclosure, example operations 400 are shown for a computing device to generate a virtual mask layer for a cutout area of ​​a display panel. Example operations 400 may be performed on a computing device or a computing system. In some embodiments, the computing device includes an integrated display; the computing system may include an external display. Example operations 400 may be performed by various components forming a computing device or a computing system, even though such various components may be remotely located. Figure 4 As shown in , at 405, region information can be obtained. The region information can indicate the size of the region of the display panel. The region may not have functional pixels. In some cases, the region can be physically cut out of the display panel. The region information can be obtained from local memory, or can be transmitted from a storage space outside the computing device or computing system (e.g., from a server, cloud or other network system). In some embodiments, the region information includes the size of the region and the location of the region.

[0064] At 410, a mask layer is generated, for example by a processor or processing unit of a computing device or computing system. The mask layer may include a size, shape, and position based on the obtained area information. For example, the mask layer may be as small as possible while being sufficient to cover the cutout area in the display panel. In some embodiments, the mask layer may be generated according to certain programmed or user-input preferences, rules, or algorithms. For example, the mask layer may be required to be concentric with the front camera below the display panel and pass through the cutout area, have a symmetrical shape, and a predefined fillet radius, etc. These preferences, rules, or algorithms can ensure that the generated mask layer provides an aesthetically pleasing appearance on the display panel and hides manufacturing defects in the area.

[0065] At 415, an image is synthesized based on the mask layer and the one or more layers. The mask layer prevents any portion of the one or more layers from being displayed that overlaps with the mask layer. In some embodiments, the mask layer includes functional pixels that continuously display black (or the darkest setting) on ​​the display panel to simulate a traditional mask layer implemented using black paint or the like. The synthesized image can display black, pure black, a shade, or the color of the portion of the one or more layers that overlaps with the mask layer. At 420, the synthesized image is output on the display panel.

[0066] In some embodiments, optionally, at 425, a second mask layer can be generated based on the region information. For example, the second mask layer can be suitable for different software applications, environments (e.g., temperature and humidity), or lighting conditions (e.g., based on measurements of the region under different lighting conditions). Optionally, at 430, a second image can be synthesized based on the second mask layer. The second synthesized image can be based on the second mask layer and one or more other layers. The second mask layer prevents display of any portion of the other one or more layers that overlaps with the second mask layer. Optionally, at 435, the second image can be output to a display.

[0067] Figure 5 Example operations 500 for a production system are shown that provide acquired region information to a computing device to generate Figure 4 For example, the example operation 500 may be performed in a production line of a display panel or a computing device including a display panel. At 505, area information of a physically removed area of ​​a display is obtained. The physically removed area of ​​a display (or display panel) may be a perforation of a material forming the display, or in applicable embodiments, it forms a backlight of the display. A microscope, scanner, or high-resolution optical system operable to measure the sub-pixel size of a display panel may be used to measure the cutout area.

[0068] In some embodiments, the local high-resolution measurement results of the cutout area can be combined with the position of the area and the overall size of the display panel measured by the overall positioning system, which can accurately measure larger overall sizes but may not be able to measure at the sub-pixel resolution of the cutout area. In other embodiments, the optical system can be configured to obtain the overall size and position of the area, as well as the sub-pixel information of the area. For example, the display panel can be powered on and lit during the measurement to contrast the area without functional pixels. Such a comparison enables a microscope, scanner or other optical system to record the shape, size and position of the cutout area into the area information.

[0069] At 510, it is determined whether the acquired area is within the tolerance of the design specification. For example, the production line may define an error tolerance corresponding to the qualified rate. When the measured cutout area is within the error tolerance of the design specification (such as, for example, within a change of three pixels), then at 515, the default area information may be saved. For example, the default area information may be used in an ideal situation, such as Figure 2 The situation shown, where the mask layer completely overlaps the region. After saving the default region information, operation 500 continues with the next display panel to measure its region.

[0070] When the measured area is not within the error tolerance of the design specification (for example, having different sizes, positions, or both), the acquired regional information is stored separately for the display panel being measured. For example, the disclosed technology can record regional information for each display panel to record device-specific data for device-specific adjustments. In some embodiments, the acquired regional information is stored at a server and then transmitted to an independent computing device that matches the corresponding regional information. In other embodiments, the acquired regional information is directly stored at an independent computing device where the display panel is installed. After storing the acquired regional information, operation 500 can continue with the next display panel and continue the measurement process.

[0071] In certain aspects of operation 500, different methods may be used to obtain the actual hole punch shape and location for each computing device. A typical method may include powering on the full screen for maximum brightness and capturing the display using a high-resolution camera.

[0072] In certain aspects of operation 500, the server can use the captured image of the display as input. The server can generate a mask with the exact hole shape and transmit the mask to a computing device (such as, for example, a smart phone, a tablet device, or a laptop computer). In some embodiments, the server can further perform post-processing and generate device-specific diameters and locations for each measured area so as to perfectly cover the actual area. The server can then transmit an image of the mask layer to the computing device.

[0073] In one configuration, a method or apparatus for display processing is provided. The apparatus may be a processing unit, a display processor, a display processing unit (DPU), a graphics processing unit (GPU), a video processor, or some other processor that can perform display processing. In some examples, the apparatus may be a processor 120 within a computing device 104, or may be some other hardware within the computing device 104, or another device.

[0074] According to the present disclosure, the term "or" may be interpreted as "and / or" in the absence of a context that dictates otherwise. Furthermore, while phrases such as "one or more" or "at least one" and the like may have been used for some features disclosed herein but not for other features, in the absence of a context that dictates otherwise, features that do not use such language may be interpreted as having such meanings implied.

[0075] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any function, processing unit, technology described herein, or other module is implemented in software, such function, processing unit, technology described herein, or other module may be stored on a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media may include computer data storage media or communication media, including any media that facilitates the transfer of a computer program from one place to another. In this manner, a computer-readable medium may generally correspond to: (1) a non-transitory tangible computer-readable storage medium; or (2) a communication medium such as a signal or carrier waveform. A data storage medium may be any available medium that one or more computers or one or more processors can access to obtain instructions, codes, and / or data structures for implementing the technology described in this disclosure. By way of example, but not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks, where disks typically reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included within the scope of protection of computer-readable media. A computer program product may include a computer-readable medium.

[0076] The code may be executed by one or more processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), arithmetic logic units (ALUs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, as used herein, the term "processor" may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0077] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Rather, as described above, the various units may be combined in any hardware unit or provided by a collection of cooperating hardware units (which include one or more processors as described above) in combination with appropriate software and / or firmware.

[0078] Various examples have been described. These and other examples are within the scope of protection of the following claims.

Claims

1. A computing device, comprising: A display including a cutout area having no functional pixels; Non-volatile memory; as well as A processor configured to: Retrieving from the non-volatile memory region information indicating a size of the cutout region and a location of the cutout region on the display, wherein the region information is specific to the computing device; generating a mask layer around the cutout area having a size, shape and position based on the area information; compositing an image based on the mask layer and one or more layers, the mask layer preventing display of any portions of the one or more layers that overlap with the mask layer; and The image is output on the display.

2. The computing device of claim 1, wherein: The area information includes the size of the cutout area and the position of the cutout area.

3. The computing device of claim 1, wherein: The region information includes a mask based on the size of the cutout region and the position of the cutout region.

4. The computing device of claim 1, wherein: The processor is further configured to: generating a second mask layer based on the region information; synthesizing a second image based at least on the second mask layer; as well as The second image is output on the display.

5. The computing device of claim 1 , further comprising: A camera is located below the cutout area of ​​the display.

6. The computing device of claim 1, wherein: The processor is further configured to receive the region information from a second computing device and store the region information in the non-volatile memory.

7. The computing device of claim 1, wherein: The processor includes a display processing unit.

8. The computing device of claim 1, wherein: The mask layer completely overlaps the cutout area.

9. The computing device of claim 1, wherein: The cutout area is physically cut out from the display.

10. The computing device of claim 9, wherein: The processor is further configured to adjust the mask layer based on the cutout area.

11. A method of display processing at a computing device, the computing device comprising: A display including a cutout area without functional pixels, a non-volatile memory, and a processor, the method comprising: Retrieving from the non-volatile memory region information indicating a size of the cutout region and a location of the cutout region on the display, wherein the region information is specific to the computing device; generating a mask layer around the cutout area having a size, shape and position based on the area information; compositing an image based on the mask layer and one or more layers, the mask layer preventing display of any portions of the one or more layers that overlap with the mask layer; and The image is output on the display.

12. The method according to claim 11, wherein: The area information includes the size of the cutout area and the position of the cutout area.

13. The method according to claim 11, wherein: The region information includes a mask based on the size of the cutout region and the position of the cutout region.

14. The method according to claim 11, further comprising: generating a second mask layer based on the region information; synthesizing a second image based at least on the second mask layer; as well as The second image is output on the display.

15. The method according to claim 11, further comprising: The region information is received from a second computing device and the region information is stored in the non-volatile memory.

16. The method according to claim 11, wherein: The mask layer completely overlaps the cutout area.

17. The method according to claim 11, wherein: The cutout area is physically cut out from the display.

18. The method according to claim 17, further comprising: The mask layer is adjusted based on the cutout area.

Images