Fingerprint spot display method and related device

Abstract

This application relates to the field of image processing technology, providing a fingerprint spot display method and related equipment. The fingerprint spot display method is applied to an electronic device, which includes a display driver chip and a display screen. The method includes: receiving a touch operation on the display screen; determining image data of a target interface based on the touch operation; and the display driver chip controlling the display screen to display the target interface in a highlight mode based on the image data, including a mask layer. This effectively avoids screen flickering or blackout phenomena that may occur during fingerprint recognition, improving fingerprint recognition speed.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a fingerprint spot display method and related equipment. Background Technology

[0002] Fingerprint recognition technology is widely used in electronic devices. For example, optical fingerprint recognition technology displays a fingerprint spot of a specific shape and brightness on the fingerprint recognition area of ​​the screen. When a user presses their finger on the fingerprint spot, the electronic device can collect the user's fingerprint information, enabling functions such as fingerprint unlocking, fingerprint payment, and fingerprint photography. However, during the fingerprint recognition process, screen flickering or blackouts may occur due to timing issues. Summary of the Invention

[0003] This application provides a fingerprint spot display method and related device, which can effectively avoid possible screen flickering or blackout phenomena during fingerprint recognition and improve fingerprint recognition speed.

[0004] A first aspect provides a fingerprint spot display method applied to an electronic device, the electronic device including a display driver chip and a display screen, the method including: receiving a touch operation on the display screen; determining image data of a target interface based on the touch operation; and the display driver chip controlling the display screen to display the target interface in a highlight mode based on the image data including a mask layer.

[0005] In this embodiment, the display driver chip determines whether the received image data includes a mask layer. If a mask layer is included, the display screen is controlled to display the target interface in a highlight mode. That is, when displaying the target interface frame image, the display driver chip enables the mask image and the highlight mode to take effect simultaneously in the frame image, effectively avoiding possible screen flickering or blackout phenomena during fingerprint recognition and improving fingerprint recognition speed.

[0006] In one possible implementation, the method further includes: generating a first mark based on image data including a mask layer; and controlling the display screen to display the target interface in a highlight mode based on the image data including the mask layer, including: the display driver chip acquiring the first mark and image data, and controlling the display screen to display the target interface in a highlight mode based on the first mark.

[0007] By generating a first marker, the first marker and the mask layer can be simultaneously transmitted to the display driver chip. Based on the recognition of the first marker, the display driver chip can control the display screen to display the target interface in a high-brightness mode, thereby effectively avoiding screen flickering or blackout phenomena that may occur during fingerprint recognition and improving fingerprint recognition speed.

[0008] In one possible implementation, generating the first mark based on the image data including the mask layer includes: adding the first mark on the mask layer based on the image data including the mask layer.

[0009] By adding a first marker to the mask layer, the first marker is transmitted to the display driver chip along with the mask layer. The display driver chip can then control the display screen to display the target interface in a highlight mode based on the recognition of the first marker, thereby effectively avoiding screen flickering or blackout phenomena that may occur during fingerprint recognition and improving fingerprint recognition speed.

[0010] In one possible implementation, the electronic device further includes a display driver, and the image data of the target interface further includes a spot layer, wherein the spot layer is used to display the fingerprint spot area of ​​the target interface, and the mask layer is used to display the mask area of ​​the target interface other than the fingerprint spot area; the display driver transmits the image data and the first mark to the display driver chip.

[0011] The image data and the first mark are synchronously transmitted to the display driver chip through the display driver. The display driver chip can then control the display screen to display the target interface in a high-brightness mode based on the recognition of the first mark, thereby effectively avoiding screen flickering or blackout phenomena that may occur during fingerprint recognition and improving fingerprint recognition speed.

[0012] In one possible implementation, the display driver chip includes a communication module, a tag recognition module, a gamma module, and an image processing module; the display driver chip acquires a first tag and image data, and controls the display screen to display the target interface in a high-brightness mode based on the first tag, including: the communication module acquires the first tag and image data; the communication module transmits the first tag to the tag recognition module; the tag recognition module controls the gamma module to switch to high-brightness mode based on the first tag, and controls the image processing module to display the target interface according to the image data.

[0013] By setting a marker recognition module within the display driver chip, the marker recognition module identifies the first marker and controls the display screen to display the target interface in a high-brightness mode, thereby effectively avoiding screen flickering or blackout phenomena that may occur during fingerprint recognition and improving fingerprint recognition speed.

[0014] In one possible implementation, the method further includes: generating a second marker based on image data excluding the mask layer; and using the second marker, the marker recognition module controls the image processing module to display the target interface according to the image data.

[0015] A second aspect provides an electronic device, comprising: a processor and a memory; the memory for storing a computer program, the computer program including program instructions; and the processor for invoking the computer program to perform any of the methods described above.

[0016] The third aspect provides a chip system applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform any of the methods described above.

[0017] The fourth aspect provides a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform any of the methods described herein.

[0018] The fifth aspect provides a computer program product that stores at least one instruction, which, when executed by a processor, implements any of the methods described above.

[0019] The technical effects achieved by the second, third, fourth, and fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect, and will not be repeated here. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application.

[0021] Figure 2 This is an example of an optical under-display fingerprint acquisition method provided in this application.

[0022] Figure 3 This is a schematic diagram of fingerprint recognition provided in an embodiment of this application.

[0023] Figure 4 This is a schematic diagram of the hard spot scheme provided in an embodiment of this application.

[0024] Figure 5 This is a schematic diagram of a soft spot scheme provided in an embodiment of this application.

[0025] Figures 6A to 6C This is a schematic diagram of a splash screen provided for an embodiment of this application.

[0026] Figures 7A to 7C This is a schematic diagram of flashing black provided for an embodiment of this application.

[0027] Figure 8 This is a schematic flowchart of a fingerprint spot display method provided in an embodiment of this application.

[0028] Figure 9 This is a schematic flowchart of another fingerprint spot display method provided in an embodiment of this application.

[0029] Figures 10A to 10D This is a schematic diagram of a fingerprint spot display provided in an embodiment of this application.

[0030] Figure 11This is a schematic flowchart of another fingerprint spot display method provided in an embodiment of this application.

[0031] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] In this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. "At least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and drawings of this application are used to distinguish similar objects and not to describe a specific order or sequence. Words such as "exemplary" or "for example" are used to indicate illustrative, exemplary, or descriptive purposes. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0033] In this embodiment, the operating system of the electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. The software structure of the electronic device will be described below using a layered architecture as an example.

[0034] Layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example... Figure 1 As shown, the Android system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, hardware abstraction layer, and kernel layer.

[0035] The application layer can include a series of application packages. Application packages can include applications such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0036] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0037] The application framework layer can include fingerprint services, window managers, content providers, view systems, phone managers, resource managers, notification managers, etc. The application framework layer can provide fingerprint recognition functionality-related APIs for fingerprint recognition applications in the application layer, and provide fingerprint services to implement fingerprint recognition functionality.

[0038] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0039] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0040] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0041] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0042] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0043] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0044] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0045] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0046] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0047] System libraries can include multiple functional modules. For example: graphics engines, surface synthesizers (SurfaceFlinger), media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0048] The graphics engine is responsible for drawing graphics.

[0049] SurfaceFlinger is responsible for combining different surfaces into a single framebuffer, and then displaying the contents of this framebuffer on the screen.

[0050] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0051] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The hardware abstraction layer is the layer between the system library and the kernel layer. The hardware abstraction layer can include the fingerprint (FP) abstraction layer and the hardware composer (HWC).

[0052] The fingerprint abstraction layer is used to report fingerprint events to the fingerprint service in the application framework layer.

[0053] HWC is a driver abstraction layer for a dedicated chip for layer compositing. For example, in the embodiments of this application, HWC is used to connect SurfaceFlinger and the display driver. That is, HWC is a communication bridge between SurfaceFlinger and the display driver, so that the layers composed by SurfaceFlinger can be transmitted to the display driver for display.

[0054] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, touch drivers, fingerprint drivers, audio drivers, and sensor drivers.

[0055] In some embodiments, the electronic device may further include a display driver chip, a touch sensor, and a display screen.

[0056] The display screen is used to display images, videos, etc. The display screen may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (ALED), a flexible light-emitting diode, a MiniLED, a MicroLED, a Micro-OLED, a quantum dot LED, etc. In some embodiments, the electronic device may include one or more display screens.

[0057] In addition to the display panel, a display screen may also include a corresponding display driver chip. In some implementations, one display screen may be configured with one display driver chip. In the case of multiple display screens, the electronic device may include multiple display driver chips.

[0058] The display driver chip sends drive signals and data to the display panel, controlling screen brightness and color to display images, letters, and other graphic content. The touch sensor detects user touch operations on the display screen.

[0059] A touch sensor, also known as a "touch device," is a device that can be placed on a display screen. A touch sensor and a display screen together form a touchscreen, also called a "touchscreen." The touch sensor detects touch operations applied to or near it. It then transmits the detected touch operation to an application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen. In some embodiments, the touch sensor may also be located on the surface of the electronic device, in a different position than the display screen.

[0060] Please see Figure 2 This document provides an example of the principle behind optical under-display fingerprint collection. The screen of an electronic device may include: a cover glass, an organic light-emitting diode (OLED) module, and an optical fingerprint module. The cover glass may also be referred to as a CoverGlass or CG cover. The optical fingerprint module may include, but is not limited to, the following sensors: Complementary Metal-Oxide-Semiconductor (CMOS) and Thin Film Transistor (TFT).

[0061] When a user places their finger on a specific area of ​​the screen, such as the fingerprint recognition area, the organic light-emitting diode module emits output light (e.g., ...). Figure 2(As shown by the solid line) Illuminates the fingerprint recognition area of ​​the screen through the covering glass. This output light is reflected by the fingerprint recognition area to obtain reflected light (such as...). Figure 2 (As shown by the dashed line), reflected light passes through the gap between the cover glass and the OLED module and returns to the sensor of the optical fingerprint module. Due to the unevenness of a fingerprint, the output light undergoes varying degrees of refraction and reflection. The light reflected back to the optical fingerprint module has different brightness levels depending on the angle of refraction; the ridge areas reflect darker light, while the valley areas reflect brighter light. The optical fingerprint module receives the reflected light and generates a multi-grayscale fingerprint image based on the differences in brightness, where ridges are presented as black and valleys as white.

[0062] Please refer to the following: Figure 3 Optical fingerprint technology converts the reflected light from a finger into a fingerprint image that can be processed by an electronic device. The sensor in the optical fingerprint module collects the reflected light from the user's finger, generating a raw image (i.e., a multi-grayscale fingerprint image). The electronic device processes the raw image to obtain a processed image. This image processing may include, but is not limited to, normalization, segmentation, enhancement, binarization, and thinning. Feature extraction is performed on the processed image to obtain feature information. The electronic device performs feature matching, comparing the feature information with fingerprint templates stored in a database to determine if a match is found. If the imaging result is good, the fingerprint template can be updated to achieve more accurate fingerprint recognition. Based on the feature matching result, the electronic device outputs the recognition result, informing the user whether a match was successful.

[0063] During fingerprint recognition, the electronic device can display the fingerprint light spot. The output light emitted by the organic light-emitting diode module (such as...) Figure 1 (As shown by the solid line) The light spot displayed on the screen through the overlay glass is the fingerprint spot. The fingerprint spot refers to a bright area emitted by the screen in a specific region, which is typically used to illuminate the user's finger for fingerprint recognition. This spot area is where the optical fingerprint module captures fingerprint information. When the user's finger presses on this fingerprint spot area, light emitted from the screen shines onto the finger, and then the light is reflected by the finger. The reflected light containing fingerprint information is received by the optical fingerprint module under the screen.

[0064] In this application embodiment, the way the electronic device displays fingerprint light spots may include soft light spot scheme and hard light spot scheme.

[0065] Please see Figure 4 The process of the hard spot scheme is illustrated by example.

[0066] The touch driver receives fingerprint events and reports them to the fingerprint driver, which then reports them to the fingerprint abstraction layer. The fingerprint abstraction layer then reports the fingerprint events to the fingerprint service.

[0067] In the hard spot scheme, the fingerprint hardware abstraction layer notifies the display driver to illuminate a localized light spot (LHBM) based on fingerprint events. The fingerprint service sends a drawing request to the drawing engine based on the fingerprint events. The drawing engine draws the relevant interface layers according to the drawing request. SurfaceFlinger composites the drawn layers to obtain a composite frame. SurfaceFlinger sends the composite frame containing all layer information and necessary data such as the layer compositing method to HWC. HWC processes the received data to obtain the image data of the final display image. The display driver receives the image data of the final display image output by HWC. The light spot is directly displayed by DDIC, not controlled by the upper layer. The registers directly store the light spot information and coordinate information, which are superimposed by DDIC.

[0068] Please see Figure 5 The process of a soft spot scheme is illustrated by example.

[0069] The differences between soft and hard light spot schemes include at least the following: In the soft light spot scheme, the fingerprint hardware abstraction layer does not notify the display driver to illuminate local light spots; in the soft light spot scheme, the light spot and highlight information are all sent and controlled by the upper layer. The layers drawn by the graphics engine according to the drawing request include, in addition to the relevant layers of the interface to be displayed (such as animation layers), mask layers and light spot layers. SurfaceFlinger or HWC adds light spot markers to the mask layer. HWC transmits the light spot markers to the display driver, and the display driver generates highlight instructions based on the light spot markers. The display driver transmits the highlight instructions and the image data of the final displayed image to the display driver integrated circuit (DDIC) chip. The display driver chip controls the screen to enter High Brightness Mode (HBM) according to the highlight instructions, and the display driver chip also controls the display to display the corresponding image based on the received image data.

[0070] In the soft fingerprint sensor solution, a mask layer is drawn on the upper layer to cover the area outside the fingerprint spot, and the screen needs to be controlled to enter a high-brightness mode for fingerprint image acquisition. Since the soft fingerprint sensor solution supports fingerprint spot display in video mode, it can be used in display driver chips that do not contain independent memory, such as random access memory (RAM), so it is less expensive than the hard fingerprint sensor solution.

[0071] The inventors discovered the following problems with the soft spot solution during the implementation of the embodiments of this application: First, the display effect is poor, and screen flickering or blackouts are prone to occur. Second, the unlocking process is time-consuming due to its complexity.

[0072] The following example uses a mobile phone as an example of fingerprint light display application in fingerprint unlocking to illustrate the screen flickering phenomenon.

[0073] Figure 6A Display the image data of frame N-1, such as Figure 6A As shown, the phone's display shows the lock screen interface, and the user touches the fingerprint unlock area on the lock screen interface with their finger. Figure 6B Display the image data of the Nth frame, such as Figure 6B As shown, the phone's screen is flickering. From the user's perspective, it can be seen... Figure 6B The display brightness relative to Figure 6A Increase. Figure 6C Display the image data of the (N+1)th frame, such as Figure 6C As shown, the phone's display still shows the lock screen. Here, N is an integer greater than 1.

[0074] The following example uses a mobile phone as an example of fingerprint light display application in fingerprint payment to illustrate the phenomenon of flashing black.

[0075] Figure 7A Display the image data of frame N-1, such as Figure 7A As shown, the phone's screen displays the payment interface, and the user touches the fingerprint unlock area of ​​the payment interface with their finger. Figure 7B Display the image data of the Nth frame, such as Figure 7B As shown, the phone's screen flickers black. From the user's perspective, it can be seen... Figure 7B The display brightness relative to Figure 7A reduce. Figure 7C Display the image data of the (N+1)th frame, such as Figure 7C As shown, the phone's screen still displays the payment interface.

[0076] The inventors discovered during the implementation of the embodiments of this application that the primary cause of screen flickering or blackout phenomena is that the mask layer and the highlight mode do not take effect in the same frame. When the highlight mode takes effect before the mask layer, screen flickering occurs; conversely, blackout phenomena occur, affecting the display effect and consequently the user experience, such as the fingerprint unlocking experience. Due to various reasons, the mask layer and highlight mode, which should take effect in the same frame, do not take effect in the same frame in electronic devices. For example, when other instructions, such as frame rate switching or backlight switching instructions, are issued simultaneously with the highlight instruction, the highlight instruction is easily delayed by one frame. This is because the display driver chip does not synchronously receive the highlight instruction and mask layer output after processing by the display driver, or is limited by the timing requirements of the display driver chip's underlying data and instruction processing. Furthermore, during fingerprint recognition, the highlight mode can be used to increase screen brightness, enabling the optical fingerprint module to capture fingerprint details more clearly. If the highlight instruction is delayed, it means that the optical fingerprint module receives insufficient light signal, resulting in inaccurate fingerprint information capture, prolonging the fingerprint data acquisition and processing time, and thus slowing down the fingerprint recognition speed.

[0077] In view of this, this application provides a fingerprint spot display method, which automatically identifies fingerprint events frame by frame through a display driver chip, and then automatically controls the synchronous activation of a highlight mode based on the fingerprint event recognition result, ensuring that the highlight mode and the mask layer are effective in the same frame at the underlying layer, avoiding screen flickering or blackout phenomena, thereby improving the fingerprint recognition speed.

[0078] Please see Figure 8 The present application provides an exemplary description of the fingerprint spot display method flow.

[0079] When a user touches the fingerprint recognition area of ​​an electronic device with their finger, the touch sensor detects the touch event and reports an interrupt to send a trigger signal to the touch driver.

[0080] The touch driver receives fingerprint events and reports the fingerprint events to the fingerprint driver.

[0081] The fingerprint driver reports fingerprint events layer by layer, and then passes them to the fingerprint service through the fingerprint abstraction layer.

[0082] The fingerprint service processes fingerprint events to display fingerprint spots. Based on the fingerprint events, the fingerprint service generates a drawing request and sends it to the drawing engine. The drawing request indicates the target interface to be drawn. In the lock screen unlocking scenario, the target interface can be the lock screen interface, which may include an Always On Display (AOD) interface. In the fingerprint payment scenario, the target interface can be the payment interface.

[0083] The drawing engine renders layers of the target interface based on the rendering request. These layers can include, but are not limited to: a light spot layer, a mask layer, and an animation layer. The light spot layer displays the fingerprint light spot area of ​​the target interface, while the mask layer displays the masked area of ​​the target interface excluding the fingerprint light spot area. Animation layers can contain visual animations to provide a better user experience.

[0084] In some embodiments, the graphics engine generates corresponding drawing instructions based on the drawing request. These instructions define how to draw the fingerprint image or other related elements on the screen. The graphics engine can use various graphics APIs (such as OpenGL, Skia, etc.) to optimize the drawing process and leverage the parallel processing capabilities of the GPU to accelerate drawing. SurfaceFlinger receives drawing instructions or image data from the graphics engine and manages them as layers. SurfaceFlinger combines multiple layers (such as light spot layers, mask layers, and animation layers) into a complete image based on the Z-axis sorting and compositing strategy of the layers.

[0085] HWC obtains the layers passed by SurfaceFlinger and generates the first marker based on the target interface's layers, including the mask layer.

[0086] In some embodiments, HWC performs layer compositing and layer recognition to determine whether a mask layer exists in the image frame. If it does, a first marker is added to the mask layer.

[0087] HWC passes the target interface's layers and first marker to the display driver.

[0088] The display driver synchronously transmits the layers and first marker of the target interface to the display driver chip of the screen.

[0089] Since the first mark is carried by the mask layer and transmitted to the display driver chip through the same transmission path, it can be ensured that the display driver chip receives both the mask layer and the first mark at the same time.

[0090] The display driver chip controls the display screen to show the target interface in a high-brightness mode based on the first marker.

[0091] Specifically, based on the first marker, the display driver chip can determine that the received image data includes a mask layer, thereby automatically controlling the display screen to enter a high-brightness mode.

[0092] In this embodiment, the display driver does not need to generate backlight switching instructions based on the light spot markings. Instead, the display driver synchronously transmits the mask layer and the first mark to the display driver chip. The display driver chip then controls the simultaneous entry into the highlight mode within the same frame. Based on the display driver chip's frame-by-frame data processing logic, this ensures that the mask layer and the highlight mode take effect in the same frame. This effectively avoids poor display effects caused by the display driver chip not receiving the processed highlight instructions and mask layer synchronously, or by the timing requirements of the display driver chip's data and instruction processing, which prevent the highlight mode and mask layer from taking effect in the same frame. Furthermore, since the display driver chip directly controls the screen to enter the highlight mode without waiting for the upper-level highlight instructions, there is no delay problem, effectively optimizing fingerprint recognition time and improving fingerprint recognition speed.

[0093] Please see Figure 9 This paper exemplarily introduces another fingerprint spot display method flow provided by an embodiment of the present application.

[0094] The display screen includes a display driver chip and a display panel. The display driver chip may include, but is not limited to, a Mobile Industry Processor Interface (MIPI) driver module, a Display Stream Compression (DSC) module, a marker recognition module, a cache module, a SubPixel Rendering (SPR) module, a gamma module, a demura module, an IR drop compensation module, and an output module.

[0095] In this embodiment, the fingerprint service notifies the drawing engine to draw layers based on fingerprint events, sending a drawing request to the drawing engine. The drawing engine draws the layers of the fingerprint recognition-related interface according to the drawing request. SurfaceFlinger composites the layers drawn by the drawing engine. HWC determines whether the image composited by SurfaceFlinger has a mask layer. If a mask layer exists, a first flag is written. If no mask layer exists, a second flag is written.

[0096] In the embodiments of this application, the first flag or the second mark can be an attribute or configuration parameter of the layer. During the transmission process, the first flag or the second mark can be processed and transmitted as part of the layer data.

[0097] In this embodiment, the information of the mask layer and the markers is synchronously transmitted to the display driver after being compressed by the display stream. The display driver does not perform any special processing on the information of the markers (such as generating highlight instructions). Instead, it transmits the image data and the information of the markers synchronously to the display driver chip through MIPI transmission lines after data conversion and MIPI encapsulation.

[0098] The MIPI driver module of the display driver chip receives image data and marker information (such as a first marker or a second marker). The image data and marker information are decompressed by the display stream compression module and then transmitted to the marker recognition module.

[0099] When the marker recognition module detects the first marker, it determines that a mask layer exists in the current frame. The marker recognition module communicates with the gamma module and notifies the gamma module to perform a set of settings or parameter settings related to gamma correction in order to switch to highlight mode.

[0100] The marker recognition module identifies the information of the marker. Regardless of whether the first marker is identified, the marker recognition module temporarily stores the image data in the cache module. Subsequently, the image data stream is processed by the sub-pixel rendering (SPR) module, gamma module, uniformization module, voltage drop compensation module, etc., and then output to the display panel by the output module.

[0101] When the marker recognition module detects the second marker, it determines that there is no mask layer, which means it is not a fingerprint recognition scenario. The underlying layer then proceeds with the normal image display data processing flow. That is, the marker recognition module temporarily stores the image data in the cache module. Subsequently, the image data is processed by the Sub Pixel Rendering (SPR) module, gamma module, uniformization module, voltage drop compensation module, etc., and then output to the display panel by the output module.

[0102] In some embodiments, once the marker recognition module notifies the gamma module to switch to high brightness mode, backlight switching will not occur again within one frame.

[0103] Taking the application of fingerprint spot display method to fingerprint unlocking as an example.

[0104] Figure 10A Display the image data of the Nth frame, such as Figure 10A As shown, the phone displays the lock screen. The user touches the fingerprint unlock area with their finger. Before displaying the next frame of image data, as... Figure 10B and Figure 10C As shown, the display driver chip receives both the mask layer and the first marker simultaneously, and controls the automatic switching of the highlight mode based on the first marker. Figure 10D Display the image data of the (N+1)th frame, such as Figure 10D As shown, in the same frame, the mask layer and the highlight mode are active simultaneously.

[0105] In this embodiment, the problem of poor display effect is solved by synchronizing the mask layer and highlight mode at the underlying level. Specifically, the display driver chip automatically identifies the presence of a mask layer frame by frame and simultaneously activates the highlight mode when a mask layer exists in the current frame. A marker recognition module is designed inside the display driver chip to identify a first marker. Since this first marker is carried by the mask layer, its detection indicates that the mask layer data has been transmitted to the display driver chip. The display driver chip then directly controls the entry into highlight mode without waiting for display driver processing. This means that the highlight mode is automatically activated without waiting for the issuance and processing of highlight commands, thus eliminating the latency issue. Based on the underlying frame-by-frame data processing logic, the mask layer and highlight command are ensured to take effect in the same frame. This underlying synchronization ensures that the frame data of the user's screen is synchronized with the display of the mask layer and highlight mode, effectively solving the problem of poor display effect caused by asynchrony. In addition, the fingerprint recognition process relies on the screen entering a high-brightness mode to capture fingerprint images. By directly controlling the screen to enter high-brightness mode from the bottom layer, there is no need to wait for the high-brightness command processing from the upper layer, so there is no delay problem, effectively optimizing the fingerprint recognition time and improving the fingerprint recognition speed.

[0106] Please see Figure 11 This paper exemplarily introduces another fingerprint spot display method provided by an embodiment of the present application. This method can be applied to the above-mentioned electronic device and may include steps S101 to S103.

[0107] Step S101: Receive touch operation on the display screen.

[0108] When a user touches the display screen with their finger, such as touching the fingerprint unlock area, the electronic device receives the user's touch operation on the display screen.

[0109] Specifically, when a user touches the fingerprint unlock area on the display screen, the electronic device's touch sensor detects the touch event in the fingerprint recognition area and receives the touch operation on the display screen. The touch sensor then reports the fingerprint event up the chain of command to the fingerprint service.

[0110] Step S102: Determine the image data of the target interface based on the touch operation.

[0111] In this embodiment, the electronic device determines the image data of the target interface based on the touch operation. Specifically, the fingerprint service generates a drawing request based on the fingerprint event. The drawing engine draws the layers of the target interface according to the drawing request, thus obtaining the layer data of the target interface.

[0112] In step S103, the display driver chip controls the display screen to display the target interface in a highlight mode based on the image data, including the mask layer.

[0113] In this embodiment of the application, the layer data of the target interface can be transmitted to the display driver chip via the graphics engine, SurfaceFlinger, HWC and display driver.

[0114] In this embodiment, the display driver chip controls the display screen to display the target interface in a highlight mode based on the layer data, including the mask layer. That is, if the display driver chip determines that the layer data is generated based on the fingerprint event and includes the mask layer, it can control the display screen to display the target interface in a highlight mode.

[0115] In some embodiments, if the display driver chip determines that the image data, including a mask layer and a spot layer, is generated based on a fingerprint event and includes a mask layer, then the display screen can be controlled to display the target interface in a highlight mode.

[0116] In some embodiments, the fingerprint spot display method further includes: generating a first mark based on image data including a mask layer.

[0117] In some embodiments, the fingerprint spot display method further includes: generating a second mark based on image data excluding a mask layer.

[0118] In some embodiments, a first marker may be generated by a fingerprint service, a drawing engine, SurfaceFlinger, HWC, or a display driver based on image data including a mask layer, ensuring that the mask layer and the first marker are simultaneously transmitted to the display driver chip. Alternatively, a second marker may be generated by a fingerprint service, a drawing engine, a hardware compositor, or a display driver based on image data excluding a mask layer, ensuring that both the mask layer and the second marker are simultaneously transmitted to the display driver chip.

[0119] This can be performed by any of the fingerprint service, drawing engine, SurfaceFlinger, HWC, and display driver: generating a first mark based on image data including a mask layer, or generating a second mark based on image data excluding a mask layer.

[0120] Specifically, the generation of a second mark based on image data excluding the mask layer can be performed by any one of the fingerprint service, drawing engine, SurfaceFlinger, HWC, and display driver: it can include adding a first mark on the mask layer based on image data including the mask layer.

[0121] In some embodiments, the display driver chip controls the display screen to display the target interface in a highlight mode based on image data including a mask layer, including: the display driver chip acquiring a first marker and image data, and controlling the display screen to display the target interface in a highlight mode based on the first marker.

[0122] In some embodiments, the electronic device further includes a display driver, and the image data of the target interface further includes a spot layer. The display driver transmits the image data and a first marker to the display driver chip; the display driver chip acquires the first marker and the image data, and controls the display screen to display the target interface in a high-brightness mode based on the first marker, including: the display driver chip drives the display screen to display a fingerprint spot area according to the spot layer, drives the display screen to display a mask area according to the mask layer, and drives the display screen to display in a high-brightness mode according to the first marker.

[0123] In some embodiments, the display driver chip includes a communication module, a marker recognition module, a gamma module, and an image processing module; the display driver chip acquires a first marker and image data, and controls the display screen to display the target interface in a high-brightness mode based on the first marker, including: the communication module acquires the first marker and image data; the communication module transmits the first marker to the marker recognition module; the marker recognition module controls the gamma module to switch to a high-brightness mode based on the first marker, and controls the image processing module to display the target interface according to the image data.

[0124] The layer processing module may include, but is not limited to: subpixel rendering module, uniformization module, voltage drop compensation module, and output module.

[0125] In some embodiments, the marker recognition module controls the image processing module to display the target interface based on the image data, based on the second marker.

[0126] Existing technologies such as Figure 5 The soft spot scheme shown converts the spot markers into highlight instructions or backlight adjustment instructions after processing by the display driver. However, in actual data transmission, there are logical and conflict issues in sending images and instructions. The data of the mask layer and the backlight adjustment instructions (or highlight instructions) may not be transmitted to the display driver chip at the same time (in the same frame). The data of the mask layer and the backlight adjustment instructions (or highlight instructions) may be transmitted to the display driver chip across frames. Furthermore, due to the timing limitations of the underlying processing instructions and image data, the display driver chip may not be able to process the mask layer and highlight instructions in the same frame, which may still lead to problems such as screen flickering or blackouts.

[0127] In this embodiment, the mask layer and highlight mode are activated simultaneously within the same frame through low-level synchronization, effectively avoiding poor display effects such as screen flickering or blackouts. The synchronous transmission of information related to the mask layer and highlight mode markers (first markers) is implemented within the underlying display driver chip. The display driver chip automatically identifies the markers frame by frame. When the display driver chip identifies the first marker, i.e., when it receives the mask layer data, it automatically controls the synchronous activation of the highlight mode, effectively solving the problems of poor display effects and slow fingerprint recognition speed.

[0128] Please see Figure 12 The hardware structure of an electronic device is illustrated by example.

[0129] Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0130] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0131] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0132] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0133] The processor 110 may also include a memory for storing instructions and data. In one embodiment of this application, the memory in the processor 110 is a cache memory. The memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instructions or data again, it can directly retrieve them from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0134] The fingerprint spot display method provided in this application embodiment can be applied to the above-mentioned electronic devices.

[0135] The chip provided in this application embodiment can implement the fingerprint spot display method in the above method embodiments.

[0136] This application provides a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to execute the fingerprint spot display method described above.

[0137] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the fingerprint spot display method in the above method embodiments.

[0138] This application also provides a computer storage medium including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the fingerprint spot display method as described above.

[0139] In this application, the electronic devices, computer storage media, computer program products, or chip systems provided in the embodiments are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0142] The unit described as a separate component may or may not be physically separate. The component shown as a unit can be one physical unit or multiple physical units, that is, it can be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0143] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application.

Claims

1. A method for displaying fingerprint light spots, characterized in that, The method is applied to an electronic device, which includes a display driver chip and a display screen, wherein the display driver chip includes a communication module, a marker recognition module, a gamma module, and an image processing module; the method includes: Receive touch operations on the display screen; The image data of the target interface is determined based on the touch operation; Based on the image data, including the mask layer, a first marker is generated; The communication module acquires the first marker and the image data, and transmits the first marker to the marker recognition module; based on the first marker, the marker recognition module controls the gamma module to switch to a highlight mode, and controls the image processing module to display the target interface according to the image data.

2. The method as described in claim 1, characterized in that, The generation of the first marker based on the image data, including the mask layer, includes: Based on the image data, including the mask layer, a first mark is added to the mask layer.

3. The method as described in claim 1 or 2, characterized in that, The electronic device further includes a display driver, and the image data of the target interface further includes a spot layer, wherein the spot layer is used to display the fingerprint spot area of ​​the target interface, and the mask layer is used to display the mask area of ​​the target interface other than the fingerprint spot area; The display driver transmits the image data and the first tag to the display driver chip.

4. The method as described in claim 1 or 2, characterized in that, The method further includes: A second marker is generated based on the image data excluding the mask layer; The marker recognition module controls the image processing module to display the target interface based on the second marker.

5. An electronic device, characterized in that, include: Processor and memory; The memory is used to store computer programs, the computer programs including program instructions; The processor is configured to invoke the computer program to execute the method as described in any one of claims 1 to 4.

6. A chip system, said chip system being used in an electronic device, characterized in that, The chip system includes one or more processors, which are configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product stores at least one instruction that, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images