Fingerprint light spot display method and related equipment

Abstract

The invention relates to the technical field of image processing, and provides a fingerprint light spot display method and related device.The fingerprint light spot display method is applied to electronic equipment, the electronic equipment comprises a display driving chip and a display screen, and the method comprises the steps that touch operation on the display screen is received; determining image data of the target interface according to the touch operation; and the display driving chip controls the display screen to display a target interface in a highlight mode based on the mask layer of the image data. The phenomenon of screen flashing or blackening possibly occurring in the fingerprint identification process can be effectively avoided, and the fingerprint identification speed is increased.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a fingerprint spot display method and related equipment. Background Art

[0002] Fingerprint recognition technology is widely used in electronic devices. For example, optical fingerprint recognition technology can display a fingerprint spot of a specific shape and brightness in the fingerprint recognition area on the display. When the user's finger presses on the fingerprint spot, the device can collect the user's fingerprint information and realize functions such as fingerprint unlocking, fingerprint payment, and fingerprint photography. However, during the fingerprint recognition process, the screen may flicker or go black due to timing issues. Summary of the Invention

[0003] The embodiments of the present application provide a fingerprint spot display method and related equipment, which can effectively avoid possible screen flickering or black flashing during the fingerprint recognition process and improve the fingerprint recognition speed.

[0004] A first aspect provides a fingerprint spot display method, which is applied to an electronic device, the electronic device including a display driver chip and a display screen. The method includes: receiving a touch operation on the display screen; determining image data of a target interface based on the touch operation; and the display driver chip controlling the display screen to display the target interface in a highlight mode based on the image data including a mask layer.

[0005] In an embodiment of the present application, the display driver chip is used to determine whether the received image data includes a mask layer. If it includes a mask layer, the display screen is controlled to display the target interface in a highlight mode. That is, when displaying the frame image of the target interface, the display driver chip is used to implement the mask image and the highlight mode at the same time in the frame image, thereby effectively avoiding possible screen flashing or black flashing during the fingerprint recognition process and improving the fingerprint recognition speed.

[0006] In one possible implementation, the method further includes: generating a first mark based on the image data including a mask layer; the display driver chip controls the display screen to display the target interface in a highlight mode based on the image data including the mask layer, including: the display driver chip obtains the first mark and the image data, and based on the first mark, controls the display screen to display the target interface in a highlight mode.

[0007] By generating a first mark, the first mark and the mask layer can be simultaneously transmitted to the display driver chip, so that the display driver chip can control the display screen to display the target interface in a highlight mode based on the recognition of the first mark, thereby effectively avoiding the screen flashing or black flashing phenomenon that may occur during the fingerprint recognition process, and improving the fingerprint recognition speed.

[0008] In one possible implementation, based on the image data including a mask layer, generating the first mark includes: based on the image data including the mask layer, adding the first mark on the mask layer.

[0009] By adding a first mark on the mask layer, the first mark is transmitted to the display driver chip along with the mask layer. The display driver chip can control the display screen to display the target interface in a high-brightness mode based on the recognition of the first mark, thereby effectively avoiding the screen flashing or black flashing phenomenon that may occur during the fingerprint recognition process, and improving the fingerprint recognition speed.

[0010] In one possible implementation, the electronic device also includes a display driver, and the image data of the target interface also includes a light spot layer, wherein the light spot layer is used to display the fingerprint light spot area of the target interface, and the mask layer is used to display the mask area of the target interface except the fingerprint light spot area; the display driver transmits the image data and the first mark to the display driver chip.

[0011] The image data and the first mark are synchronously transmitted to the display driver chip through the display driver. Based on the recognition of the first mark, the display driver chip can control the display screen to display the target interface in a high-brightness mode, thereby effectively avoiding the screen flashing or black flashing phenomenon that may occur during the fingerprint recognition process, and improving the fingerprint recognition speed.

[0012] In one possible implementation, the display driver chip includes a communication module, a mark recognition module, a gamma module and an image processing module; the display driver chip obtains a first mark and image data, and based on the first mark, controls the display screen to display the target interface in a highlight mode, including: the communication module obtains the first mark and image data; the communication module transmits the first mark to the mark recognition module; the mark recognition module controls the gamma module to switch to the highlight mode based on the first mark, and controls the image processing module to display the target interface according to the image data.

[0013] By setting a mark recognition module in the display driver chip, the mark recognition module recognizes the first mark and controls the display screen to display the target interface in a highlighted mode, thereby effectively avoiding the screen flashing or black flashing phenomenon that may occur during the fingerprint recognition process and improving the fingerprint recognition speed.

[0014] In one possible implementation, the method further includes: generating a second marker based on that the image data does not include a mask layer; and the marker recognition module controls the image processing module to display a target interface according to the image data based on the second marker.

[0015] A second aspect provides an electronic device, comprising: a processor and a memory; the memory is used to store a computer program, the computer program including program instructions; the processor is used to call the computer program and execute any of the above methods.

[0016] The third aspect provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute any of the above methods.

[0017] A fourth aspect provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, causes the electronic device to execute any one of the methods described above.

[0018] A fifth aspect provides a computer program product, which stores at least one instruction, and when the at least one instruction is executed by a processor, implements any of the above methods.

[0019] The technical effects obtained in the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application.

[0021] Figure 2 This is the intention of optical under-screen fingerprint collection provided in the embodiment of this application.

[0022] Figure 3 A schematic diagram of fingerprint recognition provided in an embodiment of the present application.

[0023] Figure 4 Schematic diagram of the hard spot solution provided in an embodiment of the present application.

[0024] Figure 5 Schematic diagram of the soft spot solution provided in an embodiment of the present application.

[0025] Figures 6A to 6C A schematic diagram of a splash screen provided in an embodiment of the present application.

[0026] 7A to 7C A schematic diagram of flashing black provided in an embodiment of the present application.

[0027] Figure 8 A schematic flow chart of a fingerprint spot display method provided in an embodiment of the present application.

[0028] Figure 9 A schematic flow chart of another fingerprint spot display method provided in an embodiment of the present application.

[0029] 10A to 10D A schematic diagram of a fingerprint spot display provided in an embodiment of the present application.

[0030] Figure 11A schematic flow chart of another fingerprint spot display method provided in an embodiment of the present application.

[0031] Figure 12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] As used herein, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element specified by the phrase "comprising an" does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. "At least one" means one or more, and "a plurality" means two or more. "And / or" describes an association between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," and so on (if any) in the specification, claims, and drawings of this application are used to distinguish similar objects, not to describe a particular order or sequential sequence. Words such as "exemplary" or "for example" are used to indicate an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0033] In the embodiment of the present application, the operating system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The following describes the software structure of the electronic device using a layered architecture as an example.

[0034] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. Figure 1 As shown in FIG, the Android system is divided into five layers, which are, from top to bottom, the application layer, the application framework layer, the Android runtime (Android runtime) and system library, the hardware abstraction layer, and the kernel layer.

[0035] The application layer can include a series of application packages, including camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0036] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0037] The application framework layer may include fingerprint services, window managers, content providers, view systems, telephony managers, resource managers, notification managers, etc. The application framework layer may provide fingerprint recognition application-related APIs for fingerprint recognition functions, and provide fingerprint services for fingerprint recognition applications to implement fingerprint recognition functions.

[0038] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0039] Content providers are used to store and retrieve data and make it accessible to applications. Data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0040] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0041] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0042] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0043] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0044] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0045] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0046] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0047] The system library can include multiple functional modules, such as the drawing engine, surface compositor (SurfaceFlinger), media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0048] The drawing engine is responsible for drawing graphics.

[0049] SurfaceFlinger is responsible for compositing different surfaces into a single framebuffer and then displaying the contents of this framebuffer on the screen.

[0050] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0051] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing. The hardware abstraction layer is a layer between the system library and the kernel layer. The hardware abstraction layer can include the fingerprint (FP) abstraction layer and the hardware composer (HWC).

[0052] The fingerprint abstraction layer is used to report fingerprint events to the fingerprint service of the application framework layer.

[0053] HWC is the driver abstraction layer of the layer synthesis dedicated chip. For example, in the embodiment of the present application, HWC is used to connect SurfaceFlinger and the display driver, that is, HWC is the communication bridge between SurfaceFlinger and the display driver, so that the layer synthesized by SurfaceFlinger is transmitted to the display driver for display.

[0054] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, touch driver, fingerprint driver, audio driver, and sensor driver.

[0055] In some embodiments, the electronic device may further include a display driver chip, a touch sensor, and a display screen.

[0056] A display screen is used to display images, videos, and the like. The display screen may include a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (LED), a MiniLED, a MicroLED, a Micro-oLED, a quantum dot light-emitting diode, or the like. In some embodiments, an electronic device may include one or more display screens.

[0057] In addition to the display panel, the display screen may also include a corresponding display driver chip. In some implementations, one display screen may be configured with one display driver chip. If there are multiple display screens, the electronic device may include multiple display driver chips.

[0058] The display driver chip sends drive signals and data to the display panel, controlling the screen's brightness and color to display images, such as pictures and letters. The touch sensor senses user touch on the display.

[0059] A touch sensor, also known as a "touch device," can be mounted on a display screen. The touch sensor and the display screen together form a touch screen, also known as a "touch screen." The touch sensor is used to detect touch operations applied to or near the touch sensor. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen. In other embodiments, the touch sensor can also be mounted on the surface of the electronic device, in a location different from that of the display screen.

[0060] See also Figure 2 , an example is given to introduce the principle of optical under-screen fingerprint collection. The screen of an electronic device may include: cover glass, an organic light-emitting diode module, and an optical fingerprint module. Among them, the cover glass can also be called CoverGlass cover or CG cover. The optical fingerprint module may include but is not limited to the following sensors: Complementary Metal-Oxide-Semiconductor (CMOS) and Thin Film Transistor (TFT).

[0061] The user places his finger on a specific area on the screen, such as the fingerprint recognition area. The output light emitted by the organic light emitting diode module (such as Figure 2The fingerprint recognition area of the screen is illuminated through the cover glass (shown by the solid line in the middle), and the output light is reflected by the fingerprint recognition area to obtain reflected light (as shown in FIG. Figure 2 The reflected light passes through the gap between the cover glass and the OLED module and returns to the optical fingerprint module's sensor. Due to the unevenness of the fingerprint, the output light undergoes varying degrees of refraction and reflection. The light reflected back to the optical fingerprint module has varying degrees of brightness due to the different refraction angles, with ridges reflecting darker and valleys reflecting brighter. The optical fingerprint module receives the reflected light and generates a multi-grayscale fingerprint image based on the brightness differences, with ridges appearing black and valleys appearing white.

[0062] Please also refer to Figure 3 , through optical imaging technology, the reflected light reflected from the finger is converted into a fingerprint image that can be processed by the electronic device. The sensor of the optical fingerprint module collects the reflected light reflected from the user's finger and generates an original image (i.e., a multi-grayscale fingerprint image). The electronic device performs image processing on the original image to obtain a processed image. The image processing may include but is not limited to: normalization, segmentation, enhancement, binarization, and refinement. Feature extraction is performed on the processed image to obtain feature information. The electronic device performs feature matching on the feature information, that is, compares the feature information with the fingerprint template stored in the database to determine whether it matches. If the imaging result is good, the fingerprint template can be updated to obtain a more accurate fingerprint recognition effect. The electronic device outputs the recognition result based on the result of the feature matching and informs the user whether the match is successful.

[0063] During the fingerprint recognition process, the electronic device can display the fingerprint spot. The output light emitted by the organic light emitting diode module (such as Figure 1 The light spot that appears on the screen through the cover glass (shown by the solid line in the middle) is called the fingerprint spot. A fingerprint spot is a bright area of light emitted from a specific area of the screen. This area is typically used to illuminate the user's finger for fingerprint recognition. This spot is where the optical fingerprint module captures fingerprint information. When the user's finger presses against this fingerprint spot, light from the screen shines on the finger, which then reflects off the finger. The reflected light, containing the fingerprint information, is received by the optical fingerprint module under the display.

[0064] In the embodiment of the present application, the manner in which the electronic device displays the fingerprint spot may include a soft spot scheme and a hard spot scheme.

[0065] See also Figure 4 , which exemplifies the process of the hard spot solution.

[0066] The touch driver receives fingerprint events and reports them to the fingerprint driver. The fingerprint driver reports the fingerprint events to the fingerprint abstraction layer. The fingerprint abstraction layer reports the fingerprint events to the fingerprint service.

[0067] In the hard light spot solution, the fingerprint hardware abstraction layer notifies the display driver to light up the local light spot (LHBM) based on the fingerprint event. The fingerprint service sends a drawing request to the drawing engine based on the fingerprint event. The drawing engine draws the layers of the relevant interface according to the drawing request. SurfaceFlinger synthesizes the drawn layers to obtain a synthesized frame. SurfaceFlinger sends the synthesized frame containing all layer information and the necessary data such as the layer synthesis method to the HWC. The HWC synthesizes the received data to obtain the image data of the final display image. The display driver receives the image data of the final display image output by the HWC. The light spot is directly displayed by the DDIC and is not controlled by the upper layer. The register directly stores the light spot information and coordinate information, which is superimposed by the DDIC.

[0068] See also Figure 5 , which exemplifies the process of the soft spot solution.

[0069] The difference between the soft spot solution and the hard spot solution includes at least the following: in the soft spot solution, the fingerprint hardware abstraction layer does not notify the display driver to light up the local spot, and the spot and highlight information of the soft spot are all sent down and controlled by the upper layer. The layers drawn by the drawing engine according to the drawing request include, in addition to the relevant layers of the interface to be displayed (such as the motion effect layer), the mask layer and the spot layer. SurfaceFlinger or HWC adds a spot mark on the mask layer. The HWC transmits the spot mark to the display driver, and the display driver generates a highlight instruction based on the spot mark. The display driver transmits the highlight instruction and the image data of the final display image to the display driver chip (Display Driver Integrated Circuit, DDIC) of the display screen. The display driver chip controls the screen to enter the highlight mode (High Brightness Mode, HBM) according to the highlight instruction, and the display driver chip also controls the display screen to display the corresponding image based on the received image data.

[0070] In the soft spot solution, a mask layer is drawn on the upper layer to cover the area outside the fingerprint spot, and the screen must be controlled to enter high-brightness mode for fingerprint capture. Because the soft spot solution supports fingerprint spot display in video mode, it can be used in display driver chips that do not include independent memory such as random access memory (RAM), making it more cost-effective than the hard spot solution.

[0071] While implementing the embodiments of the present application, the inventors discovered that the soft spot solution has the following problems: first, the display effect is poor, and screen flickering or black flashing may occur easily. Second, the unlocking process is complicated, resulting in a long time consumption.

[0072] The following uses a mobile phone as an electronic device and the application of fingerprint light display in fingerprint unlocking as an example to illustrate the screen flickering phenomenon.

[0073] Figure 6A Display the N-1 frame image data, such as Figure 6A As shown, the phone's display screen displays the lock screen interface, and the user touches the fingerprint unlocking area of the lock screen interface with their finger. Figure 6B Display the Nth frame image data, such as Figure 6B As shown, the phone's display screen flickers. From the user's perspective, you can see Figure 6B The display brightness is relative to Figure 6A Increase. Figure 6C Display the N+1 frame image data, such as Figure 6C As shown, the display screen of the mobile phone still displays the lock screen interface. Wherein, N is an integer greater than 1.

[0074] The following uses a mobile phone as an electronic device and the fingerprint light display used in fingerprint payment as an example to illustrate the flash-black phenomenon.

[0075] Figure 7A Display the N-1 frame image data, such as Figure 7A As shown, the display screen of the mobile phone displays the payment interface, and the user touches the fingerprint unlocking area of the payment interface with his finger. Figure 7B Display the Nth frame image data, such as Figure 7B As shown, the phone's display screen flashes black. From the user's perspective, you can see Figure 7B The display brightness is relative to Figure 7A reduce. Figure 7C Display the N+1 frame image data, such as Figure 7C As shown, the phone's display still displays the payment interface.

[0076] When implementing the embodiments of the present application, the inventors found that the main reason for the screen flickering or black flash phenomenon is that the mask layer and the highlight mode are not effective in the same frame. When the highlight mode is effective earlier than the mask layer, the screen flickering phenomenon will occur, otherwise the black flash phenomenon will occur, affecting the display effect and thus affecting the user experience, such as the fingerprint unlocking experience. Due to various reasons, the mask layer and the highlight mode that should be effective in the same frame of the electronic device are not effective in the same frame. For example, when there are other instructions such as frame rate switching or backlight switching instructions and highlight instructions issued simultaneously, it is easy for the highlight instruction to be delayed by one frame. This is because the display driver chip does not synchronously receive the highlight instruction and mask layer output after display driver processing or is limited by the timing requirements of the display driver chip's underlying processing data and instructions. In addition, during the fingerprint recognition process, the highlight mode can be used to increase the screen brightness, allowing the optical fingerprint module to capture fingerprint details more clearly. If the highlight instruction is delayed, it means that the optical fingerprint module receives insufficient light signals, resulting in the inability to accurately capture fingerprint information, extending the collection and processing time of fingerprint data, thereby slowing down the fingerprint recognition speed.

[0077] In view of this, an embodiment of the present application provides a fingerprint spot display method, which automatically identifies fingerprint events frame by frame through a display driver chip, and then based on the recognition results of the fingerprint events, the display driver chip automatically controls the synchronous opening of the highlight mode to ensure that the highlight mode and the mask layer at the bottom layer are effective in the same frame, avoiding screen flashing or black flashing, thereby improving the fingerprint recognition speed.

[0078] See also Figure 8 , an exemplary introduction to the fingerprint spot display method process provided in an embodiment of the present application.

[0079] When a user touches the fingerprint recognition area of an electronic device, the touch sensor detects the touch event and generates an interrupt signal to send a trigger signal to the touch driver.

[0080] The touch driver receives fingerprint events and reports them to the fingerprint driver.

[0081] The fingerprint driver reports the fingerprint event layer by layer and passes it to the fingerprint service through the fingerprint abstraction layer.

[0082] The fingerprint service processes fingerprint events to display fingerprint spots. Based on the fingerprint events, the fingerprint service generates drawing requests and sends them to the drawing engine. Drawing requests indicate the target interface to be drawn. In the lock screen unlock scenario, the target interface can be the lock screen, which can include the Always On Display (AOD) interface. In the fingerprint payment scenario, the target interface can be the payment interface.

[0083] The drawing engine draws the target interface's layers based on the drawing request. These layers may include, but are not limited to, a spot layer, a mask layer, and an animation layer. The spot layer displays the fingerprint spot area of the target interface, while the mask layer displays the masked area of the target interface excluding the fingerprint spot area. The animation layer may include visual effects to provide a better user interaction experience.

[0084] In some embodiments, the drawing engine generates corresponding drawing instructions based on the drawing request. These instructions define how to draw the fingerprint image or other related elements on the screen. The drawing engine can use various graphics APIs (such as OpenGL, Skia, etc.) to optimize the drawing process and use the parallel processing capabilities of the GPU to accelerate the drawing. SurfaceFlinger receives drawing instructions or image data from the drawing engine and manages it as a layer. SurfaceFlinger combines multiple layers (such as spot layers, mask layers, and motion effect layers) into a complete picture based on the Z-axis sorting and synthesis strategy of the layers.

[0085] HWC obtains the layer passed by SurfaceFlinger and generates a first mark based on the layer of the target interface including the mask layer.

[0086] In some embodiments, the HWC synthesizes the layers and performs layer recognition to determine whether there is a mask layer in the frame image. If so, a first mark is added to the mask layer.

[0087] The HWC passes the target interface's layer and first markup to the display driver.

[0088] The display driver synchronously transmits the layer of the target interface and the first mark to the display driver chip of the display screen.

[0089] Since the first mark is carried by the mask layer and transmitted to the display driver chip through the same transmission path, it is possible to ensure that the display driver chip receives the mask layer and the first mark at the same time.

[0090] The display driver chip controls the display screen to display the target interface in a highlight mode based on the first mark.

[0091] Specifically, the display driver chip can determine that the received image data includes a mask layer based on the first mark, thereby automatically controlling the display screen to enter a high-brightness mode.

[0092] In the embodiment of the present application, there is no need for the display driver to generate a backlight switching instruction based on the light spot mark. Instead, the display driver synchronously transmits the mask layer and the first mark to the display driver chip, and the bottom layer of the display driver chip controls the synchronous entry into the highlight mode in the same frame. Based on the logic of the bottom layer of the display driver chip processing data frame by frame, it can ensure that the mask layer and the highlight mode take effect in the same frame, effectively avoiding the poor display effect caused by the display driver chip not synchronously receiving the highlight instruction and mask layer processed by the display driver or being limited by the timing requirements of the bottom layer of the display driver chip processing data and instructions, so that the highlight mode and the mask layer cannot take effect in the same frame. Furthermore, the bottom layer of the display driver chip directly controls the screen to enter the highlight mode, without waiting for the highlight instruction from the upper layer, so there is no delay problem, which effectively optimizes the fingerprint recognition time and improves the fingerprint recognition speed.

[0093] See also Figure 9 , exemplarily introduces another fingerprint spot display method process provided by an embodiment of the present application.

[0094] The display screen includes a display driver chip and a display panel. The display driver chip may include, but is not limited to, a Mobile Industry Processor Interface (MIPI) driver module, a Display Stream Compression (DSC) module, a marker recognition module, a cache module, a Sub-Pixel Rendering (SPR) module, a gamma module, a demura module, an IR Drop compensation module, and an output module.

[0095] In this embodiment of the present application, the fingerprint service notifies the drawing engine to draw a layer based on a fingerprint event and sends a drawing request to the drawing engine. The drawing engine draws the layer of the fingerprint recognition-related interface based on the drawing request. SurfaceFlinger composites the layers drawn by the drawing engine. The HWC determines whether the image composited by SurfaceFlinger contains a mask layer. If a mask layer exists, a first flag is written. If not, a second flag is written.

[0096] In the embodiment of the present application, the first mark or the second mark can be an attribute or configuration parameter of the layer. During the transmission process, the first mark or the second mark can be processed and transmitted as part of the layer data.

[0097] In an embodiment of the present application, the information of the mask layer and the mark is synchronously transmitted to the display driver after display stream compression processing. The display driver does not perform special processing on the mark information (such as generating a highlight instruction), but instead processes the image data and the mark information through data conversion and MIPI packaging, and then synchronously transmits it to the display driver chip through the MIPI transmission line.

[0098] The MIPI driver module of the display driver chip receives the image data and the marking information (such as the first marking or the second marking). The image data and the marking information are decompressed by the display stream compression module and then transmitted to the marking recognition module.

[0099] When the marker recognition module recognizes the first marker, it determines that there is a mask layer in the current frame. The marker recognition module communicates with the gamma module and notifies the gamma module to perform a set of settings or parameter settings related to gamma correction to switch to highlight mode.

[0100] The mark recognition module recognizes the mark information. Regardless of whether the first mark is recognized, the mark recognition module temporarily stores the image data in the cache module. Then, the image data stream is processed by the sub-pixel rendering (SPR) module, the gamma module, the equalization module, the voltage drop compensation module, etc., and then output to the display panel by the output module.

[0101] When the mark recognition module recognizes the second mark, it determines that there is no mask layer, which indicates a non-fingerprint recognition scenario. The underlying layer follows the normal image display data processing flow, that is, the mark recognition module temporarily stores the image data in the cache module, and then the image data is processed by the sub-pixel rendering (SPR) module, gamma module, uniformization module, voltage drop compensation module, etc., and then outputted to the display panel by the output module.

[0102] In some embodiments, after the finger mark recognition module notifies the gamma module to switch to the high-brightness mode, the backlight switching is no longer performed within one frame.

[0103] Take the application of fingerprint spot display method in fingerprint unlocking as an example.

[0104] Figure 10A Display the Nth frame image data, such as Figure 10A As shown, the phone displays the lock screen interface. The user touches the fingerprint unlocking area. Before displaying the next frame of image data, Figure 10B and Figure 10C As shown, the display driver chip receives the mask layer and the first mark at the same time, and controls the automatic switching of the highlight mode based on the first mark. Figure 10D Display the N+1 frame image data, such as Figure 10D As shown, in the same frame, the mask layer and highlight mode are effective at the same time.

[0105] In an embodiment of the present application, the mask layer and the highlight mode are synchronized at the bottom layer, that is, the display driver chip automatically identifies whether there is a mask layer frame by frame, and synchronously turns on the highlight mode when there is a mask layer in the current frame, so as to solve the problem of poor display effect. Specifically, a mark recognition module is designed inside the display driver chip for identifying the first mark. Since the first mark is carried by the mask layer, when the first mark is recognized, it means that the data of the mask layer has been passed to the display driver chip. The display driver chip directly controls the entry into the highlight mode without waiting for the display driver to process, that is, the highlight mode is automatically realized without waiting for the issuance and processing of the highlight instruction. Therefore, there is no delay problem in the issuance and processing of the highlight instruction. Based on the logic of the bottom frame data processing, it is ensured that the mask layer and the highlight instruction can take effect in the same frame. By means of bottom synchronization, it is ensured that the frame data of the user-end screen realizes the synchronization of the mask layer and the highlight mode display, effectively solving the problem of poor display effect caused by asynchrony. In addition, the fingerprint recognition process relies on the screen entering highlight mode to capture fingerprints. By directly controlling the screen to enter highlight mode through the bottom layer, there is no need to wait for the upper layer's highlight command processing, so there will be no delay problem, which effectively optimizes the fingerprint recognition time and improves the fingerprint recognition speed.

[0106] See also Figure 11 , another fingerprint spot display method provided in an embodiment of the present application is exemplarily introduced. The method can be applied to the above-mentioned electronic device and can include steps S101 to S103.

[0107] Step S101: receiving a touch operation on the display screen.

[0108] When a user touches the display screen with a finger, such as touching the fingerprint unlocking area of the display screen, the electronic device receives the user's touch operation on the display screen.

[0109] Specifically, when a user touches the fingerprint unlocking area of the display screen, the touch sensor of the electronic device detects the touch event in the fingerprint recognition area and receives the touch operation on the display screen. The touch sensor reports the fingerprint event layer by layer to the fingerprint service.

[0110] Step S102: determining image data of the target interface according to the touch operation.

[0111] In the embodiment of the present application, the electronic device determines the image data of the target interface according to the touch operation. Specifically, the fingerprint service generates a drawing request according to the fingerprint event. The drawing engine draws the layer of the target interface according to the drawing request to obtain the layer data of the target interface.

[0112] Step S103 : The display driver chip controls the display screen to display the target interface in a highlight mode based on the image data including the mask layer.

[0113] In an embodiment of the present application, the layer data of the target interface can be transmitted to the display driver chip via the drawing engine, SurfaceFlinger, HWC and display driver.

[0114] In an embodiment of the present application, the display driver chip controls the display screen to display the target interface in a highlight mode based on the layer data including the mask layer. That is, the display driver chip determines that the layer data is generated based on a fingerprint event trigger, and the layer data includes the mask layer, and then the display screen can be controlled to display the target interface in a highlight mode.

[0115] In some embodiments, the display driver chip determines that the layer data is generated based on a fingerprint event trigger and that the layer data includes a mask layer based on the image data including a mask layer and a spot layer, and can control the display screen to display the target interface in a highlight mode.

[0116] In some embodiments, the fingerprint spot display method further includes: generating a first mark based on the image data including a mask layer.

[0117] In some embodiments, the fingerprint spot display method further includes: generating a second mark based on the image data not including a mask layer.

[0118] In some embodiments, the fingerprint service, graphics engine, SurfaceFlinger, HWC, or display driver can generate a first mark based on the image data including the mask layer, ensuring that the mask layer and the first mark are simultaneously transmitted to the display driver chip. The fingerprint service, graphics engine, hardware compositor, or display driver can also generate a second mark based on the image data not including the mask layer, ensuring that the mask layer and the second mark are simultaneously transmitted to the display driver chip.

[0119] Among them, any one of the fingerprint service, the drawing engine, the SurfaceFlinger, the HWC and the display driver can be executed: generating a first mark based on the image data including the mask layer, or generating a second mark based on the image data not including the mask layer.

[0120] Specifically, the step of generating the second mark based on that the image data does not include the mask layer may include: adding the first mark on the mask layer based on that the image data includes the mask layer, performed by any one of the fingerprint service, the drawing engine, the SurfaceFlinger, the HWC and the display driver.

[0121] In some embodiments, the display driver chip includes a mask layer based on the image data, and controls the display screen to display the target interface in a highlight mode, including: the display driver chip obtains a first mark and image data, and based on the first mark, controls the display screen to display the target interface in a highlight mode.

[0122] In some embodiments, the electronic device further comprises a display driver, and the image data of the target interface further comprises a light spot layer. The display driver transmits the image data and the first marker to a display driver chip; the display driver chip obtains the first marker and the image data, and based on the first marker, controls the display screen to display the target interface in a highlight mode, including: the display driver chip drives the display screen to display the fingerprint light spot area according to the light spot layer, drives the display screen to display the mask area according to the mask layer, and drives the display screen to display in a highlight mode according to the first marker.

[0123] In some embodiments, the display driver chip includes a communication module, a tag recognition module, a gamma module and an image processing module; the display driver chip obtains a first tag and image data, and based on the first tag, controls the display screen to display the target interface in a highlight mode, including: the communication module obtains the first tag and image data; the communication module transmits the first tag to the tag recognition module; the tag recognition module controls the gamma module to switch to the highlight mode based on the first tag, and controls the image processing module to display the target interface according to the image data.

[0124] The layer processing module may include but is not limited to: a sub-pixel rendering module, a homogenization module, a voltage drop compensation module, and an output module.

[0125] In some embodiments, the marker recognition module controls the image processing module to display the target interface according to the image data based on the second marker.

[0126] Existing technologies such as Figure 5 The soft spot solution shown in the figure converts the spot mark into a highlight instruction or a backlight adjustment instruction after being processed by the display driver. However, in actual data transmission, there will be logic and conflict problems in sending images and instructions. The data of the mask layer and the backlight adjustment instruction (or highlight instruction) are not necessarily transmitted to the display driver chip at the same time (in the same frame). The data of the mask layer and the backlight adjustment instruction (or highlight instruction) may be transmitted to the display driver chip across frames. Moreover, due to the timing limitations of the underlying processing instructions and image data, the display driver chip may not be able to process the mask layer and the highlight instruction in the same frame, which may still cause problems such as screen flickering or black flashing.

[0127] In the embodiment of the present application, the mask layer and highlight mode are implemented in the same frame through underlying synchronization, effectively avoiding problems such as screen flickering or black flashing, etc. The synchronous transmission of the information (first mark) of the mark related to the mask layer and highlight mode is combined and implemented within the underlying display driver chip. The display driver chip automatically recognizes it frame by frame. When the display driver chip recognizes the first mark, that is, the display driver chip receives the data of the mask layer, it automatically controls the synchronization and turns on the highlight mode, effectively solving the problems of poor display effect and slow fingerprint recognition speed.

[0128] See also Figure 12 , which gives an example of the hardware structure of an electronic device.

[0129] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a Subscriber Identification Module (SIM) card interface 195. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0130] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0131] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0132] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0133] Processor 110 may also include a memory for storing instructions and data. In one embodiment of the present application, the memory in processor 110 is a cache memory. The memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use an instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces processor 110 latency, and thus improves system efficiency.

[0134] The fingerprint spot display method provided in the embodiments of the present application can be applied to the above-mentioned electronic devices.

[0135] The chip provided in the embodiments of the present application can implement the fingerprint spot display method in the above-mentioned method embodiments.

[0136] An embodiment of the present application provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the above fingerprint spot display method.

[0137] The embodiments of the present application further provide a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the fingerprint spot display method in the above-mentioned method embodiments.

[0138] An embodiment of the present application further provides a computer storage medium including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the fingerprint spot display method as described in the above embodiment.

[0139] Among them, the electronic device, computer storage medium, computer program product or chip system provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0140] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0143] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A fingerprint spot display method, characterized in that: Applied to an electronic device, the electronic device includes a display driver chip and a display screen, and the method includes: receiving a touch operation on the display screen; Determining image data of a target interface according to the touch operation; The display driver chip controls the display screen to display the target interface in a highlight mode based on the image data including a mask layer.

2. The method according to claim 1, wherein: The method further comprises: generating a first mark based on the image data including the mask layer; The display driver chip controls the display screen to display the target interface in a highlight mode based on the image data including the mask layer, comprising: The display driver chip obtains the first mark and the image data, and controls the display screen to display the target interface in a highlight mode based on the first mark.

3. The method according to claim 2, wherein: The generating a first mark based on the image data including the mask layer comprises: Based on the image data including the mask layer, a first mark is added to the mask layer.

4. The method according to claim 2 or 3, wherein: The electronic device further includes a display driver, and the image data of the target interface further includes a spot layer, wherein the spot layer is used to display a fingerprint spot area of the target interface, and the mask layer is used to display a mask area of the target interface excluding the fingerprint spot area; The display driver transmits the image data and the first flag to the display driver chip.

5. The method according to claim 2 or 3, wherein: The display driver chip includes a communication module, a mark recognition module, a gamma module and an image processing module; The display driver chip acquires the first mark and the image data, and controls the display screen to display the target interface in a highlight mode based on the first mark, including: The communication module acquires the first mark and the image data; The communication module transmits the first mark to the mark recognition module; The mark recognition module controls the gamma module to switch to a highlight mode based on the first mark, and controls the image processing module to display the target interface according to the image data.

6. The method according to claim 5, wherein: The method further comprises: generating a second mark based on that the image data does not include the mask layer; The mark recognition module controls the image processing module to display the target interface according to the image data based on the second mark.

7. An electronic device, characterized in that: include: processor and memory; The memory is used to store a computer program, wherein the computer program includes program instructions; The processor is configured to call the computer program to execute the method according to any one of claims 1 to 6.

8. A chip system, which is applied to electronic equipment, characterized in that: The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the method as claimed in any one of claims 1 to 6.

9. A computer storage medium, characterized in that The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product stores at least one instruction, and when the at least one instruction is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images