Handwritten input display method and related device

Abstract

Provided in the present application are a handwritten input display method and a related device. An electric device determines a dirty region of a first image frame and a dirty region of a second image frame on the basis of collected stroke information, and determines a first target refresh region on the basis of the dirty region of the first image frame and the dirty region of the second image frame; and if the first target refresh region is greater than a first threshold value and a lifting event is detected, when the next screen refresh signal is detected, the electronic device draws a third image frame on the basis of the dirty region of the first image frame, and displays the drawn third image frame. In the present application, the time interval between the occurrence of a lifting event and the arrival of the next screen refresh signal for screen content can be used, and a dirty region of an image frame in a buffer queue is consumed on the basis of a screen refresh rate, such that a dirty region of the image frame after the lifting event is not affected by the dirty region of the image frame before the lifting event, and thus the technical problem of display latency of a handwritten input is solved, thereby improving the responsiveness of handwritten input display.

Description

Handwriting input display methods and related equipment

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411196598.6, filed on August 28, 2024, entitled "Handwriting Input Display Method and Related Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of touch technology, and in particular to a handwriting input display method and related equipment. Background Technology

[0004] Nowadays, touch devices, such as smartphones, tablets, e-readers, and smartwatches, are widely used in users' daily lives. Handwriting input via fingers or styluses on the touchscreens of these devices has become a common method of human-computer interaction. When a finger or stylus touches the touchscreen, the device draws the refresh area of ​​the image frame and sends the drawing to the screen for display. However, if a full refresh of the current screen's image frame is triggered during handwriting input, subsequent image frames will also be refreshed and drawn using the entire area of ​​their respective frames. This results in excessively long drawing times for each image frame, causing the displayed handwriting to lag behind the actual position of the finger or stylus, leading to poor responsiveness in handwriting input. Summary of the Invention

[0005] In view of the above, it is necessary to provide a handwriting input display method and related equipment to reduce the latency of handwriting input display and improve the responsiveness of handwriting input display.

[0006] In a first aspect, this application provides a handwriting input display method applied to an electronic device, the electronic device including a display screen, the method comprising: in response to a handwriting input operation on the display screen, acquiring reporting information generated by the handwriting input operation; if a screen refresh signal is detected, determining a dirty region of a first image frame and a dirty region of a second image frame based on the reporting information, wherein the first image frame is displayed before the second image frame; determining a first target refresh region based on the union region of the dirty regions of the first image frame and the dirty regions of the second image frame; if the first target refresh region is greater than a first threshold and a lift-off event is detected, and in the case of detecting a next screen refresh signal, drawing a third image frame based on the dirty region of the first image frame, and displaying the drawn third image frame.

[0007] The above technical solution, when the first target refresh area is greater than a first threshold and a lift-off event is detected, and the next screen refresh signal is detected, draws a third image frame based on the dirty area of ​​the first image frame and displays the drawn third image frame. It can utilize the time between the screen content lift-off event and the next screen refresh signal to consume the dirty area of ​​the image frame in the buffer queue according to the screen refresh rate, such as consuming the dirty area exceeding a preset threshold or globally drawing the corresponding dirty area, so that the dirty area of ​​the image frame after the lift-off event is not affected by the dirty area of ​​the image frame of the previous lift-off event, thus solving the technical problem of handwriting input display delay and improving the responsiveness of handwriting input display.

[0008] In one embodiment of this application, after the first target refresh area is greater than a first threshold and a lift-up event is detected, the method further includes: drawing a second image frame based on the first target refresh area, and displaying the drawn second image frame. The above technical solution, by drawing a second image frame based on the first target refresh area and displaying the drawn second image frame after detecting a lift-up event, can refresh the screen content at the time the lift-up event occurs (e.g., the lift-up moment).

[0009] In one embodiment of this application, drawing a third image frame based on the dirty regions of a first image frame includes: determining a target dirty region based on the dirty regions of the first image frame; obtaining a second target refresh region based on the union of the target dirty region, the dirty regions of a second image frame, and the dirty regions of a preset number of first image frames preceding the second image frame; drawing a third image frame based on the second target refresh region according to the next screen refresh signal, and displaying the drawn third image frame. The above technical solution can determine the second target refresh region based on the dirty regions of the first image frame and draw the third image frame based on the second target refresh region, thereby consuming the dirty regions of image frames in the buffer queue.

[0010] In one embodiment of this application, determining the preset quantity includes: determining the preset quantity based on the number of buffers in the buffer queue of the electronic device; if the number of buffers is N, the preset quantity is N-1, where N is a positive integer greater than 1.

[0011] In one embodiment of this application, after displaying the completed third image frame, the method further includes: if a preset condition is not met, obtaining an updated second target refresh area based on the union of the target dirty area, the dirty area of ​​the second image frame, and the dirty areas of the N-2 first image frames preceding the second image frame; drawing the third image frame based on the updated second target refresh area according to the next screen refresh signal, and displaying the completed third image frame. The above technical solution, in the case where the preset condition is not met, draws the third image frame based on the updated second target refresh area according to the next screen refresh signal and displays the completed third image frame, which can continuously consume the dirty areas of image frames in the buffer queue based on the screen refresh signal.

[0012] In one embodiment of this application, the method further includes: if the second target refresh area is less than or equal to a second threshold, determining that a preset condition is met; if the second target refresh area is greater than the second threshold, determining that the preset condition is not met. The above technical solution, when the second target refresh area is greater than the second threshold, draws a third image frame based on the updated second target refresh area according to the next screen refresh signal, and displays the drawn third image frame, thus continuously consuming the dirty areas of image frames in the buffer queue based on the screen refresh signal.

[0013] In one embodiment of this application, the method further includes: if a press event is detected, determining that a preset condition is met; if no press event is detected, determining that the preset condition is not met. The above technical solution, in the case where no press event is detected, draws a third image frame based on the updated second target refresh area according to the next screen refresh signal, and displays the drawn third image frame, thus continuously consuming the dirty areas of image frames in the buffer queue based on the screen refresh signal.

[0014] In one embodiment of this application, drawing a third image frame based on the dirty regions of a first image frame includes: determining a target dirty region based on the dirty regions of the first image frame; using the target dirty region as a second target refresh region; drawing the third image frame based on the second target refresh region according to the next screen refresh signal; and displaying the drawn third image frame. The above technical solution can use the target dirty region as the second target refresh region and draw the third image frame based on the second target refresh region, thereby consuming the dirty regions of image frames in the buffer queue.

[0015] In one embodiment of this application, determining the target dirty region based on the dirty region of the first image frame includes: identifying the first image frame adjacent to the second image frame from all the first image frames, and using the dirty region of the first image frame adjacent to the second image frame as the target dirty region. This technical solution, by using the target dirty region as the second target refresh area, can reduce the rendering time of the third image frame.

[0016] In one embodiment of this application, drawing a third image frame based on the dirty area of ​​a first image frame includes: calling the view.invalidate() interface to draw the third image frame according to the second target refresh area.

[0017] In one embodiment of this application, the method further includes: setting a handwriting area for the handwriting input operation based on the reporting information generated by the handwriting input operation; and increasing the sampling rate of the display screen in the handwriting area. The above technical solution, by increasing the sampling rate of the handwriting area, can shorten the time it takes for the display screen to detect the reporting information generated by the handwriting input operation, thereby reducing the delay between detecting the reporting information and displaying the handwriting, and improving the responsiveness of the handwriting.

[0018] Secondly, this application provides an electronic device, which includes a memory and a processor: wherein the memory is used to store program instructions; and the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device performs the above-described handwriting input display method.

[0019] Thirdly, this application provides a computer storage medium that stores program instructions. When the program instructions are run on an electronic device, the electronic device executes the above-described handwriting input display method.

[0020] Fourthly, this application provides a computer program product, including computer program instructions, which, when executed by a processor, implement the above-described handwriting input display method.

[0021] Furthermore, the technical effects brought about by the second to fourth aspects can be found in the descriptions of the methods in the above-mentioned method section, and will not be repeated here. Attached Figure Description

[0022] Figure 1A is a schematic diagram of the state of the electronic device display screen and stylus provided in the embodiment of this application.

[0023] Figure 1B is a schematic diagram of the stylus provided in the embodiment of this application writing on the display screen.

[0024] Figure 2 is a schematic diagram of the process of refreshing the screen content of the display screen provided in the embodiment of this application.

[0025] Figure 3 is a schematic diagram of the buffer queue used for round-robin storage of image frames provided in an embodiment of this application.

[0026] Figure 4 is a schematic diagram of the dirty area of ​​the image frame and the screen content stored in the buffer queue provided in the embodiment of this application.

[0027] Figure 5 is a software architecture diagram of the electronic device provided in the embodiments of this application.

[0028] Figure 6 is a flowchart of the handwriting input display method provided in the embodiment of this application.

[0029] Figure 7 is a schematic diagram of the structure of the display screen provided in the embodiment of this application.

[0030] Figure 8 is a schematic diagram of the dirty area in an image frame of handwriting provided in an embodiment of this application.

[0031] Figure 9 is a schematic diagram of the dirty region of the image frame stored in the buffer queue provided in the embodiment of this application.

[0032] Figure 10 is a hardware structure diagram of the electronic device provided in an embodiment of this application. Detailed Implementation

[0033] For ease of understanding, some concepts related to the embodiments of this application are illustrated and explained by way of example.

[0034] It should be noted that the terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this application's specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. It should be understood that, unless otherwise stated, " / " in this application means "or". For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. "At least one" refers to one or more. "More than one" refers to two or more. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, and a, b, and c (seven cases).

[0036] To better understand the handwriting input display method provided in the embodiments of this application, the application scenarios of the handwriting input display method of this application will be described first below.

[0037] To better understand the handwriting input display method provided in the embodiments of this application, the application scenarios of the handwriting input display method of this application will be described first below.

[0038] In scenarios where users engage in handwriting input on electronic devices, such as drawing, marking, or typing, they typically use their fingers or styluses to touch the screen of the electronic device and move them across the screen to draw handwriting.

[0039] Referring to Figure 1A, this is a schematic diagram of the state of the electronic device display screen and stylus provided in an embodiment of this application. Taking a user drawing handwriting with a stylus as an example, when the stylus touches the display screen of the electronic device and moves on the screen, the display screen detects the reporting points generated by the stylus at a certain sampling rate, draws the reporting points, and connects adjacent reporting points to generate and display the handwriting. When the stylus touches the touch screen to perform handwriting input, the touch device draws the refresh area of ​​the image frame and sends it to the screen for display after completion to obtain the handwriting. However, when performing handwriting input on a touchscreen, especially when the stylus touches the screen, if the entire area of ​​the current screen content's image frame is refreshed, multiple subsequent image frames will also be refreshed and drawn with the entire area of ​​their respective image frames. This results in excessively long drawing times for the image frames, causing the displayed handwriting (as shown in Figure 1B, drawing point a) to be delayed compared to the actual position of the finger or pen tip (as shown in Figure 1B, actual point b). Consequently, the responsiveness of the handwriting input is poor after the pen touches the screen.

[0040] The following explanation uses a user drawing handwriting with a stylus as an example. Referring to Figure 2, when a user draws handwriting on the display screen, a touch event is generated. The screen responds to the touch event to determine the reporting information generated by the handwriting input and sends the touch event and reporting information to the application. When the application's main thread detects the first vertical synchronization signal, it responds to the touch event, determines the refresh area (or dirty area) of the image frame corresponding to the currently displayed screen content based on the reporting information, and draws on the refresh area of ​​the image frame. The refresh area of ​​the image frame is the dirty area, which refers to the image area in the image frame that needs to be drawn; it can represent the updated portion between consecutive image frames.

[0041] An application rendering thread calls OpenGL to perform hardware rendering on the GPU. OpenGL is a cross-platform graphics API that defines a standard software interface for 2D / 3D graphics processing hardware; the GPU is a powerful image processing processor. For complex graphics drawing tasks such as creating 3D models, graphic transformations, texture mapping, lighting and material settings, image enhancement features, and extended bitmap displays, hardware rendering is typically used. Because hardware rendering is performed in a separate render thread, it not only relieves pressure on the main thread but also achieves hardware acceleration.

[0042] For example, the `draw()` function for this dirty region can be recorded as a `DrawOp` in a list (`DisplayList`). This list can be given to the application rendering thread other than the main thread, which then calls OpenGL to render the image on the GPU. For example, the rendering thread retrieves the drawn image frame, renders it, and stores the rendered image frame in a buffer queue. In one embodiment of this application, the first vertical synchronization signal is an `app Vsync` signal. The `app Vsync` signal is a periodic signal, and the period interval between two `app Vsync` signals is the same as the time interval of the periodic signal corresponding to the screen refresh rate. The image frame is a frame of screen content.

[0043] Referring to Figure 3, the buffer queue contains N buffers, where N is a positive integer greater than 1. In some embodiments of this application, the buffer queue stores rendered image data in a round-robin manner. This round-robin storage of image data is represented as follows: after the N buffers in the buffer queue are filled with N image frames and each buffer has stored one image frame, the (N+1)th image frame is stored in the first buffer, the (N+2)th image frame is stored in the second buffer, and so on, until the 2Nth image frame is stored in the Nth buffer, completing one round of storage. In some embodiments of this application, the buffer queue is a circular buffer.

[0044] When the second vertical synchronization signal is detected, SurfaceFlinger retrieves an image frame from the buffer queue corresponding to the indicator flag, and updates the indicator flag to point to the next buffer in the buffer queue. The indicator flag indicates the storage location of the image frame of the screen content to be displayed. In other words, SurfaceFlinger can retrieve the image frame of the screen content to be displayed from the buffer queue based on the indicator flag.

[0045] SurfaceFlinger synthesizes the acquired image frames (e.g., single-frame synthesis) by calling the Hardware Composer HAL (HWC) and sends the synthesized image frames to the screen for display. The second vertical synchronization signal is the SurfaceFlinger Vsync signal of the timer, where the SurfaceFlinger Vsync signal is a periodic signal, and the period interval between the two SurfaceFlinger Vsync signals is the same as the time interval of the periodic signal corresponding to the screen refresh rate.

[0046] However, to avoid losing screen content, when drawing the image frame of the currently displayed screen content, the dirty region of that image frame is combined with the dirty regions of the remaining image frames stored in the buffer queue (excluding the buffer indicated by the indicator flag) to obtain a union region. The image is then drawn based on this union region, rendered, and stored in the buffer queue. For example, if the number of buffers in the buffer queue is N, after determining the image frame of the screen content to be displayed, the dirty region of this image frame is combined with the dirty region of the (N-1)th image frame stored in the buffer queue before this image frame to obtain the union region. The image frame is then drawn based on the union region, and after rendering, it is stored in the corresponding buffer in the buffer queue according to the indicator mark. This causes the dirty region of the image frame stored in the buffer queue to accumulate, resulting in a larger dirty region of the image frame and a longer time to draw the dirty region. Consequently, the image frame of the screen content to be displayed cannot be displayed on the screen according to the refresh rate, resulting in poor handwriting input responsiveness.

[0047] In a further embodiment, due to business requirements (e.g., the need to save the image content at the moment of pen lift), it is necessary to globally render the image frame of the screen content at the moment of pen lift. Global rendering refers to rendering the entire view of the screen content. The moment of pen lift refers to the point in time when the stylus or finger is lifted from the display screen, and the display screen generates a lift event based on the user's lift operation. Global rendering of the image frame corresponding to the moment of pen lift will cause the dirty areas of the N-1 image frames after the user's next pen press to be globally rendered, resulting in a longer rendering time for the N-1 image frames after the pen press and a decrease in responsiveness when displaying these N-1 image frames. The moment of pen press refers to the point in time when the stylus or finger presses the display screen, and the display screen generates a press event based on the user's press operation.

[0048] Referring to Figure 4, taking N=4 as an example, the dirty area of ​​the image frame at the moment of pen lifting is a "red man," and the dirty area corresponding to the "red man" is a globally drawn region. The dirty area of ​​the first image frame after the next pen-falling moment is a "green man," the second image frame after the pen-falling moment is a "yellow man," and the third image frame after the pen-falling moment is a "blue man." When drawing the image frames for the "green man," "yellow man," and "blue man," the dirty area must be combined with the dirty area of ​​the image frame corresponding to the "red man" before drawing. Since the dirty area of ​​the image frame corresponding to the "red man" is a globally drawn region, the dirty areas of the three images after the next pen-falling moment are all affected by the image frame at the moment of pen lifting, resulting in longer drawing times for the three images after the next pen-falling moment and poorer responsiveness when displaying them.

[0049] To address the technical problem of decreased responsiveness when using a stylus for handwriting input on a touchscreen, this application provides a handwriting input display method. This method is applied in an electronic device. Referring to Figure 5, which shows the software architecture of the electronic device provided in this application embodiment, a layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, the Android system can be divided into four layers, from top to bottom: application layer 101, framework layer 102, Android runtime and system library 103, and kernel layer 104.

[0050] The application layer can include a series of application packages. For example, application packages can include applications such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, SMS, device control services, etc.

[0051] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes predefined functions. For example, it may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0052] The window manager manages window programs. It can obtain screen size, determine the presence of a status bar, lock the screen, and capture screenshots. The content provider stores and retrieves data, making it accessible to applications. This data can include videos, images, audio, made and received calls, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. The phone manager provides communication functionality for electronic devices, such as managing call status (including connection and disconnection). The resource manager provides applications with various resources, such as localized strings, icons, images, layout files, and video files. The notification manager allows applications to display notifications in the status bar, conveying informational messages that disappear automatically after a short pause without user interaction. For example, the notification manager is used to notify of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the system's top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, causing electronic devices to vibrate, and flashing indicator lights.

[0053] The Android Runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries.

[0054] The application layer and framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0055] System libraries can include multiple functional modules. For example, a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL).

[0056] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0057] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0058] The kernel layer is the core of the operating system of an electronic device. It is the first layer of software extension based on the hardware, providing the most basic functions of the operating system. It is the foundation for the operation of the operating system, responsible for managing system threads, memory, device drivers, file systems, and network systems, and determining the system's performance and stability. For example, the kernel can determine the timing of an application's operation on a certain part of the hardware.

[0059] The kernel layer includes hardware-dependent programs such as interrupt handlers and device drivers, as well as basic, common, and frequently running modules such as clock management and thread scheduling modules, and critical data structures. The kernel layer can be located in the processor or embedded in internal memory.

[0060] Referring to Figure 6, a flowchart of a handwriting input display method provided in an embodiment of this application is shown. The method illustrated in Figure 6 includes one or more steps, but does not constitute a limitation of this application. Furthermore, the order of the steps in the method is merely an example, and the order of the steps can be changed. Additional steps may be added or steps may be reduced without departing from the disclosure of this application. The method includes the following steps.

[0061] Step S601: In response to the handwriting input operation on the display screen, collect the reporting information generated by the handwriting input operation.

[0062] In one embodiment of this application, the electronic device collects the reporting information generated by the user's handwriting input operation through the display screen (touch screen), and draws an image of the collected reporting information according to the screen refresh rate. The drawn image is then rendered and synthesized before being sent to the display screen for display.

[0063] In one embodiment of this application, handwriting input is an operation in which a user touches or moves their finger or stylus on the display screen. The stylus can be a capacitive pen, such as an active capacitive pen or a passive capacitive pen. A capacitive pen is a pen made of conductive material, possessing conductive properties, used for touching capacitive screens to complete human-computer interaction operations. It is an auxiliary device that uses conductive material to mimic the human body (usually the human finger) to complete human-computer interaction. The following explanation uses the example of a user drawing handwriting with a stylus.

[0064] Referring to Figure 7, which is a schematic diagram of the structure of a display screen provided in an embodiment of this application, in one embodiment of this application, the display screen of the electronic device is a touch screen. The display screen includes a display layer 201 and a touch layer 202. The touch layer 202 includes multiple sensing channels 203 and driving channels 204, and multiple electrodes are provided on both the sensing channels 203 and the driving channels 204.

[0065] In one embodiment of this application, if a stylus is detected touching the display screen, the display screen generates a touch event, which includes at least one of a press event and a movement event. In another embodiment of this application, if a press event or a movement event is detected, the display screen responds to the press event or movement event by collecting the reporting information generated by the handwriting input operation, and sends the press event or movement event and the reporting information to the application.

[0066] In one embodiment of this application, the acquisition of reporting information generated by handwriting input operations is performed by the kernel layer, including: acquiring the reporting information generated by handwriting input operations at a first sampling rate. The first sampling rate is the default sampling rate of the display screen, and the sampling rate is the number of times the touch layer acquires touch signals (reporting points) per second. Optionally, the first sampling rate is 240Hz.

[0067] In one embodiment of this application, the reporting information generated by handwriting input is the coordinate information of the reporting point. Users can use a stylus to perform handwriting input operations on the display screen, such as drawing lines, markings, writing characters, and drawing images. Referring to Figure 7, a two-dimensional coordinate system is constructed on the display screen, including an X-axis and a Y-axis perpendicular to the X-axis. The X-axis and Y-axis coordinates are coordinates on a coordinate system established with the edges of the display screen as coordinate axes. The acquisition of the reporting information generated by the handwriting input operation includes: when the stylus touches the display screen, the electrodes in the stylus generate a sensing capacitance with the electrodes on the sensing channel and the driving channel. The display screen forms a reporting point of the stylus at the location where the sensing capacitance is generated. The sensing channel of the touch layer reports the generated sensing capacitance to the display driving circuit. The display driving circuit obtains the X-axis coordinate of the stylus reporting point based on the capacitance change of the sensing channel. The driving channel reports the generated sensing capacitance to the display driving circuit, and the display driving circuit obtains the Y-axis coordinate of the stylus reporting point based on the capacitance change of the driving channel. The kernel layer obtains the reporting information of the stylus by acquiring the X-axis and Y-axis coordinates of the reporting points obtained by the display driver circuit, and then uploads the reporting information to the application.

[0068] In another embodiment of this application, a user can use their finger to perform handwriting input operations on the display screen, such as drawing lines, marking, writing characters, and drawing images. Collecting the reporting point information generated by the handwriting input operation includes: when the user's finger touches the display screen, the finger, being a conductor, can generate a sensed capacitance with the electrodes on the sensing channel and driving channel. The display screen forms a reporting point of the finger at the location where the sensed capacitance is generated. The sensing channel of the touch layer is used to obtain the X-axis coordinate of the reporting point, and the driving channel is used to obtain the Y-axis coordinate of the reporting point. The kernel layer obtains the reporting point information generated by the finger by acquiring the X-axis and Y-axis coordinates of the reporting point collected by the display screen.

[0069] In one embodiment of this application, the display screen may include a short edge and a long edge. If the sensing channel is set parallel to the short edge and the driving channel is set parallel to the long edge, the sensing channel acquires the X-axis coordinate of the reported point, and the driving channel acquires the Y-axis coordinate of the reported point. In another embodiment of this application, if the sensing channel is set parallel to the long edge and the driving channel is set parallel to the short edge, the sensing channel acquires the Y-axis coordinate of the reported point, and the driving channel acquires the X-axis coordinate of the reported point.

[0070] Step S602: If a screen refresh signal is detected, determine the dirty area of ​​the second image frame and N-1 dirty areas of the first image frames according to the reporting information, wherein each first image frame is displayed before the second image frame.

[0071] In one embodiment of this application, the application's main thread determines that a screen refresh signal has been detected upon detecting a first vertical synchronization signal. The application's main thread then determines the dirty regions of the second image frame and N-1 first image frames based on the reporting information. In another embodiment of this application, the second image frame is the image frame of the screen content currently to be displayed, and the N-1 first image frames are stored in a buffer queue and displayed before the second image frame.

[0072] In one embodiment of this application, determining the dirty region of the second image frame and the dirty regions of N-1 first image frames based on the reporting information includes: acquiring the reporting information of each first image frame, and determining a first rectangular region of each first image frame as the dirty region of the corresponding first image frame based on the reporting information of each first image frame; acquiring the reporting information of the second image frame, and determining a second rectangular region as the dirty region of the second image frame based on the reporting information of the second image frame;

[0073] In one embodiment of this application, determining the second rectangular region based on the reporting information of the second image frame includes: determining the maximum X-axis coordinate Xmax, the minimum X-axis coordinate Xmin, the maximum Y-axis coordinate Ymax, and the minimum Y-axis coordinate Ymin from the reporting information of the second image frame, and using the coordinate points (Xmax, Ymax), (Xmax, Ymin), (Xmin, Ymax), and (Xmin, Ymin) as the coordinates of the four vertices of the second rectangular region.

[0074] In one embodiment of this application, the specific implementation of determining the first rectangular region of each first image frame based on the reporting information of each first image frame is similar to the implementation of determining the second rectangular region based on the reporting information of the second image frame, and will not be described again here.

[0075] Step S603: Based on the union of the dirty regions of the second image frame and the dirty regions of N-1 first image frames, the first target refresh region is obtained.

[0076] In one embodiment of this application, the main thread performs a union calculation on the dirty areas of the second image frame and the dirty areas of N-1 first image frames to obtain a union region, and uses the union region as the first target refresh region. The following example uses N=4. Referring to Figure 8, which is a schematic diagram of the dirty areas of the handwriting image frame provided in this embodiment, the dirty area of ​​the second image frame of the handwriting is a rectangular region R1, and the dirty areas of the three first image frames of the handwriting are rectangular regions R2, R3, and R4, respectively. The main thread uses the union region R of rectangular regions R1, R2, R3, and R4 as the first target refresh region.

[0077] Step S604: Determine whether the refresh area of ​​the first target is greater than the first threshold.

[0078] In one embodiment of this application, the main thread determines the comparison result between the first target refresh area and the first threshold. If the first target refresh area is greater than the first threshold, step S605 is executed; if the first target refresh area is less than or equal to the first threshold, step S612 is executed. In one embodiment of this application, the first threshold can be set to a certain proportion of the view area; for example, the first threshold can be 50% of the view area of ​​an image frame.

[0079] Step S605: Determine whether a lift event has occurred. If a lift event has occurred, proceed to step S606; otherwise, proceed to step S612.

[0080] In one embodiment of this application, when the display screen detects that the stylus is not in contact with the display screen, it determines that a lift-up event has occurred and sends the lift-up event to the application.

[0081] Step S606: Draw the second image frame according to the first target refresh area, and display the drawn second image frame.

[0082] In one embodiment of this application, the application's main thread calls the view.invalidate() interface to draw a second image frame based on the first target refresh area.

[0083] In one embodiment of this application, the application rendering thread obtains the second image frame after it has been drawn by the application main thread, renders the drawn second image frame, and stores the rendered second image frame in a buffer in the buffer queue. When a second vertical synchronization signal is detected, SurfaceFlinger obtains the second image frame from the buffer queue, and synthesizes the obtained second image frame by calling HWC, and sends the synthesized second image frame to the display screen for display.

[0084] Step S607: Determine the target dirty region based on the dirty regions of N-1 first image frames.

[0085] In one embodiment of this application, upon detecting a screen refresh signal, the application's main thread determines the target dirty region based on N-1 first image frames. In another embodiment, a first image frame adjacent to the second image frame can be determined from the N-1 first image frames, and the dirty region of this adjacent first image frame is used as the target dirty region. In this embodiment, since the content difference between adjacent image frames and the second image frame is minimal before the second image frame, using the dirty region of the first image frame adjacent to the second image frame as the target dirty region by the application's main thread can reduce the rendering time of the subsequent third image frame. The third image frame is the image frame displayed on the screen after the second image frame.

[0086] In another embodiment of this application, the application main thread can use the dirty region of any one of the N-1 first image frames preceding the second image frame as the target region.

[0087] In another embodiment of this application, the application main thread can also determine the target dirty region based on N-1 first image frames when the app Vsync signal is detected.

[0088] Step S608: Based on the union of the target dirty region, the dirty region of the second image frame, and the dirty regions of the N-1 first image frames preceding the second image frame, the second target refresh region is obtained.

[0089] In one embodiment of this application, the main thread performs a union calculation on the target dirty region, the dirty region of the second image frame, and the dirty regions of the N-2 first image frames preceding the second image frame to obtain a union region, and uses the union region as the second target refresh region.

[0090] In one embodiment of this application, step S608 may further be: using the target dirty region as the second target refresh region. In this embodiment, using the target dirty region as the second target refresh region can reduce the rendering time of the subsequent third image frame.

[0091] Step S609: Draw a third image frame based on the second target refresh area according to the screen refresh rate, and display the drawn third image frame.

[0092] In one embodiment of this application, the application's main thread calls the `view.invalidate()` interface to draw a third image frame based on the second target refresh area. In another embodiment, the application rendering thread obtains the third image frame drawn by the application's main thread, renders the drawn third image frame, and stores the rendered third image frame in a buffer queue. When a second vertical synchronization signal is detected, SurfaceFlinger obtains the third image frame from the buffer queue, composites the obtained third image frame by calling HWC, and sends the composited third image frame to the display screen for display.

[0093] In this embodiment of the application, during the time when the screen content (such as handwriting content) does not change after the lift event, a third image frame is drawn based on the second target refresh area according to the screen refresh rate (or screen refresh signal), and the drawn third image frame is displayed, which can consume the dirty area of ​​the image frame in the buffer queue.

[0094] Step S610: Determine whether the electronic device meets the preset conditions. If the preset conditions are not met, proceed to step S611; if the preset conditions are met, proceed to step S601 to collect the reporting information generated by the handwriting input operation through the display screen.

[0095] Step S611: Let N = N-1, update N according to the value of N-1, and then proceed to step S608 to determine the second target refresh area to be updated.

[0096] In step S612, a second image frame is drawn according to the first target refresh area, and the drawn second image frame is displayed. After step S612 is completed, step S601 is executed to continue collecting the reporting information generated by handwriting input operations through the display screen.

[0097] In one embodiment of this application, determining whether a preset condition is met includes: if the second target refresh area is less than a second threshold, it is determined that the preset condition is met; if the second target refresh area is greater than or equal to the second threshold, it is determined that the preset condition is not met. In one embodiment of this application, the second threshold is 30% of the view area of ​​an image frame.

[0098] In one embodiment of this application, determining whether a preset condition is met includes: if a press event is detected, determining that the preset condition is met; if no press event is detected, determining that the preset condition is not met. In this embodiment of the application, if a press event is detected after a release event, it indicates that handwriting needs to be drawn based on the handwriting input operation of the stylus on the display screen. Therefore, refreshing the screen content based on the second target refresh area is stopped, and the dirty area is determined based on the user's actual handwriting input operation to refresh the screen content.

[0099] In this embodiment of the application, when it is determined that the updated second target refresh area is less than the second threshold or a pressing event is detected, the point information generated by the handwriting input operation continues to be collected through the display screen.

[0100] In the above embodiments of this application, after a lift-off event is detected, a target dirty region is determined based on N-1 first image frames. A second target refresh region is obtained based on the union of the target dirty region and the dirty regions of the second image frames. A third image frame is drawn based on the second target refresh region according to the screen refresh rate, and the drawn third image frame is displayed. Thus, after a lift-off event is detected, this application can utilize the time interval between the lift-off event and the next press event to consume the dirty regions of image frames in the buffer queue according to the screen refresh rate (or screen refresh signal). For example, it can consume dirty regions exceeding a preset threshold or globally draw the corresponding dirty regions, so that the dirty regions of image frames after the press event are not affected by the dirty regions of image frames from the previous lift-off event. This solves the technical problem of handwriting input display delay and improves the responsiveness of handwriting input display. For example, referring to Figure 9, it is a schematic diagram of the dirty regions of image frames stored in the buffer queue provided in the embodiments of this application. If the dirty area of ​​the second image frame stored in the buffer queue is the global drawing area and a lift-off event is detected, the electronic device can clear the dirty area of ​​the second image frame in the buffer queue by performing N consecutive screen refreshes based on the dirty area of ​​the third image frame after the second image frame, according to the refresh frequency.

[0101] In one embodiment of this application, the method further includes: setting a handwriting area for the handwriting input operation based on the reporting information generated by the handwriting input operation; and increasing the sampling rate of the display screen in the handwriting area according to a preset ratio.

[0102] In one embodiment of this application, the handwriting area is a partial area of ​​the display screen. Setting the handwriting area for handwriting input is executed at the application layer, specifically including: determining the movement direction of the handwriting input operation based on the reporting information generated by the handwriting input operation; setting the handwriting area for the handwriting input operation based on the movement direction of the handwriting input operation; determining the tilt direction of the stylus based on the reporting information of the stylus; and setting the handwriting area of ​​the stylus based on the tilt direction of the stylus.

[0103] In one embodiment of this application, the application layer transmits the set handwriting area to the kernel layer, and the kernel layer increases the sampling rate of the display screen in the handwriting area. Increasing the sampling rate of the display screen in the handwriting area includes increasing the sampling rate of the display screen in the handwriting area according to a preset ratio. Optionally, the increased sampling rate is 480Hz.

[0104] In one embodiment of this application, increasing the sampling rate of the display screen in the handwriting area further includes: turning off signal sampling in other areas of the display screen besides the handwriting area.

[0105] It should be noted that, assuming the display screen has 5*8 sensing and driving channels, after setting the handwriting area, the number of sensing and driving channels occupied by the handwriting area becomes 3*3. Because the number of sensing and driving channels is reduced, from a hardware perspective, the overall sampling area of ​​the display screen is reduced, and the amount of data the display screen needs to collect is reduced. Therefore, under the premise of unchanged hardware specifications, the sampling rate of the handwriting area can be improved. After improving the sampling rate of the handwriting area, the time it takes for the display screen to detect the reporting point generated by the handwriting input operation can be shortened, thereby reducing the delay between detecting the handwriting input operation, generating the reporting point, and displaying the handwriting, thus improving the responsiveness of handwriting.

[0106] Referring to Figure 10, the electronic device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose any special restrictions on the specific type of the electronic device 100.

[0107] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, Universal Serial Bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0108] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0109] Processor 110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0110] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0111] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0112] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI) interface, a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0113] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0114] The I2S interface can be used for audio communication.

[0115] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals.

[0116] The UART interface is a general-purpose serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication.

[0117] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In some embodiments, the processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0118] The GPIO interface can be configured via software. The GPIO interface can be configured as either control signals or data signals.

[0119] USB interface 130 is an interface that conforms to the USB standard specification, specifically it can be a Mini USB interface, Micro USB interface, USB Type C interface, etc.

[0120] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0121] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0122] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0123] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0124] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0125] Random access memory can include static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and double data rate synchronous dynamic random-access memory (DDR SDRAM, such as fifth-generation DDR SDRAM, which is generally called DDR5 SDRAM). Non-volatile memory can include disk storage devices and flash memory.

[0126] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multi-media card (eMMC), etc.

[0127] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0128] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0129] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0130] Internal memory 121 or external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by processor 110. The one or more computer programs include multiple instructions, which, when executed by processor 110, can implement the handwriting input display method executed on electronic device 100 in the above embodiments, so as to realize the handwriting input display function of electronic device 100.

[0131] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0132] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0133] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on the electronic device 100, the electronic device 100 performs the aforementioned related method steps to implement the handwriting input display method in the above embodiment.

[0134] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the handwriting input display method described in the above embodiment.

[0135] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the handwriting input display method in the above method embodiments.

[0136] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0139] The unit described as a separate component may or may not be physically separate. The component shown as a unit can be one physical unit or multiple physical units, that is, it can be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A handwriting input display method, applied to electronic devices, characterized in that, The electronic device includes a display screen, and the method includes: In response to a handwriting input operation on the display screen, the point reporting information generated by the handwriting input operation is collected; If a screen refresh signal is detected, the dirty areas of the first image frame and the second image frame are determined according to the reported information, wherein the first image frame is displayed before the second image frame; The first target refresh area is determined based on the union of the dirty regions of the first image frame and the dirty regions of the second image frame. If the first target refresh area is greater than the first threshold and a lift-off event is detected, and the next screen refresh signal is detected, a third image frame is drawn based on the dirty area of ​​the first image frame, and the drawn third image frame is displayed.

2. The handwriting input display method as described in claim 1, characterized in that, After the first target refresh area is greater than the first threshold and the lift-off event is detected, the method further includes: The second image frame is drawn according to the first target refresh area, and the drawn second image frame is displayed.

3. The handwriting input display method as described in claim 1, characterized in that, The step of drawing the third image frame based on the dirty area of ​​the first image frame includes: The target dirty region is determined based on the dirty region in the first image frame; The second target refresh area is obtained by combining the target dirty area, the dirty area of ​​the second image frame, and the dirty area of ​​a preset number of first image frames preceding the second image frame. Based on the next screen refresh signal, the third image frame is drawn based on the second target refresh area, and the drawn third image frame is displayed.

4. The handwriting input display method as described in claim 3, characterized in that, The determination of the preset quantity includes: determining the preset quantity based on the number of buffers in the buffer queue of the electronic device. If the number of buffers is N, the preset quantity is N-1, where N is a positive integer greater than 1.

5. The handwriting input display method as described in claim 4, characterized in that, After displaying the completed third image frame, the method further includes: If the preset conditions are not met, the updated second target refresh area is obtained based on the union of the target dirty area, the dirty area of ​​the second image frame, and the dirty areas of the N-2 first image frames preceding the second image frame. Based on the next screen refresh signal, the third image frame is drawn based on the updated second target refresh area, and the drawn third image frame is displayed.

6. The handwriting input display method as described in claim 5, characterized in that, The method further includes: If the second target refresh area is less than or equal to the second threshold, it is determined that the preset condition is met; If the second target refresh area is larger than the second threshold, it is determined that the preset condition is not met.

7. The handwriting input display method as described in claim 5, characterized in that, The method further includes: If a press event is detected, it is determined that the preset conditions are met; If the press event is not detected, it is determined that the preset condition is not met.

8. The handwriting input display method as described in claim 1, characterized in that, The step of drawing a third image frame based on the dirty area of ​​the first image frame includes: The target dirty region is determined based on the dirty region in the first image frame; The target dirty area is used as the second target refresh area; Based on the next screen refresh signal, the third image frame is drawn based on the second target refresh area, and the drawn third image frame is displayed.

9. The handwriting input display method as described in claim 3 or 8, characterized in that, The step of determining the target dirty region based on the dirty region of the first image frame includes: Identify the first image frame adjacent to the second image frame from all the first image frames, and take the dirty region of the first image frame adjacent to the second image frame as the target dirty region.

10. The handwriting input display method as described in claim 3, characterized in that, The step of drawing a third image frame based on the dirty area of ​​the first image frame includes: The third image frame is drawn by calling the view.invalidate() interface based on the second target refresh area.

11. The handwriting input display method as described in claim 1, characterized in that, The method further includes: Based on the reporting information generated by the handwriting input operation, the handwriting area of ​​the handwriting input operation is set; Increase the sampling rate of the display screen in the handwriting area.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor: The memory is used to store program instructions; The processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device performs the handwriting input display method as described in any one of claims 1 to 11.

13. A computer storage medium, characterized in that, The computer storage medium stores program instructions that, when executed on the electronic device, cause the electronic device to perform the handwriting input display method as described in any one of claims 1 to 11.

14. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed by the processor, they implement the handwriting input display method as described in any one of claims 1 to 11.

Images