Display device and writing optimization method

Abstract

The invention provides display equipment and a writing optimization method, which are used for adjusting the length of predicted handwriting based on a screen refreshing opportunity and effectively reducing handwriting delay. The display device includes a display unit and a control circuit, wherein: the display unit is configured to display content; the control circuit comprises a processor and a memory, the processor is used for reading a program in the memory and executing the following steps: receiving current handwriting information handwritten by a user, and the current handwriting information comprises a handwriting point position; the next refresh moment of the display unit is obtained, a prediction position corresponding to current handwriting information at the next refresh moment is determined, and the current handwriting information comprises handwriting information received before the next refresh moment; and determining predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and drawing and displaying the predicted handwriting of the current handwriting information.

Description

Display device and writing optimization method Technical Field

[0001] The present disclosure relates to the field of data display technology, and in particular to a display device and a writing optimization method. Background Art

[0002] In the prior art, electronic writing whiteboards are widely used in scenarios such as education, corporate meetings, and remote collaboration. However, traditional electronic writing whiteboards have the problem of large writing delay.

[0003] The performance of an electronic writing whiteboard's touch panel, display, and processor hardware can limit handwriting response speed. When hardware performance is insufficient to meet real-time response requirements, handwriting latency can increase. Large handwriting latency can cause users to experience noticeable discomfort while writing, resulting in a poor writing experience and impacting writing quality and efficiency.

[0004] Summary of the Invention

[0005] The present disclosure provides a display device and a writing optimization method for adjusting the predicted handwriting length based on the screen refresh timing, effectively reducing handwriting delay, and improving writing fluency, thereby significantly improving the user experience of an electronic writing whiteboard.

[0006] In a first aspect, an embodiment of the present disclosure provides a display device, the display device including a display unit and a control circuit, wherein:

[0007] The display unit is configured to display content;

[0008] The control circuit includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0009] receiving current handwriting information of a user's handwriting, wherein the current handwriting information includes a handwriting point position;

[0010] Obtaining a next refresh time of the display unit, and determining a predicted position corresponding to the current handwriting information at the next refresh time, wherein the current handwriting information includes handwriting information received before the next refresh time;

[0011] According to the predicted position corresponding to the current handwriting information, the predicted handwriting of the current handwriting information is determined, and the predicted handwriting of the current handwriting information is drawn and displayed.

[0012] In a second aspect, an embodiment of the present disclosure provides a display device, the display device including a display unit and a control circuit, wherein:

[0013] The display unit is configured to display content;

[0014] The control circuit includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0015] Receive the current handwriting information of the user during the refresh cycle of the display unit;

[0016] Determining the predicted handwriting corresponding to the current handwriting information at the next refresh time of the display unit, drawing and displaying the predicted handwriting of the current handwriting information;

[0017] Continue to receive the next handwriting information in the next refresh cycle, draw and display the predicted handwriting of the next handwriting information, and stop when receiving the user's lift signal.

[0018] In a third aspect, an embodiment of the present disclosure further provides a display device, the display device including a display unit and a control circuit, wherein:

[0019] The display unit is configured to display content;

[0020] The control circuit includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0021] Receiving handwriting information of a user and obtaining a refresh time of a display unit, wherein the handwriting information includes a position of a handwriting point;

[0022] Determine the predicted handwriting corresponding to the handwriting information at the refresh time, draw and display the predicted handwriting.

[0023] In a fourth aspect, the present disclosure also provides a writing optimization method, which includes:

[0024] receiving current handwriting information of a user's handwriting, wherein the current handwriting information includes a handwriting point position;

[0025] Obtaining a next refresh time of the display unit, and determining a predicted position corresponding to the current handwriting information at the next refresh time, wherein the current handwriting information includes handwriting information received before the next refresh time;

[0026] According to the predicted position corresponding to the current handwriting information, the predicted handwriting of the current handwriting information is determined, and the predicted handwriting of the current handwriting information is drawn and displayed.

[0027] In a fifth aspect, the present disclosure also provides a writing optimization method, which includes:

[0028] Receive the current handwriting information of the user during the refresh cycle of the display unit;

[0029] Determining the predicted handwriting corresponding to the current handwriting information at the next refresh time of the display unit, drawing and displaying the predicted handwriting of the current handwriting information;

[0030] Continue to receive the next handwriting information in the next refresh cycle, draw and display the predicted handwriting of the next handwriting information, and stop when receiving the user's lift signal.

[0031] In a sixth aspect, an embodiment of the present disclosure further provides a writing optimization method, the method comprising:

[0032] Receiving handwriting information of a user and obtaining a refresh time of a display unit, wherein the handwriting information includes a position of a handwriting point;

[0033] Determine the predicted handwriting corresponding to the handwriting information at the refresh time, draw and display the predicted handwriting.

[0034] In a seventh aspect, an embodiment of the present disclosure further provides a computer storage medium on which a computer program is stored, which, when executed by a processor, is used to implement the steps of the method described in any one of the above-mentioned fourth aspects.

[0035] In an eighth aspect, the present disclosure provides a computer program product, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method described in any one of the fourth aspects.

[0036] These and other aspects of the present disclosure will become more readily apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] FIG1 is a schematic diagram of handwriting prediction for an electronic writing whiteboard provided by an embodiment of the present disclosure;

[0039] FIG2 is a schematic diagram of a display device provided by an embodiment of the present disclosure;

[0040] FIG3 is a schematic diagram of drawing a method for predicting handwriting positions in real time according to an embodiment of the present disclosure;

[0041] FIG4 is a schematic diagram of a screen refresh provided by an embodiment of the present disclosure;

[0042] FIG5 is a flowchart of an implementation of writing optimization provided by an embodiment of the present disclosure;

[0043] 6A-6B are schematic diagrams of writing optimization based on refresh timing according to an embodiment of the present disclosure;

[0044] FIG7 is a schematic diagram of a refresh synchronization provided by an embodiment of the present disclosure;

[0045] FIG8 is a schematic diagram of a display device provided by an embodiment of the present disclosure;

[0046] FIG9 is a schematic diagram of another display device provided by an embodiment of the present disclosure;

[0047] FIG10 is a flowchart of an implementation of a writing optimization method provided by an embodiment of the present disclosure;

[0048] FIG11 is a schematic diagram of a writing optimization device provided by an embodiment of the present disclosure;

[0049] FIG12 is a flowchart of an implementation of a writing optimization method provided by an embodiment of the present disclosure;

[0050] FIG13 is a flowchart of an implementation of a writing optimization method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the present disclosure will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present disclosure, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are intended to fall within the scope of protection of the present disclosure.

[0052] In the embodiments of the present disclosure, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0053] The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Persons skilled in the art will appreciate that, as new application scenarios emerge, the technical solutions provided by the embodiments of the present disclosure will also be applicable to similar technical problems. In the description of the present disclosure, unless otherwise specified, "multiple" means two or more.

[0054] Before introducing the display device and writing optimization method provided by the embodiments of the present disclosure, for ease of understanding, the technical background of the embodiments of the present disclosure is first introduced in detail below.

[0055] Electronic writing whiteboards are widely used in education, corporate meetings, and remote collaboration scenarios. However, traditional electronic writing whiteboards suffer from significant handwriting latency. The hardware performance and writing principles of electronic writing whiteboards, such as the touch panel, display, and processor, can limit handwriting response speeds. When hardware performance is insufficient to meet real-time response requirements, handwriting latency can increase. As shown in Figure 1, this embodiment also provides a schematic diagram of handwriting prediction for an electronic writing whiteboard. The writing principle of an electronic writing whiteboard is as follows: the operating system acquires touch point reporting data at a fixed period, and the system also performs screen rendering and refresh at a fixed period. The touch point reporting period is independent of the screen refresh period, and the two may overlap. After the electronic whiteboard application acquires the reporting data, it constructs handwriting data and notifies the operating system to draw the constructed handwriting on the screen. However, because the operating system's reporting is a periodic event, the coordinate position contained in the reporting event may lag behind the current actual physical touch position. If handwriting prediction technology is used, that is, the next touch point position is predicted based on the current and previous groups of touch events, the prediction may be inaccurate (over-predicted or under-predicted). After a finger or pen touches the screen, the time interval between the handwriting being displayed on the screen constitutes writing delay. A large handwriting delay can cause users to experience significant discomfort during the writing process, resulting in a reduced writing experience and affecting the user's writing effect and efficiency.

[0056] As an example to solve the above technical problems, the present disclosure provides a display device and a writing optimization method, which adjusts the predicted handwriting length based on the screen refresh timing, effectively reduces handwriting delay, improves writing fluency, and thus significantly improves the user experience of the electronic writing whiteboard.

[0057] It should be noted that the display device in this embodiment refers to a large-size (generally over 50 inches) intelligent interactive display device with touch and handwriting functions; on the display device, a writing program such as an electronic whiteboard is installed, and the user can use fingers, active pens or passive pens to write on the display device. The display device can include display modules such as liquid crystal display (LCD), organic electroluminescence display (OLED), and electronic ink, which can display writing in real time.

[0058] 2 , the display device in this embodiment will be described.

[0059] The display device 100 in this embodiment includes a display unit 1040, a processor 1080 and a memory 1020, wherein the display unit 1040 includes a display panel 1041, which is used to display information input by the user or information provided to the user and various operation interfaces of the application, etc. In the embodiment of the present disclosure, it is mainly used to display the interface, shortcut window, three-dimensional menu model, menu information of menu items, etc. of the client installed in the display device 100.

[0060] Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0061] The processor 1080 is configured to read a computer program and then execute the method defined by the computer program. For example, the processor 1080 reads an application, thereby running the application on the display device 100 and displaying the application interface on the display unit 1040. The processor 1080 may include one or more general-purpose processors and may also include one or more DSPs (Digital Signal Processors) to perform related operations to implement the technical solutions provided by the embodiments of the present disclosure.

[0062] The memory 1020 generally includes internal memory and external memory. The internal memory can be a random access memory (RAM), a read-only memory (ROM), and a cache (CACHE), etc. The external memory can be a hard disk, an optical disk, a USB disk, a floppy disk or a tape drive, etc. The memory 1020 is used to store computer programs and other data. The computer program includes an application corresponding to the client, etc. Other data may include data generated after the operating system or application is run, and the data includes system data (such as configuration parameters of the operating system) and user data. In the embodiment of the present disclosure, program instructions are stored in the memory 1020, and the processor 1080 executes the program instructions in the memory 1020 to implement any one of the three-dimensional menu display methods provided by the present disclosure.

[0063] In addition, the display device 100 may further include a touch unit 1100 for receiving input digital information, word information, contact touch operations, or contactless gestures, and generating signal inputs related to user settings and function control of the display device 100. The touch unit 1100 includes, but is not limited to, an infrared touch unit, a capacitive touch unit, an electromagnetic touch unit, a camera acquisition unit, etc., wherein the camera acquisition unit is used to capture gestures of the user without touching the display screen. When the touch unit 1100 includes an infrared touch unit or an electromagnetic touch unit, the touch unit 1100 and the display unit 1040 may be stacked. For example, when a user performs a touch operation on the touch screen, the touch unit 1100 may collect the user's touch operation on or near it (such as an operation performed by the user using a finger, a stylus, or any other suitable object or accessory on the display panel 1041) and drive the corresponding connection device according to a pre-set program.

[0064] Optionally, the touch control unit 1100 may include two parts: a touch detection device and a touch processor. The touch detection device detects the user's touch direction, detects the signal caused by the touch operation, and transmits the signal to the touch processor; the touch processor receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 1080. It can also receive commands sent by the processor 1080 and execute them. In the embodiment of the present disclosure, if the user clicks on the application, the touch detection device in the touch control unit 1100 detects a touch operation, and sends the signal corresponding to the detected touch operation to the touch processor. The touch processor converts the signal into touch point coordinates and sends them to the processor 1080. The processor 1080 determines the operation that the user needs to perform based on the received touch point coordinates.

[0065] The display panel 1041 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 1040 and the touch unit 1100, the display device 100 can also include an input unit 1030. The input unit 1030 can include an image input device 1031 and other input devices 1032. The other input devices 1032 can be, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, and the like.

[0066] In addition to the above, the display device 100 may also include a power supply 1090 for powering other modules, an audio circuit 1060, a near-field communication module 1070, and an RF circuit 1010. The display device 100 may also include one or more sensors 1050, such as an accelerometer, a light sensor, a pressure sensor, etc. The audio circuit 1060 specifically includes a speaker 1061 and a microphone 1062. For example, the display device 100 can collect the user's voice through the microphone 1062 to perform corresponding operations.

[0067] As an embodiment, the number of the processors 1080 may be one or more, and the processor 1080 and the memory 1020 may be coupled or relatively independently configured.

[0068] As an embodiment, the processor 1080 is configured to perform the following steps:

[0069] Step a, receiving current handwriting information of the user, wherein the current handwriting information includes the position of the handwriting points;

[0070] In some embodiments, the processor is specifically configured to obtain current handwriting information of the user through the application layer.

[0071] During implementation, the system touch point reporting event cycle is fixed, but since the system needs to perform unified point reporting processing before sending it to the writing application (such as the Android system needs to be processed by the Framework (framework layer) and then sent to the application layer), it should be noted that point reporting refers to the display unit sensor collecting the user's touch data, filtering the noise, and transmitting it to the system's framework layer. The framework layer generates touch events based on the touch data transmitted by the sensor, and uses the touch events to convert them into point reporting events. The point reporting events include the handwriting point position matching the screen, such as the handwriting point coordinates, the handwriting point touch time, touch size, touch device (stylus), touch ID and other point reporting information.

[0072] Since touch events are currently generated through the Framework and converted into point reporting events and then distributed to the application layer, the present disclosure provides a point reporting acceleration solution, which directly obtains the user's handwriting information through the application layer, thereby reducing the cycle time for the application layer to finally obtain the point reporting.

[0073] In some embodiments, this embodiment provides a point reporting acceleration solution, the specific process is as follows:

[0074] (a1) reading touch events through the application layer and converting the touch events into touch point reporting events;

[0075] (a2) determining, based on the handwriting point information included in the touch point reporting event, that current handwriting information of the user's handwriting has been received;

[0076] The touch event is generated based on a touch signal generated by user handwriting, and the touch point reporting event includes handwriting point information corresponding to the touch signal.

[0077] During implementation, the application layer directly reads the touch events under the system path ( / dev / input) and converts the touch events into touch point reporting events such as coordinates, time, size, touch device, touch ID, etc. that match the screen, thereby allowing the Android application to bypass the processing logic of the Framework layer and directly obtain the touch point reporting information (i.e., handwriting point information), reducing the delay in system point reporting, so that the application layer can directly obtain touch events.

[0078] During implementation, the touch events in this embodiment are briefly explained. Touch events (TouchEvent) are a type of event that describes the state change of a finger or stylus on a touch plane (such as a touch screen, touchpad, etc.). This type of event is used to describe one or more touch points (or handwriting points), and can detect the movement of touch points, the increase and decrease of touch points, etc. Each touch (Touch) object represents a touch point, and each touch point is described by its position, size, shape, pressure level and target element. The touch list (TouchList) object represents a list of multiple touch points. Touch events also provide synchronous touches at different locations on the same surface. Touch events include three interface objects: TouchEvent: represents an event generated when a state of a touch point on the surface changes; Touch: represents a separate contact point between the user and the touch surface; TouchList: represents a group of Touch, used for multi-touch situations.

[0079] The touch events in this embodiment include touch events generated by a user performing handwriting touch on the display unit, touch events generated by a user performing single-point or multi-point touch on the display unit, and the like.

[0080] In some embodiments, this embodiment provides another solution for accelerating the reporting of points. The specific process is as follows:

[0081] (b1) using cross-process communication to receive touch events transmitted by the framework layer through the application layer, and converting the touch events into touch point reporting events;

[0082] (b2) determining, based on the handwriting point information included in the touch point reporting event, that current handwriting information of the user's handwriting has been received;

[0083] During implementation, touch events can be obtained through the InputFlinger service of the framework layer, bypassing the processing of touch events by the InputFlinger service of the framework layer. Cross-process communication (framework layer and application layer) such as pipelines can be directly used to enable the framework layer to directly transfer data with the application layer. The application layer directly obtains the touch events, thereby converting the touch events into touch point reporting events, thereby reducing the delay in system event distribution and touch point reporting.

[0084] InputFlinger consists of two important classes that handle the reading and dispatching of all events. The InputReader class reads input events generated by all devices (including single-touch, multi-touch, keystrokes, electromagnetic pens, etc.). The InputDispatcher class is responsible for distributing events. In this embodiment, cross-process communication is achieved through channels created at the Linux layer.

[0085] Step b, obtaining the next refresh time of the display unit, and determining the predicted position corresponding to the current handwriting information at the next refresh time, wherein the current handwriting information includes the handwriting information received before the next refresh time;

[0086] It should be noted that the refresh time in this embodiment refers to the time when the hardware of the display unit is refreshed. Optionally, the refresh process of the display unit is usually from top to bottom, vertically refreshing, so the refresh time of the display unit includes the refresh time when the vertical refresh position is the upper left corner origin (0, 0) when the display unit starts vertical refreshing, corresponding to the vertical refresh position (0, 0). Among them, this embodiment does not impose too many restrictions on the specific refresh method of the display unit. The above-mentioned vertical refresh method is only an example, and multiple refresh methods such as horizontal refresh and left-to-right refresh may also be included.

[0087] In some embodiments, each refresh time of the display unit is determined by:

[0088] Acquire a refresh event and a refresh cycle received by the display unit before the next refresh time, wherein the refresh event includes at least one refresh time of the display unit refresh;

[0089] The next refresh time of the display unit is determined according to the at least one refresh time and the refresh cycle.

[0090] In practice, the refresh time can be recorded only once after power-on, and the next refresh time can be calculated based on the refresh cycle and the number of refreshes. Alternatively, each refresh time can be recorded, and the next refresh time can be calculated based on the refresh cycle and the current refresh time. This embodiment does not impose any restrictions on the specific method of calculating the refresh time. The refresh cycle is calculated based on the refresh rate of the current display unit, and the refresh rate of the current display unit is known.

[0091] During implementation, the refresh event can be obtained from the system operation layer, for example, the vertical synchronization VSync (Vertical Synchronization) event can be directly obtained from the system operation layer, and the vertical synchronization event can be determined as a refresh event; or, the vertical synchronization VSync event can be obtained from the writing application, and the vertical synchronization event can be determined as a refresh event. The Android system can obtain the refresh event notification of the display unit through Choreographer. Optionally, the above-mentioned writing optimization method can also be used as a functional module of the writing application. The writing application obtains the refresh event through the functional module, and the fixed refresh time of the display unit can be obtained in combination with the refresh rate of the current display unit (such as the refresh cycle of a 60Hz display unit is 16.6ms). Among them, the vertical synchronization VSync event is used to provide a signal to the outside before the screen is refreshed. The host side selects an appropriate strategy to complete the refresh of the screen based on this signal to avoid the mismatch between data refresh and screen scanning.

[0092] After the refresh event is obtained, the next refresh time T is calculated according to the refresh time and refresh cycle of the current display unit. n , if the current refresh time is T and the refresh rate is 60Hz, then the next refresh time is T n =T+16.6ms.

[0093] In some embodiments, the processor is specifically configured to determine the predicted position corresponding to the current handwriting information at the next refresh time based on the reception time of the current handwriting information and the next refresh time.

[0094] In some embodiments, the predicted position of the current handwriting information is determined by the following steps:

[0095] b1) determining an initial predicted position corresponding to the current handwriting information at the next refresh time according to the reception time of the current handwriting information and the next refresh time;

[0096] The receiving time of the current handwriting information refers to the time when the application layer receives the touch point reporting event report, that is, when the touch point reporting event is generated, the time when the touch point reporting event is reported to the application layer is used as the receiving time of the current handwriting information in this embodiment.

[0097] Optionally, initialize and configure the whiteboard writing application, and set the corresponding touch event listener and other parameters. When a finger or stylus touches the display unit (screen), the touch sensor receives the touch data (handwriting information) and feeds the data back to the writing application. If more than three touch points have been received, linear interpolation or Kalman filtering can be used to combine the previous handwriting information (including the xy coordinates, timestamps, etc. of the current handwriting point and the previous N handwriting points) to predict the initial predicted position (predict_x, predict_y) of the current handwriting information when it moves from the reception time to the next refresh time.

[0098] b2) determining an adjustment factor based on the movement angles of N pieces of handwriting information preceding the current handwriting information;

[0099] Optionally, N-1 handwriting vectors are determined based on N handwriting information before the current handwriting information; a moving angle is determined based on the angle between the N-1 handwriting vectors, and an adjustment factor is determined based on the moving angle.

[0100] In practice, the adjustment factor can be determined by the following steps:

[0101] According to the positions of the first three handwriting points (including the current handwriting point), the vector vec1 from the first handwriting point to the second handwriting point and the vector vec2 from the second handwriting point to the third handwriting point are calculated respectively, and the moving angle angle of the three handwriting points is calculated by the two vectors;

[0102] According to the moving angles angle of the three handwriting points, the curvature-based adjustment factor F is calculated. A nonlinear function can be used to calculate the value, such as: F = 1 / (1+k×angle1 / π); where k represents the preset coefficient and angle1 represents the moving angle.

[0103] b3) adjusting the initial predicted position according to the adjustment factor to determine the predicted position corresponding to the current handwriting information at the next refresh time.

[0104] During implementation, the predicted initial position of the handwriting point is adjusted according to the adjustment factor based on the curvature, and the predicted position corresponding to the current handwriting information at the next refresh time is obtained.

[0105] In some embodiments, when a touch point event is received in the current refresh cycle, the handwriting prediction direction can be calculated and the next screen refresh time T n The difference ΔT between the triggering moment of the current touch point event (which can also be understood as the moment when the application layer receives the current handwriting information) is used to predict the predicted position of the current track point at the next refresh time. For example, the moving speed v of the current handwriting information can be used to calculate the position of the next screen refresh time Tn In this case, the touch distance to be extended in the handwriting prediction direction is L=v×ΔT, thereby predicting the predicted position corresponding to the current handwriting information at the next refresh time.

[0106] Step c: determining the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and drawing and displaying the predicted handwriting of the current handwriting information.

[0107] In some embodiments, the processor is specifically configured to: determine the actual handwriting of the current handwriting information based on the current handwriting information, and display the actual handwriting. In implementation, the actual handwriting of the current handwriting information refers to the handwriting formed by the actual touch point of the user between the previous handwriting information and the current handwriting information.

[0108] In some embodiments, the actual handwriting corresponding to the previous handwriting information is determined based on the previous handwriting information and the current handwriting information; the predicted handwriting corresponding to the previous handwriting information is cleared, and the actual handwriting corresponding to the previous handwriting information is drawn and displayed.

[0109] In practice, the actual handwriting corresponding to the previous handwriting information refers to the actual handwriting from the previous handwriting information to the current handwriting information, that is, the user's actual touch handwriting. After the actual handwriting is determined, it is first drawn through off-screen rendering, and then replaces the previously predicted handwriting of the previous handwriting information.

[0110] During implementation, actual handwriting points can be used to construct an actual handwriting trajectory, and the actual handwriting can be drawn on an off-screen rendered handwriting image in memory. As shown in Figure 3, this embodiment also provides a schematic diagram for drawing a real-time predicted handwriting position, which is used to predict the position of handwriting in real time, reduce the difference between actual touch and predicted handwriting, and effectively reduce handwriting latency; optimize the distance between predicted points by adjusting the curvature of the touch point trajectory to improve prediction accuracy; and adopt a cyclical method of predicting and correcting handwriting to improve the fluency and accuracy of writing, thereby enhancing the user experience of the electronic writing whiteboard. After obtaining the predicted handwriting corresponding to the current handwriting point (i.e., the next predicted handwriting in the figure), the actual handwriting trajectory from the previous handwriting information to the current handwriting information is drawn through off-screen rendering, i.e., the next actual handwriting in the figure (the user's actual touch trajectory data). If there is a predicted handwriting corresponding to the previous handwriting information on the current screen, the rectangular area involved in the predicted handwriting corresponding to the previous handwriting information predicted at the time of the last touch on the screen and the rectangular area involved in the actual handwriting from the previous handwriting information to the current handwriting information are processed, and the pixels of these two areas are cleared; the pixels of these two rectangular areas are replaced with the pixels of the actual handwriting currently rendered off-screen. The predicted handwriting of the current handwriting information (i.e., the new predicted handwriting in the figure) is drawn in real time. At this time, the drawing process of a touch event is completed, and the next handwriting point is waiting to be received. When the finger or stylus leaves the screen, the last actual handwriting point is connected with the previous actual handwriting point to generate a complete actual handwriting.

[0111] It should be noted that the off-screen rendering (Off-Screen Rendering) in this embodiment, as the name implies, refers to the GPU (Graphics Processing Unit) opening a new buffer outside the current screen buffer for rendering operations. If we want to display content on the display screen, we need at least a frame buffer as large as the screen pixel data volume as a pixel data storage area, and this is also where the GPU stores the rendering results. If sometimes due to some limitations, the rendering results cannot be written directly to the frame buffer, but are temporarily stored in another memory area and then written to the frame buffer, then this process is called off-screen rendering. The rendering result first passes through the off-screen buffer and then to the frame buffer.

[0112] During implementation, since drawing work is performed after each touch reporting event, and since the refresh cycle is different from the touch reporting cycle and does not necessarily occur synchronously, the screen refresh event and the touch reporting event may be staggered, or multiple touch reporting events may occur in one screen refresh cycle, and only the result of the last drawing will be displayed in one screen refresh cycle, which may lead to multiple unnecessary handwriting drawing events; on the other hand, handwriting drawing takes a certain amount of time, such as E ms. If the screen refresh event is triggered when the handwriting is being drawn, it may cause the handwriting currently being drawn to be completely or partially unable to be displayed on the screen, and the current part of the handwriting needs to wait for the next screen refresh event to be displayed on the screen, which will increase the writing delay experience. If the screen refresh rate is 60Hz, the writing delay of the current handwriting will increase by 16.6ms.

[0113] In some embodiments, this embodiment dynamically adjusts screen rendering work based on drawing timing, and the processor is specifically configured to execute:

[0114] Determine a drawing time of the predicted handwriting of the current handwriting information, wherein the drawing time is before the next refresh time; when the drawing time is reached, draw the predicted handwriting of the current handwriting information.

[0115] In practice, the prior art immediately draws the predicted handwriting after calculating the predicted position. However, in the present disclosure, after determining the predicted position corresponding to the current handwriting information at the next refresh time and performing off-screen rendering of the actual handwriting, the drawing operation is not performed immediately. The off-screen rendering of the actual handwriting includes clearing the predicted handwriting corresponding to the previous handwriting point and drawing the actual handwriting from the previous handwriting point to the current handwriting point through off-screen rendering, but the drawing is not immediately displayed. When the drawing time is reached, the actual handwriting obtained by off-screen rendering is displayed, and the predicted handwriting of the current handwriting information is drawn.

[0116] In some embodiments, the drawing time is determined as follows:

[0117] (1) Determine the rendering time of the predicted handwriting of the current handwriting information;

[0118] During implementation, the rendering time includes at least one of the time required to calculate the predicted position and predicted handwriting, the time required to clear the predicted handwriting corresponding to the previous handwriting information, the time required to draw the actual handwriting from the previous handwriting information to the current handwriting information off-screen, and the time required to draw the predicted handwriting.

[0119] In some embodiments, the rendering duration is determined by any one or more of the following methods:

[0120] Method 1a: determining the rendering duration according to a preset duration;

[0121] In implementation, the rendering duration can be set to a fixed preset duration.

[0122] Mode 1b: Count the historical rendering durations corresponding to multiple historical handwriting information before the current handwriting information, perform smoothing filtering on the multiple historical rendering durations, and determine the rendering duration according to the smoothing filtering result.

[0123] During implementation, the historical rendering durations corresponding to N historical note information can be counted, and the average of the N historical rendering durations can be taken, and the average value can be determined as the rendering duration. Optionally, when the number of received handwriting information is less than N, the preset duration is determined as the rendering duration. When the number of received handwriting information is greater than or equal to N, the N consecutive historical handwriting information before the current handwriting information is used to calculate the average of the historical rendering durations corresponding to the N historical handwriting information, and the average value can be used as the rendering duration.

[0124] (2) Determine the drawing time of the predicted handwriting of the current handwriting information according to the rendering time and the next refresh time, wherein the drawing time is before the next refresh time.

[0125] When the drawing time is between the current refresh time and the next refresh time, the drawing time decreases as the rendering time increases.

[0126] In practice, drawing time = next refresh time - rendering time. For example, if the rendering time is E (ms) and the next refresh time is T, then drawing time = TE; if the rendering time is E+1 (ms) and the next refresh time is T, then drawing time = TE-1. That is, the longer the rendering time, the earlier the drawing time.

[0127] In some embodiments, when the drawing time is between the receiving time of the current handwriting information and the next refresh time, the drawing time is advanced as the rendering time increases.

[0128] In some embodiments, if the time interval between the receiving time of the current handwriting information and the next refresh time is less than or equal to the rendering time, the predicted handwriting of the current handwriting information is drawn after the next refresh time.

[0129] In some embodiments, if the time interval between the receiving time of the current handwriting information and the next refresh time is less than or equal to the rendering time, the predicted handwriting of the current handwriting information is drawn after the next refresh time.

[0130] During implementation, if the time interval between the reception of the current handwriting information (i.e., the triggering time of the touch reporting event) and the next refresh time is less than or equal to the rendering time, the predicted handwriting of the current handwriting information can be cached and drawn after the next refresh time. The predicted handwriting of the cached current handwriting information can be drawn a millisecond (e.g., 2 to 3ms) after the next refresh time to avoid starting the rendering work while the screen refresh is still in progress at the current position due to the operating environment.

[0131] If the time interval between the receiving time of the current handwriting information and the next refreshing time is greater than the rendering time, the predicted handwriting of the current handwriting information can be drawn.

[0132] In some embodiments, the timing of drawing the predicted handwriting is as follows:

[0133] If the drawing time is earlier than the receiving time of the current handwriting information, the predicted handwriting of the current handwriting information is drawn after the next refresh time.

[0134] In some embodiments, the timing of drawing the predicted handwriting is as follows:

[0135] If the next handwriting information is received between the drawing time and the next refresh time, the predicted handwriting of the next handwriting information is drawn after the next refresh time.

[0136] In practice, after the refresh event is triggered, the next refresh time T is calculated based on the current refresh rate (e.g., the refresh cycle of a 60Hz screen is 16.6ms). n At the same time, the time required for single prediction of handwriting drawing is set to T ms. n -E; when the above timing is triggered, draw the actual handwriting between the previous handwriting information and the current handwriting information, and draw the predicted handwriting of the current handwriting information at the next refresh time.

[0137] If in T n -E moment and T nWhen a new touch event is received between moments, the handwriting information corresponding to the new touch event to be drawn cannot be displayed on the screen at the next refresh moment when the drawing operation is performed at this moment. Therefore, the drawing operation generated by the new touch event will take effect after the next refresh cycle (and no new touch event is triggered in the next refresh cycle); that is, when the predicted handwriting is being drawn in the current refresh cycle, a new touch event is received, and the drawing operation generated by the new touch event takes effect after the next screen refresh cycle. After the predicted handwriting of the current handwriting information is drawn, the next predicted handwriting information is received after the next touch event is triggered, the predicted handwriting of the current handwriting information is cleared, and the actual handwriting of the current handwriting information to the next predicted handwriting information is drawn off-screen, and the predicted handwriting corresponding to the next predicted handwriting information is determined, and the actual handwriting is displayed, and the predicted handwriting corresponding to the next predicted handwriting information is drawn. The above steps start from the user's touch press, and are executed repeatedly according to the refresh cycle and the touch reporting cycle, and the handwriting prediction and actual handwriting are corrected in real time until the touch is lifted and stops.

[0138] In actual application, the writing delay can be reduced through the above steps. However, when writing on the upper half of the screen (display unit), the writing delay is relatively consistent with the expected result. However, when writing on the lower half of the screen, the writing delay is significantly different from that on the upper half of the screen, and the writing delay increases as the touch position moves downward. As shown in Figure 4, this embodiment provides a screen refresh schematic diagram. Since the screen refresh action occurs during the entire screen refresh cycle, generally speaking, the screen refresh process is from top to bottom, which is a vertical refresh cycle. A refresh event is generated after a refresh cycle is completed. Therefore, the refresh event used in the above process is the start refresh time corresponding to the top of the screen. The actual screen refresh time corresponding to different positions will increase with the distance between the position and the top of the screen. The difference between the time represented by the screen refresh event increases; the change in the actual physical touch position during this increased time difference will be reflected in the final measured writing delay. Therefore, when writing at the top and bottom of the screen, the writing delay difference can be as large as one screen cycle (for example, if the screen refresh rate is 60Hz, this difference is 16.6ms).

[0139] Based on this, this embodiment provides a process for determining the predicted position based on the position and the next refresh time. The predicted handwriting length is dynamically determined by corresponding the touch events at different positions on the screen to the screen refresh time. The specific determination process is as follows:

[0140] Determine the refresh time increment according to the position of the current handwriting information on the display unit;

[0141] According to the refresh time increment and the next refresh time, the predicted position corresponding to the current handwriting information at the next refresh time is determined.

[0142] In some embodiments, the farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the vertical direction, the larger the refresh time increment; and / or, the farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the horizontal direction, the larger the refresh time increment.

[0143] In some embodiments, the predicted position corresponding to the current handwriting information at the next refresh time is determined according to the refresh time increment and the next refresh time in the following manner:

[0144] Determining a predicted time based on the next refresh time and the refresh time increment;

[0145] Determine the predicted position corresponding to the current handwriting information at the predicted time;

[0146] The predicted position corresponding to the current handwriting information at the predicted time is determined as the predicted position corresponding to the current handwriting information at the next refresh time.

[0147] In implementation, if the current handwriting information is at the coordinate origin of the display unit, the predicted position corresponding to the current handwriting information is directly determined using the next refresh time.

[0148] In some embodiments, the refresh time increment may be determined by any one or more of the following methods:

[0149] 1) Determine the refresh time increment based on the ratio of the vertical coordinate of the current handwriting information to the height of the display unit and the refresh cycle of the display unit;

[0150] 2) Determine the refresh time increment based on the ratio of the horizontal coordinate of the current handwriting information to the width of the display unit and the refresh cycle of the display unit.

[0151] In the implementation, it is assumed that the width of the display unit (screen) is W, the height is H, the origin of the screen coordinates is the top corner (0, 0), and the screen refresh cycle is R ms; let the time represented by the screen refresh event in this screen refresh cycle be T s , this T s This is the moment when the screen starts to refresh vertically, and the corresponding vertical position of the screen is 0. When writing, after obtaining the currently triggered touch point event, the actual vertical refresh time T corresponding to the current handwriting information is calculated based on the y coordinate of the current handwriting information. an =T s +R×y / H.

[0152] The next refresh time is calculated and predicted through the above steps, and the predicted position corresponding to the current handwriting information is calculated, wherein the adjusted next refresh time T is used. an Replace the next refresh time T used in the calculationn , to ensure that the predicted handwriting length is dynamically adjusted to coincide with the next screen refresh time corresponding to the current position; the adjusted next refresh time T is used in the process of drawing operation timing an By calculating the predicted handwriting drawing time based on the current handwriting information and the rendering time, the current writing position refresh time can be maximized to fit the correct time in the screen refresh cycle.

[0153] To sum up, the present invention can adjust the length of the predicted handwriting based on the screen refresh timing on the one hand, and dynamically adjust the screen rendering work through timed drawing moments on the other hand. Finally, it can dynamically adjust the predicted handwriting length by corresponding the screen refresh moments to the touch events at different positions on the screen.

[0154] As shown in FIG5 , this embodiment provides an implementation process of writing optimization, and the specific implementation process is as follows:

[0155] Step 500: obtaining the current handwriting information of the user through the application layer;

[0156] Optionally, the touch event is read through the application layer, or the touch event transmitted by the framework layer is received through the application layer by using cross-process communication, and the touch event is converted into a touch point reporting event; based on the handwriting point information contained in the touch point reporting event, the current handwriting information of the user's handwriting is determined to be received.

[0157] Step 501: Obtain a refresh event and a refresh cycle of a display unit, wherein the refresh event includes the current refresh time, and determine the next refresh time of the display unit according to the current refresh time and the refresh cycle;

[0158] Step 502: Determine a refresh time increment based on the position of the current handwriting information on the display unit; and determine a predicted position corresponding to the current handwriting information at the next refresh time based on the refresh time increment and the next refresh time.

[0159] Step 503: Determine an adjustment factor based on the movement angles of the N handwriting information preceding the current handwriting information, adjust the initial predicted position based on the adjustment factor, and determine the predicted position corresponding to the current handwriting information at the next refresh time;

[0160] Step 504: Determine the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information;

[0161] Step 505: determining the rendering time of the predicted handwriting of the current handwriting information, and determining the drawing time of the predicted handwriting of the current handwriting information according to the rendering time, the refresh time increment and the next refresh time;

[0162] The longer the rendering time is, the earlier the drawing time is.

[0163] Step 506: determining the actual handwriting between the previous handwriting information and the current handwriting information based on the previous handwriting information and the current handwriting information; clearing the predicted handwriting of the previous handwriting information, and drawing and displaying the actual handwriting between the previous handwriting information and the current handwriting information;

[0164] Step 507: When the drawing time arrives, the actual handwriting obtained by off-screen rendering is displayed, and the predicted handwriting of the current handwriting information is drawn;

[0165] Repeat steps 500 to 507 until it is detected that the user touches the device and stops.

[0166] As shown in Figures 6A and 6B, this embodiment also provides a schematic diagram of writing optimization based on refresh time, wherein the next refresh time is calculated based on the current refresh time and the refresh cycle / frequency, a touch point reporting event is generated within the current refresh cycle, the handwriting information of the user is received, the rendering time E (ms) of the predicted handwriting of the current handwriting information is determined, the drawing time is determined based on the rendering time and the next refresh time, and when the drawing time is reached, the predicted handwriting is started to be drawn, and when the next refresh time is reached, the completed predicted handwriting is synchronously displayed, and the touch point reporting event continues to be generated within the next refresh cycle, the next handwriting information is received, and the drawing time of the predicted handwriting of the next handwriting information continues to be determined. When the drawing time of the predicted handwriting of the next handwriting information is reached, the predicted handwriting is started to be drawn, and the above process is repeated until it stops when a signal that the user touches up is received.

[0167] As shown in Figure 7, this embodiment also provides a refresh synchronization diagram, where the refresh period is R (ms), the touch reporting period is T (ms), and the rendering time is E (ms). By obtaining the screen refresh timing in real time, the predicted handwriting drawing length is dynamically adjusted, and the distance between the rendered handwriting and the screen touch position when the screen is refreshed is reduced, effectively reducing writing delays; by obtaining the screen refresh timing in real time, the screen rendering work is dynamically adjusted to reduce unnecessary drawing work, thereby reducing the number of screen refresh opportunities missed due to time-consuming rendering operations, and effectively improving the problem of writing pauses caused by unrendered cycles; by dynamically adjusting the predicted handwriting length according to the screen refresh time corresponding to the touch events at different positions on the screen, the distance between the rendered handwriting and the screen touch position when the screen is refreshed is reduced, effectively reducing writing delays.

[0168] Based on the same inventive concept, the embodiment of the present disclosure further provides a display device, which solves the problem based on a principle similar to that of the above-mentioned display device. Therefore, the implementation of the display device can refer to the above-mentioned implementation, and the repeated parts will not be repeated.

[0169] As shown in FIG8 , the display device includes a display unit 800 and a control circuit 801 , wherein:

[0170] The display unit is configured to display content;

[0171] The control circuit includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0172] Receive the current handwriting information of the user during the refresh cycle of the display unit;

[0173] Determining the predicted handwriting corresponding to the current handwriting information at the next refresh time of the display unit, drawing and displaying the predicted handwriting of the current handwriting information;

[0174] Continue to receive the next handwriting information in the next refresh cycle, draw and display the predicted handwriting of the next handwriting information, and stop when receiving the user's lift signal.

[0175] Based on the same inventive concept, the embodiment of the present disclosure also provides another display device, which solves the problem based on a similar principle to the above-mentioned display device. Therefore, the implementation of the display device can refer to the above-mentioned implementation, and the repeated parts will not be repeated.

[0176] As shown in FIG9 , the display device includes a display unit 900 and a control circuit 901 , wherein:

[0177] The display unit is configured to display content;

[0178] The control circuit includes a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0179] Receiving handwriting information of a user and obtaining a refresh time of a display unit, wherein the handwriting information includes a position of a handwriting point;

[0180] Determine the predicted handwriting corresponding to the handwriting information at the refresh time, draw and display the predicted handwriting.

[0181] Based on the same inventive concept, the embodiment of the present disclosure further provides a writing optimization method, as shown in FIG10 . The specific implementation process of the method is as follows:

[0182] Step 1000: Receive current handwriting information of the user, wherein the current handwriting information includes the position of the handwriting points;

[0183] Step 1001: Obtain the next refresh time of the display unit, and determine the predicted position corresponding to the current handwriting information at the next refresh time, wherein the current handwriting information includes the handwriting information received before the next refresh time;

[0184] Step 1002: Determine the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and draw and display the predicted handwriting of the current handwriting information.

[0185] As an optional implementation, the method further includes:

[0186] The actual handwriting of the current handwriting information is determined according to the current handwriting information, and the actual handwriting is displayed.

[0187] As an optional implementation, the next refresh time is determined by:

[0188] Acquire a refresh event and a refresh cycle received by the display unit before the next refresh time, wherein the refresh event includes at least one refresh time of the display unit refresh;

[0189] The next refresh time of the display unit is determined according to the at least one refresh time and the refresh cycle.

[0190] As an optional implementation manner, determining the predicted position corresponding to the current handwriting information at the next refresh time includes:

[0191] According to the receiving time of the current handwriting information and the next refreshing time, the predicted position corresponding to the current handwriting information at the next refreshing time is determined.

[0192] As an optional implementation manner, determining the predicted position corresponding to the current handwriting information at the next refresh time according to the reception time of the current handwriting information and the next refresh time includes:

[0193] Determine the initial predicted position of the current handwriting information at the next refresh time according to the reception time of the current handwriting information and the next refresh time;

[0194] Determine an adjustment factor based on the movement angles of N handwriting information preceding the current handwriting information;

[0195] The initial predicted position is adjusted according to the adjustment factor to determine the predicted position corresponding to the current handwriting information at the next refresh time.

[0196] As an optional implementation manner, the step of drawing the predicted handwriting of the current handwriting information includes:

[0197] Determining a predicted handwriting drawing time of current handwriting information, wherein the drawing time is before the next refresh time;

[0198] When the drawing time is reached, the predicted handwriting of the current handwriting information is drawn.

[0199] As an optional implementation manner, determining the predicted handwriting drawing time of the current handwriting information includes:

[0200] Determine the rendering time of the predicted handwriting of the current handwriting information;

[0201] The drawing time of the predicted handwriting of the current handwriting information is determined according to the rendering time and the next refresh time.

[0202] As an optional implementation,

[0203] When the drawing time is between the receiving time of the current handwriting information and the next refreshing time, the drawing time is advanced as the rendering time increases.

[0204] As an optional implementation manner, the determining of the predicted handwriting drawing time of the current handwriting information according to the rendering duration and the next refresh time further includes:

[0205] If the time interval between the reception time of the current handwriting information and the next refresh time is less than or equal to the rendering time, the predicted handwriting of the current handwriting information is drawn after the next refresh time.

[0206] As an optional implementation manner, the step of drawing the predicted handwriting of the current handwriting information further includes:

[0207] If the drawing time is earlier than the receiving time of the current handwriting information, then after the next refresh time, the predicted handwriting of the current handwriting information is drawn; and / or,

[0208] If the next handwriting information is received between the drawing time and the next refresh time, the predicted handwriting of the next handwriting information is drawn after the next refresh time.

[0209] As an optional implementation manner, determining the rendering duration of the predicted handwriting of the current handwriting information includes:

[0210] Determine the rendering duration according to a preset duration; and / or,

[0211] The historical rendering durations corresponding to a plurality of historical handwriting information before the current handwriting information are counted, and the rendering duration is determined according to the plurality of historical rendering durations.

[0212] As an optional implementation, the method further includes:

[0213] determining the actual handwriting between the previous handwriting information and the current handwriting information according to the previous handwriting information and the current handwriting information;

[0214] Clear the predicted handwriting of the previous handwriting information, draw and display the actual handwriting between the previous handwriting information and the current handwriting information.

[0215] As an optional implementation manner, determining the predicted position corresponding to the current handwriting information at the next refresh time includes:

[0216] Determine the refresh time increment according to the position of the current handwriting information on the display unit;

[0217] According to the refresh time increment and the next refresh time, the predicted position corresponding to the current handwriting information at the next refresh time is determined.

[0218] As an optional implementation,

[0219] The farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the vertical direction is, the larger the refresh time increment is; and / or,

[0220] The farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the horizontal direction is, the greater the refresh time increment is.

[0221] As an optional implementation manner, determining the predicted position corresponding to the current handwriting information at the next refresh time according to the refresh time increment and the next refresh time includes:

[0222] Determining a predicted time based on the next refresh time and the refresh time increment;

[0223] Determine the predicted position corresponding to the current handwriting information at the predicted time;

[0224] The predicted position corresponding to the current handwriting information at the predicted time is determined as the predicted position corresponding to the current handwriting information at the next refresh time.

[0225] As an optional implementation manner, determining the refresh time increment according to the position of the current handwriting information on the display unit includes:

[0226] Determining a refresh time increment based on a ratio of the vertical coordinate of the current handwriting information to the height of the display unit and a refresh cycle of the display unit; and / or,

[0227] The refresh time increment is determined according to the ratio of the horizontal coordinate of the current handwriting information to the width of the display unit and the refresh cycle of the display unit.

[0228] As an optional implementation manner, the receiving of current handwriting information of the user includes:

[0229] Obtain the current handwriting information of the user through the application layer.

[0230] As an optional implementation manner, the receiving of current handwriting information of the user includes:

[0231] Reading touch events through the application layer, or receiving touch events transmitted by the framework layer through the application layer using cross-process communication, and converting the touch events into touch point reporting events;

[0232] Determining the current handwriting information of the user's handwriting is received according to the handwriting point information included in the touch point reporting event;

[0233] The touch event is generated based on a touch signal generated by a user's handwriting, and the touch point reporting event includes handwriting point information corresponding to the touch signal.

[0234] Based on the same inventive concept, the present disclosure also provides a writing optimization device, as shown in FIG11 , which includes:

[0235] The handwriting receiving module 1100 is used to receive the current handwriting information of the user's handwriting, wherein the current handwriting information includes the position of the handwriting point;

[0236] The position prediction module 1101 is used to obtain the next refresh time of the display unit and determine the predicted position corresponding to the current handwriting information at the next refresh time, wherein the current handwriting information includes the handwriting information received before the next refresh time;

[0237] The handwriting drawing module 1102 is used to determine the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and draw and display the predicted handwriting of the current handwriting information.

[0238] As an optional implementation, the method further includes an actual handwriting drawing module specifically configured to:

[0239] The actual handwriting of the current handwriting information is determined according to the current handwriting information, and the actual handwriting is displayed.

[0240] As an optional implementation, the location prediction module 1101 is specifically configured to determine the next refresh time in the following manner:

[0241] Acquire a refresh event and a refresh cycle received by the display unit before the next refresh time, wherein the refresh event includes at least one refresh time of the display unit refresh;

[0242] The next refresh time of the display unit is determined according to the at least one refresh time and the refresh cycle.

[0243] As an optional implementation manner, the location prediction module 1101 is specifically configured to:

[0244] According to the receiving time of the current handwriting information and the next refreshing time, the predicted position corresponding to the current handwriting information at the next refreshing time is determined.

[0245] As an optional implementation manner, the location prediction module 1101 is specifically configured to:

[0246] Determine the initial predicted position of the current handwriting information at the next refresh time according to the reception time of the current handwriting information and the next refresh time;

[0247] Determine an adjustment factor based on the movement angles of N handwriting information preceding the current handwriting information;

[0248] The initial predicted position is adjusted according to the adjustment factor to determine the predicted position corresponding to the current handwriting information at the next refresh time.

[0249] As an optional implementation, the handwriting drawing module 1102 is specifically configured to:

[0250] Determining a predicted handwriting drawing time of current handwriting information, wherein the drawing time is before the next refresh time;

[0251] When the drawing time is reached, the predicted handwriting of the current handwriting information is drawn.

[0252] As an optional implementation, the handwriting drawing module 1102 is specifically configured to:

[0253] Determine the rendering time of the predicted handwriting of the current handwriting information;

[0254] The drawing time of the predicted handwriting of the current handwriting information is determined according to the rendering time and the next refresh time.

[0255] As an optional implementation,

[0256] When the drawing time is between the receiving time of the current handwriting information and the next refreshing time, the drawing time is advanced as the rendering time increases.

[0257] As an optional implementation, the handwriting drawing module 1102 is further configured to:

[0258] If the time interval between the reception time of the current handwriting information and the next refresh time is less than or equal to the rendering time, the predicted handwriting of the current handwriting information is drawn after the next refresh time.

[0259] As an optional implementation, the handwriting drawing module 1102 is further configured to:

[0260] If the drawing time is earlier than the receiving time of the current handwriting information, then after the next refresh time, the predicted handwriting of the current handwriting information is drawn; and / or,

[0261] If the next handwriting information is received between the drawing time and the next refresh time, the predicted handwriting of the next handwriting information is drawn after the next refresh time.

[0262] As an optional implementation, the handwriting drawing module 1102 is specifically configured to:

[0263] Determine the rendering duration according to a preset duration; and / or,

[0264] The historical rendering durations corresponding to a plurality of historical handwriting information before the current handwriting information are counted, and the rendering duration is determined according to the plurality of historical rendering durations.

[0265] As an optional implementation, the handwriting drawing module 1102 is further configured to:

[0266] determining the actual handwriting between the previous handwriting information and the current handwriting information according to the previous handwriting information and the current handwriting information;

[0267] Clear the predicted handwriting of the previous handwriting information, draw and display the actual handwriting between the previous handwriting information and the current handwriting information.

[0268] As an optional implementation manner, the location prediction module 1101 is specifically configured to:

[0269] Determine the refresh time increment according to the position of the current handwriting information on the display unit;

[0270] According to the refresh time increment and the next refresh time, the predicted position corresponding to the current handwriting information at the next refresh time is determined.

[0271] As an optional implementation,

[0272] The farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the vertical direction is, the larger the refresh time increment is; and / or,

[0273] The farther the distance between the position of the current handwriting information and the coordinate origin of the display unit in the horizontal direction is, the greater the refresh time increment is.

[0274] As an optional implementation manner, the location prediction module 1101 is specifically configured to:

[0275] Determining a predicted time based on the next refresh time and the refresh time increment;

[0276] Determine the predicted position corresponding to the current handwriting information at the predicted time;

[0277] The predicted position corresponding to the current handwriting information at the predicted time is determined as the predicted position corresponding to the current handwriting information at the next refresh time.

[0278] As an optional implementation manner, the location prediction module 1101 is specifically configured to:

[0279] Determining a refresh time increment based on a ratio of the vertical coordinate of the current handwriting information to the height of the display unit and a refresh cycle of the display unit; and / or,

[0280] The refresh time increment is determined according to the ratio of the horizontal coordinate of the current handwriting information to the width of the display unit and the refresh cycle of the display unit.

[0281] As an optional implementation manner, the handwriting receiving module 1100 is specifically configured to:

[0282] Obtain the current handwriting information of the user through the application layer.

[0283] As an optional implementation manner, the handwriting receiving module 1100 is specifically configured to:

[0284] Reading touch events through the application layer, or receiving touch events transmitted by the framework layer through the application layer using cross-process communication, and converting the touch events into touch point reporting events;

[0285] Determining the current handwriting information of the user's handwriting is received according to the handwriting point information included in the touch point reporting event;

[0286] The touch event is generated based on a touch signal generated by a user's handwriting, and the touch point reporting event includes handwriting point information corresponding to the touch signal.

[0287] Based on the same inventive concept, the present disclosure also provides a writing optimization method, as shown in FIG12 . The specific implementation process of the method is as follows:

[0288] Step 1200: receiving the current handwriting information of the user during the refresh cycle of the display unit;

[0289] Step 1201: determine the predicted handwriting corresponding to the current handwriting information at the next refresh time of the display unit, draw and display the predicted handwriting of the current handwriting information;

[0290] Step 1202: Continue to receive the next handwriting information in the next refresh cycle, draw and display the predicted handwriting of the next handwriting information, and stop when receiving the user's lift signal.

[0291] Based on the same inventive concept, the present disclosure also provides a writing optimization method, as shown in FIG13 . The specific implementation process of the method is as follows:

[0292] Step 1300: Receive handwriting information of the user and obtain a refresh time of the display unit, wherein the handwriting information includes the position of the handwriting point;

[0293] Step 1301: Determine the predicted handwriting corresponding to the handwriting information at the refresh time, draw and display the predicted handwriting.

[0294] Based on the same inventive concept, embodiments of the present disclosure provide a computer storage medium comprising computer program code. When the computer program code is executed on a computer, the computer executes any of the aforementioned writing optimization methods. Because the principles underlying the aforementioned computer storage medium and the writing optimization method are similar, the implementation of the aforementioned computer storage medium can be referred to as the implementation of the method, and any repetitions will not be repeated.

[0295] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0296] Based on the same inventive concept, embodiments of the present disclosure further provide a computer program product, comprising: computer program code, which, when executed on a computer, causes the computer to perform any of the aforementioned handwriting optimization methods. Because the principles underlying the aforementioned computer program products are similar to those underlying the handwriting optimization methods, the implementation of the aforementioned computer program products can be referenced to the implementation of the aforementioned methods, and any repetitions will not be repeated.

[0297] The computer program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0298] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0299] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0300] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0301] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0302] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A display device, wherein, The display device includes a display unit and a control circuit, where: The display unit is configured to display content; The control circuit includes a processor and a memory. The memory is used to store programs executable by the processor, and the processor is used to read the programs in the memory and execute the following steps: Receive the current handwriting information written by the user, where the current handwriting information includes the positions of handwriting points; Obtain the next refresh moment of the display unit, and determine the predicted position corresponding to the current handwriting information at the next refresh moment, where the current handwriting information includes the handwriting information received before the next refresh moment; Determine the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and draw and display the predicted handwriting of the current handwriting information.

2. The device according to claim 1, wherein, The processor is specifically further configured to execute: Determine the actual handwriting of the current handwriting information according to the current handwriting information, and display the actual handwriting.

3. The device according to claim 1, wherein The processor is specifically configured to execute: Obtain the refresh events and refresh periods received by the display unit before the next refresh moment, where the refresh events include the refresh moments of at least one display unit refresh; Determine the next refresh moment of the display unit according to the at least one refresh moment and the refresh period.

4. The device according to claim 1, wherein The processor is specifically configured to execute: Determine the predicted position corresponding to the current handwriting information at the next refresh moment according to the reception moment and the next refresh moment of the current handwriting information.

5. The device according to claim 4, wherein, The processor is specifically configured to execute: Determine the initial predicted position corresponding to the current handwriting information at the next refresh moment according to the reception moment and the next refresh moment of the current handwriting information; Determine an adjustment factor according to the moving angles of the N handwriting information before the current handwriting information; Adjust the initial predicted position according to the adjustment factor to determine the predicted position corresponding to the current handwriting information at the next refresh moment.

6. The device according to claim 1, wherein, The processor is specifically configured to execute: Determine the drawing moment of the predicted handwriting of the current handwriting information, where the drawing moment is before the next refresh moment; When the drawing moment arrives, draw the predicted handwriting of the current handwriting information.

7. The device according to claim 6, wherein, The processor is specifically configured to execute: Determine the rendering duration of the predicted handwriting of the current handwriting information; Determine the drawing moment of the predicted handwriting of the current handwriting information according to the rendering duration and the next refresh moment.

8. The device according to claim 7, where, When the drawing moment is between the reception moment of the current handwriting information and the next refresh moment, the drawing moment advances as the rendering duration increases.

9. The apparatus according to claim 7, wherein, The processor is specifically configured to execute: If the time interval between the reception moment of the current handwriting information and the next refresh moment is less than or equal to the rendering duration, draw the predicted handwriting of the current handwriting information after the next refresh moment.

10. The device according to claim 6, wherein, The processor is specifically configured to execute: If the drawing moment is earlier than the reception moment of the current handwriting information, draw the predicted handwriting of the current handwriting information after the next refresh moment; and / or, If the next handwriting information is received between the drawing moment and the next refresh moment, then after the next refresh moment, predict the handwriting of the next handwriting information.

11. The device according to claim 7, wherein, The processor is specifically configured to execute: Determine the rendering duration according to a preset duration; and / or, Count the historical rendering durations corresponding to a plurality of historical handwriting information before the current handwriting information, and determine the rendering duration according to the plurality of historical rendering durations.

12. The device according to claim 1, wherein, The processor is specifically further configured to execute: Determine the actual handwriting between the previous handwriting information and the current handwriting information according to the previous handwriting information and the current handwriting information; Clear the predicted handwriting of the previous handwriting information, and draw and display the actual handwriting between the previous handwriting information and the current handwriting information.

13. The device according to claim 1, wherein, The processor is specifically configured to execute: Determine a refresh time increment according to the position of the current handwriting information on the display unit; Determine the predicted position of the current handwriting information at the next refresh moment according to the refresh time increment and the next refresh moment.

14. The device according to claim 13, wherein The farther the distance from the position of the current handwriting information to the coordinate origin of the display unit in the vertical axis direction, the greater the refresh time increment; and / or, The farther the distance from the position of the current handwriting information to the coordinate origin of the display unit in the horizontal axis direction, the greater the refresh time increment.

15. The apparatus according to claim 13, wherein, The processor is specifically configured to execute: Determine a predicted moment according to the next refresh moment and the refresh time increment; Determine the predicted position of the current handwriting information at the predicted moment; Determine the predicted position of the current handwriting information at the next refresh moment as the predicted position of the current handwriting information at the predicted moment.

16. The device according to claim 13, wherein, The processor is specifically configured to execute: Determine the refresh time increment according to the ratio of the ordinate of the current handwriting information to the height of the display unit and the refresh period of the display unit; and / or, Determine the refresh time increment according to the ratio of the abscissa of the current handwriting information to the width of the display unit and the refresh period of the display unit.

17. The device according to claim 1, wherein, The processor is specifically configured to execute: Obtain the current handwriting information written by the user through the application layer.

18. The device according to claim 17, wherein, The processor is specifically configured to execute: Read a touch event through the application layer, or receive a touch event transmitted by the framework layer through the application layer by using cross-process communication, and convert the touch event into a touch reporting point event; Determine the current handwriting information written by the user according to the handwriting point information included in the touch reporting point event; wherein the touch event is generated by triggering according to a touch signal written by the user, and the touch reporting point event includes handwriting point information corresponding to the touch signal. The display device includes a display unit and a control circuit, wherein:

19. A display device, wherein, The display unit is configured to display content; The control circuit includes a processor and a memory, the memory is used to store programs executable by the processor, and the processor is used to read the programs in the memory and execute the following steps: Receive the current handwriting information written by the user within the refresh period of the current display unit; ​ Determine the predicted handwriting corresponding to the current handwriting information at the next refresh moment of the display unit, and draw and display the predicted handwriting of the current handwriting information; Continue to receive the next handwriting information within the next refresh cycle, draw and display the predicted handwriting of the next handwriting information, and stop until a lift signal from the user is received.

20. A display device, wherein, The display device includes a display unit and a control circuit, where: The display unit is configured to display content; The control circuit includes a processor and a memory. The memory is used to store programs executable by the processor. The processor is used to read the programs in the memory and execute the following steps: Receive the handwriting information written by the user, and obtain the refresh moment of the display unit, where the handwriting information includes the positions of handwriting points; Determine the predicted handwriting corresponding to the handwriting information at the refresh moment, and draw and display the predicted handwriting.

21. A writing optimization method, wherein, The method includes: Receive the current handwriting information written by the user, where the current handwriting information includes the positions of handwriting points; Obtain the next refresh moment of the display unit, and determine the predicted position corresponding to the current handwriting information at the next refresh moment, where the current handwriting information includes the handwriting information received before the next refresh moment; Determine the predicted handwriting of the current handwriting information according to the predicted position corresponding to the current handwriting information, and draw and display the predicted handwriting of the current handwriting information.

22. A computer storage medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the steps of the method described in claim 21.

Images