Refresh method, device and readable storage medium for electronic ink screen

Abstract

The present invention discloses a refresh method, device, and readable storage medium for an electronic ink display. The method comprises: responding to an initialization instruction and acquiring drawn handwriting into a buffer; determining an update region based on the buffer; determining whether the update region is a handwriting layer; and if so, sending the update region to the electronic ink display for display. This improves the refresh speed of the electronic ink display in handwriting scenarios. The present invention can be widely applied to the field of electronic ink display refresh.

Description

Technical Field

[0001] The present invention relates to the field of electronic ink screens, and in particular to a refreshing method, device and readable storage medium for an electronic ink screen. Background Art

[0002] E-ink screens are increasingly becoming the preferred reading device for many people. However, on the Android platform, due to the physical characteristics of the screen, E-ink screens have a slow refresh rate. In handwriting scenarios, the native refresh mechanism experiences a delay exceeding 200ms, accompanied by flickering in other areas. This significantly impacts the user experience when handwriting. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, device, and readable storage medium for refreshing an electronic ink screen, so as to increase the refresh speed of the electronic ink screen in handwriting scenarios.

[0004] One aspect of the present invention provides a refresh method for an electronic ink screen, comprising: responding to an initialization instruction and acquiring drawn handwriting into a buffer; determining an update area based on the buffer; determining whether the update area is a handwriting layer, and if the update area is a handwriting layer, sending the update area to the electronic ink screen for display.

[0005] According to some embodiments of the present invention, responding to the initialization instruction includes: responding to the initialization instruction and stopping layer synthesis; applying for a buffer according to the size of the handwriting area, and storing a handle of the buffer.

[0006] According to some embodiments of the present invention, determining whether the update area is a handwriting layer further includes: if the update area is a normal layer, performing layer synthesis on the update area to obtain a target update layer, and sending the target update layer to a buffer.

[0007] According to some embodiments of the present invention, after the step of synthesizing the update area into layers to obtain the target update layer and sending the target update layer to the buffer, it also includes: obtaining the target update layer from the buffer and sending the target update layer to the electronic ink screen for display.

[0008] According to some embodiments of the present invention, the determining whether the update area is a handwriting layer further includes: determining whether the update area is a conflict area; if the update area is a conflict area, waiting for the conflicted pipeline to be updated, and sending the conflict area that has been updated to a buffer, wherein the conflict area is the area where the handwriting is being updated.

[0009] According to some embodiments of the present invention, after the pipeline waiting to be conflicted is updated and the conflicting area that has been updated is sent to the buffer, it also includes: obtaining the conflicting area that has been updated from the buffer and sending the conflicting area to the electronic ink screen for display.

[0010] Another aspect of the present invention provides a refresh device for an electronic ink screen, comprising: a first module for responding to an initialization instruction and acquiring drawn handwriting into a buffer; a second module for determining an update area based on the buffer; and a third module for determining whether the update area is a handwriting layer. If the update area is a handwriting layer, the update area is sent to the electronic ink screen for display.

[0011] Another aspect of the present invention provides an electronic device, comprising a processor and a memory; the memory is used to store a program; the processor executes the program to implement any one of the above-described methods for refreshing an electronic ink screen.

[0012] The electronic device according to the embodiment of the present invention has at least the same beneficial effects as the above-mentioned refresh method of the electronic ink screen.

[0013] Another aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement any one of the above-mentioned methods for refreshing an electronic ink screen.

[0014] The computer-readable storage medium according to the embodiment of the present invention has at least the same beneficial effects as the above-mentioned method for refreshing the electronic ink screen.

[0015] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0016] The embodiments of the present invention achieve the goal of not obtaining input events through the framework by responding to initialization instructions and directly acquiring the drawn notes into the buffer, and directly processing and drawing the handwriting in the handwriting service, thereby simplifying the input process. The update area is determined based on the buffer, and it is determined whether the update area is a handwriting layer. If the update area is a handwriting layer, the update area is sent to the electronic ink screen for display. This bypasses the Android native layer synthesis and sends the handwritten content directly to the display driver for display, thereby achieving synchronous refresh. By simplifying the input process, the synchronous refresh mechanism improves the refresh speed of the electronic ink screen in the handwriting scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a framework diagram of refreshing the electronic ink screen provided by an embodiment of the present invention;

[0019] Figure 2 This is a flowchart of the steps of the refresh method of the electronic ink screen provided by an embodiment of the present invention;

[0020] Figure 3 This is a schematic block diagram of a refreshing device for an electronic ink screen provided by an embodiment of the present invention;

[0021] Figure 4 is a schematic block diagram of a device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0023] On the Android platform, electronic ink screens have a slow refresh rate due to the physical characteristics of the screen. In handwriting scenarios, the native refresh mechanism delays over 200ms and is accompanied by flickering in other areas. To address the slow refresh rate of electronic ink screens in handwriting scenarios, the industry currently has the following solutions:

[0024] (1) Set a specific refresh mode to handle the screen refresh method in handwriting scenarios.

[0025] (2) The handwriting area is updated and displayed. This part of the industry is deeply customized, from the application end to the underlying driver, and the entire channel cooperates to achieve handwriting refresh.

[0026] (3) Use Eink TCON hardware to process handwritten content.

[0027] However, the existing technology has the following problems:

[0028] (1) The Android native refresh framework process is complex. In the handwriting scenario, updating a small area will also cause the entire screen to be synthesized, consuming system resources and time.

[0029] (2) It takes time for Android electromagnetic pen data to be reported through the native input framework, which increases the delay of handwriting input.

[0030] (3) Sharing the buffer between the user layer and the driver layer requires repeatedly mapping and unmaping virtual addresses through the buffer handle, which increases system latency.

[0031] (4) The refresh mechanism of the electronic ink screen will cause refresh waiting when handling refresh area conflicts, increasing the handwriting display delay.

[0032] To this end, a refresh method, device and readable storage medium for an electronic ink screen are proposed to improve the refresh speed of the electronic ink screen in handwriting scenarios, which can be used in Android electronic ink screen products.

[0033] The following is an explanation of the terms that appear in the article:

[0034] SurfaceFlinger: is a process that accepts graphics display data from multiple sources, synthesizes them, and then sends them to the display device.

[0035] Handle: It is the foundation of all Windows programming. A handle is a unique integer value, a 4-byte (8-byte in 64-bit programs) value, used to identify different objects in an application and different instances of the same type.

[0036] Eink HWC: It is the HAL layer module for window (Layer) synthesis and display in Android, and its implementation is device-specific.

[0037] Buffer: A buffer is a section of memory that temporarily stores input and output data.

[0038] Eink200: Electronic ink. It is a new material that integrates chemistry, physics, and electronics. It is characterized by readability, flexibility, easy and inexpensive manufacturing, and low power consumption.

[0039] DE: DE refers to the display engine, which is the display synthesis module in the SoC.

[0040] TCON: also called logic board, screen driver board, central control board.

[0041] Reference Figure 1 , provides a framework diagram for executing refresh of the electronic ink screen in a specific embodiment:

[0042] Applications include: handwriting applications; frameworks include: window management services, handwriting services, and SurfaceFlinger; hardware abstraction layers include: Eink HWC; screen drivers include: Input, DE, and Eink200.

[0043] A handwriting service has been added to the Android native display framework to manage the handwriting refresh process. Handwriting applications must integrate with both the native display framework and the newly added handwriting service interface. During handwriting initialization, the handwriting service is notified to enable handwriting mode and stop SurfaceFlinger composition to avoid a full screen refresh. A Surface representing the handwriting area is also passed in. The handwriting service requests a display buffer based on the surface size, passes the handle to the HWC, and starts a thread for drawing the handwriting. During the handwriting process, the handwriting service receives input events and directly draws the handwriting, updating the handwriting area in real time. The handwriting service receives input events directly from the driver, bypassing the input service layer. After receiving the updated area, the screen driver directly updates the display buffer to the e-ink screen. Conflicting areas are merged until all current handwriting updates are complete before being sent to the screen. On the Android platform, the characteristics of E-Ink displays can cause significant delays in handwriting updates in conventional applications. Therefore, by enabling a separate handwriting service to handle input events and display refreshes, this fully utilizes the characteristics of E-Ink displays for display updates, reducing handwriting update latency and improving update speed. It should be noted that the handwriting service can also be directly integrated into the native display framework, not existing as a separate service, but providing similar functionality. Alternatively, the handwriting service work can be performed by the handwriting application, which directly connects to the hardware abstraction layer and can also provide similar functionality.

[0044] refer to Figure 2 , the method of the embodiment of the present invention includes steps S100-S300.

[0045] Step S100, responding to the initialization instruction and acquiring the drawn handwriting into the buffer.

[0046] Specifically, in response to the initialization instruction, the handwriting mode is triggered, and SurfaceFlinger is stopped to allow layer synthesis. This is to avoid screen refresh synthesis. Afterwards, a buffer is requested based on the size of the Surface in the handwriting area, and the handle is transferred to the HWC to start the thread for drawing the handwriting. The above are all initialization work before handwriting. During the handwriting process, after the screen driver receives the update, it obtains the drawn handwriting into the buffer. It should be noted that in the current common technology, the buffer is shared between the user layer and the driver layer, and the virtual address needs to be repeatedly mapped and unmapped through the buffer handle, which increases system latency. By transferring the handle to the HWC, the handwriting content buffer is maintained using the same physical address, so there is no need to remap and unmap each time the content is updated. The display driver map and unmap process is optimized to reduce driver latency. By managing the application surface layer, the Android SurfaceFlinger synthesis mechanism is bypassed. The UI layer in the handwriting area no longer triggers layer updates in handwriting mode.

[0047] Step S200: determining an update area according to the buffer zone.

[0048] Specifically, during the handwriting process, the corresponding handwriting is drawn into the buffer according to the electromagnetic pen input, and the update area is sent down.

[0049] Step S300: determine whether the updated area is a handwriting layer. If the updated area is a handwriting layer, send the updated area to the electronic ink screen for display.

[0050] In step S300, the following steps are also included:

[0051] If the update area is a normal layer, the update area is layer synthesized to obtain the target update layer, and the target update layer is sent to the buffer; the target update layer is obtained from the buffer and sent to the electronic ink screen for display.

[0052] If the update area is the handwriting layer, determine whether the update area is a conflicting area. If the update area is a conflicting area, wait for the conflicting pipeline to be updated and send the updated conflicting area to the buffer, where the conflicting area is the area where the handwriting is being updated; obtain the updated conflicting area from the buffer and send the conflicting area to the electronic ink screen for display.

[0053] Specifically, the HWC determines whether the updated area is the handwriting layer. If so, it then determines whether the update is a conflicting area. Non-conflicting areas of the handwriting layer are sent to eink for display. For conflicting areas, the HWC waits for the update to complete before merging the conflicting areas and sending them to eink for display. If the HWC determines that the updated area is not the handwriting layer, it indicates that the update was made by a standard application, SurfaceFlinger. Therefore, the HWC sends the updated area to the DE for layer merging, resulting in the target updated layer, which is then sent to eink for display. The eink screen driver includes an ioctl interface, allowing applications to obtain the DMA address corresponding to the buffer, which the driver directly uses. Through the service layer, the display layer is directly connected to the handwriting area, bypassing Android's native layer synthesis when processing the handwriting area, and the handwriting content is sent directly to the display driver for display. When a handwriting update occurs, the HWC layer is notified to refresh the updated area without refreshing the entire screen. Input events are not received through the framework, but are directly processed and drawn in the handwriting service. This reduces latency and improves speed.

[0054] In another embodiment, the display path for the update area is specifically:

[0055] After DE synthesis, the upper layer is written back to the memory buffer. Eink retrieves the display content from the memory buffer and displays it on the screen. Through the DE write-back function, the conflicting areas of the handwriting are temporarily blocked from display. After the handwriting is completed, the conflicting areas are merged and updated to the screen.

[0056] On the one hand, referring to Figure 3 This embodiment provides a refreshing device for an electronic ink screen, which at least includes: a first module 410, a second module 420, and a third module 430.

[0057] Specifically, the first module 410 responds to the initialization instruction and obtains the drawn handwriting into the buffer. The second module 420 is connected to the first module 410 and obtains the buffer in the first module 410 to determine the update area. The third module 430 is connected to the second module 420 to determine whether the update area is a handwriting layer. If so, the update area is sent to the display driver of the electronic ink screen for display. It can be understood that the device includes such Figure 1 framework and can perform Figure 2 method.

[0058] Furthermore, developing this application through an app is essentially the same as developing a regular Android app, requiring only the setting of special variable flags. This allows for high development versatility. Customized for handwriting scenarios, the performance approaches that of hardware implementations, providing an experience similar to that of writing on paper. Using a software implementation eliminates the need for hardware IP modules, saving costs.

[0059] Reference Figure 4 This embodiment provides an electronic device comprising a processor 510 and a memory 520 coupled to the processor 510. The memory 520 stores program instructions executable by the processor 510. When the processor 510 executes the program instructions stored in the memory 520, the aforementioned target risk website detection method is implemented. The processor 510 may also be referred to as a CPU (Central Processing Unit). The processor 510 may be an integrated circuit chip with signal processing capabilities. The processor 510 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or any conventional processor. The memory 520 may include various components (e.g., machine-readable media), including but not limited to random access memory components, read-only components, and any combination thereof. The memory 520 may also include instructions (e.g., software) (e.g., stored on one or more machine-readable media); the instructions implement the target risk website detection method described in the aforementioned embodiment. The electronic device has the function of carrying and running the software system for target risk website detection provided by the embodiment of the present invention, for example, a personal computer (PC), a mobile phone, a smart phone, a personal digital assistant (PDA), a wearable device, a handheld computer (PPC), a tablet computer, etc.

[0060] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 2 The method shown.

[0061] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0062] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0063] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0064] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0065] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0066] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0067] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0068] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0069] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for refreshing an electronic ink screen, characterized in that: include: Respond to the initialization instruction and get the drawn handwriting into the buffer; The responding to the initialization instruction includes: responding to the initialization instruction, stopping layer synthesis; applying for a buffer according to the size of the handwriting area, and storing a handle of the buffer; determining an update area according to the buffer zone; Determine whether the update area is a handwriting layer. If the update area is a handwriting layer, determine whether the update area is a conflicting area. If the update area is a conflicting area, wait for the conflicting pipeline to be updated, send the updated conflicting area to a buffer, retrieve the updated conflicting area from the buffer, and send the conflicting area to the electronic ink screen for display. The conflicting area is the area where the handwriting is being updated. For the handwriting layer that is a conflicting area, wait for the updated stroke to be completed before merging the conflicting areas. If the update area is not a conflicting area, send the updated area to the electronic ink screen for display. The handwriting layer represents the handwriting area UI layer. If the update area is a normal layer, the update area is layer synthesized to obtain a target update layer, and the target update layer is sent to a buffer; the target update layer is obtained from the buffer, and the target update layer is sent to the electronic ink screen for display.

2. The refresh method of the electronic ink screen according to claim 1, characterized in that: After the step of synthesizing the update area into layers to obtain a target update layer and sending the target update layer to the buffer, the method further includes: The target update layer is obtained from the buffer, and the target update layer is sent to the electronic ink screen for display.

3. The refresh method of the electronic ink screen according to claim 1, characterized in that: After the pipeline waiting for conflict is updated and the conflicting area is sent to the buffer, the method further includes: The updated conflicting area is obtained from the buffer, and the conflicting area is sent to the electronic ink screen for display.

4. A refreshing device for an electronic ink screen, characterized in that: include: The first module is used to respond to the initialization instruction and obtain the drawn handwriting into the buffer; The responding to the initialization instruction includes: responding to the initialization instruction, stopping layer synthesis; applying for a buffer according to the size of the handwriting area, and storing a handle of the buffer; A second module is configured to determine an update area according to the buffer zone; The third module is configured to determine whether the update area is a handwriting layer. If the update area is a handwriting layer, determine whether the update area is a conflicting area. If the update area is a conflicting area, wait for the conflicting pipeline to be updated, send the updated conflicting area to a buffer, retrieve the updated conflicting area from the buffer, and send the conflicting area to the electronic ink screen for display. The conflicting area is the area where the handwriting is being updated. For the handwriting layer that is a conflicting area, wait for the updated stroke to be completed before merging the conflicting areas. If the update area is not a conflicting area, send the updated area to the electronic ink screen for display. The handwriting layer represents the UI layer of the handwriting area. If the update area is a normal layer, the update area is layer synthesized to obtain a target update layer, and the target update layer is sent to a buffer; the target update layer is obtained from the buffer, and the target update layer is sent to the electronic ink screen for display.

5. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 3.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images