Touch processing method and device based on graphic display system, electronic device, and storage medium

Abstract

The present application is about a touch processing method and device, electronic device, and storage medium based on a graphics display system. The method includes: in response to detecting a touch operation, generating a touch event, and sending the touch event to the thread of the current UI window; triggering the thread of the current UI window to cache the touch event and generate an update window request; after detecting a synchronization event, responding to the update window request, obtaining the touch point information in the touch event, and determining the smooth touch point information based on the touch point information; sending the smooth touch point information to the target layout control, triggering the target layout control to determine the dirty area to be updated based on the smooth touch point information; triggering the rendering thread to render the dirty area and update the interface graphics. The present application avoids jitter and frame skipping in the graphical interface, providing users with a better user experience.

Description

Technical Field

[0001] Embodiments of the present application relate to interface graphics update technology, and more particularly to a touch processing method and device, electronic device, and storage medium based on a graphics display system. Background Art

[0002] In some operating systems, the rendering frame rate of the graphics display system is inconsistent with the frequency of touch reporting. Taking the Linux operating system as an example, the rendering frame rate of the graphics display system is usually 16ms / 60hz. The reporting frequency of the touch device set for the electronic device is usually 10ms / 90hz. Therefore, when the touch reporting event is synchronized to the display system through the input system, there is generally a delay in the final refresh display on the screen. In addition, due to the inconsistency between the rendering frame rate and the frequency of touch reporting, jitter and frame skipping of the graphical interface will often occur. Specifically, the UI process undergoes multiple cross-process transmissions before receiving the Touch event. Affected by the thread and process scheduling, the event reception time will fluctuate greatly and easily generate jitter. In one frame time, the UI process may receive multiple Touch events, and only the latest reporting point is processed and updated to the UI, which will cause point loss and frame skipping. This is a very poor user experience. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a touch processing method and device based on a graphics display system, an electronic device, and a storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of an embodiment of the present application, a touch processing method based on a graphics display system is provided, comprising:

[0005] In response to detecting a touch operation, a touch event is generated, and the touch event is sent to a thread of a current user interface (UI) window;

[0006] Triggering the thread of the current UI window to cache the touch event and generate a window update request;

[0007] After detecting the synchronization event, responding to the window update request, acquiring the touch point information in the cached touch event, and determining the smooth touch point information based on the touch point information;

[0008] Sending the information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be updated based on the information of the smooth touch point;

[0009] The rendering thread is triggered to render the dirty area and update the interface graphics.

[0010] In one embodiment, determining the smooth touch point information based on the touch point information includes:

[0011] In the touch events in the thread cache of the current UI window, a set number of touch points closest to a sampling time are searched, and the time and position information of the smoothed touch points are determined based on the time information of the found touch points and the sampling time.

[0012] In one embodiment, the set number is two;

[0013] The determining the time and position information of the smooth touch point includes:

[0014] When it is determined that the sampling time is between the times of the two touch points, a first offset between the time of a first touch point and the time of a second touch point among the two touch points, and a second offset between the sampling time and the time of the second touch point are determined, and a first weight is determined based on the first offset and the second offset; and the time and position information of the smooth touch point are calculated based on the time of the second touch point, the first weight and the first offset.

[0015] In one embodiment, the set number is two;

[0016] The determining the time and position information of the smooth touch point includes:

[0017] When it is determined that the sampling time is outside the time of the two touch points, a first offset between the time of a first touch point and the time of the second touch point among the two touch points, and a third offset between the time of the first touch point and the sampling time are determined, and a second weight is determined based on the first offset and the third offset; the time and position information of the smooth touch point are calculated based on the time of the first touch point, the second weight and the inverse of the first offset.

[0018] In one embodiment, the method further comprises:

[0019] A scheduling strategy is set for the touch event; wherein the scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than the scheduling priority of other threads or processes; when scheduling the touch event, replacing the time slice polling scheduling method with the priority scheduling method.

[0020] In one embodiment, the method further comprises:

[0021] When detecting that the application or UI window corresponding to the touch event is switched to the background, lowering the priority of the touch event; or

[0022] When it is detected that the application or UI window corresponding to the touch event of low priority is switched from the background to the foreground, the priority of the touch event is restored.

[0023] According to a second aspect of an embodiment of the present application, a touch control processing device based on a graphics display system is provided, comprising:

[0024] A first detection unit, used to detect a touch operation;

[0025] A generating unit, configured to generate a touch event in response to detecting a touch operation;

[0026] A first transmission unit, used to send the touch event to the thread of the current UI window;

[0027] A first triggering unit, configured to trigger the thread of the current UI window to cache the touch event and generate a window update request;

[0028] A second detection unit, configured to detect a synchronization event;

[0029] a determining unit, configured to respond to the window update request, obtain information of the touch point in the touch event, and determine information of the smooth touch point based on the information of the touch point;

[0030] A second transmission unit, used for sending the information of the smooth touch point to a target layout control;

[0031] A second triggering unit, used for triggering the target layout control to determine a dirty area to be updated based on the information of the smooth touch point;

[0032] The third triggering unit is used to trigger the rendering thread to render the dirty area and update the interface graphics.

[0033] In one embodiment, the determining unit is further configured to:

[0034] In the touch events in the thread cache of the current UI window, a set number of touch points closest to a sampling time are searched, and the time and position information of the smoothed touch points are determined based on the time information of the found touch points and the sampling time.

[0035] In one embodiment, the set number is two;

[0036] The determining unit is further configured to:

[0037] When it is determined that the sampling time is between the times of the two touch points, a first offset between the time of a first touch point and the time of a second touch point among the two touch points, and a second offset between the sampling time and the time of the second touch point are determined, and a first weight is determined based on the first offset and the second offset; and the time and position information of the smooth touch point are calculated based on the time of the second touch point, the first weight and the first offset.

[0038] In one embodiment, the set number is two;

[0039] The determining unit is further configured to:

[0040] When it is determined that the sampling time is outside the time of the two touch points, a first offset between the time of a first touch point and the time of the second touch point among the two touch points, and a third offset between the time of the first touch point and the sampling time are determined, and a second weight is determined based on the first offset and the third offset; the time and position information of the smooth touch point are calculated based on the time of the first touch point, the second weight and the inverse of the first offset.

[0041] In one embodiment, the apparatus further comprises:

[0042] A setting unit is used to set a scheduling strategy for the touch event; wherein the scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than the scheduling priority of other threads or processes; when scheduling the touch event, replacing the time slice polling scheduling method with the priority scheduling method.

[0043] In one embodiment, the apparatus further comprises:

[0044] A second detection unit is configured to lower the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or

[0045] When it is detected that the application or window corresponding to the touch event of low priority is switched from the background to the foreground, the priority of the touch event is restored.

[0046] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a processor, a memory, and an executable program stored in the memory and capable of being run by the processor, wherein the processor executes the steps of the touch processing method based on a graphics display system when running the executable program.

[0047] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which an executable program is stored. When the executable program is executed by a processor, the steps of the touch processing method based on the graphics display system are implemented.

[0048] In the embodiment of the present application, when a touch operation is detected, a corresponding touch event is generated for the current display interface according to the current UI window and the corresponding interface display, and the generated touch event is sent to the thread of the corresponding UI window; the embodiment of the present application does not respond to the touch event immediately, but caches the touch event, and after the graphic update synchronization event arrives, the smooth touch point is determined based on the touch point in the cached touch event, the dirty area to be updated is determined based on the smooth touch point, and the rendering thread is triggered to render the dirty area and update the interface graphics. The embodiment of the present application actively smoothes multiple touch points, and for high-frequency point reporting events, it can be updated very smoothly to the update of the display interface, avoiding jitter and frame skipping of the graphic interface, and providing users with a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A schematic diagram of a flow chart of a touch processing method based on a graphic display system according to an embodiment of the present application;

[0051] Figure 2 A schematic diagram showing a comparison between a Vsync update period and a Touch event reporting period in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of a graphical interface update cycle according to an embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the structure of a touch processing device based on a graphic display system according to an embodiment of the present application;

[0054] Figure 5 A structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] The essence of the technical solution of the embodiment of the present application is explained in detail below with reference to the accompanying drawings.

[0056] Figure 1FIG. 1 is a flow chart of a touch processing method based on a graphic display system according to an embodiment of the present application. Figure 1 As shown, the touch processing method based on the graphic display system of the embodiment of the present application includes the following processing steps:

[0057] Step 101: In response to detecting a touch operation, a touch event is generated, and the touch event is sent to a thread of the current UI window.

[0058] In the embodiment of the present application, when a user performs a touch operation (Touch) on the touch screen of the electronic device, the touch device detects the touch operation and generates a touch event. In the embodiment of the present application, the touch event may also be referred to as a Touch event.

[0059] Step 102: trigger the thread of the current UI window to cache the touch event and generate a window update request.

[0060] In an embodiment of the present application, as an implementation method, the Touch event is reported from the kernel driver to the user-mode receiving thread at a frequency of 10ms / 90hz. After receiving the Touch event, the user-mode receiving thread will distribute the event to the system's window manager. After receiving the Touch event, the window manager will make the following processing based on different scenario decisions: intercept the Touch event to perform global gesture actions; or distribute the Touch event to the system UI to perform actions such as dragging the status bar; or distribute the Touch event to the current top-level window application process for processing, such as common operations such as clicks and slides by the client. In an embodiment of the present application, the window manager implements the scheduling and management of the UI window thread, and the window manager generates the UI window thread.

[0061] In the embodiment of the present application, after receiving the Touch event on the main thread, the UI window of the relevant application process caches the Touch event in the cache queue and requests to update the window, and the buffer queue corresponds to the window one by one. When the Vsync synchronization event arrives, the window update request is responded to.

[0062] Step 103: after detecting the synchronization event, respond to the window update request, obtain the touch point information in the touch event, and determine the smooth touch point information based on the touch point information.

[0063] In the embodiment of the present application, the synchronization event is to receive the vertical synchronization signal Vsync for the graphical interface, which is used for vertical synchronization, and its main function is to prevent screen tearing of the interface graphics. When Vsync is detected, the update window request is responded to. In the embodiment of the present application, the graphical interface is not updated directly using the information of the touch points in the cache queue, but the touch points corresponding to the two most recent touch operations are obtained, and the sampling time points of the responses are determined, so as to determine the smooth touch points, and use the smooth touch points as the accurate dirty area to achieve the update rendering of the interface graphics.

[0064] Specifically, in the touch events in the thread cache of the current UI window, a set number of touch points closest to the sampling time are searched, and the time and position information of the smoothed touch points are determined based on the time information of the found touch points and the sampling time. The set number here is 2. Figure 2 Schematic diagram of comparison between the Vsync update period and the Touch event reporting period in the embodiment of the present application. Figure 2 As shown, since the Touch event reporting cycle is short, for example, 10ms, and the Vsync update cycle is long, for example, 16ms, when a synchronization event arrives, two Touch events may be received, for example, at the 32nd ms, two Touch events at the 20th ms and 30th ms will be received. In view of this situation, the embodiment of the present application needs to perform smoothing processing to better achieve the continuity of the image interface refresh, so that the graphic interface refresh is smoother without jitter and frame skipping.

[0065] Specifically, when it is determined that the sampling time is between the times of the two touch points, a first offset between the time of a first touch point and the time of the second touch point among the two touch points, and a second offset between the sampling time and the time of the second touch point are determined, and a first weight is determined based on the first offset and the second offset; and the time and position information of the smooth touch point are calculated based on the time of the second touch point, the first weight and the first offset.

[0066] The specific calculation method is as follows:

[0067] TouchTimeDiff=LastTouch-FirstTouch

[0068] TouchSampleDiff=SampleTime-FirstTouch

[0069]

[0070] Result=FirstTouch+(LastTouch-FirstTouch)×alpha

[0071] Here, SampleTime represents the sampling time, FirstTouch represents the timestamp of the second touch point, and LastTouch represents the timestamp of the first touch point. Alpha represents the first weight, which means that a larger Alpha means that the timestamp Result of the smoothed touch point is more inclined to the timestamp of the second touch point, and a smaller Alpha means that the sampling result is more inclined to the timestamp of the first touch point. TouchTimeDiff represents the first offset, and TouchSampleDiff represents the second offset; in the embodiment of the present application, by smoothing two or more touch points in the same Vsync update cycle, the graphical interface based on the Touch event is updated more smoothly.

[0072] As an implementation method, when it is determined that the sampling time is outside the time of the two touch points, a first offset between the time of the first touch point and the time of the second touch point of the two touch points, and a third offset between the time of the first touch point and the sampling time are determined, and a second weight is determined based on the first offset and the third offset; and the time information of the smoothed touch point is calculated based on the time of the first touch point, the second weight, and the inverse of the first offset. In the embodiment of the present application, the sampling time is outside the time of the two touch points, which means that the sampling time is later than the touch time of the last touch point.

[0073] The specific calculation method is as follows:

[0074] TouchTimeDiff=LastTouch-FirstTouch

[0075] TouchSampleDiff=LastTouch-SampleTime

[0076]

[0077] Result=LastTouch+(FirstTouch-LastTouch)×alpha

[0078] Here, SampleTime represents the sampling moment, FirstTouch represents the timestamp of the second touch point, and LastTouch represents the timestamp of the first touch point. Alpha represents the second weight, which means that a larger Alpha means that the timestamp Result of the smoothed touch point is more inclined to the sampling moment, and a smaller Alpha means that the sampling result is more inclined to the timestamp of the second touch point. TouchTimeDiff represents the first offset, and TouchSampleDiff represents the second offset; in the embodiment of the present application, by smoothing two or more touch points in the same Vsync update cycle, the graphical interface based on the Touch event is updated more smoothly.

[0079] In the embodiment of the present application, the sampling time is selected as 5 ms before the arrival of the Vsync event. Of course, other times may also be selected, such as 6 ms, 7 ms, 4 ms, etc. before the arrival of the Vsync event.

[0080] In the embodiment of the present application, when there is only one touch event in the cache queue, the dirty area of ​​the graphic control is directly marked using the touch point corresponding to the touch event in the cache queue.

[0081] In the embodiment of the present application, the position of the touch point with a later timestamp can be selected as the position information of the smooth touch point. This is because, for the operator, the touch operation can be considered to be close in position at the millisecond level. As an implementation method, the position of the touch point with a later timestamp can also be selected as the position information of the smooth touch point. Of course, the position information of the smooth touch point can also be determined in a weighted position manner based on the above-mentioned method of determining the smooth touch point.

[0082] Step 104: Send the information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be updated based on the information of the smooth touch point.

[0083] In the embodiment of the present application, after the relevant information of the smooth touch point is determined, it is sent to the target layout control such as the graphic control, so that the target layout control such as the graphic control determines the dirty area to be updated based on the time information of the smooth touch point. In the embodiment of the present application, the graphical interface is not updated according to the actual touch point, but when the synchronization event arrives, the two cached actual touch points are smoothed, so that the update of the graphical interface based on the touch operation is more uniform, and frame skipping and jittering will not occur, making the graphical interface more stable, thereby improving the user experience.

[0084] Step 105, triggering the rendering thread to render the dirty area and update the interface graphics.

[0085] In the embodiment of the present application, after the graphic control marks the dirty area, it sends a rendering event to the rendering thread, synchronously updates the dirty area rendering tree, submits the rendering result, and displays the updated graphic interface. In this way, the embodiment of the present application can perfectly and smoothly update the high-frequency reporting events to the display system.

[0086] Although not explicitly described, the touch processing method disclosed in the embodiments of the present application can be applied to various operating systems, including but not limited to Linux, Windows, Android, etc.

[0087] In one embodiment of the present application, the disclosed method is applied to a Linux operating system, i.e., GNU / Linux, such as Ubuntu, Debian, RedHat, etc. The Linux operating system is not limited to running on a physical machine, but can also be an operating system hosted on other operating systems, such as WSL2 in a Windows system.

[0088] Figure 3 This is a schematic diagram of the graphical interface update cycle of the embodiment of the present application, such as Figure 3 As shown, by comparing the Touch point positions before and after sampling, it can be seen that through the above-mentioned smooth sampling processing of the embodiment of the present application, the sliding behavior of moving 10 pixels every 10ms is smoothed to achieve a graphical interface update of 16 pixels every 16ms. The inter-frame deviation of the graphical interface update is 0, which completely eliminates the jitter and frame skipping between frames, greatly improves the display quality of the graphical interface, and enhances the user experience.

[0089] In the embodiment of the present application, in order to ensure that the system can give priority to responding to touch events, a higher priority is set for the touch events so that the touch events can be scheduled first. Specifically, a scheduling strategy is set for the touch events; wherein the scheduling strategy includes: setting a scheduling priority for the touch events, the scheduling priority of the touch events being higher than the scheduling priorities of other threads or processes; when scheduling the touch events, the time slice polling scheduling is replaced by the priority scheduling.

[0090] Dynamically configure the CPU scheduling policy of the Touch receiving process to improve the throughput of Touch events during the receiving and distribution process, thereby reducing the fluctuation problem caused by cross-process or cross-thread. Specifically, you can directly access the input system process driven by the Touch device, configure the FIO scheduling policy to make the CPU running priority of Touch higher than other processes, and use the thread priority queue policy instead of the time slice polling scheduling policy. When the Touch receiving thread can always be at the head of the priority queue, it will not be easily preempted by other threads.

[0091] In the embodiment of the present application, the priority adjustment of the touch event is also supported. Specifically, when it is detected that the application or UI window corresponding to the touch event is switched to the background, the priority of the touch event is lowered; or, when it is detected that the application or UI window corresponding to the low-priority touch event is switched from the background to the foreground, the priority of the touch event is restored.

[0092] In the embodiment of the present application, the CPU scheduling priority of the main thread can be dynamically changed according to the different running states of different client application processes. For example, when the process switches to the foreground and starts to respond to UI operations, the CPU scheduling priority of the main thread is increased, and when the process switches to the background, the CPU scheduling priority of the main thread is reduced to give up the CPU.

[0093] The embodiment of the present application actively smoothes multiple touch points and can very smoothly update high-frequency point reporting events to the update of the display interface, thereby avoiding jitter and frame skipping of the graphical interface and providing users with a better user experience.

[0094] Figure 4 FIG. 1 is a schematic diagram of the structure of a touch processing device based on a graphic display system according to an embodiment of the present application. Figure 4 As shown, the touch processing device based on the graphics display system of the embodiment of the present application includes:

[0095] A first detection unit 40, used to detect a touch operation;

[0096] A generating unit 41, configured to generate a touch event in response to detecting a touch operation;

[0097] A first transmission unit 42, configured to send the touch event to a thread of the current UI window;

[0098] A first triggering unit 43 is used to trigger the thread of the current UI window to cache the touch event and generate a window update request;

[0099] A second detection unit 44, configured to detect synchronization events;

[0100] A determination unit 45, configured to respond to the window update request, obtain information of the touch point in the touch event, and determine information of the smooth touch point based on the information of the touch point;

[0101] A second transmission unit 46, configured to send the information of the smooth touch point to a target layout control;

[0102] A second triggering unit 47, configured to trigger the target layout control to determine a dirty area to be updated based on the information of the smooth touch point;

[0103] The third triggering unit 48 is used to trigger the rendering thread to render the dirty area and update the interface graphics.

[0104] In one embodiment, the determining unit 45 is further configured to:

[0105] In the touch events in the thread cache of the current UI window, a set number of touch points closest to a sampling time are searched, and the time and position information of the smoothed touch points are determined based on the time information of the found touch points and the sampling time.

[0106] In one embodiment, the set number is two;

[0107] Correspondingly, the determining unit 45 is further configured to:

[0108] When it is determined that the sampling time is between the times of the two touch points, a first offset between the time of a first touch point and the time of a second touch point among the two touch points, and a second offset between the sampling time and the time of the second touch point are determined, and a first weight is determined based on the first offset and the second offset; and the time and position information of the smooth touch point are calculated based on the time of the second touch point, the first weight and the first offset.

[0109] In one embodiment, the set number is two;

[0110] Correspondingly, the determining unit 45 is further configured to:

[0111] When it is determined that the sampling time is outside the time of the two touch points, a first offset between the time of a first touch point and the time of the second touch point among the two touch points, and a third offset between the time of the first touch point and the sampling time are determined, and a second weight is determined based on the first offset and the third offset; the time and position information of the smooth touch point are calculated based on the time of the first touch point, the second weight and the inverse of the first offset.

[0112] In one embodiment, Figure 4 On the basis of the touch control processing device based on the graphic display system shown in the figure, the touch control processing device based on the graphic display system in the embodiment of the present application further includes:

[0113] Set the unit ( Figure 4 ), which is used to set a scheduling strategy for the touch event; wherein the scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than the scheduling priority of other threads or processes; when scheduling the touch event, replacing the time slice polling scheduling method with the priority scheduling method.

[0114] In one embodiment, Figure 4 On the basis of the touch control processing device based on the graphic display system shown in the figure, the touch control processing device based on the graphic display system in the embodiment of the present application further includes:

[0115] The second detection unit ( Figure 4 (not shown) for lowering the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or

[0116] When it is detected that the application or window corresponding to the touch event of low priority is switched from the background to the foreground, the priority of the touch event is restored.

[0117] In an exemplary embodiment, the first detection unit 40, the generation unit 41, the first transmission unit 42, the first trigger unit 43, the second detection unit 44, the determination unit 45, the second transmission unit 46, the second trigger unit 47, the third trigger unit 48, the setting unit and the second detection unit, etc. can be implemented by one or more central processing units (CPU, Central Processing Unit), application specific integrated circuits (ASIC, Application Specific Integrated Circuit), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (PLD, Programmable Logic Device), complex programmable logic devices (CPLD, Complex Programmable Logic Device), field programmable gate arrays (FPGA, Field-Programmable Gate Array), general-purpose processors, controllers, microcontrollers (MCU, Micro Controller Unit), microprocessors (Microprocessors), or other electronic components, and are used to execute the steps of the touch processing method based on the graphic display system of the aforementioned embodiment.

[0118] In the disclosed embodiment, Figure 4 The specific manner in which each unit in the touch control processing device based on the graphic display system performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0119] Below, reference Figure 5The electronic device 11 according to the embodiment of the present application is described in detail. The specific implementation of the electronic device 11 includes but is not limited to various forms of computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, tablet computers, wearable devices, and other similar computing devices.

[0120] like Figure 5 As shown, the electronic device 11 includes one or more processors 111 and a memory 112 .

[0121] The processor 111 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

[0122] The memory 112 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 111 may run the program instructions to implement the touch processing method based on the graphics display system of each embodiment of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0123] In one example, the electronic device 11 may further include: an input device 113 and an output device 114, which are connected via a bus system and / or other forms of connection mechanisms ( Figure 5 ) interconnected.

[0124] The input device 113 may include, for example, a keyboard, a mouse, etc.

[0125] The output device 114 can output various information to the outside, including the determined distance information, direction information, etc. The output device 114 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0126] Of course, to simplify, Figure 5Only some of the components related to the present application in the electronic device 11 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 11 may also include any other appropriate components.

[0127] The embodiment of the present application further records a storage medium on which an executable program is stored. The executable program is executed by a processor to execute the steps of the touch processing method based on the graphic display system of the above embodiment.

[0128] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

[0129] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0130] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.

[0131] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0133] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0134] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0135] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0136] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A touch processing method based on a graphics display system, characterized in that: The method comprises: In response to detecting a touch operation, generating a touch event, and sending the touch event to a thread of the current UI window; Triggering the thread of the current UI window to cache the touch event and generate a window update request; After detecting the synchronization event, responding to the update window request, obtaining the information of the touch points in the touch event, and searching for the two touch points closest to the sampling time in the touch event; When it is determined that the sampling time is between the times of the two touch points, a first offset between the time of a first touch point and the time of a second touch point of the two touch points and a second offset between the sampling time and the time of the second touch point are determined, and a first weight is determined based on the first offset and the second offset; the time and position information of the smoothed touch point is calculated based on the time of the second touch point, the first weight and the first offset; or, When it is determined that the sampling time is outside the time of the two touch points, a first offset between the time of a first touch point and the time of a second touch point of the two touch points, and a third offset between the time of the first touch point and the sampling time are determined, and a second weight is determined based on the first offset and the third offset; and the time and position information of the smoothed touch point are calculated based on the time of the first touch point, the second weight and the inverse of the first offset; Sending the time and position information of the smooth touch point to the target layout control, triggering the target layout control to determine the dirty area to be updated based on the time and position information of the smooth touch point; The rendering thread is triggered to render the dirty area and update the interface graphics.

2. The method according to claim 1, characterized in that The method further comprises: Setting a scheduling strategy for the touch event; The scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than the scheduling priority of other threads or processes; when scheduling the touch event, replacing the time slice polling scheduling method with the priority scheduling method.

3. The method according to claim 1, characterized in that: The method further comprises: When detecting that the application or UI window corresponding to the touch event is switched to the background, lowering the priority of the touch event; or When it is detected that the application or UI window corresponding to the touch event of low priority is switched from the background to the foreground, the priority of the touch event is restored.

4. A touch processing device based on a graphics display system, characterized in that: The device comprises: A first detection unit, used to detect a touch operation; A generating unit, configured to generate a touch event in response to detecting a touch operation; A first transmission unit, used to send the touch event to the thread of the current UI window; A first triggering unit, configured to trigger the thread of the current UI window to cache the touch event and generate a window update request; A second detection unit, configured to detect a synchronization event; a determination unit, configured to respond to the update window request, obtain the information of the touch points in the cache touch event, search for the two touch points closest to the sampling time in the touch event, determine a first offset between the time of a first touch point and the time of a second touch point among the two touch points, and a second offset between the sampling time and the time of the second touch point when determining that the sampling time is between the times of the two touch points, and determine a first weight based on the first offset and the second offset; calculate the time and position information of the smooth touch point based on the time of the second touch point, the first weight and the first offset, or, determine that the sampling time is outside the times of the two touch points, determine the first offset between the time of the first touch point and the time of the second touch point among the two touch points, and a third offset between the time of the first touch point and the sampling time, and determine a second weight based on the first offset and the third offset; calculate the time and position information of the smooth touch point based on the time of the first touch point, the second weight and the inverse of the first offset; A second transmission unit, used for sending the time and position information of the smooth touch point to a target layout control; A second triggering unit, used for triggering the target layout control to determine a dirty area to be updated based on the time and position information of the smooth touch point; The third triggering unit is used to trigger the rendering thread to render the dirty area and update the interface graphics.

5. The device according to claim 4, characterized in that The device also includes: A setting unit is used to set a scheduling strategy for the touch event; wherein the scheduling strategy includes: setting a scheduling priority for the touch event, the scheduling priority of the touch event is higher than the scheduling priority of other threads or processes; when scheduling the touch event, replacing the time slice polling scheduling method with the priority scheduling method.

6. The device according to claim 4, characterized in that The device also includes: A second detection unit is configured to lower the priority of the touch event when detecting that the application or window corresponding to the touch event is switched to the background; or When it is detected that the application or window corresponding to the touch event of low priority is switched from the background to the foreground, the priority of the touch event is restored.

7. An electronic device, comprising a processor, a memory, and an executable program stored in the memory and capable of being run by the processor, wherein the processor executes the steps of the touch processing method based on a graphics display system as described in any one of claims 1 to 3 when running the executable program.

8. A storage medium storing an executable program, wherein when the executable program is executed by a processor, the steps of the touch processing method based on a graphics display system as described in any one of items 1 to 3 are implemented.

Images