Smart touch event consumption

Abstract

A method for processing a touch event of a touch screen of a computing device by a processing system includes receiving a touch event after a start rendering frame and before a start rendering sequential frame (906); setting a touch event processing point (902, 904) prior to a starting point for starting rendering of the sequential frame; and processing the touch event at a touch event processing point before the starting point for starting rendering the sequential frame regardless of how the synchronization signal is displayed (910, 912).

Description

Background Technology

[0001] Input and display systems are implemented in touchscreen computing devices to respond to touch input to the touchscreen and to display the response to the touch input at the touchscreen. Touch input sampling and display refresh rates can use different rates, where one or more touch inputs may be sampled before rendering the display is performed, but not processed in time to be rendered in the display. The difference between touch input sampling and display refresh rates can result in latency in real-time visual feedback based on user input. Summary of the Invention

[0002] Various aspects provide methods, including methods for processing touch events of a touch-sensitive display or touchscreen of a computing device by a processing system, and means for implementing such methods. Aspects may include: receiving a touch event after the start of a rendering frame and before the start of rendering a sequential frame; setting a touch event processing point prior to the start point for starting rendering a sequential frame in response to receiving the touch event; and processing the touch event at the touch event processing point prior to the start point for starting rendering a sequential frame regardless of display synchronization signals. Some aspects may also include rendering the touch event within the sequential frame.

[0003] In some aspects, receiving a touch event after the start of rendering a frame and before the start of rendering a sequential frame occurs after the display synchronization signal is received during the rendering of a frame.

[0004] Some aspects may also include identifying the starting point for beginning the rendering of sequential frames. In some aspects, identifying the starting point for beginning the rendering of sequential frames may include identifying the end of a queue buffer configured to store display data for the frames. In some aspects, identifying the starting point for beginning the rendering of sequential frames may include identifying a sleep period prior to beginning the rendering of sequential frames.

[0005] Another aspect includes a processing system comprising a memory and one or more processors configured to perform operations of any of the methods outlined above. Another aspect includes a non-transitory processor-readable storage medium storing processor-executable software instructions configured to cause one or more processors of the processing system to perform operations of any of the methods outlined above. Another aspect includes a processing system having components for performing the functions of any of the methods outlined above. Attached Figure Description

[0006] The accompanying drawings, incorporated herein and forming part of this specification, illustrate exemplary embodiments of various implementations and, together with the general description given above and the detailed description given below, serve to interpret the features of the claims.

[0007] Figure 1 This is a component block diagram illustrating an example processing system suitable for implementing various implementation schemes.

[0008] Figure 2 This is a component block diagram illustrating examples of computing devices suitable for implementing various implementation schemes.

[0009] Figure 3 These are component block diagrams and flowcharts illustrating existing touch input sampling and display refresh examples.

[0010] Figure 4 These are component block diagrams and flowcharts illustrating examples of touch input sampling and display refresh according to various implementation schemes.

[0011] Figure 5 This is a flowchart illustrating an example of existing touch input sampling and display refresh.

[0012] Figure 6 This is a flowchart illustrating examples of touch input sampling and display refresh according to various implementation schemes.

[0013] Figure 7 These are component block diagrams and flowcharts illustrating examples of settings for displaying refreshed touch event handling points according to various implementation schemes.

[0014] Figure 8 These are component block diagrams and flowcharts illustrating examples of settings for displaying refreshed touch event handling points according to various implementation schemes.

[0015] Figure 9 This is a flowchart illustrating an example method for implementing touch input sampling and display refresh according to the implementation scheme.

[0016] Figure 10 This is a component block diagram illustrating an example mobile computing device suitable for implementing various implementation schemes.

[0017] Figures 11A-11C This is a component block diagram illustrating an example embedded vehicle computing system suitable for implementing various implementation schemes. Detailed Implementation

[0018] Various embodiments will be described in detail with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and embodiments are for illustrative purposes and are not intended to limit the scope of the claims.

[0019] Various implementations include methods for implementing touch events of a touch-sensitive display or touchscreen of a computing device that are processed (also referred to as consumed) by a computing system, and processing systems and / or computing devices that implement such methods. Implementations may include processing touch events received between the start of a rendering frame and the start of a rendering sequence frame, regardless of display synchronization signals. Processing one or more touch events may be based on touch event processing points set prior to the start point for beginning the rendering sequence frame.

[0020] The term "computing device" as used herein refers to a resident computing device, including personal computers, desktop computers, all-in-one computers, workstations, supercomputers, mainframe computers, embedded computers (such as those embedded in vehicles and other larger systems), computer systems within or configured for use in vehicles, servers, multimedia computers, and game consoles. The terms "computing device" and "mobile computing device" are used interchangeably herein to refer to any or all of the following: cellular phones, smartphones, personal or mobile multimedia players, personal data assistants (PDAs), laptop computers, tablet computers, convertible laptops / tablets (2-in-1 computers), smartbooks, ultrabooks, netbooks, handheld computers, wireless email receivers, internet-enabled multimedia cellular phones, mobile game consoles, wireless game controllers, and similar personal electronic devices including memory and programmable processors. The term "touchscreen" as used herein refers to any type of display used in computing devices that is configured to detect touches (and is therefore touch-sensitive) on a display surface and convert touches and swipes into information that a processing system can interpret as input (referred to as "touch events"). The term "touchscreen computing device" is used to refer to any computing device that has a touch-sensitive, combined or composite input and visual image / video display.

[0021] For ease of explanation and clarity, while various embodiments are described in terms of code (e.g., processor-executable instructions), the same principle applies to any data, such as code, program data, or other information stored in memory. The terms “code,” “data,” and “information” are used interchangeably herein and are not intended to limit the scope of the claims and specification to the types of code, data, or information used as examples in describing the various embodiments.

[0022] Input and display systems are implemented in touchscreen computing devices to respond to touch input to the touchscreen and to display the response to the touch input at the touchscreen. Touch input sampling and display refresh rates can use different rates, where one or more touch inputs may be sampled before rendering the display is performed, but not processed in time to be rendered in the display. The difference between touch input sampling and display refresh rates can result in latency in real-time visual feedback based on user input.

[0023] For example, the input and display systems are two separate modules in the Android operating system. Touch events cannot be processed immediately after a Touch Interruption Request (IRQ) is reported; they are processed in batches upon receiving a Vertical Sync (VSync) signal. If a touch event is received between a rendering of a display and a rendering of a sequential display, and the VSync signal is received after the start of the sequential rendering, the touch event will not be rendered until at least the next rendering of the next sequential display. Therefore, the delay in rendering touch events is at least one additional rendering cycle from the time the touch event is received.

[0024] The implementation addresses and overcomes the aforementioned latency issues in real-time visual feedback based on user input by enabling touch events received between displayed renderings and sequentially displayed renderings to be processed before the sequentially displayed renderings, rather than after the sequentially displayed renderings and before at least the next sequentially displayed rendering. In other words, the implementation eliminates latency for at least one rendering cycle from the time a touch event is received.

[0025] A display including any sequential display can be a visual image and / or video rendered on the display device of a touchscreen computing device. The unit of the display can include frames rendered on the display device of the touchscreen computing device, including any sequential frames. A sequential display including any sequential frames can be a display / frame immediately following another display / frame. In other words, no other (sequential) display / frames may be rendered between displays / frames and sequential displays / frames. The display device can be a touch-sensitive, combined, or composite input and visual image / video display device.

[0026] Various implementations may include receiving one or more touch events between the starting point for rendering a frame on the display device of the touchscreen computing device and the starting point for rendering a sequence of frames on the display device of the touchscreen computing device. These one or more touch events may include data about a user's touch on the display device of the touchscreen computing device, such as the timing of the touch, the coordinates of the touch on the display, and, in some devices, the pressure applied to the display, pressure-time distribution, etc. A single touch event may be a tap. Examples of multiple touch events may include double (or triple) taps, touch and swipe input, touch and drawing or trajectory input. As used herein, the reference to "touch event" is intended to encompass both single touch events and various types of multiple touch events, and therefore encompasses one or more touch events.

[0027] One or more touch events received after the start point for rendering a frame may not be processed in time to render that one or more touch events in the frame. In the case of touch events received before the start point for rendering a sequential frame, one or more touch events may be processed in time to render that one or more touch events in the sequential frame.

[0028] One or more touch events can be processed before rendering sequential frames and can be rendered within sequential frames. Processing one or more touch events can begin at a touch event processing point prior to the start point used to render the sequential frame, regardless of display synchronization signals (such as VSync, FreeSync, G-Sync, etc.). For example, a touch event processing point can be a specified measurement, such as time, event, etc., prior to the start point used to render the sequential frame. In some implementations where a display synchronization signal is received, the display synchronization signal can be ignored. In some implementations, a display synchronization signal may not be implemented.

[0029] In some implementations, the starting point for rendering the sequential frame can be identified based on one or more factors, such as the end point of the queue buffer before the sequential frame, a sleep timer in the application logic thread, etc. Touch event handling points can be set before the starting point for rendering the sequential frame. For example, a touch event handling point can be set before the end point of the queue buffer before the sequential frame. As another example, a touch event handling point can be set before the end of a sleep timer in the application logic thread. One or more touch events can be processed at the touch event handling point before the starting point for rendering the sequential frame and rendered within the sequential frame.

[0030] Figure 1 This is a component block diagram illustrating an example computing device 100 suitable for implementing any of the various implementation schemes. The various implementation schemes can be implemented on multiple single-processor and multi-processor computer systems (including processing systems).

[0031] refer to Figure 1The illustrated example computing device 100 (which may be a system-in-package in some embodiments) may include any combination of processing systems 102, 104 coupled to a clock 106, a voltage regulator 108, at least one subscriber identity module (SIM) 168 and / or SIM interface, DRAM 170, a general-purpose flash storage (UFS) device 172, and a wireless transceiver 166 configured to transmit and receive wireless communications to and from wireless computing devices such as base stations, wireless devices, and / or computing devices via an antenna (not shown). In some embodiments, the first processing system 102 may operate as a central processing unit (CPU) of the computing device 100, which implements instructions by executing arithmetic, logic, control, and input / output (I / O) operations specified by instructions from a software application. In some embodiments, the second processing system 104 may operate as a dedicated processing unit. For example, the second processing system 104 can operate as a dedicated 5G processing unit responsible for managing high-capacity, high-speed (e.g., 5Gbps) and / or ultra-high frequency short-wavelength (e.g., 28GHz millimeter wave spectrum) communications.

[0032] The term "System-on-a-Chip" (SoC) is used herein to refer to a collection of interconnected electronic circuits, generally but not exclusively including processing devices, memory, and communication interfaces. Processing systems 102 and 104 may include various types of processors, some of which may include multiple processor cores. Non-limiting examples of processors that may be included in computing device 100 and implemented in or coupled to processing systems 102 and 104 include general-purpose processors, central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), accelerated processing units (APUs), security processing units (SPUs), neural network processing units (NPUs), subsystem processors for specific components of computing devices (such as image processors in camera subsystems or display processors in display devices), auxiliary processors, single-core processors, multi-core processors, controllers, and microcontrollers. Processing systems 102 and 104 may also embody other hardware and hardware combinations, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), other programmable logic devices, discrete gate logic components, transistor logic components, performance monitoring hardware, watchdog hardware, and time references. An integrated circuit can be configured such that its components reside on a single piece of semiconductor material; this can be called a system-on-a-chip (SoC).

[0033] Processing systems 102 and 104 may be implemented in one or more SoCs and / or may include circuitry coupled to multiple chips of one or more SoCs. Computing device 100 may include more than one processing system 102 or 104, thereby increasing the number of processors, wherein any one or more processors may include multiple processor cores. Computing device 100 may also include other processors (not shown) not associated with processing system 12. Processors may each be configured for a specific purpose that may be the same as or different from other processors of computing device 100. One or more of processors and processor cores with the same or different configurations may be grouped together.

[0034] The first processing system 102 may include a digital signal processor (DSP) 110, a modem processor 112, a graphics processor 114, an application processor (AP) 116, one or more coprocessors 118 (e.g., vector coprocessors) connected to one or more of these processors, memory 120, custom circuitry 122, system components and resources 124, a host controller 162, an interconnect / bus module 126, one or more sensors 130 (e.g., accelerometers, temperature sensors, pressure sensors, optical sensors, infrared sensors, analog sound sensors, etc.), a thermal management unit 132, and a thermal power envelope (TPE) component 134, or more thereof. The second processing system 104 may include a low-power processor 152, a power management unit 154, an interconnect / bus module 164, a BT controller 156, memory 158, and various additional processors 160, such as application processors, packet processors, etc.

[0035] Each processor 110, 112, 114, 116, 118, 152, 160 may include one or more cores, and each processor / core may perform operations independently of the other processors / cores. For example, the first processing system 102 may include an operating system of a first type (e.g., UNIX-based). ® OS, LINUX ® iOS ® MACOS ® ANDROID ® Processors that execute a second type of operating system (e.g., Microsoft Windows) and processors that run a second type of operating system (e.g., Microsoft Windows). ® The processor is a processor of type 110, 112, 114, 116, 118, 152, and 160. Furthermore, any or all of these processors may be included as part of a processor cluster architecture (e.g., synchronous processor cluster architecture, asynchronous or heterogeneous processor cluster architecture, etc.). The term "processing system" is used herein to encompass all possible processor types and architectures.

[0036] The first processing system 102 and the second processing system 104 may include various system components, resources, and custom circuitry for managing sensor data, analog-to-digital conversion, wireless data transmission, and performing other specialized operations, such as decoding data packets and processing encoded audio and video signals for rendering in web browsers or audio / video applications. For example, the system components and resources 124 of the first processing system 102 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components for supporting processors and software clients running on computing devices. System components and resources 124 and / or custom circuitry 122 may also include circuitry for interfacing with peripheral devices such as cameras, electronic display devices, wireless communication devices, external memory chips, etc.

[0037] The first processing system 102 and the second processing system 104 may communicate via an interconnect / bus module 150. In some embodiments, the interconnect / bus module may be a connection established by transceiver (i.e., receive and transmit) components within both processing system 102 and processing system 104. For example, the low-power processor 152 may include a universal asynchronous receiver-transmitter (UART), and the application processor 116 may include a multi-signal message (MSM) UART driver communicatively connected to the UART of the low-power processor 152.

[0038] Various processors 110, 112, 114, 116, and 118 can be interconnected via interconnect / bus module 126 to one or more memory elements 120, system components and resources 124, custom circuitry 122, and thermal management unit 132. Similarly, low-power processor 152 can be interconnected via interconnect / bus module 164 to power management unit 154, BT controller 156, memory 158, and various additional processors 160. Interconnect / bus modules 126, 150, and 164 may include arrays of reconfigurable logic gates and / or implement bus architectures (e.g., CoreConnect, AMBA, etc.). Interconnect / bus modules 126, 150, and 164 may include any number of interconnect buses and bridges, depending on the specific application of processing systems 102 and 104 and overall design constraints. Communication may be provided by advanced interconnects, such as high-performance on-chip networks (NoC).

[0039] The first processing system 102 and / or the second processing system 104 may further include input / output modules (not illustrated) for communicating with resources outside the processing system, such as clock 106, voltage regulator 108, one or more wireless transceivers 166, and at least one SIM 168 and / or SIM interface (i.e., an interface for accepting one or more SIM cards)). Resources outside the processing system (e.g., clock 106, voltage regulator 108) may be shared by two or more internal processing system processors / cores. At least one SIM 168 (or one or more SIM cards coupled to one or more SIM interfaces) may store information supporting multiple subscriptions, including a first 5G NR subscription and a second 5G NR subscription, etc.

[0040] In addition to the example computing device 100 discussed above, various implementations can be implemented in a wide variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0041] In some implementations, the various processors of processing system 102 and processing system 104 may reside within the same processing system. For example, application processor 116 and low-power processor 152 may reside within the same processing system, such as in a single processing system of a wearable device.

[0042] Figure 2 This is a component block diagram illustrating examples of touchscreen computing devices according to various implementation schemes. (Reference) Figures 1-2 Touchscreen computing device 202 (e.g., Figure 1 The computing device 100 may include a processing system 206 (e.g., Figure 1 Processing systems 102 and 104 in the process may include those coupled to electronic storage device 204 (e.g., Figure 1 The memory 120, 158 in the memory and the wireless transceiver 266 (e.g., Figure 1 One or more processors (e.g., wireless transceiver 166) in the wireless transceiver 166 Figure 1 Processors 110, 112, 114, 116, 118, 152, and 160 are mentioned.

[0043] In some embodiments, the touchscreen computing device 202 may be part of a communication system 200, which may include at least the touchscreen computing device 202 and a wireless computing device 222 (e.g., Figure 1The touchscreen computing device 202 and the wireless computing device 222 can communicate via a wireless communication network 220. A wireless transceiver 266 can be configured to receive messages to be transmitted from the processing system 206, and to transmit such messages to the wireless computing device 222 via the wireless communication network 220 via an antenna (not shown). Similarly, the wireless transceiver 266 can be configured to receive messages transmitted from the wireless computing device 222 via the wireless communication network 220, and to transmit messages (e.g., via a modem that demodulates the messages) Figure 1 The modem 152 in the middle transmits the data to the processing system 206.

[0044] The processing system 206 can be configured via machine-readable instructions 208. Machine-readable instructions 208 may include one or more instruction modules 210, 212, 214, 216, and 218. Instruction modules 210 to 218 may include computer program modules. Instruction modules 210 to 218 may include one or more of a touch event module 210, a frame rendering start point module 212, a touch event processing point module 214, a touch event processing module 216, a frame rendering module 218, and / or other instruction modules.

[0045] Touch event module 210 can be configured to receive touch events. For example, touch event module 210 can receive data from one or more touch inputs by a user to the display device (not shown) of touchscreen computing device 202. The data from one or more touch inputs may include one or more of the following: position, size, direction of movement, speed of movement, time, duration, number of touches, pressure, pattern, etc. on the display device.

[0046] The frame rendering start point module 212 can be configured to identify the starting point for rendering frames (including sequential frames). When a touch event is received before rendering a sequential frame, the frame rendering start point module 212 can identify the starting point for rendering the sequential frame. The starting point for rendering the sequential frame can be identified based on one or more factors, such as the end point of the queue buffer before the sequential frame, a sleep timer in the application logic thread, etc.

[0047] Touch event handling point module 214 can be configured to set touch event handling points, which can be points where processing of one or more touch events can begin. Touch event handling points can be set before the start point for rendering sequential frames. For example, a touch event handling point can be set before the end point of the queue buffer before the sequential frame. As another example, a touch event handling point can be set before the end of a sleep timer in an application logic thread. In some implementations, touch event handling points can be set as constant and / or variable measurements before the start point for rendering sequential frames. For example, the measurement can be time, such as measurements in milliseconds (ms), which can include fractions of 1 ms up to 100 ms, such as 1 ms. As another example, the measurement can be events, such as multiple process loops, clock loops, etc. Setting touch event handling points can include generating a signal and sending it to processing system 206, setting a flag to be read by processing system 206 at a register (not shown), etc.

[0048] Touch event handling module 216 can be configured to identify touch event handling points and handle one or more touch events. Identifying touch event handling points may include interpreting indications configured to indicate touch event handling points to processing system 206, such as signals, flags in registers, etc.

[0049] Processing one or more touch events may begin at a touch event processing point prior to the starting point used to render the sequential frame. Processing one or more touch events may include converting data from one or more touch inputs by the user to the display device of the touchscreen computing device 202 into data for rendering the sequential frame on the display device. For example, a touch event may be converted into movement of an item in the sequential frame relative to the rendered frame. As another example, a touch event may be converted into manipulation of the sequential frame relative to the rendered frame, such as scaling, panning, rotating, skewing, changing the viewpoint, etc. As yet another example, a touch event may be converted into implementing application-specific actions in the sequential frame, such as game-based actions, including defensive actions, attack actions, and item state change actions.

[0050] The frame rendering module 218 can be configured to render sequential frames using data generated by processing one or more touch events for rendering sequential frames on a display device.

[0051] Electronic storage device 204 may include a non-transitory storage medium that electronically stores information. The electronic storage medium of electronic storage device 204 may include one or both of a system storage device that is integrated with (i.e., substantially non-removable) the touchscreen computing device 202 and / or a removable storage device that is removably connected to the touchscreen computing device 202 via, for example, a port (e.g., a Universal Serial Bus (USB) port, a FireWire port, etc.) or a drive (e.g., a disk drive, etc.). Electronic storage device 204 may include one or more of optically readable storage media (e.g., optical discs, etc.), magnetically readable storage media (e.g., magnetic tape, magnetic hard disk drives, floppy disk drives, etc.), charge-based storage media (e.g., EEPROM, RAM, etc.), solid-state storage media (e.g., flash drives, etc.), and / or other electronically readable storage media. Electronic storage device 204 may include one or more virtual storage resources (e.g., cloud storage devices, virtual private networks, and / or other virtual storage resources). Electronic storage device 204 may store software algorithms, information determined by processing system 206, information received from wireless computing device 222, one or more touch events, data for rendering sequential frames, or other information that enables touchscreen computing device 202 to function as described herein.

[0052] The processing system 206 may be configured to provide information processing capabilities in the touchscreen computing device 202. Therefore, the processing system 206 may include one or more of the following: a digital processor, an analog processor, digital circuitry designed to process information, analog circuitry designed to process information, a state machine, and / or other mechanisms for electronically processing information.

[0053] Although processing system 206 is illustrated as a single entity, this is for illustrative purposes only. In some embodiments, processing system 206 may include multiple processing units and / or processor cores. The processing units may be physically located within the same device, or processing system 206 may represent the processing functionality of multiple devices operating collaboratively.

[0054] Processing system 206 may be configured to execute modules 210 to 218 and / or other modules by means of: software; hardware; firmware; a combination of software, hardware, and / or firmware; and / or other mechanisms for configuring processing capabilities on processing system 206. As used herein, the term "module" may refer to any component or set of components that performs the functionality attributable to that module. This may include one or more physical processors, processor-readable instructions, circuitry, hardware, storage media, or any other component during the execution of processor-readable instructions.

[0055] The description of the functionality provided by the different modules 210 to 218 described herein is for illustrative purposes and is not intended to be limiting, as any one of modules 210 to 218 may provide more or less functionality than described. For example, one or more modules 210 to 218 may be eliminated, and some or all of their functionality may be provided by other modules 210 to 218. As another example, the processing system 206 may be configured to execute one or more additional modules that may perform some or all of the functionality belonging to one of the modules 210 to 218.

[0056] Figure 3 These are component block diagrams and flowcharts illustrating existing touch input sampling and display refresh examples. (Reference) Figures 1-3 Operating system 300 (e.g., based on Android) ® WINDOWS ® iOS ® MACOS ® LINUX ® Operating systems, etc.) can be used on touchscreen computing devices (e.g., Figure 1 The computing device 100 in Figure 2 The processing system of the touch screen computing device 202 (e.g., Figure 1 and Figure 2 The operating system 300 is implemented in the processing systems 102, 104, and 206. The operating system 300 may include the input system 302 and the display system 304.

[0057] Input system 302 may be configured to receive touch interrupt requests (IRQs) from the display device (not shown) of the touchscreen computing device at process 306. For example, input system 302 may be configured to receive one or more touch IRQs generated in response to one or more touch inputs from a user to the display device controller (not shown).

[0058] A touch IRQ can trigger input system 302 to read a touch event at process 308 and dispatch the touch event to the application. For example, input system 302 can retrieve the touch event from memory (e.g., ...). Figure 1 The memory in the 120, 158, Figure 2 The electronic storage device 204 reads touch events associated with received touch IRQs. The input system 302 can read data from one or more touch inputs by the user to the display device included in the touch events. The input system 302 can send touch events to applications implemented via the operating system 300 in the processing system.

[0059] Display system 304 may receive touch events from input system 302 at process 310. Display system 304 may receive data from one or more touch inputs by the user to the display device included in the touch events. Figure 3 In the illustrated example, display system 304 may receive a touch event at process 310, while simultaneously processing an application queue buffer at process 318a. The queue buffer may include data for displaying rendered frames on the display device. After processing the queue buffer at process 318a, display system 304 may render frames for the application at process 316a.

[0060] At process 312, display system 304 may receive a display synchronization signal. In some examples, the display synchronization signal may be a periodic signal configured to control the processing of one or more touch events (such as a touch event received at process 310) received prior to receiving the display synchronization signal at process 312. In response to receiving the display synchronization signal at process 312, display system 304 may process the touch event at process 314. Processing the touch event may include converting data from one or more touch inputs by the user to the display device of the touchscreen computing device into data for rendering a frame. Rendering a frame may include data for generating a frame for display on the display device, which may be stored in a queue buffer.

[0061] exist Figure 3 In the illustrated example, display system 304 may receive a display synchronization signal at process 312 and process a touch event at process 314, while simultaneously rendering a frame at process 316a. Receiving the display synchronization signal at process 312 prompts display system 304 to process the touch event at process 314. Although the touch event has been received at process 310 before rendering the frame at process 316a, display system 304 cannot process the touch event before rendering the frame at process 316a so that it will be included in the frame rendered at process 316a. Display system 304 may store the data from the touch event processed at process 314 for rendering a frame on the display device in a cache (e.g., ...). Figure 1 The memory in the 120, 158, Figure 2 The electronic storage device 204 is used for later processing.

[0062] After the rendering frame is completed at process 316a, the display system 304 can queue and buffer the data of the rendered frame for display on the display device at process 318b. Figure 3 As illustrated in the example, after processing the queue buffer at process 318b, the display system 304 renders a frame at process 316b using data from the touch event data processed in process 314 for rendering the frame on the display device.

[0063] exist Figure 3 In the illustrated example, although the display system 304 receives a touch event in process 310 before rendering a frame in process 316a, it does not process the touch event received in process 310 until it receives a display synchronization signal in process 312 while rendering the frame in process 316a. In this case, waiting to receive the display synchronization signal in process 312 prevents the display system 304 from processing the touch event received in process 310 before rendering the frame in process 316a, and the touch event cannot be included in the frame rendered in process 316a. Instead, the display system 304 is forced to wait to include the touch event received in process 310 while rendering the frame in process 316b. This forced waiting results in a delay of at least one rendering process 316a between when the display system 304 receives the touch event in process 310 and when it includes the touch event in the frame rendered in process 316b.

[0064] Figure 4 These are component block diagrams and flowcharts illustrating examples of touch input sampling and display refresh according to various implementation schemes. (Reference) Figures 1-4 The operating system 400 (e.g., operating systems based on Android, Windows, iOS, macOS, Linux, etc.) can implement one or more modules (e.g., Figure 2 Touchscreen computing devices (e.g., modules 210 to 218) Figure 1 The computing device 100 in Figure 2 The processing system of the touch screen computing device 202 (e.g., Figure 1 and Figure 2 The operating system 400 is implemented at processing systems 102, 104, and 206. The operating system 400 may include, as referenced herein. Figure 3 The input system 302 and the display system 404 are described.

[0065] Input system 302 can be configured to receive one or more touch IRQs from the display device (not shown) of the touchscreen computing device at processes 406a, 406b, and 406c. For example, input system 302 can be configured to receive one or more touch IRQs generated in response to one or more touch inputs from a user to the display device from a display device controller (not shown).

[0066] A touch IRQ can trigger input system 302 to read touch events at processes 408a, 408b, and 408c and dispatch the touch events to the application. For example, input system 302 can retrieve touch events from memory (e.g., ...). Figure 1 The memory in the 120, 158, Figure 2The electronic storage device 204 reads touch events associated with received touch IRQs. The input system 302 can read data from one or more touch inputs by the user to the display device included in the touch events. The input system 302 can send touch events to applications implemented via the operating system 400 in the processing system.

[0067] Display system 404 may receive touch events from input system 302 at processes 410a, 410b, and 410c. Display system 304 may receive one or more touch inputs from the user to the display device included in the touch events. Figure 4 In the illustrated example, display system 404 may receive touch events in processes 410a, 410b, and 410c between the time display system 304 begins rendering frames for the application at process 416a and the time display system 304 begins rendering sequential frames for the application at process 416b.

[0068] In some embodiments, display system 304 may receive touch events in processes 410a, 410b, 410c before and / or after display system 404 may receive a display synchronization signal in process 412. In some embodiments, there may be no display synchronization signal. Regardless of when and / or whether a display synchronization signal is received in process 412, all touch events received in processes 410a, 416b, 416c between the start of frame rendering in process 410a and the start of sequential frame rendering in process 410b may be processed by display system 404 in process 414. Processing touch events may include converting data from one or more touch inputs by a user to the display device of a touchscreen computing device into data for rendering frames on the display device. Processing touch events may include storing the data for rendering frames on the display device in a cache (e.g., ...). Figure 1 The memory in the 120, 158, Figure 2 The touch event can be handled at the touch event handling point before the start of rendering the sequential frame at process 416b, based on the starting point used for rendering the sequential frame at process 416b.

[0069] Display system 404 can process the queue buffer at process 418a and subsequently render the frame at process 416a. Figure 4 In the illustrated example, after the queue buffer is processed at process 418a and the frame rendering begins at process 416a, touch events received at processes 410a, 410b, and 410c are received, and the received touch events may not be included in the queue buffer process at process 418a and the frame rendered at process 416a.

[0070] Display system 404 can process the queue buffer for the frames rendered in process 416a at process 418b and subsequently render sequential frames at process 416b. Figure 4 In the illustrated example, touch events are received at processes 410a, 410b, and 410c and processed at process 414 before the rendering of a frame begins at process 416a, and all received and processed touch events are included in the sequential frames rendered at process 416b.

[0071] exist Figure 4 In the illustrated example, if a touch event is received in processes 410a, 410b, and 410c between the start of rendering a frame at process 416a and the start of rendering a sequential frame at process 416b, the received touch event can be processed at process 414 and rendered in a sequential frame at process 416b. Regardless of when the touch event is received relative to a display synchronization signal and / or whether the display synchronization signal is present, touch events received in processes 410a, 410b, and 410c can be processed at process 414 and rendered in a sequential frame at process 416b. Without the control of the display synchronization signal, received touch events can be processed without waiting for the display synchronization signal, thus avoiding any delay in processing and rendering received touch events. By processing all received touch events at process 414 at a touch event processing point prior to the start point for rendering a sequential frame at process 416b, the delay in rendering received touch events caused by the control of the display synchronization signal in processing received touch events is avoided.

[0072] exist Figure 4 The number of touch events received and processed between the start of rendering consecutive frames (frames and sequential frames) in the illustrated example is for illustrative purposes. Those skilled in the art will recognize that fewer and / or more touch events can be received and processed between the start of rendering consecutive frames.

[0073] Figure 5 This is a flowchart illustrating an example of existing touch input sampling and display refresh. (Reference) Figures 1-5 The process flow timing diagram 500 illustrates the timing for receiving one or more touch events 504a, 504b, 504c, processing one or more touch events 506a, 506b, and rendering one or more touch events in one or more frames 510a, 510b, 510c relative to the display synchronization signal 502. The process flow timing diagram 500 can be illustrated in a touchscreen computing device (e.g., Figure 1 The computing device 100 in Figure 2 The processing system of the touch screen computing device 202 (e.g., Figure 1 and Figure 2The process is implemented at processing system 102, 104, 206.

[0074] exist Figure 5 In the illustrated example, a first touch event 504a is received during rendering frame N-1 510a and before the first edge 508a of the display synchronization signal 502. The first edge 508a of the display synchronization signal 502 occurs during rendering frame N-1 510a. The first touch event 506a is processed in response to the first edge 508a of the display synchronization signal 502. The first touch event is rendered in frame N 510b following the first edge 508a of the display synchronization signal 502.

[0075] During rendering frame N-1 510a, after the first edge 508a of display synchronization signal 502 and before rendering frame N 510b and the second edge 508b of display synchronization signal 502, a second touch event 504b is received. The second edge 508b of display synchronization signal 502 occurs during rendering frame N 510b. A second touch event 506b is processed in response to the second edge 508b of display synchronization signal 502. The second touch event is rendered in frame N+1 510c, following the second edge 508b of display synchronization signal 502.

[0076] A third touch event 504c is received after rendering frame N-1 510a, after the first edge 508a of display synchronization signal 502, and before rendering frame N 510b and the second edge 508b of display synchronization signal 502. The second edge 508b of display synchronization signal 502 occurs during rendering frame N 510b. The third touch event 506b is processed in response to the second edge 508b of display synchronization signal 502. The third touch event is rendered in frame N+1 510c, following the second edge 508b of display synchronization signal 502.

[0077] exist Figure 5In the illustrated example, the edges 508a and 508b of the display sync signal 502 control the timing of processing one or more touch events 506a and 506b so that these one or more touch events are included in the rendering of one or more frames 510a, 510b, and 510c. One or more touch events cannot be rendered before they are processed. For example, received touch events 504b and 504c cannot be processed before the edge 508b of the display sync signal 502. Although received before rendering frame N 510b, the received touch events 504b and 504c cannot be included in the rendering frame N 510b because the edge 508b of the display sync signal 502 occurs during the rendering of frame N 510b, since the received touch events 504b and 504c were not processed 506b before they were received. During rendering frame N 510b, touch events 504b and 504c received at 506b are processed based on edge 508b of display synchronization signal 502, and must wait to be included in rendering frame N+1 510c. Therefore, the received touch events 504b and 504c are delayed for at least the loop of rendering frame N 510b.

[0078] Figure 6 This is a flowchart illustrating examples of touch input sampling and display refresh according to various implementation schemes. (Reference) Figures 1-6 The process flow timing diagram 600 illustrates the timing for receiving one or more touch events 504a, 504b, 504c, processing one or more touch events 602, and rendering one or more touch events in one or more frames 510a, 510b, 510c relative to the display synchronization signal 502. The process flow timing diagram 600 can be illustrated in a touchscreen computing device (e.g., Figure 1 The computing device 100 in Figure 2 The processing system of the touch screen computing device 202 (e.g., Figure 1 and Figure 2 The process is implemented at processing systems 102, 104, and 206. The process flow sequence diagram 600 can illustrate the implementation of one or more modules (e.g., Figure 2 Modules 210 to 218 (210 to 218) and / or may include implementations of a display system (e.g., Figure 4 The operating system of the display system 404 in the middle (e.g., Figure 4 The process implemented in the processing system of the operating system 400.

[0079] exist Figure 6In the illustrated example, the first touch event 504a is received during rendering frame N-1 (frame) 510a and before rendering frame N (sequential frame) 510b. Processing of the first touch event 602 begins at the touch event processing point before rendering frame N 510b. After processing 602, the first touch event is rendered during rendering frame N 510b.

[0080] The second touch event 504b is received during rendering frame N-1 510a and before rendering frame N 510b. Processing of the second touch event 602 begins at the touch event processing point before rendering frame N 510b. After 602 is processed, the second touch event is rendered during rendering frame N510b.

[0081] The third touch event 504c is received after rendering frame N-1 510a and before rendering frame N 510b. The third touch event 602 is processed at the touch event processing point before rendering frame N 510b. After 602 is processed, the third touch event is rendered during rendering frame N510b.

[0082] exist Figure 6 In the illustrated example, each of touch events 504a, 504b, and 504c is received between the start rendering frame N-1 510a and the start rendering frame N 510b. All received touch events 504a, 504b, and 504c are processed at a touch event processing point 602 prior to rendering frame N 510b. All received touch events 504a, 504b, and 504c are processed regardless of the edges 508a and 508b of the display synchronization signal 502. In other words, regardless of when the touch events are received 504a, 504b, and 504c relative to the edges 508a and 508b of the display synchronization signal 502, the received touch events 504a, 504b, and 504c are processed at a touch event processing point prior to rendering frame N 510b. The processing of touch event 602 is not controlled by the edges 508a and 508b of the display synchronization signal 502, and touch event 602 can be processed without waiting for the edges 508a and 508b of the display synchronization signal 502. Therefore, all received touch events 504a, 504b, and 504c of 602 are processed and included in the rendering frame N 510b before rendering frame N 510b. The received touch events 504a, 504b, and 504c do not need to wait until after the start of rendering frame N 510b to be processed and do not need to wait to be included in rendering frame N+1 510c.

[0083] exist Figure 6The number of touch events received and processed between the start of rendering consecutive frames (frames and sequential frames) in the illustrated example is for illustrative purposes. Those skilled in the art will recognize that fewer and / or more touch events can be received and processed between the start of rendering consecutive frames.

[0084] Figure 7 and Figure 8 These are component block diagrams and flowcharts illustrating examples of settings for displaying touch event handling points for refresh, based on various implementation schemes. (Reference) Figures 1-8 Implement one or more modules (e.g., Figure 2 Modules 210 to 218 (210 to 218) and / or may include implementations of a display system (e.g., Figure 4 The operating system of the display system 404 in the middle (e.g., Figure 4 Touchscreen computing devices (e.g., operating system 400) Figure 1 The computing device 100 in Figure 2 The processing system of the touch screen computing device 202 (e.g., Figure 1 and Figure 2 The processing systems 102, 104, and 206 in the middle can implement the following description.

[0085] Touch event handling points 706, 806 can be configured to prompt processing of one or more touch events received between the start point for rendering a frame and the start point for rendering sequential frames 704, 804. The start point for rendering sequential frames 704, 804 can be a point where rendering of sequential frames 702, 802 can begin. Touch event handling points 706, 806 can be set before the start point for rendering sequential frames 704, 804. In some embodiments, touch event handling points 706, 806 can be set as a constant and / or variable measurement before the start point for rendering sequential frames 704, 804. For example, the measurement can be time, such as measured in milliseconds (ms), which can include fractions of 1 ms up to 100 ms, such as 1 ms. As another example, the measurement can be an event, such as multiple process cycles, clock cycles, etc. Setting touch event handling points 706, 806 can include generating a signal and sending it to the processing system, setting a flag to be read by the processing system at a register (not shown), etc.

[0086] In some implementations, the starting point for rendering sequence frames 704 and 806 can be identified before setting touch event handling points 706 and 806. Then, touch event handling points 706 and 806 can be set based on them. For example, as... Figure 7 As illustrated, the touch event handling point 706 can be set before the end point of the queue buffer 700 before the rendering sequence frame 704. As another example, such as...Figure 8 As illustrated, touch event handling point 806 can be set before the end of sleep timer 800 in the application logic thread.

[0087] refer to Figure 7 The queue buffer 700 may include data for displaying rendered frames on the display device of the touchscreen computing device. The queue buffer 700 may include a limited amount of data for displaying the rendered frames, and rendering of sequential frames 704 may begin when all data at the queue buffer is read. The end of the queue buffer 700, or the end of the process for reading the queue buffer 700, may approximate the starting point for rendering sequential frames 704. The end of the queue buffer 700 may be identified, for example, based on the size of the queue buffer 700 and / or the amount of process and / or time required to read the queue buffer 700. Using the end of the queue buffer 700 to indicate the starting point for rendering sequential frames 704, a touch event handling point 706 may be set. For example, the touch event handling point 706 may be set as a constant and / or variable measurement prior to the starting point for rendering sequential frames 704.

[0088] refer to Figure 8 The sleep timer 800 may include data for waiting for the rendering of the sequential frame 804. The sleep timer 800 may include a finite amount of data and / or a finite time period for waiting for the rendering of the sequential frame 804, and the sequential frame 804 may be rendered when the sleep timer 800 expires. The end of the sleep timer 800, or the end of the process for implementing the sleep timer 800, may substantially indicate the starting point for rendering the sequential frame 804. The end of the sleep timer 800 may be identified, for example, based on the data of the sleep timer 800 and / or the amount of process and / or time required for the sleep timer 800 to expire. Using the end of the sleep timer 800 to indicate the starting point for rendering the sequential frame 804, a touch event handling point 806 may be set. For example, the touch event handling point 806 may be set as a constant and / or variable measurement prior to the starting point for rendering the sequential frame 804.

[0089] Figure 9 This is a flowchart illustrating example methods for implementing touch input sampling and display refresh according to various implementation schemes. (Reference) Figures 1-9 Method 900 can be used on a computing device (e.g., Figure 1 The computing device 100 in Figure 2 In a touchscreen computing device 202, in hardware, in a processing system (e.g., Figure 1 The processing systems 102 and 104, and processors 110, 112, 114, 116, 118, 152, and 160 are mentioned. Figure 2 The software executed in the processing system 206 (e.g., Figure 2Modules 210 to 218 in Figure 4 The method 900 is implemented in the operating system 400, display system 404, or in a combination of a processing system and dedicated hardware, including other individual components such as various memory / caches and various memory / cache controllers. Components used to implement method 900 may include the processing system (e.g., Figure 1 and Figure 2 Processing systems 102, 104, 206) or other processors (e.g., Figure 1 The processors in the method are 110, 112, 114, 116, 118, 152, and 160. Furthermore, one or more processors may be configured with software or firmware to perform some or all of the operations of method 900. To cover alternative configurations implemented in various embodiments, the hardware implementing method 900 is referred to herein as a "processing device".

[0090] In block 902, the processing system may identify a starting point for beginning the rendering of sequential frames. In some embodiments, the processing system that identifies the starting point for beginning the rendering of sequential frames in block 902 may include a processing system (e.g., Figure 1 and Figure 2 Processing systems 102, 104, 206 in the system), or one or more processors (e.g., Figure 1 Processors 110, 112, 114, 116, 118, 152, 160 in the process), and frame rendering start point modules (e.g., Figure 2 The frame rendering start point module 212 in the system), the operating system (e.g., Figure 4 The operating system 400) and / or display system (e.g., Figure 4 (Display system 404). The starting point for rendering sequential frames can be identified based on data and / or procedures to be used and / or implemented before rendering sequential frames begins. The end and / or completion of data and / or procedures to be used and / or implemented before rendering sequential frames can occur approximately at the start of rendering sequential frames. Therefore, the end and / or completion of data and / or procedures to be used and / or implemented before rendering sequential frames can be used to indicate the starting point at which rendering sequential frames can begin.

[0091] For example, queue buffers (e.g., Figure 7 The queue buffer 700 in the rendering sequence (e.g., Figure 7 The end point before sequential frame 704 in the sequence can be identified as the starting point for rendering sequential frames (e.g., for rendering...). Figure 7 The starting point of sequence frame 702 in the document). As another example, a sleep timer in an application logic thread (e.g., Figure 8 The sleep timer 800 in the rendering sequence (e.g., Figure 8The end of the sequential frame (804) can be identified as the starting point for rendering sequential frames (e.g., for rendering...). Figure 8 (The starting point of sequential frame 802 in the sequence).

[0092] In box 904, the processing system can set touch event handling points. Touch event handling points (e.g., Figure 7 and Figure 8 Touch event handling points 706, 806 in block 902 can be set before the start point for rendering sequential frames identified in block 902. In some embodiments, touch event handling points can be set as constant and / or variable measurements before the start point for rendering sequential frames. For example, the measurement can be time, such as measurements in milliseconds (ms), which can include fractions of 1 ms up to 100 ms, such as 1 ms. As another example, the measurement can be events, such as multiple processing loops, clock loops, etc. Setting touch event handling points can include generating signals and sending them to the processing system, setting flags to be read by the processing system at registers, etc. In some embodiments, the processing system that sets touch event handling points in block 904 can include a processing system, one or more processors, touch event handling point modules (e.g., Figure 2 The touch event handling point module 214), operating system and / or display system.

[0093] In box 906, the processing system may receive touch events. Touch events may include user interaction with a touchscreen computing device (e.g., ...). Figure 1 The computing device 100 in Figure 2 The touchscreen computing device 202 contains one or more touch input data of its display device. The one or more touch input data may include one or more of the following: touch position, size, direction of movement, speed of movement, time, duration, number of touches, pressure, pattern, etc., on the display device. Touch events may be associated with touch IRQs that trigger the reading of touch events and the dispatching of touch events to a processing system. The processing system may receive the read and dispatched touch events. In some embodiments, the processing system receiving touch events in block 906 may include a processing system, one or more processors, a touch event module (e.g., ... Figure 2 The touch event module 210), operating system and / or display system.

[0094] In determination block 908, the processing system may identify whether a touch event handling point has been reached. As discussed with respect to setting a touch event handling point in block 904, the touch event handling point may take various forms, such as a signal to the processing system, a flag in a register to be read by the processing system, etc. The processing system may receive and / or read touch event handling points in any form to identify whether a touch event handling point has been reached. Failure to receive and / or read touch event handling points in any form may indicate to the processing system that a touch event handling point has not been reached. In some embodiments, identifying whether a touch event handling point has been reached in determination block 908 may include the processing system, one or more processors, a touch event handling module (e.g., ...). Figure 2 The touch event handling module 216), operating system and / or display system.

[0095] In response to identifying that a touch event handling point has been reached (i.e., determining box 908 = "Yes"), the processing system may process one or more touch events in box 910. The one or more touch events may include touch events received in box 906 between the start rendering frame and the start rendering sequence frame. Processing one or more touch events in box 910 may begin at a touch event handling point prior to the start point for the rendering sequence frame.

[0096] Processing one or more touch events in box 910 may include converting data from one or more touch inputs by a user to the display device of a touchscreen computing device into data for rendering sequential frames on the display device. For example, a touch event may be converted into movement of an item in a sequential frame relative to a rendered frame. As another example, a touch event may be converted into manipulation of a sequential frame relative to a rendered frame, such as scaling, panning, rotating, skewing, changing the viewpoint, etc. As yet another example, a touch event may be converted into implementing application-specific actions in a sequential frame, such as game-based actions, including defensive actions, attack actions, and item state change actions. The processing system may process one or more touch events regardless of the display synchronization signal. In some embodiments, one or more touch events in box 910 may be processed by the processing system, one or more processors within the processing system, a touch event processing module, an operating system, and / or a display system.

[0097] In block 912, the processing device may render sequential frames. Rendering sequential frames may include, for example, instructing a display device via a display device controller to generate a display based on data used for rendering sequential frames on the display device. In some embodiments, the processing system that processes one or more touch events in block 912 may include a processing system, one or more processors, a frame rendering module (e.g., Figure 2 The frame rendering module 218), operating system and / or display system.

[0098] In response to identifying that a touch event processing point has not been reached (i.e., confirmation box 908 = "No"), the processing system may cache one or more touch events, i.e., box 914. The processing system may store one or more touch events (including data from one or more touch inputs by the user to the display device of the touchscreen computing device) in memory (e.g., ...). Figure 1 The memory in the 120, 158, Figure 2 (Electronic storage device 204 in the middle). In response to identifying that a touch event processing point has been reached (i.e., confirmation box 908 = "Yes"), one or more touch events can be cached for access to one or more touch events.

[0099] An instance of any of boxes 902 to 918 may be implemented continuously, periodically, and / or sporadically by the processing system. An instance of any of boxes 902 to 918 may be implemented serially and / or in parallel. For example, an instance of box 906 may be implemented in parallel with an instance of any of boxes 902, 904, and 908 to 914. As another example, instances of boxes 902 and 904 may be implemented in parallel with an instance of any of boxes 906 to 914.

[0100] According to various implementation plans (including but not limited to the references above) Figures 1-9 The system described in the implementation scheme can be implemented in a wide variety of computing systems, including mobile computing devices. Examples of these mobile computing devices suitable for use with various implementation schemes are shown in [the following text is missing]. Figure 10 The mobile computing device 1000 may include a processor 1002 coupled to a touchscreen controller 1004 and internal memory 1006. The processor 1002 may be one or more multi-core integrated circuits designated for general or specific processing tasks. The internal memory 1006 may be volatile or non-volatile memory, and may also be secure and / or encrypted memory, or insecure and / or unencrypted memory, or any combination thereof. Examples of available memory types include, but are not limited to, DDR, low-power DDR (LPDDR), graphics DDR (GDDR), Wide-Input DDR, RAM, static RAM (SRAM), dynamic RAM (DRAM), parametric RAM (P-RAM), resistive RAM (R-RAM), magnetoresistive RAM (M-RAM), spin-transfer torque RAM (STT-RAM), and embedded DRAM. The touchscreen controller 1004 and processor 1002 may also be coupled to a touchscreen panel 1012, such as a resistive-sensing touchscreen, a capacitive-sensing touchscreen, an infrared-sensing touchscreen, etc. Additionally, the display device of the mobile computing device 1000 does not need to have touchscreen capability.

[0101] Mobile computing device 1000 may have one or more radio transceivers 1008 (e.g., Peanut, Bluetooth, ZigBee, Wi-Fi, RF radio) and antennas 1010 coupled to each other and / or coupled to processor 1002 for transmitting and receiving communications. The transceivers 1008 and antennas 1010 may be used with the circuitry mentioned above to implement various wireless transmission protocol stacks and interfaces. Mobile computing device 1000 may include a cellular wireless modem chip 1016 that enables communication via a cellular network and is coupled to the processor.

[0102] Mobile computing device 1000 may include a peripheral device connection interface 1018 coupled to processor 1002. The peripheral device connection interface 1018 may be configured individually to accept one type of connection, or it may be configured to accept various types of shared or proprietary physical and communication connections, such as Universal Serial Bus (USB), FireWire, Thunderbolt, or PCIe. The peripheral device connection interface 1018 may also be coupled to a similarly configured peripheral device connection port (not shown).

[0103] The mobile computing device 1000 may also include a speaker 1014 for providing audio output. The mobile computing device 1000 may also include a housing 1020 for accommodating all or some of the components described herein, the housing being constructed of plastic, metal, or a combination of materials. The mobile computing device 1000 may include a power source 1022 coupled to the processor 1002, such as a disposable battery or a rechargeable battery. The rechargeable battery may also be coupled to a peripheral device connection port to receive charging current from a source external to the mobile computing device 1000. The mobile computing device 1000 may also include a physical button 1024 for receiving user input. The mobile computing device 1000 may also include a power button 1026 for turning the mobile computing device 1000 on and off.

[0104] According to various implementation plans (including but not limited to the references above) Figures 1-9 The methods described in the implementation scheme and the apparatus for implementing such methods can be implemented in a wide variety of computing systems, including the embedded transportation computing system 1100, an example of which is shown in [example of embedded transportation computing system 1100]. Figures 11A-11CThe embedded vehicle computing system 1100 may include a vehicle control unit 1140 (such as an ECU), which may include a processor, such as a CPU, an artificial intelligence (AI) processor, etc. The embedded vehicle computing system 1100 may include multiple sensors 1142 to 1170, including a Global Navigation Satellite System (GNSS) receiver 1142, an accelerometer 1144, occupancy sensors 1146, 1148, 1150, and 1152, tire pressure sensors 1154 and 1156, a camera 1158 and 1160, a microphone 1162 and 1164, an impact sensor 1166, and external sensors 1168 and 1170.

[0105] Multiple sensors 1142 to 1170 installed in or on a vehicle can be used for various purposes, such as navigation, collision avoidance, and providing sensor data about objects and people in or on the vehicle. Sensors 1142 to 1170 may include one or more of a wide variety of sensors capable of detecting various types of information useful for navigation and collision avoidance. Each sensor 1142 to 1170 can communicate wirelessly with the control unit 1140 and with each other.

[0106] Sensors may include one or more cameras 1158, 1160 or other optical or photoelectric sensors. Sensors may also include other types of object detection and ranging sensors, such as external sensors 1168, 1170, IR sensors, and ultrasonic sensors. Sensors may also include tire pressure sensors 1154, 1156, humidity sensors, temperature sensors, satellite GNSS sensors 1142, control input sensors 1145, accelerometers 1144, vibration sensors, gyroscopes, gravimeters, impact sensors 1166, force gauges, stress gauges, strain sensors, fluid sensors, chemical sensors, gas analyzers, pH sensors, radiation sensors, Geiger counters, neutron detectors, biomaterial sensors, microphones 1162, 1164, occupancy sensors 1146, 1148, 1150, 1152, proximity sensors, and other sensors.

[0107] The vehicle control unit 1140 may include one or more processors configured with processor-executable instructions to perform navigation and collision avoidance operations using information received from various sensors, particularly cameras 1158 and 1160. In some embodiments, the control unit 1140 may supplement the processing of camera images with distance and relative positioning (e.g., relative azimuth) obtainable from external sensors 1168 and 1170. The control unit 1140 may be further configured to control the steering, braking, and speed of the vehicle using information about other vehicles determined using various embodiments. The vehicle control unit 1140 may include one or more processors configured with processor-executable instructions to receive information from sensors 1142 through 1170 and perform operations using such information, as further described herein. In various embodiments, the vehicle control unit 1140 may include V2X onboard equipment of the vehicle, may be a component of the vehicle's V2X onboard equipment, or may communicate with the vehicle's V2X onboard equipment.

[0108] Figure 11C This is a component block diagram illustrating an embedded vehicle computing system 1100, including components and support systems, suitable for implementing various embodiments. The embedded vehicle computing system 1100 may include a control unit 1140, which may include various circuitry and devices for controlling the operation of the vehicle. The control unit 1140 may include a processor 1140a (such as a CPU, AI processor, etc.), a memory 1140b, an input module 1140c, an output module 1140d, and a radio module 1140e. The control unit 1140 may be coupled to and configured to control the driving control components 1172a, the navigation components 1172b, and one or more sensors 1172c of the embedded vehicle computing system 1100.

[0109] The control unit 1140 can communicate with the V2X onboard equipment 1140f. The processor 1140a may be configured with processor-executable instructions to control the vehicle's maneuvering, navigation, and / or other operations, including operations in various implementations, including acquiring and analyzing real-world vehicle operation data collected from sensor 1172c. The processor 1140a may be coupled to memory 1140b. The V2X onboard equipment 1140f may include one or more processors 1140g configured with processor-executable instructions to perform various operations in various implementations, including transmitting real-world vehicle operation data collected from sensor 1172c between the embedded vehicle computing system 1140 and computing devices on wireless communication devices 1112 and / or communication networks (e.g., core network 1132) via radio module 1140e.

[0110] Radio module 1140e can be configured for wireless communication. Radio module 1140e can exchange signals (e.g., command signals for controlling maneuver, signals from navigation facilities, data signals, etc.) with a network transceiver (e.g., base station 1110) via communication link 1122, and can provide these signals to processors 1140a, 1140g, and / or navigation unit 1172b. In some embodiments, radio module 1140e enables embedded vehicle computing system 1100 to communicate with wireless communication device 1112 via wireless communication link 1124. Wireless communication link 1124 can be a bidirectional or unidirectional communication link and can use one or more communication protocols.

[0111] The input module 1140c can receive sensor data from one or more vehicle sensors 1172c and electronic signals from other components, including the driving control component 1172a and the navigation component 1172b. The output module 1140d can communicate with or activate various components of the embedded vehicle computing system 1100, including the driving control component 1172a, the navigation component 1172b, and the sensors 1172c.

[0112] Control unit 1140 may be coupled to driving control component 1172a to control physical elements of the vehicle related to its maneuverability and navigation, such as engine, motor, throttle, steering elements, flight control elements, braking or deceleration elements, etc. Driving control component 1172a may also include components for controlling other devices of the vehicle, including interior environmental controls (e.g., air conditioning and heating), exterior and / or interior lighting, interior and / or exterior information displays (which may include displays or other devices for displaying information), safety devices (e.g., haptic devices, auditory alarms, etc.), and other similar devices.

[0113] Control unit 1140 may be coupled to navigation component 1172b and may receive data from navigation component 1172b and be configured to use such data to determine the current position and orientation of the vehicle, as well as the appropriate route toward the destination. Navigation component 1172b may include or be coupled to a GNSS receiver system (e.g., one or more Global Positioning System (GPS) receivers) that enables embedded vehicle computing system 1100 to use GNSS signals to determine its current location. Alternatively or additionally, navigation component 1172b may include radio navigation receivers for receiving navigation beacons or other signals from radio nodes such as Wi-Fi access points, cellular network sites, radio stations, telecomputing devices, other vehicles, etc. Through control of driving control element 1172a, processor 1140a may control the vehicle for navigation and maneuvering. Processors 1140a, 1140g and / or navigation component 1172b may be configured to communicate with network elements (such as servers in a communication network (e.g., core network 1132)) via wireless communication links 1122, 1126 to receive commands for controlling maneuvers, receive useful data in navigation, provide real-time positioning reports, etc.

[0114] The control unit 1140 may be coupled to one or more sensors 1172c. Sensors 1172c may include sensors 1142 to 1170 as described, and may be configured to provide various data to processors 1140a, 1140g.

[0115] Although the control unit 1140 is described as comprising separate components, in some embodiments, some or all of the components (e.g., processor 1140a, memory 1140b, input module 1140c, output module 1140d, and radio module 1140e) may be integrated into a single device or module (such as a processing system processing device). Such a processing system processing device may be configured for use in a vehicle and is configured with processor-executable instructions, such as those executed in processor 1140a, to perform navigation and collision avoidance operations.

[0116] Specific implementation examples are described in the following paragraphs. While some of the specific implementation examples described below are based on example systems, devices, or methods, other example implementations may include: example systems or devices implemented as methods for performing operations of example systems or devices, as discussed in the following paragraphs; systems, devices, or methods implemented by computing device processing systems configured to perform operations of example systems, devices, or methods, as discussed in the following paragraphs; one or more processing devices of computing devices configured to perform operations of example systems, devices, or methods; computing devices having a touchscreen and processing systems configured to cause computing devices to perform operations of example systems, devices, or methods; example systems, devices, or methods implemented by computing devices, as discussed in the following paragraphs, which include components for performing the functions of example systems, devices, or methods; and example systems, devices, or methods implemented as non-transitory processor-readable storage media having processor-executable instructions stored thereon, these processor-executable instructions being configured to cause the processor of the computing device to perform operations of the example systems, devices, or methods, as discussed in the following paragraphs.

[0117] Example 1. A method for processing touch events of a touch screen of a computing device by a processing system, the method comprising: receiving a touch event after the start of rendering a frame and before the start of rendering a sequential frame; setting a touch event processing point prior to a start point for starting rendering of the sequential frame in response to receiving the touch event; and processing the touch event at the touch event processing point prior to the start point for starting rendering of the sequential frame regardless of a display synchronization signal.

[0118] Example 2. According to the method of Example 1, the method further includes rendering the touch event in the sequential frame.

[0119] Example 3. The method according to any one of Examples 1 or 2, wherein receiving the touch event after the start of rendering the frame and before the start of rendering the sequential frames occurs after receiving the display synchronization signal during the rendering of the frame.

[0120] Example 4. The method according to any one of Examples 1 or 2, wherein receiving the touch event after the start of rendering the frame and before the start of rendering the sequential frames occurs before receiving the display synchronization signal during the rendering of the frame.

[0121] Example 5. The method according to any one of Examples 1 to 4, the method further includes identifying the starting point for starting to render the sequential frames.

[0122] Example 6. According to the method of Example 5, wherein identifying the starting point for beginning to render the sequential frame includes identifying the end of a queue buffer configured to store display data of the frame.

[0123] Example 7. According to the method of Example 5, wherein identifying the starting point for starting to render the sequential frames includes identifying a sleep period prior to starting to render the sequential frames.

[0124] Computer program code or "program code" intended to be executed on a programmable processor to perform operations of various implementation schemes may be written in a high-level programming language (such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic), a structured query language (such as Transact-SQL), Perl, or in a variety of other programming languages. As used herein, program code or program stored on a computer-readable storage medium may refer to machine language code (such as object code) in a format that is understandable by a processor.

[0125] The foregoing method descriptions and process flowcharts are provided as illustrative examples only and are not intended to require or imply that the operations of the various embodiments must be performed in the order presented. As those skilled in the art will appreciate, the operations of the foregoing embodiments can be performed in any order. Words such as “afterward,” “then,” “next,” etc., are not intended to restrict the order of operations; these words are only used to guide the reader through the description of the method. Furthermore, any reference to singular claim elements (e.g., references using the articles “a,” “an,” or “described”) should not be construed as limiting that element to the singular.

[0126] The various exemplary logic blocks, modules, circuits, and algorithmic operations described in conjunction with various implementation schemes can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and operations have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. While those skilled in the art may implement the described functionality in different ways for each specific application, such implementation decisions should not be construed as departing from the scope of the claims.

[0127] Hardware used to implement the various exemplary logics, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic unit, discrete hardware component, or any combination thereof designed to perform the functions described herein. While the general-purpose processor may be a microprocessor, in alternative embodiments, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0128] In one or more embodiments, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on a non-transitory computer-readable medium or a non-transitory processor-readable medium. The operation of the methods or algorithms disclosed herein can be implemented in a processor-executable software module that may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium can be any storage medium accessible by a computer or processor. By way of example and not limitation, such non-transitory computer-readable or processor-readable media can include RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically reproduce data, while optical discs optically reproduce data using lasers. Combinations of the above can also be included within the scope of non-transitory computer-readable and processor-readable media. In addition, the operation of a method or algorithm may reside as a single piece of code and / or instruction, or any combination or set of code and / or instructions, on a non-transitory processor-readable medium and / or computer-readable medium that may be incorporated into a computer program product.

[0129] The above description of the disclosed embodiments is provided to enable any person skilled in the art to implement or use the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and implementations without departing from the scope of the claims. Therefore, this disclosure is not intended to be limited to the embodiments and implementations described herein, but should be granted the broadest scope consistent with the appended claims and the principles and novel features disclosed herein.

Claims

1. A method for processing, by a processing system, a touch event of a touchscreen of a computing device, the method comprising: receiving a touch event after starting to render a frame and before starting to render a sequential frame; setting a touch event processing point before a starting point for starting to render the sequential frame in response to receiving the touch event; and processing the touch event at the touch event processing point before the starting point for starting to render the sequential frame regardless of a display synchronization signal.

2. The method of claim 1, further comprising rendering the touch event in the sequential frame.

3. The method of claim 1, wherein receiving the touch event after starting to render the frame and before starting to render the sequential frame occurs after receiving the display synchronization signal during rendering of the frame.

4. The method of claim 1, wherein receiving the touch event after starting to render the frame and before starting to render the sequential frame occurs before receiving the display synchronization signal during rendering of the frame.

5. The method of claim 1, further comprising identifying the starting point for starting to render the sequential frame.

6. The method of claim 5, wherein identifying the starting point for starting to render the sequential frame comprises identifying an end of a queue buffer configured to store display data of the frame.

7. The method of claim 5, wherein identifying the starting point for starting to render the sequential frame comprises identifying a dormancy period before starting to render the sequential frame.

8. A computing device, the computing device comprising: a touchscreen; a memory; and a processing system having one or more processors coupled to the touchscreen and the memory, and configured to: receive a touch event after starting to render a frame and before starting to render a sequential frame on the touchscreen; set a touch event processing point before a starting point for starting to render the sequential frame in response to receiving the touch event; and process the touch event at the touch event processing point before the starting point for starting to render the sequential frame regardless of a display synchronization signal.

9. The computing device of claim 8, wherein the one or more processors are further configured to render the touch event in the sequential frame.

10. The computing device of claim 8, wherein the one or more processors are further configured to receive the touch event after starting to render the frame and before starting to render the sequential frame after receiving the display synchronization signal during rendering of the frame.

11. The computing device of claim 8, wherein the one or more processors are further configured to receive the touch event after starting to render the frame and before starting to render the sequential frame before receiving the display synchronization signal during rendering of the frame.

12. The computing device of claim 8, wherein the one or more processors are further configured to identify the starting point for beginning rendering of the sequential frame.

13. The computing device of claim 8, wherein the one or more processors are further configured to identify the starting point for beginning rendering of the sequential frame based on identifying an end of a queue buffer configured to store display data for the frame.

14. The computing device of claim 8, wherein the one or more processors are further configured to identify the starting point for beginning rendering of the sequential frame based on identifying a sleep period prior to beginning rendering of the sequential frame.

15. A computing device, the computing device comprising: a touch screen; means for receiving a touch event after beginning rendering of a frame and before beginning rendering of a sequential frame on the touch screen; means for setting a touch event processing point prior to a starting point for beginning rendering of the sequential frame in response to receiving the touch event; and means for processing the touch event at the touch event processing point prior to the starting point for beginning rendering of the sequential frame regardless of a display synchronization signal.

16. The computing device of claim 15, further comprising means for rendering the touch event in the sequential frame.

17. The computing device of claim 15, wherein the means for receiving the touch event after beginning rendering of the frame and before beginning rendering of the sequential frame is implemented after receiving the display synchronization signal during rendering of the frame.

18. The computing device of claim 15, wherein the means for receiving the touch event after beginning rendering of the frame and before beginning rendering of the sequential frame is implemented before receiving the display synchronization signal during rendering of the frame.

19. The computing device of claim 15, further comprising means for identifying the starting point for beginning rendering of the sequential frame.

20. The computing device of claim 19, wherein the means for identifying the starting point for beginning rendering of the sequential frame comprises means for identifying an end of a queue buffer configured to store display data for the frame.

21. The computing device of claim 19, wherein the means for identifying the starting point for beginning rendering of the sequential frame comprises means for identifying a sleep period prior to beginning rendering of the sequential frame.

22. A non-transitory processor-readable medium having stored thereon processor- executable instructions configured to cause one or more processors of a processing system of a computing device to perform operations comprising: receiving a touch event after beginning rendering of a frame and before beginning rendering of a sequential frame; setting a touch event processing point prior to a starting point for beginning rendering of the sequential frame in response to receiving the touch event; and ​ handling the touch event at the touch event processing point prior to the start point for beginning rendering of the sequential frame regardless of a display synchronization signal.

23. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations further including rendering the touch event in the sequential frame.

24. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations such that receiving the touch event after beginning rendering of the frame and before beginning rendering of the sequential frame occurs after receiving the display synchronization signal during rendering of the frame.

25. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations such that receiving the touch event after beginning rendering of the frame and before beginning rendering of the sequential frame occurs before receiving the display synchronization signal during rendering of the frame.

26. The non-transitory processor-readable medium of claim 22, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations further including identifying the start point for beginning rendering of the sequential frame.

27. The non-transitory processor-readable medium of claim 26, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations such that identifying the start point for beginning rendering of the sequential frame includes identifying an end of a queue buffer configured for storing display data of the frame.

28. The non-transitory processor-readable medium of claim 26, wherein the stored processor-executable instructions are configured to cause one or more processors of the processing system to perform operations such that identifying the start point for beginning rendering of the sequential frame includes identifying a dormant period prior to beginning rendering of the sequential frame.

Images