Screen display method, electronic equipment and corresponding device

Abstract

The invention provides a screen display method, electronic equipment and a corresponding device.The method comprises the steps that firstly, in the process that the electronic equipment displays a target scene, touch operation for the electronic equipment is received; then, based on the frame rate of the target scene displayed by the electronic equipment, the first touch event and the point report rate of the electronic equipment, a second touch event matched with the frame rate is determined, and the first touch event is a touch event generated according to the touch operation; the second touch event comprises a second moment corresponding to the second touch event and a second touch coordinate corresponding to the second touch event; and displaying an interface corresponding to the target scene based on the second touch event. According to the method, the second touch event is re-determined on the basis of the first touch event, the second touch event is matched with the frame rate of the target scene displayed by the electronic equipment, the second touch event used for interface display can be determined according to the frame rate of the target scene, and therefore the chirality can be improved.

Description

Screen display method, electronic equipment and corresponding device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 27, 2023, with application number 202311833651.4 and invention name “A screen display method, electronic device and corresponding device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a screen display method, electronic equipment and corresponding devices. Background Art

[0003] With the popularity of touch-screen devices, touch operations on the screens of touch-screen devices (such as sliding operations and long pressing operations) have become an important way of human-computer interaction, and the hand tracking performance of the device has become an important indicator affecting user experience.

[0004] Chirality is a parameter that describes how quickly a touchscreen device responds to a user's touch operation. It can be defined as the time between the moment a user's finger touches the screen and the moment the corresponding image is displayed. The faster the response, the higher the chirality, and the better the user experience.

[0005] With the development of technology, the hardware configuration used in touch screen devices is getting higher and higher, but the hand tracking performance is often low and cannot meet user needs. In order to improve the user experience, a solution that can improve the hand tracking performance of touch screen devices is urgently needed.

[0006] Summary of the Invention

[0007] In order to improve the hand tracking performance of an electronic device, embodiments of the present application provide a screen display method, an electronic device, and corresponding devices.

[0008] In a first aspect, the present application provides a screen display method, comprising:

[0009] During the process of displaying the target scene on the electronic device, receiving a touch operation on the electronic device;

[0010] Determining, based on a frame rate of the target scene displayed by the electronic device, the first touch event, and a reporting rate of the electronic device, a second touch event adapted to the frame rate, wherein the first touch event is a touch event generated according to the touch operation, and the second touch event includes: a second time corresponding to the second touch event and second touch coordinates corresponding to the second touch event;

[0011] Based on the second touch event, an interface corresponding to the target scene is displayed.

[0012] The solution provided in the embodiment of the present application does not directly display the interface based on the first touch event, but re-determines the second touch event based on the first touch event, and the second touch event is adapted to the frame rate of the electronic device displaying the target scene, that is, the electronic device takes into account the frame rate of the target scene when displaying the interface, and can determine the second touch event for interface display based on the frame rate of the target scene. Therefore, even if the sampling rate and reporting rate of the electronic device are low, the hand tracking performance can be improved.

[0013] In an optional design, determining the second touch event adapted to the frame rate based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device includes:

[0014] Determining a duration threshold corresponding to the target scene;

[0015] Based on the duration threshold and each display moment corresponding to the frame rate, the second moment corresponding to each second touch event is determined, and the duration between the display moment and the second moment is not less than the duration threshold.

[0016] Through this design, the duration threshold can be determined according to the target scene, and then the second moment can be determined according to the duration threshold and the various display moments corresponding to the frame rate. That is, when determining the second moment, the target scene and frame rate are taken into consideration, thereby further improving the adaptability of the second touch event to the target scene.

[0017] In an optional design, determining the second touch event adapted to the frame rate based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device includes:

[0018] determining a first coefficient based on each reporting time corresponding to the second time and the reporting rate;

[0019] The second touch coordinates are determined based on the first coefficient and the first touch coordinates of the first touch operation corresponding to the respective reporting moments.

[0020] In an optional design, determining the first coefficient based on each reporting time corresponding to the second time and the reporting rate includes:

[0021] Determining a first difference and a second difference respectively, wherein the first difference is the difference between the second reporting time and the first reporting time, the second reporting time and the first reporting time are adjacent reporting times, the second reporting time is later than the first reporting time, and the second difference is the difference between the second reporting time and the second time corresponding to the second reporting time;

[0022] A ratio of the second difference to the first difference is determined, where the ratio is the first coefficient.

[0023] In an optional design, if the first coefficient is greater than 0, determining the second touch coordinates based on the first coefficient and the first touch coordinates of the first touch operation corresponding to each reporting point time includes:

[0024] Determining a first product of a second coordinate and the first coefficient, and determining a second product of the first coordinate and the second coefficient, wherein the first coordinate is the first touch coordinate of the first touch operation corresponding to the first point reporting time, the second coordinate is the first touch coordinate of the first touch operation corresponding to the second point reporting time, and the second coefficient is the difference between 1 and the first coefficient;

[0025] determining a sum of the first product and the second product, wherein the second touch coordinate is the sum of the first product and the second product;

[0026] If the first coefficient is less than 0, determining the second touch coordinates based on the first coefficient and the first touch coordinates of the first touch operation corresponding to each reporting point time includes:

[0027] determining a third product of the first coordinate and the first coefficient, and determining a fourth product of the second coordinate and the second coefficient;

[0028] The sum of the third product and the fourth product is determined, and the second touch coordinate is the sum of the third product and the fourth product.

[0029] In an optional design, determining a duration threshold corresponding to the target scene includes:

[0030] Determining a duration threshold corresponding to the target scene based on a correspondence between different frame rate ranges and different duration thresholds, and a frame rate at which the electronic device displays the target scene;

[0031] or,

[0032] Based on the correspondence between different application types and different duration thresholds, and the application type corresponding to the target scenario, the duration threshold corresponding to the target scenario is determined.

[0033] In an optional design, displaying an interface corresponding to the target scene based on the second touch event includes:

[0034] generating a corresponding image frame based on the second touch coordinates corresponding to the second touch event;

[0035] During the process of refreshing the electronic device according to the refresh rate, determining the image frame refreshed within the same refresh cycle;

[0036] A corresponding interface is generated and displayed based on the target image frame refreshed within the same refresh cycle, and the target image frame is generated according to the target touch event. Among the second touch events corresponding to the image frame refreshed within the refresh cycle, the second moment corresponding to the target touch event is the latest.

[0037] In this design, since the target image frame is generated according to the target touch event, the interface of the electronic device is displayed accordingly according to the second touch event determined latest, thereby reducing the time length between the moment the electronic device receives the touch operation and the moment the screen displays the corresponding picture, and correspondingly improving the hand tracking performance of the electronic device.

[0038] In an optional design, the first duration is twice the second duration, wherein the first duration is the duration of the reporting period corresponding to the reporting rate of the electronic device, and the second duration is the duration between the second moments corresponding to two adjacent second touch events.

[0039] In an optional design, determining the second touch event adapted to the frame rate based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device includes:

[0040] Determining a third touch event among the first touch events, where the first touch events include the third touch event and a fourth touch event, and the fourth touch event is a touch event generated according to first n touch screen signals, where n is a positive integer;

[0041] Based on the frame rate of the electronic device displaying the target scene, the third touch event, and the reporting rate of the electronic device, a second touch event adapted to the frame rate is determined.

[0042] Since the fourth touch event is a touch event generated based on the first n touch screen signals, that is, the fourth touch event is a touch event generated based on an earlier touch screen signal, after the fourth touch event is generated, the user's finger has often moved to other positions. Therefore, there is no need to determine the second touch event through the fourth touch event. Through this design, the lag of the screen display interface can be reduced, and the tracking performance of the electronic device can be further improved.

[0043] In a second aspect, the present application provides an electronic device comprising: a processor and a memory; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method described in the first aspect.

[0044] In a third aspect, the present application provides a computer storage medium, wherein the computer storage medium stores a computer program or instructions. When the computer program or instructions are executed, the method described in the first aspect is executed.

[0045] In a fourth aspect, the present application provides a chip system, which includes a processor coupled to a memory and configured to execute a computer program or instruction stored in the memory. When the computer program or instruction is executed, the method described in the first aspect is executed.

[0046] The solution provided in the embodiment of the present application does not directly display the interface based on the first touch event, but re-determines the second touch event based on the first touch event, and the second touch event is adapted to the frame rate of the electronic device displaying the target scene, that is, the electronic device takes into account the frame rate of the target scene when displaying the interface, and can determine the second touch event for interface display based on the frame rate of the target scene. Therefore, even if the sampling rate and reporting rate of the electronic device are low, the hand tracking performance can be improved.

[0047] For example, if the electronic device displays the target scene at a high frame rate, adjacent second touch events are closer, the time interval between adjacent second touch events is shorter than the time interval between adjacent first touch events, and the image frames generated by the second touch events are generated at a higher frequency than the image frames generated by the first touch events. Therefore, when the interface is displayed based on the second touch event, the interface displayed by the electronic device is smoother and the corresponding hand tracking is higher, thereby improving the hand tracking of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] FIG1 is an example diagram of a software architecture of an electronic device;

[0050] FIG2 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0051] FIG3 is an example diagram of a software architecture of an electronic device provided in an embodiment of the present application;

[0052] FIG4 is a schematic diagram of a workflow of a screen display method provided in an embodiment of the present application;

[0053] FIG5 is a schematic diagram of the workflow of another screen display method provided in an embodiment of the present application;

[0054] FIG6( a ) is a timeline diagram illustrating an exemplary screen display method according to an embodiment of the present application;

[0055] FIG6( b ) is another example diagram of a time axis of the screen display method provided in an embodiment of the present application;

[0056] FIG7 is a schematic diagram of the workflow of another screen display method provided in an embodiment of the present application;

[0057] FIG8( a ) is another timeline diagram illustrating a screen display method according to an embodiment of the present application;

[0058] FIG8( b ) is another timeline diagram illustrating a screen display method according to an embodiment of the present application;

[0059] FIG9 is a schematic diagram of signals in a screen display method provided by an embodiment of the present application;

[0060] FIG10 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0062] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0063] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0064] To make the description of the following embodiments clear and concise, a brief introduction to the related technologies is first given:

[0065] Some electronic devices have screens that include a display (e.g., an LCD) and a touch panel (TP) with a built-in touch sensor. These electronic devices are referred to as touchscreen devices. Users can interact with these devices by performing touch operations on them, causing the electronic device to perform the function corresponding to the touch operation and displaying the corresponding interface, thus satisfying the user's control needs for the electronic device.

[0066] In the current technical solution, when a user performs a touch operation on an electronic device, a corresponding touch screen signal can be generated when the user's finger contacts the screen. The touch sensor in the TP can sample the touch screen signal at a certain sampling rate and generate a corresponding touch event based on the sampled touch screen signal. The touch event may include the coordinates of the user's finger in contact with the screen when the touch screen signal is generated.

[0067] Then, the module inside the electronic device can report the touch event according to the corresponding reporting rate. After receiving the touch event, the module used to draw the image frame performs the corresponding layer drawing, rendering, and synthesis operations based on the touch event to obtain the image frame. The reporting rate is the frequency at which the TP reports the touch event after receiving the touch operation.

[0068] The driver of the display screen will refresh according to the corresponding refresh rate so that the display screen displays the corresponding image frame, thereby changing the picture displayed on the display screen.

[0069] For example, refer to the example diagram of a current software architecture shown in Figure 1, in which the software architecture of the electronic device includes: a driver layer (i.e., Driver layer), a hardware abstraction layer (HAL), a framework layer (framework, FWK) and an application layer (application, APP), and the Driver layer is connected to the hardware layer (i.e., Hardware layer).

[0070] The Hardware layer includes TP and liquid crystal display (LCD). After TP generates a touch event, it can transmit the touch event to the TP driver (i.e., TP driver) in the Driver layer. The TP driver then transmits it to the input kernel module (i.e., Input Kernel) in the Driver layer. The Input Kernel then transmits it to the input consumer module (i.e., InputConsumer) in the FWK layer through the HAL layer. The view and target module (i.e., ViewRootimpl) in the FWK layer obtains the touch event from InputConsumer and reports it to the application module (i.e., APP module) in the APP layer according to a preset reporting rate. After receiving the touch event, the application module generates a corresponding vertical synchronization signal (i.e., vsync signal) based on the touch event. Then, the corresponding modules in each layer can perform layer drawing, rendering, and synthesis based on the vsync signal to obtain an image frame.

[0071] After obtaining the image frame, the display kernel module (i.e., the Display Kernel module) in the Driver layer refreshes it according to the corresponding refresh rate, and then the LCD displays the image frame refreshed in each refresh cycle, thereby changing the picture displayed on the LCD.

[0072] According to the above description, after the user touches the TP, the display screen of the electronic device will display the corresponding picture.

[0073] In actual use, users sometimes pay attention to how quickly an electronic device responds to a touch operation. Chirality can be used to characterize this speed. Chirality can be defined as the time from the moment a user's finger touches the screen to the moment the corresponding image is displayed. The faster the response speed, the higher the chirality, the smoother the screen display, and the better the user experience.

[0074] As the hardware configuration performance of electronic devices continues to improve, users also hope that the electronic devices have higher hand tracking performance in certain scenarios (such as using the electronic devices to play games).

[0075] As can be seen from the above description, the chirality of an electronic device is usually related to the sampling rate and the reporting rate of the electronic device. Currently, when electronic devices leave the factory, the sampling rate and the reporting rate are usually set to fixed values.

[0076] However, with the advancement of technology, the hardware configuration of electronic devices has been greatly improved. However, the sampling rate and reporting rate are often set when the hardware configuration is relatively low. In other words, the sampling rate and reporting rate of current electronic devices are often low. This results in the chirality of current electronic devices being lower than the chirality supported by the hardware configuration. Therefore, there is an urgent need for a solution to improve the chirality of electronic devices.

[0077] To address the above issues, embodiments of the present application provide a screen display method, electronic device, and apparatus for improving the hand tracking performance of electronic devices. The method provided in embodiments of the present application can be executed by an electronic device that can receive touch operations and display a corresponding interface.

[0078] The electronic device may be a wireless terminal, a vehicle-mounted wireless terminal, a portable device, a wearable device (such as a watch and an electronic bracelet), a mobile phone (or a "cellular" phone), a portable, pocket-sized, or handheld terminal, and the like, such as a personal communication service (PCS) phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), and the like. The wireless terminal may also be a subscriber unit, an access terminal, a user terminal, a user agent, a user device, or a user equipment (UE), and the present application does not limit the type of electronic device.

[0079] Taking a mobile phone as an example of the above-mentioned electronic device, in this embodiment, the structure of the electronic device can be as shown in Figure 2, where Figure 2 is a structural diagram of an electronic device provided by an embodiment of the present application.

[0080] As shown in FIG2 , the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0081] Furthermore, when the electronic device is a mobile phone, the electronic device may also include: antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone interface 170D, and subscriber identification module (SIM) card interface 195, etc.

[0082] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0083] The processor 110 may include one or more processing units, wherein different processing units may be independent devices or integrated into one or more processors. The processor 110 may also be provided with a memory for storing instructions and data.

[0084] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0085] The charging management module 140 is configured to receive charging input from a charger. The power management module 141 is configured to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160.

[0086] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to electronic devices. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to electronic devices. In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0087] Electronic devices implement display functions through a graphics processing unit (GPU), display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0088] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display screen 194 of the electronic device can display a series of graphical user interfaces (GUIs), which are the main screens of the electronic device.

[0089] The electronic device can realize the shooting function through the ISP, camera 193, video codec, GPU, display 194 and application processor.

[0090] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage. The internal memory 121 can be used to store computer-executable program code, which includes instructions. The processor 110 executes the instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device.

[0091] The electronic device can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0092] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0093] Speaker 170A, also known as a "horn," is used to convert audio electrical signals into sound signals. An electronic device can listen to music or make hands-free calls through speaker 170A. Receiver 170B, also known as a "handset," is used to convert audio electrical signals into sound signals. Microphone 170C, also known as a "microphone" or "speakerphone," is used to convert sound signals into electrical signals. When making a call or sending a voice message, a user can speak by placing their mouth close to microphone 170C to input the sound signal into microphone 170C. Headphone jack 170D is used to connect wired headphones.

[0094] The pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor 180A can be provided on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall effect sensor. The electronic device can use the magnetic sensor 180D to detect the opening and closing of the flip cover. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). The distance sensor 180F is used to measure distance. The proximity light sensor 180G can include a light-emitting diode (LED) and a light detector. The ambient light sensor 180L is used to sense the brightness of ambient light. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect temperature. The touch sensor 180K is also called a "touch control device." The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen." The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The buttons 190 can be mechanical buttons, touch buttons, or virtual buttons. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light that can be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0095] Furthermore, operating systems run on the above components, such as the iOS operating system developed by Apple, the Android open-source operating system developed by Google, and the Windows operating system developed by Microsoft. Application programs can be installed and run on these operating systems.

[0096] In order to clarify the functional operations performed by various software architectures within the electronic device when the electronic device executes the solution disclosed in this application, the embodiment of this application also discloses the software structure of the electronic device.

[0097] The operating system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device.

[0098] Figure 3 is a software structure diagram of an embodiment of the electronic device provided by this application. Referring to Figure 3, the layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided from top to bottom into an application layer (i.e., APP layer), an application framework layer (i.e., FWK layer), a hardware abstraction layer (HAL), and a driver layer (i.e., Driver layer), and the Driver layer is connected to the hardware layer (i.e., Hardware layer).

[0099] The application layer (ie, APP layer) may include a series of application packages.

[0100] The application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc. In addition, it may also include other application modules, such as game, shopping, reading, communication and other application modules (i.e., APP modules) installed by users.

[0101] As shown in Figure 3, in an embodiment of the present application, an electronic device can display the corresponding scene of the APP by running the application module included in the application layer. In addition, the application layer may also include a scene recognition application, which can identify the scene displayed by the electronic device and identify the frame rate at which the electronic device displays the scene.

[0102] The application framework layer (FWK layer) provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0103] Among them, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0104] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0105] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0106] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0107] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0108] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0109] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0110] As shown in Figure 3, in this embodiment of the present application, the FWK layer can be further divided into the FWK-Java layer (i.e., the Java framework layer) and the FWK-Native layer (i.e., the native framework layer) from top to bottom. The FWK-Java layer includes a view root target module (i.e., ViewRootimpl) and a first image frame determination module. The FWK-Native layer includes a second image frame determination module.

[0111] Further, referring to Figure 3, the first image frame determination module may include: an editing module (i.e., Choreograhper module) and a hardware rendering module (i.e., HardWareRender module), and the second image frame determination module may include: a rendering proxy module (i.e., RenderProxy module), an enhanced cache queue module (i.e., BLASTBufferQueue module), a cache queue (i.e., BufferQueue) and a synthesis module (i.e., SurfaceFlinger module).

[0112] In addition, referring to FIG3 , the FWK-Native layer further includes: a touch service module (ie, iTouchService module), a daemon module (ie, Daemon module), and an input processing module.

[0113] Further, referring to Figure 3, the input processing module may include, in order from bottom to top: an input reading module (i.e., InputReader module), an input scheduling module (i.e., InputDispatcher), an input publishing module (i.e., InputPublisher module) and an input consumer module (i.e., InputConsumer module).

[0114] The FWK layer may also include an input transmission module (i.e., InputTransport module) and a batch processing queue (i.e., BatchQueue). The input transmission module can be set in the FWK-Java layer or in the FWK-Native layer. Correspondingly, the batch processing queue can also be set in the FWK-Java layer or in the FWK-Native layer.

[0115] The HAL is an abstract interface for device kernel drivers, providing access to underlying device APIs to higher-level Java API frameworks. The Hardware Abstraction Layer (HAL) can include multiple library modules, such as display modules, audio modules, Bluetooth modules, and Wi-Fi modules. Each module implements an interface for a specific type of hardware component. When a framework API requests access to device hardware, the Android system loads the library module for that hardware component.

[0116] 3 , in the embodiment of the present application, the HAL may include a hardware composer module (HWComposer module).

[0117] The driver layer is the layer between hardware and software. The driver layer may include display drivers, camera drivers, audio drivers, and sensor drivers.

[0118] 3 , the driver layer may further include a touch panel driver (ie, TP Driver), an input kernel (ie, Input Kernel), and a display kernel (ie, Display Kernel).

[0119] In addition, below the driver layer may be a hardware layer (ie, a hardware layer), and the hardware layer may include a touch panel TP and a display screen, and the display screen may include a liquid crystal display (LCD) and the like.

[0120] The above example uses the Android system as an example of a software system of an electronic device that adopts a layered architecture. Of course, the software system of the electronic device can also adopt other architectures, which is not limited in this application.

[0121] The following takes the scenario where an electronic device displays an interface of a target scene through a screen as an example, and describes in detail the screen display method provided in the embodiment of the present application in combination with the software and hardware of the electronic device.

[0122] While the screen of an electronic device is displaying a target scene, a user performs a touch operation on the screen of the electronic device. When the user's finger contacts the screen, a corresponding touch signal is generated. The touch sensor within the TP can detect the touch signal and generate multiple touch events based on the touch signal. The touch event may include the touch coordinates of the user's finger contacting the screen when the touch signal was generated and the time when the touch event was generated. In the solution provided in the embodiment of the present application, the touch event generated by the touch operation is referred to as a first touch event. In this case, the multiple touch events generated by the touch signal are the first touch event.

[0123] After generating the first touch event, the TP can transmit the first touch event to the touch panel driver (i.e., TP Driver) in the Driver layer, and then the TP Driver transmits the first touch event to the input kernel (i.e., Input Kernel) in the Driver layer. The Input Kernel then transmits the first touch event step by step to the batch processing queue (i.e., BatchQueue) in the FWK layer.

[0124] Among them, if the FWK layer includes an input processing module, and the input processing module includes, in order from bottom to top: an input reading module (i.e., InputReader module), an input scheduling module (i.e., InputDispatcher), an input publishing module (i.e., InputPublisher module) and an input consumer module (i.e., InputConsumer module), then during the transmission process of the first touch event, the Input Kernel transmits the first touch event to the InputReader module, and then the InputReader module transmits the first touch event to the InputDispatcher module, the InputDispatcher module transmits the received first touch event to the InputPublisher module, the InputPublisher module then transmits the received first touch event to the InputConsumer module, and then the InputConsumer module transmits the first touch event to the BatchQueue.

[0125] In addition, the scene recognition module in the application layer can identify the target scene and determine the frame rate at which the electronic device displays the target scene. The frame rate is the number of frames that display the target scene per unit time. The scene recognition then transmits the frame rate to the touch service module (i.e., the iTouchService module) in the FWK layer. The iTouchService module then transmits the frame rate to the daemon module in the FWK layer. After receiving the frame rate, the daemon module transmits the frame rate to the input transmission module (i.e., the InputTransport module) in the FWK layer.

[0126] After obtaining the touch event, BatchQueue can read the frame rate in the InputTransport module and re-determine the second touch event that is adapted to the frame rate based on the frame rate of the electronic device displaying the target scene, the first touch event corresponding to the touch operation, and the reporting rate of the electronic device.

[0127] After determining the second touch event, BatchQueue can transmit the second touch event to the view root target module (i.e., ViewRootimpl), and then ViewRootimpl reports the second touch event to the first image frame determination module. The first image frame determination module and the second image frame determination module perform drawing, synthesis, and rendering operations based on the second touch event to obtain the image frame corresponding to the second touch event.

[0128] Among them, in the process of generating image frames, the layer is first drawn according to the second touch event. The drawn layer can be stored in the cache queue (i.e., BufferQueue), and then the synthesis module (i.e., SurfaceFlinger module) reads the corresponding layer from the BufferQueue, and synthesizes and renders the read layer to obtain the corresponding image frame.

[0129] After the corresponding image frame is generated, the display screen displays the interface corresponding to the target scene according to the image frame.

[0130] The screen display method provided by this application can be implemented through the coordination of various layers of the software architecture. Since this application displays the interface corresponding to the target scene through the second touch event, and the second touch event is adapted to the frame rate of the target scene, the screen displayed by the electronic device is relatively smooth and has high chirality, which can meet the user's demand for chirality.

[0131] In order to clarify the solution provided by the present application, the solution provided by the present application is introduced and explained through various embodiments in conjunction with the accompanying drawings.

[0132] In order to improve the hand tracking performance of an electronic device, embodiments of the present application provide a screen display method, device, and electronic device.

[0133] Referring to FIG4 , the screen display method provided in the embodiment of the present application includes the following steps:

[0134] Step S11: receiving a touch operation on the electronic device while the electronic device is displaying a target scene.

[0135] Among them, the touch operation can be a click operation, a sliding operation, a click operation, a long press operation, a double-click operation and a zoom operation on the touch panel, etc., which is not limited in the embodiment of the present application.

[0136] Electronic devices can display a variety of scenes. For example, when running a gaming app, the electronic device can display a game scene; when running a shopping app, the electronic device can display a corresponding item display scene. The target scene can be any scene displayed by the electronic device; when running a reading app, the electronic device can display a corresponding book page display scene.

[0137] Furthermore, users often have different requirements for hand-tracking when displaying different scenes. For example, to provide a better gaming experience, if an electronic device displays an intense competitive game scene, the smoother the game screen and the more responsive the screen, the better the user experience. Therefore, users expect the electronic device to have high hand-tracking in this scene. On the other hand, if an electronic device displays a relatively slow-paced chess game scene, users may have lower requirements for hand-tracking.

[0138] Step S12: Based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device, determine a second touch event that is compatible with the frame rate.

[0139] In the embodiment of the present application, the second touch event is adapted to the frame rate, which means that the higher the frame rate, the shorter the time interval between adjacent second touch events, that is, the closer the adjacent second touch events are. The time interval between adjacent second touch events is the duration between the second moments corresponding to the adjacent second touch events.

[0140] The first touch event is a touch event generated according to a touch operation. After the electronic device receives the touch operation, the touch panel of the electronic device may generate a corresponding touch event according to a touch screen signal corresponding to the touch operation.

[0141] In addition, the first touch event includes: a first moment corresponding to the first touch event and first touch coordinates corresponding to the first touch event. The first moment corresponding to the first touch event can be the moment when the first touch event is generated, or can also be the moment when the electronic device receives the touch operation corresponding to the first touch event. The first touch coordinates are the coordinates of the position where the user's finger contacts the touch panel when the electronic device receives the touch operation corresponding to the first touch event.

[0142] Correspondingly, the second touch event includes: a second moment corresponding to the second touch event and second touch coordinates corresponding to the second touch event.

[0143] The second touch event is not a touch event generated based on an actual touch operation, but a touch event re-determined based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device, that is, the second touch event is a touch event corresponding to the simulated touch operation. The second moment corresponding to the second touch event can be the moment when the second touch event is generated based on the simulated touch operation, or the second moment corresponding to the second touch event can also be the moment when the electronic device receives the simulated touch operation. The second touch coordinates are the coordinates of the position where the user's finger contacts the touch panel when the electronic device receives the touch operation corresponding to the simulated touch event.

[0144] In addition, in this application, the frame rate is the number of frames displayed by the electronic device per second.

[0145] Step S13: Based on the second touch event, display the interface corresponding to the target scene.

[0146] The screen display method provided in the embodiment of the present application determines, during the process of an electronic device displaying a target scene, a second touch event adapted to the frame rate based on a first touch event corresponding to a received touch operation, a frame rate at which the electronic device displays the target scene, and a reporting rate of the electronic device, and displays an interface corresponding to the target scene based on the second touch event.

[0147] That is to say, the solution provided in the embodiment of the present application does not directly display the interface based on the first touch event, but re-determines the second touch event based on the first touch event, and the second touch event is adapted to the frame rate of the electronic device displaying the target scene, that is, the electronic device takes into account the frame rate of the target scene when displaying the interface, and can determine the second touch event for interface display based on the frame rate of the target scene. Therefore, even if the sampling rate and reporting rate of the electronic device are low, the tracking performance can be improved.

[0148] For example, if the electronic device displays the target scene at a high frame rate, adjacent second touch events are closer, the time interval between adjacent second touch events is shorter than the time interval between adjacent first touch events, and the image frames generated by the second touch events are generated at a higher frequency than the image frames generated by the first touch events. Therefore, when the interface is displayed based on the second touch event, the interface displayed by the electronic device is smoother and the corresponding hand tracking is better.

[0149] Furthermore, when determining the second touch event, the solution provided by the embodiment of the present application comprehensively considers the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device, and re-determines the second touch event that is compatible with the frame rate. That is, the solution provided by the embodiment of the present application can dynamically adapt the corresponding second touch event according to the frame rate. In this case, if the sampling rate and reporting rate of the electronic device itself are high, and the frame rate of the target scene displayed by the electronic device is low, then the time interval between adjacent second touch events determined by the solution provided by the embodiment of the present application is longer than the time interval between adjacent first touch events, and the image frames generated by the second touch event are less frequent than the image frames generated by the first touch event, thereby reducing the power consumption of the electronic device while ensuring hand tracking.

[0150] In the solution provided in the embodiment of the present application, an operation is provided for determining a second touch event adapted to the frame rate based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device. Referring to the workflow diagram shown in FIG5 , in one feasible implementation, the operation may include the following steps:

[0151] Step S121: Determine a duration threshold corresponding to the target scene.

[0152] In a feasible design, the duration threshold can be determined in the following manner:

[0153] Based on the correspondence between different frame rate ranges and different duration thresholds, and the frame rate at which the electronic device displays the target scene, the duration threshold corresponding to the target scene is determined.

[0154] In this design, a correspondence between different frame rate ranges and different duration thresholds can be established, and then the duration threshold is determined based on the correspondence and the duration range of the target scene's frame rate.

[0155] In the embodiments of the present application, if the desired chirality of the electronic device is higher, the duration threshold is generally shorter. Therefore, when setting the correspondence between different frame rate ranges and different duration thresholds, the settings can be made based on the user's chirality requirements. For example, if the user has a high chirality requirement, the duration threshold corresponding to a certain frame rate range can be set to 1ms. If the user has a low chirality requirement, the duration threshold corresponding to the frame rate range can be set to 5ms.

[0156] Alternatively, in another feasible design, the duration threshold can be set in the following manner:

[0157] Based on the correspondence between different application types and different duration thresholds, and the application type corresponding to the target scenario, the duration threshold corresponding to the target scenario is determined.

[0158] In this design, the types of each application can be pre-defined. For example, the types that can be set include game applications, shopping applications, and reading applications. In another example, the types of applications can be further subdivided based on this. For example, game applications can be divided into racing game applications, battle game applications, and puzzle game applications.

[0159] Since users have different requirements for chirality when electronic devices display target scenes of different application types, in this design, the correspondence between different application types and different time thresholds can be set, and then the time threshold corresponding to the target scene can be determined based on the application type to which the target scene belongs.

[0160] In the embodiments of the present application, if the desired chirality of the electronic device is higher, the duration threshold is generally shorter. Therefore, when setting the correspondence between different application types and different duration thresholds, the settings can be made based on the user's chirality requirements for different types of applications. For example, if the user has a higher chirality requirement for a certain type of application, the duration threshold corresponding to that application type can be set to 1ms. If the user has a lower chirality requirement for another type of application, the duration threshold corresponding to that other application type can be set to 5ms.

[0161] Through the above design, the duration threshold corresponding to the target scene can be determined based on the frame rate of the electronic device displaying the target scene and the application type corresponding to the target scene.

[0162] Step S122: determining the second moments corresponding to the second touch events based on the duration threshold and the display moments corresponding to the frame rate, wherein the duration between the display moment and the second moment is not less than the duration threshold.

[0163] In order to clearly determine the solution for the second moment corresponding to the second touch event, an embodiment of the present application provides Figures 6(a) and 6(b). In the examples corresponding to Figures 6(a) and 6(b), the horizontal axis represents the time axis, and the duration between t1 and t2 is a reporting cycle, wherein t1 is the start time of the reporting cycle, and t2 is the end time of the reporting cycle. If the reporting rate of the electronic device is 240Hz, the electronic device reports once every 4.16ms, and the duration between t1 and t2 is 4.16ms.

[0164] In both examples, frameTime is set to a display time determined by the frame rate. The frame rate refers to the number of frames displayed per second. Based on the frame rate, the time when the electronic device displays each interface can be determined, and this time is the display time.

[0165] SampleTime is set to a second moment corresponding to a second touch event, and the duration between frameTime and sampleTime is not less than the duration threshold.

[0166] In an optional design, the duration between the display moment frameTime and the second moment sampleTime corresponding to the second touch event can be set to be equal to the duration threshold. In this case, when the frame rate of the electronic device displaying the target scene is fixed, the intervals between the second moments corresponding to the various second touch events remain unchanged. Since the electronic device generates and displays the corresponding interface based on the second touch event, the intervals between the second moments corresponding to the various second touch events remain unchanged, which can maintain a relatively stable frequency for the changes in the interface displayed by the electronic device, and accordingly improve the user experience.

[0167] In addition, compared with the two examples of FIG6(a) and FIG6(b), in the example of FIG6(b), the electronic device is expected to have higher hand tracking performance, so the duration threshold of the example of FIG6(b) can be set to be smaller than the duration threshold of the example of FIG6(a).

[0168] Through the embodiments of the present application, a corresponding duration threshold can be determined based on the target scene, thereby making the duration threshold relevant to the target scene and improving the adaptability of the second touch event to the target scene. In addition, the second moment is determined based on the duration threshold and the respective display moments corresponding to the frame rate. The determination of the second moment also takes into account the frame rate at which the electronic device displays the target scene, thereby further improving the adaptability of the second touch event to the target scene.

[0169] In the solution provided in the embodiment of the present application, an operation is provided for determining a second touch event adapted to the frame rate based on the frame rate of the target scene displayed by the electronic device, the first touch event, and the reporting rate of the electronic device. Referring to the workflow diagram shown in FIG7 , in one feasible implementation, the operation may include the following steps:

[0170] Step S123: Determine a first coefficient based on the second moment and each reporting moment corresponding to the reporting rate.

[0171] In a feasible design, the first coefficient can be determined as follows:

[0172] First, determine the first difference and the second difference respectively, wherein the first difference is the difference between the second reporting moment and the first reporting moment, the second reporting moment and the first reporting moment are adjacent reporting moments, the second reporting moment is later than the first reporting moment, and the second difference is the difference between the second reporting moment and the second moment corresponding to the second reporting moment; wherein the second moment corresponding to the second reporting moment refers to the second moment corresponding to the second touch event determined based on the first moment and the second moment.

[0173] Then, a ratio of the second difference to the first difference is determined, and the ratio is a first coefficient.

[0174] Exemplarily, the first coefficient may be determined in the following manner:

[0175] Exemplarily, the first coefficient may be determined in the following manner:

[0176] alpha=float(t2-sampleTime) / (t2-t1) formula (1).

[0177] Among them, alpha is the first coefficient, t1 is the first reporting time; t2 is the second reporting time; the duration between t2 and t1 is the duration of a reporting cycle. For example, if the reporting rate is 120Hz, then (t2-t1) is 8.33ms, and if the reporting rate is 240Hz, then (t2-t1) is 4.16ms; frameTime is the display time; sampleTime is the second time corresponding to the second touch event; float represents floating point type; (t2-t1) represents the first difference, and (t2-sampleTime) represents the second difference.

[0178] According to formula (1), in the example shown in FIG6(a), the first coefficient is greater than 0, and in the example shown in FIG6(b), the first coefficient is less than 0.

[0179] Step S124 : determining the second touch coordinates based on the first coefficient and the first touch coordinates of the first touch operation corresponding to each reporting point moment.

[0180] In a feasible design, if the first coefficient is greater than 0, determining the second touch coordinates based on the first coefficient and the first touch coordinates of the first touch operation corresponding to each reporting point time includes:

[0181] First, determining a first product of the second coordinate and the first coefficient, and determining a second product of the first coordinate and the second coefficient, wherein the first coordinate is the first touch coordinate of the first touch operation corresponding to the first reporting point time, the second coordinate is the first touch coordinate of the first touch operation corresponding to the second reporting point time, and the second coefficient is the difference between 1 and the first coefficient;

[0182] Then, the sum of the first product and the second product is determined, and the second touch coordinate is the sum of the first product and the second product.

[0183] If the first coefficient is greater than 0, it corresponds to the example shown in FIG6(a). In this case, the second touch coordinates can be determined by the following method:

[0184] Xr=x2*alpha+x1*(1-alpha) formula (2);

[0185] Yr=y2*alpha+y1*(1-alpha) formula (3).

[0186] In the above formula, alpha is the first coefficient; Xr is the horizontal coordinate in the second touch coordinate; Yr is the vertical coordinate in the second touch coordinate; x1 is the horizontal coordinate in the first touch coordinate of the first touch operation corresponding to the first reporting moment; y1 is the vertical coordinate in the first touch coordinate of the first touch operation corresponding to the first reporting moment; x2 is the horizontal coordinate in the first touch coordinate of the first touch operation corresponding to the second reporting moment; y2 is the vertical coordinate in the first touch coordinate of the first touch operation corresponding to the second reporting moment; (1-alpha) is the second coefficient.

[0187] In a feasible design, if the first coefficient is less than 0, determining the second touch coordinates based on the first coefficient and the first touch coordinates of the first touch operation corresponding to each reporting point time includes:

[0188] First, a third product of the first coordinate and the first coefficient is determined, and a fourth product of the second coordinate and the second coefficient is determined;

[0189] Then, the sum of the third product and the fourth product is determined, and the second touch coordinate is the sum of the third product and the fourth product.

[0190] If the first coefficient is less than 0, it corresponds to the example shown in FIG6( b ). In this case, the second touch coordinates can be determined by the following method:

[0191] Xr=x1*alpha+x2*(1-alpha) formula (4);

[0192] Yr=y1*alpha+y2*(1-alpha) formula (5).

[0193] Formulas (2) to (5) above provide solutions for determining the second touch coordinates in two cases, respectively, when the first coefficient is greater than 0 and when the first coefficient is less than 0. Furthermore, Formulas (2) and (3) take into account the case where the first coefficient is greater than 0, and their algorithms can be called interpolation methods, while Formulas (4) and (5) take into account the case where the first coefficient is less than 0, and their algorithms can be called extrapolation methods. That is, the algorithm used in the solution provided in the embodiment of the present application can be dynamically adapted and adjusted, and can combine interpolation and extrapolation methods, so that the obtained second touch event can be adapted to the frame rate of the target scene displayed by the electronic device, thereby improving the hand tracking performance of the electronic device.

[0194] As shown in FIG6(a) and FIG6(b), the first touch event at the first reporting time is represented as (t1, x1, y1), the first touch event at the second reporting time is represented as (t2, x2, y2), and Tr=sampleTime is set, then the second touch event can be represented as (Tr, Xr, Yr).

[0195] To clarify the advantages of this application, Figures 8(a) and 8(b) are disclosed. In the examples corresponding to Figures 8(a) and 8(b), the horizontal axis represents the time axis. If the electronic device's reporting rate is 240Hz, then the electronic device reports a time every 4.16ms, and the time interval between t1 and t2 is 4.16ms. In addition, if the electronic device displays the target scene at a frame rate of 120Hz, then the time interval between adjacent display moments corresponding to this frame rate is 8.33ms.

[0196] In the example corresponding to FIG8(a), the requirement for chirality is lower than that in the example corresponding to FIG8(b), and the duration threshold in this example is 5 ms. Then, Tr is less than t2, the target coefficient is greater than 0, and the second moment Tr of the second touch event is between t1 and t2. In the example corresponding to FIG8(b), the requirement for chirality is higher than that in the example corresponding to FIG8(a), and the duration threshold is 1 ms. Then, Tr is greater than t2, the target coefficient is less than 0, and the second moment Tr of the second touch event is later than t2.

[0197] In addition, in a feasible implementation of the present application, the following conditions can be met by setting an appropriate time threshold:

[0198] The first duration is twice the second duration, wherein the first duration is the duration of the reporting period corresponding to the reporting rate of the electronic device, and the second duration is the duration between second moments corresponding to two adjacent second touch events.

[0199] In this case, the second touch event is located between two first touch events corresponding to adjacent reporting moments, which enables the electronic device to have good hand tracking performance.

[0200] Step S13 of the present application discloses an operation of displaying an interface corresponding to the target scene based on the second touch event. In a feasible implementation, step S13 can be implemented as follows:

[0201] The first step is to generate a corresponding image frame based on the second touch coordinates corresponding to the second touch event.

[0202] For example, if the second touch event is dragging an icon to change the display position of the icon on the screen, then in the generated image frame, the display position of the icon is the changed position.

[0203] The second step is to determine the image frames refreshed in the same refresh cycle during the process of the electronic device refreshing according to the refresh rate.

[0204] The electronic device will refresh according to the corresponding refresh rate and refresh to one or more image frames in each refresh cycle. For example, if the refresh rate of the electronic device is 120Hz, the electronic device will refresh every 8.33ms, that is, each refresh cycle is 8.33ms, and one or more image frames can be refreshed in one refresh cycle.

[0205] The third step is to generate and display a corresponding interface based on the target image frame refreshed within the same refresh cycle. The target image frame is generated according to the target touch event. Among the second touch events corresponding to the image frames refreshed within the refresh cycle, the second moment corresponding to the target touch event is the latest.

[0206] If one image frame is acquired within a refresh cycle, the electronic device's display screen may display the interface corresponding to that image frame. Furthermore, if multiple image frames are acquired within a refresh cycle, and each image frame is generated by its corresponding second touch event, i.e., the multiple image frames acquired within the refresh cycle are generated by multiple second touch events, the electronic device may generate and display the corresponding interface based on the image frame corresponding to the most recently determined second touch event.

[0207] In this case, the second moments corresponding to the multiple second touch events corresponding to the multiple image frames obtained within the refresh cycle can be determined, wherein the second touch event with the latest second moment is the target touch event, and the image frame generated according to the target touch event is the target image frame. The electronic device can generate and display a corresponding interface according to the target image frame.

[0208] Since the target image frame is generated according to the target touch event, the interface of the electronic device is displayed accordingly according to the latest determined second touch event, thereby reducing the time length between the moment the electronic device receives the touch operation and the moment the screen displays the corresponding picture, and correspondingly improving the hand tracking performance of the electronic device.

[0209] In the solution provided in the embodiment of the present application, step S12 provides a method for determining a second touch event that matches the frame rate based on the frame rate at which the electronic device displays the target scene, the first touch event, and the reporting rate of the electronic device. In one feasible implementation, this operation can be implemented by the following steps:

[0210] First, determining a third touch event in the first touch event, where the first touch event includes the third touch event and a fourth touch event, and the fourth touch event is a touch event generated according to first n touch screen signals, where n is a positive integer;

[0211] Then, based on the frame rate of the target scene displayed by the electronic device, the third touch event and the reporting rate of the electronic device, a second touch event adapted to the frame rate is determined.

[0212] That is, the fourth touch event in the first touch event can be deleted, only the third touch event is retained, and the second touch event is determined based on the retained third touch event. Exemplarily, n can be 1 or 2. Of course, n can also be other positive integers, and this embodiment of the application is not limited to this.

[0213] Since the fourth touch event is a touch event generated based on the first n touch screen signals, that is, the fourth touch event is a touch event generated based on an earlier touch screen signal, after the fourth touch event is generated, the user's finger has often moved to other positions. Therefore, there is no need to determine the second touch event through the fourth touch event. Through this solution, the lag of the screen display interface can be reduced, and the tracking performance of the electronic device can be further improved.

[0214] To clarify the advantages of the present application, the embodiments of the present application further provide FIG9 , which is an example diagram of a signal generated by an electronic device. Referring to FIG9 , the three signal diagrams correspond to three examples. In these three examples, the electronic device displays the same target scene. In the first example, the duration threshold is 5ms, in the second example, the duration threshold is 1ms, and in the third example, the duration threshold is 3ms. Furthermore, each example includes three types of signals, and the horizontal axes of these three types of signals are all time.

[0215] Among them, the first signal is the VSYNC-app signal generated by the application layer. Under the software architecture corresponding to Figure 3, the first signal can be a signal generated by the application module in the application layer (i.e., the APP module). Since in these three examples, the electronic devices display the same target scene and the APP module performs the same operation, the first signals in the three examples are slightly different.

[0216] The second signal corresponds to a signal for reporting a second touch event. In the software architecture corresponding to FIG. 3 , the second signal may be a signal for ViewRootimpl to report the second touch event to the Choreograhper module (the signal may be called a deliverInputEvent signal).

[0217] The third signal corresponds to the signal corresponding to the layer waiting for synthesis rendering. Under the software architecture corresponding to Figure 3, the third signal can be the signal corresponding to the layer waiting for synthesis rendering stored in the buffer queue (ie, BufferQueue).

[0218] It can be seen from Figure 9 that when the duration threshold is different, the duration between the display moment frameTime and sampleTime is different, the number of second touch events is also different, and the layers waiting for synthetic rendering stored in the corresponding cache queue (i.e., BufferQueue) are also different. Accordingly, the degree of compactness between the second signal and the third signal is also different.

[0219] If a scenario requires high chirality, a shorter duration threshold can be set for that scenario. In the first example, the duration threshold is 5ms, and in the second example, the duration threshold is 1ms. The chirality of the second example is often higher than that of the first. Figure 9 shows that the second example has a greater number of second touch events, a greater number of layers awaiting composite rendering stored in the buffer queue (i.e., BufferQueue), and a higher degree of compactness of the second and third signals than the first example, indicating that the second example has higher chirality than the first.

[0220] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0221] Corresponding to the embodiment of the aforementioned screen display method, the present application further provides an electronic device, as shown in FIG10 , wherein the electronic device includes:

[0222] Processor 1101 and memory,

[0223] The memory is used to store program instructions;

[0224] The processor 1101 is used to call and execute the program instructions stored in the memory. When the program instructions stored in the memory are executed by the processor 1101, the electronic device executes all or part of the steps in the embodiments corresponding to Figures 4, 5 and 7.

[0225] Furthermore, the electronic device may further include: a transceiver 1102 and a bus 1103 , and the memory includes a random access memory 1104 and a read-only memory 1105 .

[0226] The processor is coupled to the transceiver, random access memory, and read-only memory via a bus. When the electronic device needs to be operated, it is booted via a basic input / output system (BIOS) embedded in the read-only memory or a bootloader in an embedded system, guiding the electronic device into normal operation. After the electronic device enters normal operation, the application program and operating system are run in the random access memory, causing the electronic device to execute all or part of the steps in the embodiments corresponding to Figures 4, 5, and 7.

[0227] The electronic device of an embodiment of the present invention may correspond to the electronic device in the embodiments corresponding to Figures 4, 5 and 7 above, and the processor and storage in the electronic device may implement the functions and / or various steps and methods implemented by the electronic device in the embodiments corresponding to Figures 4, 5 and 7. For the sake of brevity, they will not be repeated here.

[0228] In a specific implementation, an embodiment of the present application further provides a computer storage medium, wherein a computer program or instruction is stored in the computer storage medium. When the computer program or instruction is executed, the computer can implement all or part of the steps in the embodiments corresponding to Figures 4, 5 and 7. The computer-readable storage medium is set in any device, and the arbitrary device can be a random-access memory (RAM). The memory can also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory can also include a combination of the above-mentioned types of memory, etc.

[0229] The present application also provides a chip system, which includes a processor coupled to a memory and configured to execute a computer program or instruction stored in the memory. When the computer program or instruction is executed, the chip system implements all or part of the steps in the embodiments corresponding to Figures 4, 5, and 7. The chip system can be composed of a chip or include a chip and other discrete components.

[0230] The various illustrative logic units and circuits described in the embodiments of the present application can be implemented or operated by a general-purpose processor, a digital information processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital information processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital information processor core, or any other similar configuration.

[0231] The steps of the methods or algorithms described in the embodiments of the present application can be directly embedded in hardware, software units executed by a processor, or a combination of the two. The software units can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), registers, hard disks, removable disks, portable compact disc read-only memory (CD-ROM), or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can also be integrated into the processor. The processor and storage medium can be provided in an ASIC, which can be provided in a user equipment (UE). Alternatively, the processor and storage medium can also be provided in different components in the UE.

[0232] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0233] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid state drive (SSD)).

[0234] The same or similar parts between the various embodiments of this specification can be referred to in conjunction with each other, and each embodiment focuses on the differences between the other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

[0235] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention or certain portions of the embodiments.

[0236] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the embodiment of the road constraint determination device disclosed in this application, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0237] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. A screen display method, characterized in that, Including: During the process of the electronic device displaying a target scene, receiving a touch operation on the electronic device; Based on the frame rate at which the electronic device displays the target scene, a first touch event, and the reporting rate of the electronic device, determining a second touch event adapted to the frame rate, where the first touch event is a touch event generated according to the touch operation, and the second touch event includes: a second moment corresponding to the second touch event and a second touch coordinate corresponding to the second touch event; Based on the second touch event, displaying an interface corresponding to the target scene.

2. The method according to claim 1, wherein The determining, based on the frame rate at which the electronic device displays the target scene, a first touch event, and the reporting rate of the electronic device, a second touch event adapted to the frame rate includes: Determining a duration threshold corresponding to the target scene; Based on the duration threshold and each display moment corresponding to the frame rate, determining the second moment corresponding to each of the second touch events, where the duration between the display moment and the second moment is not less than the duration threshold.

3. The method according to claim 1, characterized in that The determining, based on the frame rate at which the electronic device displays the target scene, a first touch event, and the reporting rate of the electronic device, a second touch event adapted to the frame rate includes: Based on the second moment and each reporting moment corresponding to the reporting rate, determining a first coefficient; Based on the first coefficient and the first touch coordinate of the first touch operation corresponding to each reporting moment, determining the second touch coordinate.

4. The method according to claim 3, wherein The determining, based on the second moment and each reporting moment corresponding to the reporting rate, a first coefficient includes: Respectively determining a first difference and a second difference, where the first difference is the difference between a second reporting moment and a first reporting moment, the second reporting moment and the first reporting moment are adjacent reporting moments, the second reporting moment is later than the first reporting moment, and the second difference is the difference between the second reporting moment and the second moment corresponding to the second reporting moment; Determining the ratio of the second difference to the first difference, and the ratio is the first coefficient.

5. The method according to claim 4, wherein If the first coefficient is greater than 0, the determining, based on the first coefficient and the first touch coordinate of the first touch operation corresponding to each reporting moment, the second touch coordinate includes: Determining a first product of the second coordinate and the first coefficient, and determining a second product of the first coordinate and a second coefficient, where the first coordinate is the first touch coordinate of the first touch operation corresponding to the first reporting moment, the second coordinate is the first touch coordinate of the first touch operation corresponding to the second reporting moment, and the second coefficient is the difference between 1 and the first coefficient; Determining the sum of the first product and the second product, and the second touch coordinate is the sum of the first product and the second product; If the first coefficient is less than 0, the determining, based on the first coefficient and the first touch coordinate of the first touch operation corresponding to each reporting moment, the second touch coordinate includes: Determine a third product of the first coordinate and the first coefficient, and determine a fourth product of the second coordinate and the second coefficient; Determine the sum of the third product and the fourth product, and the second touch coordinate is the sum of the third product and the fourth product.

6. The method according to claim 2, characterized in that, The determining the duration threshold corresponding to the target scene includes: Based on the correspondence between different frame rate ranges and different duration thresholds, and the frame rate at which the electronic device displays the target scene, determine the duration threshold corresponding to the target scene; Or, Based on the correspondence between different application program types and different duration thresholds, and the application program type corresponding to the target scene, determine the duration threshold corresponding to the target scene.

7. The method according to claim 1, characterized in that, The displaying the interface corresponding to the target scene based on the second touch event includes: Generate a corresponding image frame based on the second touch coordinate corresponding to the second touch event; During the process of the electronic device refreshing at the refresh rate, determine the image frames refreshed within the same refresh period; Generate and display a corresponding interface based on the target image frames refreshed within the same refresh period, the target image frames are generated according to the target touch event, and among the second touch events corresponding to the image frames refreshed within the refresh period, the second moment corresponding to the target touch event is the latest.

8. The method according to claim 2, wherein, The first duration is twice the second duration, where the first duration is the duration of the reporting period corresponding to the reporting rate of the electronic device, and the second duration is the duration between the second moments respectively corresponding to two adjacent second touch events.

9. The method according to claim 1, wherein The determining the second touch event adapted to the frame rate based on the frame rate at which the electronic device displays the target scene, the first touch event, and the reporting rate of the electronic device includes: Determine a third touch event in the first touch event, the first touch event includes the third touch event and a fourth touch event, and the fourth touch event is a touch event generated according to the first n touch screen signals, where n is a positive integer; Based on the frame rate at which the electronic device displays the target scene, the third touch event, and the reporting rate of the electronic device, determine the second touch event adapted to the frame rate.

10. An electronic device, characterized in that, Comprising: A processor and a memory; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1-9.

11. A computer storage medium, characterized in that, The computer storage medium stores a computer program or instructions, and when the computer program or instructions are executed, the method according to any one of claims 1-9 is executed.

12. A chip system, characterized in that, The chip system includes a processor, the processor is coupled to the memory and is configured to execute the computer program or instructions stored in the memory, and when the computer program or instructions are executed, the method according to any one of claims 1-9 is executed.

Images