Image display method and related apparatus

Abstract

Provided in the present application are an image display method and a related apparatus. In the method, if the period of time for drawing and rendering a layer exceeds a single signal period, after the rendering is completed, within a time period between the completion of the rendering and the generation of a first vertical synchronization signal 2, composition of the rendered layer is triggered to obtain an image. By means of implementing the method, an electronic device can composite an image in a timely manner even when an abnormality occurs during the drawing and rendering of a layer, thereby avoiding delayed image display. In this way, a user can observe an image without delay, resulting in a better viewing experience.

Description

Image display method and related device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 1, 2024, with application number 202410239472.6 and application name “Image Display Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an image display method and related devices. Background Art

[0003] Electronic devices can refresh displayed images. They can obtain images by drawing, rendering, and compositing layers, and then display them on a display screen. However, with the development of electronic devices, to meet user needs, electronic devices may spend more time drawing and rendering layers to produce more detailed images. This may cause the electronic device to display a certain frame of image with delay. Summary of the Invention

[0004] This application provides an image display method and related apparatus that can ensure that images can be synthesized in a timely and rapid manner and displayed in advance when an electronic device occasionally takes too long to render a single frame (e.g., exceeding a single signal cycle). This not only avoids consuming excessive resources for storing synthesized images, but also allows users to observe high-frame-rate images with a better viewing experience, while also ensuring the stability of the displayed image.

[0005] In a first aspect, the present application provides a method for image display, which is applied to an electronic device, and the method includes: at a first moment, the electronic device draws a first layer, and then renders the first layer, and the duration of drawing and rendering the first layer does not exceed a single signal cycle, and the signal cycle is the cycle of two adjacent vertical synchronization signals generated by the electronic device; at a second moment, the electronic device synthesizes the first layer to obtain a first image, and draws a second layer, and then renders the second layer, and the duration of drawing and rendering the second layer exceeds a single signal cycle, the second moment is located after the first moment, and the second moment is separated from the first moment by a single signal cycle; at a third moment, the electronic device displays the first image, the third moment is located after the second moment, and the third moment is separated from the second moment by a single signal cycle; after the electronic device completes rendering the second layer, before the moment of generating the first vertical synchronization signal after completing rendering the second layer, the second layer is synthesized to obtain a second image; the electronic device displays the second image.

[0006] By implementing the above method, the electronic device can synthesize images in a timely manner even when abnormalities occur in the drawing and rendering layers, thereby avoiding the situation of delayed image display. In this way, the user can observe the image without delay and have a good viewing experience.

[0007] In conjunction with the first aspect, in some implementations, the electronic device displaying the second image specifically includes: displaying the second image at a fourth moment, the fourth moment being after the third moment, and the fourth moment being separated from the third moment by a single signal cycle. In this way, the electronic device can display the image normally even if an anomaly occurs in the drawing and rendering layer, providing a good user experience.

[0008] In conjunction with the first aspect, in some implementations, the electronic device displaying the second image specifically includes: displaying the second image at a sixth moment, the sixth moment being after the moment when the second image is obtained. In this way, the electronic device can display the second image after synthesizing the second image in advance.

[0009] In conjunction with the first aspect, in some implementations, before the electronic device displays the second image, the method further includes: generating, by the electronic device, a first vertical synchronization signal at the sixth moment, the first vertical synchronization signal being used to trigger the electronic device to display the second image at the sixth moment. In this way, the electronic device can trigger the early display of the second image using the newly added vertical synchronization signal 3 when the second image is synthesized in advance.

[0010] In combination with the first aspect, in some implementations, the moment when the electronic device obtains the second image is after the third moment and before the fourth moment, the fourth moment is after the third moment, the fourth moment is separated from the third moment by a single signal cycle, and the sixth moment is before the fourth moment. In this case, the electronic device can synthesize the second image based on the second layer after completing the rendering for the second layer and before the moment when the first vertical synchronization signal is generated after the second layer is rendered. The moment when the synthesis of the second image is completed is before the moment when the first vertical synchronization signal is generated after the second layer is rendered. Then, the electronic device can display the second image after completing the synthesis of the second image and before the moment when the first vertical synchronization signal is generated after the second layer is rendered.

[0011] In conjunction with the first aspect, in some implementations, the method further includes: at the third moment, the electronic device drawing a third layer and then rendering the third layer, where the drawing and rendering duration of the third layer does not exceed a single signal cycle; at the fourth moment, the electronic device synthesizing the third layer to obtain a third image; and at a fifth moment, the electronic device displaying the third image, where the fifth moment is after the fourth moment and is separated from the fourth moment by a single signal cycle. In this way, displaying the second image in advance does not affect the normal drawing, rendering, and synthesized display of the third image.

[0012] In combination with the first aspect, in some implementations, the moment when the electronic device obtains the second image is after a fourth moment, the fourth moment is after the third moment, the fourth moment is separated from the third moment by a single signal cycle, and the sixth moment is after the fourth moment. In this case, the electronic device can synthesize the second image based on the second layer after completing the rendering for the second layer and before the moment when the first vertical synchronization signal is generated after the second layer is rendered. The moment when the synthesis of the second image is completed is after the moment when the first vertical synchronization signal is generated after the second layer is rendered. Then, the electronic device can display the second image after completing the synthesis of the second image and before the moment when the first vertical synchronization signal is generated after the synthesis of the second image is completed.

[0013] In conjunction with the first aspect, in some implementations, the method further includes: at the third moment, the electronic device draws a third layer, then renders the third layer, where the duration of drawing and rendering the third layer does not exceed a single signal cycle; after the electronic device obtains the second image, the electronic device synthesizes the third layer to obtain a third image; at a fifth moment, the electronic device displays the third image, where the fifth moment is after the fourth moment and is the moment when the electronic device generates the first vertical synchronization signal after obtaining the third image. In this way, although displaying the second image in advance may affect the synthesis of the third image, it will not affect the normal display of the third image.

[0014] In combination with the first aspect, in some implementations, at a first moment, the electronic device draws a first layer, and before rendering the first layer, the method further includes: the electronic device starts a first application, and the first application includes one or more of the following: a shooting game, a fighting game, a racing game, a music rhythm game, a sports competitive game, and a multiplayer online technical competitive game.

[0015] In combination with the first aspect, in some implementations, the electronic device further includes: a drawing thread, a rendering thread, a graphics synthesizer, and a display screen, wherein, at the first moment, the electronic device draws the first layer through the drawing thread, and then renders the first layer through the rendering thread; at the second moment, the electronic device synthesizes the first layer through the graphics synthesizer to obtain the first image, and draws the second layer through the drawing thread, and then renders the second layer through the rendering thread; at the third moment, the electronic device displays the first image through the display screen; after the electronic device completes rendering the second layer, before the time when the first vertical synchronization signal is generated after completing the rendering of the second layer, the second layer is synthesized through the graphics synthesizer to obtain the second image; the electronic device displays the second image through the display screen.

[0016] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, wherein the computer program code comprises computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the method of the first aspect or any embodiment of the first aspect.

[0017] In a third aspect, an embodiment of the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG1 is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0019] FIG2 is a schematic diagram of a vertical synchronization signal provided by an embodiment of the present application;

[0020] FIG3A is a schematic diagram showing the principles of drawing layers, rendering layers, compositing layers, and displaying images in an electronic device according to an embodiment of the present application;

[0021] FIG3B is a schematic diagram of another electronic device drawing layers, rendering layers, synthesizing layers, and displaying images provided by an embodiment of the present application;

[0022] FIG3C is a schematic diagram of another electronic device drawing layers, rendering layers, synthesizing layers, and displaying images provided by an embodiment of the present application;

[0023] FIG4 is a schematic diagram of an electronic device using an image display method provided by an embodiment of the present application;

[0024] FIG5 is a flow chart of an image display method provided in an embodiment of the present application;

[0025] FIG6 is a schematic diagram of another electronic device using an image display method according to an embodiment of the present application;

[0026] FIG7 is a flow chart of another image display method provided in an embodiment of the present application;

[0027] FIG8 is a schematic diagram of another electronic device using an image display method provided by an embodiment of the present application;

[0028] FIG9 is a flow chart of another image display method provided in an embodiment of the present application;

[0029] FIG10 is an internal module interaction diagram of an electronic device using an image display method provided by an embodiment of the present application;

[0030] FIG11 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0031] FIG12 is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following is a clear and detailed description of the technical solutions in the embodiments of the present application, with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of "or." For example, A / B can represent A or B. "and / or" in the text is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0033] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0034] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0035] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0036] First, the software architecture of the electronic device provided in the embodiment of the present application is introduced.

[0037] Exemplarily, the electronic devices may be mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) and virtual reality (VR) devices, etc., which include display screens (such as touch screens). The embodiments of the present application do not impose any special restrictions on the specific form of the electronic devices.

[0038] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device.

[0039] FIG1 is a block diagram of the software structure of an electronic device according to an embodiment of the present invention.

[0040] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0041] The application layer can include a series of application packages.

[0042] As shown in FIG1 , the application package may include applications such as camera, calendar, game, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0043] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0044] As shown in Figure 1, the application framework layer includes, but is not limited to, the UI thread and the rendering thread. The UI thread is the thread running in the electronic device's central processing unit (CPU). It is used to draw layers. The rendering thread is the thread running in the electronic device's graphics processing unit (GPU). It is used to render completed layers.

[0045] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0046] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0047] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0048] The system library may include multiple functional modules, including but not limited to: InputDispatcher thread, InputReader thread, image synthesizer (Surface Flinger), etc.

[0049] Among them, the InputDispatcher thread is used to wait for new events and distribute new events to applications.

[0050] The InputReader thread is used to read and preprocess the original input event and send the input event to the queue managed by the InputDispatcher thread.

[0051] Surface Flinger is used to synthesize the rendered layers to obtain an image.

[0052] Most classes in the Framework layer act as "intermediaries" for applications to access library files in the system library. Upper-layer applications are written in Java and require the most direct Java interface support, while the system library supports the operation of another language (such as C++). Therefore, the Framework layer acts as an intermediary between the application layer and the system library. The various modules in the Framework layer do not actually implement specific functions, or only implement a part of them, but focus on the core library to complete them. The difference is that the Framework layer is mostly written in Java, while the system library is mostly written in C++.

[0053] The kernel layer is the layer between hardware and software. It includes at least the touch driver and display driver. The kernel layer receives instructions from the software layer and controls the hardware to perform tasks.

[0054] It is understood that the software architecture shown in Figure 1 does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer modules than shown, or combine or split some modules, etc.

[0055] The following is a schematic diagram of the software processing flow for displaying images provided in an embodiment of the present application, described in conjunction with FIG1 .

[0056] When the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The touch driver in the kernel layer processes the touch operation into a touch event (including touch coordinates, touch operation timestamp and other information).

[0057] The touch driver sends the raw input event to the InputReader thread of the system library. The InputReader thread pre-processes the touch event and then sends it to the InputDispatcher thread.

[0058] The application framework layer can identify the specific application corresponding to the touch event.

[0059] The InputReader thread can draw the layer corresponding to the touch event based on the information provided by the application.

[0060] The rendering thread can render the completed layer.

[0061] Surface Flinger can synthesize the layers after rendering to obtain an image.

[0062] Finally, the display driver calls the synthesized image and presents the synthesized image through the display screen.

[0063] As can be seen from the above process, before an image is displayed on the display screen, it needs to go through steps such as drawing, rendering, and synthesis. Among them, drawing layers may include: arranging elements such as text, shapes, and pictures on one layer or multiple layers according to a certain layout. Rendering layers may include: performing ray tracing, shading, shadow generation, and other processing on one or more layers that have been drawn, so that the rendered layers are as close to the visual representation of the real world as possible. Synthesizing layers to obtain an image may include: superimposing one or more rendered layers together to generate a complete image. Among them, the image obtained by synthesizing layers can be simply referred to as a synthesized image.

[0064] The following describes a schematic diagram of a vertical synchronization signal used by an electronic device to trigger drawing, rendering, and synthesis with reference to FIG2 .

[0065] As shown in Figure 2, vertical synchronization signal 1 can be used to trigger the drawing and rendering of layers. Specifically, vertical synchronization signal 1 can be used to trigger the UI thread to draw layers and the rendering thread to render the layers drawn by the UI thread. Generally, after the UI thread finishes drawing a layer, the rendering thread can immediately render the completed layer if there is no current rendering task.

[0066] The vertical synchronization signal 2 can be used to trigger the synthesis of the rendered layer to obtain an image. Specifically, the vertical synchronization signal 2 can be used to trigger Surface Flinger to synthesize the layer after the rendering thread renders to obtain an image.

[0067] The vertical synchronization signal 3 can be used to trigger hardware to refresh the image displayed on the display screen. The vertical synchronization signal 3 can be a hardware signal triggered by the display driver of the electronic device.

[0068] In an embodiment of the present application, the period for generating the vertical synchronization signal 3 can be determined by the frame rate of the display screen of the electronic device. For example, the period can be the inverse of the frame rate of the display screen. For example, the frame rate of the display screen of the electronic device can be any value such as 60 Hz, 70 Hz, 75 Hz, 80 Hz, 90 Hz, or 120 Hz. Taking the frame rate of the display screen as 60 Hz as an example, the period for generating the vertical synchronization signal 3 is 1 / 60 = 0.01667 seconds (s) = 16.667 milliseconds (ms).

[0069] In one possible implementation, the electronic device may support multiple different frame rates. The frame rate of the electronic device can switch between the above different frame rates. The frame rate in the embodiment of the present application is the frame rate currently used by the electronic device. That is, the period for generating the vertical synchronization signal 3 is the inverse of the frame rate currently used by the electronic device.

[0070] As shown in Figure 2, the electronic device generates a vertical synchronization signal 3, triggered by a hardware driver, every other signal cycle. Vertical synchronization signal 1 and vertical synchronization signal 2 are generated based on vertical synchronization signal 3. That is, vertical synchronization signal 3 can be the signal source of vertical synchronization signal 1 and vertical synchronization signal 2. Alternatively, vertical synchronization signal 1 and vertical synchronization signal 2 are synchronized with vertical synchronization signal 3. In other words, the signal cycles of vertical synchronization signal 1 and vertical synchronization signal 2 are the same as the signal cycle of vertical synchronization signal 3, and their phases are consistent.

[0071] For example, as shown in Figure 2, the signal period of vertical synchronization signal 1, the signal period of vertical synchronization signal 2, and the signal period of vertical synchronization signal 3 are the same. Moreover, as shown in Figure 2, the phases of vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are consistent. It is understandable that, in actual implementation, there may be a certain phase error between vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 due to various factors (such as processing performance). It should be noted that when understanding the method of the embodiment of the present application, the above-mentioned phase error is ignored.

[0072] In summary, vertical synchronization signal 1, vertical synchronization signal 2, and vertical synchronization signal 3 are all periodic discrete signals. For example, as shown in Figure 2, vertical synchronization signal 1 occurs every signal period, vertical synchronization signal 2 occurs every signal period, and vertical synchronization signal 3 occurs every signal period.

[0073] The signal periods of the vertical synchronization signal 1, vertical synchronization signal 2 and vertical synchronization signal 3 can all be referred to as synchronization period TZ. That is, the synchronization period in the embodiment of the present application is the inverse of the frame rate of the electronic device.

[0074] Generally, a single vertical synchronization signal can trigger a single operation, such as drawing, rendering, or compositing. The conditions required for an electronic device to trigger drawing, rendering, or compositing include receiving the results of the preceding steps and generating a vertical synchronization signal. Exemplarily, the conditions required for an electronic device to trigger rendering include generating a drawn layer and vertical synchronization signal 2. The conditions required for an electronic device to trigger compositing include generating a rendered layer and vertical synchronization signal 3.

[0075] It should be noted that the name of the vertical synchronization signal may be different in different systems or architectures. For example, in some systems or architectures, the name of the vertical synchronization signal (i.e., vertical synchronization signal 1) used to trigger the drawing of one or more layers may not be VSYNC_APP. However, no matter what the name of the vertical synchronization signal is, as long as it is a synchronization signal with similar functions and conforms to the technical ideas of the method provided in the embodiment of this application, it should be covered within the scope of protection of this application.

[0076] Furthermore, the definition of the vertical synchronization signal may also be different in different systems or architectures.

[0077] For example, in other systems or architectures, the definition of the vertical synchronization signal 1 can be: vertical synchronization signal 1 can be used to trigger the rendering of one or more layers; the definition of vertical synchronization signal 2 can be: vertical synchronization signal 2 can be used to trigger the generation of an image based on one or more layers; the definition of vertical synchronization signal 3 can be: vertical synchronization signal 3 can be used to trigger the display of an image. In the embodiment of the present application, the definition of the vertical synchronization signal is not limited. However, no matter how the vertical synchronization signal is defined, as long as it is a synchronization signal with similar functions and conforms to the technical ideas of the method provided in the embodiment of the present application, it should be covered within the scope of protection of the present application.

[0078] In combination with the above description based on Figures 1 and 2, in response to the touch operation, the application corresponding to the touch operation in the electronic device can call the UI thread to draw the layer when the vertical synchronization signal 1 is generated, and call the rendering thread to render the drawn layer. Then, when the hardware synthesizer (HWC) of the electronic device generates the vertical signal 2, it calls Surface Flinger to synthesize the rendered layer to obtain an image. Afterwards, the display driver of the electronic device displays the synthesized image on the display screen when the vertical synchronization signal 3 is generated. In an embodiment of the present application, the above-mentioned drawing and rendering, synthesis, and display are all periodic.

[0079] In another possible implementation, the electronic device may not need to respond to touch operations. Instead, the application may automatically call the UI thread to draw the layer and the rendering thread to render the drawn layer when a vertical synchronization signal 1 is generated. For example, when launching a gaming application, the electronic device may still update the displayed image even without any user operation.

[0080] The following describes an example of an electronic device drawing layers, rendering layers, compositing layers, and displaying images.

[0081] As shown in Figure 3A, illustratively, at time t1, the electronic device generates a vertical synchronization signal 1. The electronic device can respond to the vertical synchronization signal 1 at time t1 to draw a. After drawing is completed, the electronic device can immediately render a. The above-mentioned a can refer to one or more layers in the same frame. At time t2, the electronic device generates a vertical synchronization signal 2. The electronic device can respond to the vertical synchronization signal 2 at time t2 to synthesize the rendered a to obtain image A. At time t3, the electronic device generates a vertical synchronization signal 3. The electronic device can respond to the vertical synchronization signal 3 at time t3 to display image A.

[0082] Similarly, at time t2, the electronic device receives vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t2, the electronic device can draw image b. After drawing, the electronic device can immediately render image b. b can refer to one or more layers in the same frame. At time t3, the electronic device receives vertical synchronization signal 2. In response to vertical synchronization signal 2 at time t3, the electronic device can synthesize the rendered image b to obtain image B. At time t4, the electronic device receives vertical synchronization signal 3. In response to vertical synchronization signal 3 at time t4, the electronic device can display image B.

[0083] Similarly, at time t3, the electronic device generates vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t3, the electronic device can draw image c. After drawing, the electronic device can immediately render image c. This image c can refer to one or more layers in the same frame. At time t4, the electronic device generates vertical synchronization signal 2. In response to vertical synchronization signal 2 at time t4, the electronic device can synthesize the rendered image c to obtain image C. At time t5, the electronic device generates vertical synchronization signal 3. In response to vertical synchronization signal 3 at time t5, the electronic device can display image C.

[0084] From the above, we can see that if the sum of the time it takes for the electronic device to draw a and render a does not exceed a single signal cycle, the sum of the time it takes to draw b and render b does not exceed a single signal cycle, and the sum of the time it takes to draw c and render c does not exceed a single signal cycle, the electronic device can display image A from time t3 to time t4, image B from time t4 to time t5, and image C from time t5 to time t6. In other words, the electronic device can display each frame of the image without delay, and there will be no situation where an image that should be displayed in the current signal cycle is actually displayed in the next signal cycle.

[0085] However, if the sum of the time it takes for an electronic device to draw and render a layer is too long, for example, exceeding a single signal cycle, a delay in displaying the image may occur.

[0086] The following are examples of several other electronic devices drawing layers, rendering layers, compositing layers, and displaying images.

[0087] As shown in Figure 3B, at time t1, the electronic device generates vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t1, the electronic device can draw image a. After drawing, the electronic device can immediately render image a. At time t2, the electronic device generates vertical synchronization signal 2. In response to vertical synchronization signal 2 at time t2, the electronic device can synthesize the rendered image a to obtain image A. At time t3, the electronic device generates vertical synchronization signal 3. In response to vertical synchronization signal 3 at time t3, the electronic device can display image A.

[0088] Similarly, at time t2, the electronic device receives vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t2, the electronic device draws b. After drawing, the electronic device can immediately render b. However, because the electronic device takes a long time to draw b, the drawing and rendering of b cannot be completed within a single signal cycle. In other words, the electronic device has not yet completed rendering b at time t3.

[0089] Therefore, the electronic device generates vertical synchronization signal 2 at time t3. However, since rendering b is not complete at this time, the electronic device cannot respond to vertical synchronization signal 2 at time t3 to synthesize image B. Furthermore, the electronic device requires vertical synchronization signal 2 to trigger the synthesis layer to obtain the image. Furthermore, the electronic device can only respond to vertical synchronization signal 2 at time t4 to synthesize the rendered image b to obtain image B. At time t5, the electronic device generates vertical synchronization signal 3. The electronic device can respond to vertical synchronization signal 3 at time t5 to display image B.

[0090] As shown in FIG3C , at time t1, the electronic device generates a vertical synchronization signal 1. The electronic device can respond to the vertical synchronization signal 1 at time t1 and draw a. After the drawing is completed, the electronic device can immediately render a.

[0091] At time t2, the electronic device generates a vertical synchronization signal 2. In response to the vertical synchronization signal 2 at time t2, the electronic device can synthesize the rendered image a to obtain image A.

[0092] At time t3, the electronic device generates a vertical synchronization signal 3. The electronic device can display image A in response to the vertical synchronization signal 3 at time t3.

[0093] Similarly, at time t2, the electronic device receives vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t2, the electronic device draws b. After drawing, the electronic device can immediately render b. However, because the time required to draw b exceeds a single signal cycle, the electronic device cannot complete both drawing and rendering b within a single signal cycle. In other words, the electronic device has not yet completed drawing b at time t3.

[0094] Therefore, the electronic device generates vertical synchronization signal 2 at time t3. However, since rendering of image b is not complete at this time, the electronic device cannot respond to vertical synchronization signal 2 at time t3 to synthesize image B. Furthermore, the electronic device requires vertical synchronization signal 2 to trigger the synthesis layer to obtain the image. Furthermore, the electronic device can only respond to vertical synchronization signal 2 at time t4 to synthesize the rendered image b to obtain image B. At time t5, the electronic device generates vertical synchronization signal 3. The electronic device can respond to vertical synchronization signal 3 at time t5 to display image B.

[0095] At time t3, the electronic device receives vertical synchronization signal 1 and should respond to it to begin drawing c. However, because drawing b has not yet completed, the UI thread in the electronic device is occupied, preventing the start of drawing c. Therefore, the electronic device must wait until drawing b is complete and the UI thread is idle before it can begin drawing c.

[0096] As can be seen from the above, if the sum of the time it takes for the electronic device to draw and render image a does not exceed a single signal cycle, and the sum of the time it takes to draw and render image b exceeds a single signal cycle, the electronic device can display image A from time t3 to time t5, but can only display image B after time t5. In other words, the electronic device may display an image that should be displayed within the current signal cycle but actually displays it in the next signal cycle.

[0097] To reduce the occurrence of the above situation, an embodiment of the present application provides a method and electronic device for image display. In this method, if the time required to draw and render a layer exceeds a single signal cycle, the rendered layer is triggered to be synthesized to obtain an image after the rendering is completed and before the first vertical synchronization signal 2 is generated after the rendering is completed. In other words, if the electronic device misses the opportunity to synthesize the image, it will not wait for the next vertical synchronization signal 2 to arrive before triggering the synthesis of the rendered layer.

[0098] By implementing the above method, the electronic device can synthesize images in a timely manner even when abnormalities occur in the drawing and rendering layers, thereby avoiding the situation of delayed image display. In this way, the user can observe the image without delay and have a good viewing experience.

[0099] Moreover, in some game scenarios, the image display method provided by the embodiment of the present application can better meet the hand-following requirements.

[0100] In order to better understand the image display method process provided in the embodiments of the present application, an example of an electronic device using the image display method is first introduced below.

[0101] FIG4 is a schematic diagram of an image display method used by an electronic device provided in an embodiment of the present application.

[0102] As shown in Figure 4, at time t1, the electronic device generates vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t1, the electronic device may draw image a. After drawing is complete, the electronic device may immediately render image a. Furthermore, the electronic device has completed drawing and rendering image a before time t2. At time t2, the electronic device generates vertical synchronization signal 2. In response to vertical synchronization signal 2 at time t2, the electronic device may synthesize the rendered image a to obtain image A. At time t3, the electronic device generates vertical synchronization signal 3. In response to vertical synchronization signal 3 at time t3, the electronic device may display image A.

[0103] At time t2, the electronic device may receive vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t1, the electronic device may draw b. After drawing, the electronic device may immediately render b. However, the electronic device may not have completed rendering b at time t3.

[0104] In this embodiment of the present application, under special circumstances, the electronic device may not trigger image synthesis via vertical synchronization signal 2. Therefore, at time t31, when the electronic device completes rendering b, synthesis of the rendered b is triggered to obtain image B. At time t4, the electronic device generates vertical synchronization signal 3. The electronic device can respond to vertical synchronization signal 3 at time t4 and display image B.

[0105] At time t3, the electronic device generates vertical synchronization signal 1. Although the electronic device has not completed rendering b at time t3, since drawing and rendering are different threads, this does not affect the electronic device's drawing of c. Therefore, the electronic device can respond to the vertical synchronization signal 1 at time t3 and draw c. After drawing is completed, the electronic device can immediately render c. And the electronic device has completed drawing and rendering c before time t4. At time t4, the electronic device generates vertical synchronization signal 2. The electronic device can respond to the vertical synchronization signal 2 at time t4 to synthesize the rendered c to obtain image C. At time t5, the electronic device generates vertical synchronization signal 3. The electronic device can respond to the vertical synchronization signal 3 at time t5 to display image C.

[0106] In a possible implementation, the process of obtaining image C by the electronic device can refer to the process of obtaining image A, or the process of obtaining image B. That is, in the embodiment of the present application, the images synthesized multiple times by the electronic device can be triggered not by vertical synchronization signal 2, but by rendering the completed layer.

[0107] The following describes the process of the image display method provided by the embodiment of the present application in conjunction with Figure 5. In the flow chart shown in Figure 5, the time for the electronic device to delay synthesizing layers can be reduced, so that the electronic device can synthesize layers as quickly as possible, thereby ensuring that the image is displayed without delay.

[0108] For example, the execution subject of the method provided in the embodiments of the present application may be a device for processing images. The device may be any of the electronic devices described above (for example, the device may be the electronic device shown in FIG1 ). Alternatively, the device may be a central processing unit (CPU) of an electronic device, or a control module in an electronic device for executing the method provided in the embodiments of the present application.

[0109] In the embodiment of the present application, the method provided by the embodiment of the present application is introduced by taking the above-mentioned electronic device executing the image display method as an example.

[0110] The method flow includes:

[0111] S501: At a first moment, the electronic device draws and renders a first layer, wherein the duration of drawing and rendering the first layer is less than a single signal cycle.

[0112] The first moment may be the moment when the electronic device generates a vertical synchronization signal 1. The vertical synchronization signal 1 may be used to trigger the electronic device to draw a first layer. The first layer may refer to a layer that starts drawing in response to the vertical synchronization signal 1 at the first moment.

[0113] S502: At a second moment, the electronic device draws and renders a second layer, and synthesizes the first image based on the first layer, wherein the duration of drawing and rendering the second layer is greater than a single signal cycle, and the second moment is separated from the first moment by a single signal cycle.

[0114] The second moment may be the moment when the electronic device generates a vertical synchronization signal 1. The vertical synchronization signal 1 may be used to trigger the electronic device to draw the second layer. The second layer may be the layer that starts drawing in response to the vertical synchronization signal 1 at the second moment.

[0115] In the embodiment of the present application, the second moment is separated from the first moment by a single signal cycle, and the second moment may be located after the first moment.

[0116] In some implementations, the UI thread periodically begins drawing UI layers based on vertical synchronization signal 1. That is, after S401, the UI thread of the electronic device may quickly complete drawing the first layer. However, because vertical synchronization signal 1 is not generated between the completion of drawing the first layer and the second moment, the UI thread of the electronic device will not begin drawing the second layer after completing drawing the first layer. Instead, it must respond to vertical synchronization signal 1 after receiving it at the second moment and then begin drawing the second layer.

[0117] Because the duration of drawing and rendering the first layer is less than a single signal cycle, the electronic device has already completed drawing and rendering the first layer at the second moment. Therefore, the electronic device can respond to the vertical synchronization signal 2 generated at the second moment to synthesize the rendered first layer to obtain the first image.

[0118] S503: At a third moment, the electronic device displays the first image, wherein the third moment is separated from the second moment by a single signal cycle.

[0119] Since the electronic device has completed synthesis to obtain the first image at the third moment, the electronic device can refresh the image displayed by the electronic device in response to the vertical synchronization signal 3 generated at the third moment. That is, the electronic device can display the first image.

[0120] Because the time required to draw and render the second layer is longer than a single signal cycle and shorter than two signal cycles, the electronic device has not yet completed drawing and rendering the second layer at the third moment. Therefore, the electronic device cannot composite the second layer at the third moment.

[0121] S504: After completing the rendering of the second layer and before the time when the first vertical synchronization signal is generated after completing the rendering of the second layer, the electronic device synthesizes a second image based on the second layer.

[0122] In some implementations, before executing S504, the electronic device can determine whether it has missed the opportunity to synthesize the third image in response to the first vertical synchronization signal 2 (vertical synchronization signal 2 at the third moment) after the start of drawing the first layer. If so, S504 can be executed; if not, the electronic device can respond to the vertical synchronization signal 2 at the third moment and synthesize the third image based on the second layer.

[0123] Specifically, whether the opportunity to synthesize the third image in response to the vertical synchronization signal 2 at the third moment is missed can be determined by the following method:

[0124] If the duration of drawing and rendering the second layer exceeds a single signal cycle, it can be considered that the opportunity to synthesize the third image in response to the vertical synchronization signal 2 at the third moment is missed.

[0125] If the moment when the second layer is completed is after the third moment, it can be considered that the opportunity to synthesize the third image in response to the vertical synchronization signal 2 at the third moment is missed.

[0126] If the time for rendering the second layer exceeds a single signal cycle and the moment of completing the second layer is after the third moment, it can be considered that the opportunity to synthesize the third image in response to the vertical synchronization signal 2 at the third moment is missed.

[0127] In the embodiment of the present application, the method is not limited to the above method, and more methods can be included to determine whether the opportunity to synthesize the third image in response to the vertical synchronization signal 2 at the third moment is missed, and this is not limited.

[0128] In some implementations, the electronic device may generate a first message after completing rendering for the second layer. The first message may instruct Surface Flinger to immediately synthesize the second image based on the second layer upon receiving the second layer, regardless of whether vertical synchronization signal 2 is present. The first message may be considered vertical synchronization signal 2. In other words, the first message has the same or similar effect as vertical synchronization signal 2.

[0129] In one possible implementation, the electronic device may synthesize the second image immediately after completing rendering the second layer, or may synthesize the second image after a certain period of time. In the embodiments of the present application, this is not limited to this. The electronic device only needs to ensure that the second layer is synthesized to obtain the second image after completing rendering the second layer and before the first vertical synchronization signal is generated after the second layer is completed.

[0130] S505: At a fourth moment, the electronic device displays a second image, wherein the fourth moment is separated from the third moment by a single signal cycle.

[0131] Since the electronic device has completed synthesis to obtain the second image at the fourth moment, the electronic device can refresh the image displayed by the electronic device in response to the vertical synchronization signal 3 generated at the fourth moment, that is, the electronic device can display the second image.

[0132] The first moment may refer to moment t1, the second moment may refer to moment t2, the third moment may refer to moment t3, and the fourth moment may refer to moment t4. The first layer may refer to the layer that the electronic device begins drawing at moment t1, and the second layer may refer to the layer that the electronic device begins drawing at moment t2. The first image may refer to image A that the electronic device begins synthesizing at moment t2, and the second image may refer to image B that the electronic device begins synthesizing after moment t3.

[0133] In the embodiment of the present application, the application of the method flow shown in FIG5 in the example diagram shown in FIG4 is only for illustrative purposes and can also be applied in more scenarios, without limitation thereto.

[0134] As can be seen from the descriptions of Figures 4 and 5 above, the image display method provided by the embodiments of the present application can ensure that the image can be synthesized in a timely manner and displayed without delay even if the electronic device occasionally experiences a long drawing and rendering time for a single frame (e.g., exceeding a single signal cycle). Furthermore, such occasional situations will not affect other normal drawing, rendering, synthesis, and display processes.

[0135] The following describes another example of an electronic device using an image display method.

[0136] FIG6 is a schematic diagram of an image display method used by an electronic device provided in an embodiment of the present application.

[0137] As shown in Figure 6, the electronic device can respond to vertical synchronization signal 1 at time t1 to draw image a. After drawing, the electronic device can immediately render image a. The electronic device can respond to vertical synchronization signal 2 at time t2 to synthesize the rendered image a to obtain image A. The electronic device can respond to vertical synchronization signal 3 at time t3 to display image A.

[0138] At time t2, the electronic device may receive vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t1, the electronic device may draw b. After drawing, the electronic device may immediately render b. However, the electronic device may not have completed rendering b at time t3.

[0139] In an embodiment of the present application, under special circumstances, the electronic device can trigger the synthesis of the image through the above-mentioned first message. Therefore, at time t31, that is, when the electronic device completes rendering b, it triggers the synthesis of the rendered b to obtain image B.

[0140] In an embodiment of the present application, when the electronic device synthesizes an image in advance, a vertical synchronization signal 3 can be temporarily added at time t3.5. Specifically, the electronic device can determine time t3.5 based on the time when image B is obtained and the current signal cycle. At time t3.5, the electronic device has completed the synthesis of image B and can respond to the vertical synchronization signal 3 at time t3.5 to display image B. The electronic device can also display image B between time t3.5 and time t5.

[0141] The electronic device may respond to vertical synchronization signal 1 at time t3 to draw image c. After drawing is completed, the electronic device may immediately render image c. Furthermore, the electronic device may complete drawing and rendering image c before time t4. The electronic device may respond to vertical synchronization signal 2 at time t4 to synthesize the rendered image c to obtain image C. The electronic device may respond to vertical synchronization signal 3 at time t5 to display image C.

[0142] In one possible implementation, the process for the electronic device to obtain image C can refer to the process for obtaining image A, or the process for obtaining image B. That is, in this embodiment of the present application, the electronic device may not be triggered by vertical synchronization signal 2 to continuously synthesize multiple images, but may be triggered by the first message. The electronic device may also be triggered by a temporarily added vertical synchronization signal 3 to continuously display multiple images.

[0143] The following describes another image display process provided by an embodiment of the present application in conjunction with FIG7 . In the flowchart shown in FIG7 , the delay time for the electronic device to synthesize layers can be reduced, allowing the electronic device to synthesize layers as quickly as possible, thereby ensuring that the image is displayed without delay. Furthermore, the image can be displayed in advance. The method process includes:

[0144] S701: At a first moment, the electronic device draws and renders a first layer, wherein the duration of drawing and rendering the first layer is less than a single signal cycle.

[0145] S702: At a second moment, the electronic device draws and renders a second layer, and synthesizes the first image based on the first layer, wherein the duration of drawing and rendering the second layer is greater than a single signal cycle, and the second moment is separated from the first moment by a single signal cycle.

[0146] S703: At a third moment, the electronic device displays the first image, wherein the third moment is separated from the second moment by a single signal cycle.

[0147] S704: After completing rendering for the second layer and before generating the first vertical synchronization signal after completing rendering of the second layer (i.e., a fourth time), the electronic device synthesizes a second image based on the second layer. The time when synthesizing the second image is completed is between the third time and the fourth time, and the fourth time is separated from the third time by a single signal cycle.

[0148] For S701-S704, please refer to S501-S504 and related descriptions in FIG5 .

[0149] S705: After the electronic device completes synthesizing the second image and before the fourth moment, the electronic device displays the second image.

[0150] In some implementations, the electronic device may temporarily add a new vertical synchronization signal 3 after the electronic device completes synthesizing the second image but before the fourth moment, that is, at the sixth moment. The electronic device may display the second image in response to the vertical synchronization signal 3 at the sixth moment. The temporary addition of a new vertical synchronization signal 3 indicates that the duration between the vertical synchronization signal 3 generated by the electronic device after the addition and the vertical synchronization signal 3 before the addition is still separated by a single signal cycle. For example, a new vertical synchronization signal 3 is added at time t3.5, and the duration between the vertical synchronization signal generated by the electronic device at time t4 and the vertical synchronization signal generated at time t3 is separated by a single signal cycle.

[0151] In one possible implementation, the electronic device can achieve the above situation by adjusting the signal period. Specifically, the electronic device can achieve the above situation by adjusting the signal period three times: the first adjustment adjusts the single signal period to the duration between the sixth moment and the third moment, the second adjustment can adjust it to the duration between the fourth moment and the sixth moment, and the third adjustment can adjust it back to the single signal period.

[0152] Here's how to make the first adjustment:

[0153] In some implementations, the electronic device can display the second image in advance by adjusting the period of the vertical synchronization signal 3, that is, displaying the second image at the fourth moment is advanced to displaying the second image at the sixth moment. The electronic device can determine the range of adjusting the signal period of the vertical synchronization signal 3 based on one or more of the following: the current signal period, the moment when the synthesis of the second image is completed, and the moment when the last vertical synchronization signal 2 generated before the synthesis of the second image is completed (that is, the third moment). Specifically, the electronic device can set the signal period after the first adjustment to be at least the length of time between the moment when the synthesis of the second image is completed and the third moment. Moreover, the vertical synchronization signal 3 obtained after the first adjustment of the signal period, that is, the newly added vertical synchronization signal 3, does not affect the signal periods of the vertical synchronization signal 2 and the vertical synchronization signal 1.

[0154] For example, assuming that the frame rate of the current electronic device display screen is 60Hz, the electronic device determines that the electronic device completes synthesizing the second image at the intermediate moment between the third moment and the fourth moment based on the current signal cycle, the moment when the synthesis of the second image is completed, and the third moment. Then, the electronic device can temporarily adjust the frame rate of the display screen to at most 120Hz. That is, the electronic device can temporarily adjust the frame rate of the display screen to any value between 60Hz and 120Hz. For example, after adjusting the frame rate of the electronic device to 120Hz, the electronic device can generate a newly added vertical synchronization signal 3 at the intermediate moment between the third moment and the fourth moment, and display the second image in response to the newly added vertical synchronization signal 3. In this way, the electronic device can display the second image at the fastest about 9.8ms in advance.

[0155] As another example, assuming that the frame rate of the current electronic device display screen is 60Hz, the electronic device determines that the electronic device completes synthesizing the second image at a moment that is two-thirds of the signal cycle from the third moment to the fourth moment based on the current signal cycle, the moment when the synthesis of the second image is completed, and the third moment. Then, the electronic device can temporarily adjust the frame rate of the display screen to at most 90Hz. That is, the electronic device can temporarily adjust the frame rate of the display screen to any value between 60Hz and 90Hz. For example, after adjusting the frame rate of the electronic device to 90Hz, the electronic device can generate a newly added vertical synchronization signal 3 in response to a moment that is two-thirds of the signal cycle from the third moment after the third moment, and display the second image in response to the newly added vertical synchronization signal 3.

[0156] In the embodiment of the present application, the purpose of the electronic device adjusting the signal period for the first time is to minimize the electronic device storing the synthesized image so that the electronic device can display the synthesized image as soon as possible after completing the synthesized image.

[0157] It is worth noting that, regardless of the value to which the electronic device adjusts the signal period for the first time, the second vertical synchronization signal 3 generated by the electronic device after the third moment is separated from the third moment by a single signal period. In the embodiment of the present application, a single signal period may refer to a single signal period that has not been adjusted.

[0158] The first moment may refer to moment t1, the second moment may refer to moment t2, the third moment may refer to moment t3, and the fourth moment may refer to moment t4. The first layer may refer to the layer that the electronic device begins drawing at moment t1, and the second layer may refer to the layer that the electronic device begins drawing at moment t2. The first image may refer to image A that the electronic device begins synthesizing at moment t2, and the second image may refer to image B that the electronic device begins synthesizing after moment t3.

[0159] In the embodiment of the present application, the above S704 is explained by taking the time duration of drawing and rendering the second layer exceeding a single signal cycle but not exceeding two signal cycles as an example.

[0160] In another possible implementation, the duration of drawing and rendering the second layer shown in FIG7 exceeds a single signal cycle and multiple signal cycles. In this case, the electronic device can synthesize the second image based on the second layer after completing the rendering for the second layer and before the moment when the first vertical synchronization signal is generated after the second layer is rendered. The moment when the synthesis of the second image is completed is before the moment when the first vertical synchronization signal is generated after the second layer is rendered. Then, the electronic device can display the second image after completing the synthesis of the second image and before the moment when the first vertical synchronization signal is generated after the second layer is rendered.

[0161] In the embodiment of the present application, the application of the method flow shown in FIG. 7 in the example diagram shown in FIG. 6 is merely an exemplary description and can also be applied in more scenarios, without limitation thereto.

[0162] As can be seen from the descriptions of Figures 6 and 7 above, the image display method provided by the embodiments of the present application can ensure that the image can be synthesized in a timely manner and displayed in advance when the electronic device occasionally takes too long to draw and render a single frame layer (for example, exceeding a single signal cycle). This not only avoids consuming too many resources to store the synthesized image, but also allows the user to observe high-frame-rate images for a better viewing experience.

[0163] The following describes another example of an electronic device using an image display method.

[0164] FIG8 is a schematic diagram of an image display method used by an electronic device provided in an embodiment of the present application.

[0165] As shown in Figure 8, the electronic device can respond to vertical synchronization signal 1 at time t1 to draw image a. After drawing, the electronic device can immediately render image a. The electronic device can respond to vertical synchronization signal 2 at time t2 to synthesize the rendered image a to obtain image A. The electronic device can respond to vertical synchronization signal 3 at time t3 to display image A.

[0166] At time t2, the electronic device may receive vertical synchronization signal 1. In response to vertical synchronization signal 1 at time t1, the electronic device may draw b. After drawing, the electronic device may immediately render b. However, the electronic device may not have completed rendering b at time t3.

[0167] In this embodiment of the present application, under special circumstances, the electronic device can trigger the synthesis of the image through the above-mentioned first message. Therefore, at time t31, that is, when the electronic device completes rendering b, it can trigger the synthesis of the rendered b, and obtain image B after time t4.

[0168] In an embodiment of the present application, when the electronic device triggers the synthesis of an image through the above-mentioned first message, a vertical synchronization signal 3 can be temporarily added at time t4.5. Specifically, the electronic device can determine time t4.5 based on the time when image B is synthesized and the current signal cycle. At time t4.5, the electronic device has completed the synthesis of image B, and the electronic device can respond to the vertical synchronization signal 3 at time t4.5 to display image B. The electronic device can also display image B between time t4.5 and time t5.

[0169] The electronic device can respond to vertical synchronization signal 1 at time t3 and draw image c. After drawing is complete, the electronic device can immediately render image c. At time t4, the electronic device generates vertical synchronization signal 2. However, at time t4, the electronic device has not yet completed synthesizing image B. Therefore, the electronic device can respond to vertical synchronization signal 2 at time t4, wait until the synthesis of image B is complete, and then synthesize the rendered image c to obtain image C. At time t5, the electronic device generates vertical synchronization signal 3. The electronic device can respond to vertical synchronization signal 3 at time t5 and display image C.

[0170] In one possible implementation, the process for the electronic device to obtain image C can refer to the process for obtaining image A, or the process for obtaining image B. That is, in this embodiment of the present application, the electronic device may not be triggered by vertical synchronization signal 2 to continuously synthesize images, but may be triggered by the first message. The electronic device may also be triggered by a temporarily added vertical synchronization signal 3 to continuously display images.

[0171] The following is a schematic diagram of another image display process provided by an embodiment of the present application, in conjunction with Figure 9. In the flowchart shown in Figure 9, the delay time for the electronic device to synthesize layers can be reduced, allowing the electronic device to synthesize layers as quickly as possible, thereby ensuring that the image is displayed without delay. The image can also be displayed in advance. In addition, the normal display of subsequent multiple frames of images can be maintained without affecting the display. The method process includes:

[0172] S901: At a first moment, the electronic device draws and renders a first layer, wherein the duration of drawing and rendering the first layer is less than a single signal cycle.

[0173] S902: At a second moment, the electronic device draws and renders a second layer, and synthesizes the first image based on the first layer, wherein the time for drawing and rendering the second layer is greater than a single signal cycle, and the second moment is separated from the first moment by a single signal cycle.

[0174] S903: At a third moment, the electronic device draws and renders a third layer and displays the first image, wherein the third moment is separated from the second moment by two signal cycles.

[0175] The third moment may be the moment when the electronic device generates a vertical synchronization signal 1. The vertical synchronization signal 1 may be used to trigger the electronic device to draw a third layer. The third layer may be a layer that starts drawing in response to the vertical synchronization signal 1 at the third moment.

[0176] S904: After the electronic device completes rendering for the second layer and before the time when the first vertical synchronization signal is generated after the second layer is rendered (i.e., the fourth moment), the electronic device synthesizes a second image based on the second layer. The time when the second image synthesis is completed is after the fourth moment, and the fourth moment is separated from the third moment by a single signal cycle.

[0177] For S901-S904, please refer to S501-S504 and related descriptions in FIG5 .

[0178] The fact that the second image synthesis is completed at S904 after the fourth moment indicates that the electronic device has not yet completed synthesis of the second image between the third and fourth moments. Therefore, in this case, the electronic device cannot execute S705. However, in this embodiment of the present application, the second image can still be displayed in advance by temporarily adding a vertical synchronization signal 2. For details, see S906 below.

[0179] S905: After the electronic device completes synthesizing the second image, the electronic device synthesizes a third image based on the third layer.

[0180] The fact that the electronic device synthesizes the second layer after the fourth moment shows that even if the electronic device can complete rendering the third layer before the fourth moment, the electronic device can still obtain the rendered third layer. However, because the electronic device is synthesizing the second image at the fourth moment, SurfaceFlinger is busy and cannot synthesize the third image. Therefore, the electronic device needs to wait until the synthesis of the second image is completed before starting to synthesize the third image.

[0181] S906: After the electronic device completes synthesizing the second image, the electronic device displays the second image before a fifth moment, wherein the fifth moment is separated from the fourth moment by a single signal cycle.

[0182] In some implementations, the electronic device may temporarily add a new vertical synchronization signal 3 after the electronic device completes synthesis of the second image but before the fifth moment, that is, at the sixth moment. The electronic device may display the second image in response to the vertical synchronization signal 3 at the sixth moment. The temporary addition of a new vertical synchronization signal 3 indicates that the duration between the vertical synchronization signal 3 generated by the electronic device after the addition and the vertical synchronization signal 3 before the addition is still separated by a single signal cycle. For example, a new vertical synchronization signal 3 is added at time t4.5, and the duration between the vertical synchronization signal generated by the electronic device at time t5 and the vertical synchronization signal generated at time t4 is separated by a single signal cycle.

[0183] In one possible implementation, the electronic device can achieve the above situation by adjusting the signal period. Specifically, the electronic device can achieve the above situation by adjusting the signal period three times: the first adjustment adjusts the single signal period to the duration between the sixth moment and the fourth moment, the second adjustment can adjust it to the duration between the fifth moment and the sixth moment, and the third adjustment can adjust it back to the single signal period.

[0184] Here's how to make the first adjustment:

[0185] In some implementations, the electronic device can display the second image in advance by adjusting the period of the vertical synchronization signal 3, that is, displaying the second image at the fifth moment is advanced to displaying the second image at the sixth moment. Specifically, the electronic device can determine the range of adjusting the signal period of the vertical synchronization signal 3 based on one or more of the following: the current signal period, the moment when the synthesis of the second image is completed, and the moment when the last vertical synchronization signal 2 generated before the synthesis of the second image is completed (that is, the fourth moment). Specifically, the electronic device can set the signal period after the first adjustment to be at least the length of time between the moment when the synthesis of the second image is completed and the fourth moment. Moreover, the vertical synchronization signal 3 obtained after the first adjustment of the signal period, that is, the newly added vertical synchronization signal 3, does not affect the signal periods of the vertical synchronization signal 2 and the vertical synchronization signal 1.

[0186] For example, assuming that the frame rate of the current electronic device display screen is 60Hz, the electronic device determines that the electronic device completes synthesizing the second image at the middle moment between the fourth moment and the fifth moment based on the current signal cycle, the moment when the synthesis of the second image is completed, and the fourth moment. Then, the electronic device can temporarily adjust the frame rate of the display screen to at most 120Hz. That is, the electronic device can temporarily adjust the frame rate of the display screen to any value between 60Hz and 120Hz. For example, after adjusting the frame rate of the electronic device to 120Hz, the electronic device can generate a newly added vertical synchronization signal 3 at the middle moment between the fourth moment and the fifth moment, and display the second image in response to the newly added vertical synchronization signal 3. In this way, the electronic device can display the second image at the fastest about 9.8ms in advance.

[0187] As another example, assuming that the frame rate of the current electronic device display screen is 60Hz, the electronic device determines that the electronic device completes synthesizing the second image at a moment that is two-thirds of the signal cycle from the fourth moment to the fifth moment based on the current signal cycle, the moment when the synthesis of the second image is completed, and the fourth moment. Then, the electronic device can temporarily adjust the frame rate of the display screen to at most 90Hz. That is, the electronic device can temporarily adjust the frame rate of the display screen to any value between 60Hz and 90Hz. For example, after adjusting the frame rate of the electronic device to 90Hz, the electronic device can generate a newly added vertical synchronization signal 3 in response to a moment that is two-thirds of the signal cycle from the fourth moment after the fourth moment, and display the second image in response to the newly added vertical synchronization signal 3.

[0188] In the embodiment of the present application, the purpose of the electronic device adjusting the signal period for the first time is to minimize the electronic device storing the synthesized image so that the electronic device can display the synthesized image as soon as possible after completing the synthesized image.

[0189] It is worth noting that, regardless of the value to which the electronic device adjusts the signal period for the first time, the second vertical synchronization signal 3 generated by the electronic device after the fourth moment is separated from the fourth moment by a single signal period. In the embodiment of the present application, a single signal period may refer to a single signal period that has not been adjusted.

[0190] S907. At the fifth moment, the electronic device displays a third image.

[0191] Because the electronic device had already displayed the second image before the fifth moment, that is, the second image had been consumed. Furthermore, the electronic device had already completed synthesis to obtain the third image before the fifth moment. Therefore, the electronic device can refresh the image displayed by the electronic device in response to the vertical synchronization signal 3 generated at the fifth moment. In other words, the electronic device can display the third image.

[0192] In this way, after the electronic device displays the image in advance by adjusting the signal cycle for the first time, although it may affect the synthesis of the next frame of image, the electronic device neither displays the next frame of image in advance nor delays the display, thereby improving the stability of the displayed image.

[0193] The first moment may refer to moment t1, the second moment may refer to moment t2, the third moment may refer to moment t3, the fourth moment may refer to moment t4, and the fifth moment may refer to moment t5. The first layer may refer to the layer that the electronic device begins drawing at moment t1, the second layer may refer to the layer that the electronic device begins drawing at moment t2. The first image may refer to image A that the electronic device begins synthesizing at moment t2, the second image may refer to image B that the electronic device begins synthesizing after moment t3, and the third image may refer to image C that the electronic device begins synthesizing after moment t4.

[0194] In the embodiment of the present application, the above S904 is explained by taking the time duration of drawing and rendering the second layer exceeding a single signal cycle but not exceeding two signal cycles as an example.

[0195] In another possible implementation, the duration of drawing and rendering the second layer shown in FIG9 exceeds a single signal cycle and multiple signal cycles. In this case, the electronic device can synthesize the second image based on the second layer after completing the rendering for the second layer and before the moment when the first vertical synchronization signal is generated after the second layer is rendered. The moment when the synthesis of the second image is completed is after the moment when the first vertical synchronization signal is generated after the second layer is rendered. Then, the electronic device can display the second image after completing the synthesis of the second image and before the moment when the first vertical synchronization signal is generated after the synthesis of the second image is completed.

[0196] In the embodiment of the present application, the application of the method flow shown in FIG9 to the example diagram shown in FIG8 is only for illustrative purposes and can also be applied in more scenarios, without limitation thereto.

[0197] In conjunction with the descriptions of FIG8 and FIG9 above, it can be seen that the image display method provided by the embodiment of the present application can ensure that the image can be synthesized in a timely and rapid manner and displayed in advance when the electronic device occasionally takes too long to draw and render a single frame layer (for example, exceeding a single signal cycle). In this way, not only can the consumption of excessive resources for storing the synthesized image be avoided, but the user can also observe high-frame rate images with a better viewing experience, and the stability of the displayed image can be guaranteed.

[0198] In one possible implementation, the image display method provided in the embodiment of the present application can be applied to games that have high requirements for hand tracking, such as shooting games, fighting games, racing games, music rhythm games, sports competitive games, multiplayer online technical competitive games, etc.

[0199] In one possible implementation, the duration of the electronic device drawing the rendering layer may exceed two single signal cycles, or even exceed more signal cycles. The duration of the electronic device synthesizing the layer may also be too long. In this case, the image display method shown in Figures 5, 7, and 9 above can still be used. This can also improve the problem of delayed image synthesis and the problem of delayed image display.

[0200] Next, in conjunction with FIG10 , an internal module interaction diagram of an operating system of an electronic device corresponding to the image display method process provided in an embodiment of the present application is introduced.

[0201] The first application, UI thread, rendering thread, Surface Flinger, and display driver data storage module shown in Figure 10 are modules included in the electronic device for managing the process from receiving touch events to displaying images. For the definitions of these modules, please refer to the detailed description of the electronic device software architecture above. The specific functions of these modules have been recorded in detail in the following process and will not be repeated here.

[0202] S1001: A first application receives a touch event for the first application.

[0203] In some implementations, the first application may include a gaming application. Specifically, it may include an application that requires high hand tracking. The touch event may be triggered by a user operation, which may include, but is not limited to, clicking or sliding on various controls and icons in the first application.

[0204] Specifically, when the touch sensor receives a user operation, a corresponding hardware interrupt is sent to the kernel layer. The touch driver in the kernel layer processes the touch operation into a touch event and sends the touch event to the first application.

[0205] S1002: The first application sends layer information corresponding to the touch event to the UI thread.

[0206] In some implementations, after receiving the touch event, the first application can determine the content that the first application needs to display, and the content needs to be drawn, rendered and synthesized before it can be displayed. Therefore, the first application can send layer information corresponding to the touch event to the UI thread.

[0207] S1003. The UI thread draws the layer.

[0208] Specifically, the UI thread may draw a layer in response to the vertical synchronization signal 1 based on the layer information corresponding to the touch event sent by the first application.

[0209] S1004: The UI thread sends the drawn layer to the rendering thread.

[0210] Specifically, after the UI thread finishes drawing the layer, it can send the finished layer to the rendering thread.

[0211] S1005: The rendering thread renders the layer.

[0212] Specifically, the rendering thread can render the layer based on the completed layer sent by the UI thread. In some implementations, the sum of the time it takes the UI thread to draw the layer and the time it takes the rendering thread to render the layer may be less than or equal to a single signal cycle, or may be greater than a single signal cycle.

[0213] S1006. The rendering thread sends the rendered layer, the rendering duration, and the rendering completion time to Surface Flinger.

[0214] In some implementations, after completing rendering a layer, the rendering thread may send the rendered layer, the duration of the rendering, and the timestamp of the completion of the rendering to Surface Flinger.

[0215] S1007. Surface Flinger determines whether to immediately synthesize the layer based on the duration of the drawing and rendering, the time when the first vertical synchronization signal 2 is received after the drawing and rendering starts, and the time when the rendering is completed.

[0216] In some implementations, after receiving a rendered layer, Surface Flinger needs to determine whether to immediately respond to the first message to synthesize the layer to obtain an image, or wait for the first vertical synchronization signal 2 after the rendering layer is completed to arrive and then synthesize the image.

[0217] Specifically, Surface Flinger can be divided into two scenarios based on the duration of rendering, the first vertical synchronization signal 2 received after starting rendering, and the moment when rendering is completed:

[0218] Scenario 1: The rendering duration is less than or equal to a single signal cycle, and the first vertical sync signal 2 after the start of layer drawing occurs after the rendering is completed.

[0219] In the first scenario, Surface Flinger can wait for the first vertical synchronization signal 2 after the rendering of the layer is completed before synthesizing the image. For example, the rendering time of the first layer is less than a single signal cycle, and the moment of the first vertical synchronization signal 2 after the first layer is started, that is, the second moment is after the moment of completing the rendering of the first layer. Surface Flinger can respond to the first vertical synchronization signal 2 after the rendering of the first layer, that is, the vertical synchronization signal 2 at the second moment, to synthesize the first layer and obtain the first image.

[0220] The second scenario: the rendering time is longer than a single signal cycle, and / or the time of the first vertical sync signal 2 after starting to draw the layer is before the time of completing the rendering.

[0221] Assume that in the second case, Surface Flinger still waits for the first vertical synchronization signal 2 after completing the rendering layer to synthesize the image, which will not only consume storage resources, but also delay the display of the synthesized image, which will have a great impact on the user experience.

[0222] Therefore, in an embodiment of the present application, in the second scenario, Surface Flinger may not wait for the first vertical synchronization signal 2 after the rendering of the layer is completed, but instead respond to the first message to synthesize the layer to obtain the image after receiving the rendered image. For example, the rendering time of the second layer is longer than a single signal cycle, and the moment of the first vertical synchronization signal 2 after the second layer is started to be rendered, that is, the third moment is before the moment of completing the rendering of the second layer. After receiving the rendered second layer, Surface Flinger can respond to the first message immediately or wait for a period of time to synthesize the second layer to obtain the second image.

[0223] When Surface Flinger determines that the current scene belongs to the second scene, S1008 - 1 is executed; when Surface Flinger determines that the current scene belongs to the first scene, S1008 - 2 is executed.

[0224] S1008-1. Surface Flinger immediately synthesizes images based on the rendered layers.

[0225] Specifically, when it is determined that the second scene exists, Surface Flinger may receive a first message that instructs Surface Flinger to synthesize an image based on the rendered layer immediately or after a period of time, without considering whether vertical synchronization signal 2 is present upon receiving the completed rendering layer.

[0226] S1009-1. Surface Flinger determines the timing of a temporarily added vertical synchronization signal 3 based on the timing of the last vertical synchronization signal 2 received before completing synthesis of the second image and the timing of completing synthesis of the layer.

[0227] In some implementations, in order to not affect the user experience, that is, to avoid delayed display, Surface Flinger can determine the time to temporarily add vertical synchronization signal 3. Specifically, Surface Flinger can determine the range of the signal period for the first adjustment.

[0228] From the second case above, it can be seen that the time when the first vertical synchronization signal 2 is received after the layer starts to be drawn is before the time when the rendering is completed.

[0229] However, in the second case, determining the range of the signal period for the first adjustment may also include the following methods:

[0230] The first method is based on the time when the composite layer is completed, the time when the first vertical synchronization signal 2 is received after the layer is drawn, and the range of the signal period of the first adjustment of the current signal period.

[0231] Specifically, the first method is implemented when the moment of completing the synthesis is after the moment of the first vertical synchronization signal 2 after the start of drawing the layer, and before the moment of the first vertical synchronization signal 2 after the start of synthesizing the layer. Exemplarily, as shown in Figure 6, the moment of completing the synthesis of the second image is after the third moment and before the fourth moment. Surface Flinger can determine the range of the signal period of the first adjustment based on the current signal period, the moment of completing the synthesis of the second image, and the third moment. For the specific implementation process of the range of the signal period of the first adjustment, please refer to the relevant description of S705 above, which will not be repeated here.

[0232] The second method is to determine the range of the signal period for the first adjustment based on the moment of completion of synthesis, the moment of the first vertical synchronization signal 2 after the synthesis of the layer is started, and the current signal period.

[0233] Specifically, the second method is implemented when the moment of completing synthesis is after the moment of the first vertical synchronization signal 2 after the synthesis layer is started. Exemplarily, as shown in Figure 8, the moment of completing synthesis of the second image is after the fourth moment. Surface Flinger can determine the range of the signal period for the first adjustment based on the current signal period, the moment of completing synthesis of the second image, and the fourth moment. For the specific implementation process of the range of the signal period for the first adjustment, please refer to the relevant description of S906 above, which will not be repeated here.

[0234] S1010-1: Surface Flinger sends the synthesized image and the temporarily added vertical synchronization signal 3 to the display driver.

[0235] S1011 - 1 . The display driver generates a temporarily added vertical synchronization signal 3 , and displays a synthesized image in response to the temporarily added vertical synchronization signal 3 .

[0236] In some implementations, regardless of the signal period adjustment, a single signal period separates the last vertical synchronization signal received before the adjustment and the second vertical synchronization signal 3 received after the adjustment. That is, after the display driver recovers the signal period of the vertical synchronization signal 3, a single signal period separates every two adjacent vertical synchronization signals 3.

[0237] Exemplarily, the fourth moment is separated from the third moment by a single signal cycle, and the fifth moment is separated from the fourth moment by a single signal cycle.

[0238] S1008-2. Surface Flinger responds to vertical synchronization signal 2 and synthesizes an image based on the rendered layer.

[0239] S1009-2. Surface Flinger sends the synthesized image to the display driver.

[0240] S1010-2. The display driver responds to the vertical synchronization signal 3 to display the synthesized image.

[0241] Specifically, when it is determined that the scene belongs to the first scene, S1008-2 to S1010-1 can be executed. Exemplarily, Surface Flinger responds to vertical synchronization signal 2 at the second moment and synthesizes the first image based on the rendered first layer. Surface Flinger sends the synthesized first image to the display driver. The display driver responds to vertical synchronization signal 3 at the third moment and displays the synthesized first image.

[0242] In the embodiment of the present application, the temporarily added vertical synchronization signal 3 may be referred to as a first vertical synchronization signal.

[0243] FIG11 shows a schematic structural diagram of the electronic device 100 .

[0244] The electronic device 100 may be equipped with Or a portable terminal device with other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, an in-vehicle device, a smart home device and / or a smart city device. Without limitation, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or touch panel, a desktop computer with a touch-sensitive surface or touch panel, and other non-portable terminal devices. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0245] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0246] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0247] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0248] In an embodiment of the present application, the processor 110 may be configured to determine whether the time of drawing the rendering layer includes the moment of generating the vertical synchronization signal 2. The processor 110 may also be configured to instruct Surface Flinger to immediately synthesize the image. The processor 110 may also be configured to determine the time of the temporarily added vertical synchronization signal 3 based on the moment of completion of synthesis and the moment of the last vertical synchronization signal 2 generated before the completion of synthesis of the second image.

[0249] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0250] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0251] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0252] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0253] The power management module 141 is used to connect the battery 142 , the charging management module 140 and the processor 110 .

[0254] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0255] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0256] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0257] A modem processor may include a modulator and a demodulator.

[0258] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0259] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0260] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0261] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0262] In an embodiment of the present application, the display screen 194 can be used to display a synthesized image when a vertical synchronization signal 3 is detected.

[0263] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0264] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0265] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0266] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0267] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0268] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0269] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0270] In the embodiment of the present application, the internal memory 121 can store images synthesized by Sureface Flinger.

[0271] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0272] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage cell potential level; universal flash storage (UFS) and embedded multi media card (eMMC) can be divided into UFS and eMMC according to the storage specification.

[0273] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0274] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0275] In the embodiment of the present application, the code for implementing the image display described in the embodiment of the present application may be stored in a non-volatile memory. When running the first application, the electronic device 100 may load the executable code stored in the non-volatile memory into the random access memory.

[0276] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory.

[0277] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0278] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0279] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or listen to hands-free calls through the speaker 170A.

[0280] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0281] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0282] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0283] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0284] The gyro sensor 180B may be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor 180B.

[0285] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0286] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0287] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0288] The distance sensor 180F is used to measure distance.

[0289] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.

[0290] The ambient light sensor 180L is used to sense the brightness of the ambient light.

[0291] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0292] The temperature sensor 180J is used to detect temperature.

[0293] Touch sensor 180K, also known as "touch device".

[0294] The bone conduction sensor 180M can acquire vibration signals.

[0295] The buttons 190 include a power button, a volume button, and the like.

[0296] Motor 191 can generate vibration prompts.

[0297] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0298] FIG12 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.

[0299] As shown in FIG. 12 , the chip 1200 includes one or more (including two) processors 1201 , a communication line 1202 and a communication interface 1203 . Optionally, the chip 1200 also includes a memory 1204 .

[0300] In some embodiments, the memory 1204 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0301] The method described in the above embodiment of the present application can be applied to the processor 1201, or implemented by the processor 1201. The processor 1201 may be an integrated circuit chip with signal processing capabilities. During the implementation process, the steps of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1201. The above-mentioned processor 1201 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 1201 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0302] The steps of the method disclosed in the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art, such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read-only memory (EEPROM). The storage medium is located in the memory 1204, and the processor 1201 reads the information in the memory 1204 and performs the steps of the above method in combination with its hardware.

[0303] The processor 1201 , the memory 1204 , and the communication interface 1203 can communicate with each other via the communication line 1202 .

[0304] In the above embodiment, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory or downloaded and installed in the memory in the form of software.

[0305] The present application embodiment also provides a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrations. For example, available media can include magnetic media (e.g., floppy disk, hard disk or tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid state disk (SSD)).

[0306] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to perform the above-mentioned image processing method.

[0307] The present embodiment provides a chip. The chip includes a processor configured to invoke a computer program stored in a memory to execute the technical solution of the above embodiment. The implementation principles and technical effects are similar to those of the above-mentioned related embodiments and will not be further described here.

[0308] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program or instructions. When the computer program or instructions are executed by a processor, the above-mentioned method is implemented. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0309] As a possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; computer-readable media may include magnetic disk storage or other magnetic disk storage devices. Moreover, any connecting line may also be appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disks and optical discs as used herein include compact discs (CDs), laser discs, optical discs, DVDs, floppy disks and Blu-ray discs, where disks generally reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0310] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described herein are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0311] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0312] In short, the above description is only an embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for image display, characterized in that: The method is applied to an electronic device, and includes: At a first moment, the electronic device draws a first layer and then renders the first layer, where the duration of drawing and rendering the first layer does not exceed a single signal cycle, where the signal cycle is a cycle in which the electronic device generates two adjacent vertical synchronization signals; At a second moment, the electronic device synthesizes the first layer to obtain a first image, draws a second layer, and then renders the second layer, where the duration of drawing and rendering the second layer exceeds a single signal cycle, and the second moment is after the first moment, and the second moment is separated from the first moment by a single signal cycle; At a third moment, the electronic device displays the first image, the third moment is located after the second moment, and a single signal cycle is separated from the second moment; After the electronic device completes rendering the second layer and before a first vertical synchronization signal is generated after the second layer is rendered, synthesizing the second layer to obtain a second image; The electronic device displays the second image.

2. The method according to claim 1, characterized in that The electronic device displaying the second image specifically includes: At a fourth moment, the electronic device displays the second image. The fourth moment is located after the third moment, and a single signal cycle is separated from the third moment.

3. The method according to claim 1, characterized in that The electronic device displaying the second image specifically includes: At a sixth moment, the electronic device displays the second image, and the sixth moment is located after the moment when the second image is obtained.

4. The method according to claim 3, characterized in that At the sixth moment, before the electronic device displays the second image, the method further includes: The electronic device generates a first vertical synchronization signal at the sixth moment, where the first vertical synchronization signal is used to trigger the electronic device to display the second image at the sixth moment.

5. The method according to claim 3, characterized in that The moment when the electronic device obtains the second image is after the third moment and before the fourth moment. The fourth moment is after the third moment. The fourth moment is separated from the third moment by a single signal cycle. The sixth moment is before the fourth moment.

6. The method according to claim 5, characterized in that The method further comprises: At the third moment, the electronic device draws a third layer and then renders the third layer, and the duration of drawing and rendering the third layer does not exceed a single signal cycle; At the fourth moment, the electronic device synthesizes the third layer to obtain a third image; At a fifth moment, the electronic device displays the third image. The fifth moment is located after the fourth moment, and a single signal cycle is separated from the fourth moment.

7. The method according to claim 3, characterized in that The moment when the electronic device obtains the second image is after a fourth moment, the fourth moment is after the third moment, the fourth moment is separated from the third moment by a single signal cycle, and the sixth moment is after the fourth moment.

8. The method according to claim 7, characterized in that The method further comprises: At the third moment, the electronic device draws a third layer and then renders the third layer, and the duration of drawing and rendering the third layer does not exceed a single signal cycle; After the electronic device obtains the second image, the electronic device synthesizes the third layer to obtain a third image; At a fifth moment, the electronic device displays the third image. The fifth moment is after the fourth moment, and the fifth moment is the moment when the electronic device generates the first vertical synchronization signal after obtaining the third image.

9. The method according to any one of claims 1 to 8, characterized in that At the first moment, before the electronic device draws the first layer and then renders the first layer, the method further includes: The electronic device starts a first application, which includes one or more of the following: a shooting game, a fighting game, a racing game, a music rhythm game, a sports game, and a multiplayer online technical competition game.

10. The method according to any one of claims 1 to 8, characterized in that The electronic device further includes: a drawing thread, a rendering thread, a graphics synthesizer, and a display screen, wherein: At the first moment, the electronic device draws the first layer through the drawing thread, and then renders the first layer through the rendering thread; At the second moment, the electronic device synthesizes the first layer through the graphics synthesizer to obtain the first image, draws the second layer through the drawing thread, and then renders the second layer through the rendering thread; At a third moment, the electronic device displays the first image through the display screen; After the electronic device completes rendering the second layer and before a time when a first vertical synchronization signal is generated after the completion of rendering the second layer, synthesizing the second layer by the graphics synthesizer to obtain the second image; The electronic device displays the second image through the display screen.

11. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

Images