Image processing method, electronic device, readable medium and program product

Abstract

The invention relates to the technical field of intelligent terminals, and discloses an image processing method, electronic equipment, a readable medium and a program product. According to the image processing method, under the condition that the to-be-generated image frame comprises the display contents of the multiple applications, the first application with the maximum rendering frame rate value in the multiple applications can be determined, and the Vsync-app signals of other applications except the first application in the multiple applications are adjusted to be synchronized with the Vsync-app signal of the first application. The terminal device may then draw, render, and synthesize an image frame based on the adjusted Vsync-app signal. Therefore, even if at least two applications in the plurality of applications of the terminal equipment update the rendering content, the image synthesis module only needs to perform one-time image synthesis, so that the synthesis times of the image synthesis module are reduced, the synthesis frame rate is reduced, and the power consumption is greatly reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent terminal technology, and in particular to an image processing method, an electronic device, a readable medium and a program product. Background Art

[0002] In the process of generating a display interface, the terminal device usually draws, renders and synthesizes images based on the vertical synchronization (Vsync) signal. Different applications can draw and render their own layers based on the Vsync signal, and after the rendering is completed, the module for synthesizing images in the operating system of the terminal device (such as surface synthesis (SF), hereinafter referred to as the image synthesis module) is triggered to synthesize the rendered layers into image frames for display on the display screen.

[0003] When the interface displayed by the terminal device includes multiple application layers, after each application completes rendering the layers to be displayed, the image layering module will apply for a software Vsync signal to trigger the synthesis of image frames. In this way, when the software Vsync signals (hereinafter referred to as Vsync-sf signals) used by the multiple applications to trigger the image synthesis module to synthesize image frames are not synchronized, the frequency of synthesizing image frames by the image synthesis module will increase, and the power consumption of the terminal device will increase. For example, when the image frames displayed by the terminal device at a refresh rate of 120Hz include images of application A and application B, and the frame rate of application A and application B updating the layers is 60 frames per second (FPS), the period of the Vsync-sf signals of application A and application B is 16.66 milliseconds (ms). If the start time of the Vsync-sf signal of application A and the Vsync-sf signal of application B differs by 10ms, each Vsync-sf signal of application A and application B will trigger the image synthesis module to synthesize image frames, and the image synthesis module will synthesize image frames at a frequency of 60fps+60fps=120fps. Summary of the invention

[0004] In order to solve the problem that when the Vsync-sf signals used by multiple applications to trigger the image synthesis module to synthesize image frames are not synchronized, the frequency of the image synthesis module synthesizing image frames will increase, and the power consumption of the terminal device will increase, the embodiments of the present application provide an image processing method, an electronic device, a readable medium and a program product.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a terminal device, and the image processing method includes: running a first application and a second application, wherein the first application renders a layer based on a first rendering frame rate, and the second application renders a layer based on a second rendering frame rate; the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers; the first application renders a layer of the i+1-th frame interface of the first application at a third moment, and the second application renders a layer of the j+1-th frame interface of the second application at the third moment, wherein the first time difference between the third moment and the first moment is different from the first time interval corresponding to the first rendering frame rate, or the second time difference between the third moment and the second moment is different from the second time interval corresponding to the second rendering frame rate; after the third moment, the first application renders the layer based on the first rendering frame rate, and the second application renders the layer based on the second rendering frame rate.

[0006] In some embodiments, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment.

[0007] In an embodiment of the present application, after the third moment, the time interval between the layers of adjacent frame interfaces drawn by the first application is the first time interval corresponding to the first rendering frame rate. For example, the time interval between the layer of the i+1 frame interface drawn by the first application and the layer of the i+2 frame interface is the first time interval, and the time interval between the layer of the i+2 frame interface and the layer of the i+3 frame interface is the first time interval.

[0008] After the third moment, the time interval between the layers of adjacent frame interfaces drawn by the second application is the second time interval corresponding to the second rendering frame rate. For example, the time interval between the layer of the j+1 frame interface and the layer of the j+2 frame interface drawn by the second application is the second time interval, and the time interval between the layer of the j+2 frame interface and the layer of the j+3 frame interface is the second time interval.

[0009] In an embodiment of the present application, after a third moment, the first application renders a layer based on a first rendering frame rate, and the second application renders a layer based on a second rendering frame rate. That is, after the third moment, the moment when the first application renders the layer based on the first rendering frame rate and the moment when the second application renders the layer based on the second rendering frame rate overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the first rendering frame rate and the layer rendered by the second application based on the second rendering frame rate can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0010] In a possible implementation, before the third moment, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than a third value of the synthesis frame rate of the terminal device.

[0011] It can be understood that when the moment when the first application renders the layer based on the first rendering frame rate and the moment when the second application renders the layer based on the second rendering frame rate are completely staggered, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than the third value of the composite frame rate of the terminal device.

[0012] In a possible implementation, when the first condition is met, the first time difference is equal to the first time interval, and the second time difference is greater than the second time interval; wherein the first condition includes any one of the following conditions: the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate; the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is greater than the second display area of ​​the interface of the second application on the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, and the distance between the interface of the first application and the center of the display screen is less than the second threshold; the first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, the distance between the interface of the first application and the center of the display screen is less than the second threshold, and the interface of the first application is on the upper layer of the interface of the second application; the first value is equal to the second value, the first display area is greater than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on the upper layer of the interface of the second application.

[0013] It can be understood that the distance between the interface of the first application and the center of the display screen can be the distance between the center of the display area of ​​the first application on the display screen and the center of the display screen. The center of the display screen refers to the geometric center of the display screen, that is, the midpoint position of the horizontal and vertical directions of the display screen. The center of the interface of the first application is the geometric center of the interface of the first application, that is, the midpoint position of the horizontal and vertical directions of the display area of ​​the first application on the display screen.

[0014] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can retain the visual advantages brought by the high rendering frame rate.

[0015] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0016] Therefore, when the first numerical value is equal to the second numerical value, the second application can be aligned with the first application when it is determined that the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen; or, the first display area is larger than the first threshold and the distance between the interface of the first application and the center of the display screen is smaller than the second threshold; or, the first display area is larger than the first threshold and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; or, the first display area is larger than the first threshold, the distance between the interface of the first application and the center of the display screen is smaller than the second threshold, and the interface of the first application is on an upper layer of the interface of the second application; or, the first display area is larger than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application.

[0017] In a possible implementation, when the first condition is met, the i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0018] In a possible implementation, the first application renders the layer of the i+1th frame interface of the first application at the third moment, and the second application renders the layer of the j+1th frame interface of the second application at the third moment, including: the first application renders the layer of the i+1th frame interface of the first application at the third moment, and the second application renders the layer of the j+1th frame interface of the second application at the third moment when at least one of the following conditions is met: the first value is an integer multiple of the second value; the ratio of the first value to the second value is greater than or equal to the first ratio.

[0019] It can be understood that the first ratio can be a positive number greater than 1, for example, the first ratio=1.5.

[0020] In a possible implementation, the second time difference is the sum of the second time interval and the delay time; delay time=first moment-second moment+first time interval-second time interval.

[0021] In some embodiments, the delay time=k times of the second time interval, 0<k<1.

[0022] In one possible implementation, the second application renders the layer of the j+1th frame interface of the second application at the third moment, including: after the second moment, delaying the Vsync-app signal of the second application by a delay time; and rendering the layer of the j+1th frame interface of the second application based on the delayed Vsync-app signal.

[0023] It can be understood that when the first value is an integer multiple of the second value (or first value / second value=N, where N is a positive integer), the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole in the time domain.

[0024] For example, the first value is 60 and the second value is 30. The Vsyns-app signal of the second application can be delayed as a whole in the time domain so that the delayed Vsyns-app signal of the second application is aligned with the Vsyns-app signal of the first application.

[0025] It can be understood that when the first frame rate is an integer multiple of the second frame rate, if the Vsync-app signals of the first application and the second application are not synchronized, the composite frame rate will be the first value + the second value; after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate will be the first value. In this way, the number of composites of the first application rendering a layer based on the first rendering frame rate and the second application rendering a layer based on the second rendering frame rate can be reduced, thereby reducing the composite frame rate and power consumption.

[0026] When the first value and the second value are both greater than or equal to a preset threshold, and the first value is not an integer multiple of the second value (or first value / second value ≠ N), the Vsync-app signal of the second application can be partially aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole in the time domain.

[0027] For example, the preset threshold is 60, the first ratio is 1.5, the first value is 90, the second value is 60, and the first value / second value = 1.5. The Vsync-app signal of the second application can be delayed as a whole in the time domain so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0028] It can be understood that if the rendering frame rate of the application is greater than or equal to a preset threshold, for example, the rendering frame rate of the application is greater than or equal to 60fps, it is determined that the current scene is a high refresh rate scene.

[0029] When the rendering frame rates of the first application and the second application are both greater than or equal to the preset threshold, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, even if the second application loses a frame or has a certain delay, the user does not notice it. Therefore, in some embodiments, when the rendering frame rates of the first application and the second application are both greater than or equal to the preset threshold, even if the rendering frame rate of the first application is not an integer multiple of the rendering frame rate of the second application, the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the application as a whole in the time domain by a delay time T.

[0030] It can be understood that when both the first value and the second value are greater than or equal to the preset threshold, and the first value is not an integer multiple of the second value (or the first value / second value ≠ N), after the Vsync-app signals of the first application and the second application are synchronized, since the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, the synthesis frame rate after alignment will be less than the synthesis frame rate before alignment. In this way, the image synthesis module can reduce the number of synthesis times, reduce the synthesis frame rate and power consumption.

[0031] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold, and the first value is not an integer multiple of the second value (or first value / second value ≠ N), after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0032] In a possible implementation, when the second condition is met, the second time difference is equal to the second time interval, and the first time difference is greater than the first time interval, wherein the second condition includes any one of the following conditions: the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate; the second value is equal to the first value, and the second display area of ​​the interface of the second application on the display screen is greater than the first display area of ​​the interface of the first application on the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold; the second value is equal to the first value, the second display area is greater than the first threshold, and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, the distance between the interface of the second application and the center of the display screen is less than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; the second value is equal to the first value, the second display area is greater than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

[0033] It can be understood that the distance between the interface of the second application and the center of the display screen can be the distance between the center of the display area of ​​the second application on the display screen and the center of the display screen. The center of the display area of ​​the second application on the display screen refers to the geometric center of the display area of ​​the second application on the display screen, that is, the midpoint position of the display area of ​​the second application on the display screen in the horizontal and vertical directions.

[0034] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate, aligning the first application with the second application can retain the visual advantages brought by the high rendering frame rate.

[0035] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0036] Therefore, when the second numerical value is equal to the first numerical value, the first application can be aligned with the second application when it is determined that the second display area of ​​the interface of the second application on the display screen is larger than the first display area of ​​the interface of the first application on the display screen; or, the second display area is larger than the second threshold and the distance between the interface of the second application and the center of the display screen is smaller than the second threshold; or, the second display area is larger than the first threshold and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; or, the second display area is larger than the first threshold, the distance between the interface of the second application and the center of the display screen is smaller than the second threshold, and the interface of the second application is on an upper layer of the interface of the first application; or, the second display area is larger than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on an upper layer of the interface of the first application.

[0037] In a possible implementation, the jth frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0038] In an embodiment of the present application, corresponding to the scenario where the first application is aligned with the second application, that is, when the second condition is met, the jth frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0039] In a possible implementation, after the third moment, the refresh rate of the display screen of the terminal device is adjusted based on the synthetic frame rate of the terminal device.

[0040] In some embodiments, after the third moment, the refresh rate of the display screen of the terminal device can be adjusted based on the frame rate of the gear corresponding to the synthetic frame rate of the terminal device.

[0041] In some embodiments, after the third moment, when it is determined that the change in the synthetic frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen can be adjusted based on the synthetic frame rate of the terminal device.

[0042] In some embodiments, after the third moment, when it is determined that the change in the synthetic frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen can be adjusted based on the frame rate of the synthetic frame rate of the terminal device corresponding to the gear.

[0043] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing any image processing method provided by the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect.

[0044] In a third aspect, an embodiment of the present application provides a readable medium having instructions stored thereon, which, when executed on an electronic device, enables the electronic device to implement any image processing method provided by the first aspect and various possible implementations of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When executed by an electronic device, the electronic device implements any image processing method provided by the first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 According to an embodiment of the present application, a schematic diagram of a method for determining a hardware Vsync period is shown; Figure 2 According to an embodiment of the present application, a schematic diagram of generating a software Vsync signal based on a hardware Vsync signal is shown; Figure 3 A schematic flow chart of an image processing method is shown; Figure 4 According to an embodiment of the present application, a first schematic diagram of a terminal device performing layer rendering, synthesis, and image frame display is shown; Figure 5 According to an embodiment of the present application, a second schematic diagram of a terminal device performing layer rendering, synthesis, and image frame display is shown; Figure 6aAccording to an embodiment of the present application, a first flow chart of an image processing method is shown; Figure 6b According to an embodiment of the present application, a schematic diagram of a multi-window scenario is shown; Figure 6c According to an embodiment of the present application, a schematic diagram of a process of controlling a first application to render a layer of an i+1th frame interface of the first application at a third moment, and a second application to render a layer of a j+1th frame interface of the second application at a third moment is shown; Figure 7a According to an embodiment of the present application, a first timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 7b According to an embodiment of the present application, a second timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 7c According to an embodiment of the present application, a third timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 8 According to an embodiment of the present application, a fourth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 9a According to an embodiment of the present application, a fifth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 9b According to an embodiment of the present application, a sixth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig.9c According to an embodiment of the present application, a seventh timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 9d According to an embodiment of the present application, an eighth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig.9e According to an embodiment of the present application, a first schematic diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 9f According to an embodiment of the present application, a second schematic diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Figure 9g According to an embodiment of the present application, a schematic diagram of a synthetic frame rate is shown; Fig.10 According to an embodiment of the present application, a second flow chart of an image processing method is shown; Fig.11a According to an embodiment of the present application, a ninth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig.11b According to an embodiment of the present application, a tenth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig.12 According to an embodiment of the present application, a third flow chart of an image processing method is shown; Fig.13 According to an embodiment of the present application, a fourth flow chart of an image processing method is shown; Fig.14 According to an embodiment of the present application, a fifth flow chart of an image processing method is shown; Fig.15 According to an embodiment of the present application, a sixth flow chart of an image processing method is shown; Fig.16 According to an embodiment of the present application, a seventh flow chart of an image processing method is shown; Fig.17 According to an embodiment of the present application, an eighth schematic diagram of an image processing method is shown; Fig.18a According to an embodiment of the present application, a timing diagram of a Vsync-app signal of a first application, a Vsync-app signal of a second application, and a Vsync-app signal of a third application is shown; Fig.18b According to an embodiment of the present application, a timing diagram of a Vsync-sf signal is shown; Fig.19 According to an embodiment of the present application, an eleventh timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig. 20 According to an embodiment of the present application, a twelfth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown; Fig.21 According to an embodiment of the present application, a structural schematic diagram of an electronic device is shown; Fig. 22 According to an embodiment of the present application, a ninth schematic diagram of an image processing method is shown; Fig.23a According to an embodiment of the present application, a first interactive schematic diagram of an image processing method is shown; Figure 23b According to an embodiment of the present application, a second interactive schematic diagram of an image processing method is shown; Fig.24 According to an embodiment of the present application, a structural schematic diagram of an electronic device 10 is shown. DETAILED DESCRIPTION

[0047] The illustrative embodiments of the present application include, but are not limited to, an image processing method, a terminal device, a readable medium, and a program product.

[0048] The following first explains the technical terms involved in this application.

[0049] Refresh rate: refers to the number of times the terminal device display refreshes the image frame per second. The unit of refresh rate is hertz (Hz). For example, if the refresh rate of the terminal device is 60Hz, it means that the terminal device display can refresh and display the image frame 60 times per second.

[0050] Rendering frame rate: refers to the number of frames of a layer rendered by an application per unit time, measured in frames per second (fps). For example, the rendering frame rate of a video playback application is 30fps, which means that the video playback application can render 30 frames of a layer per second. It should be noted that an application can have one or more layers, and during the rendering process of a frame of a layer, part or all of the one or more layers may be updated.

[0051] Synthesis frame rate: refers to the number of image frames that can be synthesized for display by the image synthesis module of the terminal device per unit time. For example, if the synthesis frame rate of the image synthesis module is 60fps, it means that the image synthesis module can synthesize 60 image frames per second.

[0052] Vsync signal: Vsync signal includes hardware Vsync signal and software Vsync signal. The hardware Vsync signal and software Vsync signal are introduced below.

[0053] Hardware Vsync signal: The period of the hardware Vsync signal can be determined based on the inverse of the refresh rate of the terminal device display. Figure 1As shown in the figure, the hardware Vsync signal is generated by hardware. Low temperature polycrystalline silicon (LTPS) displays usually support fixed refresh rate gears, such as 60Hz, 90Hz, and 120Hz. Users can further set the refresh rate to 30Hz, 20Hz, 10Hz, 1Hz and other low refresh rates through low temperature polycrystalline oxide (LTPO) display technology.

[0054] In general, the refresh rate can be a preset fixed value, or a value that the terminal device intelligently adjusts based on the scenario. For example, the mobile phone can dynamically adjust the refresh rate according to the content; for example, for the scenario where the mobile phone plays a video, the screen refresh rate is set to 60Hz, that is, the hardware Vsync period is 16.66ms; for the scenario where the mobile phone runs a game application, the screen refresh rate is set to 90Hz or 120Hz, that is, the hardware Vsync period is 11.11ms or 8.33ms.

[0055] Software Vsync signal: The software Vsync signal is generated by the display synchronization module (DispSync) according to the hardware Vsync signal. The software Vsync signal is used to control the application to draw and render layers and the image synthesis module to synthesize image frames. The software Vsync signal includes a rendering signal (Vsync-app signal) and a synthesis signal (Vsync-sf signal). Among them, the Vsync-app signal is used to instruct the application APP to draw and render a frame of layers. Vsync-sf is used to trigger the image synthesis module to synthesize the layers rendered by the APP to obtain an image frame.

[0056] In some embodiments, the phase interval between the Vsync-app signal and the Vsync-sf signal is 0; in other embodiments, the phase interval between the Vsync-app signal and the Vsync-sf signal is a fixed value.

[0057] It is understood that the frame rate of the Vsync-app signal and the Vsync-sf signal can be determined based on the refresh rate of the screen. For example, if the refresh rate of the screen is 120Hz, the frame rate of the Vsync-app signal and the Vsync-sf signal can be 120fps, 90fps, 60fps, 30fps, etc. In some embodiments, the frame rates of the Vsync-app signal and the Vsync-sf signal of different applications are the same; in other embodiments, the frame rates of the Vsync-app signal and the Vsync-sf signal of different applications are different.

[0058] In some embodiments, Figure 2As shown, the calculation model in the display synchronization module can obtain the hardware Vsync signal and generate the software Vsync signal, namely the Vsync-app signal and the Vsync-sf signal, based on the hardware Vsync signal.

[0059] The technical solution of the present application is introduced below in conjunction with the accompanying drawings.

[0060] For ease of understanding, the process of image frame synthesis based on software Vsync signal of terminal device is first introduced. The terminal device may include application program and image synthesis module, such as Figure 3 As shown, the process of performing image frame synthesis includes: S1: Control the application to apply for Vsync-app signal.

[0061] In the embodiment of the present application, when the terminal device detects that the user opens any application of the terminal device, the control application applies to the driver for a Vsync-app signal. The application includes system applications and third-party applications. For example, the system application may be a setting application, camera, calendar, phone or information, etc., and the third-party application may be an instant messaging application, a game application, a video application, etc.

[0062] It can be understood that when the terminal device detects that the user starts the terminal device's application (including system applications such as cameras, phones, and third-party applications such as instant messaging tools or games), the terminal device's system automatically establishes a Vsync-app signal callback registration mechanism through the Choreographer framework. During the application initialization rendering pipeline stage, developers can declare the desired rendering frame rate parameters (such as setting a 90fps high refresh rate mode) through the SurfaceControl interface, and the parameters will be included in the synthesis decision tree of the image synthesis module for dynamic adaptation.

[0063] S2: Controls the application to draw and render the application's layer based on the Vsync-app signal.

[0064] In the embodiment of the present application, the terminal device can control the application to draw and render the next frame layer to be displayed on the display screen based on the Vsync-app signal when the Vsync-app signal sent by the driver is obtained. And after the application is controlled to render the next frame layer to be displayed, the next frame layer to be displayed is stored in the graphics buffer, the image synthesis module is notified to synthesize the next frame of the image to be displayed, and the Vsync-app signal is continued to be applied to the driver.

[0065] S3: Detecting that the application has finished rendering the layer, the image synthesis module is controlled to apply for a Vsync-sf signal.

[0066] In an embodiment of the present application, the terminal device detects that the application has completed layer rendering (for example, the next frame layer to be displayed is stored in the graphics buffer), and controls the image synthesis module to apply for a Vsync-sf signal from the driver.

[0067] S4: Control the image synthesis module to synthesize image frames based on the Vsync-sf signal.

[0068] In an embodiment of the present application, the terminal device can control the image synthesis module to read the next frame layer to be displayed stored in the graphics buffer based on the Vsync-sf signal after receiving the Vsync-sf signal sent by the driver, and synthesize the next frame layer to be displayed to obtain the next image frame.

[0069] Based on the above content, when the displayed image frame is obtained by superimposing the layers of multiple applications, after the rendering of the layer corresponding to each application is completed, the image synthesis module will be triggered to apply for the Vsync-sf signal and synthesize the image frame (execute the aforementioned steps S3 and S4). In the case where the Vsync-app signals of the multiple applications are not synchronized, the Vsync-sf signals of the multiple applications are also not synchronized, resulting in the image synthesis module needing to respond to the Vsync-sf signal of each application to synthesize the image frame, which will increase the synthesis frame rate of the image synthesis module synthesizing the image frame.

[0070] For example, the rendering frame rates of the first application and the second application are the same, the values ​​of the rendering frame rates of the first application and the second application are half of the refresh rate values, and the Vsync-app signals of the first application and the second application differ by one Vsync cycle. Figure 4 As shown, in the mth Vsync cycle, the display screen of the terminal device displays the image frame B0, and the first application responds to the Vsyns-app signal of the hardware Vsync signal of the mth Vsync cycle to start drawing and rendering the layer A1; after the rendering of the layer A1 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the mth Vsync cycle to start synthesizing the layer A1 to obtain the image frame B1.

[0071] In the m+1th Vsync cycle, the display screen displays image frame B1, and the second application responds to the Vsyns-app signal of the hardware Vsync signal of the m+1th Vsync cycle to start drawing and rendering layer C0. After the rendering of layer C0 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+1th Vsync cycle to start synthesizing layer A1 and layer C0 to obtain image frame B2.

[0072] In the m+2th Vsync cycle, the display screen displays image frame B2. The first application responds to the Vsyns-app signal of the hardware Vsync signal of the m+2th Vsync cycle to start drawing and rendering layer A2. After the rendering of layer A2 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+2th Vsync cycle to start synthesizing layer A2 and layer C0 to obtain image frame B3.

[0073] In the m+3th Vsync cycle, the display screen displays image frame B3, and the second application responds to the Vsyns-app signal of the hardware Vsync signal of the m+3th Vsync cycle to start drawing and rendering layer C1. After the rendering of layer C1 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+3th Vsync cycle to start synthesizing layer A2 and layer C1 to obtain image frame B4.

[0074] In the m+4th Vsync cycle, the display screen displays image frame B4. The first application responds to the Vsyns-app signal of the hardware Vsync signal of the m+4th Vsync cycle to start drawing and rendering layer A3. After the rendering of layer A3 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+4th Vsync cycle to start synthesizing layer A3 and layer C1 to obtain image frame B5.

[0075] for Figure 4 , the rendering frame rates of the first application and the second application are the same, and the Vsync-app signals of the first application and the second application differ by one Vsync cycle, that is, the Vsyns-app signals of the first application and the second application are completely staggered; this is equivalent to the synthetic frame rate of the synthetic image of the image synthesis module = the rendering frame rate of the first application + the rendering frame rate of the second application = 2× the rendering frame rate of the first application. Although the synthetic frame rate of the synthetic image of the image synthesis module is twice the rendering frame rate, it does not improve the display effect of the images of the first application and the second application, resulting in a waste of power consumption.

[0076] In view of this, an embodiment of the present application provides an image processing method, in which, when the image frame to be generated includes display contents of multiple applications, the terminal device can determine the first application with the largest rendering frame rate among the multiple applications, and adjust the Vsync-app signals of the other applications except the first application among the multiple applications to be synchronized with the Vsync-app signal of the first application. Then, the terminal device can draw, render and synthesize the image frame based on the adjusted Vsync-app signal.

[0077] Since the phase interval between the Vsync-sf signal of each application and the corresponding Vsync-app signal is the same (for example, 0 or a fixed value), in the image frame corresponding to the moment when the Vsync-app signals of the multiple applications overlap, even if at least two applications among the multiple applications update the rendering content, the image synthesis module only needs to perform image synthesis once, thereby reducing the number of synthesis times of the image synthesis module, and reducing the synthesis frame rate and power consumption.

[0078] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, aligning the Vsync-app signal of the application with a lower rendering frame rate with the Vsync-app signal of the first application with the largest rendering frame rate can retain the visual advantages brought by the high rendering frame rate.

[0079] It can be understood that the Vsync-app signal synchronization of multiple applications means that the Vsync-app signals of multiple applications are aligned in time. In the following, the value of the first frame rate of the first rendering frame rate of the first application is taken as the first value, the value of the second frame rate of the second rendering frame rate of the second application other than the first application among the multiple applications is taken as the second value, and the Vsync-app signal of the second application is adjusted to be aligned with the Vsync-app signal of the first application for explanation.

[0080] When the first value is an integer multiple of the second value (or first value / second value=N, where N is a positive integer), the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole by a delay time T in the time domain.

[0081] For example, if the first frame rate is 60fps and the second frame rate is 30fps, the second Vsyns-app signal of the second application can be delayed as a whole in the time domain by a delay time T so that the delayed second Vsyns-app signal of the second application is aligned with the first Vsyns-app signal of the first application.

[0082] It can be understood that when the first value is an integer multiple of the second value, if the Vsync-app signals of the first application and the second application are not synchronized, the synthesized frame rate will be the first frame rate + the second frame rate; after the Vsync-app signals of the first application and the second application are synchronized, the synthesized frame rate will be the first frame rate. In this way, the image synthesis module can reduce the number of synthesis times, reduce the synthesis frame rate and power consumption.

[0083] When both the first value and the second value are greater than or equal to a preset threshold, and the first value is not an integer multiple of the second value (or first value / second value ≠ N), when it is determined that the ratio of the first value to the second value is greater than or equal to the first ratio, the Vsync-app signal of the second application can be delayed as a whole by a delay time T in the time domain, so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0084] For example, the preset threshold is 60, the first ratio = 1.5, the first frame rate is 90fps, and the second frame rate is 60fps. The Vsync-app signal of the second application can be delayed by an overall delay time T in the time domain, so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0085] It can be understood that if the rendering frame rate of the application is greater than or equal to a preset threshold, for example, the rendering frame rate of the application is greater than or equal to 60fps, it is determined that the current scene is a high refresh rate scene.

[0086] When the rendering frame rates of the first application and the second application are both greater than or equal to the preset threshold, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, even if the second application loses a frame or has a certain delay, the user does not notice it. Therefore, in some embodiments, when the rendering frame rates of the first application and the second application are both greater than or equal to the preset threshold, even if the rendering frame rate of the first application is not an integer multiple of the rendering frame rate of the second application, the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the application as a whole in the time domain by a delay time T.

[0087] It can be understood that when both the first value and the second value are greater than or equal to the preset threshold, and the first frame rate is not an integer multiple of the second frame rate, after the Vsync-app signals of the first application and the second application are synchronized, since the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, the composite frame rate after alignment will be less than the composite frame rate before alignment. In this way, the image synthesis module can reduce the number of synthesis times, reduce the composite frame rate and power consumption.

[0088] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0089] In an embodiment of the present application, the delay time T = the moment when the first application draws the layer of the current frame interface - the moment when the second application draws the layer of the current frame interface + the first time interval corresponding to the first rendering frame rate of the first application - the second time interval corresponding to the second rendering frame rate of the second application.

[0090] It can be understood that the time interval represents the display time of each frame of the image. The first time interval corresponding to the first rendering frame rate=1 / the first rendering frame rate; the second time interval corresponding to the second rendering frame rate=1 / the second rendering frame rate.

[0091] For example, if the first rendering frame rate is 120fps, then the first time interval = 1 / 120 = 8.33ms; if the second rendering frame rate is 30fps, then the second time interval = 1 / 30 = 33.33ms.

[0092] In some embodiments, the delay time T=k times of the second time interval, 0<k<1.

[0093] For example, a first value of a first rendering frame rate of a first application is greater than a second value of a second rendering frame rate of a second application. Figure 4 In the scenario shown, the delay time T=t4-t3+first time interval-second time interval, that is, the Vsync-app signal of the second application rendering layer C1 is delayed from time t4 to time t5 by the delay time T, so that the Vsync-app signal used by the second application to render layer C1 is synchronized with the Vsync-app signal used by the first application to render layer A3, and the result is Figure 5 Schematic diagram shown.

[0094] In an embodiment of the present application, when the image frame to be generated includes display contents of multiple applications and the rendering frame rates of the multiple applications are all lower than the synthetic frame rate, indicating that the Vsync-app signal of at least one application among the multiple applications is not synchronized with the Vsync-app signals of other applications, the Vsync-app signals of other applications among the multiple applications except the first application can be adjusted to be synchronized with the Vsync-app signal of the first application.

[0095] In some embodiments, when there are at least two applications with the largest rendering frame rate among multiple applications, the application whose application interface has a larger display area on the display screen among the at least two applications can be selected as the first application; or, the application whose application interface has a display area on the display screen that is greater than a first threshold and a distance between the application interface and the center of the display screen that is less than a second threshold can be selected as the first application; or, the application whose interface has a display area on the display screen that is greater than the first threshold and the display area of ​​the application interface on the display screen includes the center of the display screen can be selected as the first application.

[0096] In some embodiments, when there are at least two applications with the largest rendering frame rate among multiple applications, the application whose application interface has a larger display area on the display screen can be determined from top to bottom according to the sizes of the Z-axis coordinates of the layers of the at least two applications as the first application; or, the application whose application interface has a display area on the display screen that is greater than a first threshold and a distance between the application interface and the center of the display screen that is less than a second threshold is determined as the first application; or, the application whose application interface has a display area on the display screen that is greater than the first threshold and the display area of ​​the application interface on the display screen includes the center of the display screen is determined as the first application.

[0097] It can be understood that, when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is usually easier to attract the user's visual attention. Therefore, the application whose display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer is used as the first application, and the Vsync-app signals of other applications except the first application among the multiple applications are adjusted to be synchronized with the Vsync-app signal of the first application, which can improve the user's visual experience.

[0098] Figure 6a According to an embodiment of the present application, a schematic diagram of a process of image processing is shown. It can be understood that Figure 6a Each step of the process shown in the figure is performed by the terminal device. Figure 6a The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 6a As shown, the image processing method includes: 101: It is detected that a first application renders a layer of an i-th frame interface of the first application at a first moment, and a second application renders a layer of a j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0099] In the embodiment of the present application, when the terminal device enters a preset scene, for example, a multi-window scene, a split-screen scene, a picture-in-picture playback scene, or a smart capsule playback scene, the terminal device runs a first application and a second application. The first application and the second application may be system applications or third-party applications. The system application may be a setting application, a camera, a calendar, a phone, or a short message, etc. The third-party application may be an instant messaging application, a game application, a video application, etc.

[0100] For example, the first application is a video application and the second application is a phone call. Figure 6b As shown, when the terminal device 10 displays the animation 1 of the interface 101 of the video application, it also displays the sequence frame animation of the interface 102 of the telephone.

[0101] In an embodiment of the present application, the first application may render a layer based on a first rendering frame rate, and the second application may render a layer based on a second rendering frame rate.

[0102] In the embodiments of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0103] 102: Control the first application to render a layer of the i+1th frame interface of the first application at a third moment, and the second application to render a layer of the j+1th frame interface of the second application at a third moment, wherein a first time difference between the third moment and the first moment is different from a first time interval corresponding to a first rendering frame rate, or a second time difference between the third moment and the second moment is different from a second time interval corresponding to the second rendering frame rate.

[0104] The method of controlling the first application to render the layer of the i+1th frame interface of the first application at the third moment and the second application to render the layer of the j+1th frame interface of the second application at the third moment is as follows: Figure 6c As shown, including: 1021: Obtain a first rendering frame rate of the first application, a second rendering frame rate of the second application, and a composite frame rate of the terminal device.

[0105] 1022: Based on the first rendering frame rate of the first application, the second rendering frame rate of the second application, and the composite frame rate of the terminal device, determine whether the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned.

[0106] If the judgment result is yes, the processing flow ends. If the judgment result is no, go to 1023: determine whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0107] In an embodiment of the present application, a method for determining whether the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned can be: determining whether the first value of the first rendering frame rate of the first application and the second value of the second rendering frame rate of the second application are both less than the third value of the composite frame rate of the terminal device; if the determination result is yes, the processing flow is terminated; if the determination result is no, the process goes to 1023: determining whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0108] It can be understood that when the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal are completely staggered, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both less than the third value of the composite frame rate of the terminal device. When the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal are aligned or partially aligned, the first value of the first rendering frame rate and / or the second value of the second rendering frame rate are equal to the third value of the composite frame rate of the terminal device.

[0109] For example, the first value is 30, the second value is 30, and the third value of the composite frame rate is 30, that is, the third value = the first value = the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned in each frame.

[0110] For another example, the first value is 30, the second value is 15, and the third value of the composite frame rate is 30, that is, the third value = the first value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned every 30 frames.

[0111] For another example, the first value is 15, the second value is 30, and the third value of the composite frame rate is 30, that is, the third value = the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned every 30 frames.

[0112] For another example, the first value is 30, the second value is 30, and the third value of the composite frame rate is 60, that is, the third value > the first value, and the third value > the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are not aligned in each frame.

[0113] For another example, the first value is 15, the second value is 30, and the third value of the composite frame rate is 45, that is, the third value > the first value, and the third value > the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are not aligned in each frame.

[0114] 1023: Determine whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0115] If it is determined that the first value is not an integer multiple of the second value, and the ratio of the first value to the second value is less than a preset first ratio, the processing flow ends. If it is determined that the first value is an integer multiple of the second value, or the ratio of the first value to the second value is greater than or equal to the first ratio, go to 1024: determine the main application of the first application and the second application.

[0116] It can be understood that if the first value is neither an integer multiple of the second value, and the ratio of the first value to the second value is smaller than the first ratio, even if the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned at the same time, or the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized at the same time at a certain moment, after this moment, the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are staggered, and the synthesis frame rate cannot be reduced; if the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned again after this moment, the processing flow will be increased.

[0117] It can be understood that the first ratio can be a positive number greater than 1, for example, the first ratio = 1.5. In some other embodiments, the first ratio = 1.8.

[0118] For example, the first ratio is 1.8, the preset threshold is 60, the first value is 50, the second value is 30, the first value and the second value are both less than 60, the first value / the second value=1.67<1.8, that is, the first value is not an integer multiple of the second value, the first value and the second value are both less than 60, and the ratio of the first value to the second value is also less than the first ratio, such as Figure 8 As shown, although the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized simultaneously at time t1, the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal after time t1 are staggered, and the synthesis frame rate cannot be reduced.

[0119] 1024: Determine a main application between the first application and the second application.

[0120] In an embodiment of the present application, a method for determining a main application is introduced below.

[0121] 1025: Calculate the delay time T of the non-main application, and delay the Vsync-app signal of the non-main application by the delay time T.

[0122] In an embodiment of the present application, if the second application is determined to be the main application, the Vsync-app signal of the layer of the i+1 frame interface rendered by the first application is delayed by a delay time T, so that the first application renders the layer of the i+1 frame interface of the first application at the third moment, and the second application renders the layer of the j+1 frame interface of the second application at the third moment; if the first application is determined to be the main application, the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application is delayed by a delay time T, so that the second application renders the layer of the j+1 frame interface of the second application at the third moment, and the first application renders the layer of the i+1 frame interface of the first application at the third moment.

[0123] For scenarios where the first application is the primary application, for example Figure 7a In the scenario shown in FIG. 1 , the Vsync-app signal of the second application rendering the j+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, and the result is Figure 9a The schematic diagram shown is as Figure 9a As shown, the second application renders the layer of the j+1th frame interface at the third time t4, and the first application renders the layer of the i+1th frame interface at the third time t4. Figure 7b In the scenario shown in FIG. 1 , the Vsync-app signal of the second application rendering the j+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, and the result is Figure 9b As shown in the schematic diagram, Figure 9b As shown, the second application renders the layer of the j+1th frame interface at the third time t4, and the first application renders the layer of the i+1th frame interface at the third time t4.

[0124] For the scenario where the second application is the primary application, for example, Figure 7c In the scenario shown in FIG. 1 , the Vsync-app signal of the first application rendering the i+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, thereby obtaining the following Fig.9c The schematic diagram shown is as Fig.9c As shown, the first application renders the layer of the i+1th frame interface at the third time t4, and the second application renders the layer of the j+1th frame interface at the third time t4.

[0125] 103: After the third moment, control the first application to render the layer based on the first rendering frame rate, and control the second application to render the layer based on the second rendering frame rate.

[0126] In an embodiment of the present application, after the third moment, the time interval between the layers of adjacent frame interfaces drawn by the first application is the first time interval corresponding to the first rendering frame rate. For example, the time interval between the layer of the i+1 frame interface drawn by the first application and the layer of the i+2 frame interface is the first time interval, and the time interval between the layer of the i+2 frame interface and the layer of the i+3 frame interface is the first time interval.

[0127] After the third moment, the time interval between the layers of adjacent frame interfaces drawn by the second application is the second time interval corresponding to the second rendering frame rate. For example, the time interval between the layer of the j+1 frame interface and the layer of the j+2 frame interface drawn by the second application is the second time interval, and the time interval between the layer of the j+2 frame interface and the layer of the j+3 frame interface is the second time interval.

[0128] The aligned synthetic frame rate is introduced below by taking the case where the first value is an integer multiple of the second value as an example.

[0129] For example, the first value is 30, and the second value is 30, that is, the first value is 1 times the second value. If the Vsync-app signals of the first application and the second application are not synchronized, such as Fig.9e As shown, the i-th frame layer and the j-th frame layer are rendered and synthesized at different times, and the time difference between synthesizing the i-th frame layer and synthesizing the j-th frame layer is 16.6ms, that is, the initial synthesis frame rate is 60fps.

[0130] If the first application is the main application, in the embodiment of the present application, the second Vsyns-app signal of the second application can be delayed by a delay time T in the time domain, so that the delayed second Vsyns-app signal of the second application is aligned with the first Vsyns-app signal of the first application every 30 frames, such as Figure 9f As shown, the i+1th frame layer and the j+1th frame layer are rendered and synthesized at the same time, and the resulting synthesis frame rate is as follows: Figure 9g As shown. Figure 9g As shown, if the time interval is 4s and the time interval includes the moment when the Vsync-app signals of the first application and the second application are not aligned, the image synthesis module can also synthesize 186 frames of layers within 4s, and the average synthesis frame rate is 45fps.

[0131] For example, the first value is 60, and the second value is 30, that is, the first value is twice the second value. If the Vsync-app signals of the first application and the second application are not synchronized, the initial composite frame rate = 30fps + 60fps = 90fps. In an embodiment of the present application, by delaying the second Vsyns-app signal of the second application as a whole in the time domain by a delay time T, the delayed second Vsyns-app signal of the second application is aligned with the first Vsyns-app signal of the first application every 60 frames, so that the composite frame rate is adjusted to 60fps.

[0132] For another example, the first value is twice the second value. If the Vsync-app signals of the first application and the second application are not synchronized, the initial synthetic frame rate = 1.5 × the first value. Figure 9b , by delaying the second Vsyncs-app signal of the second application rendering the j+1th frame layer from the fourth moment t3 to the third moment t4 by the delay time T, so that after the third moment t4, the Vsync-app signal of each frame layer of the second application coincides with the Vsync-app signal of the first application, and the value of the synthetic frame rate = the first value.

[0133] In an embodiment of the present application, by aligning the Vsync-app signal of an application with a lower rendering frame rate with the Vsync-app signal of an application with a higher rendering frame rate, the number of synthesis times of the rendering layer can be reduced, and the synthesis frame rate and power consumption can be reduced.

[0134] The aligned synthetic frame rate is introduced below by taking the second value being an integer multiple of the first value as an example.

[0135] For example, the second value is twice the first value. If the Vsync-app signals of the first application and the second application are not synchronized, the initial synthetic frame rate = 1.5 × the second value. Fig.9c , by delaying the first Vsyncs-app signal of the i+1th frame layer rendered by the first application from the fourth moment t3 to the third moment t4 by a delay time T, so that after the third moment t4, the Vsync-app signal of each frame layer of the first application coincides with the Vsync-app signal of the second application, and the value of the synthetic frame rate = the second value.

[0136] In an embodiment of the present application, by aligning the Vsync-app signal of an application with a lower rendering frame rate with the Vsync-app signal of an application with a higher rendering frame rate, the number of synthesis times of the rendering layer can be reduced, and the synthesis frame rate and power consumption can be reduced.

[0137] In the following, the aligned synthetic frame rate is introduced by taking the preset threshold value as 60 and the ratio of the first value to the second value being greater than or equal to the first ratio as an example.

[0138] For example, if the first value is 90 and the second value is 60, the first value is 1.5 times the second value. Figure 9d After the third moment t4, the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application again at t5, and the composite frame rate after alignment is less than the composite frame rate before alignment.

[0139] It can be understood that when both the first value and the second value are greater than or equal to the preset threshold value, and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first application and the second application are synchronized, since the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, the synthesis frame rate after alignment will be less than the synthesis frame rate before alignment. In this way, the image synthesis module can reduce the number of synthesis times, reduce the synthesis frame rate and power consumption.

[0140] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0141] In an embodiment of the present application, after the third moment, the refresh rate of the display screen of the terminal device can be adjusted based on the synthetic frame rate of the terminal device.

[0142] For example, after the third moment, the refresh rate of the display screen of the terminal device may be adjusted based on the frame rate of the gear corresponding to the composite frame rate of the terminal device. For another example, after the third moment, when it is determined that the amount of change in the composite frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the display screen of the terminal device may be adjusted based on the composite frame rate of the terminal device. For another example, after the third moment, when it is determined that the amount of change in the composite frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the display screen of the terminal device may be adjusted based on the frame rate of the gear corresponding to the composite frame rate of the terminal device.

[0143] It can be understood that adjusting the refresh rate of the display based on the composite frame rate means directly adjusting the refresh rate of the display based on the composite frame rate to optimize power consumption. For example, if the composite frame rate is 30fps, the refresh rate will be set to 30Hz to match the frame rate of the application currently running on the terminal device and reduce screen tearing. This method can improve the user experience.

[0144] Adjusting the refresh rate based on the gear corresponding to the composite frame rate means selecting a refresh rate gear that is closest or matches the composite frame rate. For example, if the composite frame rate is 29.97fps (close to 30fps), the refresh rate will be set to 30Hz. This approach ensures that the terminal device can provide the best viewing experience at different frame rates while reducing resource consumption and energy consumption. It allows the system to flexibly adjust the refresh rate in different application scenarios to adapt to different content and user needs.

[0145] In an embodiment of the present application, after the third moment, the composite frame rate of the terminal device is less than the composite frame rate of any moment before the third moment. In this way, the refresh rate of the terminal device display screen is adjusted according to the composite frame rate of the terminal device after the third moment, which can significantly improve the visual experience and optimize the performance of the terminal device.

[0146] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0147] Combine the following Fig.10 , the method for determining the main application in the first application and the second application in step S1024 in step 102 of the present application embodiment is introduced. It can be understood that Fig.10 Each step of the process shown in the figure is performed by the terminal device. Fig.10 The execution entities of each step will not be described repeatedly in the steps of the process shown. Fig.10 As shown, including: 201: Determine a maximum rendering frame rate according to the rendering frame rates of each application of the terminal device.

[0148] In an embodiment of the present application, the rendering frame rate of each application of the terminal device can be first obtained, and then the maximum rendering frame rate can be determined based on the rendering frame rate of each application of the terminal device. For example, the terminal device includes a first application and a second application, the first rendering frame rate of the first application is 60fps, and the second rendering frame rate of the second application is 60fps, then the maximum rendering frame rate is determined to be 60fps. For another example, the first rendering frame rate is 60fps, and the second rendering frame rate of the second application is 30fps, then the maximum rendering frame rate is determined to be 60fps. For another example, the first rendering frame rate is 30fps, and the second rendering frame rate of the second application is 60fps, then the maximum rendering frame rate is determined to be 60fps.

[0149] 202: Determine whether there is only one application corresponding to the maximum rendering frame rate.

[0150] If the judgment result is yes, then go to 203: determine that the application corresponding to the maximum rendering frame rate is the main application. If the judgment result is no, then go to 204: determine the display area of ​​the display area of ​​multiple applications corresponding to the maximum rendering frame rate on the display screen, the distance from the application interface to the center of the display screen, and the z-axis coordinate.

[0151] 203: Determine the application corresponding to the maximum rendering frame rate as the main application.

[0152] For example, if the first rendering frame rate is 60fps and the maximum rendering frame rate is 60fps, the first application is determined to be the main application. For another example, if the second rendering frame rate is 60fps and the maximum rendering frame rate is 60fps, the second application is determined to be the main application.

[0153] In the embodiment of the present application, if the terminal device includes a high refresh application in the application, the high refresh application can be used as the main application. It can be understood that the high refresh application can be an application with a frame rate greater than a preset frame rate, for example, the preset frame rate can be 90fps.

[0154] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can retain the visual advantages brought by the high rendering frame rate.

[0155] 204: Determine the display areas of the display regions of the multiple applications on the display screen corresponding to the maximum rendering frame rate, the distance from the application interface to the center of the display screen, and the z-axis coordinate.

[0156] In an embodiment of the present application, if the maximum rendering frame rate corresponds to multiple applications, the four vertex coordinates of the layers rendered by the multiple applications corresponding to the maximum rendering frame rate can be obtained, and the display area of ​​the display area of ​​the application on the display screen and the center point of the display area of ​​the application on the display screen are determined based on the four vertex coordinates of the layers rendered by the applications. The distance between the center of the display area of ​​the application on the display screen and the center of the display screen and the z-axis coordinate of the layer rendered by the application are determined.

[0157] It can be understood that the center of the display area applied on the display screen refers to the geometric center of the display area applied on the display screen, that is, the midpoint position of the display area applied on the display screen in the horizontal and vertical directions. The center of the display screen refers to the geometric center of the display screen, that is, the midpoint position of the display screen in the horizontal and vertical directions.

[0158] 205: Based on the z-axis coordinate, determine that among the multiple applications corresponding to the maximum rendering frame rate, the application whose display area is greater than the first threshold and whose interface is at a distance from the center of the display screen less than the second threshold is the main application, or the application whose interface includes the center of the display screen in the display area of ​​the display screen is the main application.

[0159] In an embodiment of the present application, each application of the terminal device can be traversed from large to small according to the z-axis coordinates of each application to determine that among the multiple applications corresponding to the maximum rendering frame rate, the application whose display area is greater than a first threshold and whose interface is at a distance from the center of the display screen less than a second threshold is the main application, or the application whose interface includes the center of the display screen in the display area of ​​the display screen is the main application.

[0160] For example, the applications of the terminal device include a first application and a second application. If it is determined that the first application and the second application meet the first condition, the first application is determined to be the main application. The first condition includes any one of the following conditions: the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate; the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is greater than the second display area of ​​the interface of the second application on the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, and the distance between the interface of the first application and the center of the display screen is less than the second threshold; the first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, the distance between the interface of the first application and the center of the display screen is less than the second threshold, and the interface of the first application is on the upper layer of the interface of the second application; the first value is equal to the second value, the first display area is greater than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on the upper layer of the interface of the second application.

[0161] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0162] Therefore, when the first numerical value is equal to the second numerical value, when it is determined that the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen; or, when the first display area is larger than the first threshold and the distance between the interface of the first application and the center of the display screen is smaller than the second threshold; or, when the first display area is larger than the first threshold and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; or, when the first display area is larger than the first threshold, the distance between the interface of the first application and the center of the display screen is smaller than the second threshold, and the interface of the first application is in an upper layer of the interface of the second application; or, when the first display area is larger than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is in an upper layer of the interface of the second application, the first application is used as the main application, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0163] For another example, the applications of the terminal device include a first application and a second application. If it is determined that the first application and the second application meet the second condition, the second application is determined to be the main application. The second condition includes any one of the following conditions: the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate; the second value is equal to the first value, and the second display area of ​​the interface of the second application on the display screen is greater than the first display area of ​​the interface of the first application on the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold; the second value is equal to the first value, the second display area is greater than the first threshold, and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, the distance between the interface of the second application and the center of the display screen is less than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; the second value is equal to the first value, the second display area is greater than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

[0164] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0165] Therefore, when the second numerical value is equal to the first numerical value, when it is determined that the second display area of ​​the interface of the second application on the display screen is larger than the first display area of ​​the interface of the first application on the display screen; or, the second display area is larger than the second threshold and the distance between the interface of the second application and the center of the display screen is smaller than the second threshold; or, the second display area is larger than the first threshold and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; or, the second display area is larger than the first threshold, the distance between the interface of the second application and the center of the display screen is smaller than the second threshold, and the interface of the second application is on an upper layer of the interface of the first application; or, when the second display area is larger than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on an upper layer of the interface of the first application, the second application is used as the main application, and the Vsync-app signal of the first application is aligned with the Vsync-app signal of the second application.

[0166] In an embodiment of the present application, if the first application is the main application, the delay time T of the Vsync-app signal of the second application can be calculated, so that after the second moment, the Vsync-app signal of the second application is delayed by the delay time T, and the layer of the j+1th frame interface of the second application is rendered based on the delayed Vsync-app signal.

[0167] In the embodiment of the present application, the delay time T=the first moment-the second moment+the first time interval-the second time interval. In some embodiments, the delay time T=k times of the second time interval, 0<k<1.

[0168] It can be understood that if the first application is the main application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is equal to the first time interval, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0169] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Fig.11aIn the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Fig.11a In the scenario shown, the delay time T = t4 - t3 - the first time interval, where t4 is the alignment time of the main application and the non-main application; t3 is the time when the layer of the j+1th frame interface of the second application is rendered before the delay time T of the second application.

[0170] In an embodiment of the present application, if the second application is the main application, the delay time T of the Vsync-app signal of the first application can be calculated, so that after the second moment, the Vsync-app signal of the first application is delayed by the delay time T, and the first application should render the layer of the i+1th frame interface of the first application based on the delayed Vsync-app signal.

[0171] In the embodiment of the present application, the delay time T=the second moment-the first moment+the first time interval-the second time interval. In some embodiments, the delay time T=k times of the first time interval, 0<k<1.

[0172] It can be understood that if the second application is the main application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is the sum of the first time interval and the delay time T, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the second time interval. The j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0173] For example, if the second application is the main application, Fig.9c In the scenario shown, the fifth moment is t3, and the delay time T = the second moment t2 - the first moment t1 + the first time interval - the second time interval. Fig.11b In the scenario shown, the fourth moment is t3, and the delay time T = the second moment t2 - the first moment t1 + the first time interval - the second time interval. Fig.11b In the scenario shown, the delay time T = t4 - t3 - the first time interval, where t4 is the alignment time of the main application and the non-main application; t3 is the time when the layer of the i+1th frame interface of the first application is rendered before the delay time T of the first application.

[0174] Combine the following Figures 12 to 17 , a method for aligning the Vsync-app of the first application and the second application when the Vsync-app of the first application and the second application are not aligned is introduced.

[0175] The following first introduces a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is greater than the second rendering frame rate of the second application, taking the first rendering frame rate of the first application as 60fps, the second rendering frame rate of the second application as 30fps, and the composite frame rate as 90fps as an example. Fig.12 As shown, including: 301: Run the first application and the second application.

[0176] In an embodiment of the present application, the terminal device can run any number of applications such as the first application and the second application.

[0177] 302: It is detected that the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0178] In the embodiments of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0179] 303: Determine that a first value of a first rendering frame rate of the first application and a second value of a second rendering frame rate of the second application are both smaller than a third value of a composite frame rate of the terminal device, and the first value is greater than the second value, and determine that the first application is the main application.

[0180] In the embodiment of the present application, the first value is 60, the second value is 30, and the third value is 90, that is, the first value and the second value are both smaller than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely staggered. Since the first value is greater than the second value, the first application is determined to be the main application.

[0181] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can retain the visual advantages brought by the high rendering frame rate.

[0182] 304: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1th frame interface rendered by the second application by the delay time T.

[0183] In the embodiment of the present application, the delay time T=the first moment-the second moment+the first time interval-the second time interval. In some embodiments, the delay time T=k times of the second time interval, 0<k<1.

[0184] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T=the first moment t2-the second moment t1+the first time interval-the second time interval.

[0185] 305: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0186] In the embodiment of the present application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the layer of the i-th frame interface of the first application is rendered is equal to the first time interval, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the layer of the j-th frame interface of the second application is rendered is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0187] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0188] The following takes the first rendering frame rate of the first application as 30fps, the second rendering frame rate of the second application as 60fps, and the composite frame rate as 90fps as an example to introduce a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is less than the second rendering frame rate of the second application. Fig.13 As shown, including: 401: Run the first application and the second application.

[0189] In an embodiment of the present application, the terminal device can run any number of applications such as the first application and the second application at the same time.

[0190] 402: It is detected that the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0191] In the embodiments of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0192] 403: Determine that a first value of a first rendering frame rate of the first application and a second value of a second rendering frame rate of the second application are both smaller than a third value of a composite frame rate of the terminal device, and the second value is greater than the first value, and determine that the second application is the main application.

[0193] In the embodiment of the present application, the first value is 30, the second value is 60, and the third value is 90, that is, the first value and the second value are both smaller than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely staggered. Since the second value is greater than the first value, the second application is determined to be the main application.

[0194] It is understandable that a high rendering frame rate can provide a smoother and clearer visual experience. In scenes that require high-speed dynamic images, such as e-sports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can retain the visual advantages brought by the high rendering frame rate.

[0195] 404: Calculate the delay time T, and delay the Vsync-app signal of the layer of the i+1th frame interface rendered by the first application by the delay time T.

[0196] In the embodiment of the present application, the delay time T=the second moment-the first moment+the first time interval-the second time interval. In some embodiments, the delay time T=k times of the first time interval, 0<k<1.

[0197] For example, if the second application is the main application, Fig.9c In the scenario shown, the delay time T=the second moment t2-the first moment t1+the first time interval-the second time interval.

[0198] 405: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0199] In the embodiment of the present application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the layer of the i-th frame interface of the first application is rendered is equal to the sum of the first time interval and the delay time T, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the layer of the j-th frame interface of the second application is rendered is the second time interval. The j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0200] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0201] The following takes the first rendering frame rate of the first application as 30fps, the second rendering frame rate of the second application as 30fps, and the composite frame rate as 60fps as an example to introduce a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is equal to the second rendering frame rate of the second application and the first display area of ​​the interface of the first application on the display screen is smaller than the second display area of ​​the interface of the second application on the display screen. Fig.14 As shown, including: 501: Run the first application and the second application.

[0202] In the embodiment of the present application, 501 and Fig.12 The method of the embodiment 301 shown is consistent and will not be repeated here.

[0203] 502: It is detected that the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0204] In the embodiment of the present application, 502 and Fig.12 The method of the embodiment 302 shown is consistent and will not be repeated here.

[0205] 503: Determine that a first value of the first rendering frame rate of the first application and a second value of the second rendering frame rate of the second application are both smaller than a third value of the composite frame rate of the terminal device, the first value is equal to the second value, and a first display area of ​​the interface of the first application on the display screen is larger than a second display area of ​​the interface of the second application on the display screen, and determine that the first application is the main application.

[0206] In the embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60, that is, the first value and the second value are both smaller than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely staggered. Since the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen, the first application is determined to be the main application.

[0207] In some embodiments, a first display area of ​​the first application's interface on the display screen is larger than a second display area of ​​the second application's interface on the display screen, and the first application's interface is on top of the second application's interface, determining that the first application is the primary application.

[0208] In some embodiments, if the first display area of ​​the first application interface on the display screen is smaller than the second display area of ​​the second application interface on the display screen, the second application is determined to be the main application. In some embodiments, if the first display area of ​​the first application interface on the display screen is smaller than the second display area of ​​the second application interface on the display screen, and the first application interface is at the lower layer of the second application interface, the second application is determined to be the main application.

[0209] It can be understood that an application with a larger display area of ​​its interface on the display screen is generally more likely to attract the user's visual attention.

[0210] 504: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1th frame interface rendered by the second application by the delay time T.

[0211] In the embodiment of the present application, the delay time T=the first moment-the second moment+the first time interval-the second time interval. In some embodiments, the delay time T=k times of the second time interval, 0<k<1.

[0212] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T=the first moment t2-the second moment t1+the first time interval-the second time interval.

[0213] In some embodiments, if the second application is the main application, the Vsync-app signal of the layer of the i+1 frame interface rendered by the first application is delayed by a delay time T, where the delay time T = the second moment - the first moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the first time interval, 0 < k < 1.

[0214] For example, if the second application is the main application, Fig.9c In the scenario shown, the delay time T=the second moment t2-the first moment t1+the first time interval-the second time interval.

[0215] 505: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0216] In the embodiment of the present application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the layer of the i-th frame interface of the first application is rendered is equal to the first time interval, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the layer of the j-th frame interface of the second application is rendered is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0217] In some embodiments, if the second application is the main application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is the sum of the first time interval and the delay time T, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the second time interval. The j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0218] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0219] The following takes the first rendering frame rate of the first application as 30fps, the second rendering frame rate of the second application as 30fps, and the composite frame rate as 60fps as an example to introduce a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is equal to the second rendering frame rate of the second application, the first display area of ​​the interface of the first application on the display screen is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen. Fig.15 As shown, including: 601: Run the first application and the second application.

[0220] In the embodiment of the present application, 601 and Fig.14 The method of the illustrated embodiment 501 is consistent and will not be described again here.

[0221] 602: It is detected that the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0222] In the embodiment of the present application, 602 and Fig.14 The method of the illustrated embodiment 502 is consistent and will not be described again here.

[0223] 603: Determine that a first value of the first rendering frame rate of the first application and a second value of the second rendering frame rate of the second application are both smaller than a third value of the composite frame rate of the terminal device, the first value is equal to the second value, and a first display area of ​​the interface of the first application on the display screen is greater than a first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and determine that the first application is the main application.

[0224] In the embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60, that is, the first value and the second value are both smaller than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely staggered. Since the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen, the first application is determined to be the main application.

[0225] In some embodiments, the first application is determined to be the main application if the first display area of ​​the interface of the first application on the display screen is greater than a first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application.

[0226] In some embodiments, if the second display area of ​​the second application interface on the display screen is greater than the first threshold and the display area of ​​the second application interface on the display screen includes the center of the display screen, the second application is determined to be the main application.

[0227] In some embodiments, if the second display area of ​​the second application's interface on the display screen is greater than the first threshold, the display area of ​​the second application's interface on the display screen includes the center of the display screen, and the first application's interface is at the lower layer of the second application's interface, then the second application is determined to be the main application.

[0228] It can be understood that an application interface with a larger display area on the display screen, including the center of the display screen, is generally more likely to attract the user's visual attention.

[0229] 604: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1th frame interface rendered by the second application by the delay time T.

[0230] In the embodiment of the present application, 604 and Fig.14 The method of the illustrated embodiment 504 is consistent and will not be described again here.

[0231] 605: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0232] In the embodiment of the present application, 605 and Fig.14 The method of the illustrated embodiment 505 is consistent and will not be described again here.

[0233] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0234] The following takes the case where the first rendering frame rate of the first application is 30fps, the second rendering frame rate of the second application is 30fps, and the composite frame rate is 60fps as an example, and introduces a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is equal to the second rendering frame rate of the second application, the first display area of ​​the first application interface on the display screen is greater than the first threshold, the first display area of ​​the first application interface on the display screen is greater than the first threshold, and the distance between the first application interface and the center of the display screen is less than the second threshold. Fig.16 As shown, including: 701: Run the first application and the second application.

[0235] In the embodiment of the present application, 701 and Fig.14 The method of the illustrated embodiment 501 is consistent and will not be described again here.

[0236] 702: It is detected that the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0237] In the embodiment of the present application, 702 and Fig.14 The method of the illustrated embodiment 502 is consistent and will not be described again here.

[0238] 703: Determine that a first value of the first rendering frame rate of the first application and a second value of the second rendering frame rate of the second application are both smaller than a third value of the composite frame rate of the terminal device, the first value is equal to the second value, a first display area of ​​the interface of the first application on the display screen is larger than a first threshold, and a distance between the interface of the first application and the center of the display screen is smaller than a second threshold, and determine that the first application is the main application.

[0239] In the embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60, that is, the first value and the second value are both smaller than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely staggered. Since the first value is equal to the second value, the first display area of ​​the interface of the first application on the display screen is greater than the first threshold value, and the distance between the interface of the first application and the center of the display screen is less than the second threshold value, the first application is determined to be the main application.

[0240] In some embodiments, the first application is determined to be the main application if the first display area of ​​the first application's interface on the display screen is greater than a first threshold, the distance between the first application's interface and the center of the display screen is less than a second threshold, and the first application's interface is on an upper layer of the second application's interface.

[0241] In some embodiments, if the second display area of ​​the second application interface on the display screen is greater than the first threshold and the distance between the first application interface and the center of the display screen is less than the second threshold, the second application is determined to be the main application.

[0242] In some embodiments, if the second display area of ​​the second application's interface on the display screen is greater than the first threshold, the distance between the first application's interface and the center of the display screen is less than the second threshold, and the first application's interface is at the lower layer of the second application's interface, the second application is determined to be the main application.

[0243] It can be understood that an application whose interface has a larger display area on the display screen and whose interface is at a distance from the center of the display screen less than the second threshold is generally more likely to attract the user's visual attention.

[0244] 704: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1th frame interface rendered by the second application by the delay time T.

[0245] In the embodiment of the present application, 704 and Fig.14 The method of the illustrated embodiment 504 is consistent and will not be described again here.

[0246] 705: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0247] In the embodiment of the present application, 705 and Fig.14 The method of the illustrated embodiment 505 is consistent and will not be described again here.

[0248] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0249] Combine the following Fig.17 An image processing method is provided in the embodiment of the present application for introduction. Fig.17 As shown, including: 901: It is detected that the terminal device enters the multi-window, split-screen, picture-in-picture playback, smart capsule playback and other scenarios.

[0250] In an embodiment of the present application, when the terminal device detects that the current scene is a preset scene, for example, a multi-window scene, a split-screen scene, a picture-in-picture playback scene, or a smart capsule playback scene, the rendering frame rate of each application is determined.

[0251] 902: Obtain the rendering frame rate of each application and the synthesis frame rate of the image synthesis module.

[0252] In an embodiment of the present application, the terminal device can obtain the rendering frame rate of all applications (including background applications and non-running applications) included in the terminal device and the synthesis frame rate of the image synthesis module.

[0253] 903: Determine whether the rendering frame rate of each application is lower than the synthesis frame rate of the image synthesis module.

[0254] If the judgment result is no, go to 904: exit control; if the judgment result is yes, go to 905: determine whether the rendering frame rate of the application with the highest rendering frame rate is an integer multiple (including 1) of the rendering frame rates of other applications.

[0255] It can be understood that if the rendering frame rate of each application is lower than the synthesis frame rate of the image synthesis module, it indicates that the Vsync-app signal of at least one application among the multiple applications is not synchronized with the Vsync-app signals of other applications, and the process can be turned to 905: determining whether the rendering frame rates of the application with the highest rendering frame rate are integer multiples (including 1) of the rendering frame rates of the remaining applications to determine the aligned main application; if there is at least one application with a rendering frame rate equal to the synthesis frame rate of the image synthesis module, it indicates that the Vsync-app signals of the multiple applications have been aligned and no re-alignment is required, and the process can be turned to 904: exiting the control.

[0256] 904: Exit control.

[0257] 905: Determine whether the rendering frame rate of the application with the highest rendering frame rate is an integer multiple (including 1) of the rendering frame rates of other applications.

[0258] If the judgment result is yes, go to 906: determine whether there are more than two applications with the same rendering frame rate and the rendering frame rate is the maximum rendering frame rate; if the judgment result is no, go to 904: exit control.

[0259] It can be understood that if the rendering frame rate of the application with the largest rendering frame rate is not an integer multiple of the rendering frame rates of other applications, frame loss and uneven synthetic frame rate will easily occur after the frame rates are aligned. Therefore, in an embodiment of the present application, if it is determined that the rendering frame rate of the application with the largest rendering frame rate is not an integer multiple of the rendering frame rates of other applications, go to 904: exit control.

[0260] 906: Determine whether there are two or more (including two) applications with the same rendering frame rate and the maximum rendering frame rate.

[0261] If the judgment result is yes, that is, there are multiple applications with the highest rendering frame rate, go to 907: the application with the highest rendering frame rate is used as the main application; if the judgment result is no, go to 908: calculate the display area of ​​each application with the highest rendering frame rate on the display screen, and the distance from the center point of the application interface to the center point of the display screen through the four vertex coordinates of each layer of the application, and obtain the application's position on the z-axis.

[0262] 907: Use the application with the highest rendering frame rate as the main application.

[0263] It can be understood that in some embodiments, when the rendering frame rate of an application of a terminal device is greater than a preset frame rate, the application is determined to be a high rendering frame rate application, and the high rendering frame rate application is used as the main application.

[0264] 908: Based on the four vertex coordinates of each layer of the application, calculate the display area of ​​each application with the maximum rendering frame rate on the display screen, the distance from the center point of the application interface to the center point of the display screen, and obtain the z-axis position of the application.

[0265] 909: Based on the z-axis coordinates of each application, check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0266] If the judgment result is yes, go to 910: determine whether the display area of ​​the application is large; if the judgment result is no, go to 909: based on the z-axis coordinates of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0267] In an embodiment of the present application, the layers of each application can be traversed in order from large to small according to the coordinates of the multiple applications with the largest rendering frame rates on the z-axis to determine whether the display area of ​​the application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold. If the display area of ​​the application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold, go to 910: determine whether the display area of ​​the application is larger. If the judgment result is no, determine whether the display area of ​​the next application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold.

[0268] It can be understood that in the display interface of the terminal device, the layers of the upper-level applications generally cover the layers of the lower-level applications, and in the preset scenario, the first application will exist in the foreground main interface of the terminal device. Therefore, according to the z-axis coordinates of multiple applications with the largest rendering frame rate, the rendered layers of each application are traversed from large to small, which can ensure that the main application can be traversed.

[0269] 910: Determine whether the display area of ​​the application is large.

[0270] If the judgment result is yes, go to 911: calculate the delay time T for aligning the non-main application with the main application one by one, and adopt methods such as short freezing time T (the related application has no user-perceptible services such as audio), Vsync-app signal processing delay T of the non-main application, reading the layer delay T time stored in the graphics buffer of the non-main application and rendered by the application, so that the image synthesis module can synthesize and align the non-main application, and send the non-main application to the display and synthesize it at the same time when the image synthesis module synthesizes the main application; if the judgment result is no, go to 909: based on the z-axis coordinates of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0271] In an embodiment of the present application, it can be determined whether the display area of ​​the application on the display screen is greater than a first threshold value. If the display area of ​​the application on the display screen is greater than the first threshold value, go to 911: calculate the delay time T for aligning the non-main application with the main application one by one, and adopt methods such as a short freezing time of T (the related application has no user-perceivable services such as audio), a Vsync-app signal processing delay T of the non-main application, and a layer delay T time for reading the non-main application stored in the graphics buffer and rendered by the application, so that the image synthesis module synthesizes and aligns the non-main application, and sends the non-main application to the display for synthesis at the same time when the image synthesis module synthesizes the main application; if the display area of ​​the application on the display screen is less than or equal to the first threshold value, go to 909: based on the z-axis coordinates of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0272] 911: Calculate the delay time T for aligning the non-main application with the main application one by one, and based on the delay time T, enable the image synthesis module to synthesize and align the Vsync-app signals of the main application and the non-main application.

[0273] In an embodiment of the present application, the delay time T of applications other than the first application in the application of the terminal device can be calculated, and the Vaync-app signal can be delayed according to the delay time of each application other than the first application in the application of the terminal device, so that the Vaync-app signal of the other applications other than the first application in the application of the terminal device is aligned with the Vaync-app signal of the first application, thereby reducing the synthesis frame rate of the image synthesis module.

[0274] In an embodiment of the present application, according to the application of the terminal device, the method of delaying the Vaync-app signal of each application except the first application includes: for applications that do not involve business functions that can be directly perceived by the user, the Vaync-app signal of the next frame layer of the application can be frozen for a delay time T; or, the Vaync-app signal of the next frame layer of the application is delayed for a delay time T; the image synthesis module delays the delay time T to read the layer stored in the graphics buffer and rendered by the application. Among them, the business functions that can be directly perceived by the user include audio playback function, video playback function, tactile feedback function, visual effect function, etc.

[0275] In some embodiments, the delay time T = the moment when the first application draws the layer of the current frame interface - the moment when the second application draws the layer of the current frame interface + the first time interval corresponding to the first rendering frame rate of the first application - the second time interval corresponding to the second rendering frame rate of the second application.

[0276] In some embodiments, the delay time T=k times of the second time interval, 0<k<1.

[0277] 912: The image synthesis module synthesizes at least one application rendering layer at the same time point.

[0278] In an embodiment of the present application, the image synthesis module can synthesize at least one application-rendered layer at the same time point.

[0279] 913: Set the LCD hardware screen refresh rate to the synthesis frame rate of the image synthesis module, or a gear corresponding to the synthesis frame rate of the image synthesis module.

[0280] 914: Determine whether a rendering frame rate of an application in the terminal device has changed.

[0281] If the judgment result is yes, go to 904: exit control. If the judgment result is no, go to 915: maintain frame rate alignment and LCD refresh rate configuration.

[0282] In some embodiments, if the rendering frame rate of an application in the terminal device changes, a new judgment is required, and the process may go to 902: check the rendering frame rate of each application and obtain the synthesis frame rate of the image synthesis module.

[0283] 915: Maintain frame rate alignment and LCD refresh rate configuration.

[0284] In an embodiment of the present application, layers can be rendered according to the rendering frame rate of each application and the refresh rate configuration of the LCD can be maintained. For example, the LCD hardware screen refresh rate is set to the synthesis frame rate of the image synthesis module, or the gear corresponding to the synthesis frame rate of the image synthesis module.

[0285] In the embodiment of the present application, the Vsync-app signals of each application have been aligned. After alignment, the layers are rendered according to the rendering frame rates of each application. Even if at least two applications in the terminal device update the rendering content, the number of times the image synthesis module performs image synthesis can be reduced, and the synthesis frame rate and power consumption can be reduced. In addition, setting the refresh rate of the LCD to the synthesis frame rate of the image synthesis module, or to the refresh rate of the gear corresponding to the synthesis frame rate of the image synthesis module, can optimize power consumption.

[0286] It can be understood that the image processing method provided in the embodiment of the present application is also applicable to multiple application scenarios. Fig.18a As shown, the applications of the terminal device include a first application, a second application and a third application, and the second rendering frame rate of the second application is greater than the first rendering frame rate of the first application and the third rendering frame rate of the third application. Then, the second application can be used as the main application, and the Vsync-app signal of the first application, the third Vsync-app signal of the third application and the Vsync-app signal of the second application can be aligned.

[0287] like Fig.18aAs shown, the second rendering frame rate is an integer multiple of the first rendering frame rate and the third rendering frame rate, so after the Vsync-app signal of the first application, the third Vsync-app signal of the third application and the Vsync-app signal of the second application are aligned, the first application, the second application, and the third application do not lose frames. Fig.18b It can be found that aligning the Vsync-app signal of the first application, the third Vsync-app signal of the third application, and the Vsync-app signal of the second application can significantly reduce the synthesis frame rate.

[0288] The following is an introduction to the reasons for choosing the application with the highest rendering frame rate as the main application.

[0289] It can be understood that if an application with a lower rendering frame rate is aligned with an application with a higher rendering frame rate, since the timestamp between the two frame layers rendered by the application with a higher rendering frame rate is shorter, or the interval between two adjacent frames of the application with a higher rendering frame rate is shorter, the delay time T required for the application with a lower rendering frame rate during the alignment process is shorter.

[0290] It is understandable that the scenes with high rendering frame rates are generally applied because of the customers' clicks, sliding and other operations. These operations generally have transition effects, and the transition effects generally require high refresh rate to ensure smooth effects and enhance the smoothness of customers' interactive experience.

[0291] For example, Fig.19 As shown, the first rendering frame rate of the first application is greater than the second rendering frame rate of the second application. In the process of aligning the Vsync-app signal of the second application with the Vsync-app signal of the first application, the required delay time T is less than the first time interval corresponding to the first rendering frame rate of the first application.

[0292] It can be understood that if an application with a higher rendering frame rate is aligned with an application with a lower rendering frame rate, since the timestamp between the two frames of the application with a lower rendering frame rate is longer, or the interval between the two adjacent frames of the application with a lower rendering frame rate is longer, the delay time T required for the application with a higher rendering frame rate during the alignment process is longer, and after the alignment time point, the subsequent frame of the application with a high rendering frame rate is still refreshed normally, and the subsequent frames will be aligned again. Therefore, the alignment action of the application with a higher rendering frame rate to the application with a lower rendering frame rate needs to be performed each time, which is more cumbersome to handle than aligning the application with a lower rendering frame rate to the application with a higher rendering frame rate. In addition, if the application with a higher rendering frame rate is directly aligned with the application with a lower rendering frame rate, the time for synthesizing the first frame after the alignment of the application with a high rendering frame rate will exceed the interval between the two frames corresponding to the high rendering frame rate, resulting in a large difference between the refresh time of the first frame aligned and the previous frame time, poor user experience, and one more frame stored in the graphics buffer, which is likely to cause other problems that affect the customer experience, such as frame loss in the first application, blocking of the graphics buffer, and slow release.

[0293] For example, Fig. 20 As shown in (a), the first rendering frame rate of the first application is lower than the second rendering frame rate of the second application. In the process of aligning the Vsync-app signal of the first application with the Vsync-app signal of the second application, the required delay time T is greater than the second time interval corresponding to the second rendering frame rate of the second application. At this time, it is easy to cause frame loss and graphics buffer blocking in the first application, and the graphics buffer will store one more frame.

[0294] In the embodiment of the present application, if the application with a higher rendering frame rate is aligned with the application with a lower rendering frame rate, such as Fig. 20 As shown in (b), the high rendering frame rate application does not align the first frame after the alignment starts, but aligns the low rendering frame rate application from the next frame after the alignment moment.

[0295] Combine the following Fig.21 The structure of the electronic device is described. Fig.21 As shown, the software layer of the electronic device includes the application (APP) layer, the application framework layer (also called the native framework Native layer), the hardware abstraction layer (HAL), and the kernel layer (also called the driver layer); the hardware layer includes the display screen (panel).

[0296] The application layer may include various application programs running in electronic devices, such as conference applications, video applications, and communication applications, etc. Applications in the application layer may apply for Vsync-app signals from the image display driver and draw layers based on the Vsync-app signals.

[0297] The application framework layer mainly involves image processing modules, such as: a graphics processing unit (GPU) for rendering the image to be displayed given by the application, and a surface flinger (SF) for synthesizing multiple layers of the image to be displayed. The kernel layer mainly includes the image display driver, which includes the display engine (software development engineer, SDE) and the display module serial interface (display serial interface, DSI); the image display driver is mainly used to send the image to be displayed processed and sent by the upper layer of the system (that is, other system layers above the kernel layer where the image display driver is located, such as the application layer, application framework layer, and hardware abstraction layer) to the display screen. The hardware layer mainly includes the display screen that refreshes the image to be displayed based on the call of the image display driver.

[0298] In such Fig.21 In the system framework of the electronic device 10 shown in FIG. 1 , when an APP such as a conference application, a video application or a communication application at the application layer needs to refresh an image to be displayed, such as Fig. 22As shown, the conference application, video application or communication application in the application layer can apply for the Vsync-app signal from the image display driver. The VsyncDispatchTimerQueue process in the framework layer calculates the distribution time of the Vsync-app signal. When the distribution time is reached, the driver is triggered to send the Vsync-app signal to the APP. After receiving the Vsync-app signal, the APP starts drawing a frame of image. After the APP draws a frame of image, it places the drawn image in the graphics buffer corresponding to the APP, notifies the GPU in the application framework layer to render the drawn image, notifies the image synthesis module to start synthesizing the frame of image, and continues to apply for the Vsync-app signal from the driver. The VsyncDispatchTimerQueue process in the framework layer calculates the next distribution time of the Vsync-app signal. The image synthesis module applies for the Vsync-sf signal from the driver. The VsyncDispatchTimerQueue process in the framework layer calculates the distribution time of the Vsync-sf signal. When the distribution time is reached, the driver is triggered to send the Vsync-sf signal to the APP. After receiving the Vsync-sf signal, the image synthesis module performs layer synthesis on the rendered image to obtain the display data of the image to be displayed. The image synthesis module then passes the display data of the image to be displayed to the image display driver of the kernel layer through the hardware abstraction layer. The image display driver responds to the hardware VSync signal to call DSI through SDE, and writes the display data of the image to be displayed to the image display driver chip (display driver integrated circuit, DDIC) of the display screen. The DDIC of the display screen can store the display data sent by the software layer in the buffer, thereby controlling the display panel to complete the refresh display of the image to be displayed by scanning (or reading) the display data in the buffer.

[0299] It can be understood that the APP can send rendering commands to the GPU through the graphics application programming interface (API). After receiving the rendering command from the APP application, the GPU reads the image data drawn by the APP and renders the image data. After the GPU completes the rendering, it puts the rendering result into the buffer corresponding to the APP.

[0300] Combine the following Fig.23a and Figure 23b , the image processing method provided by this application is introduced. Fig.23a and Figure 23b As shown, the image processing method includes: 801: The first application requests a Vsync-app signal from the image display driver.

[0301] In an embodiment of the present application, when a user opens a first application of a terminal device, or when the first application needs to draw a layer of the i-th frame interface, the first application can apply for a Vsync-app signal from the image display driver. The first application includes a system application and a third-party application. For example, a system application can be a setting application, a camera, a calendar, a phone, or a message, and a third-party application can be an instant conference application, a video application, a communication application, etc.

[0302] 802: The image display driver sends a Vsync-app signal to the first application.

[0303] 803: The first application renders a layer of the i-th frame interface of the first application based on the Vsync-app signal.

[0304] In the embodiment of the present application, when the first application obtains the Vsync-app signal sent by the driver, the layer of the i-th frame interface can be drawn and rendered at the first moment. After the first application renders the layer of the i-th frame interface, the layer of the i-th frame interface is stored in the graphics buffer, the image synthesis module is notified to synthesize the layer of the i-th frame interface, and the Vsync-app signal is continued to be applied to the image display driver.

[0305] 804: The first application sends the layer of the i-th frame interface of the first application to the image synthesis module.

[0306] 805: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0307] 806: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0308] 807: The image synthesis module synthesizes the layers of the i-th frame interface of the first application based on the Vsync-sf signal to obtain a first image frame.

[0309] 808: The image synthesis module sends the first image frame to the display screen.

[0310] 809: The display screen displays the first image frame.

[0311] 810: The second application requests a Vsync-app signal from the image display driver.

[0312] In the embodiment of the present application, when the user opens the second application of the terminal device, or when the second application needs to draw the layer of the j-th frame interface, the second application can apply for a Vsync-app signal to the image display driver. The second application includes system applications and third-party applications. For example, the system application can be a setting application, camera, calendar, phone or information, etc., and the third-party application can be a conference application, video application, communication application, etc.

[0313] 811: The image display driver sends a Vsync-app signal to the second application.

[0314] 812: The second application renders a layer of the j-th frame interface of the second application based on the Vsync-app signal.

[0315] In the embodiment of the present application, the second application can render the layer of the j-th frame interface of the second application based on the Vsync-app signal at the second moment. After the second application renders the layer of the j-th frame interface, the layer of the j-th frame interface is stored in the graphics buffer, the image synthesis module is notified to synthesize the layer of the j-th frame interface, and the Vsync-app signal is continued to be applied to the image display driver.

[0316] In an embodiment of the present application, the first moment is earlier than the second moment, and in other embodiments, the first moment is later than the second moment.

[0317] 813: The second application sends the layer of the j-th frame interface of the second application to the image synthesis module.

[0318] 814: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0319] 815: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0320] 816: The image synthesis module synthesizes the layer of the i-th frame interface of the first application and the layer of the j-th frame interface of the second application based on the Vsync-sf signal to obtain a second image frame.

[0321] 817: The image synthesis module sends the second image frame to the display screen.

[0322] 818: The display screen displays the second image frame.

[0323] 819: The first application requests a Vsync-app signal from the image display driver.

[0324] In the embodiment of the present application, the method of 819 is the same as that of 801 and will not be repeated here.

[0325] 820: The second application requests a Vsync-app signal from the image display driver.

[0326] In the embodiment of the present application, steps 820 and 810 are the same and will not be repeated here.

[0327] 821: The image display driver sends a Vsync-app signal to the first application.

[0328] 822: The image display driver sends a Vsync-app signal to the second application.

[0329] 823: The first application renders a layer of the (i+1)th frame interface of the first application based on the Vsync-app signal.

[0330] 824: The second application renders a layer of the j+1th frame interface of the second application based on the Vsync-app signal.

[0331] In an embodiment of the present application, at the third moment, the first application renders the layer of the i+1th frame interface of the first application based on the Vsync-app signal, and the second application renders the layer of the j+1th frame interface of the second application based on the Vsync-app signal.

[0332] It can be understood that at the third moment, the first application renders the layer of the i+1th frame interface of the first application based on the Vsync-app signal, and the second application renders the layer of the j+1th frame interface of the second application based on the Vsync-app signal. Figure 6a The embodiments shown will not be described in detail here.

[0333] 825: The first application sends the layer of the i+1th frame interface of the first application to the image synthesis module. 826: The second application sends the layer of the j+1th frame interface of the second application to the image synthesis module.

[0334] 827: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0335] 828: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0336] 829: The image synthesis module synthesizes the layer of the i+1th frame interface of the first application based on the Vsync-sf signal, and the second application sends the layer of the j+1th frame interface of the second application to the image synthesis module to obtain a third image frame.

[0337] 830: The image synthesis module sends the third image frame to the display screen.

[0338] 831: The display screen displays the third image frame.

[0339] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of synthesis of the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0340] An embodiment of the present application also provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing any image processing method provided by the various possible implementations described above.

[0341] An embodiment of the present application provides a readable medium on which instructions are stored. When the instructions are executed on an electronic device, the electronic device implements any one of the image processing methods provided by the various possible implementations described above.

[0342] An embodiment of the present application provides a computer program product, which includes computer instructions. When executed by an electronic device, the electronic device implements any one of the image processing methods provided by the various possible implementations described above.

[0343] The hardware structure of each electronic device mentioned in this application is described below by taking the electronic device 10 as an example. Fig.24 As shown, the electronic device 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101 and a display screen 102, etc.

[0344] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0345] The processor 110 may include one or more processing units, for example, a processing module or processing circuit including a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), a micro-programmed control unit (MCU), an artificial intelligence (AI) processor, or a programmable logic device (field programmable gate array, FPGA). Different processing units may be independent devices or integrated into one or more processors. A storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180.

[0346] It can be understood that the image processing method in the embodiment of the present application can be executed by the processor 110 of the corresponding electronic device. The power module 140 may include a power supply, a power management component, etc. The power supply may be a battery. The power management component is used to manage the charging of the power supply and the power supply of the power supply to other modules. In some embodiments, the power management component includes a charging management module and a power management module. The charging management module is used to receive charging input from the charger; the power management module is used to connect the power supply, the charging management module and the processor 110. The power management module receives input from the power supply and / or the charging management module to power the processor 110, the display screen 102, the camera 170, and the wireless communication module 120.

[0347] The mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), etc. The mobile communication module 130 may provide a solution for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 10. The mobile communication module 130 may receive electromagnetic waves through an antenna, filter, amplify, and process the received electromagnetic waves, and transmit them to a modulation and demodulation processor for demodulation. The mobile communication module 130 may also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through an antenna. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be disposed in the same device as at least some of the modules of the processor 110.

[0348] The wireless communication module 120 may include an antenna, and transmit and receive electromagnetic waves via the antenna. The wireless communication module 120 may provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied to the electronic device 10. The electronic device 10 may communicate with the network and other devices through wireless communication technology.

[0349] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the electronic device 10 may also be located in the same module.

[0350] The display screen 102 is used to display a human-computer interaction interface, images, videos, etc. The display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot light emitting diode (QLED), etc.

[0351] The sensor module 190 may include a proximity light sensor, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0352] The audio module 150 is used to convert digital audio information into an analog audio signal output, or to convert analog audio input into a digital audio signal. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be set in the processor 110, or some functional modules of the audio module 150 can be set in the processor 110. In some embodiments, the audio module 150 may include a speaker, an earpiece, a microphone, and an earphone interface. The camera 170 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the image signal processing (ISP) to convert it into a digital image signal. The electronic device 10 can realize the shooting function through the ISP, the camera 170, the video codec, the graphic processing unit (GPU), the display screen 102, and the application processor.

[0353] The interface module 160 includes an external memory interface, a USB interface, and a subscriber identification module (SIM) card interface. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 10. The external memory card communicates with the processor 110 through the external memory interface to implement a data storage function. The universal serial bus interface is used for the electronic device 10 to communicate with other electronic devices. The subscriber identification module card interface is used to communicate with a SIM card installed in the electronic device 10, for example, to read a phone number stored in the SIM card, or to write a phone number into the SIM card.

[0354] In some embodiments, the electronic device 10 further includes a button 101, a motor, an indicator, etc. Among them, the button 101 may include a volume button, an on / off button, etc. The motor is used to make the electronic device 10 vibrate, for example, vibrate when the user's electronic device 10 is called, so as to prompt the user to answer the call of the electronic device 10. The indicator may include a laser indicator, a radio frequency indicator, an LED indicator, etc.

[0355] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0356] Program code may be applied to input instructions to perform the functions described herein and to generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0357] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0358] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to floppy disks, optical disks, optical disks, read-only memories (compact disc-read only memories, CD-ROMs), magneto-optical disks, read-only memories (read only memories, ROMs), random access memories (random access memories, RAMs), erasable programmable read-only memories (erasable programmable read-only memories, EPROMs), electrically erasable programmable read-only memories (electrically erasable programmable read-only memories, EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0359] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0360] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0361] It should be noted that in the examples and description of this patent, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises one" does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0362] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. An image processing method, characterized in that: Applied to a terminal device, the image processing method comprises: Running a first application and a second application, wherein the first application renders a layer based on a first rendering frame rate, and the second application renders a layer based on a second rendering frame rate; The first application renders a layer of an i-th frame interface of the first application at a first moment, and the second application renders a layer of a j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers; The first application renders a layer of an i+1th frame interface of the first application at a third moment, and the second application renders a layer of a j+1th frame interface of the second application at the third moment, wherein a first time difference between the third moment and the first moment is different from a first time interval corresponding to the first rendering frame rate, or a second time difference between the third moment and the second moment is different from a second time interval corresponding to the second rendering frame rate; After the third moment, the first application renders the layer based on the first rendering frame rate, and the second application renders the layer based on the second rendering frame rate.

2. The image processing method according to claim 1, characterized in that: Before the third moment, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than a third value of the synthesis frame rate of the terminal device.

3. The image processing method according to claim 1, characterized in that: When the first condition is met, the first time difference is equal to the first time interval, and the second time difference is greater than the second time interval; wherein the first condition includes any one of the following conditions: A first value of the first rendering frame rate is greater than a second value of the second rendering frame rate; The first value is equal to the second value, and a first display area of ​​the interface of the first application on the display screen is larger than a second display area of ​​the interface of the second application on the display screen; The first value is equal to the second value, the first display area is greater than a first threshold, and the distance between the interface of the first application and the center of the display screen is less than a second threshold; The first value is equal to the second value, the first display area is greater than a first threshold, and the display area of ​​the first application interface on the display screen includes the center of the display screen; The first value is equal to the second value, the first display area is greater than a first threshold, the distance between the interface of the first application and the center of the display screen is less than a second threshold, and the interface of the first application is on an upper layer of the interface of the second application; The first value is equal to the second value, the first display area is greater than a first threshold, the interface of the first application in the display area of ​​the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application.

4. The image processing method according to claim 3, characterized in that: When the first condition is met, the i-th frame interface is the last frame interface synthesized by the first application before a fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

5. The image processing method according to claim 3 or 4, characterized in that: The first application renders a layer of an i+1th frame interface of the first application at a third moment, and the second application renders a layer of a j+1th frame interface of the second application at the third moment, including: When at least one of the following conditions is met, the first application renders a layer of the i+1th frame interface of the first application at a third moment, and the second application renders a layer of the j+1th frame interface of the second application at the third moment: The first value is an integer multiple of the second value; A ratio of the first value to the second value is greater than or equal to a first ratio.

6. The image processing method according to claim 3 or 4, characterized in that: The second time difference is the sum of the second time interval and the delay time; The delay time=the first moment-the second moment+the first time interval-the second time interval.

7. The image processing method according to claim 6, characterized in that: The second application renders a layer of the j+1th frame interface of the second application at a third moment, including: After the second moment, delaying the Vsync-app signal of the second application by the delay time; Render the layer of the j+1th frame interface of the second application based on the delayed Vsync-app signal.

8. The image processing method according to claim 1, characterized in that: When the second condition is met, the second time difference is equal to the second time interval, and the first time difference is greater than the first time interval, wherein the second condition includes any one of the following conditions: A second value of the second rendering frame rate is greater than a first value of the first rendering frame rate; The second value is equal to the first value, and a second display area of ​​the interface of the second application on the display screen is larger than a first display area of ​​the interface of the first application on the display screen; The second value is equal to the first value, the second display area is greater than a first threshold, and the distance between the interface of the second application and the center of the display screen is less than a second threshold; The second value is equal to the first value, the second display area is greater than a first threshold, and the interface of the second application in the display area of ​​the display screen includes the center of the display screen; The second value is equal to the first value, the second display area is greater than a first threshold, the distance between the interface of the second application and the center of the display screen is less than a second threshold, and the interface of the second application is on an upper layer of the interface of the first application; The second value is equal to the first value, the second display area is greater than a first threshold, the interface of the second application includes the center of the display screen in the display area of ​​the display screen, and the interface of the second application is on an upper layer of the interface of the first application.

9. The image processing method according to claim 8, characterized in that: The j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

10. The image processing method according to claim 1, characterized in that: The method comprises: After the third moment, the refresh rate of the display screen of the terminal device is adjusted based on the synthetic frame rate of the terminal device.

11. An electronic device, characterized in that: The device comprises: a memory for storing instructions executed by one or more processors of the electronic device, and the processor, which is one of the one or more processors of the electronic device, for executing the image processing method according to any one of claims 1 to 10.

12. A readable medium, characterized in that The readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the image processing method according to any one of claims 1 to 10.

13. A computer program product, characterized in that The computer program product comprises computer instructions, which, when executed by an electronic device, execute the image processing method according to any one of claims 1 to 10.

Images