Image processing method, electronic device, readable medium and program product

Abstract

The present application relates to the field of smart terminal technology, and discloses an image processing method, an electronic device, a readable medium, and a program product. In the case where the image frame to be generated includes the display content of multiple applications, the image processing method of the present application can determine the first application with the largest rendering frame rate among the multiple applications, and adjust the Vsync‑app signals of the other applications except the first application in the multiple applications to be synchronized with the Vsync‑app signal of the first application. Then, the terminal device can draw, render, and synthesize image frames based on the adjusted Vsync‑app signal. Thus, even if at least two applications among the multiple applications of the terminal device update the rendering content, the image synthesis module only needs to perform image synthesis once, which reduces the number of synthesis times of the image synthesis module, reduces the synthesis frame rate, and greatly reduces power consumption.

Description

Technical Field

[0001] The present application relates to the field of intelligent terminal technology, and in particular to an image processing method, electronic device, readable medium, and program product. Background Art

[0002] When generating a display interface, a terminal device typically draws, renders, and synthesizes images based on the vertical synchronization (Vsync) signal. Different applications can draw and render their own layers based on the Vsync signal. After rendering is complete, the terminal device's operating system's image synthesis module (such as the surface compositing module, SF) is triggered to synthesize the rendered layers into an image frame for the display.

[0003] When the interface displayed by a terminal device includes multiple application layers, after each application completes rendering the layers it needs to display, the image compositing module will request a software Vsync signal to trigger image frame synthesis. If the software Vsync signals (hereinafter referred to as Vsync-sf signals) used by these multiple applications to trigger image frame synthesis by the image synthesis module are not synchronized, the frequency of image frame synthesis by the image synthesis module will increase, increasing the power consumption of the terminal device. For example, if the image frames displayed by the terminal device at a 120Hz refresh rate include the images of application A and application B, and the frame rate of application A and application B updating the layers is 60 frames per second (FPS), the period of the Vsync-sf signals for application A and application B is 16.66 milliseconds (ms). If the start time of the Vsync-sf signal of application A and application B differs by 10ms, each Vsync-sf signal of application A and application B will trigger the image synthesis module to synthesize image frames, and the image synthesis module will synthesize image frames at a frequency of 60fps + 60fps = 120fps. Summary of the Invention

[0004] In order to solve the problem that when the Vsync-sf signals used by multiple applications to trigger the image synthesis module to synthesize image frames are not synchronized, the frequency of the image synthesis module synthesizing image frames will increase, and the power consumption of the terminal device will increase, the embodiments of the present application provide an image processing method, electronic device, readable medium and program product.

[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to a terminal device, and the image processing method includes: running a first application and a second application, wherein the first application renders a layer based on a first rendering frame rate, and the second application renders a layer based on a second rendering frame rate; the first application renders a layer of the i-th frame interface of the first application at a first moment, and the second application renders a layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers; the first application renders a layer of the i+1-th frame interface of the first application at a third moment, and the second application renders a layer of the j+1-th frame interface of the second application at the third moment, wherein the first time difference between the third moment and the first moment is different from the first time interval corresponding to the first rendering frame rate, or the second time difference between the third moment and the second moment is different from the second time interval corresponding to the second rendering frame rate; after the third moment, the first application renders the layer based on the first rendering frame rate, and the second application renders the layer based on the second rendering frame rate.

[0006] In some embodiments, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment.

[0007] In an embodiment of the present application, after the third moment, the time interval between the layers of adjacent frame interfaces drawn by the first application is the first time interval corresponding to the first rendering frame rate. For example, the time interval between the layer of the i+1 frame interface and the layer of the i+2 frame interface drawn by the first application is the first time interval, and the time interval between the layer of the i+2 frame interface and the layer of the i+3 frame interface is the first time interval.

[0008] After the third moment, the time interval between the layers of adjacent frame interfaces drawn by the second application is the second time interval corresponding to the second rendering frame rate. For example, the time interval between the layer of the j+1 frame interface and the layer of the j+2 frame interface drawn by the second application is the second time interval, and the time interval between the layer of the j+2 frame interface and the layer of the j+3 frame interface is the second time interval.

[0009] In an embodiment of the present application, after the third moment, the first application renders the layer based on the first rendering frame rate, and the second application renders the layer based on the second rendering frame rate. That is, after the third moment, the moment when the first application renders the layer based on the first rendering frame rate and the moment when the second application renders the layer based on the second rendering frame rate overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the first rendering frame rate and the layer rendered by the second application based on the second rendering frame rate are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0010] In a possible implementation, before the third moment, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than a third value of the composite frame rate of the terminal device.

[0011] It can be understood that when the moment when the first application renders the layer based on the first rendering frame rate and the moment when the second application renders the layer based on the second rendering frame rate are completely staggered, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than the third value of the composite frame rate of the terminal device.

[0012] In one possible implementation, when the first condition is met, the first time difference is equal to the first time interval, and the second time difference is greater than the second time interval; wherein the first condition includes any one of the following conditions: the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate; the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is greater than the second display area of ​​the interface of the second application on the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, and the distance between the interface of the first application and the center of the display screen is less than the second threshold; the first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, the distance between the interface of the first application and the center of the display screen is less than the second threshold, and the interface of the first application is on the upper layer of the interface of the second application; the first value is equal to the second value, the first display area is greater than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on the upper layer of the interface of the second application.

[0013] It is understood that the distance between the first application's interface and the center of the display screen can be the distance between the center of the first application's display area on the display screen and the center of the display screen. The center of the display screen refers to the geometric center of the display screen, that is, the midpoint of the display screen in the horizontal and vertical directions. The center of the first application's interface is the geometric center of the first application's interface, that is, the midpoint of the first application's display area on the display screen in the horizontal and vertical directions.

[0014] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can preserve the visual advantages of the high rendering frame rate.

[0015] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0016] Therefore, when the first value is equal to the second value, the second application can be aligned with the first application when it is determined that the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen; or, the first display area is larger than the first threshold and the distance between the interface of the first application and the center of the display screen is smaller than the second threshold; or, the first display area is larger than the first threshold and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; or, the first display area is larger than the first threshold, the distance between the interface of the first application and the center of the display screen is smaller than the second threshold, and the interface of the first application is on the upper layer of the interface of the second application; or, the first display area is larger than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on the upper layer of the interface of the second application.

[0017] In a possible implementation, when the first condition is met, the i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0018] In one possible implementation, the first application renders the layer of the i+1th frame interface of the first application at the third moment, and the second application renders the layer of the j+1th frame interface of the second application at the third moment, including: the first application renders the layer of the i+1th frame interface of the first application at the third moment, and the second application renders the layer of the j+1th frame interface of the second application at the third moment when at least one of the following conditions is met: the first value is an integer multiple of the second value; the ratio of the first value to the second value is greater than or equal to the first ratio.

[0019] It can be understood that the first ratio can be a positive number greater than 1, for example, the first ratio=1.5.

[0020] In a possible implementation, the second time difference is the sum of the second time interval and the delay time; delay time=first moment-second moment+first time interval-second time interval.

[0021] In some embodiments, the delay time=k times the second time interval, 0<k<1.

[0022] In one possible implementation, the second application renders the layer of the j+1th frame interface of the second application at the third moment, including: after the second moment, delaying the Vsync-app signal of the second application by a delay time; and rendering the layer of the j+1th frame interface of the second application based on the delayed Vsync-app signal.

[0023] It can be understood that when the first value is an integer multiple of the second value (or first value / second value=N, where N is a positive integer), the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole in the time domain.

[0024] For example, the first value is 60 and the second value is 30. The Vsyns-app signal of the second application can be delayed as a whole in the time domain so that the delayed Vsyns-app signal of the second application is aligned with the Vsyns-app signal of the first application.

[0025] It can be understood that if the first frame rate is an integer multiple of the second frame rate, if the Vsync-app signals of the first and second applications are out of sync, the composite frame rate will be the first value plus the second value. After the Vsync-app signals of the first and second applications are synchronized, the composite frame rate will be the first value. This can reduce the number of composites required for the first application to render a layer at the first rendering frame rate and the second application to render a layer at the second rendering frame rate, thereby reducing the composite frame rate and power consumption.

[0026] When both the first value and the second value are greater than or equal to a preset threshold and the first value is not an integer multiple of the second value (or when first value / second value ≠ N), the Vsync-app signal of the second application can be partially aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole in the time domain.

[0027] For example, the preset threshold is 60, the first ratio is 1.5, the first value is 90, the second value is 60, and the first value / second value = 1.5. The Vsync-app signal of the second application can be delayed as a whole in the time domain, so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0028] It can be understood that if the rendering frame rate of the application is greater than or equal to a preset threshold, for example, the rendering frame rate of the application is greater than or equal to 60fps, it is determined that the current scene is a high refresh rate scene.

[0029] When the rendering frame rates of the first and second applications are both greater than or equal to a preset threshold, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, even if the second application loses a frame or has a certain delay, the user will not notice it. Therefore, in some embodiments, when the rendering frame rates of the first and second applications are both greater than or equal to the preset threshold, even if the rendering frame rate of the first application is not an integer multiple of the rendering frame rate of the second application, the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the applications in the time domain by a delay time T.

[0030] It can be understood that if both the first value and the second value are greater than or equal to the preset threshold, and the first value is not an integer multiple of the second value (or if first value / second value ≠ N), after the Vsync-app signals of the first and second applications are synchronized, because the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, the composite frame rate after alignment will be lower than the composite frame rate before alignment. This allows the image synthesis module to reduce the number of synthesis times, lowering the composite frame rate and power consumption.

[0031] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold, and the first value is not an integer multiple of the second value (or when the first value / the second value ≠ N), after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0032] In one possible implementation, when the second condition is met, the second time difference is equal to the second time interval, and the first time difference is greater than the first time interval, wherein the second condition includes any one of the following conditions: the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate; the second value is equal to the first value, and the second display area of ​​the interface of the second application on the display screen is greater than the first display area of ​​the interface of the first application on the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold; the second value is equal to the first value, the second display area is greater than the first threshold, and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, the distance between the interface of the second application and the center of the display screen is less than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; the second value is equal to the first value, the second display area is greater than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

[0033] It is understood that the distance between the second application's interface and the center of the display screen can be the distance between the center of the second application's display area on the display screen and the center of the display screen. The center of the second application's display area on the display screen refers to the geometric center of the second application's display area on the display screen, that is, the midpoint of the second application's display area on the display screen in the horizontal and vertical directions.

[0034] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate, aligning the first application with the second application can preserve the visual advantages of the high rendering frame rate.

[0035] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0036] Therefore, when the second value is equal to the first value, the first application and the second application can be aligned when it is determined that the second display area of ​​the interface of the second application on the display screen is larger than the first display area of ​​the interface of the first application on the display screen; or, the second display area is larger than the second threshold and the distance between the interface of the second application and the center of the display screen is smaller than the second threshold; or, the second display area is larger than the first threshold and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; or, the second display area is larger than the first threshold, the distance between the interface of the second application and the center of the display screen is smaller than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; or, the second display area is larger than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

[0037] In a possible implementation, the j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0038] In an embodiment of the present application, corresponding to the scenario where the first application is aligned with the second application, that is, when the second condition is met, the j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0039] In a possible implementation, after the third moment, the refresh rate of the display screen of the terminal device is adjusted based on the synthesized frame rate of the terminal device.

[0040] In some embodiments, after the third moment, the refresh rate of the terminal device display screen can be adjusted based on the frame rate of the gear corresponding to the synthetic frame rate of the terminal device.

[0041] In some embodiments, after the third moment, when it is determined that the change in the synthetic frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen can be adjusted based on the synthetic frame rate of the terminal device.

[0042] In some embodiments, after the third moment, when it is determined that the change in the composite frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen can be adjusted based on the frame rate of the corresponding gear of the composite frame rate of the terminal device.

[0043] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing any image processing method provided by the above-mentioned first aspect and various possible implementations of the above-mentioned first aspect.

[0044] In a third aspect, an embodiment of the present application provides a readable medium having instructions stored thereon. When the instructions are executed on an electronic device, the electronic device implements any image processing method provided by the first aspect and various possible implementations of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When executed by an electronic device, the electronic device implements any image processing method provided by the first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 According to an embodiment of the present application, a schematic diagram of a method for determining a hardware Vsync period is shown;

[0047] Figure 2 According to an embodiment of the present application, a schematic diagram of generating a software Vsync signal based on a hardware Vsync signal is shown;

[0048] Figure 3 A schematic flow chart of an image processing method is shown;

[0049] Figure 4 According to an embodiment of the present application, a first schematic diagram of a terminal device performing layer rendering, synthesis, and image frame display is shown;

[0050] Figure 5According to an embodiment of the present application, a second schematic diagram of a terminal device performing layer rendering, synthesis, and image frame display is shown;

[0051] Figure 6a According to an embodiment of the present application, a first flow chart of an image processing method is shown;

[0052] Figure 6b According to an embodiment of the present application, a schematic diagram of a multi-window scenario is shown;

[0053] Figure 6c According to an embodiment of the present application, a schematic diagram of a process for controlling a first application to render a layer of an i+1th frame interface of the first application at a third moment, and a second application to render a layer of a j+1th frame interface of the second application at a third moment is shown;

[0054] Figure 7a According to an embodiment of the present application, a first timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0055] Figure 7b According to an embodiment of the present application, a second timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0056] Figure 7c According to an embodiment of the present application, a third timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0057] Figure 8 According to an embodiment of the present application, a fourth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0058] Figure 9a According to an embodiment of the present application, a fifth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0059] Figure 9b According to an embodiment of the present application, a sixth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0060] Figure 9c According to an embodiment of the present application, a seventh timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0061] Figure 9dAccording to an embodiment of the present application, an eighth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0062] Figure 9e According to an embodiment of the present application, a first schematic diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0063] Figure 9f According to an embodiment of the present application, a second schematic diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0064] Figure 9g According to an embodiment of the present application, a schematic diagram of a synthetic frame rate is shown;

[0065] Figure 10 According to an embodiment of the present application, a second flow chart of an image processing method is shown;

[0066] Figure 11a According to an embodiment of the present application, a ninth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0067] Figure 11b According to an embodiment of the present application, a tenth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0068] Figure 12 According to an embodiment of the present application, a third flow chart of an image processing method is shown;

[0069] Figure 13 According to an embodiment of the present application, a fourth flow chart of an image processing method is shown;

[0070] Figure 14 According to an embodiment of the present application, a fifth flow chart of an image processing method is shown;

[0071] Figure 15 According to an embodiment of the present application, a sixth flow chart of an image processing method is shown;

[0072] Figure 16 According to an embodiment of the present application, a seventh flow chart of an image processing method is shown;

[0073] Figure 17 According to an embodiment of the present application, an eighth schematic diagram of an image processing method is shown;

[0074] Figure 18aAccording to an embodiment of the present application, a timing diagram of a Vsync-app signal of a first application, a Vsync-app signal of a second application, and a Vsync-app signal of a third application is shown;

[0075] Figure 18b According to an embodiment of the present application, a timing diagram of a Vsync-sf signal is shown;

[0076] Figure 19 According to an embodiment of the present application, an eleventh timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0077] Figure 20 According to an embodiment of the present application, a twelfth timing diagram of a Vsync-app signal of a first application and a Vsync-app signal of a second application is shown;

[0078] Figure 21 According to an embodiment of the present application, a structural diagram of an electronic device is shown;

[0079] Figure 22 According to an embodiment of the present application, a ninth schematic diagram of an image processing method is shown;

[0080] Figure 23a According to an embodiment of the present application, a first interactive schematic diagram of an image processing method is shown;

[0081] Figure 23b According to an embodiment of the present application, a second interactive schematic diagram of an image processing method is shown;

[0082] Figure 24 According to an embodiment of the present application, a structural diagram of an electronic device 10 is shown. DETAILED DESCRIPTION

[0083] The illustrative embodiments of the present application include, but are not limited to, an image processing method, a terminal device, a readable medium, and a program product.

[0084] The following first explains the technical terms involved in this application.

[0085] Refresh rate: refers to the number of times a terminal device's display refreshes an image frame per second. The unit of refresh rate is hertz (Hz). For example, a terminal device's refresh rate of 60Hz means that the terminal device's display can refresh and display an image frame 60 times per second.

[0086] Rendering frame rate: This refers to the number of frames an application renders per unit time, measured in frames per second (fps). For example, a video playback application with a rendering frame rate of 30 fps means the application can render 30 frames per second. It should be noted that an application can have one or more layers, and during the rendering process of a single layer frame, some or all of the layers may be updated.

[0087] Synthesis frame rate: This refers to the number of image frames that can be synthesized for display by the terminal device's image synthesis module per unit time. For example, a synthesis frame rate of 60 fps means that the module can synthesize 60 image frames per second.

[0088] Vsync signal: Vsync signals include hardware Vsync signals and software Vsync signals. The following describes hardware Vsync signals and software Vsync signals respectively.

[0089] Hardware Vsync signal: The period of the hardware Vsync signal can be determined based on the inverse of the refresh rate of the terminal device display. Figure 1 As shown in the figure, the hardware Vsync signal is generated by hardware. Low-temperature polycrystalline silicon (LTPS) displays usually support fixed refresh rate gears, such as 60Hz, 90Hz, and 120Hz. Users can further set the refresh rate to low refresh rates such as 30Hz, 20Hz, 10Hz, and 1Hz through low-temperature polycrystalline oxide (LTPO) display technology.

[0090] Generally, the refresh rate can be a preset fixed value, or a value that the terminal device intelligently adjusts based on the scenario. For example, a mobile phone can dynamically adjust the refresh rate based on the content. For example, when the phone is playing a video, the screen refresh rate is set to 60Hz, which means the hardware Vsync period is 16.66ms. When the phone is running a game application, the screen refresh rate is set to 90Hz or 120Hz, which means the hardware Vsync period is 11.11ms or 8.33ms.

[0091] Software Vsync signal: The software Vsync signal is generated by the display synchronization module (DispSync) based on the hardware Vsync signal. It controls the application's layer drawing and rendering, and the image compositing module's image frame synthesis. Software Vsync signals include a rendering signal (Vsync-app) and a synthesis signal (Vsync-sf). The Vsync-app signal instructs the application to draw and render a frame. The Vsync-sf signal triggers the image compositing module to synthesize the layers rendered by the application to produce an image frame.

[0092] In some embodiments, the phase interval between the Vsync-app signal and the Vsync-sf signal is 0; in other embodiments, the phase interval between the Vsync-app signal and the Vsync-sf signal is a fixed value.

[0093] It is understood that the frame rates of the Vsync-app and Vsync-sf signals can be determined based on the screen refresh rate. For example, if the screen refresh rate is 120Hz, the frame rates of the Vsync-app and Vsync-sf signals can be 120fps, 90fps, 60fps, 30fps, etc. In some embodiments, the frame rates of the Vsync-app and Vsync-sf signals for different applications are the same; in other embodiments, the frame rates of the Vsync-app and Vsync-sf signals for different applications are different.

[0094] In some embodiments, as Figure 2 As shown, the calculation model in the display synchronization module can obtain the hardware Vsync signal and generate the software Vsync signal based on the hardware Vsync signal, namely the Vsync-app signal and the Vsync-sf signal.

[0095] The technical solution of this application is introduced below with reference to the accompanying drawings.

[0096] For ease of understanding, we first introduce the process of image frame synthesis based on software Vsync signal of terminal equipment. The terminal equipment may include application program and image synthesis module, such as Figure 3 As shown, the process of performing image frame synthesis includes:

[0097] S1: Control the application to apply for the Vsync-app signal.

[0098] In an embodiment of the present application, when a terminal device detects that a user has opened any application on the terminal device, the control application requests a Vsync-app signal from the driver. Applications include system applications and third-party applications. For example, system applications may include settings applications, camera applications, calendar applications, phone applications, or messaging applications, while third-party applications may include instant messaging applications, game applications, video applications, etc.

[0099] As you can understand, when the terminal device detects that the user has launched an application (including system applications such as the camera and phone, as well as third-party applications such as instant messaging tools or games), the terminal device system automatically establishes a Vsync-app signal callback registration mechanism through the Choreographer framework. During the application's initialization of the rendering pipeline, the developer can declare the desired rendering frame rate parameters (for example, setting a high refresh rate of 90fps) through the SurfaceControl interface. This parameter will be incorporated into the image synthesis module's synthesis decision tree for dynamic adaptation.

[0100] S2: Controls the application to draw and render the application's layer based on the Vsync-app signal.

[0101] In an embodiment of the present application, the terminal device can control the application to draw and render the next frame layer to be displayed on the display screen based on the Vsync-app signal when it obtains the Vsync-app signal sent by the driver. After the application finishes rendering the next frame layer to be displayed, it stores the next frame layer to be displayed in the graphics buffer, notifies the image synthesis module to synthesize the next frame to be displayed, and continues to request the Vsync-app signal from the driver.

[0102] S3: Detecting that the application has completed layer rendering, the image synthesis module is controlled to apply for a Vsync-sf signal.

[0103] In an embodiment of the present application, the terminal device detects that the application has completed layer rendering (for example, the next frame layer to be displayed is stored in the graphics buffer), and controls the image synthesis module to apply for a Vsync-sf signal from the driver.

[0104] S4: Control the image synthesis module to synthesize image frames based on the Vsync-sf signal.

[0105] In an embodiment of the present application, the terminal device can control the image synthesis module to read the next frame layer to be displayed stored in the graphics buffer based on the Vsync-sf signal after receiving the Vsync-sf signal sent by the driver, and synthesize the next frame layer to be displayed to obtain the next image frame.

[0106] Based on the above, when the displayed image frame is obtained by superimposing the layers of multiple applications, after the rendering of the corresponding layer of each application is completed, it will trigger the image synthesis module to request a Vsync-sf signal and synthesize the image frame (performing the aforementioned steps S3 and S4). If the Vsync-app signals of the multiple applications are not synchronized, the Vsync-sf signals of the multiple applications are also not synchronized, causing the image synthesis module to respond to the Vsync-sf signal of each application to synthesize the image frame, which will increase the synthesis frame rate of the image synthesis module synthesizing the image frames.

[0107] For example, the rendering frame rates of the first application and the second application are the same, the values ​​of the rendering frame rates of the first application and the second application are half of the refresh rate values, and the Vsync-app signals of the first application and the second application differ by one Vsync cycle. Figure 4 As shown, in the mth Vsync cycle, the display screen of the terminal device displays the image frame B0, and the first application responds to the Vsyns-app signal of the hardware Vsync signal of the mth Vsync cycle to start drawing and rendering the layer A1; after the rendering of the layer A1 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the mth Vsync cycle to start synthesizing the layer A1 to obtain the image frame B1.

[0108] In the m+1th Vsync cycle, the display screen displays image frame B1, and the second application responds to the Vsyns-app signal of the hardware Vsync signal of the m+1th Vsync cycle to start drawing and rendering layer C0. After the rendering of layer C0 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+1th Vsync cycle to start synthesizing layer A1 and layer C0 to obtain image frame B2.

[0109] In the m+2th Vsync cycle, the display screen displays image frame B2. The first application responds to the Vsyns-app signal of the hardware Vsync signal of the m+2th Vsync cycle to start drawing and rendering layer A2. After the rendering of layer A2 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+2th Vsync cycle to start synthesizing layer A2 and layer C0 to obtain image frame B3.

[0110] In the m+3th Vsync cycle, the display screen displays image frame B3. The second application responds to the Vsyns-app signal of the hardware Vsync signal of the m+3th Vsync cycle to start drawing and rendering layer C1. After the rendering of layer C1 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+3th Vsync cycle to start synthesizing layer A2 and layer C1 to obtain image frame B4.

[0111] In the m+4th Vsync cycle, the display screen displays image frame B4. The first application responds to the Vsyns-app signal of the hardware Vsync signal of the m+4th Vsync cycle, and starts to draw and render layer A3. After the rendering of layer A3 is completed, the image synthesis module responds to the Vsyns-sf signal of the hardware Vsync signal of the m+4th Vsync cycle, and starts to synthesize layer A3 and layer C1 to obtain image frame B5.

[0112] for Figure 4 The rendering frame rates of the first and second applications are the same, and the Vsync-app signals of the first and second applications differ by one Vsync cycle. This means that the Vsyns-app signals of the first and second applications are completely offset. This is equivalent to the image synthesis module synthesizing the image with a synthesis frame rate = the rendering frame rate of the first application + the rendering frame rate of the second application = 2 × the rendering frame rate of the first application. Although the synthesis frame rate of the synthesized image synthesized by the image synthesis module is twice the rendering frame rate, it does not improve the display quality of the images of the first and second applications, resulting in wasted power consumption.

[0113] In view of this, an embodiment of the present application provides an image processing method. In this method, when the image frame to be generated includes display content of multiple applications, the terminal device can determine the first application with the largest rendering frame rate among the multiple applications, and adjust the Vsync-app signals of the other applications in the multiple applications except the first application to be synchronized with the Vsync-app signal of the first application. The terminal device can then draw, render, and synthesize the image frame based on the adjusted Vsync-app signal.

[0114] Since the phase interval between the Vsync-sf signal of each application and the corresponding Vsync-app signal is the same (for example, 0 or a fixed value), in the image frame corresponding to the moment when the Vsync-app signals of the multiple applications coincide, even if at least two applications among the multiple applications update the rendering content, the image synthesis module only needs to perform image synthesis once, thereby reducing the number of synthesis times of the image synthesis module, and reducing the synthesis frame rate and power consumption.

[0115] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, aligning the Vsync-app signal of an application with a lower rendering frame rate with the Vsync-app signal of the first application with the highest rendering frame rate can preserve the visual advantages of a high rendering frame rate.

[0116] It can be understood that synchronization of the Vsync-app signals of multiple applications means that the Vsync-app signals of the multiple applications are aligned in time. The following description assumes that the value of the first frame rate of the first rendering frame rate of a first application is referred to as the first value, the value of the second frame rate of the second rendering frame rate of a second application other than the first application among the multiple applications is referred to as the second value, and the Vsync-app signal of the second application is adjusted to align with the Vsync-app signal of the first application.

[0117] When the first value is an integer multiple of the second value (or first value / second value=N, where N is a positive integer), the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the second application as a whole by a delay time T in the time domain.

[0118] For example, if the first frame rate is 60fps and the second frame rate is 30fps, the second Vsyns-app signal of the second application can be delayed as a whole in the time domain by a delay time T, so that the delayed second Vsyns-app signal of the second application is aligned with the first Vsyns-app signal of the first application.

[0119] It can be understood that when the first value is an integer multiple of the second value, if the Vsync-app signals of the first application and the second application are not synchronized, the composite frame rate will be the first frame rate + the second frame rate; after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate will be the first frame rate. This allows the image synthesis module to reduce the number of synthesis times, thereby reducing the composite frame rate and power consumption.

[0120] When both the first value and the second value are greater than or equal to a preset threshold and the first value is not an integer multiple of the second value (or when first value / second value ≠ N), upon determining that the ratio of the first value to the second value is greater than or equal to the first ratio, the Vsync-app signal of the second application can be delayed as a whole in the time domain by a delay time T, so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0121] For example, the preset threshold is 60, the first ratio = 1.5, the first frame rate is 90fps, and the second frame rate is 60fps. The Vsync-app signal of the second application can be delayed as a whole in the time domain by a delay time T, so that the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0122] It can be understood that if the rendering frame rate of the application is greater than or equal to a preset threshold, for example, the rendering frame rate of the application is greater than or equal to 60fps, it is determined that the current scene is a high refresh rate scene.

[0123] When the rendering frame rates of the first and second applications are both greater than or equal to a preset threshold, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, even if the second application loses a frame or has a certain delay, the user will not notice it. Therefore, in some embodiments, when the rendering frame rates of the first and second applications are both greater than or equal to the preset threshold, even if the rendering frame rate of the first application is not an integer multiple of the rendering frame rate of the second application, the Vsync-app signal of the second application can be aligned with the Vsync-app signal of the first application by delaying the Vsync-app signal of the applications in the time domain by a delay time T.

[0124] It can be understood that if both the first value and the second value are greater than or equal to the preset threshold, and the first frame rate is not an integer multiple of the second frame rate, after the Vsync-app signals of the first and second applications are synchronized, the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, so the composite frame rate after alignment will be lower than the composite frame rate before alignment. This allows the image synthesis module to reduce the number of synthesis times, lowering the composite frame rate and power consumption.

[0125] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0126] In an embodiment of the present application, the delay time T = the moment when the first application draws the layer of the current frame interface - the moment when the second application draws the layer of the current frame interface + the first time interval corresponding to the first rendering frame rate of the first application - the second time interval corresponding to the second rendering frame rate of the second application.

[0127] It can be understood that the time interval represents the display time of each frame of the image. The first time interval corresponding to the first rendering frame rate = 1 / the first rendering frame rate; the second time interval corresponding to the second rendering frame rate = 1 / the second rendering frame rate.

[0128] For example, if the first rendering frame rate is 120 fps, the first time interval = 1 / 120 = 8.33 ms; if the second rendering frame rate is 30 fps, the second time interval = 1 / 30 = 33.33 ms.

[0129] In some embodiments, the delay time T=k times the second time interval, 0<k<1.

[0130] For example, the first value of the first rendering frame rate of the first application is greater than the second value of the second rendering frame rate of the second application. Figure 4 In the scenario shown, the delay time T = t4 - t3 + the first time interval - the second time interval, that is, the Vsync-app signal of the second application rendering layer C1 is delayed from time t4 to time t5 by the delay time T, so that the Vsync-app signal used by the second application to render layer C1 is synchronized with the Vsync-app signal used by the first application to render layer A3, and the result is Figure 5 The schematic diagram shown.

[0131] In an embodiment of the present application, when the image frame to be generated includes display content of multiple applications and the rendering frame rates of the multiple applications are all lower than the synthetic frame rate, it is characterized that the Vsync-app signal of at least one application among the multiple applications is not synchronized with the Vsync-app signals of other applications. The Vsync-app signals of other applications except the first application among the multiple applications can be adjusted to be synchronized with the Vsync-app signal of the first application.

[0132] In some embodiments, when there are at least two applications with the largest rendering frame rate among multiple applications, the application whose application interface has a larger display area on the display screen can be selected as the first application among the at least two applications; or, the application whose application interface has a display area on the display screen that is greater than a first threshold and a distance between the application interface and the center of the display screen that is less than a second threshold can be selected as the first application; or, the application whose interface has a display area on the display screen that is greater than the first threshold and the display area of ​​the application interface on the display screen includes the center of the display screen can be selected as the first application.

[0133] In some embodiments, when there are at least two applications with the largest rendering frame rate among multiple applications, the application whose application interface has a larger display area on the display screen can be determined from top to bottom according to the size of the Z-axis coordinates of the layers of the at least two applications as the first application; or, the application whose application interface has a display area on the display screen that is greater than a first threshold and the distance between the application interface and the center of the display screen is less than a second threshold as the first application; or, the application whose interface has a display area on the display screen that is greater than the first threshold and the display area of ​​the application interface on the display screen includes the center of the display screen as the first application.

[0134] It is understood that, when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application whose interface has a larger display area on the display screen, or the application whose interface is in an upper layer is generally more likely to attract the user's visual attention. Therefore, the application whose display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application whose interface has a larger display area on the display screen, or the application whose interface is in an upper layer is used as the first application, and the Vsync-app signals of the other applications among the multiple applications except the first application are adjusted to be synchronized with the Vsync-app signal of the first application, can improve the user's visual experience.

[0135] Figure 6a According to the embodiment of the present application, a flow chart of an image processing method is shown. It can be understood that Figure 6a The execution entities of each step in the process shown are all terminal devices. Figure 6a The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 6a As shown, the image processing method includes:

[0136] 101: It is detected that a first application renders a layer of an i-th frame interface of the first application at a first moment, and a second application renders a layer of a j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0137] In an embodiment of the present application, when a terminal device enters a preset scenario, such as a multi-window scenario, a split-screen scenario, a picture-in-picture playback scenario, or a smart capsule playback scenario, the terminal device runs a first application and a second application. The first application and the second application can be system applications or third-party applications. System applications can include settings applications, cameras, calendars, phone calls, or short messaging applications, and third-party applications can include instant messaging applications, game applications, video applications, and the like.

[0138] For example, the first application is a video application and the second application is a phone application. Figure 6b As shown, when the terminal device 10 displays the animation 1 of the interface 101 of the video application, it also displays the sequence frame animation of the interface 102 of the phone.

[0139] In an embodiment of the present application, the first application may render a layer based on a first rendering frame rate, and the second application may render a layer based on a second rendering frame rate.

[0140] In the embodiment of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0141] 102: Control the first application to render the layer of the i+1th frame interface of the first application at the third moment, and the second application to render the layer of the j+1th frame interface of the second application at the third moment, wherein the first time difference between the third moment and the first moment is different from the first time interval corresponding to the first rendering frame rate, or the second time difference between the third moment and the second moment is different from the second time interval corresponding to the second rendering frame rate.

[0142] The method of controlling the first application to render the layer of the i+1 frame interface of the first application at the third moment and the second application to render the layer of the j+1 frame interface of the second application at the third moment is as follows: Figure 6c As shown, including:

[0143] 1021: Obtain a first rendering frame rate of the first application, a second rendering frame rate of the second application, and a composite frame rate of the terminal device.

[0144] 1022: Based on the first rendering frame rate of the first application, the second rendering frame rate of the second application, and the composite frame rate of the terminal device, determine whether the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned.

[0145] If the judgment result is yes, the processing flow ends; if the judgment result is no, go to 1023: determine whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0146] In an embodiment of the present application, a method for determining whether the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned can be: determining whether the first value of the first rendering frame rate of the first application and the second value of the second rendering frame rate of the second application are both less than the third value of the composite frame rate of the terminal device; if the judgment result is yes, the processing flow is terminated; if the judgment result is no, go to 1023: determine whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0147] It can be understood that when the time when the first application renders the layer based on the Vsync-app signal and the time when the second application renders the layer based on the Vsync-app signal are completely staggered, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both less than the third value of the composite frame rate of the terminal device. When the time when the first application renders the layer based on the Vsync-app signal and the time when the second application renders the layer based on the Vsync-app signal are aligned or partially aligned, the first value of the first rendering frame rate and / or the second value of the second rendering frame rate are equal to the third value of the composite frame rate of the terminal device.

[0148] For example, the first value is 30, the second value is 30, and the third value of the composite frame rate is 30, that is, the third value = the first value = the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned in each frame.

[0149] For another example, the first value is 30, the second value is 15, and the third value of the composite frame rate is 30, that is, the third value = the first value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned every 30 frames.

[0150] For another example, the first value is 15, the second value is 30, and the third value of the composite frame rate is 30, that is, the third value = the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned every 30 frames.

[0151] For another example, the first value is 30, the second value is 30, and the third value of the composite frame rate is 60, that is, the third value > the first value, and the third value > the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are not aligned in each frame.

[0152] For another example, the first value is 15, the second value is 30, and the third value of the composite frame rate is 45, that is, the third value > the first value, and the third value > the second value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are not aligned in each frame.

[0153] 1023: Determine whether the first value is an integer multiple of the second value, or whether the ratio of the first value to the second value is greater than or equal to the first ratio.

[0154] If it is determined that the first value is not an integer multiple of the second value, and the ratio of the first value to the second value is less than a preset first ratio, the processing flow ends. If it is determined that the first value is an integer multiple of the second value, or the ratio of the first value to the second value is greater than or equal to the first ratio, the process goes to 1024: determining a master application between the first application and the second application.

[0155] It can be understood that if the first value is neither an integer multiple of the second value, and the ratio of the first value to the second value is smaller than the first ratio, even if the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned at the same time, or the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized at the same time at a certain moment, after this moment, the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are staggered, and the synthesis frame rate cannot be reduced; if the Vsync-app signal of the first application and the Vsync-app signal of the second application are aligned again after this moment, the processing flow will be increased.

[0156] It is understood that the first ratio may be a positive number greater than 1, for example, the first ratio = 1.5. In other embodiments, the first ratio = 1.8.

[0157] For example, the first ratio is 1.8, the preset threshold is 60, the first value is 50, the second value is 30, the first value and the second value are both less than 60, the first value / the second value=1.67<1.8, that is, the first value is not an integer multiple of the second value, the first value and the second value are both less than 60, and the ratio of the first value to the second value is also less than the first ratio, such as Figure 8 As shown, although the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized simultaneously at time t1, the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal after time t1 are staggered, and the synthesis frame rate cannot be reduced.

[0158] 1024: Determine a main application between the first application and the second application.

[0159] In the embodiment of the present application, a method for determining the main application is introduced below.

[0160] 1025: Calculate the delay time T of the non-main application, and delay the Vsync-app signal of the non-main application by the delay time T.

[0161] In an embodiment of the present application, if the second application is determined to be the main application, the Vsync-app signal of the layer of the i+1 frame interface rendered by the first application is delayed by a delay time T, so that the first application renders the layer of the i+1 frame interface of the first application at the third moment, and the second application renders the layer of the j+1 frame interface of the second application at the third moment; if the first application is determined to be the main application, the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application is delayed by a delay time T, so that the second application renders the layer of the j+1 frame interface of the second application at the third moment, and the first application renders the layer of the i+1 frame interface of the first application at the third moment.

[0162] For scenarios where the first application is the primary application, for example Figure 7a In the scenario shown, the Vsync-app signal of the second application rendering the j+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, and the result is Figure 9a The schematic diagram shown is as follows Figure 9a As shown, the second application renders the layer of the j+1 frame interface at the third time t4, and the first application renders the layer of the i+1 frame interface at the third time t4. Figure 7b In the scenario shown, the Vsync-app signal of the second application rendering the j+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, and the result is Figure 9b As shown in the schematic diagram, Figure 9b As shown, the second application renders the layer of the j+1th frame interface at the third time t4, and the first application renders the layer of the i+1th frame interface at the third time t4.

[0163] For the scenario where the second application is the main application, for example, Figure 7c In the scenario shown in FIG. 1 , the Vsync-app signal of the first application rendering the i+1 frame interface can be delayed from the fourth moment t3 to the third moment t4 by a delay time T, thereby obtaining the following: Figure 9c The schematic diagram shown is as follows Figure 9c As shown, the first application renders the layer of the i+1th frame interface at the third time t4, and the second application renders the layer of the j+1th frame interface at the third time t4.

[0164] 103 : After the third moment, control the first application to render the layer based on the first rendering frame rate, and control the second application to render the layer based on the second rendering frame rate.

[0165] In an embodiment of the present application, after the third moment, the time interval between the layers of adjacent frame interfaces drawn by the first application is the first time interval corresponding to the first rendering frame rate. For example, the time interval between the layer of the i+1 frame interface and the layer of the i+2 frame interface drawn by the first application is the first time interval, and the time interval between the layer of the i+2 frame interface and the layer of the i+3 frame interface is the first time interval.

[0166] After the third moment, the time interval between the layers of adjacent frame interfaces drawn by the second application is the second time interval corresponding to the second rendering frame rate. For example, the time interval between the layer of the j+1 frame interface and the layer of the j+2 frame interface drawn by the second application is the second time interval, and the time interval between the layer of the j+2 frame interface and the layer of the j+3 frame interface is the second time interval.

[0167] The aligned composite frame rate is described below by taking the case where the first value is an integer multiple of the second value as an example.

[0168] For example, the first value is 30 and the second value is 30, that is, the first value is 1 times the second value. If the Vsync-app signals of the first application and the second application are not synchronized, such as Figure 9e As shown, the i-th frame layer and the j-th frame layer are rendered and synthesized at different times. The time difference between synthesizing the i-th frame layer and synthesizing the j-th frame layer is 16.6ms, that is, the initial synthesis frame rate is 60fps.

[0169] If the first application is the main application, in the embodiment of the present application, the second Vsyns-app signal of the second application can be delayed as a whole in the time domain by a delay time T, so that the delayed second Vsyns-app signal of the second application is aligned with the first Vsyns-app signal of the first application every 30 frames, such as Figure 9f As shown, the i+1th frame layer and the j+1th frame layer are rendered and synthesized at the same time, and the resulting synthetic frame rate is as follows: Figure 9g As shown. Figure 9g As shown, if the time interval is 4s and the time interval includes the moment when the Vsync-app signals of the first application and the second application are not aligned, the image synthesis module can also synthesize 186 frames within 4s, and the average synthesis frame rate is 45fps.

[0170] For example, the first value is 60 and the second value is 30, that is, the first value is twice the second value. If the Vsync-app signals of the first application and the second application are not synchronized, the initial composite frame rate = 30fps + 60fps = 90fps. In an embodiment of the present application, by delaying the second Vsyncs-app signal of the second application as a whole in the time domain by a delay time T, the delayed second Vsyncs-app signal of the second application is aligned with the first Vsyncs-app signal of the first application every 60 frames, so that the composite frame rate is adjusted to 60fps.

[0171] For another example, if the first value is twice the second value, and the Vsync-app signals of the first application and the second application are not synchronized, the initial composite frame rate = 1.5 × the first value. Figure 9b By delaying the second Vsync-app signal of the second application rendering the j+1th frame layer from the fourth moment t3 to the third moment t4 by the delay time T, so that after the third moment t4, the Vsync-app signal of each frame layer of the second application coincides with the Vsync-app signal of the first application, and the value of the synthetic frame rate = the first value.

[0172] In an embodiment of the present application, by aligning the Vsync-app signal of an application with a lower rendering frame rate with the Vsync-app signal of an application with a higher rendering frame rate, the number of times the rendering layer is synthesized can be reduced, and the synthesis frame rate and power consumption can be reduced.

[0173] The aligned composite frame rate is described below by taking the second value being an integer multiple of the first value as an example.

[0174] For example, if the second value is twice the first value, and the Vsync-app signals of the first application and the second application are not synchronized, the initial synthetic frame rate = 1.5 × the second value. Figure 9c By delaying the first Vsync-app signal of the first application rendering the i+1th frame layer from the fourth moment t3 to the third moment t4 by the delay time T, so that after the third moment t4, the Vsync-app signal of each frame layer of the first application coincides with the Vsync-app signal of the second application, and the value of the composite frame rate = the second value.

[0175] In an embodiment of the present application, by aligning the Vsync-app signal of an application with a lower rendering frame rate with the Vsync-app signal of an application with a higher rendering frame rate, the number of times the rendering layer is synthesized can be reduced, and the synthesis frame rate and power consumption can be reduced.

[0176] The aligned synthetic frame rate is described below by taking the preset threshold being 60 and the ratio of the first value to the second value being greater than or equal to the first ratio as an example.

[0177] For example, if the first value is 90 and the second value is 60, the first value is 1.5 times the second value. Figure 9d After the third moment t4, the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application, and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application again at t5, and the composite frame rate after alignment is less than the composite frame rate before alignment.

[0178] It can be understood that if both the first value and the second value are greater than or equal to the preset threshold, and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first and second applications are synchronized, the Vsync-app signal of the second application overlaps with the Vsync-app signal of the first application in at least one frame layer, so the composite frame rate after alignment will be lower than the composite frame rate before alignment. This allows the image synthesis module to reduce the number of synthesis times, lowering the composite frame rate and power consumption.

[0179] In some embodiments, when the first value and the second value are both greater than or equal to a preset threshold and the first value is not an integer multiple of the second value, after the Vsync-app signals of the first application and the second application are synchronized, the composite frame rate is equal to the first value.

[0180] In an embodiment of the present application, after the third moment, the refresh rate of the terminal device display screen can be adjusted based on the synthetic frame rate of the terminal device.

[0181] For example, after the third moment, the refresh rate of the terminal device display screen may be adjusted based on the frame rate of the gear corresponding to the composite frame rate of the terminal device. For another example, after the third moment, when it is determined that the change in the composite frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen may be adjusted based on the composite frame rate of the terminal device. For another example, after the third moment, when it is determined that the change in the composite frame rate of the terminal device within a preset period is less than a fourth threshold, the refresh rate of the terminal device display screen may be adjusted based on the frame rate of the gear corresponding to the composite frame rate of the terminal device.

[0182] Adjusting the display's refresh rate based on the composite frame rate directly adjusts the display's refresh rate based on the composite frame rate to optimize power consumption. For example, if the composite frame rate is 30 fps, the refresh rate will be set to 30 Hz to match the frame rate of the application currently running on the terminal device, reducing screen tearing. This approach can improve the user experience.

[0183] Adjusting the refresh rate based on the composite frame rate's corresponding gear level means selecting a refresh rate gear that's closest to or matches the composite frame rate. For example, if the composite frame rate is 29.97fps (close to 30fps), the refresh rate will be set to 30Hz. This approach ensures that the terminal device provides the best viewing experience at different frame rates while reducing resource and energy consumption. It allows the system to flexibly adjust the refresh rate in different application scenarios to suit different content and user needs.

[0184] In an embodiment of the present application, after the third moment, the composite frame rate of the terminal device is less than the composite frame rate of any moment before the third moment. In this way, the refresh rate of the terminal device display screen is adjusted according to the composite frame rate of the terminal device after the third moment, which can significantly improve the visual experience and optimize the performance of the terminal device.

[0185] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0186] The following combination Figure 10 , the method of determining the main application in the first application and the second application in step S1024 in step 102 of the embodiment of the present application is introduced. It can be understood that Figure 10 The execution entities of each step in the process shown are all terminal devices. Figure 10 The execution entities of each step will not be described repeatedly in the steps of the process shown. Figure 10 As shown, including:

[0187] 201: Determine a maximum rendering frame rate according to the rendering frame rates of each application on the terminal device.

[0188] In an embodiment of the present application, the rendering frame rate of each application of the terminal device can be first obtained, and then the maximum rendering frame rate can be determined based on the obtained rendering frame rate of each application of the terminal device. For example, the terminal device includes a first application and a second application, the first rendering frame rate of the first application is 60fps, and the second rendering frame rate of the second application is 60fps, then the maximum rendering frame rate is determined to be 60fps. For another example, the first rendering frame rate is 60fps and the second rendering frame rate of the second application is 30fps, then the maximum rendering frame rate is determined to be 60fps. For another example, the first rendering frame rate is 30fps and the second rendering frame rate of the second application is 60fps, then the maximum rendering frame rate is determined to be 60fps.

[0189] 202: Determine whether there is only one application corresponding to the maximum rendering frame rate.

[0190] If the judgment result is yes, then go to 203: determine that the application corresponding to the maximum rendering frame rate is the main application. If the judgment result is no, then go to 204: determine the display area of ​​the display area of ​​the multiple applications corresponding to the maximum rendering frame rate on the display screen, the distance between the application interface and the center of the display screen, and the z-axis coordinate.

[0191] 203: Determine the application corresponding to the maximum rendering frame rate as the main application.

[0192] For example, if the first rendering frame rate is 60 fps and the maximum rendering frame rate is 60 fps, the first application is determined to be the primary application. For another example, if the second rendering frame rate is 60 fps and the maximum rendering frame rate is 60 fps, the second application is determined to be the primary application.

[0193] In the embodiment of the present application, if the terminal device includes a high refresh rate application, the high refresh rate application can be used as the main application. It is understood that the high refresh rate application can be an application with a frame rate greater than a preset frame rate, for example, the preset frame rate can be 90fps.

[0194] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can preserve the visual advantages of the high rendering frame rate.

[0195] 204: Determine the display areas of the display regions of the multiple applications on the display screen corresponding to the maximum rendering frame rate, the distance from the application interface to the center of the display screen, and the z-axis coordinate.

[0196] In an embodiment of the present application, if the maximum rendering frame rate corresponds to multiple applications, the four vertex coordinates of the layers rendered by the multiple applications corresponding to the maximum rendering frame rate can be obtained, and the display area of ​​the display region of the application on the display screen and the center point of the display region of the application on the display screen are determined based on the four vertex coordinates of the layers rendered by the applications. The distance between the center of the display region of the application on the display screen and the center of the display screen and the z-axis coordinate of the layer rendered by the application are determined.

[0197] It is understood that the center of the display area applied on the display screen refers to the geometric center of the display area applied on the display screen, that is, the midpoint of the display area applied on the display screen in the horizontal and vertical directions. The center of the display screen refers to the geometric center of the display screen, that is, the midpoint of the display screen in the horizontal and vertical directions.

[0198] 205: Based on the z-axis coordinate, determine that among the multiple applications corresponding to the maximum rendering frame rate, the application whose display area is greater than the first threshold and the distance between the application interface and the center of the display screen is less than the second threshold is the main application, or the application whose interface includes the center of the display screen in the display area of ​​the display screen is the main application.

[0199] In an embodiment of the present application, each application of the terminal device can be traversed from large to small according to the z-axis coordinates of each application to determine that among the multiple applications corresponding to the maximum rendering frame rate, the application whose display area is greater than a first threshold and the distance between the application interface and the center of the display screen is less than a second threshold is the main application, or the application whose interface includes the center of the display screen in the display area of ​​the display screen is the main application.

[0200] For example, the applications of the terminal device include a first application and a second application. If it is determined that the first application and the second application meet the first condition, the first application is determined to be the main application. The first condition includes any one of the following conditions: the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate; the first value is equal to the second value, and the first display area of ​​the first application interface on the display screen is greater than the second display area of ​​the second application interface on the display screen; the first value is equal to the second value, the first display area is greater than a first threshold, and the distance between the first application interface and the center of the display screen is less than a second threshold; the first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the first application interface on the display screen includes the center of the display screen; the first value is equal to the second value, the first display area is greater than the first threshold, the distance between the first application interface and the center of the display screen is less than the second threshold, and the first application interface is on an upper layer of the second application interface; the first value is equal to the second value, the first display area is greater than the first threshold, the display area of ​​the first application interface on the display screen includes the center of the display screen, and the first application interface is on an upper layer of the second application interface.

[0201] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0202] Therefore, when the first value is equal to the second value, when it is determined that the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen; or, when the first display area is larger than the first threshold and the distance between the interface of the first application and the center of the display screen is smaller than the second threshold; or, when the first display area is larger than the first threshold and the display area of ​​the interface of the first application on the display screen includes the center of the display screen; or, when the first display area is larger than the first threshold, the distance between the interface of the first application and the center of the display screen is smaller than the second threshold, and the interface of the first application is on an upper layer of the interface of the second application; or, when the first display area is larger than the first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application, the first application is used as the main application and the Vsync-app signal of the second application is aligned with the Vsync-app signal of the first application.

[0203] For another example, the applications of the terminal device include a first application and a second application. If it is determined that the first application and the second application meet the second condition, the second application is determined to be the main application. The second condition includes any one of the following conditions: the second value of the second rendering frame rate is greater than the first value of the first rendering frame rate; the second value is equal to the first value, and the second display area of ​​the interface of the second application on the display screen is greater than the first display area of ​​the interface of the first application on the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold; the second value is equal to the first value, the second display area is greater than the first threshold, and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; the second value is equal to the first value, the second display area is greater than the first threshold, the distance between the interface of the second application and the center of the display screen is less than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; the second value is equal to the first value, the second display area is greater than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

[0204] It can be understood that when the display area of ​​the application interface on the display screen includes the center of the display screen, or the distance between the application interface and the center of the display screen is less than the second threshold, or the application interface has a larger display area on the display screen, or the application interface is in the upper layer, it is generally easier to attract the user's visual attention.

[0205] Therefore, when the second value is equal to the first value, when it is determined that the second display area of ​​the interface of the second application on the display screen is larger than the first display area of ​​the interface of the first application on the display screen; or, the second display area is larger than the second threshold and the distance between the interface of the second application and the center of the display screen is smaller than the second threshold; or, the second display area is larger than the first threshold and the display area of ​​the interface of the second application on the display screen includes the center of the display screen; or, the second display area is larger than the first threshold, the distance between the interface of the second application and the center of the display screen is smaller than the second threshold, and the interface of the second application is on the upper layer of the interface of the first application; or, when the second display area is larger than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application, the second application is used as the main application and the Vsync-app signal of the first application is aligned with the Vsync-app signal of the second application.

[0206] In an embodiment of the present application, if the first application is the main application, the delay time T of the Vsync-app signal of the second application can be calculated, so that after the second moment, the Vsync-app signal of the second application is delayed by the delay time T, and the layer of the j+1th frame interface of the second application is rendered based on the delayed Vsync-app signal.

[0207] In an embodiment of the present application, the delay time T = the first moment - the second moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the second time interval, 0 < k < 1.

[0208] It can be understood that if the first application is the primary application, the first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is equal to the first time interval, and the second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0209] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 11aIn the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 11a In the scenario shown, the delay time T = t4 - t3 - the first time interval, where t4 is the alignment time of the primary application and the non-primary application, and t3 is the time when the second application renders the layer of the j+1th frame interface of the second application before the delay time T.

[0210] In an embodiment of the present application, if the second application is the main application, the delay time T of the Vsync-app signal of the first application can be calculated, so that after the second moment, the Vsync-app signal of the first application is delayed by the delay time T, and the first application should render the layer of the i+1th frame interface of the first application based on the delayed Vsync-app signal.

[0211] In an embodiment of the present application, the delay time T = the second moment - the first moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the first time interval, 0 < k < 1.

[0212] It can be understood that if the second application is the primary application, the first time difference between the third moment when the first application renders the layer of the first application's frame interface (i+1) and the first moment when the first application renders the layer of the first application's frame interface is the sum of the first time interval and the delay time T, and the second time difference between the third moment when the second application renders the layer of the second application's frame interface (j+1) and the second moment when the second application renders the layer of the second application's frame interface is the second time interval. The j-frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0213] For example, if the second application is the main application, Figure 9c In the scenario shown, the fifth moment is t3, and the delay time T = the second moment t2 - the first moment t1 + the first time interval - the second time interval. Figure 11b In the scenario shown, the fourth moment is t3, and the delay time T = the second moment t2 - the first moment t1 + the first time interval - the second time interval. Figure 11b In the scenario shown, the delay time T = t4 - t3 - the first time interval, where t4 is the alignment time of the primary application and the non-primary application, and t3 is the time when the first application renders the layer of the i+1th frame interface of the first application before the delay time T.

[0214] The following combination Figures 12 to 17 , a method for aligning the Vsync-apps of the first application and the second application when the Vsync-apps of the first application and the second application are not aligned is introduced.

[0215] The following first introduces a method for aligning the Vsync-apps of the first application and the second application when the first rendering frame rate of the first application is greater than the second rendering frame rate of the second application, taking the first rendering frame rate of the first application as 60fps, the second rendering frame rate of the second application as 30fps, and the composite frame rate as 90fps as an example. Figure 12 As shown, including:

[0216] 301: Run the first application and the second application.

[0217] In an embodiment of the present application, the terminal device can run any number of applications such as the first application and the second application.

[0218] 302: It is detected that the first application renders the layer of the i-th frame interface of the first application at a first moment, and the second application renders the layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0219] In the embodiment of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0220] 303: Determine that a first value of the first rendering frame rate of the first application and a second value of the second rendering frame rate of the second application are both smaller than a third value of the composite frame rate of the terminal device, and the first value is greater than the second value, and determine that the first application is the main application.

[0221] In this embodiment of the present application, the first value is 60, the second value is 30, and the third value is 90. That is, the first value and the second value are both less than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely offset. Because the first value is greater than the second value, the first application is determined to be the primary application.

[0222] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can preserve the visual advantages of the high rendering frame rate.

[0223] 304: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application by the delay time T.

[0224] In an embodiment of the present application, the delay time T = the first moment - the second moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the second time interval, 0 < k < 1.

[0225] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T=the first moment t2-the second moment t1+the first time interval-the second time interval.

[0226] 305: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0227] In this embodiment of the present application, a first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is equal to the first time interval, and a second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0228] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0229] The following describes a method for aligning the Vsync-apps of the first and second applications when the first rendering frame rate of the first application is lower than the second rendering frame rate of the second application, taking the first rendering frame rate of the first application as 30fps, the second rendering frame rate of the second application as 60fps, and the composite frame rate as 90fps as an example. Figure 13 As shown, including:

[0230] 401: Run the first application and the second application.

[0231] In an embodiment of the present application, the terminal device can run any number of applications such as the first application and the second application at the same time.

[0232] 402: It is detected that the first application renders the layer of the i-th frame interface of the first application at a first moment, and the second application renders the layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0233] In the embodiment of the present application, the first moment is earlier than the second moment; in other embodiments, the first moment is later than the second moment. Figure 7a As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, that is, the first time is later than the second time. Figure 7b As shown, the first application renders the layer of the i-th frame interface at the first time t2, and the second application renders the layer of the j-th frame interface at the second time t1, and the first time is later than the second time. Figure 7c As shown, the first application renders the layer of the i-th frame interface at the first moment t1, and the second application renders the layer of the j-th frame interface at the second moment t2, and the first moment is earlier than the second moment.

[0234] 403: Determine that the first value of the first rendering frame rate of the first application and the second value of the second rendering frame rate of the second application are both smaller than the third value of the composite frame rate of the terminal device, and the second value is greater than the first value, and determine that the second application is the main application.

[0235] In this embodiment of the present application, the first value is 30, the second value is 60, and the third value is 90. That is, the first value and the second value are both less than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely offset. Since the second value is greater than the first value, the second application is determined to be the primary application.

[0236] It's understandable that a high rendering frame rate can provide a smoother, clearer visual experience. In scenarios requiring high-speed dynamic images, such as esports games and live sports events, a high rendering frame rate can significantly reduce ghosting and blurring. Therefore, when the first value of the first rendering frame rate is greater than the second value of the second rendering frame rate, aligning the second application with the first application can preserve the visual advantages of the high rendering frame rate.

[0237] 404: Calculate the delay time T, and delay the Vsync-app signal of the layer of the (i+1) frame interface rendered by the first application by the delay time T.

[0238] In an embodiment of the present application, the delay time T = the second moment - the first moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the first time interval, 0 < k < 1.

[0239] For example, if the second application is the main application, Figure 9c In the scenario shown, the delay time T=the second time instant t2-the first time instant t1+the first time interval-the second time interval.

[0240] 405: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0241] In an embodiment of the present application, a first time difference between the third moment when the first application renders the layer of the (i+1)th frame interface of the first application and the first moment when the first application renders the layer of the (i)th frame interface of the first application is equal to the sum of the first time interval and the delay time T. A second time difference between the third moment when the second application renders the layer of the (j+1)th frame interface of the second application and the second moment when the second application renders the layer of the (j)th frame interface of the second application is the second time interval. The (j)th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0242] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0243] The following describes a method for aligning the Vsync-apps of the first and second applications when the first rendering frame rate of the first application is equal to the second rendering frame rate of the second application and the first display area of ​​the first application interface on the display screen is smaller than the second display area of ​​the second application interface on the display screen, taking the first rendering frame rate of the first application, the second rendering frame rate of the second application, and the composite frame rate of 60fps as an example. Figure 14 As shown, including:

[0244] 501: Run the first application and the second application.

[0245] In the embodiment of the present application, 501 and Figure 12 The method of the embodiment 301 is consistent and will not be repeated here.

[0246] 502: It is detected that the first application renders the layer of the i-th frame interface of the first application at a first moment, and the second application renders the layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0247] In the embodiment of the present application, 502 and Figure 12 The method of the embodiment 302 is consistent and will not be repeated here.

[0248] 503: Determine that the first value of the first rendering frame rate of the first application and the second value of the second rendering frame rate of the second application are both smaller than the third value of the composite frame rate of the terminal device, the first value is equal to the second value, and the first display area of ​​the interface of the first application on the display screen is larger than the second display area of ​​the interface of the second application on the display screen, and determine that the first application is the main application.

[0249] In this embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60. That is, the first value and the second value are both less than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely offset. Because the first value is equal to the second value, and the first display area of ​​the first application interface on the display screen is larger than the second display area of ​​the second application interface on the display screen, the first application is determined to be the primary application.

[0250] In some embodiments, the first display area of ​​the first application's interface on the display screen is larger than the second display area of ​​the second application's interface on the display screen, and the first application's interface is on top of the second application's interface, determining that the first application is the main application.

[0251] In some embodiments, if the first display area of ​​the first application's interface on the display screen is smaller than the second display area of ​​the second application's interface on the display screen, the second application is determined to be the primary application. In some embodiments, if the first display area of ​​the first application's interface on the display screen is smaller than the second display area of ​​the second application's interface on the display screen, and the first application's interface is in a lower layer than the second application's interface, the second application is determined to be the primary application.

[0252] It can be understood that an application with a larger display area of ​​its interface on the display screen is generally more likely to attract the user's visual attention.

[0253] 504: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application by the delay time T.

[0254] In an embodiment of the present application, the delay time T = the first moment - the second moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the second time interval, 0 < k < 1.

[0255] For example, the first application is the main application, Figure 9a In the scenario shown, the fourth moment is t3, and the delay time T = the first moment t2 - the second moment t1 + the first time interval - the second time interval. Figure 9b In the scenario shown, the fourth moment is t3, and the delay time T=the first moment t2-the second moment t1+the first time interval-the second time interval.

[0256] In some embodiments, if the second application is the primary application, the Vsync-app signal of the layer of the first application rendering the (i+1)th frame interface is delayed by a delay time T, where the delay time T = the second moment - the first moment + the first time interval - the second time interval. In some embodiments, the delay time T = k times the first time interval, where 0 < k < 1.

[0257] For example, if the second application is the main application, Figure 9c In the scenario shown, the delay time T=the second time instant t2-the first time instant t1+the first time interval-the second time interval.

[0258] 505: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0259] In this embodiment of the present application, a first time difference between the third moment when the first application renders the layer of the i+1th frame interface of the first application and the first moment when the first application renders the layer of the i-th frame interface of the first application is equal to the first time interval, and a second time difference between the third moment when the second application renders the layer of the j+1th frame interface of the second application and the second moment when the second application renders the layer of the j-th frame interface of the second application is the sum of the second time interval and the delay time T. The i-th frame interface is the last frame interface synthesized by the first application before the fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

[0260] In some embodiments, if the second application is the primary application, the first time difference between the third moment when the first application renders the layer of the (i+1)th frame interface of the first application and the first moment when the first application renders the layer of the (i)th frame interface of the first application is the sum of the first time interval and the delay time T, and the second time difference between the third moment when the second application renders the layer of the (j+1)th frame interface of the second application and the second moment when the second application renders the layer of the (j)th frame interface of the second application is the second time interval. The (j)th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

[0261] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0262] The following describes a method for aligning the Vsync-apps of the first and second applications when the first rendering frame rate of the first application is 30fps, the second rendering frame rate of the second application is 30fps, and the composite frame rate is 60fps. Figure 15 As shown, including:

[0263] 601: Run the first application and the second application.

[0264] In the embodiment of the present application, 601 and Figure 14 The method of the embodiment 501 is consistent and will not be repeated here.

[0265] 602: It is detected that the first application renders the layer of the i-th frame interface of the first application at a first moment, and the second application renders the layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0266] In the embodiment of the present application, 602 and Figure 14 The method of the embodiment 502 is consistent and will not be repeated here.

[0267] 603: Determine that a first value of the first rendering frame rate of the first application and a second value of the second rendering frame rate of the second application are both smaller than a third value of the composite frame rate of the terminal device, the first value is equal to the second value, and a first display area of ​​the interface of the first application on the display screen is greater than a first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and determine that the first application is the main application.

[0268] In this embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60, i.e., the first value and the second value are both less than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely offset. Because the first value is equal to the second value, the first display area of ​​the first application interface on the display screen is greater than the first threshold, and the display area of ​​the first application interface on the display screen includes the center of the display screen, the first application is determined to be the primary application.

[0269] In some embodiments, the first application is determined to be the main application if the first display area of ​​the interface of the first application on the display screen is greater than a first threshold, the display area of ​​the interface of the first application on the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application.

[0270] In some embodiments, if the second display area of ​​the second application interface on the display screen is greater than the first threshold and the display area of ​​the second application interface on the display screen includes the center of the display screen, the second application is determined to be the main application.

[0271] In some embodiments, if the second display area of ​​the interface of the second application on the display screen is greater than the first threshold, the display area of ​​the interface of the second application on the display screen includes the center of the display screen, and the interface of the first application is at the lower layer of the interface of the second application, then the second application is determined to be the main application.

[0272] It can be understood that an application interface with a larger display area on the display screen, including an application at the center of the display screen, is generally more likely to attract the user's visual attention.

[0273] 604: Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application by the delay time T.

[0274] In the embodiment of the present application, 604 and Figure 14 The method of the embodiment 504 is consistent and will not be repeated here.

[0275] 605: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0276] In the embodiment of the present application, 605 and Figure 14 The method of the embodiment 505 shown is consistent and will not be repeated here.

[0277] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0278] The following describes a method for aligning the Vsync-apps of the first and second applications when the first rendering frame rate of the first application is 30fps, the second rendering frame rate of the second application is 30fps, and the composite frame rate is 60fps. Figure 16 As shown, including:

[0279] 701: Run the first application and the second application.

[0280] In the embodiment of the present application, 701 and Figure 14 The method of the embodiment 501 is consistent and will not be repeated here.

[0281] 702: It is detected that the first application renders the layer of the i-th frame interface of the first application at a first moment, and the second application renders the layer of the j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers.

[0282] In the embodiment of the present application, 702 and Figure 14 The method of the embodiment 502 is consistent and will not be repeated here.

[0283] 703: Determine that the first value of the first rendering frame rate of the first application and the second value of the second rendering frame rate of the second application are both smaller than the third value of the composite frame rate of the terminal device, the first value is equal to the second value, the first display area of ​​the interface of the first application on the display screen is larger than the first threshold, and the distance between the interface of the first application and the center of the display screen is smaller than the second threshold, and determine that the first application is the main application.

[0284] In this embodiment of the present application, the first value is 30, the second value is 30, and the third value is 60, i.e., the first value and the second value are both less than the third value, indicating that the Vsync-app signal of the first application and the Vsync-app signal of the second application are completely offset. Because the first value is equal to the second value, the first display area of ​​the first application's interface on the display screen is greater than the first threshold, and the distance between the first application's interface and the center of the display screen is less than the second threshold, the first application is determined to be the primary application.

[0285] In some embodiments, the first display area of ​​the interface of the first application on the display screen is greater than a first threshold, the distance between the interface of the first application and the center of the display screen is less than a second threshold, and the interface of the first application is on the upper layer of the interface of the second application, thereby determining that the first application is the main application.

[0286] In some embodiments, if the second display area of ​​the second application interface on the display screen is greater than the first threshold and the distance between the first application interface and the center of the display screen is less than the second threshold, the second application is determined to be the main application.

[0287] In some embodiments, if the second display area of ​​the second application's interface on the display screen is greater than a first threshold, the distance between the first application's interface and the center of the display screen is less than a second threshold, and the first application's interface is at the lower layer of the second application's interface, the second application is determined to be the main application.

[0288] It can be understood that an application whose interface has a larger display area on the display screen and whose interface is closer to the center of the display screen than the second threshold value is generally more likely to attract the user's visual attention.

[0289] 704 : Calculate the delay time T, and delay the Vsync-app signal of the layer of the j+1 frame interface rendered by the second application by the delay time T.

[0290] In the embodiment of the present application, 704 and Figure 14 The method of the embodiment 504 is consistent and will not be repeated here.

[0291] 705: Control the second application to render the layer of the j+1th frame interface of the second application at the third moment, and control the first application to render the layer of the i+1th frame interface of the first application at the third moment.

[0292] In the embodiment of the present application, 705 and Figure 14 The method of the embodiment 505 shown is consistent and will not be repeated here.

[0293] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0294] The following combination Figure 17 An image processing method is provided in the embodiment of the present application. Figure 17 As shown, including:

[0295] 901: The terminal device is detected to have entered scenarios such as multi-window, split-screen, picture-in-picture playback, and smart capsule playback.

[0296] In an embodiment of the present application, when the terminal device detects that the current scene is a preset scene, for example, a multi-window scene, a split-screen scene, a picture-in-picture playback scene, or a smart capsule playback scene, the rendering frame rate of each application is determined.

[0297] 902: Obtain the rendering frame rate of each application and the synthesis frame rate of the image synthesis module.

[0298] In an embodiment of the present application, the terminal device can obtain the rendering frame rate of all applications included in the terminal device (including background applications and non-running applications) and the synthesis frame rate of the image synthesis module.

[0299] 903: Determine whether the rendering frame rate of each application is lower than the synthesis frame rate of the image synthesis module.

[0300] If the judgment result is no, go to 904: exit control; if the judgment result is yes, go to 905: determine whether the rendering frame rate of the application with the highest rendering frame rate is an integer multiple (including 1) of the rendering frame rate of other applications.

[0301] It can be understood that if the rendering frame rate of each application is lower than the composite frame rate of the image synthesis module, it indicates that the Vsync-app signal of at least one application among the multiple applications is not synchronized with the Vsync-app signals of other applications, and you can go to 905: determine whether the rendering frame rate of the application with the largest rendering frame rate is an integer multiple (including 1) of the rendering frame rate of the remaining applications to determine the aligned main application; if there is at least one application with a rendering frame rate equal to the composite frame rate of the image synthesis module, it indicates that the Vsync-app signals of the multiple applications have been aligned and do not need to be aligned again, and go to 904: exit control.

[0302] 904: Exit control.

[0303] 905: Determine whether the rendering frame rate of the application with the highest rendering frame rate is an integer multiple (including 1) of the rendering frame rates of the other applications.

[0304] If the judgment result is yes, go to 906: determine whether there are more than two applications with the same rendering frame rate and the rendering frame rate is the maximum rendering frame rate; if the judgment result is no, go to 904: exit control.

[0305] It can be understood that if the rendering frame rate of the application with the maximum rendering frame rate is not an integer multiple of the rendering frame rates of other applications, frame loss and uneven synthetic frame rate will easily occur after aligning the frame rates. Therefore, in an embodiment of the present application, if it is determined that the rendering frame rate of the application with the maximum rendering frame rate is not an integer multiple of the rendering frame rates of other applications, go to 904: Exit control.

[0306] 906: Determine whether there are two or more (including two) applications with the same rendering frame rate and the maximum rendering frame rate.

[0307] If the judgment result is yes, that is, there are multiple applications with the highest rendering frame rate, go to 907: the application with the highest rendering frame rate is used as the main application; if the judgment result is no, go to 908: calculate the display area of ​​each application with the highest rendering frame rate on the display screen, and the distance from the center point of the application interface to the center point of the display screen through the four vertex coordinates of each layer of the application, and obtain the application's position on the z-axis.

[0308] 907: Use the application with the highest rendering frame rate as the main application.

[0309] It can be understood that in some embodiments, when the rendering frame rate of an application of the terminal device is greater than a preset frame rate, the application is determined to be a high rendering frame rate application, and the high rendering frame rate application is used as the main application.

[0310] 908: Based on the four vertex coordinates of each layer of the application, calculate the display area of ​​each application with the maximum rendering frame rate on the display screen, the distance from the center point of the application interface to the center point of the display screen, and obtain the z-axis position of the application.

[0311] 909: Based on the z-axis coordinates of each application, check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0312] If the judgment result is yes, go to 910: determine whether the display area of ​​the application is large; if the judgment result is no, go to 909: based on the z-axis coordinate of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0313] In an embodiment of the present application, the layers of each application can be traversed in order from large to small based on the coordinates of the multiple applications with the largest rendering frame rates on the z-axis to determine whether the display area of ​​the application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold. If the display area of ​​the application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold, go to 910: determine whether the display area of ​​the application is larger. If the judgment result is no, determine whether the display area of ​​the next application on the display screen includes the center of the display screen, or the distance between the interface of the application and the center of the display screen is less than the second threshold.

[0314] It can be understood that the layers of the upper applications in the display interface of the terminal device generally cover the layers of the lower applications, and in the preset scenario, there will be a first application on the foreground main interface of the terminal device. Therefore, according to the z-axis coordinates of multiple applications with the largest rendering frame rate, the rendering layers of each application are traversed from large to small, which can ensure that the main application can be traversed.

[0315] 910: Determine whether the display area of ​​the application is large.

[0316] If the judgment result is yes, go to 911: calculate the delay time T for aligning the non-main application with the main application one by one, and adopt methods such as a short freezing time T (the related application has no user-perceivable services such as audio), the Vsync-app signal processing delay T of the non-main application, and the reading of the layer delay T of the application rendered by the non-main application stored in the graphics buffer, so that the image synthesis module can synthesize and align the non-main application, and send the non-main application to the display and synthesize it at the same time when the image synthesis module synthesizes the main application; if the judgment result is no, go to 909: based on the z-axis coordinate of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0317] In an embodiment of the present application, it can be determined whether the display area of ​​the application on the display screen is greater than a first threshold. If the display area of ​​the application on the display screen is greater than the first threshold, go to 911: calculate the delay time T for aligning the non-main application with the main application one by one, and adopt methods such as a short freezing time T (the related application has no user-perceivable services such as audio), a Vsync-app signal processing delay T of the non-main application, and a layer delay T time for reading the non-main application stored in the graphics buffer and rendered by the application, so that the image synthesis module synthesizes and aligns the non-main application, and sends the non-main application to the display and synthesizes it at the same time when the image synthesis module synthesizes the main application; if the display area of ​​the application on the display screen is less than or equal to the first threshold, go to 909: based on the z-axis coordinate of each application, traverse and check the layers of each application one by one to determine whether the display area of ​​the application covers the center of the display screen or is close to the center of the display screen.

[0318] 911: Calculate the delay time T for aligning non-main applications with the main application one by one, and enable the image synthesis module to synthesize and align the Vsync-app signals of the main application and the non-main application based on the delay time T.

[0319] In an embodiment of the present application, the delay time T of applications other than the first application in the application of the terminal device can be calculated, and the Vaync-app signal can be delayed according to the delay time of each application other than the first application in the application of the terminal device, so that the Vaync-app signal of applications other than the first application in the application of the terminal device is aligned with the Vaync-app signal of the first application, thereby reducing the synthesis frame rate of the image synthesis module.

[0320] In an embodiment of the present application, a method for delaying the Vaync-app signal of each application other than the first application based on the application of the terminal device includes: for applications that do not involve business functions that can be directly perceived by the user, the Vaync-app signal of the next frame layer of the application can be frozen for a delay time T; or the Vaync-app signal of the next frame layer of the application can be delayed for a delay time T; and the image synthesis module delays the delay time T to read the application-rendered layer stored in the graphics buffer. Among them, the business functions that can be directly perceived by the user include audio playback function, video playback function, tactile feedback function, visual effects function, etc.

[0321] In some embodiments, the delay time T = the moment when the first application draws the layer of the current frame interface - the moment when the second application draws the layer of the current frame interface + the first time interval corresponding to the first rendering frame rate of the first application - the second time interval corresponding to the second rendering frame rate of the second application.

[0322] In some embodiments, the delay time T=k times the second time interval, 0<k<1.

[0323] 912: The image synthesis module synthesizes at least one layer to which rendering is applied at the same time point.

[0324] In an embodiment of the present application, the image synthesis module can synthesize at least one application-rendered layer at the same time point.

[0325] 913: Set the LCD hardware screen refresh rate to the image synthesis module synthesis frame rate, or a gear corresponding to the image synthesis module synthesis frame rate.

[0326] 914: Determine whether the rendering frame rate of any application in the terminal device has changed.

[0327] If the judgment result is yes, go to 904: exit control. If the judgment result is no, go to 915: maintain frame rate alignment and LCD refresh rate configuration.

[0328] In some embodiments, if the rendering frame rate of an application in the terminal device changes, a new judgment is required, and the process may go to 902: checking the rendering frame rate of each application and obtaining the synthesis frame rate of the image synthesis module.

[0329] 915: Maintain frame rate alignment and LCD refresh rate configuration.

[0330] In an embodiment of the present application, layers can be rendered according to the rendering frame rate of each application and the refresh rate configuration of the LCD can be maintained. For example, the LCD hardware screen refresh rate is set to the synthesis frame rate of the image synthesis module, or the gear corresponding to the synthesis frame rate of the image synthesis module.

[0331] In the embodiment of the present application, the Vsync-app signals of each application are aligned. After alignment, the layers are rendered according to the rendering frame rate of each application. Even if at least two of the multiple applications in the terminal device update the rendered content, the number of times the image synthesis module performs image synthesis can be reduced, thereby reducing the synthesis frame rate and power consumption. In addition, setting the LCD refresh rate to the synthesis frame rate of the image synthesis module, or to the refresh rate of the gear corresponding to the synthesis frame rate of the image synthesis module, can optimize power consumption.

[0332] It is understood that the image processing method provided in the embodiment of the present application is also applicable to multiple application scenarios. Figure 18a As shown, the applications of the terminal device include a first application, a second application and a third application, and the second rendering frame rate of the second application is greater than the first rendering frame rate of the first application and the third rendering frame rate of the third application. Then, the second application can be used as the main application, and the Vsync-app signal of the first application, the third Vsync-app signal of the third application and the Vsync-app signal of the second application can be aligned.

[0333] like Figure 18aAs shown, the second rendering frame rate is an integer multiple of the first rendering frame rate and the third rendering frame rate, so after the Vsync-app signal of the first application, the third Vsync-app signal of the third application and the Vsync-app signal of the second application are aligned, the first application, the second application and the third application will not lose frames. Figure 18b It can be found that aligning the Vsync-app signal of the first application, the third Vsync-app signal of the third application, and the Vsync-app signal of the second application can significantly reduce the composite frame rate.

[0334] The following describes the reasons for selecting the application with the highest rendering frame rate as the main application.

[0335] It can be understood that if an application with a lower rendering frame rate is aligned with an application with a higher rendering frame rate, since the timestamp between the two frame layers rendered by the application with a higher rendering frame rate is shorter, or the interval between two adjacent frames of the application with a higher rendering frame rate is shorter, the delay time T required for the application with a lower rendering frame rate during the alignment process is shorter.

[0336] It is understandable that scenes with high rendering frame rates are generally applied because of operations such as clicks and sliding by customers. These operations generally have transition effects, and transition effects generally require high refresh rates to ensure smooth effects and enhance the smoothness of customers' interactive experience.

[0337] For example, Figure 19 As shown, the first rendering frame rate of the first application is greater than the second rendering frame rate of the second application, and the delay time T required for the Vsync-app signal of the second application to align with the Vsync-app signal of the first application is less than the first time interval corresponding to the first rendering frame rate of the first application.

[0338] It is understandable that if an application with a higher rendering frame rate is aligned with an application with a lower rendering frame rate, since the timestamp between the two frame layers rendered by the application with a lower rendering frame rate is longer, or the interval between two adjacent frames of the application with a lower rendering frame rate is longer, the delay time T required for the application with a higher rendering frame rate during the alignment process is longer. Moreover, after the alignment time point, the subsequent frame of the application with a high rendering frame rate is still refreshed normally, and the subsequent frames are aligned again. Therefore, the alignment action of the application with a higher rendering frame rate to the application with a lower rendering frame rate needs to be performed each time, which is more cumbersome than aligning the application with a lower rendering frame rate to the application with a higher rendering frame rate. In addition, if the application with a higher rendering frame rate is directly aligned with the application with a lower rendering frame rate, the time it takes to synthesize the first frame after the high rendering frame rate application is aligned will exceed the interval between the two frames corresponding to the high rendering frame rate, resulting in a large difference between the refresh time of the first frame and the time of the previous frame, resulting in a poor user experience. The graphics buffer will store an extra frame, which can easily cause the first application to drop frames, the graphics buffer to be blocked, the release to be slow, and other problems that affect the customer experience.

[0339] For example, Figure 20 As shown in (a), the first rendering frame rate of the first application is lower than the second rendering frame rate of the second application. In the process of aligning the Vsync-app signal of the first application with the Vsync-app signal of the second application, the required delay time T is greater than the second time interval corresponding to the second rendering frame rate of the second application. At this time, it is easy to cause frame loss and graphics buffer blocking in the first application, and the graphics buffer will store one extra frame.

[0340] In the embodiment of the present application, if the application with a higher rendering frame rate is aligned with the application with a lower rendering frame rate, such as Figure 20 As shown in (b), the high rendering frame rate application does not align the first frame after the alignment starts, but aligns the low rendering frame rate application starting from the next frame after the alignment moment.

[0341] The following combination Figure 21 The structure of the electronic equipment is described. Figure 21 As shown, the software layer of the electronic device includes the application (APP) layer, the application framework layer (also called the native framework layer), the hardware abstraction layer (HAL), and the kernel layer (also called the driver layer); the hardware layer includes the display panel.

[0342] The application layer may include various applications running on electronic devices, such as conference applications, video applications, and communication applications. Applications in the application layer may request a Vsync-app signal from the image display driver and draw layers based on the Vsync-app signal.

[0343] The application framework layer primarily involves image processing modules, such as the graphics processing unit (GPU) for rendering the application's display images and the surface flinger (SF) for compositing multiple layers of the displayed image. The kernel layer primarily includes the image display driver, which includes the display engine (software development engineer (SDE)) and the display serial interface (DSI). The image display driver primarily transmits images processed and sent by upper system layers (i.e., other system layers above the kernel layer, such as the application layer, application framework layer, and hardware abstraction layer)) to the display screen. The hardware layer primarily includes the display screen, which refreshes the displayed image based on calls from the image display driver.

[0344] In such Figure 21 In the system framework of the electronic device 10 shown in FIG, when an APP such as a conference application, a video application or a communication application of the application layer needs to refresh the image to be displayed, such as Figure 22As shown, apps in the application layer, such as conferencing, video, or communication apps, can request a Vsync-app signal from the image display driver. The VsyncDispatchTimerQueue process in the framework layer calculates the Vsync-app signal's distribution time and, when the distribution time is reached, triggers the driver to send the Vsync-app signal to the app. After receiving the Vsync-app signal, the app begins drawing a frame of image. After the app draws a frame, it places the drawn image in the app's corresponding graphics buffer, notifies the GPU in the application framework layer to render the drawn image, notifies the image synthesis module to begin synthesizing the frame, and continues to request a Vsync-app signal from the driver. The VsyncDispatchTimerQueue process in the framework layer calculates the next Vsync-app signal's distribution time. The image synthesis module then requests a Vsync-sf signal from the driver. The VsyncDispatchTimerQueue process in the framework layer calculates the Vsync-sf signal's distribution time and, when the distribution time is reached, triggers the driver to send a Vsync-sf signal to the app. After receiving the Vsync-sf signal, the image synthesis module performs layer synthesis on the rendered image to obtain the display data of the image to be displayed. The image synthesis module then passes this display data to the kernel layer's image display driver through the hardware abstraction layer. The image display driver responds to the hardware VSync signal by calling the DSI through the SDE, writing the display data of the image to be displayed to the display driver integrated circuit (DDIC) of the display. The DDIC then stores the display data sent by the software layer in a buffer. By scanning (or reading) the display data in the buffer, it controls the display panel to refresh the display of the image to be displayed.

[0345] As you can understand, apps can send rendering commands to the GPU through the graphics application programming interface (API). After receiving the rendering commands from the app, the GPU reads the image data drawn by the app and renders the image data. After the GPU completes the rendering, it stores the rendering results in the buffer corresponding to the app.

[0346] The following combination Figure 23a and Figure 23b , the image processing method provided by this application is introduced. Figure 23a and Figure 23b As shown, the image processing method includes:

[0347] 801: The first application requests a Vsync-app signal from the image display driver.

[0348] In an embodiment of the present application, when a user opens a first application on a terminal device, or when the first application needs to draw a layer of the i-th frame interface, the first application may request a Vsync-app signal from the image display driver. The first application includes system applications and third-party applications. For example, system applications may include settings applications, cameras, calendars, phone calls, or messaging applications, while third-party applications may include instant conferencing applications, video applications, communication applications, etc.

[0349] 802: The image display driver sends a Vsync-app signal to the first application.

[0350] 803: The first application renders a layer of the i-th frame interface of the first application based on the Vsync-app signal.

[0351] In an embodiment of the present application, when the first application receives the Vsync-app signal sent by the driver, it can draw and render the layers of the i-th frame interface at the first moment. After the first application completes rendering the layers of the i-th frame interface, it stores the layers of the i-th frame interface in the graphics buffer, notifies the image synthesis module to synthesize the layers of the i-th frame interface, and continues to request the Vsync-app signal from the image display driver.

[0352] 804: The first application sends the layer of the i-th frame interface of the first application to the image synthesis module.

[0353] 805: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0354] 806: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0355] 807: The image synthesis module synthesizes the layers of the i-th frame interface of the first application based on the Vsync-sf signal to obtain a first image frame.

[0356] 808: The image synthesis module sends the first image frame to the display screen.

[0357] 809: The display screen displays the first image frame.

[0358] 810: The second application requests a Vsync-app signal from the image display driver.

[0359] In an embodiment of the present application, when a user opens a second application on a terminal device, or when the second application needs to draw a layer of the j-th frame interface, the second application can request a Vsync-app signal from the image display driver. Second applications include system applications and third-party applications. For example, system applications can be settings applications, cameras, calendars, phone calls, or messaging applications, while third-party applications can be conference applications, video applications, communication applications, etc.

[0360] 811: The image display driver sends a Vsync-app signal to the second application.

[0361] 812: The second application renders a layer of the j-th frame interface of the second application based on the Vsync-app signal.

[0362] In an embodiment of the present application, the second application can render the layer of the j-th frame interface of the second application based on the Vsync-app signal at the second moment. After the second application completes rendering the layer of the j-th frame interface, it stores the layer of the j-th frame interface in the graphics buffer, notifies the image synthesis module to synthesize the layer of the j-th frame interface, and continues to request the Vsync-app signal from the image display driver.

[0363] In an embodiment of the present application, the first moment is earlier than the second moment, and in other embodiments, the first moment is later than the second moment.

[0364] 813: The second application sends the layer of the j-th frame interface of the second application to the image synthesis module.

[0365] 814: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0366] 815: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0367] 816: The image synthesis module synthesizes the layer of the i-th frame interface of the first application and the layer of the j-th frame interface of the second application based on the Vsync-sf signal to obtain a second image frame.

[0368] 817: The image synthesis module sends the second image frame to the display screen.

[0369] 818: The display screen displays the second image frame.

[0370] 819: The first application requests a Vsync-app signal from the image display driver.

[0371] In the embodiment of the present application, the method of 819 is the same as that of 801 and will not be repeated here.

[0372] 820: The second application requests a Vsync-app signal from the image display driver.

[0373] In the embodiment of the present application, steps 820 and 810 are the same and will not be repeated here.

[0374] 821: The image display driver sends a Vsync-app signal to the first application.

[0375] 822: The image display driver sends a Vsync-app signal to the second application.

[0376] 823: The first application renders a layer of the (i+1)th frame interface of the first application based on the Vsync-app signal.

[0377] 824: The second application renders a layer of the j+1th frame interface of the second application based on the Vsync-app signal.

[0378] In an embodiment of the present application, at the third moment, the first application renders the layer of the i+1th frame interface of the first application based on the Vsync-app signal, and the second application renders the layer of the j+1th frame interface of the second application based on the Vsync-app signal.

[0379] It can be understood that at the third moment, the first application renders the layer of the i+1 frame interface of the first application based on the Vsync-app signal, and the second application renders the layer of the j+1 frame interface of the second application based on the Vsync-app signal. Figure 6a The embodiments shown will not be described in detail here.

[0380] 825: The first application sends the layer of the (i+1)th frame interface of the first application to the image synthesis module.

[0381] 826: The second application sends the layer of the j+1th frame interface of the second application to the image synthesis module.

[0382] 827: The image synthesis module applies for a Vsync-sf signal from the image display driver.

[0383] 828: The image display driver sends a Vsync-sf signal to the image synthesis module.

[0384] 829: The image synthesis module synthesizes the layer of the i+1th frame interface of the first application based on the Vsync-sf signal, and the second application sends the layer of the j+1th frame interface of the second application to the image synthesis module to obtain a third image frame.

[0385] 830: The image synthesis module sends the third image frame to the display screen.

[0386] 831: The display screen displays the third image frame.

[0387] In an embodiment of the present application, after the third moment, the moment when the first application renders the layer based on the Vsync-app signal and the moment when the second application renders the layer based on the Vsync-app signal overlap. Even if the first application and the second application update the rendering content, the number of times the layer rendered by the first application based on the Vsync-app signal and the layer rendered by the second application based on the Vsync-app signal are synthesized can be reduced, thereby reducing the synthesis frame rate and power consumption.

[0388] An embodiment of the present application also provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for implementing any one of the image processing methods provided by the various possible implementations mentioned above.

[0389] An embodiment of the present application provides a readable medium having instructions stored thereon. When the instructions are executed on an electronic device, the electronic device implements any one of the image processing methods provided by the various possible implementations described above.

[0390] An embodiment of the present application provides a computer program product, which includes computer instructions. When executed by an electronic device, the electronic device implements any one of the image processing methods provided by the various possible implementations described above.

[0391] The hardware structure of each electronic device mentioned in this application is described below using the electronic device 10 as an example. Figure 24 As shown, the electronic device 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101 and a display screen 102, etc.

[0392] It should be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0393] The processor 110 may include one or more processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP), a microprocessor (MCU), an artificial intelligence (AI) processor, or a processing module or processing circuit such as a field programmable gate array (FPGA). The different processing units may be independent devices or integrated into one or more processors. A storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180.

[0394] It can be understood that the image processing method in the embodiment of the present application can be executed by the processor 110 of the corresponding electronic device. The power module 140 may include a power supply, a power management component, etc. The power supply may be a battery. The power management component is used to manage the charging of the power supply and the supply of power to other modules. In some embodiments, the power management component includes a charging management module and a power management module. The charging management module is used to receive charging input from the charger; the power management module is used to connect the power supply, the charging management module and the processor 110. The power management module receives input from the power supply and / or the charging management module to power the processor 110, the display screen 102, the camera 170, and the wireless communication module 120.

[0395] The mobile communication module 130 may include, but is not limited to, an antenna, a power amplifier, a filter, a low noise amplifier (LNA), and the like. The mobile communication module 130 may provide solutions for wireless communications, including 2G / 3G / 4G / 5G, applied to the electronic device 10. The mobile communication module 130 may receive electromagnetic waves through the antenna, filter, amplify, and perform other processing on the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 130 may also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be provided in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 may be provided in the same device as at least some of the modules of the processor 110.

[0396] The wireless communication module 120 may include an antenna and transmit and receive electromagnetic waves via the antenna. The wireless communication module 120 may provide wireless communication solutions for electronic device 10, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), and other wireless communication technologies. Electronic device 10 may communicate with networks and other devices using wireless communication technologies.

[0397] In some embodiments, the mobile communication module 130 and the wireless communication module 120 of the electronic device 10 may also be located in the same module.

[0398] Display screen 102 is used to display human-computer interaction interfaces, images, videos, and the like. Display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), or a quantum dot light-emitting diode (QLED).

[0399] The sensor module 190 may include a proximity sensor, a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0400] The audio module 150 is used to convert digital audio information into an analog audio signal for output, or to convert analog audio input into a digital audio signal. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be provided in the processor 110, or some functional modules of the audio module 150 can be provided in the processor 110. In some embodiments, the audio module 150 can include a speaker, an earpiece, a microphone, and a headphone jack. The camera 170 is used to capture still images or videos. The object is projected onto the photosensitive element through the lens to generate an optical image. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the image signal processing (ISP) for conversion into a digital image signal. The electronic device 10 can implement a shooting function through the ISP, camera 170, video codec, graphics processing unit (GPU), display screen 102, and application processor.

[0401] Interface module 160 includes an external memory interface, a USB interface, and a subscriber identification module (SIM) card interface. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 10. The external memory card communicates with processor 110 via the external memory interface to implement data storage. The universal serial bus interface allows electronic device 10 to communicate with other electronic devices. The subscriber identification module card interface is used to communicate with a SIM card installed in electronic device 10, for example, to read or write phone numbers stored on the SIM card.

[0402] In some embodiments, the electronic device 10 further includes a button 101, a motor, and an indicator. The button 101 may include a volume button, an on / off button, and the like. The motor is used to vibrate the electronic device 10, for example, when the user's electronic device 10 is called, to prompt the user to answer the call. The indicator may include a laser pointer, a radio frequency indicator, an LED indicator, and the like.

[0403] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of this application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0404] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor, such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0405] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0406] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions may be distributed over a network or via other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy disks, optical disks, optical discs, compact disc-read only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable storage for transmitting information via the Internet using electrical, optical, acoustic, or other propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0407] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0408] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0409] It should be noted that in the examples and description of this patent, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article or apparatus comprising the element.

[0410] Although the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. An image processing method, characterized in that: Applied to a terminal device, the image processing method includes: Running a first application and a second application, wherein the first application renders a layer based on a first rendering frame rate, and the second application renders a layer based on a second rendering frame rate; The first application renders a layer of an i-th frame interface of the first application at a first moment, and the second application renders a layer of a j-th frame interface of the second application at a second moment, wherein the first moment is different from the second moment, and i and j are positive integers; The first application renders a layer of an i+1th frame interface of the first application at a third moment, and the second application renders a layer of a j+1th frame interface of the second application at the third moment, wherein a first time difference between the third moment and the first moment is different from a first time interval corresponding to the first rendering frame rate, or a second time difference between the third moment and the second moment is different from a second time interval corresponding to the second rendering frame rate; After the third moment, the first application renders the layer based on the first rendering frame rate, and the second application renders the layer based on the second rendering frame rate; When a first condition is met, the first time difference is equal to the first time interval, and the second time difference is greater than the second time interval; wherein the first condition includes any one of the following conditions: A first value of the first rendering frame rate is equal to a second value of the second rendering frame rate, and a first display area of ​​the first application interface on the display screen is larger than a second display area of ​​the second application interface on the display screen; The first value is equal to the second value, the first display area is greater than a first threshold, and the distance between the interface of the first application and the center of the display screen is less than a second threshold; The first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the interface of the first application on the display screen includes the center of the display screen.

2. The image processing method according to claim 1, wherein: The first value is equal to the second value, the first display area is greater than a first threshold, and the distance between the interface of the first application and the center of the display screen is less than a second threshold, including: The first value is equal to the second value, the first display area is greater than the first threshold, the distance between the interface of the first application and the center of the display screen is less than the second threshold, and the interface of the first application is on an upper layer of the interface of the second application.

3. The image processing method according to claim 1, wherein: The first value is equal to the second value, the first display area is greater than the first threshold, and the display area of ​​the first application interface on the display screen includes the center of the display screen, including: The first value is equal to the second value, the first display area is greater than the first threshold, the interface of the first application in the display area of ​​the display screen includes the center of the display screen, and the interface of the first application is on an upper layer of the interface of the second application.

4. The image processing method according to claim 1, wherein: Before the third moment, the first value of the first rendering frame rate and the second value of the second rendering frame rate are both smaller than a third value of the synthetic frame rate of the terminal device.

5. The image processing method according to claim 1, wherein: When the first condition is met, the i-th frame interface is the last frame interface synthesized by the first application before a fourth moment, and the time interval between the fourth moment and the second moment is the second time interval.

6. The image processing method according to claim 1 or 5, characterized in that: The first application renders a layer of an i+1th frame interface of the first application at a third moment, and the second application renders a layer of a j+1th frame interface of the second application at the third moment, including: When at least one of the following conditions is met, the first application renders a layer of the (i+1)th frame interface of the first application at a third moment, and the second application renders a layer of the (j+1)th frame interface of the second application at the third moment: The first value is an integer multiple of the second value; The first value and the second value are both greater than or equal to a preset threshold, the first value is not an integer multiple of the second value, and a ratio of the first value to the second value is greater than or equal to a first ratio.

7. The image processing method according to claim 1 or 5, characterized in that: The second time difference is the sum of the second time interval and the delay time; The delay time=the first moment-the second moment+the first time interval-the second time interval.

8. The image processing method according to claim 7, wherein: The second application renders a layer of the j+1th frame interface of the second application at the third moment, including: After the second moment, delaying the Vsync-app signal of the second application by the delay time; Render the layer of the j+1th frame interface of the second application based on the delayed Vsync-app signal.

9. The image processing method according to claim 1, wherein: When the second condition is met, the second time difference is equal to the second time interval, and the first time difference is greater than the first time interval, wherein the second condition includes any one of the following conditions: The second value is equal to the first value, and a second display area of ​​the interface of the second application on the display screen is larger than a first display area of ​​the interface of the first application on the display screen; The second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold; The second value is equal to the first value, the second display area is greater than the first threshold, and the interface of the second application in the display area of ​​the display screen includes the center of the display screen.

10. The image processing method according to claim 9, wherein: The second value is equal to the first value, the second display area is greater than the first threshold, and the distance between the interface of the second application and the center of the display screen is less than the second threshold, including: The second value is equal to the first value, the second display area is greater than the first threshold, the distance between the interface of the second application and the center of the display screen is less than the second threshold, and the interface of the second application is on an upper layer of the interface of the first application.

11. The image processing method according to claim 9, wherein: The second value is equal to the first value, the second display area is greater than the first threshold, and the interface of the second application in the display area of ​​the display screen includes the center of the display screen, including: The second value is equal to the first value, the second display area is greater than the first threshold, the interface of the second application in the display area of ​​the display screen includes the center of the display screen, and the interface of the second application is on the upper layer of the interface of the first application.

12. The image processing method according to claim 9, wherein: The j-th frame interface is the last frame interface synthesized by the second application before the fifth moment, and the time interval between the fifth moment and the first moment is the first time interval.

13. The image processing method according to claim 1, wherein: The method comprises: After the third moment, the refresh rate of the display screen of the terminal device is adjusted based on the synthetic frame rate of the terminal device.

14. An electronic device, characterized in that: include: A memory for storing instructions executed by one or more processors of the electronic device, and the processor, which is one of the one or more processors of the electronic device, for executing the image processing method according to any one of claims 1 to 13.

15. A readable medium, characterized in that The readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the image processing method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The computer program product comprises computer instructions, and when executed by an electronic device, the electronic device performs the image processing method according to any one of claims 1 to 13.

Images